Main Definition
File metadata is descriptive information about a file (name, size, owner, timestamps, permissions) and about its contents (format-specific tags like EXIF, DICOM, or FASTQ headers). File metadata consists of two types of data about a file:
“File system” metadata describes the file as an object in the storage system: filename, size, last-modified and last-accessed dates, ownership, and permissions. File system metadata is what traditional storage systems and scanning tools see.
“Content” metadata, also called embedded or file-format metadata, describes what’s inside the file. Content metadata is specific to the type of file: EXIF data in images, DICOM tags in medical images, headers in FASTQ genomics files, bibliographic fields in research datasets.
Two files can share identical file system metadata while holding completely different content, and content metadata is what tells them apart.
At a petabyte scale with billions of files, file metadata becomes the most important tool for understanding and managing an unstructured data environment. It’s the difference between knowing you have 50 petabytes of storage and knowing what those 50 petabytes contain, who owns them, when they were last touched, and whether they still have value.
Starfish scans and stores both file system metadata and content metadata. It extracts and indexes native file system metadata across heterogeneous storage (every enterprise NAS, HPC file system, and S3-style object store), then enriches it with deep content metadata pulled from 100+ file types plus custom tags applied by users and automated policies. The result is a searchable catalog of your all your files that provides you with the intelligence you need to understand your data value and execute sensible data lifecycle management practices.
Related Links
- Starfish Product: Metadata Extraction | Starfish Storage
- Breaking File System Scanning Barriers at Scale | Starfish Storage
- From Months to Minutes: How ASU Built a Searchable DICOM Catalog with Starfish | Starfish Storage
- Exploring Metadata Solutions for Large-Scale Data Management | Starfish Storage
- What is Metadata? | IBM
- Understanding File Metadata | Dataversity
