Frequently Asked Questions
Curious about how Starfish addresses a wide variety of file management challenges? Our FAQ section covers everything you need to know! From supporting massive amounts of files with enhanced metadata operations to seamless integration with HPC file systems, NAS, and object stores, Starfish tackles it all. Learn how Starfish automates metadata, replaces outdated file management tools, supports custom workflows, and offers advanced reporting capabilities. Explore the answers to frequently asked questions to discover how Starfish can transform your data management processes.
How is it possible that Starfish addresses so many use cases?
It’s hard to believe, but the answer is quite simple. Starfish improves upon the fundamental building blocks of file management – the FIND statement, the CSV, and the script – but with tooling to support billions of files and a novel metadata framework that turns simple scripts into sophisticated operations.
Does Starfish support both HPC file systems and enterprise NAS?
Starfish supports any and all file storage systems. We have special integrations for Spectrum Scale, Lustre, HPSS, Isilon, and Qumulo. Our superfast, multi-node crawler is great for VAST, Weka, Pure, NetApp, Panasas, BeeGFS, Quantum, and Quobyte.
What kinds of products does Starfish replace?
Most of our “competition” comes from homegrown file management and movement software that has failed to scale up or that has become too much to support. We also replace backup, archive, reporting, data governance, and data migration applications.
Does Starfish include reporting and BI tools?
Starfish comes with customizable dashboards, reports, and pivot tables. You can also attach directly to our SQL database for custom reports or for use with third-party reporting and BI platforms.
How does Starfish’s metadata system enable workflow?
Starfish can capture metadata from files and processes throughout a workflow. Starfish can also use metadata with conditional logic to drive workflows.
Where does metadata come from? Not manual entry, I hope!
Yikes, no! It would be impossible to apply metadata manually across billions of files. Starfish makes it easy to automate metadata operations. And, of course, manual entry is a last resort.
What kinds of metadata does Starfish support?
Starfish supports file system metadata, file tags, directory tags (explicit or inherited), and key-value pairs.
Does Starfish support object stores?
Starfish works with S3 and its many variants, including AWS, GCP, Azure, Swift, and the many commercial implementations of S3.
What kinds of jobs can Starfish execute?
Pretty much anything. We include commands for copy, move, delete, hash calculations, extracting metadata, analyzing text, etc. Of course, you can execute your own scripts and call any third-party APIs.
Can Starfish integrate with custom and third-party apps?
Yes, you can integrate Starfish with other apps in several ways: through our API, through native SQL, and through custom jobs.
How does Starfish’s metadata system enable reporting?
Reports are much more meaningful and specific when you have a comprehensive metadata system to add color to your files and directories.
How does Starfish’s metadata system enable data curation?
Starfish allows for curation processes to begin when files are first created. Starfish keeps track of how files and directories relate to samples, projects, programs, researchers, etc. Starfish can then pass relevant metadata to long-term curatorial applications and maintain links between such applications and the files they reference.
Can my users use Starfish to manage their own data?
Yes! A common use case for Starfish is user-guided data ROT (Redundant, Obsolete, and Trivial) cleanup and archiving. If you give users the right tools, they will manage their own data.