Use Case - Scientific Workflows
From instrument to archive, automatically
Starfish orchestrates data from creation to long-term storage. It can chain incremental workflow steps, extract metadata via Meta-x, apply tags programmatically or by users through Zones, all with lineage tracked against every file version.
Built for research environments where data lands continuously and needs to be acted on quickly.
Capture
Pick the data up where it lands
Chained incremental workflow steps move data from multiple instruments to designated analytical areas, with zero-loss standards, auto-resume, and fault tolerance.
Chained incremental steps
Data moves from multiple instruments to designated analytical areas as a sequence, not a set of manual handoffs.
Zero loss, auto-resume
Zero-loss standards, auto-resume, and fault tolerance, because instruments do not pause for a network problem.
Secure transit
Movement to object storage enforces TLS certificate validation rather than assuming the path is safe.
Meta-x extraction at capture
Internal metadata is read from scientific and media formats, including DICOM, HDF5, and BAM, as the data arrives.
Starfish Automtion Engine
See a workflow built against your own instruments
No synthetic demo. We take one real instrument path and show you what capture to archive looks like automated.
Describe
FAIR by construction, not by cleanup
Predefined configuration schemas and templates are applied via Zones, the REST API, or the CLI, and every file version carries a unique Starfish ID (SFID) so lineage is captured rather than reconstructed.
Templates
Required and optional fields, applied through Zones, the REST API, or the CLI - so description is part of the workflow.
Lineage on every version
Each file version is identified by a unique Starfish ID (SFID), which is what makes a result traceable to its source run.
Normalization maps
Data from different systems or instruments can be compared, instead of being merely co-located.
Integrity and gating
Compliance gating and ACID-compliant integrity throughout, so a non-conforming dataset does not quietly proceed.
Automate
Your logic, in your language
Custom logic runs in any language, including Bash, Python, and Go, with integrations for Airflow, n8n, Nextflow, Snakemake, and HPC schedulers - and curated data can be sent on automatically.
Any language
Bash, Python, Go, or anything else you already maintain. Starfish runs with the tools and scripts you use at scale rather than replacing them.
Workflow engine integrations
Airflow, n8n, Nextflow, Snakemake, and HPC schedulers, so Starfish fits the pipeline you already have.
Curated data, delivered
Send curated data automatically to HPC clusters, AI/ML environments, or cloud storage for backup.
Audit-ready by default
Lineage captured automatically supports funder and journal requirements on FAIR data and long-term retention.
In Practice
“Once we implemented Starfish, complaints from researchers about managing data went to nearly zero.”
Senior Programmer Analyst
A leading US research hospital
Dig Deeper
Resources for the curious technical reader

Blog
Starfish Storage automates dozens of scientific instrument workflows for a leading research hospital
When researcher time was being eroded by having to manually process dozens of instrument data workflows, Starfish Storage stepped in and set researchers free.
4-min video

Blog
How ASU scratch storage went from instability to control
ASU used Starfish to help researchers stop using scratch storage for permanent archive and bring storage utilization down to 70%
4-min read

Glossary
Get everyone using the same words
Storage admins, data owners, and finance can all say “cold data” and mean three different things. The glossary pins each term to one plain definition.
Quick reference
FAQ
Frequently asked questions
What does Starfish automate in a research workflow
Orchestration from instrument to archive: chained incremental steps that move data from multiple instruments to analytical areas, then on to HPC clusters, AI/ML environments, or cloud archive, with metadata captured along the way.
How does this support FAIR data principles?
Through structured metadata templates with required and optional fields, applied via Zones, the REST API, or the CLI, and lineage tracking where every file version carries a unique Starfish ID (SFID).
Can we keep the scripts we already have?
Yes. Custom logic runs in any language, including Bash, Python, and Go, and Starfish integrates with Airflow, n8n, Nextflow, Snakemake, and HPC schedulers.
Which scientific formats can Starfish read inside?
Meta-x extracts metadata from a wide and rapidly expanding catalog of scientific and media formats, including DICOM, HDF5, and BAM. Additional formats are supported on request.
What happens if a transfer fails midway?
Zero-loss standards, auto-resume, and fault tolerance apply, and job records mark what has already moved so a rerun does not repeat completed work.
Can data from different instruments be compared?
Yes. Normalization maps let data from different systems or instruments be compared rather than simply stored alongside each other.
Ready When You Are
Bring us the instrument nobody has time to babysit.
We will automate one real path on your own data - capture, description, and archive - and show you the result.


