Glossary Term

High performance computing

What is high performance computing?

High performance computing (HPC) links thousands of processors into clusters or supercomputers so that massive workloads can be split into smaller tasks and run simultaneously.

HPC exists because some problems simply will not finish on a single machine in any reasonable timeframe. Climate simulations, genomics pipelines, computational fluid dynamics, and large-scale AI training all fall into this category. By distributing work across thousands of processors, HPC systems turn months of computation into hours or days.

Getting that parallelism to work requires more than just adding servers. HPC environments depend on high core count processors, low-latency interconnects so nodes can communicate without bottlenecks, parallel file systems like Lustre and GPFS, Weka and VAST, and job schedulers that assign work across the cluster. The data these systems produce is almost entirely unstructured, often reaching petabytes or exabytes, which makes storage management, governance, and simple visibility into what exists genuinely difficult at that scale.

As datasets keep growing, HPC organizations need metadata-driven tools that can index and automate across billions of files, and multiple file systems, without interfering with production jobs. Starfish Storage runs in some of the most complex and fast changing HPC environments in the world, including Top500 supercomputers, DOE national laboratories, and major research universities, providing near real-time visibility, data movement, storage optimization, and job automation across their environments.