The Gruesome Job of Managing Petabytes of Scientific Data

March 26, 2024

“Oh, wow, the IT department charges us $11,000 a month for data storage. Why don’t we put some data on cheaper storage? Mark, why don’t you look into that?” the principal investigator asks the postdoc who just joined the team.

There are thousands of similar discussions in research labs around the world, sometimes because there is pricing sticker shock and at other times because the IT department is asking to curb storage consumption through quotas. In some cases, there is a quick solution (“How hard can that be?”), but in many other cases, this leads to a real quagmire. Let’s follow Mark’s journey through this quagmire and propose a solution with the help of tools such as Starfish.

Read More at Medium →

Medium

Recent Posts

Starfish Storage Wins “Data Solution of the Year for Research” in 2026 Data Breakthrough Awards Program

April 16, 2026

New White Paper: How ASU Built a Searchable DICOM Catalog for Global Health Research using Starfish

April 9, 2026

How ASU scratch storage went from instability to control

February 24, 2026

Upcoming Events

BioIT World Expo

May 18, 2026

ISC High Performance 2026

Jun 21, 2025

21-things-banner-600x600