Is Your Backup Storage Working Too Hard?
- Read time: 6 mins.
Ask most IT teams about their biggest storage challenges, and you’ll hear mostly the same things: rising costs, shrinking backup windows, and data growth that never seems to slow down.
It’s easy to assume that’s just the new normal. But a surprising amount of that pain can be traced back to a single factor: how your data is being deduplicated.
When deduplication is done well, it can dramatically reduce storage requirements and improve efficiency. When it isn’t, the costs add up quickly.
Before exploring one solution, let’s start with the numbers behind the problem—and why deduplication matters in the first place.
The Backup Retention Problem Is Bigger Than You Think
![Man in a striped shirt typing at a desktop computer in a bright, modern office.] , hmm fix](data:image/svg+xml;base64,PHN2ZyB2aWV3Qm94PSIwIDAgMTYwMCA5MDAiIHdpZHRoPSIxNjAwIiBoZWlnaHQ9IjkwMCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj48L3N2Zz4= "Man in a striped shirt typing at a desktop computer in a bright, modern office.] , hmm fix")
Think about what it takes to meet backup retention requirements for a mid-sized organization. You’re keeping 12 weekly backups. Add monthlies across three years and you’re looking at 45 or more backup copies on retention at any given time.
Now multiply. If your weekly full backup is 50TB, storing 20 copies without any deduplication means you need 1 petabyte of storage for one system. And that’s before accounting for regulatory mandates like HIPAA, Sarbanes-Oxley, or GLBA that require you to hold data for years instead of months. The longer the retention window, the more copies stack up, and the more storage you need.
Straight disk at this scale will quickly bankrupt a storage budget.
That’s the core reason data deduplication exists, as a fundamental requirement for making backup retention financially viable.
What Deduplication Does
At its core, deduplication is straightforward. The system scans your backup data, identifies repeated chunks, stores one copy, and replaces every other instance with a pointer to the original. You end up with far less data physically on disk.
The effectiveness of that process is measured as a deduplication ratio. A 20:1 ratio means what used to require 1PB of storage now requires 50TB.
However, not all deduplication achieves the same ratios, and not all deduplication work the same way. Ratios can range anywhere from 2:1 on the low end to 50:1 or higher depending on data mix, retention period, and the algorithm doing the work. The average for a well-implemented solution lands around 20:1. A weak implementation might give you 4:1, and you’d be paying for a lot more storage as a result.
There are two fundamental deduplication timing approaches:
Why Traditional Inline Deduplication Often Breaks Down at Scale
Many legacy backup appliances use inline, block-level deduplication. Data gets split into fixed chunks (typically 4KB to 10KB blocks), and each block gets a unique fingerprint (or hash). When new backup data comes in, the system checks incoming block fingerprints against a massive lookup table to find matches. The block matches don’t get stored again, while the new unique blocks make it to disk.
For every 10TB of backup data using 8KB blocks, that hash table contains approximately one billion entries. At 50TB, that’s five billion. At 200TB … you get the picture. That table has to live in memory in a single front-end controller—which means the only way to add capacity is to add disk behind that controller.
The problem with scale-up is that only one resource grows: raw disk, while processor, memory, and bandwidth stay fixed. So as data volumes increase, the time required to process deduplicated data during the backup window grows with them. Your backup jobs end up taking longer, and then your team notices, and finally leadership starts asking questions.
Restores can be even more painful. Because inline solutions store only deduplicated data, every restore request requires the system to reassemble the original file from its component chunks. This process is called rehydration. It’s compute-intensive and slow; for example, VM boots that should take minutes usually take hours.
Eventually, the front-end controller will max out. You need a new one—a disruptive, expensive replacement the industry calls a forklift upgrade. You’re essentially starting over, paying again for infrastructure you already bought.
This wasn’t a bad design choice in the early days of disk-based backup. But it wasn’t built for the data volumes organizations carry today, and it hasn’t aged well.
How ExaGrid Built a Better Architecture
ExaGrid looked at those problems and decided the architecture itself needed to change, and not just the specs. There are two decisions that define how ExaGrid is different.
Zone-Level Deduplication
Instead of breaking data into thousands of tiny blocks, ExaGrid works with larger “zones” and performs similarity detection across them. It then stores and replicates only the changed bytes. The tracking table this produces is roughly 1/1000th the size of a block-level hash table.
Because the hash table stays manageable, ExaGrid doesn’t need expensive flash storage to accelerate lookups. That keeps hardware costs lower and eliminates one of the biggest cost escalators in competing solutions. And because ExaGrid’s zone-level approach is backup-application agnostic, it works across more than 25 applications—Veeam, Veritas NetBackup, Commvault, Rubrik, IBM Spectrum Protect, and many others—without requiring any customization.
Strong deduplication ratios average 20:1, with some environments achieving 50:1 or higher depending on data mix and retention period. That translates directly into less storage purchased and less bandwidth consumed when replicating to a disaster recovery site.
The Landing Zone
When a backup job runs, ExaGrid writes data directly to a front-end disk cache called the Landing Zone at full speed, with no deduplication running inline. There’s no compute tax on your backup window. Ingest rates are as fast as the network and the backup application can push data.
The most recent backups stay in that Landing Zone in their native, non-deduplicated format. No reassembly required. When you need to restore a recent backup or boot a VM, ExaGrid pulls directly from the Landing Zone at full speed, and in its original format. What takes hours in a traditional inline system takes seconds to minutes with ExaGrid.
Deduplication happens in the background, after the backup window closes, against the long-term retention repository.
Scale-Out, Not Scale-Up
Every node added to an ExaGrid system brings its own processor, memory, network ports, and storage. When your data volume doubles, you add nodes, and all resources double with it. The backup window stays fixed regardless of how much data grows. There are no forklift upgrades and no single controller that eventually becomes a ceiling.
We can bring this out of the architecture and into operations.
Predictable backup windows. Your backup schedule shouldn’t get harder to hit as the business grows. With fixed-length backup windows regardless of data volume, ExaGrid makes SLA compliance a system property rather than a fire drill.
Faster recovery. The distance between a failure event and a restored environment shrinks substantially when recent backups are immediately accessible in their native format. For VM boots especially, seconds and minutes versus hours is a meaningful difference.
Lower total cost of ownership. Better deduplication ratios mean less storage purchased. No flash required for hash lookups. No forklift upgrades. Lower bandwidth consumption for DR replication. The cost advantages compound over time.
Application flexibility. If your organization runs multiple backup applications, or changes backup vendors down the road, ExaGrid’s architecture accommodates that without requiring a rip-and-replace.
Ransomware resilience. ExaGrid’s Retention Time-Lock feature adds an additional layer of protection against ransomware by making backup data immutable for a defined retention period, meaning attackers can’t encrypt or delete what they can’t reach.
Is Your Deduplication Working Against You?
If your backup windows have been creeping longer quarter over quarter, if restores take longer than your team can justify, or if you’re approaching another storage upgrade that feels too soon, your deduplication architecture is worth examining before you spend more money on raw capacity.
The storage problem isn’t always about having more disk. Sometimes it’s about making better use of the disk you already have.
HBS works with organizations across the country to evaluate backup environments and right-size infrastructure for where the business is going, not just where it stands today.
Reach out to the HBS team to start the conversation.
Related Content
Managed Backup and Disaster Recovery
Secure your data and peace of mind with the Backup and Disaster Recovery Services from HBS, offering expert-managed, always-on solutions.
Backup, Replication and Disaster Recovery
Ensure data resilience with HBS’s Backup, Replication and Disaster Recovery solutions. Get reliable data protection and expert support tailored to your needs.
Data Classification: What It Is and Why It Matters
Data classification helps protect your organization’s sensitive information, ensure compliance, and streamline data management.
