RAID & NAS Recovery · Pittsburgh, PA
RAID and NAS recovery.
Don't rebuild it again.
RAID 0, 1, and 5, discussed case by case. Most of the unrecoverable arrays I hear about were made unrecoverable by the third recovery attempt, not by the original failure. Power it down and label the bays before anything else.
Label the drives by bay before you remove them. Number the bays, pull one at a time, write the number on each drive, and photograph the serials in order. It takes two minutes and it is the single most useful thing you can do.
Rebuild failed, second drive dropped
Stop before the next attemptQuoted after evaluation
You replaced the failed drive, started the rebuild, and somewhere partway a second member dropped out. The controller now reports two failures in a set that tolerates one.
This is more recoverable than the error message implies, and it happens for reasons that are statistical rather than unlucky. A rebuild reads every sector of every surviving drive, with no redundancy left to cover a read error. Consumer drives are specified around one unrecoverable read error per 10^14 bits — roughly one per 12 TB read — and a rebuild across several multi-terabyte drives reads a comparable amount. Add that all your drives are the same age from the same batch with identical hours, and correlated failure is the normal case.
Here is the part the controller does not tell you: a drive marked failed is rarely dead. Controllers are conservative and drop a member that takes too long on one read. That drive typically still contains almost all of its data.
Do not force members online repeatedly. Each attempt can write new configuration metadata, and inconsistent generation counters across members make reconstructing the true array state much harder.
Synology or QNAP volume crashed
Power it down, label the baysQuoted after evaluation
The unit is beeping, the web interface reports a crashed volume, or a drive shows as failed and the rebuild will not complete.
Worth knowing what is actually inside: Synology's SHR is not a proprietary format. It is Linux mdadm software RAID with LVM layered on top, and the file system is ext4 or btrfs. QNAP is similar. That means the array can be assembled and read on a Linux system without the original NAS hardware.
Which is the good news. If the NAS unit itself died — power supply, board, heat — your data is fine. The drives are the array. A dead enclosure is an inconvenience, not a data loss event, and you do not need an identical replacement unit to get your files back.
Never accept an offer to Initialize, Create Volume, or Repair a crashed volume. Any of them can write new metadata over the array configuration that recovery depends on.
Array ran degraded for months
The most common case I seeQuoted after evaluation
One drive failed, nobody noticed because the alert emails were never configured, the array ran degraded for months, and then a second drive went.
This is the single most common NAS story that reaches me, and it is the argument for actually turning notifications on once you are back up. A degraded array is not a working array — it is an array with its safety net already spent.
The recovery approach is the same as any other multi-drive case: image every member individually, handling bad sectors carefully, then determine the layout parameters and assemble the array virtually from the images. Nothing is ever assembled on the original drives.
Where a sector is unreadable on one image, parity from the others often reconstructs it exactly. That is the whole point of RAID, and it still works during recovery even when it failed during the rebuild.
Power loss or interrupted expansion
Often very recoverableQuoted after evaluation
A surge or a hard shutdown mid-write leaves array metadata inconsistent. So does an interrupted expansion — adding a drive or converting RAID levels is a long operation that rewrites the layout, and stopping partway leaves the array mid-transition.
Power events are frequently the most recoverable version of a RAID failure, because the drives themselves are healthy and the problem is bookkeeping. Interrupted migrations are genuinely harder, because the on-disk layout is partly old and partly new.
Either way, the sequence of events matters enormously to the approach. Write down what happened and in what order.
A UPS is cheap compared to this. Worth adding once the array is back.
Out of scope
Where I'll send you elsewhere
I am one person with a bench, and I will be straight about the ceiling.
- × Arrays where multiple drives have genuine mechanical failures needing cleanroom work.
- × Large enterprise configurations beyond a handful of drives.
- × Virtualised storage environments and SAN-attached volumes.
- × Cases needing a formal chain-of-custody process for litigation.
I would rather point you to a larger lab on day one than be the reason you are down an extra week. The evaluation costs nothing either way.
What to do right now
- 01. Power the array down. Every hour it stays up is more stress on marginal drives.
- 02. Label the drives by bay and photograph the serials in order before removing anything.
- 03. Do not rebuild, do not force members online, do not initialize or clear the configuration.
- 04. Record the controller make and model, the RAID level, and the stripe size if you know it.
- 05. Write down the sequence of events — which drive failed first, what was attempted, in what order. That changes the approach more than anything else.
From the bench
Read up before you decide
- RAID 5 rebuild failed Why it happens, and what to stop doing immediately.
- Synology or QNAP failed What not to do before you call, and why the enclosure does not matter.
- Business down after a failure Triage when the priority is operating again, not recovering everything.
- Image first, then work on the copy The rule every other piece of advice follows from.
- RAID is not backup Redundancy protects against drive failure, not deletion or fire.
- How long will it take? Multi-drive imaging is the long pole. Honest ranges here.