Someone Else's Hands: The Quarterly Restore Rehearsal That Proves a Backup Works
Photo: U.S. Navy photo by Mass Communication Specialist 3rd Class Will Tyndall, Public domain, via Wikimedia Commons
Most backup platforms are honest about what they measure and quietly silent about what they do not. The dashboard confirms that a job started, transferred a quantity of data and exited without error. It says nothing about whether a finance manager can open last quarter's payroll workbook, whether the line-of-business application will start against a restored database, or whether the person attempting the recovery at four o'clock on a Friday has the permissions to attempt it at all. Those questions are answered in only one way: by restoring something, with the clock running, using people who were not involved in configuring the backup.
Why a completed job is not a recovered service
A successful backup job proves that a source was readable and a destination was writable at a given moment. Between those two points sit a dozen quiet failure modes that no console will flag: an application-consistent snapshot that captured a database mid-transaction, an encryption key held by a leaver, a retention policy that pruned the very version you needed, a Microsoft 365 tenant where a user deleted a mailbox and the recycle bin expired before anyone noticed. Ransomware operators understand this well; a common tactic is to dwell quietly until the oldest intact backup has aged out of retention before encrypting production.
For a UK SME, the commercial exposure is rarely the data itself. It is the invoice run that cannot be issued, the site that cannot take orders, the statutory records that cannot be produced. Recovery is therefore a business capability, not an IT setting, and it should be tested the way businesses test anything that matters — on a schedule, against a target, with a written result.
Set the drill up so that it is genuinely a test
A restore rehearsal that the backup administrator runs on their own machine, at a convenient hour, against a system they chose, tests the tool and confirms the administrator's familiarity. It does not test the organisation. The more useful design has four characteristics.
First, the drill should be run by a stand-in — a second-line engineer, an IT coordinator, or a managed service provider's on-call technician who has read the runbook but never built the platform. Second, it should be unannounced to that individual, with a short notice window, so that it measures discovery and interpretation rather than rehearsal. Third, it should be timed from the moment the request is raised, including the time spent finding the correct console, the right credentials, and the person who can approve access. Fourth, it should be recorded in a template that a non-technical reader could follow.
Many organisations find it useful to hold one drill a quarter at an awkward hour, deliberately — late on a Friday, or during a period of thin support cover. The point is not to manufacture stress for its own sake, but to discover which parts of the process depend on one individual being awake and available.
Choose a representative sample, not the easiest file
The temptation is to restore a small text document from last night's incremental and declare the estate resilient. A defensible sample set covers four categories.
- A user file from deep in the retention window. Something at least sixty to ninety days old, chosen from a department that has never raised a ticket. This tests retention policy, archive tiers and retrieval latency.
- A business-critical application and its data. Restore the database and the application together into an isolated environment, then confirm that the application starts, authenticates users and passes a defined functional check — a report runs, a document opens, a record can be created.
- A SaaS workload outside the virtual estate. Microsoft 365 mailboxes, SharePoint libraries and Teams channels are protected by the platform's own retention, which is not the same as a backup. Attempt a point-in-time restore of a mailbox folder and a document library, and confirm the restore scope behaved as intended rather than overwriting live data.
- A configuration or identity object. An Active Directory object, a group policy, a firewall rule set. Recovering a server without the directory context that lets it authenticate is a common and frustrating discovery.
Rotate the sample each quarter so that, over a year, most of the estate has been touched at least once.
Measure against the number you actually promised
Recovery time objective and recovery point objective are often quoted in a service schedule and rarely verified. The drill is where they acquire meaning. Record three timestamps: when the request was logged, when the data was available for use, and when the service was confirmed functional by the business owner. The gap between the first and the last is the real recovery time, and it is almost always longer than the technical restore time.
If the service schedule promises a four-hour recovery for a finance system and the drill takes nine, that is not a drill failure — it is a finding, and considerably cheaper to discover on a Tuesday morning in November than during a month-end. Where the measured figure cannot be improved economically, the honest response is to revise the objective and inform the business, not to leave a number in a document that nobody can meet.
Gather evidence that survives contact with a third party
Insurers writing cyber cover, auditors reviewing financial controls, and procurement teams assessing suppliers increasingly ask for proof rather than assurance. A useful drill record contains: date and duration; the scope restored; the names and roles of those involved; the measured recovery time and point; the outcome against the stated objectives; screenshots of the restored data with a timestamp; and a list of corrective actions with owners and dates.
Keep the records for at least the period your insurer or auditor requires, and store them somewhere outside the estate being tested. A restore report that exists only on the file server that the drill was meant to recover is an uncomfortable anecdote waiting to happen.
The gaps that only appear in someone else's hands
Almost every first drill surfaces the same short list. Credentials for the backup console sit with one engineer and are not in the password vault. The runbook references a server that was decommissioned in the spring. Multi-factor authentication for the recovery portal is tied to a phone number that changed. Restores are throttled to protect production bandwidth, and nobody remembered to lift the limit. Licensing for the isolated recovery environment has lapsed. And the business owner who must sign off the restored data is on leave.
None of these are exotic. All of them are invisible until somebody unfamiliar tries to do the work, which is precisely why the rehearsal should not be run by the person who configured the system.
Close the loop or the exercise is theatre
End each drill with no more than five corrective actions, each with a named owner and a date, and carry open items into the next quarter's agenda. Publish a one-page summary to the leadership team in plain English: what was tested, what was recovered, how long it took, what it cost in staff time, and what changes as a result.
A backup product is a means. The deliverable is a restored service, in someone else's hands, inside an agreed window, with a record to prove it. Treat the quarterly rehearsal as the product itself and the subscription becomes considerably easier to justify.