Your backup has never been restored, and that is the finding
Backups are the control everyone says they have and almost nobody has tested. Here is the check that takes an afternoon and the reason it keeps turning up something.
Ask a team whether they have backups and the answer is yes. Ask when a backup was last restored and the answer changes shape. It becomes "the job runs nightly", or "the host handles it", or a pause.
Those are not the same answer. A backup job that completes is a file that exists. A restore is the only thing that proves the file is a backup.
Why this keeps happening
Nothing tells you that a backup is broken. The job reports success, the file appears, the size looks plausible. A backup can be encrypted with a key nobody has, can exclude the one directory that matters, can capture a database mid-write in a state that will not import, or can quietly have been failing since a path changed eight months ago. Every one of those looks identical to a working backup right up until the afternoon you need it.
The check
Take the most recent backup. Restore it into a scratch environment that is not production. Bring the application up against it. Log in. Look at the newest record you expect to find.
Then write down three things: the date you did this, how long the restore took, and what was missing. That last one is usually the surprise. Uploaded files stored outside the database, environment configuration, cron entries, TLS certificates and queue state are the things that fall outside a database dump and are only noticed when something is being rebuilt under pressure.
What to fix afterwards
- Restore time is a number your business should know. If a full restore takes six hours, that is your recovery time, whatever the plan says.
- Keep at least one copy somewhere that a compromise of the production host cannot reach. Ransomware that encrypts a server usually encrypts the backup mounted on it too.
- Put the restore on a schedule. Quarterly is enough for most teams, and it is infinitely more than never.
- Write the restore procedure down. In an incident, the person doing it is often not the person who set it up.
The honest version
A backup you have not restored is not a backup, it is an intention. That sounds harsh until the day you find out, and then it sounds like the most useful sentence anyone could have said to you six months earlier.