Disaster recovery

Git disaster recovery playbook

Four failures, four recovery paths, and the one thing that makes all of them boring: a current copy somewhere else.

01 — The failures

What actually goes wrong

Outage, lockout, a destructive push, a deletion — four different failures, and each one needs a different kind of copy to get back from.

Problem

Provider outage

GitHub, GitLab and Bitbucket have all had multi-hour outages that block pushes, pulls, and webhook delivery. If the only copy of a repository lives with the provider that's down, your team is blocked until the provider recovers — not until you do.

Git Replica solves it

Work continues from the replica

Point remotes and CI at the mirror and keep working. Nothing was lost — the replica was current as of your last push, because it's an ordinary Git repository sitting on a host that isn't down.

Problem

Account or organization lockout

A suspended account, a billing dispute, or a policy flag can cut a team off from every repository under that account, sometimes for days while support works the ticket. The code isn't gone — it's just unreachable, and unreachable feels the same as gone when a deploy is due.

Git Replica solves it

A copy outside that account

A replica on a different provider or account, or an archive sitting in your own bucket, was never tied to the account that got locked. It's reachable exactly when the primary account isn't.

Problem

Destructive force-push or deleted branch

Someone force-pushes over main, or deletes a branch that turns out to matter. The history is gone from the provider the instant the push lands, and there's no undo button.

Git Replica solves it

A point-in-time archive

A live replica configured to mirror exactly will follow that push faithfully — it now shows the same damage. An archive taken before the incident does not: it's a snapshot from a moment in time, untouched by whatever happened after it. Restore the refs you lost from there.

Problem

Repository or organization deleted

A repository is deleted by mistake, or an entire organization is removed. What used to be a URL is now a 404, and reconstructing it from whatever teammates happen to have cloned locally is slow and never quite complete.

Git Replica solves it

Restore the whole repository

Unzip the last archive and git push --mirror it to a fresh remote. Every branch, tag and note comes back in that one push — there's nothing to reconstruct, because the archive already has everything in it.

02 — RPO & RTO

How long until you're working again

What matters after an incident isn't cost — it's how much work you lose and how long it takes to have a working repository again.

Scenario No backup Nightly cron Git Replica
Worst-case work lost Everything since the last local clone Up to 24 hours The last push
Time to a working repository Reconstruct from laptops Find the dump, restore, verify Clone the replica, or unzip and push
Confidence it works None Untested unless you test it Every sync is logged
03 — The runbook

Recovery, step by step

The same four steps whether the trigger was an outage or a deletion — only where the restore comes from changes.

Identify the last good copy

If the failure was about availability — an outage, a lockout — the replica is your last good copy, and it's already current. If the failure was destructive — a force-push, a deletion — go back further, to an archive from before the incident happened.

Restore

Clone the replica directly if it's untouched by the incident. Otherwise, unzip the archive you identified and git push --mirror it to a new remote — a bare mirror push brings every ref back in one step.

Repoint remotes and CI

git remote set-url for every local clone that matters, then update your CI pipeline's clone URL and any deploy scripts that reference the old remote.

Verify

Compare branch and tag counts against what you expect, and check the latest commit on your main branches. Don't call it recovered until you've looked — an incomplete restore nobody checked is just a slower version of the original incident.

04 — Test it

An untested backup is not a backup

A backup you've never restored is a hypothesis. Test it before an incident forces you to.

Restore into a scratch namespace

Pick a throwaway org or repository and actually run the restore — clone the replica, or unzip an archive and push it. It costs nothing, and it's the only way to know a restore works, instead of assuming it does because a dashboard says the last sync succeeded.

Check the sync log regularly

A binding that has been failing quietly for a week looks exactly like one that's working — until the day you need it. Every step of every sync is recorded; look at it before an incident, not during one.

Turn on failure email

Enable notifications so a broken binding tells you the day it breaks, instead of you discovering it during a recovery. It's a per-account toggle, plus a periodic summary if you'd rather review on your own schedule.

FAQ

Disaster recovery questions

What's the fastest way to recover a deleted repository?

Unzip the most recent archive and push it to a new remote with git push --mirror — every branch, tag and note comes back in that one command. If you kept a live replica of the repository instead, recovery is even faster: the replica is already a working repository, so you point people at it, or push it to a freshly created remote if you need the original URL back.

Can I recover from a force-push that overwrote history?

Yes, but not from a live replica configured to mirror exactly — it will have faithfully followed the force-push, including the history it overwrote. Recovery comes from an archive taken before the incident, a snapshot the later push can't touch. This is exactly why archives and replicas complement each other rather than substitute for one another: the replica keeps your team working day to day, and the archive is what saves you when the replica itself gets told to destroy something.

How do I test my Git backups?

Periodically restore into a scratch namespace — a throwaway repository or org — and check that branch counts, tag counts, and the latest commit match what you expect. That's the only way to know a restore actually works, rather than assuming it does because the sync log says success. Pair it with checking the sync log regularly and turning on failure email, so a broken binding doesn't sit silent for weeks.

What is a realistic RPO for Git repositories?

It depends on how the source syncs. GitHub, GitLab and Bitbucket sources are webhook-triggered — the sync starts the moment you push — so the recovery point is effectively the last push, typically seconds behind. Self-hosted and custom remotes sync on a schedule (hourly, daily, or weekly) or on demand, so their recovery point is bounded by that interval instead. Check the schedule on your slowest-syncing binding rather than assuming — that binding is your real number.

Make your next Git incident boring.

Set up a replica, an archive, or both, in under two minutes. Free for open source and private non-commercial use.

Get started free