A deployment you can roll back in sixty seconds

Most teams introduce a deployment pipeline to make releases faster. That is a real benefit, but it is not the one that matters. The benefit that matters is that a bad release becomes reversible, which changes how willing people are to ship at all.

A team that cannot roll back deploys less often, batches more changes into each release, and therefore makes each release riskier. The fear is rational and it compounds.

Rollback is a property of the build, not of the code

The common approach is to fix forward: something breaks, so you write a patch, commit it, and deploy again. That works when the fix is obvious. It fails when the site is down and you are debugging under pressure, which is precisely when you need it.

Real rollback means the previous version is still sitting there, built and ready, and switching back to it is one action. Getting there requires three things.

Build once, deploy that artifact. If your pipeline rebuilds from source on every deployment, then rolling back means rebuilding an old commit, and you are hoping the dependency tree resolves to the same thing it did last month. It often does not. Build an artifact, tag it with the commit, and deploy that same artifact to every environment.

Keep the last few releases on the server. A releases directory with timestamped or commit-tagged folders, and a symlink pointing at the current one. Rollback is repointing the symlink and restarting the process. This pattern is old, it is boring, and it works.

Never let the pipeline be the only way to deploy. If your CI provider is having an outage at the same moment you need to roll back, you need a documented manual path. One script on the server, run by hand, that does what the pipeline does.

The database is the part that does not roll back

Application code goes backwards easily. Schema changes do not. A migration that dropped a column cannot be undone by redeploying yesterday's build, because yesterday's build expects the column to exist.

The discipline that solves this is making every migration backwards compatible for one release. Adding a column is safe. Renaming one is not, so you split it: add the new column, write to both, deploy, backfill, switch reads to the new column, deploy, and only then remove the old column in a later release. It is more steps, and each individual step is safe to reverse.

If you take one thing from this: never combine a destructive migration with a feature release. Ship the feature, confirm it is healthy, then remove the old structure in a separate deployment.

What belongs in the pipeline

Keep it short enough that people do not learn to skip it.

• Install dependencies from a lockfile, so the build is reproducible.
• Type check and lint. These catch the class of error that is embarrassing rather than dangerous, but they catch it in seconds.
• Run the tests you actually have. A small suite that runs on every commit is worth more than a large one people disable.
• Build the artifact once.
• Deploy to staging, run a smoke check against a real URL, then deploy to production.

That smoke check matters more than it looks. A deployment that succeeded and a site that works are different claims. Fetch the homepage, assert a 200 and a string you expect to be present. It takes seconds and catches the deploys that technically completed while serving an error page.

Secrets do not live in the repository

Every credential the application needs comes from the environment, injected by the pipeline or set on the host. Not in the repository, not in the build image, not in a .env file that got committed once and rewritten out of history afterwards.

Two practical rules that prevent most of this: a .gitignore that covers environment files before the first commit rather than after, and different secrets in staging and production so a leak from a test environment is not a production incident.

Measure the one number that matters

Not deployment frequency. Time to restore service.

If something ships broken at 3pm, how long until the site is healthy again? If the answer is under a couple of minutes because you repoint a symlink and restart, you have a pipeline worth having. If the answer involves finding the right commit, rebuilding, and hoping, then what you have is automation, not safety.

CI/CD pipeline setup is a fixed-scope piece of work, and rollback is part of the definition of done rather than something added afterwards.