Between “done” and “live”: what CI/CD actually does
A feature can be finished for a week and still not be in anyone's hands. What sits in that gap is usually a person, a set of steps they remember, and an evening nobody wanted to spend. This is what a pipeline replaces, stage by stage.
The ritual nobody writes down
Ask a team how they release and you will often get an answer that sounds casual and takes forty minutes.
Someone runs the tests on their own machine, or means to. Someone builds the project. Someone copies the output to the server over SSH, or drags it into a panel. Someone remembers that the database needs a change, and runs it by hand, hopefully in the right order. Someone restarts the service. Then everybody watches the site for ten minutes to see if anything catches fire.
It works. It works most of the time, in fact, which is exactly what makes it dangerous. The steps live in one person's head, so releases happen when that person is available. Because each release is expensive, releases get batched — three weeks of changes in one go. And because a big batch is risky, it goes out late at night, when the fewest people will see it break and the fewest people are awake to fix it.
If a release contains forty changes and something breaks, you do not have a bug. You have forty suspects.
What the two letters actually mean
CI — continuous integration. Every change gets merged into the shared main line frequently, and every merge is verified automatically. The word doing the work is *integration*: the problem being solved is branches that live for three weeks and then collide. When everyone integrates daily, conflicts are small and constant instead of rare and enormous. The automation — run the tests on every push — is what makes integrating that often safe.
CD — continuous delivery, or continuous deployment. These are different, and the difference matters:
- Continuous delivery means the main branch is *always* in a releasable state, verified and packaged. Deploying is a decision someone makes, and it takes one click.
- Continuous deployment means there is no click. Every change that passes every check goes to production on its own.
Most teams should aim for delivery first. Deployment is the same machine with the last manual gate removed, and it is only sensible once the checks are actually trusted.
The pipeline, stage by stage
A pipeline is just these stages, in this order, every time, without a human choosing to.
1. A change is pushed Small and frequent beats large and rare. A change that touches four files is reviewable in minutes and diagnosable in seconds if it misbehaves.
2. The fast checks run first Formatting, linting, type checking, unit tests. These take under a minute, so the ordering matters: fail the cheap checks before spending money on the expensive ones. A typo in a type should never wait behind a browser test.
Two settings earn their keep immediately on any hosted runner. Cache the dependency install against a hash of the lockfile, so a branch that did not touch dependencies skips the install entirely. And cancel superseded runs — in GitHub Actions that is a concurrency group keyed on the branch with cancel-in-progress — otherwise three pushes in ten minutes means three full pipelines competing for the same runners, and the one you care about is last in the queue.
3. The build happens once This is the stage most hand-rolled processes get wrong. The project is built one time, producing one artifact, and that exact artifact is what goes to preview and then to production. Nothing is rebuilt per environment.
The reason is simple: if you build separately for staging and for production, you have tested one thing and shipped a different one. Configuration — database URLs, keys, feature switches — is read from the environment at run time, never compiled into a second build. That is the one rule from the twelve-factor list that pays for itself in the first month.
Two details make "the same artifact" literally true rather than approximately true. Reference it by content digest, not by a moving tag: latest is not a version, and neither is a tag someone can repoint. And give the pipeline short-lived credentials through OIDC federation rather than a long-lived cloud key sitting in repository secrets — a static deploy key is the most valuable secret in most organisations and the one nobody rotates.
4. A preview environment appears The change is deployed to its own temporary URL, with its own data, that nobody else can break. Reviewers open a real, working version of the change instead of reading a description of it.
This is the single most underrated stage. It turns "looks fine to me" into "I clicked it".
5. The slow checks run against the preview End-to-end tests drive a real browser through the journeys that matter. They take minutes, which is why they run here rather than first, and why there should be few of them.
6. Merge, then deploy Once everything is green and a human has approved the change, it merges — and merging is what triggers the production deployment. Not a person. Not a script somebody runs. The merge.
7. Migrations run inside the pipeline Database changes are code: written down, reviewed, versioned, and applied by the pipeline in order. Never typed into a production console at midnight.
The pattern that makes this safe is expand, then contract — four deployments where a rename looks like one:
- Add the new column, nullable. Nothing reads it yet.
- Deploy code that writes both columns and still reads the old one.
- Backfill in batches, then switch reads to the new column.
- In a later release, once nothing references it, drop the old one.
Every step is individually reversible, and at no point does the running code disagree with the schema. A single release that renames in place is reversible only in theory: roll the code back and it is now talking to a column that no longer exists.
The same care applies to the DDL itself, because migrations take locks. CREATE INDEX holds a write lock for its duration, so on a live table it is CREATE INDEX CONCURRENTLY. Adding a foreign key or check constraint validates the whole table under an exclusive lock unless you add it NOT VALID and run VALIDATE CONSTRAINT afterwards. And set a lock_timeout before any DDL: a migration that cannot get its lock should fail in two seconds and be retried, not queue behind a long-running query while every request piles up behind *it*. Adding a column with a default is the one that used to be feared and no longer needs to be — PostgreSQL 11 made it a metadata change rather than a table rewrite.
8. A smoke check runs against production Three or four assertions against the live site, immediately after deploying. The home page answers. Sign-in works. One record reads and writes. This catches the entire family of failures that pass every test and still take the site down: the environment variable that was never set in production, the migration that ran everywhere else, the build deployed to the wrong project.
9. There is a way back If the smoke check fails, the previous release is restored automatically, and if it does not fail, a person can still restore it in one step. The important part is that rollback has been *rehearsed*. A procedure nobody has ever run is a hope, not a plan.
How far back you can go depends on the deployment strategy you chose, so it is worth choosing on purpose:
- Rolling replaces instances a few at a time. Cheap, no spare capacity needed, but for a few minutes two versions are live at once — which is fine only if the schema supports both, which is the expand-and-contract discipline again.
- Blue-green brings the new version up beside the old one and switches traffic at the router. Rollback is the same switch in reverse, which makes it the fastest reversal available. It costs double capacity during the change.
- Canary sends a small share of traffic to the new version, watches error rates and latency, and proceeds or aborts on what it sees. It needs enough traffic for the sample to mean anything.
And the thing that removes most of the pressure from all three: put risky changes behind a feature flag, so deploying and releasing stop being the same event. The code ships dark on Tuesday with the flag off. On Thursday somebody turns it on for staff, then for five per cent, then for everyone — and turning it off again is a toggle rather than a deployment.
What changes when this exists
The technical description undersells it, because the real effects are about people.
Batch size collapses. Releasing becomes cheap, so it happens several times a day instead of every three weeks. Small batches mean that when something breaks, the cause is obvious — there is only one recent change to look at.
Fear leaves the room. Nobody dreads a deployment they have watched happen forty times this month and reversed twice without drama. Teams that are afraid to release stop releasing, and a product that ships once a quarter stops responding to its own users.
Releases stop depending on one person. The steps are in a file in the repository rather than in somebody's memory. People can take holidays. People can leave.
Review becomes real. A preview URL turns approval from a formality into an actual inspection.
Fridays become ordinary. The advice "never deploy on a Friday" is not a law of nature. It is a symptom of a process that cannot be undone quickly.
If you want to know whether any of this is actually working, there are four numbers worth tracking, and they have been the industry's standard set for years because they resist gaming: how often you deploy; how long a change takes to get from commit to production; what share of releases cause a problem; and how long recovery takes when one does. The useful insight from all the research into them is that the first two and the last two move *together*. Teams that ship more often recover faster, because small changes are easier to understand and easier to reverse. Shipping rarely to be safe is the thing that makes releases dangerous.
What it costs, and what it does not fix
A pipeline is not free and it is not magic.
It costs build minutes, and it costs maintenance: a pipeline is code, and it breaks like code. It costs discipline, because the moment a failing check is routinely skipped, the whole thing becomes theatre.
And it fixes nothing on its own. A pipeline is only as honest as the checks inside it. A green build with no tests behind it is a green light that means nothing at all — it says the code compiled, not that the product works. This is why the pipeline and the test suite are one piece of work rather than two: the tests give the pipeline something to say, and the pipeline makes sure the tests are actually run.
It will not fix a broken deployment target either. If production is a single server that has been hand-edited for three years, the first honest step is making that environment reproducible.
Starting on a project that already exists
None of this requires a rewrite. On a codebase that is already live, the order that pays fastest is:
- Get the build running in CI, even with no tests. Just proving the project builds from a clean checkout finds the file somebody forgot to commit.
- Add type checking and linting to the same run. Cheap, fast, immediately useful.
- Add one smoke check against production, running after each deploy.
- Automate the deployment itself, so it is one command rather than a remembered sequence — and make rollback that same command with a previous version.
- Add preview environments, so review has something to look at.
- Then start filling in the test layers, worst-risk first.
Each step is useful on its own, and each one makes the next cheaper. By step four, releasing has already stopped being an event — which was the whole point.