Why Your CI/CD Pipeline Is a Liar
Green builds. Red outages. Let’s talk about it.
Your pipeline was green.
All checks passed.
Linting? Clean.
Validation? Successful.
Approval? Clicked.
And then production immediately fell over like a folding chair at a backyard BBQ.
If the pipeline was green… how did prod still catch on fire?
Welcome to the finale of the CI/CD series.
Today we’re talking about the uncomfortable truth:
Your pipeline might be lying to you.
Lie #1 — “All Tests Passed”
This one feels comforting.
“All tests passed” sounds like:
We’re safe.
We’re mature.
We’re DevOps now.
But what did those tests actually test?
For infrastructure teams, most pipelines check structure — not intent.
Examples you’ve probably lived through:
Ansible syntax check passes… but you referenced the wrong variable group.
Terraform validates… but your plan destroys the wrong resource.
Firewall rules commit cleanly… but you just shadowed a critical ACL.
YAML lints perfectly… and still points production traffic to the wrong subnet.
Passing lint is not the same thing as validating logic.
A syntax check will happily confirm that your disaster is formatted correctly.
Most infrastructure pipelines prove that the config is legal.
They do not prove that it is smart.
Lie #2 — “It Worked in QA”
Ah yes. QA.
Where everything works.
Because QA:
Has fewer devices.
Has less traffic.
Has no mystery static routes from 2011.
Doesn’t have that one undocumented NAT rule nobody wants to touch.
Production is messy.
It has history.
It has hotfixes.
It has “temporary” changes from five years ago.
And if your environments drift — which they absolutely do — then “it worked in QA” just means:
“It worked in a cleaner universe.”
Environment drift is one of the biggest lies pipelines hide.
Your CI/CD process assumes parity.
Your network does not.
Lie #3 — “We Have a Pipeline, So We’re Mature”
This one hurts.
Some pipelines exist for one reason:
To push changes faster.
That’s it.
No diff validation.
No policy enforcement.
No blast radius awareness.
No rollback plan.
No drift detection.
Just:
Push button → Deploy everywhere → Hope.
If your pipeline’s only job is to move changes faster, it’s not CI/CD.
It’s a cannon.
Speed without guardrails is not maturity.
It’s automation with confidence issues.
Lie #4 — “Green Means Safe”
Dashboards are addictive.
Big green checkmarks.
Happy pipeline stages.
Pretty graphs.
But green only means:
“The steps we defined completed successfully.”
If the steps are shallow…
The green is meaningless.
You can have:
Zero validation of route impact.
Zero awareness of overlapping firewall rules.
Zero health checks after deployment.
And still get a big green check.
Your pipeline doesn’t know your business impact.
It only knows the commands it ran.
So What Makes a Pipeline Honest?
An honest pipeline does more than execute.
It questions the change.
At a high level, honest infrastructure pipelines include things like:
Pre-change diffs reviewed before apply.
Policy checks (not just syntax checks).
Approval gates with context.
Rollback strategies that are real — not “revert manually.”
Drift detection that catches CLI hotfixes.
Post-deploy validation (connectivity, routing, service health).
Honest pipelines try to break your change before production does.
They are guardrails — not speedrun buttons.
The Real Goal of CI/CD (For Infrastructure)
CI/CD is not about deploying faster.
It’s about feedback.
It’s about safety.
It’s about visibility.
It’s about being able to answer:
Who approved this?
What changed?
Why did it change?
Can we roll it back?
Does production actually match Git?
Automation doesn’t remove responsibility.
It removes excuses.
Want to Build a Pipeline That Doesn’t Lie?
In the Extended Version, I break down:
The four levels of infrastructure pipeline maturity
What real CI looks like for Ansible, Terraform, and firewall changes
Drift detection patterns most teams ignore
Blast radius awareness before deployment
Rollback strategies that don’t require panic
A practical CI/CD self-assessment checklist you can use immediately
If your pipeline is green but prod still makes you nervous…
that version is for you.
That’s the finale of the CI/CD series.
You don’t need trendier tools.
You need stricter feedback loops.
And maybe fewer green checkmarks you don’t trust.
— JJ – Chief Packet Pusher