A case study
Gauge 2022
Making 'is it slower?' a merge check instead of a customer complaint.
- In one line
- Load-testing harness with budget enforcement
- My role
- Engineer — harness design, CI integration, budget model
- Timeline
- 4 months · 2022
- Team
- 2 engineers, rolled out org-wide
- Built with
- Python, k6, Grafana, GitHub Actions
Gauge is a load-testing harness that turns performance budgets into a required CI check. Instead of hoping a release didn't regress, every merge runs a representative load test and fails if latency or throughput crosses a defined budget — so regressions are caught by the pipeline, not by users.
Where it started
The situation
Performance problems were discovered in production, by customers, often days after the release that caused them. Every deploy carried an unspoken bet that nothing had gotten slower, and when the bet lost, the investigation started from zero.
The constraint
A perf gate that is noisy gets disabled. Load tests are inherently variable, so the check had to distinguish a real regression from normal run-to-run jitter — a flaky gate that cries wolf is worse than no gate at all.
How it was built
Budgets as code
Each service declares latency and throughput budgets in a versioned file that lives next to the code. The budget is reviewed like any other change, so tightening or relaxing a threshold is a visible, deliberate decision rather than a silent config drift.
Representative load in CI
k6 scenarios model real traffic shapes and run against an ephemeral environment on every pull request. The result is compared to the budget, and a breach fails the merge with a report pointing at the offending endpoint.
Statistical gating, not single-run
Each run reports a distribution, and the gate compares percentiles across repeated samples rather than a single number. This is what kept false positives low enough that teams trusted the check instead of routing around it.
Trends anyone can see
Results feed Grafana dashboards, so a service's latency over time is visible to the whole team and slow drift becomes obvious long before it crosses a budget and blocks a merge.
What it changed
- Regression escapes in 14 months
- 0
- Gated on a perf budget
- Every merge
- False-positive gate rate
- <2%
- Adoption across teams
- org-wide
Looking back
The statistical gate was the whole ballgame — the first naive version compared single runs, was flaky within a week, and nearly got the check disabled before it earned trust. What I'd add sooner is automatic root-cause hints: telling a team a merge is slower is useful, but pointing at the specific query or call that regressed is what turns the gate from a blocker into a helper.