Jeffrey Zalischi

← Back to all work

A case study

Quorum 2023

One audited control plane for flags, so a config change can't ship a half-built feature.

In one line
Consensus-backed feature flag service
My role
Backend engineer — control plane, SDK, rollout tooling
Timeline
5 months · 2022–2023
Team
2 engineers, adopted across 6 product teams
Built with
Rust, etcd, gRPC, Terraform

Quorum consolidated feature flags — previously scattered across config files in several repositories — into a single consensus-backed service. Every change is audited, propagates in under a second, and can be rolled back to any prior state, which turned flags from a liability into a safe deployment tool.

Where it started

The situation

Flags lived in five config files across three repos with no single owner. A stale flag once shipped an unfinished checkout flow to every user for six hours before anyone could find the right file to flip. There was no history, no audit, and no fast way to undo.

The constraint

A flag service sits in the request path of every product it touches. It has to be fast, it has to be highly available, and a change must never leave two servers disagreeing about whether a feature is on — a split-brain flag is its own kind of outage.

How it was built

Consensus-backed state

Flag state is stored in etcd, so a change is agreed by a quorum before it takes effect and every reader converges on the same value. There is no window where two servers disagree about whether a feature is live.

A thin, fast SDK

Services embed a Rust SDK that streams flag updates over gRPC and evaluates locally against an in-memory snapshot. Flag checks cost microseconds and never block on a network call in the hot path, so adoption never came at a latency cost.

Audit and rollback as core features

Every change records who, what, and why, and the full history is addressable — rolling back is selecting a previous revision, not editing a file and redeploying. Post-incident reviews got their timeline for free.

Targeting rules

Flags support percentage rollouts and attribute-based targeting, so a feature can be released to one percent of traffic or a single internal cohort and widened with confidence rather than shipped all-or-nothing.

What it changed

Flag propagation
<1s
Audited control plane (was 5 files)
1
Adopted the SDK
6 teams
Per-flag evaluation cost
µs

Looking back

Putting audit and rollback in from the start, rather than treating them as a later 'enterprise' feature, is what made teams comfortable trusting Quorum with risky rollouts. If I revisited it I'd invest more in the local-testing story — engineers wanted to run flag rules on their laptops without the full control plane, and that ergonomic gap slowed early adoption more than any technical limitation did.