Bazel Monorepo Builds for Large Systems
Bazel makes large monorepo builds fast through hermetic, cached, incremental builds, but the cost is up-front rigor. When it pays off, and how to adopt it.
Part of Build Systems and Developer Infrastructure at Scale
A Bazel monorepo gets its speed from one idea: model the whole build as a dependency graph, rebuild only what changed, and cache every action so work done once is never repeated. For a large, multi-language codebase that has outgrown its native build tools, that is transformative. The catch is that Bazel demands up-front rigor (explicit dependencies, hermetic builds) that a small project does not need.
The honest framing is a tradeoff, not a universal win. Bazel is the right tool when build time and CI cost have become a tax you pay every day across many engineers. It is overkill for a single-language service that builds in ten seconds with the native toolchain.
Why build systems matter at scale
In a small repo, the build is invisible. In a large monorepo with hundreds of targets and several languages, the build becomes one of your biggest recurring costs: engineer wait time, CI minutes, and the friction of slow feedback loops that quietly slow every change.
A build system that rebuilds everything on every change does not scale. Past a certain size, the only way to keep builds fast is to rebuild only what changed and reuse everything else. That is exactly what Bazel is designed to do. This post is part of the Build systems series.
The cost is easy to underestimate because it is distributed rather than concentrated. A build that takes eight minutes instead of ninety seconds does not appear on any budget line, but it changes engineer behaviour in ways that compound: people batch changes into larger commits to amortise the wait, which makes review harder and regressions harder to bisect. They context-switch during builds, paying the real cost in lost focus rather than in wall-clock minutes. They run fewer tests locally and let CI find the problem, lengthening the feedback loop further. None of that shows up as “build cost” in a retrospective, and all of it is caused by build cost.
What makes Bazel fast for large monorepos?
Bazel represents your build as a directed graph of actions, each with declared inputs and outputs, and it rebuilds only the actions whose inputs changed. Every action’s result is cached and keyed by its inputs, so an unchanged target is never rebuilt. With a shared remote cache, a result computed once by CI or a teammate is reused by everyone.
The combination is what produces the speed. Incremental builds mean a one-line change rebuilds one target and its dependents, not the world. The remote cache means even a fresh checkout or a CI run mostly downloads pre-built artifacts instead of compiling them.
| Mechanism | What it does | Why it speeds builds |
|---|---|---|
| Dependency graph | Models inputs/outputs per target | Knows exactly what a change affects |
| Incremental builds | Rebuilds only changed targets + dependents | Avoids rebuilding the whole repo |
| Action caching | Caches each action keyed by inputs | Unchanged work is never repeated |
| Remote cache | Shares the cache across machines | Work done once is reused by all |
| Remote execution | Runs build actions on a cluster | Parallelism beyond one machine |
What is hermeticity in Bazel, and why does it matter?
Hermeticity means a build action depends only on its explicitly declared inputs, never on ambient state like system-installed tools or environment variables. A hermetic build is reproducible: the same inputs always produce the same output, on any machine. That property is the entire reason Bazel’s caching is safe to trust.
This is the rigor people complain about and the rigor that makes everything else work. If a build secretly depended on a tool that happened to be installed, the cache could hand you a stale or wrong result on a different machine. By forcing dependencies to be explicit, Bazel guarantees that a cache hit is always correct, which is what lets it cache aggressively across an entire fleet.
Does Bazel work with Go, Rust, Java, and Python together?
Yes. Bazel is language-agnostic by design, which is precisely why large polyglot systems adopt it. With the appropriate rule sets per language, Go, Rust, Java, Python, and others build in a single graph backed by a single cache, so a cross-language change is one coherent build rather than several disconnected ones.
This is the feature that justifies Bazel for many teams. A polyglot fleet otherwise needs a different build tool per language, each with its own caching, its own CI wiring, and no shared view of cross-language dependencies. Bazel unifies them: one command builds the Go service, the Rust hot path, and the shared protobuf definitions, and the cache spans all of it. For why those languages coexist in the first place, see Go vs Rust for Microservices: When to Choose Which.
The case that makes this concrete is generated code, and protobuf is the canonical example. A schema change should rebuild the stubs for every language that consumes it, rebuild the services depending on those stubs, and rerun exactly the affected tests — no more. Without a unified graph, that coordination is a hand-written script that regenerates everything, rebuilds everything downstream, and is wrong in one of two directions: either it over-builds and you lose the speed, or it under-builds and a service ships against a stale stub. Neither failure is obvious until it reaches production.
With one graph, the schema file is simply an input, and correctness falls out of the dependency edges rather than out of anyone’s discipline. This is worth weighing carefully during evaluation, because it is the benefit that does not have a cheap workaround. Slow builds can be survived with more CI machines; a cross-language dependency graph that only exists in someone’s head cannot be bought.
The hermeticity violations you will actually hit
“Declare your dependencies” sounds simple until a build breaks on a colleague’s machine and not yours. These are the leaks that cause it, roughly in order of how often they appear:
| Violation | How it shows up | The fix |
|---|---|---|
| System toolchain | Builds with the compiler that happens to be on PATH | Register an explicit toolchain; pin the compiler version as a dependency |
| Undeclared file reads | Works locally, fails in CI or in a sandbox | Add the file to srcs or data; enable sandboxing so it fails fast everywhere |
| Network access during build | Passes today, breaks when a registry is down or a version moves | Vendor or pin the dependency with a checksum; never fetch at build time |
| Environment variables | Different output depending on who ran it | Pass explicitly via --action_env, or eliminate the dependence |
| Timestamps and build IDs | Identical inputs produce non-identical outputs, destroying cache hits | Use fixed or stamped values kept out of the cache key |
| Absolute paths | Cache misses across machines with different checkout locations | Keep paths repo-relative |
Enable sandboxing early rather than late. It converts every one of these from an intermittent mystery into a deterministic failure at the moment the mistake is made, which is enormously cheaper than discovering it six months in when the cache silently starts serving wrong results.
The cache-hit arithmetic that decides everything
Bazel’s value is almost entirely a function of cache hit rate, and it is worth doing the arithmetic before adopting, because the result is frequently surprising.
Take a build with 1,000 actions, averaging 2 seconds each: 2,000 seconds of work, or roughly 33 minutes serialised. At a 95% cache hit rate, you execute 50 actions — about 100 seconds. At 80%, you execute 200 actions, about 400 seconds. At 50%, 1,000 seconds.
The shape of that curve is the important part: the difference between 95% and 80% is four times the build time, even though both sound like “good caching.” This is why hermeticity is not pedantry. Every non-hermetic action is a permanent cache miss for everyone, forever, and a handful of them scattered through a hot part of the graph can move you from 95% to 80% and quietly erase most of the benefit you migrated for.
It also explains why remote caching matters more than remote execution for most teams. A shared cache moves CI and every engineer onto the same hit rate; without it, each machine warms its own cache and a fresh checkout pays full price.
What does Bazel actually cost you?
The tables above are the upside. An honest assessment has to state the bill, because Bazel migrations fail regularly and they fail for predictable reasons.
The migration is not a weekend. Converting a large existing repo means writing build files for every target, untangling implicit dependencies that have accumulated for years, and discovering that a surprising amount of your build was relying on ambient state. Budget months for a large codebase, and expect the untangling — not the Bazel syntax — to be the slow part.
You inherit a second dependency ecosystem. Your language’s native package manager still exists, but Bazel needs its own view of those dependencies. Modern Bazel uses Bzlmod for this, which is a real improvement over the older workspace approach, but it is still a distinct system to learn, keep current, and debug when a transitive version resolves differently than your language’s native tool would resolve it.
Tooling and IDE integration are worse. Your editor’s language server, debugger, and test runner all expect the native layout. Bazel-generated outputs and sandboxed paths break some of those assumptions, and the plugins that bridge the gap vary in quality by language. This is a daily papercut for every engineer, not a one-time cost, and it is the complaint that shows up most in practice.
Third-party rules vary in maturity. Bazel’s language support is delivered through rule sets, and they are not uniformly polished. Some are excellent and well maintained; others lag behind their language’s current release, and you will occasionally be the person reading rule source to understand why a flag does nothing.
You need someone who owns it. Non-obviously, this is the decisive factor. A Bazel monorepo without a person or small group responsible for build health decays: hermeticity leaks accumulate, cache hit rate drifts down, everyone works around problems locally, and eventually the team concludes “Bazel is slow” when what happened is that nobody was maintaining the property that made it fast.
The honest summary: Bazel trades a large, front-loaded, ongoing complexity cost for a build that stays fast as the repo grows. That trade is excellent at sufficient scale and clearly bad below it. The failure mode is not choosing Bazel — it is choosing Bazel at a scale where the daily tax exceeds the daily saving, and then discovering the migration is hard to reverse.
The architecture I run uses Bazel across a polyglot monorepo spanning Go, Java, Elixir, and Rust services, with versioned protobuf contracts laid out by directory so a v2 can live beside a v1 rather than mutating it. That layout is not incidental to the build system — a single graph over generated protobuf stubs consumed by four languages is precisely the case where per-language build tools stop composing, and it is the clearest justification for paying the tax. For the contract-versioning discipline that goes with it, see Protobuf Schema Evolution Without Breaking Clients.
When is Bazel worth the complexity?
Bazel is worth it when you have a large, multi-language monorepo, many engineers sharing code, and build or CI time that has become a measurable cost. It is not worth it for a small or single-language project, where the native toolchain is simpler and the time saved does not repay the up-front investment.
Run this gut check before adopting:
- Is the repo large and polyglot, with shared code across languages?
- Have build and CI times become a real, recurring drag on the team?
- Do many engineers touch the repo, so a shared cache pays off across people?
- Can you fund the migration and the ongoing
BUILD-file maintenance?
If most answers are yes, Bazel pays for itself. If you are a small team on one language with fast builds, adopting Bazel imports a lot of rigor to solve a problem you do not have yet.
Bazel vs other build tools: when to switch
Switch to Bazel when your build has outgrown what a language-native or single-purpose tool can keep fast, and when the build spans multiple languages. Below that threshold, the native tool is simpler and the right call. The decision is about scale and polyglot needs, not about Bazel being “better” in the abstract.
| Build tool | Best for | Weakness at scale |
|---|---|---|
| Native (go build, cargo, etc.) | Single-language projects | No cross-language graph; rebuilds widen as repo grows |
| Make | Small/medium, simple deps | Manual dependency tracking; not hermetic or cached across machines |
| Gradle / Maven | JVM ecosystems | JVM-centric; weaker for polyglot monorepos |
| Bazel | Large polyglot monorepos | Up-front rigor and BUILD-file maintenance |
The pattern across the table is that the simpler tools are better until the repo gets large and multi-language, at which point their lack of a shared dependency graph and cross-machine cache becomes the bottleneck. Bazel inverts that: heavy to start, but its incremental, cached, hermetic model is what keeps a huge polyglot build fast. The switch is justified by the pain you are already feeling, not by anticipation.
How to adopt Bazel without a big-bang rewrite
The failure mode is trying to convert an entire large repo to Bazel in one quarter. The better path is incremental: introduce Bazel alongside the existing build, convert a few high-value targets, stand up the remote cache early (because the cache is where most of the win comes from), and expand outward as the team learns the idioms.
Prioritize the remote cache and CI integration first. A lot of Bazel’s value is realized the moment CI and engineers share a cache, even before the whole repo is converted. Tooling like gazelle can generate and maintain BUILD files for some languages, which removes much of the manual bookkeeping people fear. Migrate the slowest, most-shared parts of the build first, where the payoff is largest.
How do you keep a Bazel build fast over time?
Adoption gets all the attention, but the more common failure is a Bazel repo that was fast in year one and is slow in year three. Nothing dramatic happened; the properties that made it fast were never measured, so they eroded silently.
Four things are worth treating as monitored production metrics rather than folklore:
- Cache hit rate, tracked over time. This is the single number that predicts build time. A downward trend is the leading indicator of hermeticity decay, and it shows up long before anyone complains. If you instrument one thing, instrument this.
- The dependency fan-out of your hottest targets. A widely-depended-on target that gains a heavy new dependency quietly increases the rebuild cost of every change beneath it. A base library that pulls in something large can add minutes to everyone’s incremental build without any single commit looking expensive.
- Test target granularity. One giant test target means any change in its dependency closure reruns all of it. Splitting tests along the same seams as the code is what keeps the “only what changed” promise true for the test phase, which is usually where the time actually goes.
- Build file drift. Dependencies declared but unused, or used but undeclared and surviving only because sandboxing is disabled somewhere. Both accumulate. Automated dependency-management tooling that keeps build files in sync with source imports removes most of this class of decay.
The organisational point underneath all four: build speed is a property that must be maintained, not a state you reach. Every new engineer, every new dependency, and every deadline-driven shortcut applies pressure toward a slower build. Without someone watching the numbers, the pressure always wins, and the team’s eventual conclusion is that the tool failed rather than that the discipline lapsed.
A practical governance habit that costs little: treat a significant cache-hit-rate regression the same way you would treat a latency regression in a service. It gets noticed, it gets an owner, and it gets fixed while the cause is still one commit rather than two hundred.
What I’d do differently
The trap I would warn against is adopting Bazel for the prestige of it, on a codebase that did not need it. Bazel solves a real and painful problem, but if you do not have that problem, you have imported its considerable complexity for no return, and your team will resent every BUILD file.
If I were adopting Bazel again, I would be ruthless about sequencing: stand up the remote cache first so the speed win is immediate, convert the highest-pain targets next, and lean on generation tools to keep BUILD files from becoming a chore. And I would only start at all once build time was a number the team complained about, because that complaint is the signal that the rigor will pay for itself.
Sources
- Bazel, Why Bazel?: bazel.build/about/why
- Bazel, Remote caching: bazel.build/remote/caching
- Bazel, Hermeticity: bazel.build/basics/hermeticity
Frequently asked questions
What makes Bazel fast for large monorepos?
Bazel models the build as a dependency graph and only rebuilds what actually changed, then caches every action. With a shared remote cache, work done once by anyone (or CI) is reused by everyone, so most builds are cache hits rather than full rebuilds.
When is Bazel worth the complexity?
Bazel pays off for large, multi-language monorepos where build times and CI cost have become a real tax, and where many engineers share code. For a small single-language project, Bazel's up-front rigor usually costs more than it saves; the native toolchain is simpler.
What is hermeticity in Bazel?
Hermeticity means a build depends only on its declared inputs, not on whatever happens to be installed on the machine. Hermetic builds are reproducible and safely cacheable, because the same inputs always produce the same output regardless of where they run.
Does Bazel work with Go, Rust, Java, and Python together?
Yes. Bazel is language-agnostic and is built for polyglot monorepos. With the appropriate rule sets, Go, Rust, Java, Python, and more build in one graph with one cache, which is a major reason large polyglot systems adopt it.
What are the downsides of Bazel?
A long migration for existing repos, a second dependency ecosystem to learn alongside your language's native one, degraded IDE and debugger integration, third-party rule sets of uneven maturity, and the need for someone to own build health. Without an owner, hermeticity leaks accumulate and cache hit rate quietly decays.
Why does Bazel cache hit rate matter so much?
Because build time is almost entirely a function of it. For a 1,000-action build averaging 2 seconds per action, a 95% hit rate executes about 100 seconds of work while 80% executes about 400 seconds. Every non-hermetic action is a permanent cache miss for everyone, which is why hermeticity is not pedantry.
What breaks hermeticity in a Bazel build?
Depending on system-installed toolchains, reading undeclared files, fetching from the network during a build, reading environment variables, embedding timestamps, and using absolute paths. Enabling sandboxing early turns each of these from an intermittent mystery into a deterministic failure at the moment the mistake is introduced.