NOTE: This documentation is meant for people who are struggling with Bazel-only build failures. If instead you want a high-level guide to performing day-to-day development tasks using Bazel, check out the "Developing with Bazel" documentation.
What is Bazel? Why do I have to think about it?
Bazel is a modern build system with better performance and correctness guarantees than our old build system, make. Since weâre currently only partly through the migration to Bazel, weâre currently in a limbo where we need to ensure CockroachDB can build with both Bazel and make until weâre ready to switch over entirely.
If youâre interested in watching or contributing to the progress of the Bazel migration, you can check out the project tracker on GitHub.
CI is failing because the Bazel build is broken. What do I do?
Compared to make, Bazel is much more rigorous about properly declaring dependencies between build targets. Most issues with the Bazel build can be solved by identifying and fixing the missing dependencies.
dev generate bazel
We use an open-source tool called Gazelle which automatically fixes up all the dependencies in the workspace for all our Go and protobuf code. You can run Gazelle with ./dev generate bazel. (bors will not let you merge unless this command can be run cleanly, so you may want to make running this a part of your normal workflow.) If Gazelle notices changes, this will result in updates to BUILD.bazel files in tree, or to the root-level WORKSPACE or DEPS.bzl files. Then, re-run the Bazel build to ensure this fixes the failure. Note that ./dev generate bazel --mirror is required when vendoring new dependencies.
Updating dependencies for protos
There is a class of Bazel build errors that Gazelle canât handle, and manual intervention is necessary. In this case you typically need to manually add a dependency to the corresponding BUILD.bazel file for the package thatâs failing to build. Hereâs a current example (at time of writing).
In pkg/sql/execinfrapb/processors_bulk_io.proto you can see the following line of code:
The protobuf compiler will turn this into a .pb.go file that imports the package github.com/cockroachdb/cockroach/pkg/ccl/streamingccl; but Gazelle doesnât know that, so ./dev generate bazel wonât insert the necessary dependency. This manifests as an error when you try to bazel build:
compilepkg: missing strict dependencies:
/private/var/tmp/_bazel_ricky/be70b24e7357091e16c49d70921b7985/sandbox/darwin-sandbox/2323/execroot/cockroach/bazel-out/darwin-fastbuild/bin/pkg/sql/execinfrapb/execinfrapb_go_proto_/github.com/cockroachdb/cockroach/pkg/sql/execinfrapb/processors_bulk_io.pb.go: import of "github.com/cockroachdb/cockroach/pkg/ccl/streamingccl"
We fix this by adding the dependency ourselves, in pkg/sql/execinfrapb/BUILD.bazel:
The # keep comment is very important as it tells Gazelle not to remove a dependency it would otherwise think is unnecessary. Generally, for missing dependencies on Go code in the github.com/cockroachdb/cockroach repo, just strip out the prefix and replace it with // to get Bazel to look in the right place (for example, here weâve replaced github.com/cockroachdb/cockroach/pkg/ccl/streamingccl => "//pkg/ccl/streamingccl").
After you manually update your dependencies and ensure the build has succeeded, itâs a good idea to re-run ./dev generate bazel to have Gazelle fix everything up. This will let you know if you forgot any # keep comments.
CI is failing because a Go test is broken in Bazel. What do I do?
First, make sure you follow all of the above advice. If building the test binaries succeeds but running the test fails, and that same test failure doesnât show up in the non-Bazel CI jobs, youâve probably identified a case where Bazelâs test sandboxing has caused a breakage. Thereâs no one-size-fits-all solution to this; fixes will vary wildly depending on what the problem is and how it manifests. Generally, we recommend you ask for help (see below).
For integration tests, lint tests, or other heavyweight tests, it might be impractical to land Bazel support in the same PR, in which case it can be appropriate to tag your test as broken_in_bazel. Make sure to follow up with a member of the developer infrastructure team when you do so we can make sure the migration to Bazel gets triaged appropriately.
How to ask for help
If you donât know how to solve a build or test failure, engage with a member of the developer infrastructure team. If youâre a Cockroach Labs employee, you can reach out to #bazel on Slack. External contributors can ping #contributors on the CockroachDB Community Slack, or you can engage the reviewer on your pull request for help.
See the comment at the top of DEPS.bzl for help on injecting temporary changes to dependencies. This is useful if you want to test changes to dependencies before upstreaming them or if you want to inject debugging logic into third-party code.
Generated files handled by go:generate are not natively generated in the Bazel sandbox, so introductions of new calls to go:generate can break the Bazel build or otherwise lead to unpredictable or incorrect behavior. If you add a new go:generate somewhere, make sure the equivalent logic to generate the file is present in the Bazel build as well. Follow the directions above to ask for help if you donât know how to do this.