How to ensure your code builds with Bazel

What is Bazel? Why do I have to think about it?

Bazel is a modern build system with better performance and correctness guarantees than our old build system, make. Since we’re currently only partly through the migration to Bazel, we’re currently in a limbo where we need to ensure CockroachDB can build with both Bazel and make until we’re ready to switch over entirely.

If you’re interested in watching or contributing to the progress of the Bazel migration, you can check out the project tracker on GitHub.

How can I build CockroachDB with Bazel?

Currently, only the buildshort target fully builds with Bazel. From the root of the cockroach repo, you can build CockroachDB with bazel build pkg/cmd/cockroach-short.

CI is failing because the Bazel build is broken. What do I do?

Compared to make, Bazel is much more rigorous about properly declaring dependencies between build targets. Most issues with the Bazel build can be solved by identifying and fixing the missing dependencies.

make bazel-generate

We use an open-source tool called Gazelle which automatically fixes up all the dependencies in the workspace for all our Go and protobuf code. You can run Gazelle with make bazel-generate. (bors will not let you merge unless this command can be run cleanly, so you may want to make running this a part of your normal workflow.) If Gazelle notices changes, this will result in updates to BUILD.bazel files in tree, or to the root-level WORKSPACE or DEPS.bzl files. Then, re-run the Bazel build to ensure this fixes the failure.

Sometimes make bazel-generate hangs indefinitely for an unknown reason. This is a known issue. You can just CTRL-C and run it again.

Updating dependencies for protos

There is a class of Bazel build errors that Gazelle can’t handle, and manual intervention is necessary. In this case you typically need to manually add a dependency to the corresponding BUILD.bazel file for the package that’s failing to build. Here’s a current example (at time of writing).

In pkg/sql/execinfrapb/processors_bulk_io.proto you can see the following line of code:

1 repeated string partition_addresses = 1 [(gogoproto.customtype) = "",(gogoproto.nullable) = false]

The protobuf compiler will turn this into a .pb.go file that imports the package; but Gazelle doesn’t know that, so make bazel-generate won’t insert the necessary dependency. This manifests as an error when you try to bazel build:

1 2 compilepkg: missing strict dependencies: /private/var/tmp/_bazel_ricky/be70b24e7357091e16c49d70921b7985/sandbox/darwin-sandbox/2323/execroot/cockroach/bazel-out/darwin-fastbuild/bin/pkg/sql/execinfrapb/execinfrapb_go_proto_/ import of ""

We fix this by adding the dependency ourselves, in pkg/sql/execinfrapb/BUILD.bazel:

1 2 3 4 5 go_proto_library( name = "execinfrapb_go_proto", ... deps = [ "//pkg/ccl/streamingccl", # keep

The # keep comment is very important as it tells Gazelle not to remove a dependency it would otherwise think is unnecessary. Generally, for missing dependencies on Go code in the repo, just strip out the prefix and replace it with // to get Bazel to look in the right place (for example, here we’ve replaced => "//pkg/ccl/streamingccl").

After you manually update your dependencies and ensure the build has succeeded, it’s a good idea to re-run make bazel-generate to have Gazelle fix everything up. This will let you know if you forgot any # keep comments.

CI is failing because a Go test is broken in Bazel. What do I do?

See full article here!

First, make sure you follow all of the above advice. If building the test binaries succeeds but running the test fails, and that same test failure doesn’t show up in the non-Bazel CI jobs, you’ve probably identified a case where Bazel’s test sandboxing has caused a breakage. There’s no one-size-fits-all solution to this; fixes will vary wildly depending on what the problem is and how it manifests. Generally, we recommend you ask for help (see below).

For integration tests, lint tests, or other heavyweight tests, it might be impractical to land Bazel support in the same PR, in which case it can be appropriate to tag your test as broken_in_bazel. Make sure to follow up with a member of the developer infrastructure team when you do so we can make sure the migration to Bazel gets triaged appropriately.

How to ask for help

If you don’t know how to solve a build or test failure, engage with a member of the developer infrastructure team. If you’re a Cockroach Labs employee, you can reach out to #bazel on Slack. External contributors can ping #contributors on the CockroachDB Community Slack, or you can engage the reviewer on your pull request for help.

Miscellaneous tips

  • See the comment at the top of DEPS.bzl for help on injecting temporary changes to dependencies. This is useful if you want to test changes to dependencies before upstreaming them or if you want to inject debugging logic into third-party code.

  • Generated files handled by go:generate are not natively generated in the Bazel sandbox, so introductions of new calls to go:generate can break the Bazel build or otherwise lead to unpredictable or incorrect behavior. If you add a new go:generate somewhere, make sure the equivalent logic to generate the file is present in the Bazel build as well. Follow the directions above to ask for help if you don’t know how to do this.