We’re partway through a migration of our build system to Bazel. Bazel is a modern build system with better performance characteristics and correctness guarantees than we currently have with make/go build. Today, you can perform almost all day-to-day CRDB dev tasks with Bazel rather than with make. make is deprecated and we will remove this option for building Cockroach at some point in the next release cycle.

For the few tasks that you can't/don't know how to do with Bazel, please ask a question or contribute a solution. The Bazel migration is actively in progress and we always accept contributions even for minor/QOL improvements.

NOTE: for specific debugging tips on:
- Build failures, see “How to ensure your code builds with Bazel”
- Test failures, see “How to ensure your tests can run in the Bazel sandbox”

Prerequisites

Follow the directions on "Getting and building CockroachDB from source" or "Building from source on macOS" to get your development environment set up.

Note that you do need a full installation of XCode to build on macOS. (No, a command-line tools instance does not suffice.) If you’ve already installed a command-line tools instance as part of setting up Homebrew or other tools, switch to the full XCode instance and accept the XCode license agreement with:

sudo xcode-select -s /Applications/Xcode.app && sudo xcodebuild -license accept

Getting started

Introduction to dev

dev is a light wrapper around Bazel that is well-suited to specific CRDB development tasks. Properly, when we say dev, we’re referring to the Go binary whose source lives in cockroach/pkg/cmd/dev, side-by-side with the Cockroach source. At the top level of the repo there is a wrapper script also called dev whose only job is to build pkg/cmd/dev and then invoke it with the passed-in arguments. This means that you can always type ./dev to run the version of dev at the commit you’ve checked out, which is typically what you want to do.

Note that dev is meant to supplement bazel rather than replace it. dev makes certain tasks easier than bazel, but simple builds and tests are easy enough to run via bazel directly.

Doctor, doctor

Before trying to build anything, run the following:

./dev doctor

If dev notices any issues with your system that might prevent you from being able to build, it will let you know and tell you how to fix it. Make sure to run dev doctor before you ask for help just in case it catches something you didn’t know about.

NOTE: dev will take a while to run the first time you execute it since it needs to build itself. Be patient. (smile)

First steps: building cockroach

Start by building cockroach-short as follows:

bazel build pkg/cmd/cockroach-short

./dev build short also works as an alias. Bazel will pretty-print build output to the terminal.

You can also build the full cockroach binary which includes

bazel build pkg/cmd/cockroach --config=with_ui

./dev build (or equivalently, ./dev build cockroach) is a synonym for this.

dev build is a light wrapper for bazel build that supports aliases for common targets (for example, ./dev build crlfmt instead of the harder-to-remember bazel build @com_github_cockroachdb_crlfmt//:crlfmt). dev also copies binaries out of the Bazel output directories for you into your workspace; for example, bazel build pkg/cmd/cockroach-short puts the binary in _bazel/bin/pkg/cmd/cockroach-short/cockroach-short_/cockroach-short, but if you dev build short the binary will be staged at ./cockroach instead.

Run ./dev help build for more information about what you can do with dev build. Note that you can pass additional arguments directly to bazel build by adding them after --:

# build "verbosely", outputting all commands as they are run
./dev build short -- -s

dev lets you cross-compile with the --cross option, as in:

./dev build --cross

--cross takes an optional argument which is the platform to cross-compile to: --cross=linux, --cross=windows, --cross=macos, --cross=linuxarm, --cross=macosarm. dev will copy the built binaries into the artifacts directory in this case. Note that cross-building requires Docker. Cross-compiling should work on M1 Macs, but this support is experimental, so report issues if you should observe any.

For more debugging tips on building with Bazel, see “How to ensure your code builds with Bazel”.

Running tests

You can run all tests in a given package (for example, pkg/sql) with:

bazel test pkg/sql:all

bazel test pkg/sql/... will instead run all tests under the pkg/sql directory recursively, including all the tests under pkg/sql/types and pkg/sql/rowinfra among others.

If the test passes, you’ll see a brief message indicating which tests were run and their status:

INFO: Elapsed time: 7.842s, Critical Path: 7.26s
INFO: 48 processes: 3 internal, 45 darwin-sandbox.
INFO: Build completed successfully, 48 total actions
//pkg/sql/types:types_test                                               PASSED in 0.7s

If the test doesn’t pass, Bazel will print the location of the test’s log file:

INFO: Elapsed time: 8.763s, Critical Path: 7.94s
INFO: 46 processes: 1 internal, 45 darwin-sandbox.
INFO: Build completed, 1 test FAILED, 46 total actions
//pkg/sql/types:types_test                                               FAILED in 1.5s
  /private/var/tmp/_bazel_ricky/be70b24e7357091e16c49d70921b7985/execroot/cockroach/bazel-out/darwin-fastbuild/testlogs/pkg/sql/types/types_test/test.log

You can examine that file to see the complete logs for the test.

./dev test is provided as a shorthand for bazel test with some additional conveniences. Run ./dev test with the name of one or more packages to execute all tests from those packages:

./dev test pkg/sql

Tips and tricks

Other tasks

# Run acceptance tests
./dev acceptance
# Run compose tests
./dev compose
# Run benchmarks for pkg/sql/parser
./dev bench pkg/sql/parser
# Generate code (run this before submitting your PR).
./dev generate
# Run lints
./dev lint
# logic tests!
./dev testlogic --files=$FILES --subtests=$SUBTESTS --config=$CONFIG
# Open a container running the "bazelbuilder" image. Requires Docker.
./dev builder

dev vs. make

This is a (non-exhaustive) 1-to-1 mapping of dev commands to their make equivalents. Feel free to add to this (smile)

dev/bazel command

equivalent non-bazel command

./dev build

make build

./dev build short

make buildshort

./dev build pkg/sql/...

make build PKG=./pkg/sql/...

./dev test

make test

./dev test pkg/sql/parser -f TestParse

make test PKG=./pkg/sql/parser TESTS=TestParse

./dev test pkg/sql/parser -f TestParse --test-args '-test.count=5 -show-logs'

make test PKG=./pkg/sql/parser TESTS=TestParse TESTFLAGS='-count=5 -show-logs'

./dev bench pkg/sql/parser -f BenchmarkParse

make bench PKG=./pkg/sql/parser BENCHES=BenchmarkParse

./dev build --cross

build/builder.sh mkrelease

./dev builder

build/builder.sh

./dev testlogic base --files=fk --subtests=20042 --config=local

make testbaselogic FILES=fk SUBTESTS=20042 TESTCONFIG=local

Add build --define gotags=bazel,gss,X,Y to .bazelrc.user, then running dev

make ... TAGS=X,Y

Add gc_goopts = ["S"], to the go_library target in the BUILD.bazel file for the package you’re interested in, then running dev

make ... GOFLAGS=-gcflags=-S

General dev tips

The top-level dev script uses Bazel to build pkg/cmd/dev before running unless another dev binary with the same increasing integer ID has already been built. Generally dev will invoke the dev binary “as of” that commit, which should usually be the correct behavior. However, if the default behavior does not work for some reason, you can find all the built versions of dev under bin/dev-versions.

A (hopefully) fast and error proof dev workflow

TODO(a UI developer): fill in gaps for UI dev workflow

1. Switch to a new branch

2. If your workflow involves an IDE, generate your protos ./dev gen protobuf

3. Write some code!

4. Run a test

5. Before opening/updating a PR:

General Bazel tips

If you’re using a different ccache directory (ccache --get-config cache_dir) point to that instead.

dev vs. Bazel

You can always use bazel directly instead of going through dev but there are some things you might want to keep in mind: