Offline builds with Bazel

Unlike the make-driven build process which uses the checked-in sources in the vendor submodule, the Bazel build will download sources as needed. Under normal circumstances this should be transparent to the user, but if you are in an environment with no network access you’ll have to do some planning ahead of time to make sure you have all the dependencies needed to build. There are two general approaches here: an ad-hoc method and building a distdir.

A note about Non-Bazel tools

Bazel fetches dependencies from a bunch of tarballs/zip files over https, and it stores them in a directory called the “repository cache” (bazel info repository_cache). This is different from how the standard go toolchain handles dependencies; in particular, it has a different module cache that is not compatible with Bazel’s. Use go mod download to pre-fetch Go dependencies for use by non-Bazel tools, like your IDE or code editor.

Ad-hoc bootstrapping for offline builds

You can fetch the dependencies needed for any Bazel target with bazel fetch. This conditionally fetches all the dependencies needed for a given target. For example, you can pre-fetch all the dependencies needed to build cockroach plus all of the unit tests with the following:

bazel fetch pkg/cmd/cockroach pkg:all_tests

Note that this will not fetch dependencies for binaries besides cockroach or for any test in-tree that is not a unit test. If you need the dependencies for another binary or test, include those in the bazel fetch call as well. For example:

bazel fetch pkg/acceptance:all  # acceptance tests
bazel fetch build/bazelutil:lint  # lints
bazel fetch pkg/sql/logictest:logictest_test  # base logic tests

bazel fetch pkg/... is a heavyweight option but it does work.

It is not recommended to use bazel sync. This unconditionally fetches all dependencies that might be necessary for building any target in the project, including 10+ toolchains for cross-compiling for many targets. This all represents many GB of data you almost definitely don’t need. bazel sync is also broken on macOS machines due to the macOS filesystem being case-insensitive.

Building a distdir

Bazel has built-in support for offline builds with the --distdir flag. In this context a “distdir” is a directory containing pre-downloaded versions of any file that Bazel would otherwise have to download for the build. You can build a distdir for your current checkout of cockroach as follows:

bazel build @distdir//:archives
mkdir -p /path/to/distdir
tar -xzf _bazel/bin/external/distdir/archives.tar --strip-components=1 -C /path/to/distdir

The first command will construct a tarball _bazel/bin/external/distdir/archives.tar containing all dependencies for the cockroach you have checked out. The last extracts all of these dependencies into /path/to/distdir.

Then, you can build with the Bazel flag --distdir=/path/to/distdir. Alternatively, to have the distdir consulted by default for all Bazel actions, you can add build --distdir=/path/to/distdir to your .bazelrc.user.

Caveats

The built distdir does NOT contain everything that would be fetched by bazel sync. In particular, the cross toolchains are not included in the distdir, as they are very large and not generally useful. If you need to perform a cross-build using one of these toolchains offline, you can download the .tar.gz for the cross toolchain(s) you need and place it in the distdir manually.
The built distdir will likely contain a lot of content that you do not need to perform your build. For example, the distdir contains the Go SDK for all supported platforms, a version of NodeJS for each supported platform, and pre-built versions of the c-deps for all supported platforms. You can feel free to delete any files you know you won’t need from your local distdir to free up space.
As you might expect, the distdir you build via this method will contain only the dependencies for the version of cockroach you have checked out. If you regularly need to build several different versions of cockroach offline, you cannot build a distdir once and expect it to contain all of the dependencies needed to build any arbitrary version of cockroach. You can build a distdir for each version of cockroach you need and swap between them, or you can “merge” all of your distdirs together by extracting all of the tarballs into the same directory. To do the latter, do something like this: