Crate Universe

Crate Universe is a set of Bazel rule for generating Rust targets using Cargo.

This doc describes using crate_universe from a WORKSPACE file.

If you're using bzlmod, please see the bzlmod equivalent of this doc.

Setup

After loading rules_rust in your workspace, set the following to begin using crate_universe:

load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencies")

crate_universe_dependencies()

Note that if the current version of rules_rust is not a release artifact, you may need to set additional flags such as bootstrap = True on the crate_universe_dependencies call above or crates_repository::generator_urls in uses of crates_repository.

Rules

• crates_repository
• crates_vendor

Utility Macros

• crate_universe_dependencies
• crate.annotation
• crate.select
• crate.spec
• crate.workspace_member
• render_config
• splicing_config

Workflows

The crates_repository rule (the primary repository rule of rules_rust's cargo support) supports a number of different ways users can express and organize their dependencies. The most common are listed below though there are more to be found in the ./examples/crate_universe directory.

Cargo Workspaces

One of the simpler ways to wire up dependencies would be to first structure your project into a Cargo workspace. The crates_repository rule can ingest a root Cargo.toml file and generate dependencies from there.

load("@rules_rust//crate_universe:defs.bzl", "crates_repository")

crates_repository(
    name = "crate_index",
    cargo_lockfile = "//:Cargo.lock",
    lockfile = "//:Cargo.Bazel.lock",
    manifests = ["//:Cargo.toml"],
)

load("@crate_index//:defs.bzl", "crate_repositories")

crate_repositories()

The generated crates_repository contains helper macros which make collecting dependencies for Bazel targets simpler. Notably, the all_crate_deps and aliases macros (see Dependencies API) commonly allow the Cargo.toml files to be the single source of truth for dependencies. Since these macros come from the generated repository, the dependencies and alias definitions they return will automatically update BUILD targets.

load("@crate_index//:defs.bzl", "aliases", "all_crate_deps")
load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")

rust_library(
    name = "lib",
    aliases = aliases(),
    deps = all_crate_deps(
        normal = True,
    ),
    proc_macro_deps = all_crate_deps(
        proc_macro = True,
    ),
)

rust_test(
    name = "unit_test",
    crate = ":lib",
    aliases = aliases(
        normal_dev = True,
        proc_macro_dev = True,
    ),
    deps = all_crate_deps(
        normal_dev = True,
    ),
    proc_macro_deps = all_crate_deps(
        proc_macro_dev = True,
    ),
)

Direct Packages

In cases where Rust targets have heavy interractions with other Bazel targests (Cc, Proto, etc.), maintaining Cargo.toml files may have deminishing returns as things like rust-analyzer begin to be confused about missing targets or environment variables defined only in Bazel. In workspaces like this, it may be desirable to have a "Cargo free" setup. crates_repository supports this through the packages attribute.

load("@rules_rust//crate_universe:defs.bzl", "crate", "crates_repository", "render_config")

crates_repository(
    name = "crate_index",
    cargo_lockfile = "//:Cargo.lock",
    lockfile = "//:Cargo.Bazel.lock",
    packages = {
        "async-trait": crate.spec(
            version = "0.1.51",
        ),
        "mockall": crate.spec(
            version = "0.10.2",
        ),
        "tokio": crate.spec(
            version = "1.12.0",
        ),
    },
    # Setting the default package name to `""` forces the use of the macros defined in this repository
    # to always use the root package when looking for dependencies or aliases. This should be considered
    # optional as the repository also exposes alises for easy access to all dependencies.
    render_config = render_config(
        default_package_name = ""
    ),
)

load("@crate_index//:defs.bzl", "crate_repositories")

crate_repositories()

Consuming dependencies may be more ergonomic in this case through the aliases defined in the new repository.

load("@rules_rust//rust:defs.bzl", "rust_library", "rust_test")

rust_library(
    name = "lib",
    deps = [
        "@crate_index//:tokio",
    ],
    proc_macro_deps = [
        "@crate_index//:async-trait",
    ],
)

rust_test(
    name = "unit_test",
    crate = ":lib",
    deps = [
        "@crate_index//:mockall",
    ],
)

Binary dependencies

Neither of the above approaches supports depending on binary-only packages.

In order to depend on a Cargo package that contains binaries and no library, you will need to do one of the following:

• Fork the package to add an empty lib.rs, which makes the package visible to Cargo metadata and compatible with the above approaches;

• Or handwrite your own build target for the binary, use http_archive to import its source code, and use crates_repository to make build targets for its dependencies. This is demonstrated below using the rustfilt crate as an example.

# in WORKSPACE.bazel

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rustfilt",
    build_file = "//rustfilt:BUILD.rustfilt.bazel",
    sha256 = "c8d748b182c8f95224336d20dcc5609598af612581ce60cfb29da4dc8d0091f2",
    strip_prefix = "rustfilt-0.2.1",
    type = "tar.gz",
    urls = ["https://static.crates.io/crates/rustfilt/rustfilt-0.2.1.crate"],
)

load("@rules_rust//crate_universe:defs.bzl", "crates_repository")

crates_repository(
    name = "rustfilt_deps",
    cargo_lockfile = "//rustfilt:Cargo.lock",
    manifests = ["@rustfilt//:Cargo.toml"],
)

load("@rustfilt_deps//:defs.bzl", rustfilt_deps = "crate_repositories")

rustfilt_deps()

# in rustfilt/BUILD.rustfilt.bazel

load("@rules_rust//rust:defs.bzl", "rust_binary")

rust_binary(
    name = "rustfilt",
    srcs = glob(["src/**/*.rs"]),
    edition = "2018",
    deps = [
        "@rustfilt_deps//:clap",
        "@rustfilt_deps//:lazy_static",
        "@rustfilt_deps//:regex",
        "@rustfilt_deps//:rustc-demangle",
    ],
)

If you use either crates_repository or crates_vendor to depend on a Cargo package that contains both a library crate and binaries, by default only the library gets made available to Bazel. To generate Bazel targets for the binary crates as well, you must opt in to it with an annotation on the package:

load("@rules_rust//crate_universe:defs.bzl", "crates_repository", "crate")

crates_repository(
    name = "crate_index",
    annotations = {
        "thepackage": [crate.annotation(
            gen_binaries = True,
            # Or, to expose just a subset of the package's binaries by name:
            gen_binaries = ["rustfilt"],
        )],
    },
    # Or, to expose every binary of every package:
    generate_binaries = True,
    ...
)

Dependencies API

After rendering dependencies, convenience macros may also be generated to provide convenient accessors to larger sections of the dependency graph.

• aliases
• crate_deps
• all_crate_deps
• crate_repositories

Building crates with complicated dependencies

Some crates have build.rs scripts which are complicated to run. Typically these build C++ (or other languages), or attempt to find pre-installed libraries on the build machine.

There are a few approaches to making sure these run:

Some things work without intervention

Some build scripts will happily run without any support needed.

rules_rust already supplies a configured C++ toolchain as input to build script execution, and sets variables like CC, CXX, LD, LDFLAGS, etc as needed. Many crates which invoke a compiler with the default environment, or forward these env vars, will Just Work (e.g. if using cc-rs).

rules_rust is open to PRs which make build scripts more likely to work by default with intervention assuming they're broadly applicable (e.g. setting extra widely-known env vars is probably fine, wiring up additional toolchains like cmake that aren't present by default for most Bazel users probably isn't).

Supplying extra tools to build

Some build scripts can be made to work by pulling in some extra files and making them available to the build script.

Commonly this is done by passing the file to the build_script_data annotation for the crate, and using build_script_env to tell the build script where the file is. That env var may often use $(execroot) to get the path to the label, or $${pwd}/ as a prefix if the path given is relative to the execroot (as will frequently happen when using a toolchain).A

There is an example of this in the "complicated dependencies" section of https://github.com/bazelbuild/rules_rust/blob/main/examples/crate_universe/WORKSPACE.bazel which builds libz-ng-sys.

Building with Bazel and supplying via an override

Some build scripts have hooks to allow replacing parts that are complicated to build with output prepared by Bazel.

We can use those hooks by specifying paths (generally using the build_script_data and build_script_env annotations) and pointing them at labels which Bazel will then build. These env vars may often use $(execroot) to get the path to the label, or $${pwd}/ as a prefix if the path given is relative to the execroot (as will frequently happen when using a toolchain).

There is an example of this in the "complicated dependencies" section of https://github.com/bazelbuild/rules_rust/blob/main/examples/crate_universe/WORKSPACE.bazel which builds boring-sys.

crates_vendor

crates_vendor(name, annotations, bazel, buildifier, cargo_bazel, cargo_config, cargo_lockfile,
              generate_binaries, generate_build_scripts, generate_target_compatible_with, manifests,
              mode, packages, render_config, repository_name, splicing_config,
              supported_platform_triples, vendor_path)

A rule for defining Rust dependencies (crates) and writing targets for them to the current workspace. This rule is useful for users whose workspaces are expected to be consumed in other workspaces as the rendered BUILD files reduce the number of workspace dependencies, allowing for easier loads. This rule handles all the same workflows crate_universe rules do.

Example:

Given the following workspace structure:

[workspace]/
    WORKSPACE
    BUILD
    Cargo.toml
    3rdparty/
        BUILD
    src/
        main.rs

The following is something that'd be found in 3rdparty/BUILD:

load("@rules_rust//crate_universe:defs.bzl", "crates_vendor", "crate")

crates_vendor(
    name = "crates_vendor",
    annotations = {
        "rand": [crate.annotation(
            default_features = False,
            features = ["small_rng"],
        )],
    },
    cargo_lockfile = "//:Cargo.Bazel.lock",
    manifests = ["//:Cargo.toml"],
    mode = "remote",
    vendor_path = "crates",
    tags = ["manual"],
)

The above creates a target that can be run to write BUILD files into the 3rdparty directory next to where the target is defined. To run it, simply call:

bazel run //3rdparty:crates_vendor

Repinning / Updating Dependencies

Repinning dependencies is controlled by both the CARGO_BAZEL_REPIN environment variable or the --repin flag to the crates_vendor binary. To update dependencies, simply add the flag ro your bazel run invocation.

bazel run //3rdparty:crates_vendor -- --repin

Under the hood, --repin will trigger a cargo update call against the generated workspace. The following table describes how to control particular values passed to the cargo update command.

ATTRIBUTES

aliases

aliases(normal, normal_dev, proc_macro, proc_macro_dev, build, build_proc_macro, package_name)

Produces a map of Crate alias names to their original label

If no dependency kinds are specified, normal and proc_macro are used by default. Setting any one flag will otherwise determine the contents of the returned dict.

PARAMETERS

RETURNS

dict: The aliases of all associated packages

all_crate_deps

all_crate_deps(normal, normal_dev, proc_macro, proc_macro_dev, build, build_proc_macro,
               package_name)

Finds the fully qualified label of all requested direct crate dependencies for the package where this macro is called.

If no parameters are set, all normal dependencies are returned. Setting any one flag will otherwise impact the contents of the returned list.

PARAMETERS

RETURNS

list: A list of labels to generated rust targets (str)

crate.annotation

crate.annotation(version, additive_build_file, additive_build_file_content, alias_rule,
                 build_script_compile_data, build_script_data, build_script_tools,
                 build_script_data_glob, build_script_deps, build_script_env,
                 build_script_proc_macro_deps, build_script_rundir, build_script_rustc_env,
                 build_script_toolchains, compile_data, compile_data_glob, crate_features, data,
                 data_glob, deps, extra_aliased_targets, gen_binaries, disable_pipelining,
                 gen_build_script, patch_args, patch_tool, patches, proc_macro_deps, rustc_env,
                 rustc_env_files, rustc_flags, shallow_since, override_targets)

A collection of extra attributes and settings for a particular crate

PARAMETERS

RETURNS

string: A json encoded string containing the specified version and separately all other inputs.

crate.select

crate.select(common, selects)

A Starlark Select for crate.annotation().

PARAMETERS

RETURNS

struct: A struct representing the Starlark Select.

crate.spec

crate.spec(package, version, artifact, lib, default_features, features, git, branch, tag, rev)

A constructor for a crate dependency.

See specifying dependencies in the Cargo book for more details.

PARAMETERS

RETURNS

string: A json encoded string of all inputs

crate.workspace_member

crate.workspace_member(version, sha256)

Define information for extra workspace members

PARAMETERS

RETURNS

string: A json encoded string of all inputs

crate_deps

crate_deps(deps, package_name)

Finds the fully qualified label of the requested crates for the package where this macro is called.

PARAMETERS

RETURNS

list: A list of labels to generated rust targets (str)

crate_repositories

crate_repositories()

A macro for defining repositories for all generated crates.

RETURNS

A list of repos visible to the module through the module extension.

crate_universe_dependencies

crate_universe_dependencies(rust_version, bootstrap, kwargs)

Define dependencies of the cargo-bazel Rust target

PARAMETERS

RETURNS

list[struct(repo=str, is_dev_dep=bool)]: A list of the repositories defined by this macro.

render_config

render_config(build_file_template, crate_label_template, crate_repository_template,
              crates_module_template, default_alias_rule, default_package_name,
              generate_target_compatible_with, platforms_template, regen_command, vendor_mode,
              generate_rules_license_metadata)

Various settings used to configure rendered outputs

The template parameters each support a select number of format keys. A description of each key can be found below where the supported keys for each template can be found in the parameter docs

PARAMETERS

RETURNS

string: A json encoded struct to match the Rust config::RenderConfig struct

splicing_config

splicing_config(resolver_version)

Various settings used to configure Cargo manifest splicing behavior.

PARAMETERS

RETURNS

str: A json encoded string of the parameters provided

crates_repository

crates_repository(name, annotations, cargo_config, cargo_lockfile, generate_binaries,
                  generate_build_scripts, generate_target_compatible_with, generator,
                  generator_sha256s, generator_urls, isolated, lockfile, manifests, packages, quiet,
                  render_config, repin_instructions, repo_mapping, rust_toolchain_cargo_template,
                  rust_toolchain_rustc_template, rust_version, splicing_config,
                  supported_platform_triples)

A rule for defining and downloading Rust dependencies (crates). This rule handles all the same workflows crate_universe rules do.

Environment Variables:

Example:

Given the following workspace structure:

[workspace]/
    WORKSPACE.bazel
    BUILD.bazel
    Cargo.toml
    Cargo.Bazel.lock
    src/
        main.rs

The following is something that'd be found in the WORKSPACE file:

load("@rules_rust//crate_universe:defs.bzl", "crates_repository", "crate")

crates_repository(
    name = "crate_index",
    annotations = {
        "rand": [crate.annotation(
            default_features = False,
            features = ["small_rng"],
        )],
    },
    cargo_lockfile = "//:Cargo.Bazel.lock",
    lockfile = "//:cargo-bazel-lock.json",
    manifests = ["//:Cargo.toml"],
    # Should match the version represented by the currently registered `rust_toolchain`.
    rust_version = "1.60.0",
)

The above will create an external repository which contains aliases and macros for accessing Rust targets found in the dependency graph defined by the given manifests.

NOTE: The cargo_lockfile and lockfile must be manually created. The rule unfortunately does not yet create it on its own. When initially setting up this rule, an empty file should be created and then populated by repinning dependencies.

Repinning / Updating Dependencies

Dependency syncing and updating is done in the repository rule which means it's done during the analysis phase of builds. As mentioned in the environments variable table above, the CARGO_BAZEL_REPIN (or REPIN) environment variables can be used to force the rule to update dependencies and potentially render a new lockfile. Given an instance of this repository rule named crate_index, the easiest way to repin dependencies is to run:

CARGO_BAZEL_REPIN=1 bazel sync --only=crate_index

This will result in all dependencies being updated for a project. The CARGO_BAZEL_REPIN environment variable can also be used to customize how dependencies are updated. The following table shows translations from environment variable values to the equivilant cargo update command that is called behind the scenes to update dependencies.

If the crates_repository is used multiple times in the same Bazel workspace (e.g. for multiple independent Rust workspaces), it may additionally be useful to use the CARGO_BAZEL_REPIN_ONLY environment variable, which limits execution of the repinning to one or multiple instances of the crates_repository rule via a comma-delimited allowlist:

CARGO_BAZEL_REPIN=1 CARGO_BAZEL_REPIN_ONLY=crate_index bazel sync --only=crate_index

ATTRIBUTES

ENVIRONMENT VARIABLES

This repository rule depends on the following environment variables:

• CARGO_BAZEL_GENERATOR_URL
• CARGO_BAZEL_GENERATOR_SHA256
• CARGO_BAZEL_REPIN
• REPIN
• CARGO_BAZEL_REPIN_ONLY
• CARGO_BAZEL_ISOLATED
• CARGO_BAZEL_DEBUG