Espresso AI started as a primarily Linux shop, but as we've added more developers on OS X we've needed a way to build docker containers on OS X that are compatible with our Linux production environment.

Pants has supported running portions of the build in docker containers for a while, but the exact configuration is a little tricky, particularly if you want to build x86_64 containers on Apple ARM processors.

The recipe itself is very straight forward, adding these two sections to your root BUILD and pants.toml files will build your python binaries and docker containers for Linux/x64 on both Linux & OSX:

BUILD

__defaults__({
  pex_binary: dict(environment="linux"),
  docker_image: dict(build_platform=["linux/amd64"]),
})

local_environment(
  name="local_linux",
  compatible_platforms=["linux_x86_64"],
  fallback_environment="docker",
  docker_env_vars=["AWS_PROFILE=profile"]
)

local_environment(
  name="local_osx",
  compatible_platforms=["macos_arm64"],
)

docker_environment(
  name="docker",
  platform="linux_x86_64",
  image="python:3.11-bookworm",
  docker_env_vars=["AWS_PROFILE=profile"]
)

pants.toml

[environments-preview.names]
local = "//:local_linux"
local_osx = "//:local_osx"
docker = "//:docker"

With these snippets, we have defined 3 environments:

• A docker environment with associated image for running the linux builds
• A local linux environment that falls back to docker
• An OSX environment for running non-linux builds & other goals locally on osx

We've setup a few defaults for our repo to avoid unnecessary configuration in our BUILD files:

• All of our pex_binary targets are built using either the local or docker linux environments via the fallback mechanism
• All of our docker_image targets have the build_platform explicitly specified so that we do not inherit the platform defaults to build arm64 images.

The other targets, such as tests, get built/run/etc using one of the maching local_workspace targets.

We've also worked around a non-obvious requirement to duplicate our [docker.env_vars] setting in the docker_env_vars field of the docker_environment target. If your publish of a docker container is failing, this is likely the cause.

Troubleshooting Tips

If you are tempted to apply the build_platform argument on your individual docker_image targets, or attempt to add this to your Dockerfile, note that you need to specify this on the entire chain of docker images you depend on if you have multi-stage builds.

If you have attempted your own version of this build_platform setup before using this recipe, you may run into some caching issues and may benefit from using the --no-local-cache flag once to get builds working.

If you notice you cannot publish images with this setup, run the publish command with the PANTS_LEVEL=debug environment variable to see what environment variables are actually making it to the docker push command.

Understanding common errors

If you are receiving errors like this, it indicates you are attempting to run a linux/x64 binary on a linux/arm64 container image and need to adjust your docker image by updating the build platform as shown in the example above:

Failed to find compatible interpreter on path /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.

Examined the following interpreters:
1.) /usr/bin/python3.10 CPython==3.10.12
2.) /usr/bin/python3.11 CPython==3.11.9

No interpreter compatible with the requested constraints was found:

  A distribution for pendulum could not be resolved for /usr/bin/python3.11.
  Found 1 distribution for pendulum that do not apply:
  1.) The wheel tags for pendulum 2.1.2 are cp311-cp311-manylinux_2_35_x86_64 which do not match the supported tags of /usr/bin/python3.11:
  cp311-cp311-manylinux_2_35_aarch64