Skip to main content
Version: 2.18 (deprecated)

pyenv-python-provider

A subsystem for Pants-provided Python leveraging pyenv (https://github.com/pyenv/pyenv).

Enabling this subsystem will switch Pants from trying to find an appropriate Python on your system to using pyenv to install the correct Python(s).

The Pythons provided by Pyenv will be used to run any "user" code (your Python code as well as any Python-based tools you use, like black or pylint). The Pythons are also read-only to ensure they remain hermetic across runs of different tools and code.

The Pythons themselves are stored in your named_caches_dir: https://www.pantsbuild.org/docs/reference-global#named_caches_dir under pyenv/versions/<version>. Wiping the relevant version directory (with sudo rm -rf) will force a re-install of Python. This may be necessary after changing something about the underlying system which changes the compiled Python, such as installing an optional-at-build-time dependency like liblzma-dev (which is used for the optional module lzma).

By default, the subsystem does not pass any optimization flags to the Python compilation process. Doing so would increase the time it takes to install a single Python by about an order of magnitude (E.g. ~2.5 minutes to ~26 minutes).

Backend: pants.backend.python.providers.experimental.pyenv

Config section: [pyenv-python-provider]

Basic options​

installation_extra_env_vars​

--pyenv-python-provider-installation-extra-env-vars="['<str>', '<str>', ...]"
PANTS_PYENV_PYTHON_PROVIDER_INSTALLATION_EXTRA_ENV_VARS
pants.toml
[pyenv-python-provider]
installation_extra_env_vars = [
    '<str>',
    '<str>',
    ...,
]
default: []

Additional environment variables to include when running pyenv install.

Entries are strings in the form ENV_VAR=value to use explicitly; or just ENV_VAR to copy the value of a variable in Pants's own environment.

This is especially useful if you want to use an optimized Python (E.g. setting PYTHON_CONFIGURE_OPTS='--enable-optimizations --with-lto' and PYTHON_CFLAGS='-march=native -mtune=native') or need custom compiler flags.

Note that changes to this option result in a different fingerprint for the installed Python, and therefore will cause a full re-install if changed.

See https://github.com/pyenv/pyenv/blob/master/plugins/python-build/README.md#special-environment-variables for supported env vars.

Can be overriden by fieldpyenv_python_provider_installation_extra_env_vars on local_environment, docker_environment, or remote_environmenttargets.

Advanced options​

known_versions​

--pyenv-python-provider-known-versions="['<str>', '<str>', ...]"
PANTS_PYENV_PYTHON_PROVIDER_KNOWN_VERSIONS
pants.toml
[pyenv-python-provider]
known_versions = [
    '<str>',
    '<str>',
    ...,
]
default: 
[
  "2.3.13|macos_arm64|9105de5e5cf8dc0eca2a520ed04493d183128d46a2cfb402d4cc271af1bf144b|749323",
  "2.3.13|macos_x86_64|9105de5e5cf8dc0eca2a520ed04493d183128d46a2cfb402d4cc271af1bf144b|749323",
  "2.3.13|linux_x86_64|9105de5e5cf8dc0eca2a520ed04493d183128d46a2cfb402d4cc271af1bf144b|749323",
  "2.3.13|linux_arm64|9105de5e5cf8dc0eca2a520ed04493d183128d46a2cfb402d4cc271af1bf144b|749323"
]

Known versions to verify downloads against.

Each element is a pipe-separated string of version|platform|sha256|length or version|platform|sha256|length|url_override, where:

  • version is the version string
  • platform is one of [linux_arm64,linux_x86_64,macos_arm64,macos_x86_64]
  • sha256 is the 64-character hex representation of the expected sha256 digest of the download file, as emitted by shasum -a 256
  • length is the expected length of the download file in bytes, as emitted by wc -c
  • (Optional) url_override is a specific url to use instead of the normally generated url for this version

E.g., 3.1.2|macos_x86_64|6d0f18cd84b918c7b3edd0203e75569e0c7caecb1367bbbe409b44e28514f5be|42813. and 3.1.2|macos_arm64 |aca5c1da0192e2fd46b7b55ab290a92c5f07309e7b0ebf4e45ba95731ae98291|50926|https://example.mac.org/bin/v3.1.2/mac-aarch64-v3.1.2.tgz.

Values are space-stripped, so pipes can be indented for readability if necessary.

url_platform_mapping​

--pyenv-python-provider-url-platform-mapping="{'key1': val1, 'key2': val2, ...}"
PANTS_PYENV_PYTHON_PROVIDER_URL_PLATFORM_MAPPING
pants.toml
[pyenv-python-provider.url_platform_mapping]
key1 = val1
key2 = val2
...
default: {}

A dictionary mapping platforms to strings to be used when generating the URL to download the tool.

In --url-template, anytime the {platform} string is used, Pants will determine the current platform, and substitute {platform} with the respective value from your dictionary.

For example, if you define {"macos_x86_64": "apple-darwin", "linux_x86_64": "unknown-linux"}, and run Pants on Linux with an intel architecture, then {platform} will be substituted in the --url-template option with unknown-linux.

url_template​

--pyenv-python-provider-url-template=<str>
PANTS_PYENV_PYTHON_PROVIDER_URL_TEMPLATE
pants.toml
[pyenv-python-provider]
url_template = <str>
default: https://github.com/pyenv/pyenv/archive/refs/tags/v{version}.tar.gz

URL to download the tool, either as a single binary file or a compressed file (e.g. zip file). You can change this to point to your own hosted file, e.g. to work with proxies or for access via the filesystem through a file:$abspath URL (e.g. file:/this/is/absolute, possibly by templating the buildroot in a config file).

Use {version} to have the value from --version substituted, and {platform} to have a value from --url-platform-mapping substituted in, depending on the current platform. For example, https://github.com/.../protoc-&#123;version&#125;-&#123;platform&#125;.zip.

use_unsupported_version​

--pyenv-python-provider-use-unsupported-version=<UnsupportedVersionUsage>
PANTS_PYENV_PYTHON_PROVIDER_USE_UNSUPPORTED_VERSION
pants.toml
[pyenv-python-provider]
use_unsupported_version = <UnsupportedVersionUsage>
one of: error, warning
default: error

What action to take in case the requested version of pyenv is not supported.

Supported pyenv versions: unspecified

version​

--pyenv-python-provider-version=<str>
PANTS_PYENV_PYTHON_PROVIDER_VERSION
pants.toml
[pyenv-python-provider]
version = <str>
default: 2.3.13

Use this version of pyenv.

Deprecated options​

None

None