Interpreter compatibility
How to configure which Python version(s) your project should use.
Setting the default Python version
Pants v2.23.x is the last Pants release series which will be regularly tested with Python 2.7. Python 2.7 is no longer supported by its maintainers as of 1 January 2020. While Pants may continue to work with Python 2.7 in the near term, as of Pants v2.24.x, Pants will no longer officially support use of Python 2.7, and, consequently, will no longer be proactively tested with Python 2.7 in CI. Contributions to fix issues with Python 2.7 support will continue to be accepted, but will depend on any community contributions and will not consitute continued official support for Python 2.7.
Configure your default Python interpreter compatibility constraints in pants.toml
like this:
[python]
interpreter_constraints = ["CPython==3.11.*"]
The value can be any valid Requirement-style strings. You can use multiple strings to OR constraints, and use commas within each string to AND constraints. For example:
Constraint | What it means |
---|---|
['CPython>=3.6,<4'] | CPython 3.6+, but not CPython 4 or later |
['CPython==3.7.3'] | CPython 3.7.3 |
['PyPy'] | any version of PyPy |
['CPython==2.7.*', 'CPython>=3.5'] | CPython 2.7 or 3.5+ |
As a shortcut, you can leave off CPython
and just put the version specifier. For example, ==3.8
will be expanded automatically to CPython==3.8
.
If you use Python code on Apple's M1/M2 hardware you may need to set your interpreter constraints to Python 3.9+, as many tools, such as Black, will not install correctly on earlier Python versions on this platform.
Using multiple Python versions in the same project
Pants also allows you to specify the interpreter compatibility for particular targets. This allows you to use multiple Python versions in the same repository, such as allowing you to incrementally migrate from Python 2 to Python 3.
Use the interpreter_constraints
field on a Python target, like this:
python_sources(
name="python2_target",
interpreter_constraints=["==2.7.*"],
)
If interpreter_constraints
is left off, the target will default to the value from the option interpreter_constraints
in [python]
.
To only change the interpreter constraints for a few files, you can use the overrides
field:
python_sources(
name="lib",
overrides={
"py2.py": {"interpreter_constraints": ["==2.7.*"]},
# You can use a tuple for multiple files:
("common.py", "f.py"): {"interpreter_constraints": ["==2.7.*"]},
}
)
A particular target's interpreter constraints must be the same or a subset of all of its transitive dependencies. For example, a target marked Python 3-only cannot depend on a Python 2-only target, which Pants will validate.
It is possible for a target to have more precise interpreter constraints than its transitive
dependencies. For example, if you have a common helper file utils.py
that works with both
Python 2.7 and 3.5 (['==2.7.*', '>=3.5']
), it is legal for a dependent like app.py
to use more
precise constraints like ['==3.7.*']
. It is often useful for helper code to work with multiple
Python versions, while specific "root" targets (tests, apps, and binaries) have more specific
constraints.
Pants accepts your interpreter constraints at face value. If you use a constraint like '>=3.6'
, Pants will trust you that your code indeed works with any interpreter >= 3.6, as Pants has no way to audit if your code is actually compatible.
Instead, consider running your unit tests with every Python version you claim to support to ensure that your code really is compatible:
python_test(
source="util_test.py",
interpreter_constraints=parametrize(py2=["==2.7.*"], py3=["==3.6.*"]),
)
When you parametrize
constraints, Pants will generate multiple targets, one for each constraint, allowing you to test multiple python versions with a single command. This is somewhat similar to "matrix testing". See Target Generation for more details.
Tip: activate pants.backend.python.mixed_interpreter_constraints
We recommend adding pants.backend.python.mixed_interpreter_constraints
to backend_packages
in the [GLOBAL]
scope, which will add the new goal py-constraints
.
[GLOBAL]
backend_packages = [
"pants.backend.python",
"pants.backend.python.mixed_interpreter_constraints",
]
You can run pants py-constraints $file/$target
to see what final interpreter constraints will be used, and why. For example:
$ pants py-constraints helloworld/main.py
Final merged constraints: CPython==2.7.*,>=3.5 OR CPython>=3.5
CPython>=3.5
helloworld/main.py
CPython==2.7.* OR CPython>=3.5
helloworld/util/__init__.py
helloworld/util/config_loader.py
helloworld/util/lang.py
helloworld/util/proto/__init__.py:init
helloworld/util/proto/config.proto
py-constraints --summary
You can run pants py-constraints --summary
for Pants to generate a CSV giving an overview of your project's interpreter constraints:
Result of pants py-constraints --summary
, then importing the CSV into Google
Sheets.
We recommend then importing this CSV into a tool like Pandas or Excel to filter/sort the data.
The # Dependents
column is useful to see how impactful it is to port a file, and the # Dependencies
can be useful to see how easy it would be to port.
While every project will have different needs and scope, there are a few best practices with Pants that will allow for a more successful migration.
- Start by setting the
interpreter_constraints
option in[python]
to describe the status of the majority of your targets. If most are only compatible with Python 2, set it to['==2.7.*']
. If most are compatible with Python 2 and Python 3, set to['==2.7', '>=3.5']
. If most are only compatible with Python 3, set to[>=3.5]
. For any targets that don't match these global constraints, override with theinterpreter_constraints
field. - Run
pants py-constraints --summary
and sort by# Dependents
from Z to A to find your most-used files. Focus on getting these targets to be compatible with Python 2 and 3. You may want to also sub-sort the CSV by# Dependencies
to find what is easiest to port. - Once >40% of your targets work with both Python 2 and Python 3, change the
interpreter_constraints
option in[python]
to specify compatibility with both Python 2.7 and Python 3 so that all new code uses this by default. - For files with no or few dependencies, change them to Python 3-only when possible so that you can start using all the neat new Python 3 features like f-strings! Use the CSV from
pants py-constraints --summary
to find these. You can also do this if every dependent target works exclusively with Python 3, which you can find by theTransitive Constraints
column and by runningpants py-constraints path/to/file.py
.
Check out this blog post on Pants' own migration to Python 3 in 2019 for more general tips on Python 3 migrations.
Changing the interpreter search path
Pants will default to looking at your $PATH
to discover Python interpreters. You can change this by setting the option search_paths
in the [python-bootstrap]
scope.
You can specify absolute paths to interpreter binaries and/or to directories containing interpreter binaries. In addition, Pants understands some special symbols:
<PATH>
: read the$PATH
env var<PYENV>
: use all directories in$(pyenv root)/versions
<PYENV_LOCAL>
: the interpreter specified in the local file.python-version
<ASDF>
, all Python versions currently configured by ASDF, with a fallback to all installed versions.<ASDF_LOCAL>
, the ASDF interpreter with the version in<BUILD_ROOT>/.tool-versions
.
For example:
[python-bootstrap]
search_path = ["<PYENV>", "/opt/python3"]