Options
A deep dive into how options may be configured.
Option scopes
Options are partitioned into named scopes.
Some system-wide options belong in the global scope. For example, the --level option, which controls the logging level, is in the global scope.
Other options belong to a subsystem scope. A subsystem is simply a collection of related options, in a scope. For example, the pytest subsystem contains options related to Python's test framework pytest.
Setting options
Every option can be set in the following ways, in order of precedence:
- Via a command line flag.
- In an environment variable.
- In a config file (
pants.toml).
If an option isn't set in one of these ways, it will take on a default value.
You can inspect both the current value and the default value by using pants help $scope or pants help-advanced $scope, e.g. pants help global.
Command-line flags
Global options are set using an unqualified flag:
pants --level=debug ...
Subsystem options are set by providing the flag, with the name prefixed with the lower-case scope name and a dash. So for the option --root-patterns in the scope source:
pants --source-root-patterns="['^ext']"
Environment variables
Global options are set using the environment variable PANTS_{OPTION_NAME}:
PANTS_LEVEL=debug pants ...
Subsystem options are set using the environment variable
PANTS_{SCOPE}_{OPTION_NAME}:
PANTS_SOURCE_ROOT_PATTERNS="['^ext']" pants ...
Note that the scope and option name are upper-cased, and any dashes in the option flag name are converted to underscores: --multiword-name becomes MULTIWORD_NAME.
Config file entries
Global options are set in the GLOBAL section of the config file:
[GLOBAL]
level = "debug"
Subsystem options are set in the section named for their scope:
[source]
root_patterns = ["/src/python"]
Note that any dashes in the option flag name are converted to underscores: --multiword-name becomes multiword_name.
Config file interpolation
A string value in a config file can contain placeholders of the form %(key)s, which will be replaced with a corresponding value. The key can be one of:
- A string-valued option in the DEFAULT section of the same config file.
- A string-valued option in the same section of the config file as the value containing the placeholder.
- Any environment variable, prefixed with
env.:%(env.ENV_VAR)s. - The following special values:
%(buildroot)s: absolute path to the root of your repository.%(homedir)s: equivalent to$HOMEor~.%(user)s: the current user's username, obtained from the system password file.%(pants_workdir)s: the absolute path of the global option--pants-workdir, which defaults to{buildroot}/.pants.d/.%(pants_distdir)s: the absolute path of the global option--pants-distdir, which defaults to{buildroot}/dist/.
An interpolated value may itself contain placeholders, that will be recursively interpolated.
For example:
[DEFAULT]
domain = "my.domain"
[python-repos]
repo_host = "repo.%(domain)s"
indexes.add = ["https://%(env.PY_REPO)s@%(repo_host)s/index"]
Learn more about exporting environment variables in the .pants.bootstrap)
Bash script that is sourced before Pants runs.
Option types
Every option has a type, and any values you set must be of that type.
The option types are:
- string
- integer
- bool
- list
- dict
A list-valued option may also declare a specific type for its members (e.g., a list of strings, or a list of integers).
String and integer values
Standalone string and integer values are written without quotes. Any quotes will be considered part of the value, after shell escaping.
Command-line flags:
pants --scope-intopt=42
pants --scope-stropt=qux
Environment variables:
PANTS_SCOPE_INTOPT=42
PANTS_SCOPE_STROPT=qux
Config file entries:
[scope]
intopt = 42
stropt = "qux"
Boolean values
Boolean values can be specified using the special strings true and false. When specifying them via command-line flags you can also use the --boolopt/--no-boolopt syntax.
Command-line flags:
pants --scope-boolopt=true
pants --scope-boolopt
pants --no-scope-boolopt
Environment variables:
PANTS_SCOPE_BOOLOPT=true
Config file entries:
[scope]
boolopt = true
List values
List values are parsed as Python list literals, so you must quote string values, and you may need to apply shell-level quoting and/or escaping, as required.
Command-line flags:
pants --scope-listopt="['foo','bar']"
You can also leave off the [] to append elements. So we can rewrite the above to:
pants --scope-listopt=foo --scope-listopt=bar
Appending will add to any values from lower-precedence sources, such as config files (pants.toml) and possibly Pants's default. Otherwise, using [] will override any lower-precedence sources.
Environment variables:
PANTS_SCOPE_LISTOPT="['foo','bar']"
Like with command-line flags, you can leave off the [] to append elements:
PANTS_SCOPE_LISTOPT=foo
Config file entries:
[scope]
listopt = [
'foo',
'bar'
]
Add/remove semantics
List values have some extra semantics:
- A value can be preceded by
+, which will append the elements to the value obtained from lower-precedence sources. - A value can be preceded by
-, which will remove the elements from the value obtained from lower-precedence sources. - Multiple
+and-values can be provided, separated by commas. - Otherwise, the value replaces the one obtained from lower-precedence sources.
For example, if the value of --listopt in scope is set to [1, 2] in a config file, then
pants --scope-listopt="+[3,4]"
will set the value to [1, 2, 3, 4].
pants --scope-listopt="-[1],+[3,4]"
will set the value to [2, 3, 4], and
pants --scope-listopt="[3,4]"
will set the value to [3, 4].
The +/- syntax works in .toml files, but the entire value must be quoted:
[scope]
listopt = "+[1,2],-[3,4]"
This means that TOML treats the value as a string, instead of a TOML list.
Alternatively, you can use this syntactic sugar, which allows the values to be regular TOML lists:
[scope]
listopt.add = [1, 2]
listopt.remove = [3, 4]
But note that this only works in Pants's .toml config files, not in environment variables or command-line flags.
Dict values
Dict values are parsed as Python dict literals on the command-line and environment variables, so you must quote string keys and values, and you may need to apply shell-level quoting and/or escaping, as required.
Command-line flags:
pants --scope-dictopt="{'foo':1,'bar':2}"
Environment variables:
PANTS_SCOPE_DICTOPT="{'foo':1,'bar':2}"
Config file entries:
You can use TOML's nested table features. These are equivalent:
[scope]
dictopt = { foo = 1, bar = 2}
[scope.dictopt]
foo = 1
bar = 2
You can also use a string literal. Note the quotes:
[scope]
dictopt = """{
'foo': 1,
'bar': 2,
}"""
Add/replace semantics
- A value can be preceded by
+, which will update the value obtained from lower-precedence sources with the entries. - Otherwise, the value replaces the one obtained from lower-precedence sources.
For example, if the value of --dictopt in scope is set to {'foo', 1, 'bar': 2} in a config file, then
pants --scope-dictopt="+{'foo':42,'baz':3}"
will set the value to {'foo': 42, 'bar': 2, 'baz': 3}, and
pants --scope-dictopt="{'foo':42,'baz':3}"
will set the value to {'foo': 42, 'baz': 3}.
Reading individual option values from files
If an option value is too large or elaborate to use directly, or if you don't want to hard-code
values directly in pants.toml, you can set the value of any option to the string
@relative/path/from/repo/root/to/file (note the leading @), and the value will be read
from that file.
If the file name ends with .json or .yaml then the file will be parsed as the relevant
format, which is useful for list- and dict-valued options.
Otherwise, the file is parsed as a literal as described above for each option type.
Normally, the file must exist, and it is an error if it doesn't.
To avoid an error when the file doesn't exist, add a ? after @, for example:
@?path/that/may/not/exist - this will treat the config value in question as not
being set when the file does not exist.
Note that you can use this feature on the command-line, in an env var, or in a config file:
[scope]
opt = "@path/to/file.json"
PANTS_SCOPE_OPTION=@path/to/file.json
pants --scope-option="@path/to/file.json"
Until we resolve this issue, changing
the value in a file used with the @ syntax as described above will not invalidate the build.
For now, if such a file changes you will have to restart the Pants daemon manually. You can
do so by killing it (after using ps -ef | grep pantsd to find its pid), or by running
Pants once with --no-pantsd.
.pants.rc file
You can set up personal Pants config files, using the same TOML syntax as pants.toml. By default, Pants looks for the paths /etc/pantsrc, ~/.pants.rc, and .pants.rc in the repository root.
For example:
[python]
# Even though our repository uses 3.8+, because I have an M1,
# I must use Python 3.9+.
interpreter_constraints = ["==3.9.*"]
If you want to ban this feature, set [GLOBAL].pantsrc = false in pants.toml.
.pants.bootstrap file
If you need to set default values for environment variables without requiring all users to define them in the local
environment, you can export them in the .pants.bootstrap Bash script. This file needs to be placed in the root of your
workspace, and it is going to be sourced before invoking any Pants goal.
You can also add to this file any Bash code you want to execute before Pants runs, and any environment variables declared in this file are going to be available to any process that Pants launcher binary may start.
For example:
# these variables are defined in our CI agents,
# but this is set to support local development
export DOCKER_DEFAULT_REPO="https://hub.docker.com/"
export GIT_COMMIT="$(git rev-parse HEAD)"
If you want to learn more about how Pants launcher binary works, see the scie-pants project.