Troubleshooting / common issues
Frequently asked questions (FAQs) and known issues you may encounter.
See Getting Help. We would love to help!
If you are confused by something, likely someone else will run into the same issue. It is helpful for us to know what is going wrong so that we can improve Pants and improve this documentation.
Debug tip: enable stack traces and increase logging
Pants defaults to not displaying the full stack trace when it encounters an error. Pants also defaults to logging at the info level.
When you encounter an exception, it can help to use the global options --print-stacktrace and -ldebug, like this:
./pants --print-stacktrace -ldebug <rest of your command>
Setting the option --pex-verbosity=9 can help debug exceptions that occur when building .pex files.
Once you have this stack trace, we recommend copying it into Pastebin or a GitHub Gist, then opening a GitHub issue or posting on Slack. Someone from the Pants team would be happy to help. See Getting Help.
Debug tip: inspect the sandbox with --no-process-cleanup
Pants runs most processes in a hermetic sandbox (temporary directory), which allows for safely caching and running multiple processes in parallel.
Use the option --no-process-cleanup for Pants to log the paths to these sandboxes, and to keep them around after the run. You can then inspect them to check if the files you are expecting are present.
./pants --no-process-cleanup lint src/project/app.py
...
21:26:13.55 [INFO] preserving local process execution dir `"/private/var/folders/hm/qjjq4w3n0fsb07kp5bxbn8rw0000gn/T/process-executionQgIOjb"` for "Run isort on 1 file."
...
There is even a __run.sh script in the directory that will run the process using the same argv and environment that Pants would use.
Cache or pantsd invalidation issues
If you are using the latest stable version of Pants and still experience a cache invalidation issue: we are sorry for the trouble. We have not yet added a comprehensive goal to "clear all caches", because we are very interested in coming up with coherent solutions to potential issues (see #11167 for more information). If you experience a cache issue, please absolutely file a bug before proceeding to the following steps.
To start with, first try using --no-pantsd. If that doesn't work, you can also try --no-local-cache.
If --no-pantsd worked, you can restart pantsd, either by:
- Killing the
pantsdprocess associated with your workspace. You can useps aux | grep pantsto find the PID, thekill -9 <pid>. - Deleting the
${workspace}/.pidsdirectory.
If this resolves the issue, please report that on the ticket and attach the recent content of the .pants.d/pantsd/pantsd.log file.
If restarting pantsd is not sufficient, you can also use --no-local-cache to ignore the persistent caches. If this resolves the issue, then it is possible that the contents of the cache (at ~/.cache/pants) will be useful for debugging the ticket that you filed: please try to preserve the cache contents until it can be resolved.
Pants cannot find a file in your project
Pants may complain that it cannot find a file or directory, even though the file does indeed exist.
This error generally happens because of the option pants_ignore in the [GLOBAL] scope, but you should also check for case-mismatches in filenames ("3rdparty" vs "3rdParty"). By default, Pants will read your top-level .gitignore file to populate pants_ignore, along with ignoring dist/ and any top-level files/directories starting with ..
To override something included in your .gitignore, add a new value to pants_ignore and prefix it with !, like the below. pants_ignore uses the same syntax as gitignore.
[GLOBAL]
pants_ignore.add = ["!folder/"]
Alternatively, you can stop populating pants_ignore from your .gitignore by setting pants_ignore_use_gitignore = false in the [GLOBAL] scope.
Import errors and missing dependencies
Because Pants runs processes in hermetic sandboxes (temporary directories), Pants must properly know about your dependencies to avoid import errors.
Usually, you do not need to tell Pants about your dependencies thanks to dependency inference, but sometimes dependency inference is not set up properly or cannot work. Dependency inference can fail for several reasons, which may require that you instead explicitly set the dependencies field:
- Pants does not know about some of your project files because you do not have targets and BUILD files describing them.
- Tip: run
./pants tailor, which will generate targets for files without owners. Note that this will only teach Pants about your first-party code, but not third-party requirements, which need to be manually added.
- Tip: run
- Source roots are not set up properly.
- Pants needs to know your source roots so that it can convert source files into their language's representation, e.g. the file
src/py/project/app.pyto the Python moduleproject.app.
- Pants needs to know your source roots so that it can convert source files into their language's representation, e.g. the file
- >1 target exports the same module, which means that Pants cannot safely infer which you want to use.
- For third-party requirements, this usually happens when you have multiple targets referring to the same requirement. Was this intentional? Sometimes it can be useful to do this, such as having two conflicting versions of the same requirement. Otherwise, you can fix this by consolidating to only one target for the requirement or using explicit dependencies to disambiguate.
- For first-party code, this usually happens when you have multiple targets with the same file in their
sourcesfield. Use./pants list path/to/file.extto see all the "owners" of a particular file. - Pants will warn when this happens.
- Third-party requirements might expose different modules than Pants thinks by default.
- For example, see Third-party dependencies for how to use
module_mappingwith Python requirements.
- For example, see Third-party dependencies for how to use
- Some targets can never be inferred and must always be explicitly added via the
dependenciesfield, e.g.file/filesandresource/resourcestargets. - For Python third-party requirements, check if they have any undeclared dependencies.
When debugging, run ./pants dependencies path/to/file.ext and ./pants dependencies --transitive to see what Pants thinks a particular file depends on. If the dependency you want is not showing up, try temporarily adding the dependency explicitly to see if it fixes the issue; if it does, you can then try to figure out why dependency inference was not working, or simply use the explicit dependency.
If ./pants dependencies is behaving how you'd expect, but certain imports of first-party code are still missing, try running with --no-process-cleanup and inspect the logged sandbox to see if the files you're expecting are present. If files are missing, you may need to adjust the sources field of some of your dependencies.
"Out of space" error: set an alternative tmpdir
It may be necessary to explicitly set the directory Pants uses as a temporary directory. For example, if the system default temporary directory is a small partition, you may exhaust that temp space.
Use the global option local_execution_root_dir to change the tmpdir used by Pants.
[GLOBAL]
local_execution_root_dir = "/mnt/large-partition/tmpdir"
"No space left on device" error while watching files
On Linux, Pants uses inotify to watch all files and directories related to any particular build. Some systems have limits configured for the maximum number of files watched. To adjust the limit on file watches, you can run:
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p
How to change your cache directory
You may change any of these options in the [GLOBAL] section of your pants.toml:
| Option | What it does | Default |
|---|---|---|
local_store_dir | Stores the results of running subprocesses and of some file operations. | ~/.cache/pants/lmdb_store |
named_caches_dir | Stores the caches for certain tools used by Pants, like PEX's cache for resolving Python requirements. | ~/.cache/pants/named_caches |
pants_workdir | Stores some project-specific logs; used as a temporary directory when running ./pants repl and ./pants run.This is not used for caching. This must be relative to the build root. | <build_root>/.pants.d/ |
pants_distdir | Where Pants writes artifacts to, such as the result of ./pants package.This is not used for caching; you can delete this folder and still leverage the cache from local_store_dir.This must be relative to the build root. | <build_root>/dist/ |
For local_store_dir and named_caches_dir, you may either specify an absolute path or a relative path, which will be relative to the build root. You may use the special string %(homedir)s to get the value of ~, e.g. local_store_dir = "%(homedir)s/.custom_cache/pants/lmdb_store".
It is safe to delete these folders to free up space.
You can also change the cache used by the ./pants script described in Installing Pants, which defaults to ~/.pants/cache/setup. Either set the environment variable PANTS_SETUP_CACHE or change the Bash script directly where it defines PANTS_SETUP_CACHE. You may use an absolute path or a path relative to the build root.
"Double requirement given" error when resolving Python requirements
This is an error from pip, and it means that the same 3rd-party Python requirement—with different version constraints—appears in your dependencies.
You can use ./pants peek to help identify why the same requirement is being used more than once:
# Check the `requirements` key to see if it has the problematic requirement.
./pants filter --target-type=python_requirement | xargs ./pants peek
macOS users: issues with system Python interpreters
The macOS system Python interpreters are broken in several ways, such as sometimes resulting in:
ERROR: Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/Library/Python/3.7'
You can set the option interpreter_search_paths in the [python] scope to teach Pants to ignore the interpreters in /usr/bin. See here for more information.
"Too many open files" error
You may encounter this error when running Pants:
./pants count-loc helloworld/greet/f.py
ERROR: Could not initialize store for process cache: "Error making env for store at \"/Users/pantsbuild/.cache/pants/lmdb_store/processes/2\": Too many open files"
(Use --print-exception-stacktrace to see more error details.)
This sometimes happens because Pants uses lots of file handles to read and write to its cache at ~/.cache/pants/lmdb_store; often, this is more than your system's default.
This can be fixed by setting ulimit -n 10000. (10,000 should work in all cases, but feel free to lower or increase this number as desired.)
ulimit -nWe recommend permanently setting this by either:
- Adding
ulimit -n 10000to your./pantsscript. - Using a tool like Direnv to run
ulimit -n 10000everytime the project is loaded. - Adding
ulimit -n 10000to your global.bashrcor equivalent.
The first two approaches have the benefit that they will be checked into version control, so every developer at your organization can use the same setting.
ulimit unlimitedContrary to the name, this will not fix the issue. You must use ulimit -n instead.