typecheck

How to use MyPy.

---

Activating MyPy

To opt-in, add pants.backend.python.typecheck.mypy to backend_packages in your config file.

pants.toml

[GLOBAL]
backend_packages.add = [
  "pants.backend.python",
  "pants.backend.python.typecheck.mypy",
]

This will register a new typecheck goal:

$ ./pants typecheck helloworld/util/lang.py
$ ./pants typecheck ::

Benefit of Pants: typecheck Python 2-only and Python 3-only code at the same time

MyPy determines which Python version to use based on its python_version option. If that's undefined, MyPy uses the interpreter the tool is run with. Because you can only use one config file at a time with MyPy, you cannot normally say to use 2.7 for part of your codebase but 3.6 for the rest; you must choose a single version.

Instead, Pants will group your targets based on their interpreter constraints, and run all the Python 2 targets together and all the Python 3 targets together. It will automatically set python_version to the minimum compatible interpreter, such as a constraint like ["==2.7.*", ">3.6"] using 2.7.

To turn this off, you can still set python_version in mypy.ini or --python-version/--py2 in --mypy-args; Pants will respect the value you set.

Hook up a MyPy config file

Set the config option in the [mypy] scope:

pants.toml

[mypy]
config = "build-support/mypy.ini"

Change the MyPy version

Use the version option in the [mypy] scope:

pants.toml

[mypy]
version = "mypy==0.782"

Type stubs (.pyi files)

You can use .pyi files for both first-party and third-party code. Include the .pyi files in the sources field for python_library and python_tests targets. MyPy will use these stubs rather than looking at the implementation.

Pants's dependency inference knows to infer a dependency both on the implementation and the type stub. You can verify this by running ./pants dependencies path/to/file.py.

When writing stubs for third-party libraries, you may need the set up the [source].root_patterns option so that source roots are properly stripped. For example:

pants.toml

[source]
root_patterns = ["mypy-stubs", "src/python"]

mypy-stubs/colors.pyi

# Because we set `mypy-stubs` as a source root, this file will be
# stripped to be simply `colors.pyi`. MyPy will look at this file for
# imports of the `colors` module.

def red(s: str) -> str: ...

mypy-stubs/BUILD

python_library()

src/python/project/app.py

from colors import red

if __name__ == "__main__":
    print(red("I'm red!"))

src/python/project/BUILD

# Pants will infer a dependency both on the `ansicolors` requirement
# and our type stub.
pex_binary(sources=["app.py"])

Add a third-party plugin

Add the plugin to the extra_requirements option in the [mypy] scope, then update your mypy.ini to load the plugin:

pants.toml

[mypy]
extra_requirements = ["pydantic==1.6.1"]

mypy.ini

[mypy]
plugins =
    pydantic.mypy

For some plugins, like django-stubs, you may need to always load certain source files, such as a settings.py file. You can make sure that this source file is always used by hijacking the source_plugins option, which allows you to specify targets whose sources should always be used when running MyPy. See the below section for more information about source plugins.

For example, to fully use the django-stubs plugin, your setup might look like this:

pants.toml

[source]
root_patterns = ["src/python"]

[mypy]
extra_requirements = ["django-stubs==1.5.0"]
source_plugins = ["src/python/project:django_settings"]

mypy.ini

[mypy]
plugins =
    mypy_django_plugin.main

[mypy.plugins.django-stubs]
django_settings_module = project.django_settings

src/python/project/django_settings.py

from django.urls import URLPattern

DEBUG = True
DEFAULT_FROM_EMAIL = "[email protected]"
SECRET_KEY = "not so secret"
MY_SETTING = URLPattern(pattern="foo", callback=lambda: None)

src/python/project/BUILD

mypy_source_plugin(
    name="django-settings",
    sources=["django_settings.py"],
)

MyPy Protobuf support

Add mypy_plugin = true to the [python-protobuf] scope. See Protobuf for more information.

Add a first-party plugin

To add a MyPy plugin you wrote, define a mypy_source_plugin target with the plugin's Python file(s) included in the sources field.

Then, add plugins = path.to.module to your MyPy config file, using the name of the module without source roots. For example, if your Python file is called pants-plugins/mypy_plugins/custom_plugin.py, and you set pants-plugins as a source root, then set plugins = mypy_plugins.custom_plugin. Set the config option in the [mypy] scope in your pants.toml to point to your MyPy config file.

Finally, set the option source_plugins in the [mypy] scope to include this target's address, e.g. source_plugins = ["pants-plugins/mypy_plugins:plugin"]. This will ensure that your plugin's sources are always included in the subprocess.

For example:

pants.toml

[mypy]
config = "mypy.ini"
source_plugins = ["pants-plugins/mypy_plugins:plugin"]

mypy.ini

plugins =
    mypy_plugins.change_return_type

pants-plugins/mypy_plugins/BUILD

mypy_source_plugin(
    name="plugin",
    sources=["change_return_type.py"],
)

pants-plugins/mypy_plugins/change_return_type.py

"""A contrived plugin that changes the return type of any
function ending in `__overriden_by_plugin` to return None."""

from typing import Callable, Optional, Type

from mypy.plugin import FunctionContext, Plugin
from mypy.types import NoneType, Type as MyPyType

from plugins.subdir.dep import is_overridable_function

class ChangeReturnTypePlugin(Plugin):
    def get_function_hook(
        self, fullname: str
    ) -> Optional[Callable[[FunctionContext], MyPyType]]:
        return hook if name.endswith("__overridden_by_plugin") else None


def hook(ctx: FunctionContext) -> MyPyType:
    return NoneType()


def plugin(_version: str) -> Type[Plugin]:
    return ChangeReturnTypePlugin

A mypy_source_plugin target is treated similarly to a python_library target. For example, Python linters and formatters will run on the target.

You can depend on other targets and Pants's dependency inference will add them to the dependencies field, including any third-party requirements and python_library targets (even if their source files live in a different directory).

Other targets can depend on a mypy_source_plugin target. This allows you to write a python_tests target for this code or a python_distribution target to distribute the plugin externally.

Known limitations

Performance may be slower than normal

Pants does not yet leverage MyPy's caching mechanism and daemon, so a typical run with Pants will likely be slower than using MyPy directly.

We are working to figure out how to leverage MyPy's cache in a way that is safe and allows for things like remote execution.

Namespace packages

Pants's MyPy implementation will likely not work if you use namespace packages, either via PEP 420 or the older pkg_resources approach, even if you use MyPy's --namespace-packages option.

Further, we've internally had issues with setting --namespace-packages causing MyPy to complain about some third-party dependencies, which appears to be a MyPy bug.

Please reach out on Slack if you need support for namespace packages. We had difficulty getting MyPy to work with namespace packages independently of Pants, and it would help us to understand your use case.

Tip: only run over changed files and their dependees

When changing type hints code, you not only need to run over the changed files, but also any code that depends on the changed files:

$ ./pants --changed-since=HEAD --changed-dependees=transitive typecheck

See Advanced target selection for more information.