Skip to main content

Pants 2.10 adds Apache Thrift support for Python

· 3 min read
Eric Arellano
Eric Arellano
Pants Maintainer

Pants's codegen support solves one of the biggest problems with code generation: how to make sure that local developers, CI, and production are using the same generated code?

The Pants build system now supports generating Python code from Apache Thrift, in addition to already supporting Protobuf and gRPC.

Pants's codegen support solves one of the biggest problems with code generation: how to make sure that local developers, CI, and production are using the same generated code?

Organizations often adopt subpar solutions like writing scripts that must be invoked before building, along with checking in the generated code to version control.

Instead, Pants automates generating exactly what is needed for the current task. Rather than writing files to your repository directly—where they can be changed unintentionally—Pants runs processes like tests in hermetic sandboxes, guaranteeing you're using the most up-to-date generated files.

(If you do want to save your generated files to disk, you can optionally run ./pants export-codegen ::)

Precise generation, without boilerplate​

Pants understands which files to generate through dependency inference, which automatically maps your import statements to the rest of your code.

For example, if you have this Thrift file:

include "models/name.thrift"
namespace py codegen.models.user

Pants will understand that this Python import requires generating that Thrift file:

from codegen.models.user.ttypes import User
$ ./pants dependencies project/app.py
thrift/models/name.thrift

Dependency inference understands the unique semantics of Thrift, like what happens if you leave off namespace py. It also understands dependencies amongst Thrift files via include directives.

This fine-grained dependency information is what unlocks fine-grained caching of builds, like running tests and packaging Python binaries with Docker. If none of the inputs have changed, Pants can safely use the cache.

As discussed in our post on Pants's performance, this inference is 1) very safe and 2) very fast. Because Pants invokes processes hermetically with a sandbox, failing to infer a dependency can never cause the wrong thing to be cached. Further, the inference is fast thanks to Pants's core being implemented in Rust, along with a daemon, parallelism, and very fine-grained invalidation.

Trying out Pants​

In addition to generating Thrift and Protobuf/gRPC, Pants automates and improves the rest of your builds, including:

  • Run linters and formatters with a single consistent interface, parallelizing them all when safe. Including: Black, isort, Yapf, Flake8, Pylint, Shellcheck, Shfmt.
  • Execute tests with hermeticity, safe parallelization, and fine-grained caching.
  • Package your code into PEX binaries, PyOxidizer binaries, AWS Lambdas, Google Cloud Functions, and Python wheels/sdists.
  • Provide insight via project introspection, such as querying which files depend on which.

Check out the example-codegen repository to try out Pants 2.10's Thrift support. And let us know what you think in Slack and GitHub!

To use Pants with your project, check out our Getting started guide and our Thrift docs.