Add a formatter
How to add a new formatter to the fmt and lint goals.
In Pants, every formatter is also a linter, meaning that if you can run a tool with pants fmt, you can run the same tool in check-only mode with pants lint. Start by skimming Add a linter to familiarize yourself with how linters work.
This guide assumes that you are running a formatter that already exists outside of Pants as a stand-alone binary, such as running Black or Prettier.
If you are instead writing your own formatting logic inline, you can skip Step 1. In Step 4, you will not need to use Process.
1. Install your formatter
There are several ways for Pants to install your formatter. See Installing tools. This example will use ExternalTool because there is already a pre-compiled binary for shfmt.
You will also likely want to register some options, like --config, --skip, and --args. Options are registered through a Subsystem. If you are using ExternalTool, this is already a subclass of Subsystem. Otherwise, create a subclass of Subsystem. Then, set the class property options_scope to the name of the tool, e.g. "shfmt" or "prettier". Finally, add options from pants.option.option_types.
from pants.core.util_rules.external_tool import ExternalTool
from pants.engine.platform import Platform
from pants.option.option_types import ArgsListOption, SkipOption
class Shfmt(ExternalTool):
"""An autoformatter for shell scripts (https://github.com/mvdan/sh)."""
options_scope = "shfmt"
name = "Shfmt"
default_version = "v3.2.4"
default_known_versions = [
"v3.2.4|macos_arm64 |e70fc42e69debe3e400347d4f918630cdf4bf2537277d672bbc43490387508ec|2998546",
"v3.2.4|macos_x86_64|43a0461a1b54070ddc04fbbf1b78f7861ee39a65a61f5466d15a39c4aba4f917|2980208",
"v3.2.4|linux_arm64 |6474d9cc08a1c9fe2ef4be7a004951998e3067d46cf55a011ddd5ff7bfab3de6|2752512",
"v3.2.4|linux_x86_64|3f5a47f8fec27fae3e06d611559a2063f5d27e4b9501171dde9959b8c60a3538|2797568",
]
# We set this because we need the mapping for both `generate_exe` and `generate_url`.
platform_mapping = {
"macos_arm64": "darwin_arm64",
"macos_x86_64": "darwin_amd64",
"linux_arm64": "linux_arm64",
"linux_x86_64": "linux_amd64",
}
skip = SkipOption("fmt", "lint")
args = ArgsListOption(example="-i 2")
def generate_url(self, plat: Platform) -> str:
plat_str = self.platform_mapping[plat.value]
return (
f"https://github.com/mvdan/sh/releases/download/{self.version}/"
f"shfmt_{self.version}_{plat_str}"
)
def generate_exe(self, plat: Platform) -> str:
plat_str = self.platform_mapping[plat.value]
return f"./shfmt_{self.version}_{plat_str}"
2. Set up a FieldSet and FmtTargetsRequest
As described in Rules and the Target API, a FieldSet is a way to tell Pants which Fields you care about targets having for your plugin to work.
Usually, you should add a subclass of SourcesField to the class property required_fields, such as ShellSourceField or PythonSourceField. This means that your linter will run on any target with that sources field or a subclass of it.
Create a new dataclass that subclasses FieldSet:
from dataclasses import dataclass
from pants.engine.target import FieldSet
...
@dataclass(frozen=True)
class ShfmtFieldSet(FieldSet):
required_fields = (ShellSourceField,)
sources: ShellSourceField
Then, hook this up to a new subclass of FmtRequest.
from pants.core.goals.fmt import FmtRequest
class ShfmtRequest(FmtTargetsRequest):
field_set_type = ShfmtFieldSet
tool_subsystem = Shfmt
3. Create fmt rules
You will need a rule for fmt which takes the FmtTargetsRequest.Batch from step 3 (e.g. ShfmtRequest) as a parameter and returns a FmtResult.
@rule(desc="Format with shfmt", level=LogLevel.DEBUG)
async def shfmt_fmt(request: ShfmtRequest.Batch, shfmt: Shfmt, platform: Platform) -> FmtResult:
download_shfmt_get = Get(
DownloadedExternalTool,
ExternalToolRequest,
shfmt.get_request(platform),
)
# If the user specified `--shfmt-config`, we must search for the file they specified with
# `PathGlobs` to include it in the `input_digest`. We error if the file cannot be found.
config_digest_get = Get(
Digest,
PathGlobs(
globs=[shfmt.config] if shfmt.config else [],
glob_match_error_behavior=GlobMatchErrorBehavior.error,
description_of_origin="the option `--shfmt-config`",
),
)
downloaded_shfmt, config_digest = await MultiGet(
download_shfmt_get, config_digest_get
)
input_digest = await Get(
Digest,
MergeDigests(
(request.snapshot.digest, downloaded_shfmt.digest, config_digest)
),
)
argv = [
downloaded_shfmt.exe,
"-w",
*shfmt.args,
*request.snapshot.files,
]
process = Process(
argv=argv,
input_digest=input_digest,
output_files=request.snapshot.files,
description=f"Run shfmt on {pluralize(len(request.snapshot.files), 'file')}.",
level=LogLevel.DEBUG,
)
result = await Get(ProcessResult, Process, process)
return await FmtResult.create(request, result, output_snapshot)
The FmtRequest.Batch has .snapshot, which stores the list of files and the Digest for each source file.
If you used ExternalTool in step 1, you will use Get(DownloadedExternalTool, ExternalToolRequest) to ensure that the tool is fetched.
Use Get(Digest, MergeDigests) to combine the different inputs together, such as merging the source files and downloaded tool.
At the bottom of your file, tell Pants about your rules:
def rules():
return [
*collect_rules(),
*ShfmtRequest.rules(partitioner_type=PartitionerType.DEFAULT_SINGLE_PARTITION),
]
Finally, update your plugin's register.py to activate this file's rules. Note that we must register the rules added in Step 2, as well.
from shell import shfmt
def rules():
return [*shfmt.rules()]
Now, when you run pants fmt :: or pants lint ::, your new formatter should run.
4. Add tests (optional)
Refer to Testing rules.