Skip to main content
Version: 2.24 (dev)

GitHub Actions aarch64 runners

GitHub Actions does not have affordable hosted aarch64 runners.

For a while we enjoyed hardware donated by the Works on ARM program, but that has now been terminated, so we instead rely on self-hosted EC2 instances.

We use Runs On to simplify the management of these instances. Runs On monitors requests for runners, launches EC2 instances on the fly to satify those requests, and terminates them when the workflows complete.

The Custom AMI​

We configure Runs On to launch these instances with a custom AMI that pre-installs the Python interpreters needed to bootstrap and test Pants on aarch64. This custom AMI is built on top of the standard Runs On Ubuntu 22 ARM64 AMI.

It may occasionally be necessary to update this AMI, for example to pick up updates to the underlying standard AMI. To do so manually:

Create a temporary EC2 instance on the standard AMI​

In the AWS web console, initiate instance creation, and when prompted for an AMI, search for the latest standard AMI, as described in the Runs On images repo. In practice this means using runs-on-v2.2-ubuntu22-full-arm64 as the search term, and checking the results in the Community AMIs tab.

After selecting the AMI, open the Advanced section and set the instance's User Data to:

#!/bin/bash
systemctl start ssh

to ensure that its SSH daemon runs on startup. Then choose "pantsbuild.org.bastion" as the instance's SSH keypair.

SSH into the temporary EC2 instance​

Once the instance is running, click on "Connect" to get SSH instructions. You will need the pantsbuild.org.bastion private key, which is in our 1Password account.

Install Pythons on the instance​

Once you are in a bash prompt on the instance, run the following:

sudo apt install -y software-properties-common
sudo add-apt-repository -y ppa:deadsnakes/ppa
sudo apt update
sudo apt install -y \
  python3.7 python3.7-dev python3.7-venv \
  python3.8 python3.8-dev python3.8-venv \
  python3.9 python3.9-dev python3.9-venv

to install the necessary Pythons.

Create an AMI from the instance​

From the instance's actions menu in the web console select "Images" and then "Create image". The image name doesn't strictly matter, but ubuntu22-full-arm64-python3.7-python3.8-python3.9 is a good name for consistency (multiple AMIs may have the same name).

We hope to automate the above via Packer at some point.

Update the Runs On config​

Edit .github/runs-on.yml and update the ami field. Note that the logical image name in this file happens to be the same as the AMI name, but doesn't strictly have to be. This is the name that we target in the runs-on stanza of a CI job in generate_github_workflows.py.

Note that the new AMI will only be used in PR CI jobs after the config change is merged into the target branch, so the CI job for the update itself will still use the old AMI.

Deregister the old AMI​

After the config update has been merged into main and any release branches it needs to be in, we can deregister the old AMI in the AWS web console, to avoid confusion.