Autoscaling self-hosted Github Actions Runners in K8s with AWS Karpenter

A brief explanation of how AWS Autoscaling of EC2 EKS nodes usually work in AWS EKS Cluster:

• cluster-autoscaler installed
• permissions given to the cluster-autoscaler to modify launch templates

Today we will be understanding how we can use the brand new AWS Karpenter instead.

What is AWS Karpenter?

Karpenter is an open-source, flexible, high-performance Kubernetes cluster autoscaler built with AWS, however it works with any other cloud providers as well.

The difference here is:

• cluster-autoscaler is cloud agnostic, any Cloud Providers can work with it to autoscale accordingly.
• cluster-autoscaler uses launch template APIs to manage the autoscaling groups in AWS to scale then nodes.
• Karpenter uses AWS APIs directly to create and delete EC2 instances, assigning them as AWS EKS Nodes as required. Update: Karpenter now supports other cloud providers as well

Understanding the Github Actions Self-hosted runners arch:

Actions Runner Controller

GitHub - actions/actions-runner-controller: Kubernetes controller for GitHub Actions self-hosted…

Recently it was announced that this project is adopted by GitHub officially and they will be making a public beta on the next version’s implementation soon.

A Helm chart is also available. Note the version differences between the Application version and the Chart version, they correspond to one another but the Application version can be configured via the chart

Currently I am working at SPH Media Trust Singapore and we have open sourced our implementation via a Terraform module.

This module deploys the helm chart using Terraform’s helm_release resource and manage the CRD resources in kubernetes_manifest resources

NodeTemplate

Provisioner

Actions Controller Helm Chart

• CRD resource creation
• Controller deployment
• Webhook for autoscaling Github Actions Runners

Horizontal Runner Autoscaler

RunnerDeployment

However, there is a new version that is currently being developed as the project was officially adopted by GitHub.

Stay tuned as I update this space to use the completely rewrote version of actions controller.