Component: actions-runner-controller

This component creates a Helm release for actions-runner-controller on an EKS cluster.

Usage

Stack Level: Regional

Once the catalog file is created, the file can be imported as follows.

import:
  - catalog/eks/actions-runner-controller
  ...

The default catalog values e.g. stacks/catalog/eks/actions-runner-controller.yaml

components:
  terraform:
    eks/actions-runner-controller:
      settings:
        spacelift:
          workspace_enabled: true
      vars:
        enabled: true
        name: "actions-runner" # avoids hitting name length limit on IAM role
        chart: "actions-runner-controller"
        chart_repository: "https://actions-runner-controller.github.io/actions-runner-controller"
        chart_version: "0.21.0"
        kubernetes_namespace: "actions-runner-system"
        create_namespace: true
        resources:
          limits:
            cpu: 100m
            memory: 128Mi
          requests:
            cpu: 100m
            memory: 128Mi
        ssm_github_token_path: "/github_runners/controller_github_app_secret"
        ssm_github_webhook_secret_token_path: "/github_runners/controller_github_app_secret"
        github_app_id: "123456"
        github_app_installation_id: "234567890"
        webhook:
          enabled: true
          # gha-webhook.use1.auto.core.acme.net
          hostname_template: "gha-webhook.%[3]v.%[2]v.%[1]v.acme.net"
        timeout: 120
        runners:
          infrastructure-runner:
            type: "repository" # can be either 'organization' or 'repository'
            dind_enabled: false # If `true`, a Docker sidecar container will be deployed
            # To run Docker in Docker (dind), change image from summerwind/actions-runner to summerwind/actions-runner-dind
            image: summerwind/actions-runner
            scope: "acme/infrastructure"
            scale_down_delay_seconds: 300
            min_replicas: 1
            max_replicas: 5
            resources:
              limits:
                cpu: 200m
                memory: 256Mi
              requests:
                cpu: 100m
                memory: 128Mi
            webhook_driven_scaling_enabled: true
            pull_driven_scaling_enabled: false
            labels:
              - "Ubuntu"
              - "self-hosted"

Generating Required Secrets

AWS SSM is used to store and retrieve secrets.

Decide on the SSM path for the GitHub secret (PAT or Application private key) and GitHub webhook secret.

Since the secret is automatically scoped by AWS to the account and region where the secret is stored, we recommend the secret be stored at /github_runners/controller_github_app_secret unless you plan on running multiple instances of the controller. If you plan on running multiple instances of the controller, and want to give them different access (otherwise they could share the same secret), then you can add a path component to the SSM path. For example /github_runners/cicd/controller_github_app_secret.

ssm_github_secret_path: "/github_runners/controller_github_app_secret"

The preferred way to authenticate is by creating and installing a GitHub App. This is the recommended approach as it allows for more much more restricted access than using a personal access token, at least until fine-grained personal access token permissions are generally available. Follow the instructions here to create and install the GitHub App.

At the creation stage, you will be asked to generate a private key. This is the private key that will be used to authenticate the Action Runner Controller. Download the file and store the contents in SSM using the following command, adjusting the profile and file name. The profile should be the admin role in the account to which you are deploying the runner controller. The file name should be the name of the private key file you downloaded.

AWS_PROFILE=acme-mgmt-use2-auto-admin chamber write github_runners controller_github_app_secret -- "$(cat APP_NAME.DATE.private-key.pem)"

You can verify the file was correctly written to SSM by matching the private key fingerprint reported by GitHub with:

AWS_PROFILE=acme-mgmt-use2-auto-admin chamber read -q github_runners controller_github_app_secret | openssl rsa -in - -pubout -outform DER | openssl sha256 -binary | openssl base64

At this stage, record the Application ID and the private key fingerprint in your secrets manager (e.g. 1Password). You will need the Application ID to configure the runner controller, and want the fingerprint to verify the private key.

Proceed to install the GitHub App in the organization or repository you want to use the runner controller for, and record the Installation ID (the final numeric part of the URL, as explained in the instructions linked above) in your secrets manager. You will need the Installation ID to configure the runner controller.

In your stack configuration, set the following variables, making sure to quote the values so they are treated as strings, not numbers.

github_app_id: "12345"
github_app_installation_id: "12345"

OR (obsolete)

• A PAT with the scope outlined in this document. Save this to the value specified by ssm_github_token_path using the following command, adjusting the AWS_PROFILE to refer to the admin role in the account to which you are deploying the runner controller:

AWS_PROFILE=acme-mgmt-use2-auto-admin chamber write github_runners controller_github_app_secret -- "<PAT>"

2. If using the Webhook Driven autoscaling (recommended), generate a random string to use as the Secret when creating the webhook in GitHub.

Generate the string using 1Password (no special characters, length 45) or by running

dd if=/dev/random bs=1 count=33  2>/dev/null | base64

Store this key in AWS SSM under the same path specified by ssm_github_webhook_secret_token_path

ssm_github_webhook_secret_token_path: "/github_runners/github_webhook_secret"

Using Webhook Driven Autoscaling

To use the Webhook Driven autoscaling, you must also install the GitHub organization-level webhook after deploying the component (specifically, the webhook server). The URL for the webhook is determined by the webhook.hostname_template and where it is deployed. Recommended URL is https://gha-webhook.[environment].[stage].[tenant].[service-discovery-domain].

As a GitHub organization admin, go to https://github.com/organizations/[organization]/settings/hooks, and then:

• Click"Add webhook" and create a new webhook with the following settings:
  • Payload URL: copy from Terraform output webhook_payload_url
  • Content type: application/json
  • Secret: whatever you configured in the sops secret above
  • Which events would you like to trigger this webhook:
    • Select "Let me select individual events"
    • Uncheck everything ("Pushes" is likely the only thing already selected)
    • Check "Workflow jobs"
  • Ensure that "Active" is checked (should be checked by default)
  • Click "Add webhook" at the bottom of the settings page

After the webhook is created, select "edit" for the webhook and go to the "Recent Deliveries" tab and verify that there is a delivery (of a "ping" event) with a green check mark. If not, verify all the settings and consult the logs of the actions-runner-controller-github-webhook-server pod.

Updating CRDs

When updating the chart or application version of actions-runner-controller, it is possible you will need to install new CRDs. Such a requirement should be indicated in the actions-runner-controller release notes and may require some adjustment to our custom chart or configuration.

This component uses helm to manage the deployment, and helm will not auto-update CRDs. If new CRDs are needed, install them manually via a command like

kubectl create -f https://raw.githubusercontent.com/actions-runner-controller/actions-runner-controller/master/charts/actions-runner-controller/crds/actions.summerwind.dev_horizontalrunnerautoscalers.yaml

Useful Reference

Consult actions-runner-controller documentation for further details.

Requirements

Name	Version
terraform	>= 1.3.0
aws	>= 4.9.0
helm	>= 2.0
kubernetes	>= 2.0

Providers

Name	Version
aws	>= 4.9.0

Modules

Name	Source	Version
actions_runner	cloudposse/helm-release/aws	0.7.0
actions_runner_controller	cloudposse/helm-release/aws	0.7.0
eks	cloudposse/stack-config/yaml//modules/remote-state	1.4.1
iam_roles	../../account-map/modules/iam-roles	n/a
this	cloudposse/label/null	0.25.0

Resources

Name	Type
aws_eks_cluster_auth.eks	data source
aws_ssm_parameter.github_token	data source
aws_ssm_parameter.github_webhook_secret_token	data source

Inputs

Name	Description	Type	Default	Required
additional_tag_map	Additional key-value pairs to add to each map in tags_as_list_of_maps. Not added to tags or id. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration.	map(string)	{}	no
atomic	If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used.	bool	true	no
attributes	ID element. Additional attributes (e.g. workers or cluster) to add to id, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the delimiter and treated as a single ID element.	list(string)	[]	no
chart	Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if repository is specified. It is also possible to use the <repository>/<chart> format here if you are running Terraform on a system that the repository has been added to with helm repo add but this is not recommended.	string	n/a	yes
chart_description	Set release description attribute (visible in the history).	string	null	no
chart_repository	Repository URL where to locate the requested chart.	string	n/a	yes
chart_values	Additional values to yamlencode as helm_release values.	any	{}	no
chart_version	Specify the exact chart version to install. If this is not specified, the latest version is installed.	string	null	no
cleanup_on_fail	Allow deletion of new resources created in this upgrade when upgrade fails.	bool	true	no
context	Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as null to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged.	any	{ "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null }	no
create_namespace	Create the namespace if it does not yet exist. Defaults to false.	bool	null	no
delimiter	Delimiter to be used between ID elements. Defaults to - (hyphen). Set to "" to use no delimiter at all.	string	null	no
descriptor_formats	Describe additional descriptors to be output in the descriptors output map. Map of maps. Keys are names of descriptors. Values are maps of the form {<br/> format = string<br/> labels = list(string)<br/>} (Type is any so the map values can later be enhanced to provide additional options.) format is a Terraform format string to be passed to the format() function. labels is a list of labels, in order, to pass to format() function. Label values will be normalized before being passed to format() so they will be identical to how they appear in id. Default is {} (descriptors output will be empty).	any	{}	no
eks_component_name	The name of the eks component	string	"eks/cluster"	no
enabled	Set to false to prevent the module from creating any resources	bool	null	no
environment	ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT'	string	null	no
existing_kubernetes_secret_name	If you are going to create the Kubernetes Secret the runner-controller will use by some means (such as SOPS) outside of this component, set the name of the secret here and it will be used. In this case, this component will not create a secret and you can leave the secret-related inputs with their default (empty) values. The same secret will be used by both the runner-controller and the webhook-server.	string	""	no
github_app_id	The ID of the GitHub App to use for the runner controller.	string	""	no
github_app_installation_id	The "Installation ID" of the GitHub App to use for the runner controller.	string	""	no
helm_manifest_experiment_enabled	Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan	bool	true	no
id_length_limit	Limit id to this many characters (minimum 6). Set to 0 for unlimited length. Set to null for keep the existing setting, which defaults to 0. Does not affect id_full.	number	null	no
import_profile_name	AWS Profile name to use when importing a resource	string	null	no
import_role_arn	IAM Role ARN to use when importing a resource	string	null	no
kube_data_auth_enabled	If true, use an aws_eks_cluster_auth data source to authenticate to the EKS cluster. Disabled by kubeconfig_file_enabled or kube_exec_auth_enabled.	bool	false	no
kube_exec_auth_aws_profile	The AWS config profile for aws eks get-token to use	string	""	no
kube_exec_auth_aws_profile_enabled	If true, pass kube_exec_auth_aws_profile as the profile to aws eks get-token	bool	false	no
kube_exec_auth_enabled	If true, use the Kubernetes provider exec feature to execute aws eks get-token to authenticate to the EKS cluster. Disabled by kubeconfig_file_enabled, overrides kube_data_auth_enabled.	bool	true	no
kube_exec_auth_role_arn	The role ARN for aws eks get-token to use	string	""	no
kube_exec_auth_role_arn_enabled	If true, pass kube_exec_auth_role_arn as the role ARN to aws eks get-token	bool	true	no
kubeconfig_context	Context to choose from the Kubernetes kube config file	string	""	no
kubeconfig_exec_auth_api_version	The Kubernetes API version of the credentials returned by the exec auth plugin	string	"client.authentication.k8s.io/v1beta1"	no
kubeconfig_file	The Kubernetes provider config_path setting to use when kubeconfig_file_enabled is true	string	""	no
kubeconfig_file_enabled	If true, configure the Kubernetes provider with kubeconfig_file and use that kubeconfig file for authenticating to the EKS cluster	bool	false	no
kubernetes_namespace	The namespace to install the release into.	string	n/a	yes
label_key_case	Controls the letter case of the tags keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the tags input. Possible values: lower, title, upper. Default value: title.	string	null	no
label_order	The order in which the labels (ID elements) appear in the id. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present.	list(string)	null	no
label_value_case	Controls the letter case of ID elements (labels) as included in id, set as tag values, and output by this module individually. Does not affect values of tags passed in via the tags input. Possible values: lower, title, upper and none (no transformation). Set this to title and set delimiter to "" to yield Pascal Case IDs. Default value: lower.	string	null	no
labels_as_tags	Set of labels (ID elements) to include as tags in the tags output. Default is to include all labels. Tags with empty values will not be included in the tags output. Set to [] to suppress all generated tags. Notes: The value of the name tag, if included, will be the id, not the name. Unlike other null-label inputs, the initial setting of labels_as_tags cannot be changed in later chained modules. Attempts to change it will be silently ignored.	set(string)	[ "default" ]	no
name	ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a tag. The "name" tag is set to the full id string. There is no tag with the value of the name input.	string	null	no
namespace	ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique	string	null	no
rbac_enabled	Service Account for pods.	bool	true	no
regex_replace_chars	Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, "/[^a-zA-Z0-9-]/" is used to remove all characters other than hyphens, letters and digits.	string	null	no
region	AWS Region.	string	n/a	yes
resources	The cpu and memory of the deployment's limits and requests.	object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) })	n/a	yes
runners	Map of Action Runner configurations, with the key being the name of the runner. Please note that the name must be in kebab-case. For example: hcl organization_runner = { type = "organization" # can be either 'organization' or 'repository' dind_enabled: false # A Docker sidecar container will be deployed image: summerwind/actions-runner # If dind_enabled=true, set this to 'summerwind/actions-runner-dind' scope = "ACME" # org name for Organization runners, repo name for Repository runners scale_down_delay_seconds = 300 min_replicas = 1 max_replicas = 5 busy_metrics = { scale_up_threshold = 0.75 scale_down_threshold = 0.25 scale_up_factor = 2 scale_down_factor = 0.5 } labels = [ "Ubuntu", "core-automation", ] }	map(object({ type = string scope = string image = optional(string, "") dind_enabled = bool scale_down_delay_seconds = number min_replicas = number max_replicas = number busy_metrics = optional(object({ scale_up_threshold = string scale_down_threshold = string scale_up_adjustment = optional(string) scale_down_adjustment = optional(string) scale_up_factor = optional(string) scale_down_factor = optional(string) })) webhook_driven_scaling_enabled = bool webhook_startup_timeout = optional(string, null) pull_driven_scaling_enabled = bool labels = list(string) storage = optional(string, null) pvc_enabled = optional(string, false) resources = object({ limits = object({ cpu = string memory = string ephemeral_storage = optional(string, null) }) requests = object({ cpu = string memory = string }) }) }))	n/a	yes
s3_bucket_arns	List of ARNs of S3 Buckets to which the runners will have read-write access to.	list(string)	[]	no
ssm_github_secret_path	The path in SSM to the GitHub app private key file contents or GitHub PAT token.	string	""	no
ssm_github_webhook_secret_token_path	The path in SSM to the GitHub Webhook Secret token.	string	""	no
stage	ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release'	string	null	no
tags	Additional tags (e.g. {'BusinessUnit': 'XYZ'}). Neither the tag keys nor the tag values will be modified by this module.	map(string)	{}	no
tenant	ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for	string	null	no
timeout	Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to 300 seconds	number	null	no
wait	Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as timeout. Defaults to true.	bool	null	no
webhook	Configuration for the GitHub Webhook Server. hostname_template is the format() string to use to generate the hostname via format(var.hostname_template, var.tenant, var.stage, var.environment)" Typically something like "echo.%[3]v.%[2]v.example.com".	object({ enabled = bool hostname_template = string })	{ "enabled": false, "hostname_template": null }	no

Outputs

Name	Description
metadata	Block status of the deployed release
metadata_action_runner_releases	Block statuses of the deployed actions-runner chart releases
webhook_payload_url	Payload URL for GitHub webhook

References

• cloudposse/terraform-aws-components - Cloud Posse's upstream component
• alb-controller - Helm Chart
• alb-controller - AWS Load Balancer Controller
• actions-runner-controller Webhook Driven Scaling
• actions-runner-controller Chart Values