Skip to main content
Sponsor @cloudposse

EKS as a Foundational Platform


We first deploy the foundation for the cluster. The eks/cluster component deploys the initial EKS resources to AWS, including Auth Config mapping. We do not deploy any nodes with the cluster initially. Then once EKS is available, we connect to the cluster and start deploying resources. First is Karpenter. We deploy the Karpenter chart on a Fargate node and the IAM service role to allow Karpenter to purchase Spot Instances. Karpenter is the only resources that will be deployed to Fargate. Then we deploy Karpenter Node Pools using the CRD created by the initial Karpenter component. These provisioners will automatically launch and scale the cluster to meet our demands. Next we deploy idp-roles to manage custom roles for the cluster, and deploy metrics-server to provide access to resource metrics.

Then we connect the cluster to our network. First we must deploy the cert-manager component to provision X.509 certificates on the cluster. Then we deploy the alb-controller component to provision and associate ALBs or NLBs based on Ingress annotations that route traffic to the cluster. Then we deploy the alb-controller-ingress-group to actually create that ALB. Next, we deploy external-dns which will look for annotations to make services discoverable, and then create records in our Route 53 Hosted Zones mapping to the cluster. Finally we deploy echo-server to validate the complete setup.

info

Connecting to an EKS cluster requires a VPN connection! See ec2-client-vpn for details.

Depending on your application requirements we can also deploy a number of operators. The most common is the efs-controller, which we use to provide encrypted block storage that is not zone-locked. Other operators are optionally but often include the external-secrets-operator to automatically sync secrets from AWS SSM Parameter Store.

Monitoring and release engineering are handled separately from the components mentioned here, and we will expand of those implementations in follow up topics. For details, see the Monitoring and Release Engineering quick start documents.

Foundation​

  • eks/cluster: This component is responsible for provisioning an end-to-end EKS Cluster, including IAM role to Kubernetes Auth Config mapping.
  • eks/karpenter: Installs the Karpenter chart on the EKS cluster and prepares the environment for provisioners.
  • idp-roles: These identity provider roles specify several pre-determined permission levels for cluster users and come with bindings that make them easy to assign to Users and Groups. Use this component to define custom permission within EKS.
  • metrics-server: A Kubernetes addon that provides resource usage metrics used in particular by other addons such Horizontal Pod Autoscaler. For more, see metrics-server.
  • reloader: Installs the Stakater Reloader for EKS clusters. reloader can watch ConfigMaps and Secrets for changes and use these to trigger rolling upgrades on pods and their associated DeploymentConfigs, Deployments, Daemonsets Statefulsets and Rollouts.

Network​

  • cert-manager: A Kubernetes addon that provisions X.509 certificates.
  • alb-controller: A Kubernetes addon that, in the context of AWS, provisions and manages ALBs and NLBs based on Service and Ingress annotations. This module is also provision a default IngressClass.
    • alb-controller-ingress-group: A Kubernetes Service that creates an ALB for a specific IngressGroup. An IngressGroup is a feature of the alb-controller which allows multiple Kubernetes Ingresses to share the same Application Load Balancer.
  • external-dns: A Kubernetes addon that configures public DNS servers with information about exposed Kubernetes services to make them discoverable. This component is responsible for adding DNS records to your Route 53 Hosted Zones.
  • echo-server: The echo server is a server that sends it back to the client a JSON representation of all the data the server received. We use this component is validate a cluster deployment.

Storage​

Additional Operators​

  • external-secrets-operator: This component (ESO) is used to create an external SecretStore configured to synchronize secrets from AWS SSM Parameter store as Kubernetes Secrets within the cluster.