Add EKS addons and the required IRSA to the `eks` component
What
- Deprecate the
eks-iam
component - Add EKS addons and the required IRSA for the addons to the
eks
component - Add ability to specify configuration values and timeouts for addons
- Add ability to deploy addons to Fargate when necessary
- Add ability to omit specifying Availability Zones and infer them from private subnets
- Add recommended but optional and requiring opt-in: use a single Fargate Pod Execution Role for all Fargate Profiles
Why
The
eks-iam
component is not in use (we now create the IAM roles for Kubernetes Service Accounts in the https://github.com/cloudposse/terraform-aws-helm-release module), and has very old and outdated codeAWS recommends to provision the required EKS addons and not to rely on the managed addons (some of which are automatically provisioned by EKS on a cluster)
Some EKS addons (e.g.
vpc-cni
andaws-ebs-csi-driver
) require an IAM Role for Kubernetes Service Account (IRSA) with specific permissions. Since these addons are critical for cluster functionality, we create the IRSA roles for the addons in theeks
component and provide the role ARNs to the addonsSome EKS addons can be configured. In particular,
coredns
requires configuration to enable it to be deployed to Fargate.Users relying on Karpenter to deploy all nodes and wanting to deploy
coredns
oraws-ebs-csi-driver
addons need to deploy them to Fargate or else the EKS deployment will fail.Enable DRY specification of Availability Zones, and use of AZ IDs, by reading the VPCs AZs.
A cluster needs only one Fargate Pod Execution Role, and it was a mistake to provision one for every profile. However, making the change would break existing clusters, so it is optional and requires opt-in.
References
- https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html
- https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html#creating-an-add-on
- https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html
- https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html
- https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html#cni-iam-role-create-role
- https://aws.github.io/aws-eks-best-practices/networking/vpc-cni/#deploy-vpc-cni-managed-add-on
- https://docs.aws.amazon.com/eks/latest/userguide/csi-iam-role.html
- https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons
- https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#csi-iam-role
- https://github.com/kubernetes-sigs/aws-ebs-csi-driver