Module: `eks-node-group`

Terraform module to provision an EKS Node Group for Elastic Container Service for Kubernetes.

Instantiate it multiple times to create many EKS node groups with specific settings such as GPUs, EC2 instance types, or autoscale parameters.

IMPORTANT: This module provisions an EKS Node Group nodes globally accessible by SSH (22) port. Normally, AWS recommends that no security group allows unrestricted ingress access to port 22 .

Introduction

Usage

Major Changes (breaking and otherwise)

With the v2.0.0 (a.k.a. v0.25.0) release of this module, it has undergone major breaking changes and added new features. Please see the migration document for details.

For a complete example, see examples/complete.

For automated tests of the complete example using bats and Terratest (which tests and deploys the example on AWS), see test.

Terraform Version

Terraform version 1.0 is out. Before that, there was Terraform version 0.15, 0.14, 0.13 and so on. The v2.0.0 release of this module drops support for Terraform 0.13. That version is old and has lots of known issues. There are hardly any breaking changes between Terraform 0.13 and 1.0, so please upgrade to the latest Terraform version before raising any issues about this module.

Sources of Information

The code examples below are manually updated and have a tendency to fall out of sync with actual code, particularly with respect to usage of other modules. Do not rely on them.
The documentation on this page about this module's inputs, outputs, and compliance is all automatically generated and is up-to-date as of the release date. After the code itself, this is your best source of information.
The code in examples/complete is automatically tested before every release, so that is a good place to look for verified example code. Keep in mind, however, it is code for testing, so it may not represent average use cases or best practices.
Of course, the READMEs and examples/complete directories in the other modules' GitHub repos are more authoritative with respect to how to use those modules than this README is.

Example Code

provider "aws" {
  region = var.region
}

module "label" {
  source = "cloudposse/label/null"
  # Cloud Posse recommends pinning every module to a specific version
  # version  = "x.x.x"

  namespace  = var.namespace
  name       = var.name
  stage      = var.stage
  delimiter  = var.delimiter
  attributes = ["cluster"]
  tags       = var.tags
}

locals {
  # Prior to Kubernetes 1.19, the usage of the specific kubernetes.io/cluster/* resource tags below are required
  # for EKS and Kubernetes to discover and manage networking resources
  # https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking
  tags = { "kubernetes.io/cluster/${module.label.id}" = "shared" }
}

module "vpc" {
  source = "cloudposse/vpc/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "1.x.x"

  cidr_block = "172.16.0.0/16"

  tags    = local.tags
  context = module.label.context
}

module "subnets" {
  source = "cloudposse/dynamic-subnets/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "2.x.x"

  availability_zones   = var.availability_zones
  vpc_id               = module.vpc.vpc_id
  igw_id               = [module.vpc.igw_id]
  ipv4_cidr_block      = [module.vpc.vpc_cidr_block]
  nat_gateway_enabled  = true
  nat_instance_enabled = false

  tags    = local.tags
  context = module.label.context
}

module "eks_cluster" {
  source = "cloudposse/eks-cluster/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version = "2.x.x"

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.subnets.public_subnet_ids

  kubernetes_version    = var.kubernetes_version
  oidc_provider_enabled = true

  context = module.label.context
}

module "eks_node_group" {
  source = "cloudposse/eks-node-group/aws"
  # Cloud Posse recommends pinning every module to a specific version
  # version     = "2.x.x"

  instance_types        = [var.instance_type]
  subnet_ids            = module.subnets.public_subnet_ids
  min_size              = var.min_size
  max_size              = var.max_size
  cluster_name          = module.eks_cluster.eks_cluster_id
  create_before_destroy = true
  kubernetes_version    = var.kubernetes_version == null || var.kubernetes_version == "" ? [] : [var.kubernetes_version]

  # Enable the Kubernetes cluster auto-scaler to find the auto-scaling group
  cluster_autoscaler_enabled = var.autoscaling_policies_enabled

  context = module.label.context

  # Ensure the cluster is fully created before trying to add the node group
  module_depends_on = [module.eks_cluster.kubernetes_config_map_id]
}

Windows Managed Node groups

Windows managed node-groups have a few pre-requisites.

Your cluster must contain at least one linux based worker node
Your EKS Cluster must have the AmazonEKSVPCResourceController and AmazonEKSClusterPolicy policies attached
Your cluster must have a config-map called amazon-vpc-cni with the following content

apiVersion: v1
kind: ConfigMap
metadata:
name: amazon-vpc-cni
namespace: kube-system
data:
enable-windows-ipam: "true"

Windows nodes will automatically be tainted

kubernetes_taints = [{
  key    = "WINDOWS"
  value  = "true"
  effect = "NO_SCHEDULE"
}]

Any pods that target Windows will need to have the following attributes set in their manifest

  nodeSelector:
    kubernetes.io/os: windows
    kubernetes.io/arch: amd64

https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html

Requirements

Name	Version
terraform	>= 1.3.0
aws	>= 4.48
random	>= 2.0

Providers

Name	Version
aws	>= 4.48
random	>= 2.0

Modules

Name	Source	Version
label	cloudposse/label/null	0.25.0
ssh_access	cloudposse/security-group/aws	2.2.0
this	cloudposse/label/null	0.25.0

Resources

Name	Type
aws_eks_node_group.cbd	resource
aws_eks_node_group.default	resource
aws_iam_policy.ipv6_eks_cni_policy	resource
aws_iam_role.default	resource
aws_iam_role_policy_attachment.amazon_ec2_container_registry_read_only	resource
aws_iam_role_policy_attachment.amazon_eks_worker_node_policy	resource
aws_iam_role_policy_attachment.existing_policies_for_eks_workers_role	resource
aws_iam_role_policy_attachment.ipv6_eks_cni_policy	resource
aws_launch_template.default	resource
random_pet.cbd	resource
aws_ami.selected	data source
aws_eks_cluster.this	data source
aws_iam_policy_document.assume_role	data source
aws_iam_policy_document.ipv6_eks_cni_policy	data source
aws_launch_template.this	data source
aws_partition.current	data source

Inputs

Name	Description	Type	Default	Required
additional_tag_map	Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration.	`map(string)`	`{}`	no
after_cluster_joining_userdata	Additional `bash` commands to execute on each worker node after joining the EKS cluster (after executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production	`list(string)`	`[]`	no
ami_image_id	AMI to use. Ignored if `launch_template_id` is supplied.	`list(string)`	`[]`	no
ami_release_version	EKS AMI version to use, e.g. For AL2 "1.16.13-20200821" or for bottlerocket "1.2.0-ccf1b754" (no "v") or for Windows "2023.02.14". For AL2, bottlerocket and Windows, it defaults to latest version for Kubernetes version.	`list(string)`	`[]`	no
ami_type	Type of Amazon Machine Image (AMI) associated with the EKS Node Group. Defaults to `AL2_x86_64`. Valid values: `AL2_x86_64, AL2_x86_64_GPU, AL2_ARM_64, CUSTOM, BOTTLEROCKET_ARM_64, BOTTLEROCKET_x86_64, BOTTLEROCKET_ARM_64_NVIDIA, BOTTLEROCKET_x86_64_NVIDIA, WINDOWS_CORE_2019_x86_64, WINDOWS_FULL_2019_x86_64, WINDOWS_CORE_2022_x86_64, WINDOWS_FULL_2022_x86_64`.	`string`	`"AL2_x86_64"`	no
associate_cluster_security_group	When true, associate the default cluster security group to the nodes. If disabled the EKS managed security group will not be associated to the nodes and you will need to provide another security group that allows the nodes to communicate with the EKS control plane. Be aware that if no `associated_security_group_ids` or `ssh_access_security_group_ids` are provided, then the nodes will have no inbound or outbound rules.	`bool`	`true`	no
associated_security_group_ids	A list of IDs of Security Groups to associate the node group with, in addition to the EKS' created security group. These security groups will not be modified.	`list(string)`	`[]`	no
attributes	ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element.	`list(string)`	`[]`	no
before_cluster_joining_userdata	Additional `bash` commands to execute on each worker node before joining the EKS cluster (before executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production	`list(string)`	`[]`	no
block_device_map	Map of block device name specification, see launch_template.block-devices.	`map(object({ no_device = optional(bool, null) virtual_name = optional(string, null) ebs = optional(object({ delete_on_termination = optional(bool, true) encrypted = optional(bool, true) iops = optional(number, null) kms_key_id = optional(string, null) snapshot_id = optional(string, null) throughput = optional(number, null) volume_size = optional(number, 20) volume_type = optional(string, "gp3") })) }))`	`{ "/dev/xvda": { "ebs": {} } }`	no
block_device_mappings	DEPRECATED: Use `block_device_map` instead. List of block device mappings for the launch template. Each list element is an object with a `device_name` key and any keys supported by the `ebs` block of `launch_template`.	`list(any)`	`null`	no
bootstrap_additional_options	Additional options to bootstrap.sh. DO NOT include `--kubelet-additional-args`, use `kubelet_additional_options` var instead.	`list(string)`	`[]`	no
capacity_type	Type of capacity associated with the EKS Node Group. Valid values: "ON_DEMAND", "SPOT", or `null`. Terraform will only perform drift detection if a configuration value is provided.	`string`	`null`	no
cluster_autoscaler_enabled	Set `true` to label the node group so that the Kubernetes Cluster Autoscaler will discover and autoscale it. Note that even when `false`, EKS will set the `k8s.io/cluster-autoscaler/enabled` label to `true` on the node group.	`bool`	`false`	no
cluster_name	The name of the EKS cluster	`string`	n/a	yes
context	Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged.	`any`	`{ "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null }`	no
cpu_options	Configuration for the `cpu_options` Configuration Block of the launch template. Leave list empty for defaults. Pass list with single object with attributes matching the `cpu_options` block to configure it. Note that this configures the launch template only. Some elements will be ignored by the Auto Scaling Group that actually launches instances. Consult AWS documentation for details.	`list(any)`	`[]`	no
create_before_destroy	Set true in order to create the new node group before destroying the old one. If false, the old node group will be destroyed first, causing downtime. Changing this setting will always cause node group to be replaced.	`bool`	`false`	no
delimiter	Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all.	`string`	`null`	no
descriptor_formats	Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `{<br/> format = string<br/> labels = list(string)<br/>}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty).	`any`	`{}`	no
desired_size	Initial desired number of worker nodes (external changes ignored)	`number`	n/a	yes
detailed_monitoring_enabled	The launched EC2 instance will have detailed monitoring enabled. Defaults to false	`bool`	`false`	no
ebs_optimized	Set `false` to disable EBS optimization	`bool`	`true`	no
ec2_ssh_key_name	SSH key pair name to use to access the worker nodes	`list(string)`	`[]`	no
enabled	Set to false to prevent the module from creating any resources	`bool`	`null`	no
enclave_enabled	Set to `true` to enable Nitro Enclaves on the instance.	`bool`	`false`	no
environment	ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT'	`string`	`null`	no
force_update_version	When updating the Kubernetes version, force Pods to be removed even if PodDisruptionBudget or taint/toleration issues would otherwise prevent them from being removed (and cause the update to fail)	`bool`	`false`	no
id_length_limit	Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`.	`number`	`null`	no
instance_types	Instance types to use for this node group (up to 20). Defaults to ["t3.medium"]. Must be empty if the launch template configured by `launch_template_id` specifies an instance type.	`list(string)`	`[ "t3.medium" ]`	no
kubelet_additional_options	Additional flags to pass to kubelet. DO NOT include `--node-labels` or `--node-taints`, use `kubernetes_labels` and `kubernetes_taints` to specify those."	`list(string)`	`[]`	no
kubernetes_labels	Key-value mapping of Kubernetes labels. Only labels that are applied with the EKS API are managed by this argument. Other Kubernetes labels applied to the EKS Node Group will not be managed.	`map(string)`	`{}`	no
kubernetes_taints	List of `key`, `value`, `effect` objects representing Kubernetes taints. `effect` must be one of `NO_SCHEDULE`, `NO_EXECUTE`, or `PREFER_NO_SCHEDULE`. `key` and `effect` are required, `value` may be null.	`list(object({ key = string value = string effect = string }))`	`[]`	no
kubernetes_version	Kubernetes version. Defaults to EKS Cluster Kubernetes version. Terraform will only perform drift detection if a configuration value is provided	`list(string)`	`[]`	no
label_key_case	Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`.	`string`	`null`	no
label_order	The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present.	`list(string)`	`null`	no
label_value_case	Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`.	`string`	`null`	no
labels_as_tags	Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. Notes: The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored.	`set(string)`	`[ "default" ]`	no
launch_template_id	The ID (not name) of a custom launch template to use for the EKS node group. If provided, it must specify the AMI image ID.	`list(string)`	`[]`	no
launch_template_version	The version of the specified launch template to use. Defaults to latest version.	`list(string)`	`[]`	no
max_size	Maximum number of worker nodes	`number`	n/a	yes
metadata_http_endpoint_enabled	Set false to disable the Instance Metadata Service.	`bool`	`true`	no
metadata_http_put_response_hop_limit	The desired HTTP PUT response hop limit (between 1 and 64) for Instance Metadata Service requests. The default is `2` to allows containerized workloads assuming the instance profile, but it's not really recomended. You should use OIDC service accounts instead.	`number`	`2`	no
metadata_http_tokens_required	Set true to require IMDS session tokens, disabling Instance Metadata Service Version 1.	`bool`	`true`	no
min_size	Minimum number of worker nodes	`number`	n/a	yes
module_depends_on	Can be any value desired. Module will wait for this value to be computed before creating node group.	`any`	`null`	no
name	ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input.	`string`	`null`	no
namespace	ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique	`string`	`null`	no
node_group_terraform_timeouts	Configuration for the Terraform `timeouts` Configuration Block of the node group resource. Leave list empty for defaults. Pass list with single object with attributes matching the `timeouts` block to configure it. Leave attribute values `null` to preserve individual defaults while setting others.	`list(object({ create = string update = string delete = string }))`	`[]`	no
node_role_arn	If provided, assign workers the given role, which this module will not modify	`list(string)`	`[]`	no
node_role_cni_policy_enabled	When true, the `AmazonEKS_CNI_Policy` will be attached to the node IAM role. This used to be required, but it is now recommended that this policy be attached only to the `aws-node` Kubernetes service account. However, that is difficult to do with Terraform, so this module defaults to the old pattern.	`bool`	`true`	no
node_role_permissions_boundary	If provided, all IAM roles will be created with this permissions boundary attached.	`string`	`null`	no
node_role_policy_arns	List of policy ARNs to attach to the worker role this module creates in addition to the default ones	`list(string)`	`[]`	no
placement	Configuration for the `placement` Configuration Block of the launch template. Leave list empty for defaults. Pass list with single object with attributes matching the `placement` block to configure it. Note that this configures the launch template only. Some elements will be ignored by the Auto Scaling Group that actually launches instances. Consult AWS documentation for details.	`list(any)`	`[]`	no
regex_replace_chars	Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits.	`string`	`null`	no
replace_node_group_on_version_update	Force Node Group replacement when updating to a new Kubernetes version. If set to `false` (the default), the Node Groups will be updated in-place	`bool`	`false`	no
resources_to_tag	List of auto-launched resource types to tag. Valid types are "instance", "volume", "elastic-gpu", "spot-instances-request", "network-interface".	`list(string)`	`[ "instance", "volume", "network-interface" ]`	no
ssh_access_security_group_ids	Set of EC2 Security Group IDs to allow SSH access (port 22) to the worker nodes. If you specify `ec2_ssh_key`, but do not specify this configuration when you create an EKS Node Group, port 22 on the worker nodes is opened to the Internet (0.0.0.0/0)	`list(string)`	`[]`	no
stage	ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release'	`string`	`null`	no
subnet_ids	A list of subnet IDs to launch resources in	`list(string)`	n/a	yes
tags	Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module.	`map(string)`	`{}`	no
tenant	ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for	`string`	`null`	no
update_config	Configuration for the `eks_node_group` `update_config` Configuration Block. Specify exactly one of `max_unavailable` (node count) or `max_unavailable_percentage` (percentage of nodes).	`list(map(number))`	`[]`	no
userdata_override_base64	Many features of this module rely on the `bootstrap.sh` provided with Amazon Linux, and this module may generate "user data" that expects to find that script. If you want to use an AMI that is not compatible with the Amazon Linux `bootstrap.sh` initialization, then use `userdata_override_base64` to provide your own (Base64 encoded) user data. Use "" to prevent any user data from being set. Setting `userdata_override_base64` disables `kubernetes_taints`, `kubelet_additional_options`, `before_cluster_joining_userdata`, `after_cluster_joining_userdata`, and `bootstrap_additional_options`.	`list(string)`	`[]`	no

Outputs

Name	Description
eks_node_group_arn	Amazon Resource Name (ARN) of the EKS Node Group
eks_node_group_cbd_pet_name	The pet name of this node group, if this module generated one
eks_node_group_id	EKS Cluster name and EKS Node Group name separated by a colon
eks_node_group_launch_template_id	The ID of the launch template used for this node group
eks_node_group_launch_template_name	The name of the launch template used for this node group
eks_node_group_remote_access_security_group_id	The ID of the security group generated to allow SSH access to the nodes, if this module generated one
eks_node_group_resources	List of objects containing information about underlying resources of the EKS Node Group
eks_node_group_role_arn	ARN of the worker nodes IAM role
eks_node_group_role_name	Name of the worker nodes IAM role
eks_node_group_status	Status of the EKS Node Group
eks_node_group_tags_all	A map of tags assigned to the resource, including those inherited from the provider default_tags configuration block.
eks_node_group_windows_note	Instructions on changes a user needs to follow or script for a windows node group in the event of a custom ami

Module: eks-node-group

Introduction​

Usage​

Major Changes (breaking and otherwise)​

Terraform Version​

Sources of Information​

Example Code​

Windows Managed Node groups​

Requirements​

Providers​

Modules​

Resources​

Inputs​

Outputs​