Count vs For Each

Terraform in Depth

This article is part of our "Terraform in Depth" series, where we dive into the details of Terraform that require a deeper understanding and longer explanation than are required for our other Terraform articles.

Count vs For Each

When you are dynamically creating multiple instances of a resource in Terraform, you have two options: count and for_each. Both of these options allow you to create multiple instances of a resource, but they have different use cases and different implications for your Terraform code.

There are 2 key considerations when using count or for_each:

1. Addressing: Terraform must be able to determine the "address" of each resource instance during the planning phase. This is discussed in the next section.
2. Stability: When using count, resources whose configuration has not changed can nevertheless be destroyed and recreated by Terraform because they have moved to a new address. Using for_each usually avoids this issue. This is discussed further below.

The tl;dr: Use for_each when possible, and count when you can't use for_each.

Background: Terraform Resource Addressing

During the planning phase, Terraform must be able to determine the "address" of each resource instance. The address of a resource instance is a unique identifier that Terraform uses to track the state of the resource. The address is a combination of the resource type, the resource name, and, when that is not unique due to count or for_each, the index or key of the resource instance, possibly along with other information.

For example:

locals {
  availability_zone_ids = ["usw2a", "usw2b"]
}
resource "aws_eip" "pub" {
  count = length(local.availability_zone_ids)
}

will generate resources with addresses like aws_eip.pub[0] and aws_eip.pub[1].

resource "aws_eip" "pub" {
  for_each = toset(local.availability_zone_ids)
}

will generate resources with addresses like aws_eip.pub["usw2a"] and aws_eip.pub["usw2b"]. The values supplied to for each (either the strings in a set of strings, or the keys of a map) are used as the keys in the addresses.

info

Although documentation and commentary often refer to the requirement that Terraform must know at plan time how many instances of a resource to create, it is more accurate to say that Terraform must know at plan time the address of each instance of a resource under its management. This is because the address is used to as the key to the data structure that stores the state of the resource, and Terraform must be able to access that data during the plan phase to compare it to the desired state of the resource and compute the necessary changes.

If some address cannot be determined at plan time, terraform plan will fail with an error. This issue is discussed in greater detail in Error: Values Cannot Be Determined Until Apply. The main reason not to use for_each is that the values supplied to it would not be known until apply time.

Count is Easier to Determine, but Less Stable

The count option operates on simple integers: you specify the number of resource instances you want to create (n), and Terraform will create that many (0 to n-1). Because you can often know at plan time how many instances of a resource you will need without knowing exact details of each instance, count is almost always easier to use than for_each. However, count is less stable than for_each, which makes it less desirable.

Use count for Simple Optional Cases

When you have a simple case where you know you want to create zero or one instance of a resource, particularly as the result of a boolean input variable, count is the best choice. For example:

resource "aws_instance" "bastion" {
  count = var.bastion_enabled ? 1 : 0
  # ...
}

None of the drawbacks of count versus for_each apply when you are never creating more than one instance, so the advantages of count versus for_each favor using count in this case. This is, in fact, the most common usage of count in Cloud Posse's Terraform modules.

Similarly, to avoid using two variables for an optional resource, for example vpc_enabled and vpc_ipv4_cidr_block, you can use a single variable and toggle the option based on whether or not it is supplied.

caution

Do not condition the creation of a resource on the value of a variable. Instead, place the value in a list and condition the creation of the resource on the length of the list. This is discussed in greater detail below.

It can be tempting, and indeed early Cloud Posse modules did this, to use the value of a variable to determine whether or not to create a resource.

# DO NOT DO THIS
variable "vpc_ipv4_cidr_block" {
  type = string
  default = null
}
resource "aws_vpc" "vpc" {
  # This fails when var.vpc_ipv4_cidr_block is computed during the apply phase
  count = var.vpc_ipv4_cidr_block == null ? 0 : 1
  cidr_block = var.vpc_ipv4_cidr_block
}

The problem with this approach is that it requires the value of var.vpc_ipv4_cidr_block to be known at plan time, which is frequently not the case. Often, the value supplied will be generated or computed during the apply phase, and this whole construct fails in this scenario.

The recommended way to toggle an option by supplying a value is to supply the value inside a list, and toggle the option based on the length of the list.

variable "vpc_ipv4_cidr_block" {
  type = list(string)
  default = []
  # Accepting the value as a list can lead a casual user to think that
  # they can supply multiple values, and that each value will be used
  # somehow, perhaps to create multiple VPCs. To prevent confusion or surprise,
  # add a validation rule. Without this kind of validation rule, the user
  # will not get any feedback that their additional list items are being ignored.
  validation {
    condition     = length(var.vpc_ipv4_cidr_block) < 2
    error_message = <<-EOT
      The list should contain at most 1 CIDR block.
      If the list is empty, no VPC will be created.
      EOT
  }
}

resource "aws_vpc" "vpc" {
  count = length(var.vpc_ipv4_cidr_block)
  cidr_block = var.vpc_ipv4_cidr_block[count.index]
}

This allows the user to choose the option without having to supply the value as configuration available at plan time.

The Instability of count

The problem with count is that when you use it with a list of configuration values, the resource instances are configured according to their index in the list. If you add or remove an item from the list, the index of other items in the list will change, and so even though a resource configuration has not changed in any fundamental way, the configuration will now apply to a different instance of the resource, effectively causing Terraform to destroy it in one index and recreate it in another.

For example, consider the case where you want to create a set of IAM Users. We will illustrate the problem with count here, and then show how to use for_each to avoid the problem in the next section.

To create a reusable module that creates IAM users, you might do something like this:

variable "users" {
  type = list(string)
}
resource "aws_iam_user" "example" {
  count = length(var.users)
  name  = var.users[count.index]
}
output "users" {
  value = aws_iam_user.example
}

Say you first deploy this configuration with the following input:

module "users" {
  source = "./iam_users"
  users = ["Dick", "Harry"]
}
output "ids" {
  value = { for v in module.users.users : v.name => v.id }
}

You will get a plan like this (many elements omitted):

  # module.users.aws_iam_user.example[0] will be created
  + resource "aws_iam_user" "example" {
      + name          = "Dick"
    }

  # module.users.aws_iam_user.example[1] will be created
  + resource "aws_iam_user" "example" {
      + name          = "Harry"
    }
Changes to Outputs:
  + ids     = {
      + Dick  = "Dick"
      + Harry = "Harry"
    }

This is all fine, until you realize you left out "Tom". You revise your root module like this:

module "users" {
  users = ["Tom", "Dick", "Harry"]
}

You will get a plan like this (many elements omitted):

  # module.users.aws_iam_user.example[0] will be updated in-place
  ~ resource "aws_iam_user" "example" {
        id            = "Dick"
      ~ name          = "Dick" -> "Tom"
        tags          = {}
        # (5 unchanged attributes hidden)
    }

  # module.users.aws_iam_user.example[1] will be updated in-place
  ~ resource "aws_iam_user" "example" {
        id            = "Harry"
      ~ name          = "Harry" -> "Dick"
        tags          = {}
        # (5 unchanged attributes hidden)
    }

  # module.users.aws_iam_user.example[2] will be created
  + resource "aws_iam_user" "example" {
      + id            = (known after apply)
      + name          = "Harry"
    }

Plan: 1 to add, 2 to change, 0 to destroy.

Changes to Outputs:
  ~ ids     = {
      ~ Dick  = "Dick" -> "Harry"
      ~ Harry = "Harry" -> (known after apply)
      + Tom   = "Dick"
    }

Note that because "Tom" was inserted at the beginning of the list, all the other elements moved to new addresses, so all 3 users are going to be modified. In most cases, the existing 2 resources would be destroyed and 3 new resources would be created. That would be bad enough if, for example, the resources were NAT gateways or VPCs.

In this particular case, it is even worse! In this case (and for some other resources), existing resources will be updated in place, potentially causing serious problems when those resources are referenced by ID. If you are lucky, you will get an error message like this:

Error: updating IAM User (Harry): EntityAlreadyExists: User with name Dick already exists.

If you are unlucky (or if you run terraform apply 3 times), the change will go through, and user "Dick" will be renamed user "Tom", meaning that whatever access Dick had, Tom now gets. Likewise, user Dick is renamed Harry, getting Harry's access, and Harry get the newly created user. For example, Tom can now log in with user name "Tom" using Dick's password, while Harry will be locked out as a new user. This nightmare scenario has a lot to do with peculiarities of the implementation of IAM principals, but gives you an idea of what can happen when you use count with a list of resource configurations.

Note: The above behavior was actually observed using Terraform v1.5.7 and AWS provider v5.38.0. Hopefully something less dangerous and confusing happens with the current versions of the tools when you try this yourself, but nevertheless be prepared for behavior like this.

note

All of this instability is a direct consequence of resource configuration being address by its position in a list of configurations. When items are added to or deleted from the list, or when the list provided in a random order (as used to happen with many AWS data sources), resources may be needlessly affected. The answer to this is for_each, but that is not without its own limitations.

For Each is Stable, But Not Always Feasible to Use

The Stability of for_each

In large part to address the instability of count, Terraform introduced the ability to use for_each to create multiple instances of a resource. Where count takes a single integer, for_each takes a set of strings, either explicitly or as the keys of a map. When you use for_each, the instance addresses are the string values of the set of strings passed to for_each.

We can rewrite the IAM User example using for_each like this:

variable "users" {
  type = set(string)
}
resource "aws_iam_user" "example" {
  for_each = var.users
  name  = each.key
}

Now, if deploy the code with Dick and Harry, and then you add Tom to the list of users, the plan will look like this (many elements omitted):

  # module.users.aws_iam_user.example["Tom"] will be created
  + resource "aws_iam_user" "example" {
      + name          = "Tom"
    }
Changes to Outputs:
  ~ ids     = {
      + Tom   = (known after apply)
        # (2 unchanged attributes hidden)
    }

This is what we want! Nothing has changed in the code regarding Dick or Harry, so nothing has changed in the infrastructure regarding Dick or Harry.

The problems with for_each

If for_each is so much better, why is it not used by everybody all the time? The answer is because the keys supplied to for_each must be known at plan time. It used to be the case that data sources were not read during the plan phase, so something like:

data "aws_availability_zones" "available" {
  state = "available"
}
resource "aws_subnet" "zone" {
  for_each = data.aws_aws_availability_zones.available.names
}

would fail because the zone names would not be available during the planning phase. Hashicorp has worked to make more data available during planning to mitigate such problems, with one major improvement being reading data sources during planning. Thus you will see a lot of old code using count that could now be rewritten to use for_each and require a recent version of Terraform.

A still current issue would be trying to create a dynamic number of compute instances and assign IP addresses to them. The most obvious way to do this would be to use for_each with a map of instance IDs to IP addresses, but the instance IDs are not known until the instances are created, so that would fail. In some cases, possibly such as this one, where the configuration for all the resources is the same, you can generate keys using range() so that the association between a compute instance and an IP address remains stable and is not dependent on the order in which instance IDs are returned by the cloud provider.

In other cases, such as when the configurations vary, using proxy keys like this has all the same problems as count, in which case using count is better because it is simpler and all of the issues with count are already understood.

::: note Another limitation, though not frequently encountered, is that "sensitive" values, such as sensitive input variables, sensitive outputs, or sensitive resource attributes, cannot be used as arguments to for_each. As stated previously, the value supplied to for_each is used as part of the resource address, and as such, it will always be disclosed in UI output, which is why sensitive values are not allowed. Attempts to use sensitive values as for_each arguments will result in an error. :::

Ideally, as we saw with IAM users in the examples above, the user would supply static keys in the initial configuration, and then they would always be known and usable in for_each, while allowing the user to add or remove instances without affecting the others. The main obstacle to this is when the user does not know how many instances they will need. For example, if they need one for each availability zone, the number they need will depend on the region they are deploying to, and they may want to adjust the configuration for each region in that way.

Conclusion

In conclusion, use for_each when you can, and count when you must. Finding suitable keys to use with for_each can be a challenge, but it is often worth the effort to avoid the instability of count. Module authors should be sensitive to the needs of their users and provide for_each where possible, but consider using count where it seems likely the user may have trouble providing suitable keys.

One possible solution for module authors (though generally not advisable due to the complexity it introduces) is to accept a list of object that have an optional key attribute, and use that attribute as the key for for_each if it is present, and use count if it is not. This presents a consistent interface to the user, and allows them to use for_each when they can, and count when they must. It does introduce complexity and new failure modes, such as when some elements have keys and others do not, or when duplicate keys are present, or again if the keys are not known at plan time, so this particular solution should be approached with caution. Weigh the consequences of the complexity against the benefits of the stability of for_each. For many kinds of resources, having them be destroyed and recreated is of little practical consequence, so the instability of count is not worth the added complexity and potential for failure that for_each introduces.

Further Reading

• Error: Values Cannot Be Determined Until Apply.
• Terraform Best Practices