Count vs For Each
This article is part of our "Terraform in Depth" series, where we dive into the details of Terraform that require a deeper understanding and longer explanation than are required for our other Terraform articles.
When you are dynamically creating multiple instances of a resource in Terraform,
you have two options: count and for_each. Both of these options allow you to
create multiple instances of a resource, but they have different use cases
and different implications for your Terraform code.
There are 2 key considerations when using count or for_each:
- Addressing: Terraform must be able to determine the "address" of each resource instance during the planning phase. This is discussed in the next section.
- Stability: When using
count, resources whose configuration has not changed can nevertheless be destroyed and recreated by Terraform because they have moved to a new address. Usingfor_eachusually avoids this issue. This is discussed further below.
Use for_each when possible, and count when you can't use for_each.
Background: Terraform Resource Addressing
During the planning phase, Terraform must be able to
determine the "address"
of each resource instance. The address of a resource
instance is a unique identifier that Terraform uses to track the state of
the resource. The address is a combination of the resource type, the
resource name, and, when that is not unique due to count or for_each,
the index or key of the resource instance, possibly along with other
information.
For example:
locals {
availability_zone_ids = ["usw2a", "usw2b"]
}
resource "aws_eip" "pub" {
count = length(local.availability_zone_ids)
}
will generate resources with addresses like aws_eip.pub[0] and aws_eip.pub[1].
resource "aws_eip" "pub" {
for_each = toset(local.availability_zone_ids)
}
will generate resources with addresses like aws_eip.pub["usw2a"] and
aws_eip.pub["usw2b"]. The values supplied to for each (either the strings
in a set of strings, or the keys of a map) are used as the keys in the
addresses.
Although documentation and commentary often refer to the requirement that Terraform must know at plan time how many instances of a resource to create, it is more accurate to say that Terraform must know at plan time the address of each instance of a resource under its management. This is because the address is used to as the key to the data structure that stores the state of the resource, and Terraform must be able to access that data during the plan phase to compare it to the desired state of the resource and compute the necessary changes.
If some address cannot be determined at plan time, terraform plan will
fail with an error. This issue is discussed in greater detail in Error: Values Cannot Be Determined Until Apply.
The main reason not to use for_each is that the values supplied to it
would not be known until apply time.
Count is Easier to Determine, but Less Stable
The count option operates on simple integers: you specify the number of
resource instances you want to create (n), and Terraform will create that
many (0 to n-1).
Because you can often know at plan time how many instances of a resource you
will need without knowing exact details of each instance, count is almost
always easier to use than for_each. However, count is less stable than
for_each, which makes it less desirable.
Use count for Simple Optional Cases
When you have a simple case where you know you want to create zero or one
instance of a resource, particularly as the result of a boolean input variable,
count is the best choice. For example:
resource "aws_instance" "bastion" {
count = var.bastion_enabled ? 1 : 0
# ...
}
None of the drawbacks of count versus for_each apply when you are never
creating more than one instance, so the advantages of count versus for_each
favor using count in this case. This is, in fact, the most common usage of
count in Cloud Posse's Terraform modules.
Similarly, to avoid using two variables for an optional resource, for
example vpc_enabled and vpc_ipv4_cidr_block, you can use a single
variable and toggle the option based on whether or not it is supplied.
Do not condition the creation of a resource on the value of a variable. Instead, place the value in a list and condition the creation of the resource on the length of the list. This is discussed in greater detail below.
It can be tempting, and indeed early Cloud Posse modules did this, to use the value of a variable to determine whether or not to create a resource.
# DO NOT DO THIS
variable "vpc_ipv4_cidr_block" {
type = string
default = null
}
resource "aws_vpc" "vpc" {
# This fails when var.vpc_ipv4_cidr_block is computed during the apply phase
count = var.vpc_ipv4_cidr_block == null ? 0 : 1
cidr_block = var.vpc_ipv4_cidr_block
}
The problem with this approach is that it requires the value of var.vpc_ipv4_cidr_block
to be known at plan time, which is frequently not the case. Often, the value
supplied will be generated or computed during the apply phase, and this
whole construct fails in this scenario.
The recommended way to toggle an option by supplying a value is to supply the value inside a list, and toggle the option based on the length of the list.
variable "vpc_ipv4_cidr_block" {
type = list(string)
default = []
# Accepting the value as a list can lead a casual user to think that
# they can supply multiple values, and that each value will be used
# somehow, perhaps to create multiple VPCs. To prevent confusion or surprise,
# add a validation rule. Without this kind of validation rule, the user
# will not get any feedback that their additional list items are being ignored.
validation {
condition = length(var.vpc_ipv4_cidr_block) < 2
error_message = <<-EOT
The list should contain at most 1 CIDR block.
If the list is empty, no VPC will be created.
EOT
}
}
resource "aws_vpc" "vpc" {
count = length(var.vpc_ipv4_cidr_block)
cidr_block = var.vpc_ipv4_cidr_block[count.index]
}
This allows the user to choose the option without having to supply the value as configuration available at plan time.
The Instability of count
The problem with count is that when you use it with a list of
configuration values, the resource instances are configured according to their
index in the list. If you add or remove an item from the list, the index of
other items in the list will change, and so even though a resource configuration
has not changed in any fundamental way, the configuration will now
apply to a different instance of the resource, effectively causing Terraform
to destroy it in one index and recreate it in another.
For example, consider the case where you want to create a set of IAM Users.
We will illustrate the problem with count here, and then show how to use
for_each to avoid the problem in the next section.
To create a reusable module that creates IAM users, you might do something like this:
variable "users" {
type = list(string)
}
resource "aws_iam_user" "example" {
count = length(var.users)
name = var.users[count.index]
}
output "users" {
value = aws_iam_user.example
}
Say you first deploy this configuration with the following input:
module "users" {
source = "./iam_users"
users = ["Dick", "Harry"]
}
output "ids" {
value = { for v in module.users.users : v.name => v.id }
}
You will get a plan like this (many elements omitted):
# module.users.aws_iam_user.example[0] will be created
+ resource "aws_iam_user" "example" {
+ name = "Dick"
}
# module.users.aws_iam_user.example[1] will be created
+ resource "aws_iam_user" "example" {
+ name = "Harry"
}
Changes to Outputs:
+ ids = {
+ Dick = "Dick"
+ Harry = "Harry"
}
This is all fine, until you realize you left out "Tom". You revise your root module like this:
module "users" {
users = ["Tom", "Dick", "Harry"]
}
You will get a plan like this (many elements omitted):
# module.users.aws_iam_user.example[0] will be updated in-place
~ resource "aws_iam_user" "example" {
id = "Dick"
~ name = "Dick" -> "Tom"
tags = {}
# (5 unchanged attributes hidden)
}
# module.users.aws_iam_user.example[1] will be updated in-place
~ resource "aws_iam_user" "example" {
id = "Harry"
~ name = "Harry" -> "Dick"
tags = {}
# (5 unchanged attributes hidden)
}
# module.users.aws_iam_user.example[2] will be created
+ resource "aws_iam_user" "example" {
+ id = (known after apply)
+ name = "Harry"
}
Plan: 1 to add, 2 to change, 0 to destroy.
Changes to Outputs:
~ ids = {
~ Dick = "Dick" -> "Harry"
~ Harry = "Harry" -> (known after apply)
+ Tom = "Dick"
}
Note that because "Tom" was inserted at the beginning of the list, all the other elements moved to new addresses, so all 3 users are going to be modified. In most cases, the existing 2 resources would be destroyed and 3 new resources would be created. That would be bad enough if, for example, the resources were NAT gateways or VPCs.
In this particular case, it is even worse! In this case (and for some other resources), existing resources will be updated in place, potentially causing serious problems when those resources are referenced by ID. If you are lucky, you will get an error message like this:
Error: updating IAM User (Harry): EntityAlreadyExists: User with name Dick already exists.
If you are unlucky (or if you run terraform apply 3 times), the change
will go through, and user "Dick" will be renamed user "Tom", meaning that
whatever access Dick had, Tom now gets. Likewise, user Dick is renamed Harry,
getting Harry's access, and Harry get the newly created user. For example, Tom
can now log in with user name "Tom" using Dick's password, while Harry will be
locked out as a new user. This nightmare scenario has a lot to do with
peculiarities of the implementation of IAM principals, but gives you an idea
of what can happen when you use count with a list of resource configurations.
Note: The above behavior was actually observed using Terraform v1.5.7 and AWS provider v5.38.0. Hopefully something less dangerous and confusing happens with the current versions of the tools when you try this yourself, but nevertheless be prepared for behavior like this.
All of this instability is a direct consequence of resource configuration
being address by its position in a list of configurations. When items are
added to or deleted from the list, or when the list provided in a random order
(as used to happen with many AWS data sources), resources may be needlessly
affected. The answer to this is for_each, but that is not without its own
limitations.
For Each is Stable, But Not Always Feasible to Use
The Stability of for_each
In large part to address the instability of count, Terraform introduced
the ability to use for_each to create multiple instances of a resource.
Where count takes a single integer, for_each takes a set of strings,
either explicitly or as the keys of a map. When you use for_each, the
instance addresses are the string values of the set of strings passed to
for_each.
We can rewrite the IAM User example using for_each like this:
variable "users" {
type = set(string)
}
resource "aws_iam_user" "example" {
for_each = var.users
name = each.key
}
Now, if deploy the code with Dick and Harry, and then you add Tom to the list of users, the plan will look like this (many elements omitted):
# module.users.aws_iam_user.example["Tom"] will be created
+ resource "aws_iam_user" "example" {
+ name = "Tom"
}
Changes to Outputs:
~ ids = {
+ Tom = (known after apply)
# (2 unchanged attributes hidden)
}
This is what we want! Nothing has changed in the code regarding Dick or Harry, so nothing has changed in the infrastructure regarding Dick or Harry.
The problems with for_each
If for_each is so much better, why is it not used by everybody all the time?
The answer is because the keys supplied to for_each must be known at plan
time. It used to be the case that data sources were not read during the plan
phase, so something like:
data "aws_availability_zones" "available" {
state = "available"
}
resource "aws_subnet" "zone" {
for_each = data.aws_aws_availability_zones.available.names
}
would fail because the zone names would not be available during the planning
phase. Hashicorp has worked to make more data available during planning to
mitigate such problems, with one major improvement being reading data sources
during planning. Thus you will see a lot of old code using count that
could now be rewritten to use for_each and require a recent version of
Terraform.
A still current issue would be trying to create a dynamic number of compute
instances and assign IP addresses to them. The most obvious way to do this
would be to use for_each with a map of instance IDs to IP addresses, but
the instance IDs are not known until the instances are created, so that
would fail. In some cases, possibly such as this one, where the
configuration for all the resources is the same, you can generate keys using
range() so that the association between a compute instance and an IP
address remains stable and is not dependent on the order in which instance
IDs are returned by the cloud provider.
In other cases, such as when the configurations vary, using proxy keys like
this has all the same problems as count, in which case using count is
better because it is simpler and all of the issues with count are already
understood.
::: note
Another limitation, though not frequently encountered, is that "sensitive"
values, such as sensitive input variables, sensitive outputs, or sensitive
resource attributes, cannot be used as arguments to for_each. As stated
previously, the value supplied to for_each is used as part of the resource
address, and as such, it will always be disclosed in UI output, which is why
sensitive values are not allowed. Attempts to use sensitive values as
for_each arguments will result in an error.
:::
Ideally, as we saw with IAM users in the examples above, the user would
supply static keys in the initial configuration, and then they would always
be known and usable in for_each, while allowing the user to add or remove
instances without affecting the others. The main obstacle to this is when
the user does not know how many instances they will need. For example, if
they need one for each availability zone, the number they need will depend
on the region they are deploying to, and they may want to adjust the
configuration for each region in that way.
Conclusion
In conclusion, use for_each when you can, and count when you must.
Finding suitable keys to use with for_each can be a challenge, but it is
often worth the effort to avoid the instability of count. Module authors
should be sensitive to the needs of their users and provide for_each where
possible, but consider using count where it seems likely the user may have
trouble providing suitable keys.
One possible solution for module authors (though generally not advisable due
to the complexity it introduces) is to accept a list of object that have an
optional key attribute, and use that attribute as the key for for_each if
it is present, and use count if it is not. This presents a consistent
interface to the user, and allows them to use for_each when they can, and
count when they must. It does introduce complexity and new failure modes,
such as when some elements have keys and others do not, or when duplicate
keys are present, or again if the keys are not known at plan time, so this
particular solution should be approached with caution. Weigh the
consequences of the complexity against the benefits of the stability of
for_each. For many kinds of resources, having them be destroyed and
recreated is of little practical consequence, so the instability of count
is not worth the added complexity and potential for failure that for_each
introduces.