Module: security-group
Terraform module to create AWS Security Group and rules.
Usage
This module is primarily for setting security group rules on a security group. You can provide the ID of an existing security group to modify, or, by default, this module will create a new security group and apply the given rules to it.
This module can be used very simply, but it is actually quite complex because it is attempting to handle numerous interrelationships, restrictions, and a few bugs in ways that offer a choice between zero service interruption for updates to a security group not referenced by other security groups (by replacing the security group with a new one) versus brief service interruptions for security groups that must be preserved.
Avoiding Service Interruptions
It is desirable to avoid having service interruptions when updating a security group. This is not always possible due to the way Terraform organizes its activities and the fact that AWS will reject an attempt to create a duplicate of an existing security group rule. There is also the issue that while most AWS resources can be associated with and disassociated from security groups at any time, there remain some that may not have their security group association changed, and an attempt to change their security group will cause Terraform to delete and recreate the resource.
The 2 Ways Security Group Changes Cause Service Interruptions
Changes to a security group can cause service interruptions in 2 ways:
- Changing rules may be implemented as deleting existing rules and creating new ones. During the period between deleting the old rules and creating the new rules, the security group will block traffic intended to be allowed by the new rules.
- Changing rules may alternately be implemented as creating a new security group with the new rules and replacing the existing security group with the new one (then deleting the old one). This usually works with no service interruption in the case where all resources that reference the security group are part of the same Terraform plan. However, if, for example, the security group ID is referenced in a security group rule in a security group that is not part of the same Terraform plan, then AWS will not allow the existing (referenced) security group to be deleted, and even if it did, Terraform would not know to update the rule to reference the new security group.
The key question you need to answer to decide which configuration to use is "will anything break
if the security group ID changes". If not, then use the defaults create_before_destroy = true and
preserve_security_group_id = false and do not worry about providing "keys" for
security group rules. This is the default because it is the easiest and safest solution when
the way the security group is being used allows it.
If things will break when the security group ID changes, then set preserve_security_group_id
to true. Also read and follow the guidance below about keys and
limiting Terraform security group rules to a single AWS security group rule
if you want to mitigate against service interruptions caused by rule changes.
Note that even in this case, you probably want to keep create_before_destroy = true because otherwise,
if some change requires the security group to be replaced, Terraform will likely succeed
in deleting all the security group rules but fail to delete the security group itself,
leaving the associated resources completely inaccessible. At least with create_before_destroy = true,
the new security group will be created and used where Terraform can make the changes,
even though the old security group will still fail to be deleted.
The 3 Ways to Mitigate Against Service Interruptions
Security Group create_before_destroy = true
The most important option is create_before_destroy which, when set to true (the default),
ensures that a new replacement security group is created before an existing one is destroyed.
This is particularly important because a security group cannot be destroyed while it is associated with
a resource (e.g. a load balancer), but "destroy before create" behavior causes Terraform
to try to destroy the security group before disassociating it from associated resources,
so plans fail to apply with the error
Error deleting security group: DependencyViolation: resource sg-XXX has a dependent object
With "create before destroy" and any resources dependent on the security group as part of the same Terraform plan, replacement happens successfully:
- New security group is created
- Resource is associated with the new security group and disassociated from the old one
- Old security group is deleted successfully because there is no longer anything associated with it
(If there is a resource dependent on the security group that is also outside the scope of the Terraform plan, the old security group will fail to be deleted and you will have to address the dependency manually.)
Note that the module's default configuration of create_before_destroy = true and
preserve_security_group_id = false will force "create before destroy" behavior on the target security
group, even if the module did not create it and instead you provided a target_security_group_id.
Unfortunately, just creating the new security group first is not enough to prevent a service interruption. Keep reading.
Setting Rule Changes to Force Replacement of the Security Group
A security group by itself is just a container for rules. It only functions as desired when all the rules are in place.
If using the Terraform default "destroy before create" behavior for rules, even when using create_before_destroy for the
security group itself, an outage occurs when updating the rules or security group, because the order of operations is:
- Delete existing security group rules (triggering a service interruption)
- Create the new security group
- Associate the new security group with resources and disassociate the old one (which can take a substantial amount of time for a resource like a NAT Gateway)
- Create the new security group rules (restoring service)
- Delete the old security group
To resolve this issue, the module's default configuration of create_before_destroy = true and
preserve_security_group_id = false causes any change in the security group rules
to trigger the creation of a new security group. With that, a rule change causes operations to occur in this order:
- Create the new security group
- Create the new security group rules
- Associate the new security group with resources and disassociate the old one
- Delete the old security group rules
- Delete the old security group
Preserving the Security Group
There can be a downside to creating a new security group with every rule change.
If you want to prevent the security group ID from changing unless absolutely necessary, perhaps because the associated
resource does not allow the security group to be changed or because the ID is referenced somewhere (like in
another security group's rules) outside of this Terraform plan, then you need to set preserve_security_group_id to true.
The main drawback of this configuration is that there will normally be a service outage during an update, because existing rules will be deleted before replacement rules are created. Using keys to identify rules can help limit the impact, but even with keys, simply adding a CIDR to the list of allowed CIDRs will cause that entire rule to be deleted and recreated, causing a temporary access denial for all of the CIDRs in the rule. (For more on this and how to mitigate against it, see The Importance of Keys below.)
Also note that setting preserve_security_group_id to true does not prevent Terraform from replacing the
security group when modifying it is not an option, such as when its name or description changes.
However, if you can control the configuration adequately, you can maintain the security group ID and eliminate
impact on other security groups by setting preserve_security_group_id to true. We still recommend
leaving create_before_destroy set to true for the times when the security group must be replaced,
to avoid the DependencyViolation described above.