# The Cloud Posse Reference Architecture > The turnkey architecture for AWS, Datadog & GitHub Actions to get up and running quickly using the Atmos open source framework. This file contains all documentation content in a single document following the llmstxt.org standard. ## Best Practices > > Physics is the law, everything else is a recommendation. > Anyone can break laws created by people, but I have yet to see anyone break the laws of physics. > — **Elon Musk** > --- ## Developer Best Practices import DocCardList from '@theme/DocCardList'; import Intro from '@site/src/components/Intro'; ### Recommendations --- ## Editor Config Best Practices import Intro from '@site/src/components/Intro'; The EditorConfig enables developers to define and maintain consistent coding styles between different editors and IDEs. It consists of a simple file format (`.editorconfig`) for defining coding styles such as tabs vs spaces. Most text editors support the format and adhere to defined styles. The config files are easily readable and they work nicely with version control systems. ## Example Place this file in the root of your git repository. ### `.editorconfig` ```ini # top-most EditorConfig file root = true # Unix-style newlines with a newline ending every file [*] end_of_line = lf insert_final_newline = true # Matches multiple files with brace expansion notation # Set default charset [*.{js,py}] charset = utf-8 # 4 space indentation [*.py] indent_style = space indent_size = 4 # Override for Makefile [{Makefile, makefile, GNUmakefile}] indent_style = tab indent_size = 4 [Makefile.*] indent_style = tab indent_size = 4 # Indentation override for all JS under lib directory [lib/**.js] indent_style = space indent_size = 2 # Matches the exact files either package.json or .travis.yml [{package.json,.travis.yml}] indent_style = space indent_size = 2 [shell] indent_style = tab indent_size = 4 [*.sh] indent_style = tab indent_size = 4 ``` ## Editor Plugins Find all plugins here: http://editorconfig.org/#download - [Vim](https://github.com/editorconfig/editorconfig-vim#readme) - [Visual Studio](https://marketplace.visualstudio.com/items?itemName=EditorConfigTeam.EditorConfig) ## References - http://editorconfig.org/ --- ## Sign Your GitHub Commits with SSH If you are already using SSH to authenticate to GitHub, it is very easy to sign all your commits as well, as long as you have already installed Git 2.34.0 or later. (Note, there may be problems with OpenSSH 8.7. Use an earlier or later version. I have this working with OpenSSH 8.1p1.) ### Configure git to sign all your commits with an SSH key ```bash git config --global gpg.format ssh git config --global commit.gpgsign true git config --global tag.gpgsign true ``` ### Configure git with the public key to use when signing Set `KEY_FILE` to the file containing your SSH public key ```bash KEY_FILE=~/.ssh/id_ed25519.pub git config --global user.signingKey "$(head -1 $KEY_FILE)" ``` Add your SSH public key to GitHub as a signing key, much the same way you added it as an authentication key, but choose "Signing Key" instead of "Authentication Key" under "Key type", even if you already have it uploaded as an authentication key. Detailed instructions are available [here](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account#adding-a-new-ssh-key-to-your-account). We suggest using the same key you use to authenticate with, so that signing is the same as pulling and pushing, but you can use a different key if you want to be prompted for a password with every commit. --- ## Makefile Best Practices import Intro from '@site/src/components/Intro'; GNU Makefiles are a convenient way for last-mile automation across multiple tool sets. We used to rely more heavily on Makefiles, but have since transitioned our usage predominantly into Atmos itself. That said, here is a collection of some of the best practices we’ve amassed over the years from extensively leveraging Makefiles. ## Avoid using Evals The use of `$(eval ...)` leads to very confusing execution paths, due to the way `make` evaluates a target. When `make` executes a target, it preprocesses all `$(....)` interpolations and renders the template. After that, it executes, line-by-line each command in the target. ## Namespace targets Over time, the number of targets in a `Makefile` will grow. We recommend namespacing all targets. For example: ``` docker/build: docker build -t example/test . ``` ## Use `/` as a target namespace delimiter When naming target names, we recommend using `/` as the delimiter rather than `:` or `-`. Further more, we recommend sticking all targets within a namespace into a separate file. E.g. `Makefile.docker` for all targets that begin with `docker/`. For example, stick this in `Makefile.docker` ``` docker/build: docker build -t example/test . ``` ## Avoid using `:` in target names While it's possible to use `:` as the delimiter in target names, there is a big gotcha: it breaks target dependencies. For example: ``` docker\:deps: docker pull example/base-image docker\:build: docker:deps docker build -t example/test ``` In this example, `make` will silently ignore calling the target dependency of `docker:deps`. Escaping the target dependency (e.g. `docker\:deps`) has no effect. ## Use `include` Avoid sticking every target in the same `Makefile` for the same reason we don't stick all code in the same source file. We typically recommend adding something like this to the top of our `Makefile`: ``` -include tasks/Makefile.* ``` :::info > The leading `-` tells `make` not to error if the `tasks/` folder is empty. ::: ## Define sane defaults for environment variables No one likes to pass 20 arguments to `make`. Set sane defaults for all variables using the `?=` operator. For example: ``` DOCKER_TAG ?= latest ``` ## Pass Environment Variables like Function Arguments The nice thing about `make` is it will automatically export all arguments in `key=value` notation as environment variables. This let's us call `make` targets like functions. e.g. ``` make docker/build DOCKER_TAG=dev ``` ## Write small targets Make is an excellent language for gluing together various tools in your toolchain. It's an easy trap to stick an entire `bash` script inside of a target. From experience, these targets become error prone and difficult to maintain for anyone but a seasoned `make` programmer. Instead, stick complex logic inside of shell scripts and call those shell scripts from a target. ## Use target dependencies A target can have dependencies called automatically prior to executing the target. If anyone of the dependencies fails, the execution aborts and the target will not be called. For example: ``` deps: @which docker build: deps @docker build -t example/test . ``` ## Use standard target names in root `Makefile` The entry-level `Makefile` should define these standard targets across all projects. This makes it very easy for anyone to get started who is familiar with `make`. * `deps` * `build` * `install` * `default` * `all` *IMPORTANT:* All leading whitespace should be tabbed (`^T`) ## Help Target Our standard `help` target. This will automatically generate well-formatted output for any target that has a `###` comment preceding it. ![Example Help Target Output](/assets/7ee92cd-Screen_Shot_2018-04-01_at_12.03.15_AM.png) Simply add this code snippet to your `Makefile` and you'll get this functionality. ``` ## This help screen help: @printf "Available targets:\n\n" @awk '/^[a-zA-Z\-\_0-9%:\\]+/ { \ helpMessage = match(lastLine, /^## (.*)/); \ if (helpMessage) { \ helpCommand = $$1; \ helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \ gsub("\\\\", "", helpCommand); \ gsub(":+$$", "", helpCommand); \ printf " \x1b[32;01m%-35s\x1b[0m %s\n", helpCommand, helpMessage; \ } \ } \ { lastLine = $$0 }' $(MAKEFILE_LIST) | sort -u @printf "\n" ``` # Default target Add this to the top of your `Makefile` to automatically call `help` if no target passed. ``` default: help ``` --- ## Markdown Best Practices import Intro from '@site/src/components/Intro'; Most of our documentation is provided in Markdown format. Here are some of the conventions and best practices we follow when writing Markdown. Please note that we use the term Markdown loosely to refer to GitHub-flavored Markdown, and we also use quite a bit of MDX, which is what all of our documentation in Docusaurus uses. ## Code Blocks Use code blocks for anything more than 1 line. Use `code` for inline code, filenames, commands, etc. ### Code Block ~~~~markdown ``` # This is a code block ``` ~~~~ ### Table of Options Use tables to communicate lists of options. Here's an example: ##### Table of Options ```markdown | Name | Default | Description | Required | |:-----------|:-------:|:-------------------------------------------|:--------:| | namespace | | Namespace (e.g. `cp` or `cloudposse`) | Yes | | stage | | Stage (e.g. `prod`, `dev`, `staging`) | Yes | | name | | Name (e.g. `bastion` or `db`) | Yes | | attributes | [] | Additional attributes (e.g. `policy`) | No | | tags | {} | Additional tags (e.g. `map("Foo","XYZ")`) | No | ``` * `:--------:` should be used for “Default” and “Required” values * `:---------` should be used for all other columns * Use `value` for all values Which will render to something like this: ![Example Markdown Table Rendering](/assets/8d8cdf3-image_23.png) ![Example Markdown Table Rendering](/assets/a2761a9-image_22.png) ## Feature List Formatting Use this format describe the features & benefits. ### Feature List Example ```markdown 1. **Feature 1** - Explanation of benefits 2. **Feature 2** - Explanation of benefits ``` ## Use Block Quotes Reference copyrighted text, quotes, and other unoriginal copy using `>` ### Block Quote Example ```markdown > Amazon Simple Storage Service (Amazon S3) makes it simple and practical to collect, store, and analyze data - regardless of format – all at massive scale. ``` --- ## Password Management We strongly advise all companies to use "1Password for Teams" as their password management solution. ## Features - Shared MFA - useful for root accounts like AWS - MFA Integration with Duo - Groups - Slack Integration - Cloud Storage - Cross-platform support (OSX, Windows, Linux, & Web) ## Alternatives - LastPass --- ## Semantic Versioning We practice [Semantic Versioning](https://semver.org/) for all projects (e.g. GitHub Tags/Releases, Helm Charts, Terraform Modules, Docker Images). Using this versioning standard helps to reduce the entropy related to [Dependency Hell](https://en.wikipedia.org/wiki/Dependency_hell).
Image credit: [Gopher Academy](https://blog.gopheracademy.com/advent-2015/semver/)
## Semantics Generally, all of our versions follow this convention: `X.Y.Z` (e.g. `1.2.3`). Sometimes, we'll use this format: `X.Y.Z-branch` when we need to disambiguate between versions existing in multiple branches.
Major Releases
These are releases when `X` changes. These releases will typically have major changes in the interface. These releases may not be backward-compatible.
Minor Releases
These are releases when `Y` changes. These releases bundle new features, but the interface should be largely the same. Minor releases should be backward-compatible.
Patch Releases
These are releases when `Z` changes. These releases are typically bug fixes which do not introduce new features. These releases are backward-compatible.
We use GitHub tags & releases for all versioning. All docker images follow the same convention. ## Versioning ### 0.X.Y We always start projects off at `0.1.0`. This is our first release of any project. While we try to keep our interfaces stable, as long as `X=0`, it indicates that our code does not yet have a stable API and may vary radically between minor releases. ### 1.X.Y+ As soon as our code reaches `1.X.Y`, the interface should be relatively stable - that is not changing much between minor releases. ## Implementation Managing semantic versions should be automated just like everything else in our infrastructure. The [`build-harness`](/learn/toolchain/#build-harness) is used by our CI/CD process to automatically generate versions based on git history. --- ## Docker Best Practices import Intro from '@site/src/components/Intro'; Docker best practices that we follow are listed here. Note that this is not an exhaustive list, but rather some of the ones that have stood out for us as practical ways of leveraging Docker together with the Cloud Posse reference architecture. ## Inheritance Inheritance is when you use `FROM some-image:1.2.3` (vs `FROM scratch`) in a `Dockerfile`. We recommend to leverage lean base images (E.g. `alpine` or `busybox`). Try to leverage the same base image in as many of your images as possible for faster `docker pulls`. :::info - https://docs.docker.com/engine/reference/builder/#from ::: ## Multi-stage Builds There are two ways to leverage multi-stage builds: 1. *Build-time Environments* The most common application of multi-stage builds is for using a build-time environment for compiling apps, and then a minimal image (E.g. `alpine` or `scratch`) for distributing the resultant artifacts (e.g. statically-linked `go` binaries). 2. *Multiple-Inheritance* We like to think of "multi-stage builds" as a mechanism for "multiple inheritance" as it relates to docker images. While not technically the same thing, using multi-stage images makes it possible to `COPY --from=other-image` to keep things very DRY. :::info - https://docs.docker.com/develop/develop-images/multistage-build/ - https://blog.alexellis.io/mutli-stage-docker-builds/ ::: ## Use Scratch Base Image One often overlooked, ultimately lean base-image is the `scratch` image. This is an empty filesystem which allows one to copy/distribute the minimal set of artifacts. For languages that can compile statically linked binaries, using the `scratch` base image (e.g. `FROM scratch`) is the most secure way as there will be no other exploitable packages bundled in the image. We use this pattern for our [`terraform-root-modules`](https://github.com/cloudposse/terraform-root-modules) distribution of terraform reference architectures. ## Configure Cache Storage Backends When using BuildKit, you should configure a [cache storage backend](https://docs.docker.com/build/cache/backends/) that is suitable for your build environment. Layer caching significantly speeds up builds by reusing layers from previous builds, and is enabled by default as BuildKit has a dedicated local cache. However, in a CI/CD build environment such as GitHub Actions, an external cache storage backend is essential as there is little to no persistence between builds. Fortunately, Cloud Posse's [cloudposse/github-action-docker-build-push](https://github.com/cloudposse/github-action-docker-build-push) action uses `gha` (the [GitHub Actions Cache](https://docs.github.com/en/rest/actions/cache)) by default. Thus, even without any additional configuration, the action will automatically cache layers between builds. When using self-hosted GitHub Actions Runners in an AWS environment, however, we recommend using [ECR as a remote cache storage backend](https://aws.amazon.com/blogs/containers/announcing-remote-cache-support-in-amazon-ecr-for-buildkit-clients/). Using ECR as the remote cache backend—especially in conjunction with a [VPC endpoint for ECR](https://docs.aws.amazon.com/AmazonECR/latest/userguide/vpc-endpoints.html)—results in reduced NAT Gateway costs and faster layered cache imports when compared to the GitHub Actions Cache. The following example demonstrates how to configure the [cloudposse/github-action-docker-build-push](https://github.com/cloudposse/github-action-docker-build-push) action to use ECR as the remote cache storage backend: ```diff - name: Build id: build uses: cloudposse/github-action-docker-build-push@main with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" + cache-from: "type=registry,ref=registry.hub.docker.com/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:cache" + cache-to: "mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=registry.hub.docker.com/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:cache" ``` For more information with regards to the `cache-from` and `cache-to` options, please refer to the [docker buildx documentation](https://docs.docker.com/reference/cli/docker/buildx/build/#options). --- ## GitHub Feature Branches import Intro from '@site/src/components/Intro' The Feature Branch Workflow is a requirement for CI/CD. It's a process by which all feature development takes place in a dedicated branch instead of the `master` branch. This makes it easy for multiple developers to collaborate on a particular feature while at the same time ensuring that the master branch remains stable." ## Problem When you're working on a project, there will be a bunch of different features or ideas in progress at any given time, not all of which are ready for prime time. Further more, as business priorities change, you might need to prioritize certain features and put others on the back burner. At the same time, business requirements mandate that you have a stable version that can be deployed at any given time. We know code can never be entirely bug-free. Furthermore, once deployed there can be unintended consequences. Other times, managers simply change their mind and decide that a certain feature was premature or unnecessary. To mitigate the impact of these events, we need the ability to rollback a feature or cut-bait. **TL;DR:** If everyone is working on the same branch such as master, it pollutes the commit history making it all but impossible to figure out which commits relate to specific features making rollbacks impossible. ## Solution To solve this problem, the standard workflow called _branching_ should be used religiously. Any time a new feature is developed it must be worked on in a separate branch. When you create a branch, you're creating an environment where you can freely test out new ideas without impacting others because changes made on a branch don't affect the `master` branch (or any other one). Furthermore, no two developers should ever commit to or work on the same branch at the same time (unless they have permission from the branch stakeholder). Instead, they should create a new branch. The next thing that needs to happen is that the `master` branch is treated as the Holy Grail. Every effort is made to ensure it's stable and can be deployed to production at any time. Once a feature is considered ready, the developer submits a Pull Request (or PR) and assigns it to a Subject Matter Expert (SME) or peer for review. On the surface, this is what a well-formatted Pull Request looks like: ![Example Pull Request](/assets/e802ae2-image_3.png) A _Pull Request_ allows many things to happen: - **Title**: A "human readable" title that represents the feature! ![Example Pull Request Title](/assets/2d4fce9-image.png) - **Description**: A long description that details **_What_** was changed, **_Why_** it was deemed necessary, and any other **_References_** that might be useful (E.g. Jira ticket) - **Comments**: let anyone provide arbitrary feedback viewable by everyone. - **Diffs**: show what changed between this feature and the current master branch - **Formal Code Review Process:** let multiple people contribute to the code review process by submitting comments on a line-by-line basis. Having these code reviews formally documented serves as an excellent teaching tool. Over time, the reviews become faster and faster as developers learn what is expected. ![Example of Code Review](/assets/9df4fad-image_2.png) - **Merging**: Once the PR is approved, the developer can squash and merge their code into the master branch. Squashing allows the master branch to have a very clean commit history where every commit corresponds to a PR. ![Example of Merging](/assets/2b3e7eb-image_4.png) - **Clean Commit History**: means that every change to the master branch is documented and justified. No one is sneaking in changes. ![Example of Clean Commit History](/assets/b3dae79-image_5.png) - **History of Features** and when they were added ![History of Features](/assets/f9a3727-image_7.png) - **Reverting**: If a feature needs to be removed, with the click of a single button it can be removed from the `master` branch ![Example of Reverting Changes](/assets/28887e9-image_8.png) ## Technical Details ### Create a Branch Whenever you begin work on a new feature or bugfix, it's important that you create a new branch. Not only is it proper git workflow, but it also keeps your changes organized and separated from the master branch so that you can easily submit and manage multiple pull requests for every task you complete. To create a new branch and start working on it: ```shell # Checkout the master branch - you want your new branch to come from master git checkout master # Pull down the latest changes git pull origin master # Create a new branch, with a descriptive name (e.g. implement-xyz-widget) git checkout -b newfeature ``` Now, go to town hacking away. When you're ready, push the changes up to the origin. ```shell git push origin newfeature ``` Now check out how to create [Pull Requests](/best-practices/github/github-pull-requests)! --- ## GitHub Pull Requests ## Submitting a Pull Request Prior to submitting your pull request, you might want to do a few things to clean up your branch and make it as simple as possible for the original repo's maintainer to test, accept, and merge your work. If any commits have been made to the upstream master branch, you should rebase your development branch so that merging it will be a simple fast-forward that won't require any conflict resolution work. ``` # Fetch upstream master and merge with your repo's master branch git pull origin master --rebase ``` Follow the prompts to correct any code conflicts. Any file that is conflicted needs to be manually reviewed. After you fix the problems run: ``` git add filename git rebase --continue ``` Once that is happy, push the rebased changes back to the origin. ``` git push origin newfeature -f ``` Then follow these instructions once you're ready: https://help.github.com/articles/creating-a-pull-request/ ## Pull Request Template Use the following markdown template to describe the Pull Request. ``` ## what * ...high-level explanation of what this PR accomplishes... ## why * ...business justifications for making the changes... ## references * ...related pull requests, issues, documents, or research... ``` **Pro Tip:** Use a `.github/pull_request_template.md` file to automatically populate this template when creating new Pull Requests. :::info https://help.github.com/articles/creating-a-pull-request-template-for-your-repository/ ::: --- ## GitHub Best Practices ## Use `.gitignore` Use a `.gitignore` file in the root of every repo to exclude files that should never be committed. Here's an example of the [`.gitignore`](https://github.com/cloudposse/docs/blob/master/.gitignore) from our documentation repository. ```txt title=".gitignore example" .DS_Store .envrc .env deploy.toml deploy.yaml test test.toml test.yaml .htmltest.*.yaml node_modules .build-harness build-harness/ public/* algolia/* tmp/* .gitkeep *.swp .idea *.iml package-lock.json static/components/* static/styleguide/* themes/cloudposse/static/css/* themes/cloudposse/static/js/* static/webfonts/* static/css/* static/js/* ``` --- ## Terraform Best Practices These are the *opinionated* best-practices we follow at Cloud Posse. They are inspired by years of experience writing terraform and borrow on the many other helpful resources like those by [HashiCorp](https://www.terraform.io/docs/cloud/guides/recommended-practices/index.html). See our general [Best Practices](/best-practices/) which also apply to Terraform. ## Variables ### Use upstream module or provider variable names where applicable When writing a module that accepts `variable` inputs, make sure to use the same names as the upstream to avoid confusion and ambiguity. ### Use all lower-case with underscores as separators Avoid introducing any other syntaxes commonly found in other languages such as CamelCase or pascalCase. For consistency we want all variables to look uniform. This is also inline with the [HashiCorp naming conventions](https://www.terraform.io/docs/extend/best-practices/naming.html). ### Use positive variable names to avoid double negatives All `variable` inputs that enable/disable a setting should be formatted `...._enabled` (e.g. `encryption_enabled`). It is acceptable for default values to be either `false` or `true`. ### Use description field for all inputs All `variable` inputs need a `description` field. When the field is provided by an upstream provider (e.g. `terraform-aws-provider`), use same wording as the upstream docs. ### Use sane defaults where applicable Modules should be as turnkey as possible. The `default` value should ensure the most secure configuration (E.g. with encryption enabled). ### Use `nullable = false` where appropriate When passing an argument to a resource, passing `null` means to use the default value. Prior to Terraform version 1.1.0, passing `null` to a module input set that value to `null` rather than to the default value. Starting with Terraform version 1.1.0, variables can be declared as `nullable = false` which: 1. Prevents the variable from being set to `null`. 2. Causes the variable to be set to the default value if `null` is passed in. You should always use `nullable = false` for all variables which should never be set to `null`. This is particularly important for lists, maps, and objects, which, if not required, should default to empty values (i. e. `{}` or `[]`) rather than `null`. It can also be useful to set strings to default to `""` rather than `null` and set `nullable = false`. This will simplify the code since it can count on the variable having a non-null value. The default `nullable = true` never needs to be explicitly set. Leave variables with the default `nullable = true` if a null value is acceptable. ### Use feature flags, list, or map inputs for optional functionality All Cloud Posse modules should respect the [null-label](https://github.com/cloudposse/terraform-null-label) `enabled` feature flag, and when `enabled` is `false`, create no resources and generate null outputs (or, in the case of output lists, maps, and objects, empty values may be acceptable to avoid having other modules consuming the outputs fail due to having a null rather than empty value). Optional functionality should be toggled in either of 2 ways: 1. Use of a feature flag. Specifically, an input variable of type `bool` with a name ending in `_enabled`. Use this mechanism if the option requires no further configuration, e.g. `iam_role_enabled` or `s3_bucket_enabled`. Feature flags should always be `nullable = false`, but the default value can be `true` or `false` as desired. 2. If an optional feature requires further configuration, use a `list` or `map` input variable, with an empty input disabling the option and non-empty input providing configuration. In this case, only use a separate feature flag if the list or map input may still cause problems due to relying on computed problems during the plan phase. See [Count vs. For Each](/learn/component-development/terraform-in-depth/terraform-count-vs-for-each) and [Terraform Errors When Planning](/learn/component-development/terraform-in-depth/terraform-unknown-at-plan-time) for more information. 3. It is never acceptable for an optional feature of the _module_ to be toggled by the value of a string or number input variable, due to the issues explained in [Terraform Errors When Planning](/learn/component-development/terraform-in-depth/terraform-unknown-at-plan-time). However, if an optional feature of a _resource_ may be toggled by such an input if that is the behavior of the resource and the input has the same name as the resource argument. ### Use objects with optional fields for complex inputs When a module requires a complex input, use an object with optional fields. This provides documentation and plan-time validation while avoiding type conversion errors, and allows for future expansion without breaking changes. Make as many fields as possible optional, provide defaults at every level of nesting, and use `nullable = false` if possible. :::caution Extra (or Misspelled) Fields in Object Inputs Will Be Silently Ignored If you use an object with defaults as an input, Terraform will not give any indication if the user provides extra fields in the input object. This is particularly a problem if they misspelled an optional field name, because the misspelled field will be silently ignored, and the default value the user intended to override will silently be used. This is [a limitation of Terraform](https://github.com/hashicorp/terraform/issues/29204#issuecomment-1989579801). Furthermore, there is no way to add any checks for this situation, because the input will have already been transformed (unexpected fields removed) by the time any validation code runs. This makes using an object a trade-off versus using separate inputs, which do not have this problem, or `type = any` which allows you to write validation code to catch this problem and additional code to supply defaults for missing fields. ::: Reserve `type = any` for exceptional cases where the input is highly variable and/or complex, and the module is designed to handle it. For example, the configuration of a [Datadog synthetic test](https://registry.terraform.io/providers/DataDog/datadog/latest/docs/reference/synthetics_test) is both highly complex and [the Cloud Posse module](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/synthetics) accepts both an object derived from the `synthetics_test` resource schema or an object derived from the JSON output of the Datadog API. In this rare case, attempting to maintain a type definition would not only be overly complex, it would slow the adoption of future additions to the interface, and so `type = any` is appropriate. ### Prefer a single object over multiple simple inputs for related configuration When reviewing Cloud Posse modules as examples, you may notice that they often use a large number of input variables of simple types. This is because in the early development of Terraform, there was no good way to define complex objects with defaults. However, now that Terraform supports complex objects with field-level defaults, we recommend using a single object input variable with such defaults to group related configuration, taking into consideration the trade-offs listed in the [above caution](#use-objects-with-optional-fields-for-complex-inputs). This makes the interface easier to understand and use. For example, prefer: ```hcl variable "eip_timeouts" { type = object({ create = optional(string) update = optional(string) delete = optional(string, "30m") })) default = {} nullable = false } ``` rather than: ```hcl variable "eip_create_timeout" { type = string default = null } variable "eip_update_timeout" { type = string default = null } variable "eip_delete_timeout" { type = string default = "30m" } ``` However, using an object with defaults versus multiple simple inputs is not without trade-offs, as explained in the [above caution](#use-objects-with-optional-fields-for-complex-inputs). There are a few ways to mitigate this problem besides using separate inputs: - If all the defaults are null or empty, you can use a `map(string)` input variable and use the `keys` function to check for unexpected fields. This catches errors, but has the drawback that it does not provide documentation of what fields are expected. - You can use `type = any` for inputs, but then you have to write the extra code to validate the input and supply defaults for missing fields. You should also document the expected fields in the input description. - If all you are worried about is misspelled field names, you can make the correctly spelled field names required, ensuring they are supplied. Alternatively, if the misspelling is predictable, such as you have a field named `minsize` but people are likely to try to supply `min_size`, you can make the misspelled field name optional with a sentinel value and then check for that value in the validation code. ### Use custom validators to enforce custom constraints Use the `validation` block to enforce custom constraints on input variables. A custom constraint is one that, if violated, would **_not_** otherwise cause an error, but would cause the module to behave in an unexpected way. For example, if the module takes an optional IPv6 CIDR block, you might receive that in a variable of `list(string)` for reasons explained [here](#use-feature-flags-list-or-map-inputs-for-optional-functionality). Use a custom validator to ensure that the list has at most one element, because if it has more than one, the module will ignore the extra elements, while a reasonable person might expect them to be used in some way. At the same time, is it not necessary to use a custom validator to enforce that the input is a valid IPv6 CIDR block, because Terraform will already do that for you. This is perhaps better illustrated by an example of a pseudo-enumeration. Terraform does not support real enumerations, so they are typically implemented as strings with a limited number of acceptable values. For example, Say you have a resource that takes a `frequency` input that is a string that must be either `DAILY` or `WEEKLY`. Even though the value of the string is very restricted, you should not use a custom validator to enforce that for two reasons. 1. Terraform (technically, the Terraform resource provider) will already enforce that the string is one of those two values and produce an informative error message if it is not. 2. If you use a custom validator to enforce that the string is one of those two values, and then a later version of the resource adds a new option to the enumeration, such as "HOURLY", the custom validator will prevent the module from using the new value, even though the module would function perfectly were it not for the validator. This adds work and delay to the adoption of underlying enhancements to the resource, without providing enough benefit to be worth the extra effort. ### Use variables for all secrets with no `default` value and mark them "sensitive" All `variable` inputs for secrets must never define a `default` value. This ensures that `terraform` is able to validate user input. The exception to this is if the secret is optional and will be generated for the user automatically when left `null` or `""` (empty). Use `sensitive = true` to mark all secret variables as sensitive. This ensures that the value is not printed to the console. ## Outputs ### Use description field for all outputs All outputs must have a `description` set. The `description` should be based on (or adapted from) the upstream terraform provider where applicable. Avoid simply repeating the variable name as the output `description`. ### Use well-formatted snake case output names Avoid introducing any other syntaxes commonly found in other languages such as CamelCase or pascalCase. For consistency, we want all variables to look uniform. It also makes code more consistent when using outputs together with terraform [`remote_state`](https://www.terraform.io/docs/providers/terraform/d/remote_state.html) to access those settings from across modules. ### Never output secrets Secrets should never be outputs of modules. Rather, they should be written to secure storage such as AWS Secrets Manager, AWS SSM Parameter Store with KMS encryption, or S3 with KMS encryption at rest. Our preferred mechanism on AWS is using SSM Parameter Store. Values written to SSM are easily retrieved by other terraform modules, or even on the command-line using tools like [chamber](https://github.com/segmentio/chamber) by Segment.io. We are very strict about this in our components (a.k.a root modules), the top-most module, because these sensitive outputs are easily leaked in CI/CD pipelines (see [`tfmask`](https://github.com/cloudposse/tfmask) for masking secrets in output only as a last resort). We are less sensitive to this in modules that are typically nested inside of other modules. Rather than outputting a secret, you may output plain text indicating where the secret is stored, for example `RDS master password is in SSM parameter /rds/master_password`. You may also want to have another output just for the key for the secret in the secret store, so the key is available to other programs which may be able to retrieve the value given the key. :::warning Regardless of whether the secret is output or not, the fact that a secret is known to Terraform means that its value is stored in plaintext in the Terraform state file. Storing values in SSM Parameter Store or other places does not solve this problem. Even if you store the value in a secure place using some method other than Terraform, if you read it into Terraform, it will be stored in plaintext in the state file. To keep the secret out of the state file, you must both store and retrieve the secret outside of Terraform. This is a limitation of Terraform that [has been discussed](https://github.com/hashicorp/terraform/issues/516) practically since Terraform's inception, so a near-term solution is unlikely. ::: ### Use symmetrical names We prefer to keep terraform outputs symmetrical as much as possible with the upstream resource or module, with exception of prefixes. This reduces the amount of entropy in the code or possible ambiguity, while increasing consistency. Below is an example of what **not* to do. The expected output name is `user_secret_access_key`. This is because the other IAM user outputs in the upstream module are prefixed with `user_`, and then we should borrow the upstream's output name of `secret_access_key` to become `user_secret_access_key` for consistency. ![Terraform outputs should be symmetrical](/assets/terraform-outputs-should-be-symmetrical.png) ### Export all Attributes of a Resource While it's important to explicitly output the most relevant attributes from a resource, there are cases where exposing the entire resource object is beneficial. By outputting the full resource, you provide downstream modules and users with greater flexibility and easier interoperability—especially when composing infrastructure across multiple modules. This approach also future-proofs your module. As providers evolve and introduce new attributes, consumers can immediately access those attributes without requiring the module to be updated with new outputs. ```hcl output "_" { value = . description = "All attributes of [`.`](link-to-aws-provider-resource-attribute-docs)" } ``` For example, if the module contained the resource address `aws_ec2_instance.default` then the output would be defined like this. ```hcl output "aws_ec2_instance_default" { value = aws_ec2_instance.default description = "All attributes of [`aws_ec2_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance#attribute-reference)" } ``` ## Language ### Use indented `HEREDOC` syntax Using `<<-EOT` (as opposed to `<0.12.26` or `>= 0.13, < 0.15`. Always use `>=` and enforce Terraform versions in your environment by controlling which CLI you use. ### Use encrypted S3 bucket with versioning, encryption and strict IAM policies We recommend not commingling state in the same bucket as other data. This could cause the state to get overridden or compromised. Note, the state contains cached values of all outputs. Consider isolating sensitive areas like production configuration and audit trails (separate buckets, separate organizations). **Pro Tip:** Using the [`terraform-aws-tfstate-backend`](https://github.com/cloudposse/terraform-aws-tfstate-backend) to easily provision buckets for each stage. ### Use Versioning on State Bucket ### Use Encryption at Rest on State Bucket ### Use `.gitignore` to exclude terraform state files, state directory backups and core dumps ``` .terraform .terraform.tfstate.lock.info *.tfstate *.tfstate.backup ``` ### Use `.dockerigore` to exclude terraform statefiles from builds Example: ``` **/.terraform* ``` ### Use a programmatically consistent naming convention All resource names (E.g. things provisioned on AWS) must follow a consistent convention. The reason this is so important is that modules are frequently composed inside of other modules. Enforcing consistency increases the likelihood that modules can invoke other modules without colliding on resource names. To enforce consistency, we require that all modules use the [`terraform-null-label`](https://github.com/cloudposse/terraform-null-label) module. With this module, users have the ability to change the way resource names are generated such as by changing the order of parameters or the delimiter. While the module is opinionated on the parameters, it's proved invaluable as a mechanism for generating consistent resource names. ## Module Design ### Small Opinionated Modules We believe that modules should do one thing very well. But in order to do that, it requires being opinionated on the design. Simply wrapping terraform resources for the purposes of modularizing code is not that helpful. Implementing a specific use-case of those resource is more helpful. ### Composable Modules Write all modules to be easily composable into other modules. This is how we're able to achieve economies of scale and stop re-inventing the same patterns over and over again. ## Module Usage ### Use Terraform registry format with exact version numbers There are many ways to express a module's source. Our convention is to use Terraform registry syntax with an explicit version. ``` source = "cloudposse/label/null" version = "0.25.0" ``` The reason to pin to an explicit version rather than a range like `>= 0.25. 0` is that any update is capable of breaking something. Any changes to your infrastructure should be implemented and reviewed under your control, not blindly automatic based on when you deployed it. :::info Prior to Terraform v0.13, our convention was to use the pure git url: ```hcl source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.16.0" ``` Note that the `ref` always points explicitly to a `tags` pinned to a specific version. Dropping the `tags/` qualifier means it could be a branch or a tag; we prefer to be explicit. ::: --- ## Code of Conduct import Intro from '@site/src/components/Intro'; In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation. ## Our Standards Examples of behavior that contributes to creating a positive environment include: - Using welcoming and inclusive language - Being respectful of differing viewpoints and experiences - Gracefully accepting constructive criticism - Focusing on what is best for the community - Showing empathy towards other community members Examples of unacceptable behavior by participants include: - The use of sexualized language or imagery and unwelcome sexual attention or advances - Trolling, insulting/derogatory comments, and personal or political attacks - Public or private harassment - Publishing others' private information, such as a physical or electronic address, without explicit permission - Other conduct which could reasonably be considered inappropriate in a professional setting ## Our Responsibilities Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior. Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful. ## Scope This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers. ## Enforcement Instances of abusive, harassing or otherwise, unacceptable behavior may be reported by contacting the project team at hello@cloudposse.com. The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately. Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership. ## Attribution This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, available at [http://contributor-covenant.org/version/1/4][version] [homepage]: http://contributor-covenant.org [version]: http://contributor-covenant.org/version/1/4/ --- ## Need help? Join our community! import DocCardList from '@theme/DocCardList' import Intro from '@site/src/components/Intro' import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Note from '@site/src/components/Note' import PrimaryCTA from '@site/src/components/PrimaryCTA' Cloud Posse has a great community of active users who are more than willing to help each other. So, join us! ## Get Started today ## Sign up for our SweetOps Slack Join our free SweetOps Slack community, hosted by Cloud Posse with more than 9,500 members worldwide. While we primarily address questions in channels like `#atmos`, `#aws`, `#cloudposse`, `#geodesic`, `#github-actions`, and `#terraform`, feel free to engage in any channel. Sign up here: [https://slack.sweetops.com](https://slack.sweetops.com). Archives are searchable at [archive.sweetops.com](https://archive.sweetops.com). ## Join our GitHub Discussions [Cloud Posse GitHub Discussions](https://github.com/orgs/cloudposse/discussions) is a great place to ask questions, share ideas, and get feedback from the community. We have a dedicated section for support and questions related to the reference architecture. Please search through existing discussions before creating a new one. If you do not find an answer, please feel free to create a new discussion. Essential Support guarantees priority responses, so you always have a direct line to Cloud Posse’s expertise. Learn more about our support offerings at [https://cloudposse.com/support](https://cloudposse.com/support). ## Attend Weekly Office Hours [Sign up](https://cloudposse.com/office-hours/) for our free public “Office Hours” are held every Wednesday at 11:30am PT (14:30 ET) via Zoom. Past recordings are available on our [YouTube channel](https://youtube.com/cloudposse). These public calls are hosted weekly by Cloud Posse. They are a good way to keep up with the latest developments and trends in our DevOps community. Sign Up for Office Hours ## Are you more into email? Try our Newsletter Sign up for [Cloud Posse's Weekly Newsletter](https://newsletter.cloudposse.com) to get the latest news about things happening in our community and other news about building Open Source infrastructure—straight into your inbox. ## Found a Bug or Issue? Please report it in [our issue tracker](https://github.com/cloudposse/docs/issues) --- --- ## Contact Us import Intro from '@site/src/components/Intro'; We'd love to hear from you! Reach out using any of the options below in the medium most convenient for you. Here's how to get in touch with us:
Email
hello@cloudposse.com
Website
cloudposse.com
GitHub
github.com/cloudposse
Schedule Time
cloudposse.com/meet
Slack
Newsletter
cloudposse.com/newsletter
LinkedIn
linkedin.com/company/cloudposse
## Partnership Opportunities Cloud Posse welcomes all partnership inquiries, including partnerships with other DevOps practitioners, freelancers and consultancies who want to leverage our methodologies with their customers. Please drop us a line at [hello@cloudposse.com](mailto:hello@cloudposse.com). --- ## Terraform Automated Testing import Intro from '@site/src/components/Intro'; import Step from '@site/src/components/Step'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; Cloud Posse's Terraform modules use a comprehensive automated testing strategy that combines static code analysis and integration tests. Our testing approach ensures code quality through automated linting and formatting checks, while integration tests validate that modules work correctly in real-world scenarios. Tests can be run locally during development and are automatically triggered through our CI/CD pipeline. All of our Terraform modules have automated tests. We have two sets of checks: ### Static Code Analysis The first set of checks is executed through the feature-branch workflow, which can be found [here](https://github.com/cloudposse/github-actions-workflows-terraform-module/blob/main/.github/workflows/feature-branch.yml) This workflow generates some documentation and performs basic sanity checks, including linting and formatting. These checks are lightweight and can be executed without requiring any special permissions. Consequently, they *are automatically run* on every commit. Before committing and pushing your changes, you can and should run this set of checks locally by executing the following command on your host ``` pre-commit run --all-files ``` Running these checks locally incorporates all the required changes that otherwise would block your PR. ### Integration Tests The second set of checks consists of Terraform integration tests that validate the functionality and integration of the module. These tests are performed using the [`terratest`](https://github.com/gruntwork-io/terratest) library, specifically designed for infrastructure testing, and do more in-depth integration tests of module functionality. Unlike the first set of checks, these integration tests are *executed only on request*, and only by authorized contributors. We use ChatOps to trigger this workflow. ## Philosophy of Terraform Integration Testing At a minimum, we ensure that all of our modules cleanly `plan`, `apply` and `destroy`. This catches 80% of the problems with only 20% of the effort. We also test than then the `enabled` input is set to `false`, no resources are created. Ideally we would like to test that the resources are properly created, but often this is difficult to verify programmatically, in which case we settle for spot checking that the dynamic outputs match our expectations. At the same time, we do not want to waste effort retesting what has already been tested by HashiCorp and their providers. For example, we have our [`terraform-aws-s3-bucket`](https://github.com/cloudposse/terraform-aws-s3-bucket) module that creates an S3 bucket. We don't need to test that a bucket is created; we assume that would be caught by the upstream terraform tests. But we do want to [test that the bucket name](https://github.com/cloudposse/terraform-aws-s3-bucket/blob/master/test/src/examples_complete_test.go#L38) is what we expect it to be, since this is something under our control. ## Using ChatOps To Trigger Integration Tests In addition to automatic triggers, tests can be run on demand via "ChatOps". (You will need to have at least `triage` level of access to a repo to use ChatOps commands.) Typically, tests are run by a Cloud Posse contributor or team member as part of a PR review. Tests are initiated by posting GitHub comments on the PR. Currently supported commands are the following: | Command | Description | | ------------ | --------------------------------------------------- | | `/terratest` | Run the `terratest` integration tests in `test/src` | Terraform tests run against our [testing infrastructure](https://github.com/cloudposse/testing.cloudposse.co) that we host in an isolated account on AWS, strictly for the purposes of testing infrastructure. ChatOps is powered by [GitHub Actions](https://github.com/features/actions) and the [slash-dispatch-command](https://github.com/peter-evans/slash-command-dispatch). The terratest workflow is defined in the [`cloudposse/actions`](https://github.com/cloudposse/actions/blob/master/.github/workflows/terratest-command.yml) repository. The benefit with this is that we have one place to control the testing workflow for all of our hundreds of terraform modules. The downside, however, with dispatched workflows is that the _workflows_ always run from the `main` branch. ## Manually triggering a shared workflow Here's a list a workflows you might want to trigger manually should things go wrong on GitHub side or with our configuration. - `feature-branch` can be triggered anytime by labeling/unlabeling PR with any label. - `release-branch` is the same to creating a GH release manually. We have created a complimentary workflow `release-published` for this case: it will fulfill the missing parts once you create a release manually. Note that you are skipping tests before release in this case. - `scheduled` can be triggered anytime from GitHub UI, it has a *workflow_dispatch* trigger for this purpose. ## Running Terraform Tests locally We use [Atmos](https://atmos.tools) to streamline how Terraform tests are run. It centralizes configuration and wraps common test workflows with easy-to-use commands. All tests are located in the `test/` folder. Under the hood, tests are powered by Terratest together with our internal [Test Helpers](https://github.com/cloudposse/test-helpers) library, providing robust infrastructure validation. Setup dependencies: - Install Atmos ([installation guide](https://atmos.tools/install/)) - Install Go [1.24+ or newer](https://go.dev/doc/install) - Install Terraform or OpenTofu To run tests: - Run all tests: ```sh atmos test run ``` - Clean up test artifacts: ```sh atmos test clean ``` - Explore additional test options: ```sh atmos test --help ``` The configuration for test commands is centrally managed. To review what's being imported, see the [`atmos.yaml`](https://raw.githubusercontent.com/cloudposse/.github/refs/heads/main/.github/atmos/terraform-module.yaml) file. Learn more about implementing [custom commands](https://atmos.tools/core-concepts/custom-commands/) with atmos. ## ChatOps Configuration If you're a contributor who wants to initialize one of our terraform modules, this is the process. Note, if a repo has already been configured for ChatOps, there's no need to proceed with these steps. To initialize one of our modules with ChatOps, run the following commands: 1. Install Atmos ([installation guide](https://atmos.tools/install/)) 1. `git clone` the terraform module repository 1. `cd $repo` to enter the repository directory 1. `git add *` to add the changes 1. Add the build badge to the `README.yaml` under the `badges` section. 1. `atmos docs generate readme` to rebuild the `README.md` (remember, never edit the `README.md` manually since it's generated from the `README.yaml`) 1. Open up a Pull Request with the changes. Here is a [good example](https://github.com/cloudposse/atmos/pull/555). 1. Request a Code Review in the [`#pr-reviews`](https://slack.cloudposse.com) Slack channel (and *big* thanks for your contribution!) --- ## Code Review Guidelines import Steps from '@site/src/components/Steps'; Here are some of our tips for conducting *Code Reviews* the SweetOps way. If you haven't already, become familiar with our [Best Practices](/best-practices) and [Terraform Best Practices](/best-practices/terraform). 1. Use the ["Suggest"](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests/incorporating-feedback-in-your-pull-request) feature as much as possible. This makes it quick and easy for the contributor to accept or dismiss the recommendations. 1. Use proper markdown in suggestions (e.g. code blocks) 1. Always be polite and appreciative of the contributions! 1. Use emoticons to up-vote other comments (rather than `+1` comments) 1. Use ChatOps command `/terratest` to run integration tests 1. Recommend changes to better conform to our best-practices 1. Quote comments you're replying to make your responses more clear ### Specifics for Terraform Modules We use automated testing to enforce certain standards for our Terraform modules. Currently these are run via GitHub Actions, and you can look at the logs of failing tests by clicking the `Details` link in the PR status list. Here is a partial list of rules that are enforced: - All modules referenced must be pinned to an exact, numbered version. Cannot be `master` or a range like `>= 0.9.0` - All providers must have version pinning of the form `>=` (can be `>= x.x` or `>= x.x.x`). More restrictive pinning is not allowed. - All modules that no longer support Terraform versions older than 0.12.26 must be upgraded to refer to providers using Terraform Registry format (explicit `source` field). - All modules must have their `README` updated to the current standard. **Note:** `README.md` is generated by tooling from `README.yaml`. Anything you want to update in the `README` must be updated in `README.yaml` or it will simply be overwritten. Updating the `README` usually requires nothing more than regenerating it. - All modules must comply exactly with Terraform formatting standards used by `terraform fmt` We have tooling to help with some of this. Before opening a PR, but after making all your changes, run ``` make pr/auto-format ``` in the root directory of the repository. That will format your Terraform code and rebuild the README. (If you have done that and the tests still complain about a bad `README`, it is possible you have cached an old version of the builder Docker image. Try updating it with `make init && make builder/pull` and run `make pr/auto-format` again.) --- ## Component Testing import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import CodeBlock from '@theme/CodeBlock'; import CollapsibleText from '@site/src/components/CollapsibleText'; import PillBox from '@site/src/components/PillBox'; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; import Step3VendorYaml from '@site/examples/component-testing/step-3/vendor.yaml'; import Step3VpcYaml from '@site/examples/component-testing/step-3/catalog-vpc.yaml'; import Step3Stack from '@site/examples/component-testing/step-3/stack.yaml'; import Step4UsecaseYaml from '@site/examples/component-testing/step-4/usecase.yaml'; import Step4Stack from '@site/examples/component-testing/step-4/stack.yaml'; import Step7UsecaseYaml from '@site/examples/component-testing/step-7/usecase.yaml'; import Step7Stack from '@site/examples/component-testing/step-7/stack.yaml'; import Intro from '@site/src/components/Intro'; This documentation will guide you through our comprehensive strategy for testing Terraform components, provide step-by-step instructions and practical examples to help you validate your component configurations effectively. Whether you're setting up initial tests, adding dependencies, or verifying output assertions, you'll find the resources you need to ensure robust and reliable deployments. ## Context Our component testing strategy is a direct outcome of [our migration to a dedicated GitHub Organization for components](/components/#terraform-component-github-repository-has-moved). This separation allows each component to live in its own repository, enabling independent versioning and testing. It not only improves the reliability of each component but also empowers the community to contribute via pull requests confidently. With integrated testing for every PR, we can ensure high quality and build trust in each contribution. For more information on building and maintaining components, please refer to our [Component Development Guide](/learn/component-development/), which provides detailed insights into best practices, design principles, and the overall process of component development. ## Prerequisites 1. Install Terraform / Tofu - Ensure you have [Terraform](https://www.terraform.io/downloads.html) or [OpenTofu](https://opentofu.org/docs/intro/install/) installed on your machine. 1. Install Atmos - [Atmos](https://atmos.tools/install/) is a tool for managing Terraform environments. 1. Install Golang - Go is a programming language that you'll need to run the tests. - Download and install Go from the [official Go website](https://golang.org/dl/). - Make sure to set up your Go environment correctly by following the [Getting Started with Go](https://golang.org/doc/install/source) guide. 1. Authenticate on AWS - Ensure you have the necessary AWS credentials configured on your machine. You can do this by setting up the AWS CLI and running `aws configure`, where you'll input your AWS Access Key, Secret Key, region, and output format. - Refer to the [AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html) for more details. ## Test Framework Component testing framework assumes that each component's repo structure follows the convention when all component terraform source code would be stored in `src` directory and everything related to tests will be placed in `test` directory. Tests consists of two coupled parts - atmos configuration fixtures and tests written on Go code. Repo structure should be simular to this one: ```console component-root/ ├── src/ # Component source directory │ └── main.tf └── test/ # Tests directory ├── fixtures/ # Atmos configurations ├── component_test.go # Tests ├── go.mod └── go.sum ``` ### Atmos configuration fixtures Atmos configuration fixtures provides minimal settings to deploy the component and it's dependencies on test account during test run. The difference with a regular atmos configuration are: 1. All components deployed on one stack `default-test` in one `us-east-2` region. 2. Use single aws account for all test resources. If component assumes the cross region or cross account interaction, the configuration still deploys it to the same actual aws account. 3. Mock `account-map` component to skip role assuming and always use current AWS credentials provided with environment variables 4. Configure terraform state files storage to local directory at a path provided by test framework with environment variable `COMPONENT_HELPER_STATE_DIR` This configuration is common for all components and could be copied from [template repo](https://github.com/cloudposse-terraform-components/template/tree/main/test). Fixtures directory structure looks like ```console fixtures/ ├── stacks/ | ├── catalog/ | | ├── usecase/ | | | ├── basic.yaml | | | └── disabled.yaml | | └── account-map.yaml │ └── orgs/default/test/ | ├── _defaults.yaml | └── tests.yaml ├── atmos.yaml └── vendor.yaml ``` For most components, avoid any changes to these files 1. `atmos.yaml` - shared atmos config common for all test cases 2. `stacks/catalog/account-map.yaml` - Mock `account-map` configuration makes any environment/stack/tenant to be backed with the single AWS test account 3. `stacks/orgs/default/test/_defaults.yaml` - Configure terraform state backend to local directory and define shared variables for `default-test` This files and directories contains custom configurations specific for a testing component: 1. `vendor.yaml` - Vendor configuration for all component dependencies 2. `stacks/catalog/` - Store all dependencies configuration files in the dir 3. `stacks/catalog/usecases` - Store configuration of the testing component's use cases 4. `stacks/catalog/usecases/basic.yaml` - Predefined file for basic configuration of the testing component's use case 5. `stacks/catalog/usecases/disabled.yaml` - Predefined file for the `disabled` configuration use case (when variable `enabled: false`) 6. `stacks/orgs/default/test/tests.yaml` - Include all dependencies and use cases configurations to deploy them for `default-test` stack ### Tests (Golang) Component tests are written on go lang as this general purpose language is standard defacto for cloud compute engineering Under the hood tests uses several libraries with helper functions 1. `github.com/cloudposse/test-helpers/atmos/component-helper` - Component testing framework provides 2. `github.com/cloudposse/test-helpers/atmos` - Atmos API 3. `github.com/cloudposse/test-helpers/aws` - Test helpers interact with AWS 4. `github.com/cloudposse/terratest/aws` - Test helpers provided by GruntWork 5. `github.com/aws/aws-sdk-go-v2` - AWS API You can specify any additional dependency libraries by running `go get {library name}`. Test framework extends `github.com/stretchr/testify/suite` to organize test suites. Regular test file structure follow this example: ```go title="test/component_test.go" package test import ( "context" "testing" "fmt" "strings" helper "github.com/cloudposse/test-helpers/pkg/atmos/component-helper" awsHelper "github.com/cloudposse/test-helpers/pkg/aws" "github.com/cloudposse/test-helpers/pkg/atmos" "github.com/gruntwork-io/terratest/modules/aws" "github.com/stretchr/testify/assert" metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" ) type ComponentSuite struct { helper.TestSuite } // Functions Test prefix are entrypoint for `go test` func TestRunSuite(t *testing.T) { # Define test suite instance suite := new(ComponentSuite) // Add dependency to the dependencies queue suite.AddDependency(t, "vpc", "default-test", nil) // Run test suite helper.Run(t, suite) } // Test suite methods prefixed with `Test` are tests // Test basic usecase func (s *ComponentSuite) TestBasic() { const component = "example/basic" const stack = "default-test" const awsRegion = "us-east-2" // Destroy test component defer s.DestroyAtmosComponent(s.T(), component, stack, nil) // Deploy test component options, _ := s.DeployAtmosComponent(s.T(), component, stack, nil) assert.NotNil(s.T(), options) // Get test component output id := atmos.Output(s.T(), options, "eks_cluster_id") assert.True(s.T(), strings.HasPrefix(id, "eg-default-ue2-test-")) // Test component drift s.DriftTest(component, stack, nil) } // Test disabled use case func (s *ComponentSuite) TestEnabledFlag() { const component = "example/disabled" const stack = "default-test" // Verify no resources created when `enabled: false` s.VerifyEnabledFlag(component, stack, nil) } ``` ### CLI Flags Cheat Sheet A test suite run consists of the following phases all of which can be controlled by passing flags: | Phase | Description | Skip flag | |------------|-------------------------------------------|----------------------------| | Setup | Setup test suite and deploy dependencies |`--skip-setup` | | Test | Deploy the component |`--only-deploy-dependencies`| | Teardown | Destroy all dependencies |`--skip-teardown` | This is possible to enable/disable steps on each phase more precisely | Phase | Description | Skip flag | |------------|--------------------------------------------------------|-----------------------------| | Setup | Vendor dependencies |`--skip-vendor` | | Setup | Deploy component dependencies |`--skip-deploy-dependencies` | | Test | Deploy the component |`--skip-deploy-component` | | Test | Perform assertions | | | Test | Destroy the deployed component (on defer) |`--skip-destroy-component` | | Teardown | Destroy all dependencies |`--skip-destroy-dependencies`| Here is the useful combination of flags. | Command | Description | |-----------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------| |`go test -timeout 1h --only-deploy-dependencies --skip-destroy-dependencies` | Deploy dependencies only. | |`go test -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-destroy-component` | Deploy testing component. Use previously deployed dependencies. Do not destroy anything. Useful when you are working on deploying use case | |`go test -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-deploy-component --skip-destroy-component`| Do not deploy or destroy anything. Useful when you are working on tests asserts | |`go test -timeout 1h --skip-deploy-dependencies --skip-deploy-component` | Destroy component and its dependencies. Useful when your tests are done to clean up all resources | [Read more about the test helpers framework](https://github.com/cloudposse/test-helpers/blob/main/pkg/atmos/component-helper/README.md) ## Write Tests Writing tests for your Terraform components is essential for building trust in the component's reliability and enabling safe acceptance of community contributions. By implementing comprehensive tests, we can confidently review and merge pull requests while ensuring the component continues to function as expected. ### Copy the test scaffold files If you missed the test scaffold files, copy the contents from [this GitHub repository](https://github.com/cloudposse-terraform-components/template/tree/main/test) into your component repository. This will provide you with the necessary structure and example tests to get started. The repo structure should looks like the following: ```console ├── src/ │ └── main.tf └── test/ ├── fixtures/ │ ├── stacks/ │ | ├── catalog/ │ | | ├── usecase/ │ | | | ├── basic.yaml │ | | | └── disabled.yaml │ | | └── account-map.yaml │ │ └── orgs/default/test/ │ | ├── _defaults.yaml │ | └── tests.yaml │ ├── atmos.yaml │ └── vendor.yaml ├── component_test.go ├── go.mod └── go.sum ``` ### Run Initial Tests Navigate to the `test` directory and run tests in your terminal by running ```console cd test go test -v -timeout 1h --only-deploy-dependencies ``` ➜ test git:(main) go test -v -timeout 1h --only-deploy-dependencies === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 14:13:42 INFO TestRunSuite: deploy dependencies → started 2025/03/07 14:13:42 INFO no dependencies to deploy 2025/03/07 14:13:42 INFO TestRunSuite: deploy dependencies → completed 2025/03/07 14:13:42 INFO TestRunSuite: setup → completed 2025/03/07 14:13:42 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (8.28s) PASS ok test 9.142s ### Add Dependencies Identify any additional dependencies your component require. Skip this step if the component doesn't have any dependencies. 1. Add dependency to the vendor file {Step3VendorYaml} 1. Add atmos component configurations {Step3VpcYaml} 1. Import the dependent component for `default-test` stack {Step3Stack} 1. Add the dependent component to test suite with Go code - By default, the test suite will add a unique random value to the `attributes` terraform variable. - This is to avoid resource naming collisions with other tests that are using the same component. - But in some cases, you may need to pass unique value to specific input for the component. Check out the advanced example for the most common use-case with the `dns-delegated` domain name. ```go title="test/component_test.go" package test import ( "testing" helper "github.com/cloudposse/test-helpers/pkg/atmos/component-helper" ) type ComponentSuite struct { helper.TestSuite } func (s *ComponentSuite) TestBasic() { // Add empty test // Suite setup would not be executed without at least one test } func TestRunSuite(t *testing.T) { suite := new(ComponentSuite) // Deploy the dependent vpc component suite.AddDependency(t, "vpc", "default-test", nil) helper.Run(t, suite) } ``` ```go title="test/component_test.go" package test import ( "testing" helper "github.com/cloudposse/test-helpers/pkg/atmos/component-helper" ) type ComponentSuite struct { helper.TestSuite } func TestRunSuite(t *testing.T) { suite := new(ComponentSuite) subdomain := strings.ToLower(random.UniqueId()) inputs := map[string]interface{}{ "zone_config": []map[string]interface{}{ { "subdomain": subdomain, "zone_name": "components.cptest.test-automation.app", }, }, } suite.AddDependency(t, "dns-delegated", "default-test", &inputs) helper.Run(t, suite) } ``` 1. Deploy dependencies ```console go test -v -timeout 1h --only-deploy-dependencies --skip-destroy-dependencies ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → started 2025/03/07 17:38:24 INFO deploying dependency component=vpc stack=default-test TestRunSuite 2025-03-07T17:38:24+01:00 retry.go:91: atmos [terraform apply vpc -s default-test -input=false -auto-approve -var attributes=["rydpt4"] -no-color -lock=false] TestRunSuite 2025-03-07T17:38:24+01:00 logger.go:67: Running command atmos with args [terraform apply vpc -s default-test -input=false -auto-approve -var attributes=["rydpt4"] -no-color -lock=false] ... 2025/03/07 17:43:27 INFO TestRunSuite: deploy dependencies → completed 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed 2025/03/07 17:43:27 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (322.74s) PASS ok test 324.052s ### Add Test Use-Cases 1. Add atmos configuration for the component use case {Step4UsecaseYaml} 1. Import the use case for `default-test` stack {Step4Stack} 1. Write tests ```go title="test/component_test.go" package test import ( "testing" helper "github.com/cloudposse/test-helpers/pkg/atmos/component-helper" ) type ComponentSuite struct { helper.TestSuite } func TestRunSuite(t *testing.T) { suite := new(ComponentSuite) suite.AddDependency(t, "vpc", "default-test", nil) helper.Run(t, suite) } func (s *ComponentSuite) TestBasic() { const component = "example-component/basic" const stack = "default-test" const awsRegion = "us-east-2" // How to read outputs from the dependent component // vpcOptions, err := s.GetAtmosOptions("vpc", stack, nil) // id := atmos.Output(s.T(), vpcOptions, "id") inputs := map[string]interface{}{ // Add other inputs that are required for the use case } defer s.DestroyAtmosComponent(s.T(), component, stack, &inputs) options, _ := s.DeployAtmosComponent(s.T(), component, stack, &inputs) assert.NotNil(s.T(), options) } ``` 1. Deploy test component ```console go test -v -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-destroy-component --skip-teardown ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → skipped 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed ... 2025/03/07 17:43:27 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (322.74s) --- PASS: TestRunSuite/TestBasic (3.19s) PASS ok test 324.052s ### Add Assertions 1. Include assertions Within your test, include assertions to validate the expected outcomes. Use Go's testing package to assert conditions that must be true for the test to pass. This will help ensure that your component behaves as expected. ```go title="test/component_test.go" package test import ( "testing" "github.com/cloudposse/test-helpers/pkg/atmos" helper "github.com/cloudposse/test-helpers/pkg/atmos/component-helper" "github.com/stretchr/testify/assert" ) type ComponentSuite struct { helper.TestSuite } func TestRunSuite(t *testing.T) { suite := new(ComponentSuite) suite.AddDependency(t, "vpc", "default-test", nil) helper.Run(t, suite) } func (s *ComponentSuite) TestBasic() { const component = "example-component/basic" const stack = "default-test" const awsRegion = "us-east-2" // How to read outputs from the dependent component // vpcOptions, err := s.GetAtmosOptions("vpc", stack, nil) // id := atmos.Output(s.T(), vpcOptions, "id") inputs := map[string]interface{}{ // Add other inputs that are required for the use case } defer s.DestroyAtmosComponent(s.T(), component, stack, &inputs) options, _ := s.DeployAtmosComponent(s.T(), component, stack, &inputs) assert.NotNil(s.T(), options) // How to read string output from the component output1 := atmos.Output(s.T(), options, "output_name_1") assert.Equal(s.T(), "expected_value_1", output1) // How to read list of strings output from the component output2 := atmos.OutputList(s.T(), options, "output_name_2") assert.Equal(s.T(), "expected_value_2", output2[0]) assert.ElementsMatch(s.T(), ["expected_value_2"], output2) // How to read map of objects output from the component output3 := atmos.OutputMapOfObjects(s.T(), options, "output_name_3") assert.Equal(s.T(), "expected_value_3", output3["key"]) // How to read struct output from the component type outputStruct struct { keyName string `json:"key"` } output4 := outputStruct{} atmos.OutputStruct(s.T(), options, "output_name_4", &output4) assert.Equal(s.T(), "expected_value_4", output4["keyName"]) } ``` 1. Run test ```console go test -v -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-destroy-component --skip-teardown ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → skipped 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed ... 2025/03/07 17:43:27 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (322.74s) --- PASS: TestRunSuite/TestBasic (3.19s) PASS ok test 324.052s ### Add Drift Detection Test The drifting test ensures that the component is not change any resources on rerun with the same inputs. 1. Add a "drifting test" check ```go title="test/component_test.go" func (s *ComponentSuite) TestBasic() { const component = "example-component/basic" const stack = "default-test" const awsRegion = "us-east-2" inputs := map[string]interface{}{} defer s.DestroyAtmosComponent(s.T(), component, stack, &inputs) options, _ := s.DeployAtmosComponent(s.T(), component, stack, &inputs) assert.NotNil(s.T(), options) // ... // Just add this line to the check for drift s.DriftTest(component, stack, &inputs) } ``` 1. Run test ```console go test -v -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-destroy-component --skip-teardown ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → skipped 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed ... 2025/03/07 17:43:27 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (322.74s) --- PASS: TestRunSuite/TestBasic (3.19s) PASS ok test 324.052s ### Test `disabled` Use-case All components should avoid creating any resources if the `enabled` input is set to `false`. 1. Add atmos configuration for the component use case {Step7UsecaseYaml} 1. Import the use case for `default-test` stack {Step7Stack} 1. Add a "disabled" use case test ```go title="test/component_test.go" // ... func (s *ComponentSuite) TestEnabledFlag() { const component = "example-component/disabled" const stack = "default-test" s.VerifyEnabledFlag(component, stack, nil) } ``` 1. Run test ```console go test -v -timeout 1h --skip-deploy-dependencies --skip-destroy-dependencies --skip-destroy-component --skip-teardown ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → skipped 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed ... 2025/03/07 17:43:27 WARN TestRunSuite: teardown → skipped --- PASS: TestRunSuite (322.74s) --- PASS: TestRunSuite/TestBasic (3.19s) --- PASS: TestRunSuite/TestEnabledFlag (1.02s) PASS ok test 324.052s ### Tear Down Resources Tear down the test environment ```console go test -v -timeout 1h --skip-deploy-dependencies ``` === RUN TestRunSuite 2025/03/07 14:13:34 INFO TestRunSuite: setup → started 2025/03/07 14:13:34 INFO TestRunSuite: tests will be run in temp directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212 2025/03/07 14:13:34 INFO TestRunSuite: terraform state for tests will be saved in state directory path=/var/folders/1l/hcm6nfms6g58mdrpwcxklsvh0000gn/T/atmos-test-helper3047340212/state 2025/03/07 14:13:34 INFO TestRunSuite: setup/bootstrap temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → started 2025/03/07 14:13:34 INFO TestRunSuite: setup/copy component to temp dir → completed 2025/03/07 14:13:34 INFO TestRunSuite: vendor dependencies → started TestRunSuite 2025-03-07T14:13:35+01:00 retry.go:91: atmos [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Running command atmos with args [vendor pull] TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Processing vendor config file 'vendor.yaml' TestRunSuite 2025-03-07T14:13:35+01:00 logger.go:67: Pulling sources for the component 'account-map' from 'github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=1.520.0' into 'components/terraform/account-map' 2025/03/07 14:13:42 INFO TestRunSuite: vendor dependencies → completed 2025/03/07 17:38:24 INFO TestRunSuite: deploy dependencies → completed 2025/03/07 17:43:27 INFO TestRunSuite: setup → completed ... 2025/03/07 17:43:27 WARN TestRunSuite: teardown → completed --- PASS: TestRunSuite (322.74s) --- PASS: TestRunSuite/TestBasic (3.19s) --- PASS: TestRunSuite/TestEnabledFlag (1.02s) PASS ok test 324.052s ## FAQ ### Why do my tests fail when looking up remote state for components? If you encounter an error like: ``` Error: Attempt to get attribute from null value ... │ module.s3_bucket.outputs is null ... This value is null, so it does not have any attributes. ``` This typically occurs when using an older version of the remote-state module. The solution is to upgrade to version `1.8.0` or higher of the `cloudposse/stack-config/yaml//modules/remote-state` module. For example: ```hcl module "s3_bucket" { source = "cloudposse/stack-config/yaml//modules/remote-state" version = "1.8.0" component = var.destination_bucket_component_name context = module.this.context } ``` ### How do I handle dependencies in my tests? When testing components that depend on other infrastructure (like EKS clusters, VPCs, or other foundational components), you need to configure and deploy these dependencies in your test suite. This is done by adding dependencies to the stack test fixtures and deploying before running the tests. For example: ```go func TestRunSuite(t *testing.T) { suite := new(ComponentSuite) // Add dependencies suite.AddDependency(t, "s3-bucket/cloudwatch", "default-test", nil) helper.Run(t, suite) } ``` --- ## GitHub Contributors ## About Cloud Posse maintains 300+ projects under our GitHub organization. All of our projects have stemmed from past consulting engagements with our customers. Everything we do is Open Sourced under our GitHub under the permissive APACHE2 license. With so many projects, however, it wouldn't be possible to maintain all of them if we didn't have the support from our community and some tools to make life easier. ## Our Tools Here are some of the tools we depend on for running our Open Source organization. | Tool | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------------ | | `geodesic` | [Cloud Automation Shell](https://github.com/cloudposse/geodesic) that we use as the base Docker image for many of our projects | | `build-harness` | Our collection of GNU-style `Makefiles` for building stuff | | `README.yaml` | Our specification for generating beautiful READMEs | | `packages` | Our toolchain that we rely on throughout our projects | | GitHub Actions | Our CI/CD Platform | ## How to Become a Contributor Becoming a contributor is easy. Just start opening Pull Requests with enhancements, bug fixes, or other improvements. Once we take notice, we'll reach out to you. We recommend that you start by participating in the [`#pr-reviews`](https://slack.cloudposse.com/) channel. This way we'll work with you directly. ## Responsibilities * Participate in Code Reviews. Help us out by reviewing pull requests from our community. * Report issues and provide feedback for how we can improve processes. * Help answer questions from our community of tens of thousands users from around the world. * Cutting releases when Pull Requests are merged to master. ## Current Contributors We're grateful to our volunteers helping to review community pull requests. | Avatar | GitHub Username | Name | | -------------------------------------------------------------------------------- | -------------------------------------------- | ------------- | | ![osterman](https://img.cloudposse.com/75x75/http://github.com/osterman.png) | [@osterman](https://github.com/osterman) | Erik Osterman | | ![aknysh](https://img.cloudposse.com/75x75/http://github.com/aknysh.png) | [@aknysh](https://github.com/aknysh) | Andriy Knyshh | | ![goruha](https://img.cloudposse.com/75x75/http://github.com/goruha.png) | [@goruha](https://github.com/goruha) | Igor Rodionov | | ![nuru](https://img.cloudposse.com/75x75/http://github.com/nuru.png) | [@nuru](https://github.com/nuru) | Jeremy | | ![jamengual](https://img.cloudposse.com/75x75/http://github.com/jamengual.png) | [@jamengual](https://github.com/jamengual) | PePe Amengual | | ![adamcrews](https://img.cloudposse.com/75x75/http://github.com/adamcrews.png) | [@adamcrews](https://github.com/jamengual) | Adam Crews | | ![nitrocode](https://img.cloudposse.com/75x75/http://github.com/nitrocode.png) | [@nitrocode](https://github.com/nitrocode) | Ronak | | ![RothAndrew](https://img.cloudposse.com/75x75/http://github.com/RothAndrew.png) | [@RothAndrew](https://github.com/RothAndrew) | Andrew Roth | | ![Gowiem](https://img.cloudposse.com/75x75/http://github.com/Gowiem.png) | [@Gowiem](https://github.com/Gowiem) | Matt Gowie | --- ## Contributor Tips & Tricks import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Note from '@site/src/components/Note'; This document is intended to describe common and not-so-common processes that the contributor team executes as part of maintaining the 300+ open source projects within the Cloud Posse Organization. ## Tips & Tricks ### Update Multiple Repos at Once To update many of the open source repos with a common change such as updating Terraform `required_version` pinning, adding GitHub actions, or updating pinned providers, the contributor team has adopted using [microplane](https://github.com/Clever/microplane). This tool allows us to execute automated changes across dozens or even hundreds of our open source repos, which saves many hours of contributor time. Here is a standard usage pattern that contributors can adopt to specific changes as they see fit: 1. [Download the microplane binary from their releases page](https://github.com/Clever/microplane/releases) 1. Open your terminal, rename and add the downloaded binary into your $PATH, and add execution privileges to the binary: 1. `mv ~/Downloads/mp-0.0.21-darwin-amd64 /usr/local/bin/mp && chmod 755 /usr/local/bin/mp` 1. Add a [GH Personal Access Token](https://docs.github.com/en/github/authenticating-to-github/creating-a-personal-access-token) to your shell's environment variables under the variable `GITHUB_API_TOKEN`: 1. `export GITHUB_API_TOKEN=$YOUR_TOKEN` 1. Change to an empty directory that you can use as a scratch workspace for your Many Repos change: 1. `mkdir ~/mp_workspace && cd ~/mp_workspace` 1. Initialize microplane: 1. `mp init --all-repos "cloudposse"` 1. Initializing creates an `mp/` folder in your current directory, finds all of the Cloud Posse public projects, and then creates an `mp/init.json` file describing them. 1. NOTE: microplane supposedly has the ability to search against an organization and narrow the returned repos that end up in `init.json`, but that functionality seems buggy and not working. We do our repo filtering manually in the next step. 1. Manually edit `mp/init.json` to only include the repos which you want to make changes against. 1. Your editor's multi-select and edit capabilities are your friend here! (or maybe some [`jq`](https://stedolan.github.io/jq/) if that's your thing) 1. Duplicate the original `init.json` so you don't need to re-run `mp init` in the case that you want to start fresh. 1. Run microplane's clone command to pull all of the repos specified in `init.json` down to your local machine for mass changes: 1. `mp clone` 1. Create a bash script to facilitate the changes that you're attempting to make against the many repos you've specified. 1. Use the microplane `-r` or `--repo` flag to operate on a single repo for testing your script prior to making the changes across all repos. 1. Go through the full microplane process (complete the following steps) for this single repo test-run and get it signed off by the other Cloud Posse `#contributors` to ensure everyone agrees with the change prior to making it. 1. Once you've got a script that is working like you expect, you can run the microplane plan command. This step executes the given script across all the repos specified in `init.json` and then commits the result. You should format your `mp plan` command as follows: ```bash mp plan -b $YOUR_BRANCH_NAME -m "[AUTOMATED] $YOUR_PR_TITLE ## What 1. $INFO_ON_YOUR_CHANGE_NUMBER_ONE 1. $INFO_ON_YOUR_CHANGE_NUMBER_TWO ## Why 1. $INFO_ON_WHY_YOU_MADE_YOUR_CHANGE_NUMBER_ONE 1. $INFO_ON_WHY_YOU_MADE_YOUR_CHANGE_NUMBER_TWO" \ -- \ sh -c $PWD/$YOUR_SCRIPT_NAME.sh ``` the quotes around the message passing this as an argument to `-m` (message) 1. Verify one of the repos that you're updating exemplifies the changes you're trying to make: 1. `cd mp/terraform-aws-tfstate-backend/plan/planned/` 1. `git show` 1. Confirm the changes and commit message that you see are what you want to push. 1. If everything looks legit, ship it: 1. `mp push -a $YOUR_GITHUB_USERNAME` That should cycle through all of the repos in `init.json`, pushing them to the branch you specified, and creating a PR against the `master` branch. Now go forth and run `/test all` against all of those PRs and ask some kind soul to help you get them merged 😎. --- ## GitHub Contributors FAQ import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; Welcome to the Cloud Posse Contributors FAQ! This guide answers common questions about contributing to our open source projects. Whether you're a first-time contributor or a seasoned maintainer, you'll find helpful information about our processes, best practices, and community guidelines. If you don't find what you're looking for here, feel free to reach out through our community channels. ## How do I ask questions? There are several ways to get help and ask questions: 1. GitHub Discussions - Visit our [GitHub Discussions](https://github.com/orgs/cloudposse/discussions) - Search existing discussions to see if your question has already been answered - Create a new discussion if you can't find an answer 2. SweetOps Slack - Join our [SweetOps Slack workspace](https://slack.sweetops.com/) - Recommended channels: - `#terraform` - For Terraform-related questions - `#aws` - For AWS-specific questions - `#cloudposse` - For general Cloud Posse questions - `#atmos` - For Atmos-related questions - Please search the channel history before asking questions 3. **GitHub Issues** - For bug reports or feature requests, create an issue in the relevant repository - Make sure to follow the issue template and provide all requested information 4. **Documentation** - Check our [documentation](/learn/) for answers to common questions - Many questions can be answered by reading the relevant documentation ## How do I see all open Pull Requests? You can find all open Pull Requests by going to [GitHub and searching for open PRs](https://github.com/pulls?q=is%3Apr+is%3Aopen+org%3Acloudposse). ## What if we approve and merge a Pull Request with a problem? We encourage everyone who uses our modules to practice version pinning. So while we try to ensure `master` is always stable, we're not concerned if we occasionally break things. Also, we believe in a blameless culture. We rather figure out and fix why something happened than blame or chastise our volunteers. ## What are your best-practices we should follow? See our [Terraform Best Practices](/best-practices/terraform) and [Best Practices](/best-practices/). These are just some guidelines to follow and we're open to your feedback! ## What benefits do I receive as a contributor? As a contributor, you'll be able to expedite the reviews of Pull Requests for your organization by having a direct line of communication with our community of volunteers. ## Are contributors paid? All of our contributors are volunteers. Granted, some of our "volunteers" happen to work for Cloud Posse. They get paid! =) ## How do contributors collaborate? Contributors participate in a private Slack channel on the [SweetOps Slack team](https://slack.sweetops.com/) and via GitHub on issues and pull requests. ## When do we cut new releases? We cut a release every single merge to `master`. ## What is our versioning strategy? We practice [`semver`](https://semver.org). Our versioning strategy allows us to systematically and consistently increase patch, minor and major releases. When in doubt, bump the minor release. Following this strategy allows us to move quickly, release often while enabling our community to version pin for stability, and still convey the *semantics* of the kind of change that happened. 1. **Patch Releases** We bump the patch release for bug fixes of *existing* functionality or small updates to documentation 2. **Minor Releases** Projects that are `< 1.x`, every merge to `master` else is a minor release. This is the proper [semver convention](https://semver.org/#spec-item-4) for `0.x.y` releases. - While we always try to ensure the interfaces won't change radically, we cannot promise that will remain the case, especially when the tool itself (e.g. `terraform` is not yet `1.0`). - Once the interface is more or less guaranteed to be stable we will release a 1.0. 3. **Major Releases** The major version is milestone-driven (e.g. `> 1.x`). The first milestone is always stability. A major release will correspond to the previous minor release that closes out that milestone. - The 1.0 milestone doesn’t happen until we have had a very long burn-in period where it is stable and the interface works. For comparison, the `terraform` language is still `0.x` since July 28, 2014. - **After 1.0** all major releases are driven by achieving a particular feature set A common strategy practiced by other organizations is to bump the major release when there's a “known breaking change” and usually includes many changes all at once. This is typically practiced post-1.0 and it's still somewhat arbitrary and difficult to verify. Philosophically speaking, every change is breaking for somebody. For example, if a project has a bug, chances are that someone has implemented a workaround for that bug. If we release a bug fix as a patch release, that could very well be a breaking change for anyone who had a workaround. By releasing frequently on every commit to `master`, we allow the greatest number of users to benefit from the work we do. If we break something, no big deal. Users should always practice strict version pinning - never using `master` directly. That way, users can just pin to the previous release of of a module. As a small organization managing *hundreds* of projects, attempting a formal release schedule for each project is not feasible. ## How do we create a new release? As a member of the `@cloudposse/contributors` team, create a new release, use the [built-in GitHub release functionality](https://help.github.com/en/enterprise/2.13/user/articles/creating-releases). Please do not create releases manually by creating tags and pushing them as this lacks all the metadata associated with a release, which can have a rich markdown description. All GitHub releases also have tags, but not all tags have a GitHub release. :::caution Versions must follow the [`semver`](https://semver.org) convention. Do not prefix releases with a version specifier (e.g. a *good* version is `0.1.0` and a *bad* version is `v0.1.0`). ::: ## Why are releases not always in sequential order? Some of our `terraform` modules support backwards compatibility with HCLv1 (pre terraform 0.12). You'll notice these projects usually have a branch named `0.11/master`. When we accept a bugfix for one of these projects and merge to `master`, we will cut a patch release against the last minor release version for terraform 0.11. :::info We're not accepting new features for pre-terraform-0.12 modules. ::: ## Why is my Terraform Pull Request not yet reviewed or merged? If your Pull Request is to upgrade a Terraform module from HCLv1 to HCLv2, then chances are we haven't approved it because it does not have `terratest` integration tests. As a general policy, we're only upgrading modules to HCLv2 that have `terratest` integration tests. Attempting to maintain stability with hundreds of modules is only possible with integration testing. :::info All Terraform Modules updated to HCL2 **must** have `terratest` integration tests. ::: ## Do we have to update integration tests? We do not expect contributors to be experts at integration testing or writing Golang. For that reason, we do not require that Open Source community contributors update integration tests. However, if existing tests break due to changes in a Pull Request, we will not accept the contributions until the tests pass. ## How are Pull Requests merged? Can I merge my own Pull Requests? Once a Pull Request is approved and tests pass, then it may be merged. Anyone with permissions to merge is permitted to merge. Note, if any changes are pushed to the branch, the approval is automatically dismissed. This is why we let you merge your own PRs. Approvers are free to leave a Pull Request open so that the originator of the PR may have the recourse to change things if something comes up. While we try to keep `master` stable, it's just a best-effort. If something goes wrong, it's better that we address what broke down procedurally (e.g. improving tests, communication, etc.), than micro-managing the merging process. We recommend users version pin to releases for stability and never pin to master. After merging a Pull Request to `master`, then cut a release. We cut a release for every merge to master. If it's a bug fix, bump the patch release (e.g. `0.0.x`). If it's a new feature, bump the minor (e.g. `0.x.0`). It's that easy! Review the rest of this FAQ for more details on our `semver` strategy. ## What are the merge constraints? All of our GitHub repositories implement the following convention with branch protections: 1. At least (1) approver determined by the [`CODEOWNER`](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners) file 2. Required tests passing --- ## Our GitHub import Steps from '@site/src/components/Steps'; import Note from '@site/src/components/Note'; ## Our Commitment We commit to always provide free and public access to our Open Source repositories. If you see repository on our GitHub today, then it will be here tomorrow and thereafter in perpetuity. From time to time, we might decide we can no longer maintain a repository. If that happens, we will mark it as "archived" on GitHub. This will ensure you will continue to have access to the code. ## Getting Involved The best way to get involved is to checkout our "[Choose Your Path](/intro/path/)" guide. Then join us in our Slack [`#community`](https://cloudposse.com/slack/) channel to get support or talk with others in the community. ## Contributing Cloud Posse accepts contributions from the community. In the interest of fostering an open and welcoming environment, we have a strict [code of conduct](/community/code-of-conduct) to ensure participation in our projects and our community is a harassment-free experience for everyone. If you want to make some big changes or don't know where to begin, it's best to first get in touch. You can discuss the change you wish to make via GitHub issues, [email](mailto:hello@cloudposse.com), or join our [`#community`](https://cloudposse.com/slack/) channel. In general, PRs are welcome. We follow the typical "fork-and-pull" Git workflow. 1. Fork the repo on GitHub 2. Clone the project to your own machine 3. Commit changes to your own branch 4. Push your work back up to your fork 5. Submit a Pull request so that we can review your changes Be sure to merge the latest from "upstream" before making a pull request! ## Maintenance Cloud Posse actively maintains all projects on [our GitHub](https://github.com/cloudposse/). ## Deprecation and Archival Process From time to time, we may need to deprecate and archive repositories that are no longer actively maintained. We follow a structured process to ensure transparency and give the community adequate notice. This process applies to all Cloud Posse repositories, including: - Terraform modules in the [cloudposse](https://github.com/cloudposse/) organization - Terraform components in the [cloudposse-terraform-components](https://github.com/cloudposse-terraform-components) organization - GitHub Actions and other tooling ### Step 1: Create and Pin a GitHub Issue Create a comprehensive GitHub issue that includes: - Detailed explanation of the deprecation - Timeline and key dates - Migration path or alternatives - Answers to common questions - Contact information for support Pin this issue to the repository so it appears at the top of the Issues tab. ### Step 2: Add Deprecation Notice to README Add a deprecation warning at the top of the `README.md` using a GitHub-style warning admonition: ```markdown > [!WARNING] > **Deprecated**: This repository is deprecated and will be archived on [DATE]. > Please see [Issue #XXX](link-to-pinned-issue) for more information. ``` ### Step 3: Update README.yaml If the project uses `README.yaml` for generating documentation, add the `deprecated` field: ```yaml deprecated: notice: |- This module is deprecated and will be archived on [DATE]. Please see the [pinned issue](link-to-pinned-issue) for details and migration guidance. Consider using [alternative-module](link) as a replacement. ``` After updating `README.yaml`, regenerate the `README.md`: ```bash atmos docs generate readme ``` ### Step 4: Publish Blog Post Announcement Create a blog post announcing the deprecation. This post should: - Link to the pinned GitHub issue - Explain the reason for deprecation - Provide the timeline and deprecation date - Offer migration guidance or alternatives - Direct readers where to ask questions This ensures the broader community is aware of the deprecation, even if they're not actively monitoring the repository. ### Step 5: Submit Pull Request Create a pull request with the changes from Steps 2-3. This PR provides visibility to those monitoring repository activity. The PR should: - Clearly state the reason for deprecation - Specify the planned deprecation date - Provide migration guidance or alternative solutions (if applicable) - Reference the GitHub issue created in Step 1 - Tag relevant stakeholders for visibility ### Step 6: Wait Until Deprecation Date Allow sufficient time (typically 90+ days) for the community to: - Migrate away from the deprecated component - Ask questions and get support - Complete any in-flight work ### Step 7: Archive the Repository Once the deprecation date has passed: 1. Update the `.github/settings.yml` file in the repository: ```yaml repository: archived: true ``` 2. Commit and merge this change 3. The repository will be automatically archived by GitHub settings automation Once archived, the repository becomes read-only but remains publicly accessible for historical reference. ### Step 8: Update Blog Post Update the blog post from Step 4 to reflect that the repository has been archived: - Add a note that the deprecation period has ended - Confirm the repository is now archived - Remind readers that the code remains publicly accessible for historical reference ## GitHub Projects There's a lot going on in our GitHub. With over [200+ Open Source repositories](https://github.com/cloudposse/), keeping track of all the [Open Issues](https://github.com/search?q=org%3Acloudposse+type%3Aissues+is%3Aopen), Feature Requests, and Pull Requests is a fulltime job. We use Kanban board to manage our [Open Source projects](https://github.com/orgs/cloudposse/projects/3). (**Help wanted!**) ## Something Missing? [Get in touch](/community/contact-us) with us. --- ## Office Hours Registration import HubspotForm from 'react-hubspot-form' import { YouTubePlaylist } from '@codesweetly/react-youtube-playlist'; console.log('Submit!')} onReady={(form) => console.log('Form ready!')} loading={Loading...} /> ## Past Recordings --- ## #refarch ## Join our Slack Community! Cloud Posse has a great community of active users who are more than willing to help each other. So, join us! --- ## Community Support import Intro from '@site/src/components/Intro'; Cloud Posse is an **open source company**. Everything we develop is open source and free to use under permissive licenses, but **Support is not included**. :::tip If your team depends on our work, the best way to support Cloud Posse—and to ensure your team gets the help it needs—is by subscribing to one of our [premium support options](/support). ::: ## Getting Help We have a [Slack community](https://cloudposse.com/slack) and a [GitHub Discussions forum](https://github.com/orgs/cloudposse/discussions) for questions and collaboration relating to our open source projects. ### Recommended etiquette: - **Ask one question at a time** — split complex topics into separate posts. - **Keep it high-level and clear** — we prioritize general questions that help the broader community. - **Deeper technical or commercial-related questions** may be answered from time to time, **solely at our discretion**. ## Free Tier Support **Free Tier Support does not include** support for: - [Quickstart](/quickstart), [Jumpstart](/jumpstart), or other Commercial Reference Architectures - [Cloud Posse Components](/components) - [GitHub Actions and workflows](/github-actions) - **Deliverables from paid implementations or services** Our projects are open source and free to use — **but support is not included**. We occasionally answer complex or commercial-related questions here, but this is voluntary and unscheduled. If your team depends on our work, we strongly encourage you to [subscribe to premium support](https://cloudposse.com/support). **That’s what makes open source work.** --- ## Terraform Components import Intro from "@site/src/components/Intro"; import DocCardList from "@theme/DocCardList"; This is a library of reusable Terraform "root module" components. :::info ## Terraform Component GitHub Repository Has Moved! The GitHub repository for Cloud Posse's Terraform components has migrated to a [dedicated GitHub organization](https://github.com/cloudposse-terraform-components). All documentation remains here, but all future updates, contributions, and issue tracking for the source code should now be directed to the respective repositories in the new organization. [Learn more](/learn/maintenance/tutorials/how-to-update-components-yaml-to-new-organization/) about updating your references to point to the new repositories. ::: --- ## access-analyzer This component is responsible for configuring AWS Identity and Access Management Access Analyzer within an AWS Organization. IAM Access Analyzer helps you identify the resources in your organization and accounts, such as Amazon S3 buckets or IAM roles, shared with an external entity. This lets you identify unintended access to your resources and data, which is a security risk. IAM Access Analyzer identifies resources shared with external principals by using logic-based reasoning to analyze the resource-based policies in your AWS environment. For each instance of a resource shared outside of your account, IAM Access Analyzer generates a finding. Findings include information about the access and the external principal granted to it. You can review findings to determine if the access is intended and safe or if the access is unintended and a security risk. In addition to helping you identify resources shared with an external entity, you can use IAM Access Analyzer findings to preview how your policy affects public and cross-account access to your resource before deploying resource permissions. The findings are organized in a visual summary dashboard. The dashboard highlights the split between public and cross-account access findings, and provides a breakdown of findings by resource type. IAM Access Analyzer analyzes only policies applied to resources in the same AWS Region where it's enabled. To monitor all resources in your AWS environment, you must create an analyzer to enable IAM Access Analyzer in each Region where you're using supported AWS resources. AWS Identity and Access Management Access Analyzer provides the following capabilities: - IAM Access Analyzer external access analyzers help identify resources in your organization and accounts that are shared with an external entity. - IAM Access Analyzer unused access analyzers help identify unused access in your organization and accounts. - IAM Access Analyzer validates IAM policies against policy grammar and AWS best practices. - IAM Access Analyzer custom policy checks help validate IAM policies against your specified security standards. - IAM Access Analyzer generates IAM policies based on access activity in your AWS CloudTrail logs. Here's a typical workflow: **Delegate Access Analyzer to another account**: From the Organization management (root) account, delegate administration to a specific AWS account within your organization (usually the security account). **Create Access Analyzers in the Delegated Administrator Account**: Enable the Access Analyzers for external access and unused access in the delegated administrator account. ## Deployment Overview ```yaml components: terraform: access-analyzer/defaults: metadata: component: access-analyzer type: abstract vars: enabled: true global_environment: gbl account_map_tenant: core root_account_stage: root delegated_administrator_account_name: core-mgt accessanalyzer_service_principal: "access-analyzer.amazonaws.com" accessanalyzer_organization_enabled: false accessanalyzer_organization_unused_access_enabled: false organizations_delegated_administrator_enabled: false ``` ```yaml import: - catalog/access-analyzer/defaults components: terraform: access-analyzer/root: metadata: component: access-analyzer inherits: - access-analyzer/defaults vars: organizations_delegated_administrator_enabled: true ``` ```yaml import: - catalog/access-analyzer/defaults components: terraform: access-analyzer/delegated-administrator: metadata: component: access-analyzer inherits: - access-analyzer/defaults vars: accessanalyzer_organization_enabled: true accessanalyzer_organization_unused_access_enabled: true unused_access_age: 30 ``` ### Provisioning Delegate Access Analyzer to the security account: ```bash atmos terraform apply access-analyzer/root -s plat-dev-gbl-root ``` Provision Access Analyzers for external access and unused access in the delegated administrator (security) account in each region: ```bash atmos terraform apply access-analyzer/delegated-administrator -s plat-dev-use1-mgt ``` ## References - https://aws.amazon.com/iam/access-analyzer/ - https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html - https://repost.aws/knowledge-center/iam-access-analyzer-organization - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/accessanalyzer_analyzer - https://github.com/hashicorp/terraform-provider-aws/issues/19312 - https://github.com/hashicorp/terraform-provider-aws/pull/19389 - https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_delegated_administrator ## Variables ### Required Variables
`accessanalyzer_organization_enabled` (`bool`) required
Flag to enable the Organization Access Analyzer
`accessanalyzer_organization_unused_access_enabled` (`bool`) required
Flag to enable the Organization unused access Access Analyzer
`account_map_tenant` (`string`) required
The tenant where the `account_map` component required by remote-state is deployed
`delegated_administrator_account_name` (`string`) required
The name of the account that is the AWS Organization Delegated Administrator account
`organizations_delegated_administrator_enabled` (`bool`) required
Flag to enable the Organization delegated administrator
`region` (`string`) required
AWS Region
### Optional Variables
`accessanalyzer_service_principal` (`string`) optional
The Access Analyzer service principal for which you want to make the member account a delegated administrator **Default value:** `"access-analyzer.amazonaws.com"`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`organization_management_account_name` (`string`) optional
The name of the AWS Organization management account **Default value:** `null`
`root_account_stage` (`string`) optional
The stage name for the Organization root (management) account. This is used to lookup account IDs from account names using the `account-map` component. **Default value:** `"root"`
`unused_access_age` (`number`) optional
The specified access age in days for which to generate findings for unused access **Default value:** `30`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_organizations_delegated_administrator_id`
AWS Organizations Delegated Administrator ID
`aws_organizations_delegated_administrator_status`
AWS Organizations Delegated Administrator status
`organization_accessanalyzer_id`
Organization Access Analyzer ID
`organization_unused_access_accessanalyzer_id`
Organization unused access Access Analyzer ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_accessanalyzer_analyzer.organization`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/accessanalyzer_analyzer) (resource) - [`aws_accessanalyzer_analyzer.organization_unused_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/accessanalyzer_analyzer) (resource) - [`aws_organizations_delegated_administrator.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_delegated_administrator) (resource) ## Data Sources The following data sources are used by this module: --- ## account This component is responsible for provisioning the full account hierarchy along with Organizational Units (OUs). It includes the ability to associate Service Control Policies (SCPs) to the Organization, each Organizational Unit and account. :::note Part of a [cold start](https://docs.cloudposse.com/layers/accounts/prepare-aws-organization/) so it has to be initially run with `SuperAdmin` role. ::: In addition, it enables [AWS IAM Access Analyzer](https://docs.aws.amazon.com/IAM/latest/UserGuide/what-is-access-analyzer.html), which helps you identify the resources in your organization and accounts, such as Amazon S3 buckets or IAM roles, that are shared with an external entity. This lets you identify unintended access to your resources and data, which is a security risk. Access Analyzer identifies resources that are shared with external principals by using logic-based reasoning to analyze the resource-based policies in your AWS environment. For each instance of a resource that is shared outside of your account, Access Analyzer generates a finding. Findings include information about the access and the external principal that it is granted to. You can review findings to determine whether the access is intended and safe, or the access is unintended and a security risk. ## Usage **Stack Level**: Global **IMPORTANT**: Account Name building blocks (such as tenant, stage, environment) must not contain dashes. Doing so will lead to unpredictable resource names as a `-` is the default delimiter. Additionally, account names must be lower case alphanumeric with no special characters. For example: | Key | Value | Correctness | | ---------------- | --------------- | ----------- | | **Tenant** | foo | ✅ | | **Tenant** | foo-bar | ❌ | | **Environment** | use1 | ✅ | | **Environment** | us-east-1 | ❌ | | **Account Name** | `core-identity` | ✅ | Here is an example snippet for how to use this component. Include this snippet in the stack configuration for the management account (typically `root`) in the management tenant/OU (usually something like `mgmt` or `core`) in the global region (`gbl`). You can insert the content directly, or create a `stacks/catalog/account.yaml` file and import it from there. ```yaml components: terraform: account: settings: spacelift: workspace_enabled: false backend: s3: role_arn: null vars: enabled: true account_email_format: aws+%s@example.net account_iam_user_access_to_billing: ALLOW organization_enabled: true aws_service_access_principals: - cloudtrail.amazonaws.com - guardduty.amazonaws.com - ipam.amazonaws.com - ram.amazonaws.com - securityhub.amazonaws.com - servicequotas.amazonaws.com - sso.amazonaws.com - securityhub.amazonaws.com - auditmanager.amazonaws.com enabled_policy_types: - SERVICE_CONTROL_POLICY - TAG_POLICY organization_config: root_account: name: core-root stage: root tenant: core tags: eks: false accounts: [] organization: service_control_policies: - DenyEC2InstancesWithoutEncryptionInTransit organizational_units: - name: core accounts: - name: core-artifacts tenant: core stage: artifacts tags: eks: false - name: core-audit tenant: core stage: audit tags: eks: false - name: core-auto tenant: core stage: auto tags: eks: true - name: core-corp tenant: core stage: corp tags: eks: true - name: core-dns tenant: core stage: dns tags: eks: false - name: core-identity tenant: core stage: identity tags: eks: false - name: core-network tenant: core stage: network tags: eks: false - name: core-security tenant: core stage: security tags: eks: false service_control_policies: - DenyLeavingOrganization - name: plat accounts: - name: plat-dev tenant: plat stage: dev tags: eks: true - name: plat-sandbox tenant: plat stage: sandbox tags: eks: true - name: plat-staging tenant: plat stage: staging tags: eks: true - name: plat-prod tenant: plat stage: prod tags: eks: true service_control_policies: - DenyLeavingOrganization service_control_policies_config_paths: # These paths specify where to find the service control policies identified by SID in the service_control_policies sections above. - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/cloudwatch-logs-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/deny-all-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/iam-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/kms-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/organization-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/route53-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/s3-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/ec2-policies.yaml" ``` ## First Time Organization Setup Your AWS Organization is managed by the `account` component, along with accounts and organizational units. However, because the AWS defaults for an Organization and its accounts are not exactly what we want, and there is no way to change them via Terraform, we have to first provision the AWS Organization, then take some steps on the AWS console, and then we can provision the rest. ### Use AWS Console to create and set up the Organization Unfortunately, there are some tasks that need to be done via the console. Log into the AWS Console with the root (not SuperAdmin) credentials you have saved in 1Password. #### Request an increase in the maximum number of accounts allowed :::warning Make sure your support plan for the _root_ account was upgraded to the "Business" level (or Higher). This is necessary to expedite the quota increase requests, which could take several days on a basic support plan. Without it, AWS support will claim that since we’re not currently utilizing any of the resources, so they do not want to approve the requests. AWS support is not aware of your other organization. If AWS still gives you problems, please escalate to your AWS TAM. ::: 1. From the region list, select "US East (N. Virginia) us-east-1". 2. From the account dropdown menu, select "My Service Quotas". 3. From the Sidebar, select "AWS Services". 4. Type "org" in the search field under "AWS services" 5. Click on "AWS Organizations" in the "Service" list 6. Click on "Default maximum number of accounts", which should take you to a new view 7. Click on "Request quota increase" on the right side of the view, which should pop us a request form 8. At the bottom of the form, under "Change quota value", enter the number you decided on in the previous step (probably "20") and click "Request" #### (Optional) Create templates to request other quota increases New accounts start with a low limit on the number of instances you can create. However, as you add accounts, and use more instances, the numbers automatically adjust up. So you may or may not want to create a template to generate automatic quota increase requests, depending on how many instances per account you expect to want to provision right away. Create a [Quota request template](https://docs.aws.amazon.com/servicequotas/latest/userguide/organization-templates.html) for the organization. From the Sidebar, click "Quota request template" Add each EC2 quota increase request you want to make: 1. Click "Add Quota" on the right side of the view 2. Under "Region", select your default region (repeat with the backup region if you are using one) 3. Under "Service", type "EC2" and select "Amazon Elastic Compute Cloud (Amazon EC2)" 4. Under "Quota", find the quota you want to increase. The likely candidates are: 5. type "stand" and select "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) Instances" 6. type "stand" and select "All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Request" 7. type "g i" and select "Running On-Demand G Instances" 8. type "all g" and select "All G Spot Instance Requests" 9. Under "Desired quota value" enter your desired default quota 10. Click "Add" After you have added all the templates, click "Enable" on the Quota request template screen to enable the templates. #### Enable resource sharing with AWS Organization [AWS Resource Access Manager (RAM)](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) lets you share your resources with any AWS account or through AWS Organizations. If you have multiple AWS accounts, you can create resources centrally and use AWS RAM to share those resources with other accounts. Resource sharing through AWS Organization will be used to share the Transit Gateway deployed in the `network` account with other accounts to connect their VPCs to the shared Transit Gateway. This is a one-time manual step in the AWS Resource Access Manager console. When you share resources within your organization, AWS RAM does not send invitations to principals. Principals in your organization get access to shared resources without exchanging invitations. To enable resource sharing with AWS Organization via AWS Management Console - Open the Settings page of AWS Resource Access Manager console at [https://console.aws.amazon.com/ram/home#Settings](https://console.aws.amazon.com/ram/home#Settings) - Choose "Enable sharing with AWS Organizations" To enable resource sharing with AWS Organization via AWS CLI ``` √ . [xamp-SuperAdmin] (HOST) infra ⨠ aws ram enable-sharing-with-aws-organization { "returnValue": true } ``` For more information, see: - [https://docs.aws.amazon.com/ram/latest/userguide/what-is.html](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) - [https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html) - [https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-ram.html](https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-ram.html) ### Import the organization into Terraform using the `account` component After we are done with the above ClickOps and the Service Quota Increase for maximum number of accounts has been granted, we can then do the rest via Terraform. In the Geodesic shell, as SuperAdmin, execute the following command to get the AWS Organization ID that will be used to import the organization: ``` aws organizations describe-organization ``` From the output, identify the _organization-id_: ``` { "Organization": { "Id": "o-7qcakq6zxw", "Arn": "arn:aws:organizations:: ... ``` Using the example above, the _organization-id_ is o-7qcakq6zxw. In the Geodesic shell, as SuperAdmin, execute the following command to import the AWS Organization, changing the stack name `core-gbl-root` if needed, to reflect the stack where the organization management account is defined, and changing the last argument to reflect the _organization-id_ from the output of the previous command. ``` atmos terraform import account --stack core-gbl-root 'aws_organizations_organization.this[0]' 'o-7qcakq6zxw' ``` ### Provision AWS OUs and Accounts using the `account` component AWS accounts and organizational units are generated dynamically by the `terraform/account` component using the configuration in the `gbl-root` stack. :::important In the rare case where you will need to be enabling non-default AWS Regions, temporarily comment out the `DenyRootAccountAccess` service control policy setting in `gbl-root.yaml`. You will restore it later, after enabling the optional Regions. See related: [Decide on Opting Into Non-default Regions](https://docs.cloudposse.com/layers/network/design-decisions/decide-on-opting-into-non-default-regions/) ::: :::tip #### You must wait until your quota increase request has been granted If you try to create the accounts before the quota increase is granted, you can expect to see failures like `ACCOUNT_NUMBER_LIMIT_EXCEEDED`. ::: In the Geodesic shell, execute the following commands to provision AWS Organizational Units and AWS accounts: ``` atmos terraform apply account --stack gbl-root ``` Review the Terraform plan, _**ensure that no new organization will be created**_ (look for `aws_organizations_organization.this[0]`), type "yes" to approve and apply. This creates the AWS organizational units and AWS accounts. ### Configure root account credentials for each account Note: unless you need to enable non-default AWS regions (see next step), this step can be done later or in parallel with other steps, for example while waiting for Terraform to create resources. **For** _**each**_ **new account:** 1. Perform a password reset by attempting to [log in to the AWS console](https://signin.aws.amazon.com/signin) as a "root user", using that account's email address, and then clicking the "Forgot password?" link. You will receive a password reset link via email, which should be forwarded to the shared Slack channel for automated messages. Click the link and enter a new password. (Use 1Password or [Random.org](https://www.random.org/passwords) to create a password 26-38 characters long, including at least 3 of each class of character: lower case, uppercase, digit, and symbol. You may need to manually combine or add to the generated password to ensure 3 symbols and digits are present.) Save the email address and generated password as web login credentials in 1Password. While you are at it, save the account number in a separate field. 2. Log in using the new password, choose "My Security Credentials" from the account dropdown menu and set up Multi-Factor Authentication (MFA) to use a Virtual MFA device. Save the MFA TOTP key in 1Password by using 1Password's TOTP field and built-in screen scanner. Also, save the Virtual MFA ARN (sometimes shown as "serial number"). 3. While logged in, enable optional regions as described in the next step, if needed. 4. (Optional, but highly recommended): [Unsubscribe](https://pages.awscloud.com/communication-preferences.html) the account's email address from all marketing emails. ### (Optional) Enable regions Most AWS regions are enabled by default. If you are using a region that is not enabled by default (such as Middle East/Bahrain), you need to take extra steps. 1. While logged in using root credentials (see the previous step), in the account dropdown menu, select "My Account" to get to the [Billing home page](https://console.aws.amazon.com/billing/home?#/account). 2. In the "AWS Regions" section, enable the regions you want to enable. 3. Go to the IAM [account settings page](https://console.aws.amazon.com/iam/home?#/account_settings) and edit the STS Global endpoint to create session tokens valid in all AWS regions. You will need to wait a few minutes for the regions to be enabled before you can proceed to the next step. Until they are enabled, you may get what look like AWS authentication or permissions errors. After enabling the regions in all accounts, re-enable the `DenyRootAccountAccess` service control policy setting in `gbl-root.yaml` and rerun ``` atmos terraform apply account --stack gbl-root ``` ## Variables ### Required Variables
`account_email_format` (`string`) required
Email address format for the accounts (e.g. `aws+%s@example.com`)
`aws_service_access_principals` (`list(string)`) required
List of AWS service principal names for which you want to enable integration with your organization. This is typically in the form of a URL, such as service-abbreviation.amazonaws.com. Organization must have `feature_set` set to ALL. For additional information, see the [AWS Organizations User Guide](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_integrate_services.html)
`enabled_policy_types` (`list(string)`) required
List of Organizations policy types to enable in the Organization Root. Organization must have feature_set set to ALL. For additional information about valid policy types (e.g. SERVICE_CONTROL_POLICY and TAG_POLICY), see the [AWS Organizations API Reference](https://docs.aws.amazon.com/organizations/latest/APIReference/API_EnablePolicyType.html)
`organization_config` (`any`) required
Organization, Organizational Units and Accounts configuration
`region` (`string`) required
AWS Region
`service_control_policies_config_paths` (`list(string)`) required
List of paths to Service Control Policy configurations
### Optional Variables
`account_iam_user_access_to_billing` (`string`) optional
If set to `ALLOW`, the new account enables IAM users to access account billing information if they have the required permissions. If set to `DENY`, then only the root user of the new account can access account billing information **Default value:** `"DENY"`
`organization_enabled` (`bool`) optional
A boolean flag indicating whether to create an Organization or use the existing one **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`account_arns`
List of account ARNs (excluding root account)
`account_ids`
List of account IDs (excluding root account)
`account_info_map`
Map of account names to eks: boolean, account hosts at least one EKS cluster id: account id (number) stage: (optional) the account "stage" tenant: (optional) the account "tenant"
`account_name_id_map`
Map of account names to account IDs (including root account)
`account_names_account_arns`
Map of account names to account ARNs (excluding root account)
`account_names_account_ids`
Map of account names to account IDs (excluding root account)
`account_names_account_scp_arns`
Map of account names to SCP ARNs for accounts with SCPs
`account_names_account_scp_ids`
Map of account names to SCP IDs for accounts with SCPs
`eks_accounts`
List of EKS accounts
`non_eks_accounts`
List of non EKS accounts
`organization_arn`
Organization ARN
`organization_id`
Organization ID
`organization_master_account_arn`
Organization master account ARN
`organization_master_account_email`
Organization master account email
`organization_master_account_id`
Organization master account ID
`organization_scp_arn`
Organization Service Control Policy ARN
`organization_scp_id`
Organization Service Control Policy ID
`organizational_unit_arns`
List of Organizational Unit ARNs
`organizational_unit_ids`
List of Organizational Unit IDs
`organizational_unit_names_organizational_unit_arns`
Map of Organizational Unit names to Organizational Unit ARNs
`organizational_unit_names_organizational_unit_ids`
Map of Organizational Unit names to Organizational Unit IDs
`organizational_unit_names_organizational_unit_scp_arns`
Map of OU names to SCP ARNs
`organizational_unit_names_organizational_unit_scp_ids`
Map of OU names to SCP IDs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `accounts_service_control_policies` | 0.15.1 | [`cloudposse/service-control-policies/aws`](https://registry.terraform.io/modules/cloudposse/service-control-policies/aws/0.15.1) | Provision Accounts Service Control Policies `organization_service_control_policies` | 0.15.1 | [`cloudposse/service-control-policies/aws`](https://registry.terraform.io/modules/cloudposse/service-control-policies/aws/0.15.1) | Provision Organization Service Control Policy `organizational_units_service_control_policies` | 0.15.1 | [`cloudposse/service-control-policies/aws`](https://registry.terraform.io/modules/cloudposse/service-control-policies/aws/0.15.1) | Provision Organizational Units Service Control Policies `service_control_policy_statements_yaml_config` | 1.0.2 | [`cloudposse/config/yaml`](https://registry.terraform.io/modules/cloudposse/config/yaml/1.0.2) | Convert all Service Control Policy statements from YAML config to Terraform list `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_organizations_account.organization_accounts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_account) (resource) - [`aws_organizations_account.organizational_units_accounts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_account) (resource) - [`aws_organizations_organization.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_organization) (resource) - [`aws_organizations_organizational_unit.child`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_organizational_unit) (resource) - [`aws_organizations_organizational_unit.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_organizational_unit) (resource) ## Data Sources The following data sources are used by this module: - [`aws_organizations_organization.existing`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/organizations_organization) (data source) --- ## account-map This component is responsible for provisioning information only: it simply populates Terraform state with data (account ids, groups, and roles) that other root modules need via outputs. ## Usage ## Pre-requisites - [account](https://docs.cloudposse.com/components/library/aws/account) must be provisioned before [account-map](https://docs.cloudposse.com/components/library/aws/account-map) component ## Usage **Stack Level**: Global Here is an example snippet for how to use this component. Include this snippet in the stack configuration for the management account (typically `root`) in the management tenant/OU (usually something like `mgmt` or `core`) in the global region (`gbl`). You can include the content directly, or create a `stacks/catalog/account-map.yaml` file and import it from there. ```yaml components: terraform: account-map: vars: enabled: true # Set profiles_enabled to false unless we are using AWS config profiles for Terraform access. # When profiles_enabled is false, role_arn must be provided instead of profile in each terraform component provider. # This is automatically handled by the component's `provider.tf` file in conjunction with # the `account-map/modules/iam-roles` module. profiles_enabled: false root_account_aws_name: "aws-root" root_account_account_name: root identity_account_account_name: identity dns_account_account_name: dns audit_account_account_name: audit # The following variables contain `format()` strings that take the labels from `null-label` # as arguments in the standard order. The default values are shown here, assuming # the `null-label.label_order` is # ["namespace", "tenant", "environment", "stage", "name", "attributes"] # Note that you can rearrange the order of the labels in the template by # using [explicit argument indexes](https://pkg.go.dev/fmt#hdr-Explicit_argument_indexes) just like in `go`. # `iam_role_arn_template_template` is the template for the template [sic] used to render Role ARNs. # The template is first used to render a template for the account that takes only the role name. # Then that rendered template is used to create the final Role ARN for the account. iam_role_arn_template_template: "arn:%s:iam::%s:role/%s-%s-%s-%s-%%s" # `profile_template` is the template used to render AWS Profile names. profile_template: "%s-%s-%s-%s-%s" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`root_account_aws_name` (`string`) required
The name of the root account as reported by AWS
### Optional Variables
`account_configuration_export_enabled` (`bool`) optional
If true, the account configuration information will be exported to a file under `account-info/` **Default value:** `true`
`artifacts_account_account_name` (`string`) optional
The short name for the artifacts account **Default value:** `"artifacts"`
`audit_account_account_name` (`string`) optional
The short name for the audit account **Default value:** `"audit"`
`aws_config_identity_profile_name` (`string`) optional
The AWS config profile name to use as `source_profile` for credentials. **Default value:** `null`
`dns_account_account_name` (`string`) optional
The short name for the primary DNS account **Default value:** `"dns"`
`iam_role_arn_template_template` (`string`) optional
The template for the template used to render Role ARNs. The template is first used to render a template for the account that takes only the role name. Then that rendered template is used to create the final Role ARN for the account. Default is appropriate when using `tenant` and default label order with `null-label`. Use `"arn:%s:iam::%s:role/%s-%s-%s-%%s"` when not using `tenant`. Note that if the `null-label` variable `label_order` is truncated or extended with additional labels, this template will need to be updated to reflect the new number of labels. **Default value:** `"arn:%s:iam::%s:role/%s-%s-%s-%s-%%s"`
`identity_account_account_name` (`string`) optional
The short name for the account holding primary IAM roles **Default value:** `"identity"`
`import_organization_accounts` (`bool`) optional
Retrieve accounts from AWS Organizations and import them into the account map. Set false for brownfield environments where you want to curate the list of accounts manually via the `account` component with a static backend. Note that the brownfield `account` component needs to include the `root` account in the `account_names_account_ids` map, whereas the greenfield `account` component does not. **Default value:** `true`
`legacy_terraform_uses_admin` (`bool`) optional
If `true`, the legacy behavior of using the `admin` role rather than the `terraform` role in the `root` and identity accounts will be preserved. The default is to use the negations of the value of `terraform_dynamic_role_enabled`. **Default value:** `null`
`profile_template` (`string`) optional
The template used to render AWS Profile names. Default is appropriate when using `tenant` and default label order with `null-label`. Use `"%s-%s-%s-%s"` when not using `tenant`. Note that if the `null-label` variable `label_order` is truncated or extended with additional labels, this template will need to be updated to reflect the new number of labels. **Default value:** `"%s-%s-%s-%s-%s"`
`profiles_enabled` (`bool`) optional
Whether or not to enable profiles instead of roles for the backend. If true, profile must be set. If false, role_arn must be set. **Default value:** `false`
`root_account_account_name` (`string`) optional
The short name for the root account **Default value:** `"root"`
`terraform_dynamic_role_enabled` (`bool`) optional
If true, the IAM role Terraform will assume will depend on the identity of the user running terraform **Default value:** `false`
`terraform_role_name_map` (`map(string)`) optional
Mapping of Terraform action (plan or apply) to aws-team-role name to assume for that action **Default value:** ```hcl { "apply": "terraform", "plan": "planner" } ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`account_info_map`
A map from account name to various information about the account. See the `account_info_map` output of `account` for more detail.
`all_accounts`
A list of all accounts in the AWS Organization
`artifacts_account_account_name`
The short name for the artifacts account
`audit_account_account_name`
The short name for the audit account
`aws_partition`
The AWS "partition" to use when constructing resource ARNs
`cicd_profiles` OBSOLETE
dummy results returned to avoid breaking code that depends on this output
`cicd_roles` OBSOLETE
dummy results returned to avoid breaking code that depends on this output
`dns_account_account_name`
The short name for the primary DNS account
`eks_accounts`
A list of all accounts in the AWS Organization that contain EKS clusters
`full_account_map`
The map of account name to account ID (number).
`helm_profiles` OBSOLETE
dummy results returned to avoid breaking code that depends on this output
`helm_roles` OBSOLETE
dummy results returned to avoid breaking code that depends on this output
`iam_role_arn_templates`
Map of accounts to corresponding IAM Role ARN templates
`identity_account_account_name`
The short name for the account holding primary IAM roles
`non_eks_accounts`
A list of all accounts in the AWS Organization that do not contain EKS clusters
`org`
The name of the AWS Organization
`profiles_enabled`
Whether or not to enable profiles instead of roles for the backend
`root_account_account_name`
The short name for the root account
`root_account_aws_name`
The name of the root account as reported by AWS
`terraform_access_map`
Mapping of team Role ARN to map of account name to terraform action role ARN to assume For each team in `aws-teams`, look at every account and see if that team has access to the designated "apply" role. If so, add an entry ` = "apply"` to the `terraform_access_map` entry for that team. If not, see if it has access to the "plan" role, and if so, add a "plan" entry. Otherwise, no entry is added.
`terraform_dynamic_role_enabled`
True if dynamic role for Terraform is enabled
`terraform_profiles`
A list of all SSO profiles used to run terraform updates
`terraform_role_name_map`
Mapping of Terraform action (plan or apply) to aws-team-role name to assume for that action
`terraform_roles`
A list of all IAM roles used to run terraform updates
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `aws`, version: `>= 4.9.0` - `local`, version: `>= 1.3` - `utils`, version: `>= 1.10.0` ### Providers - `aws`, version: `>= 4.9.0` - `local`, version: `>= 1.3` - `utils`, version: `>= 1.10.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `accounts` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `atmos` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`local_file.account_info`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) ## Data Sources The following data sources are used by this module: - [`aws_organizations_organization.organization`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/organizations_organization) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`utils_describe_stacks.team_roles`](https://registry.terraform.io/providers/cloudposse/utils/latest/docs/data-sources/describe_stacks) (data source) - [`utils_describe_stacks.teams`](https://registry.terraform.io/providers/cloudposse/utils/latest/docs/data-sources/describe_stacks) (data source) --- ## iam-roles # Submodule `iam-roles` This submodule is used by other modules to determine which IAM Roles or AWS CLI Config Profiles to use for various tasks, most commonly for applying Terraform plans. ## Special Configuration Needed In order to avoid having to pass customization information through every module that uses this submodule, if the default configuration does not suit your needs, you are expected to add `variables_override.tf` to override the variables with the defaults you want to use in your project. For example, if you are not using "core" as the `tenant` portion of your "root" account (your Organization Management Account), then you should include the `variable "overridable_global_tenant_name"` declaration in your `variables_override.tf` so that `overridable_global_tenant_name` defaults to the value you are using (or the empty string if you are not using `tenant` at all). ## Variables ### Required Variables
### Optional Variables
`overridable_global_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
`overridable_global_stage_name` (`string`) optional
The stage name for the organization management account (where the `account-map` state is stored) **Default value:** `"root"`
`overridable_global_tenant_name` (`string`) optional
The tenant name used for organization-wide resources **Default value:** `"core"`
`privileged` (`bool`) optional
True if the Terraform user already has access to the backend **Default value:** `false`
`profiles_enabled` (`bool`) optional
Whether or not to use profiles instead of roles for Terraform. Default (null) means to use global settings. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`audit_terraform_profile_name`
The AWS config profile name for Terraform to use to provision resources in the "audit" role account, when profiles are in use
`audit_terraform_role_arn`
The AWS Role ARN for Terraform to use to provision resources in the "audit" role account, when Role ARNs are in use
`aws_partition`
The AWS "partition" to use when constructing resource ARNs
`current_account_account_name`
The account name (usually `-`) for the account configured by this module's inputs. Roughly analogous to `data "aws_caller_identity"`, but returning the name of the caller account as used in our configuration.
`dns_terraform_profile_name`
The AWS config profile name for Terraform to use to provision DNS Zone delegations, when profiles are in use
`dns_terraform_role_arn`
The AWS Role ARN for Terraform to use to provision DNS Zone delegations, when Role ARNs are in use
`global_environment_name`
The `null-label` `environment` value used for regionless (global) resources
`global_stage_name`
The `null-label` `stage` value for the organization management account (where the `account-map` state is stored)
`global_tenant_name`
The `null-label` `tenant` value used for organization-wide resources
`identity_account_account_name`
The account name (usually `-`) for the account holding primary IAM roles
`identity_terraform_profile_name`
The AWS config profile name for Terraform to use to provision resources in the "identity" role account, when profiles are in use
`identity_terraform_role_arn`
The AWS Role ARN for Terraform to use to provision resources in the "identity" role account, when Role ARNs are in use
`org_role_arn`
The AWS Role ARN for Terraform to use when SuperAdmin is provisioning resources in the account
`profiles_enabled`
When true, use AWS config profiles in Terraform AWS provider configurations. When false, use Role ARNs.
`terraform_profile_name`
The AWS config profile name for Terraform to use when provisioning resources in the account, when profiles are in use
`terraform_role_arn`
The AWS Role ARN for Terraform to use when provisioning resources in the account, when Role ARNs are in use
`terraform_role_arns`
All of the terraform role arns
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `awsutils`, version: `>= 0.16.0` ### Providers - `awsutils`, version: `>= 0.16.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `always` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`awsutils_caller_identity.current`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/data-sources/caller_identity) (data source) None --- ## roles-to-principals # Submodule `roles-to-principals` This submodule is used by other modules to map short role names and AWS SSO Permission Set names in accounts designated by short account names (for example, `terraform` in the `dev` account) to full IAM Role ARNs and other related tasks. ## Special Configuration Needed As with `iam-roles`, in order to avoid having to pass customization information through every module that uses this submodule, if the default configuration does not suit your needs, you are expected to add `variables_override.tf` to override the variables with the defaults you want to use in your project. For example, if you are not using "core" as the `tenant` portion of your "root" account (your Organization Management Account), then you should include the `variable "overridable_global_tenant_name"` declaration in your `variables_override.tf` so that `overridable_global_tenant_name` defaults to the value you are using (or the empty string if you are not using `tenant` at all). ## Variables ### Required Variables
### Optional Variables
`overridable_global_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
`overridable_global_stage_name` (`string`) optional
The stage name for the organization management account (where the `account-map` state is stored) **Default value:** `"root"`
`overridable_global_tenant_name` (`string`) optional
The tenant name used for organization-wide resources **Default value:** `"core"`
`overridable_team_permission_set_name_pattern` (`string`) optional
The pattern used to generate the AWS SSO PermissionSet name for each team **Default value:** `"Identity%sTeamAccess"`
`overridable_team_permission_sets_enabled` (`bool`) optional
When true, any roles (teams or team-roles) in the identity account references in `role_map` will cause corresponding AWS SSO PermissionSets to be included in the `permission_set_arn_like` output. This has the effect of treating those PermissionSets as if they were teams. The main reason to set this `false` is if IAM trust policies are exceeding size limits and you are not using AWS SSO. **Default value:** `true`
`permission_set_map` (`map(list(string))`) optional
Map of account:[PermissionSet, PermissionSet...] specifying AWS SSO PermissionSets when accessed from specified accounts **Default value:** `{ }`
`privileged` (`bool`) optional
True if the default provider already has access to the backend **Default value:** `false`
`role_map` (`map(list(string))`) optional
Map of account:[role, role...]. Use `*` as role for entire account **Default value:** `{ }`
`teams` (`list(string)`) optional
List of team names to translate to AWS SSO PermissionSet names **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_partition`
The AWS "partition" to use when constructing resource ARNs
`full_account_map`
Map of account names to account IDs
`permission_set_arn_like`
List of Role ARN regexes suitable for IAM Condition `ArnLike` corresponding to given input `permission_set_map`
`principals`
Consolidated list of AWS principals corresponding to given input `role_map`
`principals_map`
Map of AWS principals corresponding to given input `role_map`
`team_permission_set_name_map`
Map of team names (from `var.teams` and `role_map["identity"]) to permission set names
## Dependencies ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `always` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a None --- ## team-assume-role-policy This submodule generates a JSON-encoded IAM Policy Document suitable for use as an "Assume Role Policy". You can designate both who is allowed to assume a role and who is explicitly denied permission to assume a role. The value of this submodule is that it allows for many ways to specify the "who" while at the same time limiting the "who" to assumed IAM roles: - All assumed roles in the `dev` account: `allowed_roles = { dev = ["*"] }` - Only the `admin` role in the dev account: `allowed_roles = { dev = ["admin"] }` - A specific principal in any account (though it must still be an assumed role): `allowed_principal_arns = arn:aws:iam::123456789012:role/trusted-role` - A user of a specific AWS SSO Permission Set: `allowed_permission_sets = { dev = ["DeveloperAccess"] }` ## Usage ```hcl module "assume_role" { source = "../account-map/modules/team-assume-role-policy" allowed_roles = { dev = ["admin"] } context = module.this.context } resource "aws_iam_role" "default" { assume_role_policy = module.assume_role.policy_document # ... } ``` ## Variables ### Required Variables
### Optional Variables
`allowed_permission_sets` (`map(list(string))`) optional
Map of account:[PermissionSet, PermissionSet...] specifying AWS SSO PermissionSets allowed to assume the role when coming from specified account **Default value:** `{ }`
`allowed_principal_arns` (`list(string)`) optional
List of AWS principal ARNs allowed to assume the role. **Default value:** `[ ]`
`allowed_roles` (`map(list(string))`) optional
Map of account:[role, role...] specifying roles allowed to assume the role. Roles are symbolic names like `ops` or `terraform`. Use `*` as role for entire account. **Default value:** `{ }`
`denied_permission_sets` (`map(list(string))`) optional
Map of account:[PermissionSet, PermissionSet...] specifying AWS SSO PermissionSets denied access to the role when coming from specified account **Default value:** `{ }`
`denied_principal_arns` (`list(string)`) optional
List of AWS principal ARNs explicitly denied access to the role. **Default value:** `[ ]`
`denied_roles` (`map(list(string))`) optional
Map of account:[role, role...] specifying roles explicitly denied permission to assume the role. Roles are symbolic names like `ops` or `terraform`. Use `*` as role for entire account. **Default value:** `{ }`
`global_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
`iam_users_enabled` (`bool`) optional
True if you would like IAM Users to be able to assume the role. **Default value:** `false`
`privileged` (`bool`) optional
True if the default provider already has access to the backend **Default value:** `false`
`trusted_github_org` (`string`) optional
The GitHub organization unqualified repos are assumed to belong to. Keeps `*` from meaning all orgs and all repos. **Default value:** `"cloudposse"`
`trusted_github_repos` (`list(string)`) optional
A list of GitHub repositories allowed to access this role. Format is either "orgName/repoName" or just "repoName", in which case "cloudposse" will be used for the "orgName". Wildcard ("*") is allowed for "repoName". **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`github_assume_role_policy`
JSON encoded string representing the "Assume Role" policy configured by the inputs
`policy_document`
JSON encoded string representing the "Assume Role" policy configured by the inputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `aws`, version: `>= 4.9.0` ### Providers - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `allowed_role_map` | latest | [`../roles-to-principals`](https://registry.terraform.io/modules/../roles-to-principals/) | n/a `denied_role_map` | latest | [`../roles-to-principals`](https://registry.terraform.io/modules/../roles-to-principals/) | n/a `github_oidc_provider` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_arn.allowed`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) (data source) - [`aws_arn.denied`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.github_oidc_provider_assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) None --- ## account-quotas This component is responsible for requesting AWS Service Quota increases. We recommend making requests here rather than in `account-settings` because `account-settings` is a restricted component that can only be applied by SuperAdmin. ## Usage **Stack Level**: Global and Regional (depending on quota) Global resources must be provisioned in `us-east-1`. Put them in the `gbl` stack, but set `region: us-east-1` in the `vars` section. You can refer to services either by their exact full name (e.g. `service_name: "Amazon Elastic Compute Cloud (Amazon EC2)"`) or by the service code (e.g. `service_code: "ec2"`). Similarly, you can refer to quota names either by their exact full name (e.g. `quota_name: "EC2-VPC Elastic IPs"`) or by the quota code (e.g. `quota_code: "L-0263D0A3"`). You can find service codes and full names via the AWS CLI (be sure to use the correct region): ```bash aws --region us-east-1 service-quotas list-services ``` You can find quota codes and full names, and also whether the quotas are adjustable or global, via the AWS CLI, but you will need the service code from the previous step: ```bash aws --region us-east-1 service-quotas list-service-quotas --service-code ec2 ``` If you make a request to raise a quota, the output will show the requested value as `value` while the request is pending. ### Special Usage Notes Even though the Terraform will submit the support request, you may need to follow up with AWS Support to get the request approved, via the AWS Console or email. #### Resources are destroyed on change Because the AWS API often returns default values rather than configured or applicable values for a given quota, we must ignore the value returned by the API or else face perpetual drift. To allow us to change the value in the future, even though we are ignoring it, we encode the value in the resource key, so that a change of value will result in a new resource being created and the old one being destroyed. Destroying the old resource has no actual effect (it does not even close an open request), so it is safe to do. ### Example Here's an example snippet for how to use this component. ```yaml components: terraform: account-quotas: vars: quotas: vpcs-per-region: service_code: vpc quota_name: "VPCs per Region" value: 10 vpc-elastic-ips: service_code: ec2 quota_name: "EC2-VPC Elastic IPs" value: 10 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`quotas` optional
Map of quotas to set. Map keys are arbitrary and are used to allow Atmos to merge configurations. Delete an inherited quota by setting its key's value to null. You only need to provide one of either name or code for each of "service" and "quota". If you provide both, the code will be used. **Type:** ```hcl map(object({ service_name = optional(string) service_code = optional(string) quota_name = optional(string) quota_code = optional(string) value = number })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`quotas`
Full report on all service quotas managed by this component.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_servicequotas_service_quota.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/servicequotas_service_quota) (resource) ## Data Sources The following data sources are used by this module: - [`aws_servicequotas_service.by_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/servicequotas_service) (data source) - [`aws_servicequotas_service_quota.by_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/servicequotas_service_quota) (data source) --- ## account-settings This component is responsible for provisioning account level settings: IAM password policy, AWS Account Alias, EBS encryption, and Service Quotas. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. It's suggested to apply this component to all accounts, so create a file `stacks/catalog/account-settings.yaml` with the following content and then import that file in each account's global stack (overriding any parameters as needed): ```yaml components: terraform: account-settings: vars: enabled: true minimum_password_length: 20 maximum_password_age: 120 budgets_enabled: true budgets_notifications_enabled: true budgets_slack_webhook_url: https://url.slack.com/abcd/1234 budgets_slack_username: AWS Budgets budgets_slack_channel: aws-budgets-notifications budgets: - name: 1000-total-monthly budget_type: COST limit_amount: "1000" limit_unit: USD time_unit: MONTHLY - name: s3-3GB-limit-monthly budget_type: USAGE limit_amount: "3" limit_unit: GB time_unit: MONTHLY notification: - comparison_operator: GREATER_THAN notification_type: FORECASTED threshold_type: PERCENTAGE threshold: 80 subscribers: - slack - comparison_operator: GREATER_THAN notification_type: FORECASTED threshold_type: PERCENTAGE # We generate two forecast notifications. This makes sure that notice is taken, # and hopefully action can be taken to prevent going over budget. threshold: 100 subscribers: - slack - comparison_operator: GREATER_THAN notification_type: ACTUAL threshold_type: PERCENTAGE threshold: 100 subscribers: - slack service_quotas_enabled: true service_quotas: - quota_name: Subnets per VPC service_code: vpc value: 250 - quota_code: L-E79EC296 # aka `Security Groups per Region` service_code: vpc value: 500 - quota_code: L-F678F1CE # aka `VPC per Region` service_code: vpc value: null ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`budgets` (`any`) optional
A list of Budgets to be managed by this module, see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/budgets_budget#argument-reference for a list of possible attributes. For a more specific example, see `https://github.com/cloudposse/terraform-aws-budgets/blob/master/examples/complete/fixtures.us-east-2.tfvars`. **Default value:** `[ ]`
`budgets_enabled` (`bool`) optional
Whether or not this component should manage AWS Budgets **Default value:** `false`
`budgets_notifications_enabled` (`bool`) optional
Whether or not to setup Slack notifications for Budgets. Set to `true` to create an SNS topic and Lambda function to send alerts to a Slack channel. **Default value:** `false`
`budgets_slack_channel` (`string`) optional
The name of the channel in Slack for notifications. Only used when `budgets_notifications_enabled` is `true` **Default value:** `""`
`budgets_slack_username` (`string`) optional
The username that will appear on Slack messages. Only used when `budegets_notifications_enabled` is `true` **Default value:** `""`
`budgets_slack_webhook_url` (`string`) optional
The URL of Slack webhook. Only used when `budgets_notifications_enabled` is `true` **Default value:** `""`
`maximum_password_age` (`number`) optional
The number of days that an user password is valid **Default value:** `190`
`minimum_password_length` (`string`) optional
Minimum number of characters allowed in an IAM user password. Integer between 6 and 128, per https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_passwords_account-policy.html **Default value:** `14`
`service_quotas` (`list(any)`) optional
A list of service quotas to manage or lookup. To lookup the value of a service quota, set `value = null` and either `quota_code` or `quota_name`. To manage a service quota, set `value` to a number. Service Quotas can only be managed via `quota_code`. For a more specific example, see https://github.com/cloudposse/terraform-aws-service-quotas/blob/master/examples/complete/fixtures.us-east-2.tfvars. **Default value:** `[ ]`
`service_quotas_enabled` (`bool`) optional
Whether or not this component should handle Service Quotas **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`account_alias`
Account alias
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `budgets` | 0.7.1 | [`cloudposse/budgets/aws`](https://registry.terraform.io/modules/cloudposse/budgets/aws/0.7.1) | n/a `iam_account_settings` | 0.5.0 | [`cloudposse/iam-account-settings/aws`](https://registry.terraform.io/modules/cloudposse/iam-account-settings/aws/0.5.0) | IAM Account Settings provides a CIS-compliant IAM Password Policy out of the box It also sets the account alias for the current account. `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `service_quotas` | 0.1.0 | [`cloudposse/service-quotas/aws`](https://registry.terraform.io/modules/cloudposse/service-quotas/aws/0.1.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ebs_encryption_by_default.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ebs_encryption_by_default) (resource) ## Data Sources The following data sources are used by this module: --- ## acm This component is responsible for requesting an ACM certificate for a domain and adding a CNAME record to the DNS zone to complete certificate validation. The ACM component is to manage an unlimited number of certificates, predominantly for vanity domains. While the [dns-primary](https://docs.cloudposse.com/components/library/aws/dns-primary) component has the ability to generate ACM certificates, it is very opinionated and can only manage one zone. In reality, companies have many branded domains associated with a load balancer, so we need to be able to generate more complicated certificates. We have, as a convenience, the ability to create an ACM certificate as part of creating a DNS zone, whether primary or delegated. That convenience is limited to creating `example.com` and `*.example.com` when creating a zone for `example.com`. For example, Acme has delegated `acct.acme.com` and in addition to `*.acct.acme.com` needed an ACM certificate for `*.usw2.acct.acme.com`, so we use the ACM component to provision that, rather than extend the DNS primary or delegated components to take a list of additional certificates. Both are different views on the Single Responsibility Principle. ## Usage **Stack Level**: Global or Regional Here's an example snippet for how to use this component. ```yaml components: terraform: acm: settings: spacelift: workspace_enabled: true vars: enabled: true domain_name: acme.com process_domain_validation_options: false validation_method: DNS # NOTE: The following subject alternative name is automatically added by the module. # Additional entries can be added by providing this input. # subject_alternative_names: # - "*.acme.com" ``` ACM using a private CA ```yaml components: terraform: acm: settings: spacelift: workspace_enabled: true vars: enabled: true domain_name: acme.com process_domain_validation_options: false dns_private_zone_enabled: true certificate_authority_component_name: private-ca-subordinate certificate_authority_stage_name: pca certificate_authority_environment_name: use2 certificate_authority_component_key: subordinate ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`certificate_authority_component_key` (`string`) optional
Use this component key e.g. `root` or `mgmt` to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_component_name` (`string`) optional
Use this component name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_enabled` (`bool`) optional
Whether to use the certificate authority or not **Default value:** `false`
`certificate_authority_environment_name` (`string`) optional
Use this environment name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_stage_name` (`string`) optional
Use this stage name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_export` (`bool`) optional
Specifies whether the certificate can be exported **Default value:** `false`
`dns_delegated_component_name` (`string`) optional
Use this component name to read from the remote state to get the dns_delegated zone ID **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
Use this environment name to read from the remote state to get the dns_delegated zone ID **Default value:** `"gbl"`
`dns_delegated_stage_name` (`string`) optional
Use this stage name to read from the remote state to get the dns_delegated zone ID **Default value:** `null`
`dns_private_zone_enabled` (`bool`) optional
Whether to set the zone to public or private **Default value:** `false`
`domain_name` (`string`) optional
Root domain name **Default value:** `""`
`domain_name_prefix` (`string`) optional
Root domain name prefix to use with DNS delegated remote state **Default value:** `""`
`process_domain_validation_options` (`bool`) optional
Flag to enable/disable processing of the record to add to the DNS zone to complete certificate validation **Default value:** `false`
`subject_alternative_names` (`list(string)`) optional
A list of domains that should be SANs in the issued certificate **Default value:** `[ ]`
`subject_alternative_names_prefixes` (`list(string)`) optional
A list of domain prefixes to use with DNS delegated remote state that should be SANs in the issued certificate **Default value:** `[ ]`
`validation_method` (`string`) optional
Method to use for validation, DNS or EMAIL **Default value:** `"DNS"`
`zone_name` (`string`) optional
Name of the zone in which to place the DNS validation records to validate the certificate. Typically a domain name. Default of `""` actually defaults to `domain_name`. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the certificate
`domain_name`
Certificate domain name
`domain_validation_options`
CNAME records that are added to the DNS zone to complete certificate validation
`id`
The ID of the certificate
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 6.4.0, < 7.0.0` ### Providers - `aws`, version: `>= 6.4.0, < 7.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 0.18.1 | [`cloudposse/acm-request-certificate/aws`](https://registry.terraform.io/modules/cloudposse/acm-request-certificate/aws/0.18.1) | https://github.com/cloudposse/terraform-aws-acm-request-certificate `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `private_ca` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.acm_arn`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## alb This component is responsible for provisioning a generic Application Load Balancer. It depends on the `vpc` and `dns-delegated` components. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: alb: vars: https_ssl_policy: ELBSecurityPolicy-FS-1-2-Res-2020-10 health_check_path: /api/healthz ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_logs_enabled` (`bool`) optional
A boolean flag to enable/disable access_logs **Default value:** `true`
`access_logs_prefix` (`string`) optional
The S3 log bucket prefix **Default value:** `""`
`access_logs_s3_bucket_id` (`string`) optional
An external S3 Bucket name to store access logs in. If specified, no logging bucket will be created. **Default value:** `null`
`acm_component_name` (`string`) optional
Atmos `acm` component name **Default value:** `"acm"`
`alb_access_logs_s3_bucket_force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the ALB access logs S3 bucket so that the bucket can be destroyed without error **Default value:** `false`
`cross_zone_load_balancing_enabled` (`bool`) optional
A boolean flag to enable/disable cross zone load balancing **Default value:** `true`
`deletion_protection_enabled` (`bool`) optional
A boolean flag to enable/disable deletion protection for ALB **Default value:** `false`
`deregistration_delay` (`number`) optional
The amount of time to wait in seconds before changing the state of a deregistering target to unused **Default value:** `15`
`dns_acm_enabled` (`bool`) optional
If `true`, use the ACM ARN created by the given `dns-delegated` component. Otherwise, use the ACM ARN created by the given `acm` component. **Default value:** `false`
`dns_delegated_component_name` (`string`) optional
Atmos `dns-delegated` component name **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
`dns-delegated` component environment name **Default value:** `null`
`drop_invalid_header_fields` (`bool`) optional
Indicates whether HTTP headers with header fields that are not valid are removed by the load balancer (true) or routed to targets (false). **Default value:** `false`
`health_check_healthy_threshold` (`number`) optional
The number of consecutive health checks successes required before considering an unhealthy target healthy **Default value:** `2`
`health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `15`
`health_check_matcher` (`string`) optional
The HTTP response codes to indicate a healthy check **Default value:** `"200-399"`
`health_check_path` (`string`) optional
The destination for the health check request **Default value:** `"/"`
`health_check_port` (`string`) optional
The port to use for the healthcheck **Default value:** `"traffic-port"`
`health_check_timeout` (`number`) optional
The amount of time to wait in seconds before failing a health check request **Default value:** `10`
`health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before considering the target unhealthy **Default value:** `2`
`http2_enabled` (`bool`) optional
A boolean flag to enable/disable HTTP/2 **Default value:** `true`
`http_enabled` (`bool`) optional
A boolean flag to enable/disable HTTP listener **Default value:** `true`
`http_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in HTTP security group **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`http_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to HTTP ingress security group **Default value:** `[ ]`
`http_port` (`number`) optional
The port for the HTTP listener **Default value:** `80`
`http_redirect` (`bool`) optional
A boolean flag to enable/disable HTTP redirect to HTTPS **Default value:** `true`
`https_enabled` (`bool`) optional
A boolean flag to enable/disable HTTPS listener **Default value:** `true`
`https_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in HTTPS security group **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`https_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to HTTPS ingress security group **Default value:** `[ ]`
`https_port` (`number`) optional
The port for the HTTPS listener **Default value:** `443`
`https_ssl_policy` (`string`) optional
The name of the SSL Policy for the listener **Default value:** `"ELBSecurityPolicy-TLS13-1-2-2021-06"`
`idle_timeout` (`number`) optional
The time in seconds that the connection is allowed to be idle **Default value:** `60`
`internal` (`bool`) optional
A boolean flag to determine whether the ALB should be internal **Default value:** `false`
`ip_address_type` (`string`) optional
The type of IP addresses used by the subnets for your load balancer. The possible values are `ipv4` and `dualstack`. **Default value:** `"ipv4"`
`lifecycle_rule_enabled` (`bool`) optional
A boolean that indicates whether the s3 log bucket lifecycle rule should be enabled. **Default value:** `true`
`stickiness` optional
Target group sticky configuration **Type:** ```hcl object({ cookie_duration = number enabled = bool }) ``` **Default value:** `null`
`target_group_name` (`string`) optional
The name for the default target group, uses a module label name if left empty **Default value:** `""`
`target_group_port` (`number`) optional
The port for the default target group **Default value:** `80`
`target_group_protocol` (`string`) optional
The protocol for the default target group HTTP or HTTPS **Default value:** `"HTTP"`
`target_group_target_type` (`string`) optional
The type (`instance`, `ip` or `lambda`) of targets that can be registered with the target group **Default value:** `"ip"`
`vpc_component_name` (`string`) optional
Atmos `vpc` component name **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_logs_bucket_id`
The S3 bucket ID for access logs
`alb_arn`
The ARN of the ALB
`alb_arn_suffix`
The ARN suffix of the ALB
`alb_dns_name`
DNS name of ALB
`alb_name`
The ARN suffix of the ALB
`alb_zone_id`
The ID of the zone which ALB is provisioned
`default_target_group_arn`
The default target group ARN
`http_listener_arn`
The ARN of the HTTP forwarding listener
`http_redirect_listener_arn`
The ARN of the HTTP to HTTPS redirect listener
`https_listener_arn`
The ARN of the HTTPS listener
`listener_arns`
A list of all the listener ARNs
`security_group_id`
The security group ID of the ALB
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `local`, version: `>= 2.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `alb` | 2.3.1 | [`cloudposse/alb/aws`](https://registry.terraform.io/modules/cloudposse/alb/aws/2.3.1) | n/a `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## amplify This component is responsible for provisioning AWS Amplify apps, backend environments, branches, domain associations, and webhooks. ## Usage **Stack Level**: Regional Here's an example for how to use this component: ```yaml # stacks/catalog/amplify/defaults.yaml components: terraform: amplify/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true # https://docs.aws.amazon.com/amplify/latest/userguide/setting-up-GitHub-access.html github_personal_access_token_secret_path: "/amplify/github_personal_access_token" platform: "WEB" enable_auto_branch_creation: false enable_basic_auth: false enable_branch_auto_build: true enable_branch_auto_deletion: false iam_service_role_enabled: false environment_variables: {} dns_delegated_component_name: "dns-delegated" dns_delegated_environment_name: "gbl" ``` ```yaml # stacks/catalog/amplify/example.yaml import: - catalog/amplify/defaults components: terraform: amplify/example: metadata: # Point to the Terraform component component: amplify inherits: # Inherit the default settings - amplify/defaults vars: name: "example" description: "example Amplify App" repository: "https://github.com/cloudposse/amplify-test2" platform: "WEB_COMPUTE" enable_auto_branch_creation: false enable_basic_auth: false enable_branch_auto_build: true enable_branch_auto_deletion: false iam_service_role_enabled: true # https://docs.aws.amazon.com/amplify/latest/userguide/ssr-CloudWatch-logs.html iam_service_role_actions: - "logs:CreateLogStream" - "logs:CreateLogGroup" - "logs:DescribeLogGroups" - "logs:PutLogEvents" custom_rules: [] auto_branch_creation_patterns: [] environment_variables: NEXT_PRIVATE_STANDALONE: false NEXT_PUBLIC_TEST: test _LIVE_UPDATES: '[{"pkg":"node","type":"nvm","version":"16"},{"pkg":"next-version","type":"internal","version":"13.1.1"}]' environments: main: branch_name: "main" enable_auto_build: true backend_enabled: false enable_performance_mode: false enable_pull_request_preview: false framework: "Next.js - SSR" stage: "PRODUCTION" environment_variables: {} develop: branch_name: "develop" enable_auto_build: true backend_enabled: false enable_performance_mode: false enable_pull_request_preview: false framework: "Next.js - SSR" stage: "DEVELOPMENT" environment_variables: {} domain_config: enable_auto_sub_domain: false wait_for_verification: false sub_domain: - branch_name: "main" prefix: "example-prod" - branch_name: "develop" prefix: "example-dev" subdomains_dns_records_enabled: true certificate_verification_dns_record_enabled: false ``` The `amplify/example` YAML configuration defines an Amplify app in AWS. The app is set up to use the `Next.js` framework with SSR (server-side rendering) and is linked to the GitHub repository "https://github.com/cloudposse/amplify-test2". The app is set up to have two environments: `main` and `develop`. Each environment has different configuration settings, such as the branch name, framework, and stage. The `main` environment is set up for production, while the `develop` environments is set up for development. The app is also configured to have custom subdomains for each environment, with prefixes such as `example-prod` and `example-dev`. The subdomains are configured to use DNS records, which are enabled through the `subdomains_dns_records_enabled` variable. The app also has an IAM service role configured with specific IAM actions, and environment variables set up for each environment. Additionally, the app is configured to use the Atmos Spacelift workspace, as indicated by the `workspace_enabled: true` setting. The `amplify/example` Atmos component extends the `amplify/defaults` component. The `amplify/example` configuration is imported into the `stacks/mixins/stage/dev.yaml` stack config file to be provisioned in the `dev` account. ```yaml # stacks/mixins/stage/dev.yaml import: - catalog/amplify/example ``` You can execute the following command to provision the Amplify app using Atmos: ```shell atmos terraform apply amplify/example -s ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`auto_branch_creation_config` optional
The automated branch creation configuration for the Amplify app **Type:** ```hcl object({ basic_auth_credentials = optional(string) build_spec = optional(string) enable_auto_build = optional(bool) enable_basic_auth = optional(bool) enable_performance_mode = optional(bool) enable_pull_request_preview = optional(bool) environment_variables = optional(map(string)) framework = optional(string) pull_request_environment_name = optional(string) stage = optional(string) }) ``` **Default value:** `null`
`auto_branch_creation_patterns` (`list(string)`) optional
The automated branch creation glob patterns for the Amplify app **Default value:** `[ ]`
`basic_auth_credentials` (`string`) optional
The credentials for basic authorization for the Amplify app **Default value:** `null`
`build_spec` (`string`) optional
The [build specification](https://docs.aws.amazon.com/amplify/latest/userguide/build-settings.html) (build spec) for the Amplify app. If not provided then it will use the `amplify.yml` at the root of your project / branch. **Default value:** `null`
`certificate_verification_dns_record_enabled` (`bool`) optional
Whether or not to create DNS records for SSL certificate validation. If using the DNS zone from `dns-delegated`, the SSL certificate is already validated, and this variable must be set to `false`. **Default value:** `false`
`custom_rules` optional
The custom rules to apply to the Amplify App **Type:** ```hcl list(object({ condition = optional(string) source = string status = optional(string) target = string })) ``` **Default value:** `[ ]`
`description` (`string`) optional
The description for the Amplify app **Default value:** `null`
`dns_delegated_component_name` (`string`) optional
The component name of `dns-delegated` **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
The environment name of `dns-delegated` **Default value:** `"gbl"`
`domain_config` optional
Amplify custom domain configuration **Type:** ```hcl object({ domain_name = optional(string) enable_auto_sub_domain = optional(bool, false) wait_for_verification = optional(bool, false) sub_domain = list(object({ branch_name = string prefix = string })) }) ``` **Default value:** `null`
`enable_auto_branch_creation` (`bool`) optional
Enables automated branch creation for the Amplify app **Default value:** `false`
`enable_basic_auth` (`bool`) optional
Enables basic authorization for the Amplify app. This will apply to all branches that are part of this app. **Default value:** `false`
`enable_branch_auto_build` (`bool`) optional
Enables auto-building of branches for the Amplify App **Default value:** `true`
`enable_branch_auto_deletion` (`bool`) optional
Automatically disconnects a branch in the Amplify Console when you delete a branch from your Git repository **Default value:** `false`
`environment_variables` (`map(string)`) optional
The environment variables for the Amplify app **Default value:** `{ }`
`environments` optional
The configuration of the environments for the Amplify App **Type:** ```hcl map(object({ branch_name = optional(string) backend_enabled = optional(bool, false) environment_name = optional(string) deployment_artifacts = optional(string) stack_name = optional(string) display_name = optional(string) description = optional(string) enable_auto_build = optional(bool) enable_basic_auth = optional(bool) enable_notification = optional(bool) enable_performance_mode = optional(bool) enable_pull_request_preview = optional(bool) environment_variables = optional(map(string)) framework = optional(string) pull_request_environment_name = optional(string) stage = optional(string) ttl = optional(number) webhook_enabled = optional(bool, false) })) ``` **Default value:** `{ }`
`github_personal_access_token_secret_path` (`string`) optional
Path to the GitHub personal access token in AWS Parameter Store **Default value:** `"/amplify/github_personal_access_token"`
`iam_service_role_actions` (`list(string)`) optional
List of IAM policy actions for the AWS Identity and Access Management (IAM) service role for the Amplify app. If not provided, the default set of actions will be used for the role if the variable `iam_service_role_enabled` is set to `true`. **Default value:** `[ ]`
`iam_service_role_arn` (`list(string)`) optional
The AWS Identity and Access Management (IAM) service role for the Amplify app. If not provided, a new role will be created if the variable `iam_service_role_enabled` is set to `true`. **Default value:** `[ ]`
`iam_service_role_enabled` (`bool`) optional
Flag to create the IAM service role for the Amplify app **Default value:** `false`
`oauth_token` (`string`) optional
The OAuth token for a third-party source control system for the Amplify app. The OAuth token is used to create a webhook and a read-only deploy key. The OAuth token is not stored. **Default value:** `null`
`platform` (`string`) optional
The platform or framework for the Amplify app **Default value:** `"WEB"`
`repository` (`string`) optional
The repository for the Amplify app **Default value:** `null`
`subdomains_dns_records_enabled` (`bool`) optional
Whether or not to create DNS records for the Amplify app custom subdomains **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amplify App ARN
`backend_environments`
Created backend environments
`branch_names`
The names of the created Amplify branches
`default_domain`
Amplify App domain (non-custom)
`domain_association_arn`
ARN of the domain association
`domain_association_certificate_verification_dns_record`
The DNS record for certificate verification
`domain_associations`
Domain associations
`name`
Amplify App name
`sub_domains`
DNS records and the verified status for the subdomains
`webhooks`
Created webhooks
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `amplify_app` | 1.2.0 | [`cloudposse/amplify-app/aws`](https://registry.terraform.io/modules/cloudposse/amplify-app/aws/1.2.0) | n/a `certificate_verification_dns_record` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | Create the SSL certificate validation record `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `subdomains_dns_record` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | Create DNS records for the subdomains `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.github_pat`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## api-gateway-account-settings This component is responsible for setting the global, regional settings required to allow API Gateway to write to CloudWatch logs. Every AWS region you want to deploy an API Gateway to must be configured with an IAM Role that gives API Gateway permissions to create and write to CloudWatch logs. Without this configuration, API Gateway will not be able to send logs to CloudWatch. This configuration is done once per region regardless of the number of API Gateways deployed in that region. This module creates an IAM role, assigns it the necessary permissions to write logs and sets it as the "CloudWatch log role ARN" in the API Gateway configuration. ## Usage **Stack Level**: Regional The following is a snippet for how to use this component: ```yaml components: terraform: api-gateway-account-settings: settings: spacelift: workspace_enabled: true vars: enabled: true tags: Service: api-gateway ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role_arn`
Role ARN of the API Gateway logging role
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `api_gateway_account_settings` | 0.9.0 | [`cloudposse/api-gateway/aws//modules/account-settings`](https://registry.terraform.io/modules/cloudposse/api-gateway/aws/modules/account-settings/0.9.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## api-gateway-rest-api This component is responsible for deploying an API Gateway REST API. ## Usage **Stack Level**: Regional The following is a snippet for how to use this component: ```yaml components: terraform: api-gateway-rest-api: vars: enabled: true name: api openapi_config: openapi: 3.0.1 info: title: Example API Gateway version: 1.0.0 paths: "/": get: x-amazon-apigateway-integration: httpMethod: GET payloadFormatVersion: 1.0 type: HTTP_PROXY uri: https://api.ipify.org "/{proxy+}": get: x-amazon-apigateway-integration: httpMethod: GET payloadFormatVersion: 1.0 type: HTTP_PROXY uri: https://api.ipify.org ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_log_format` (`string`) optional
The format of the access log file. **Default value:** `" {\n \"requestTime\": \"$context.requestTime\",\n \"requestId\": \"$context.requestId\",\n \"httpMethod\": \"$context.httpMethod\",\n \"path\": \"$context.path\",\n \"resourcePath\": \"$context.resourcePath\",\n \"status\": $context.status,\n \"responseLatency\": $context.responseLatency,\n \"xrayTraceId\": \"$context.xrayTraceId\",\n \"integrationRequestId\": \"$context.integration.requestId\",\n \"functionResponseStatus\": \"$context.integration.status\",\n \"integrationLatency\": \"$context.integration.latency\",\n \"integrationServiceStatus\": \"$context.integration.integrationStatus\",\n \"authorizeResultStatus\": \"$context.authorize.status\",\n \"authorizerServiceStatus\": \"$context.authorizer.status\",\n \"authorizerLatency\": \"$context.authorizer.latency\",\n \"authorizerRequestId\": \"$context.authorizer.requestId\",\n \"ip\": \"$context.identity.sourceIp\",\n \"userAgent\": \"$context.identity.userAgent\",\n \"principalId\": \"$context.authorizer.principalId\",\n \"cognitoUser\": \"$context.identity.cognitoIdentityId\",\n \"user\": \"$context.identity.user\"\n}\n"`
`deregistration_delay` (`number`) optional
The amount of time to wait in seconds before changing the state of a deregistering target to unused **Default value:** `15`
`enable_private_link_nlb` (`bool`) optional
A flag to indicate whether to enable private link. **Default value:** `false`
`enable_private_link_nlb_deletion_protection` (`bool`) optional
A flag to indicate whether to enable private link deletion protection. **Default value:** `false`
`endpoint_type` (`string`) optional
The type of the endpoint. One of - PUBLIC, PRIVATE, REGIONAL **Default value:** `"REGIONAL"`
`fully_qualified_domain_name` (`string`) optional
The fully qualified domain name of the API. **Default value:** `null`
`logging_level` (`string`) optional
The logging level of the API. One of - OFF, INFO, ERROR **Default value:** `"INFO"`
`metrics_enabled` (`bool`) optional
A flag to indicate whether to enable metrics collection. **Default value:** `true`
`openapi_config` (`any`) optional
The OpenAPI specification for the API **Default value:** `{ }`
`rest_api_policy` (`string`) optional
The IAM policy document for the API. **Default value:** `null`
`stage_name` (`string`) optional
The name of the stage **Default value:** `""`
`throttling_burst_limit` (`number`) optional
The API request burst limit **Default value:** `-1`
`throttling_rate_limit` (`number`) optional
The API request rate limit **Default value:** `-1`
`xray_tracing_enabled` (`bool`) optional
A flag to indicate whether to enable X-Ray tracing. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the REST API
`created_date`
The date the REST API was created
`execution_arn`
The execution ARN part to be used in lambda_permission's source_arn when allowing API Gateway to invoke a Lambda function, e.g., arn:aws:execute-api:eu-west-2:123456789012:z4675bid1j, which can be concatenated with allowed stage, method and resource path.The ARN of the Lambda function that will be executed.
`id`
The ID of the REST API
`invoke_url`
The URL to invoke the REST API
`root_resource_id`
The resource ID of the REST API's root
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `api_gateway_rest_api` | 0.9.0 | [`cloudposse/api-gateway/aws`](https://registry.terraform.io/modules/cloudposse/api-gateway/aws/0.9.0) | n/a `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `nlb` | 0.18.2 | [`cloudposse/nlb/aws`](https://registry.terraform.io/modules/cloudposse/nlb/aws/0.18.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_api_gateway_base_path_mapping.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_base_path_mapping) (resource) - [`aws_api_gateway_domain_name.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_domain_name) (resource) - [`aws_route53_record.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) ## Data Sources The following data sources are used by this module: - [`aws_acm_certificate.issued`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/acm_certificate) (data source) - [`aws_route53_zone.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## argocd-github-repo This component is responsible for creating and managing an ArgoCD desired state repository. ## Usage **Stack Level**: Regional The following are example snippets of how to use this component: ```yaml # stacks/argocd/repo/default.yaml components: terraform: argocd-repo: vars: enabled: true github_user: ci-acme github_user_email: ci@acme.com github_organization: ACME github_codeowner_teams: - "@ACME/cloud-admins" - "@ACME/cloud-posse" # the team must be present in the org where the repository lives # team_slug is the name of the team without the org # e.g. `@cloudposse/engineering` is just `engineering` permissions: - team_slug: admins permission: admin - team_slug: bots permission: admin - team_slug: engineering permission: push ``` ```yaml # stacks/argocd/repo/non-prod.yaml import: - catalog/argocd/repo/defaults components: terraform: argocd-deploy-non-prod: component: argocd-repo settings: spacelift: workspace_enabled: true vars: name: argocd-deploy-non-prod description: "ArgoCD desired state repository (Non-production) for ACME applications" environments: - tenant: mgmt environment: uw2 stage: sandbox ``` ```yaml # stacks/mgmt-gbl-corp.yaml import: --- - catalog/argocd/repo/non-prod ``` If the repository already exists, it will need to be imported (replace names of IAM profile var file accordingly): ```bash $ export TF_VAR_github_token_override=[REDACTED] $ atmos terraform varfile argocd-deploy-non-prod -s mgmt-gbl-corp $ cd components/terraform/argocd-repo $ terraform import -var "import_profile_name=eg-mgmt-gbl-corp-admin" -var-file="mgmt-gbl-corp-argocd-deploy-non-prod.terraform.tfvars.json" "github_repository.default[0]" argocd-deploy-non-prod $ atmos terraform varfile argocd-deploy-non-prod -s mgmt-gbl-corp $ cd components/terraform/argocd-repo $ terraform import -var "import_profile_name=eg-mgmt-gbl-corp-admin" -var-file="mgmt-gbl-corp-argocd-deploy-non-prod.terraform.tfvars.json" "github_branch.default[0]" argocd-deploy-non-prod:main $ cd components/terraform/argocd-repo $ terraform import -var "import_profile_name=eg-mgmt-gbl-corp-admin" -var-file="mgmt-gbl-corp-argocd-deploy-non-prod.terraform.tfvars.json" "github_branch_default.default[0]" argocd-deploy-non-prod ``` ## Variables ### Required Variables
`github_codeowner_teams` (`list(string)`) required
List of teams to use when populating the CODEOWNERS file. For example: `["@ACME/cloud-admins", "@ACME/cloud-developers"]`.
`github_organization` (`string`) required
GitHub Organization
`github_user` (`string`) required
Github user
`github_user_email` (`string`) required
Github user email
`gitignore_entries` (`list(string)`) required
List of .gitignore entries to use when populating the .gitignore file. For example: `[".idea/", ".vscode/"]`.
`region` (`string`) required
AWS Region
### Optional Variables
`bypass_pull_request_actors` (`list(string)`) optional
List of GitHub usernames and team slugs that can bypass pull request requirements **Default value:** `[ ]`
`create_repo` (`bool`) optional
Whether or not to create the repository or use an existing one **Default value:** `true`
`deploy_keys_enabled` (`bool`) optional
Enable GitHub deploy keys for the repository. These are used for Argo CD application syncing. Alternatively, you can use a GitHub App to access this desired state repository. **Default value:** `true`
`description` (`string`) optional
The description of the repository **Default value:** `null`
`environments` optional
Environments to populate `applicationset.yaml` files and repository deploy keys (for ArgoCD) for. `auto-sync` determines whether or not the ArgoCD application will be automatically synced. `ignore-differences` determines whether or not the ArgoCD application will ignore the number of replicas in the deployment. Read more on ignore differences here: https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs Example: ``` tenant: plat environment: use1 stage: sandbox auto-sync: true ignore-differences: - group: apps kind: Deployment json-pointers: - /spec/replicas ``` **Type:** ```hcl list(object({ tenant = optional(string, null) environment = string stage = string attributes = optional(list(string), []) auto-sync = bool ignore-differences = optional(list(object({ group = string, kind = string, json-pointers = list(string) })), []) })) ``` **Default value:** `[ ]`
`github_base_url` (`string`) optional
This is the target GitHub base API endpoint. Providing a value is a requirement when working with GitHub Enterprise. It is optional to provide this value and it can also be sourced from the `GITHUB_BASE_URL` environment variable. The value must end with a slash, for example: `https://terraformtesting-ghe.westus.cloudapp.azure.com/` **Default value:** `null`
`github_default_notifications_enabled` (`string`) optional
Enable default GitHub commit statuses notifications (required for CD sync mode) **Default value:** `true`
`github_notifications` (`list(string)`) optional
ArgoCD notification annotations for subscribing to GitHub. The default value given uses the same notification template names as defined in the `eks/argocd` component. If want to add additional notifications, include any existing notifications from this list that you want to keep in addition. **Default value:** ```hcl [ "notifications.argoproj.io/subscribe.on-deploy-started.app-repo-github-commit-status: \"\"", "notifications.argoproj.io/subscribe.on-deploy-started.argocd-repo-github-commit-status: \"\"", "notifications.argoproj.io/subscribe.on-deploy-succeeded.app-repo-github-commit-status: \"\"", "notifications.argoproj.io/subscribe.on-deploy-succeeded.argocd-repo-github-commit-status: \"\"", "notifications.argoproj.io/subscribe.on-deploy-failed.app-repo-github-commit-status: \"\"", "notifications.argoproj.io/subscribe.on-deploy-failed.argocd-repo-github-commit-status: \"\"" ] ```
`github_token_override` (`string`) optional
Use the value of this variable as the GitHub token instead of reading it from SSM **Default value:** `null`
`manifest_kubernetes_namespace` (`string`) optional
The namespace used for the ArgoCD application **Default value:** `"argocd"`
`permissions` optional
A list of Repository Permission objects used to configure the team permissions of the repository `team_slug` should be the name of the team without the `@{org}` e.g. `@cloudposse/team` => `team` `permission` is just one of the available values listed below **Type:** ```hcl list(object({ team_slug = string, permission = string })) ``` **Default value:** `[ ]`
`push_restrictions_enabled` (`bool`) optional
Enforce who can push to the main branch **Default value:** `true`
`required_pull_request_reviews` (`bool`) optional
Enforce restrictions for pull request reviews **Default value:** `true`
`restrict_pushes_blocks_creations` (`bool`) optional
Setting this to `false` allows people, teams, or apps to create new branches matching this rule **Default value:** `true`
`slack_notifications_channel` (`string`) optional
If given, the Slack channel to for deployment notifications. **Default value:** `""`
`ssm_github_api_key` (`string`) optional
SSM path to the GitHub API key **Default value:** `"/argocd/github/api_key"`
`ssm_github_deploy_key_format` (`string`) optional
Format string of the SSM parameter path to which the deploy keys will be written to (%s will be replaced with the environment name) **Default value:** `"/argocd/deploy_keys/%s"`
`use_local_github_credentials` (`bool`) optional
Use local GitHub credentials from environment variables instead of SSM **Default value:** `false`
`vulnerability_alerts_enabled` (`bool`) optional
Enable security alerts for vulnerable dependencies **Default value:** `false`
`web_commit_signoff_required` (`bool`) optional
Require contributors to sign off on web-based commits **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`deploy_keys_ssm_path_format`
SSM Parameter Store path format for the repository's deploy keys
`deploy_keys_ssm_paths`
SSM Parameter Store paths for the repository's deploy keys
`repository`
Repository name
`repository_default_branch`
Repository default branch
`repository_description`
Repository description
`repository_git_clone_url`
Repository git clone URL
`repository_http_clone_url`
Repository HTTP clone URL
`repository_ssh_clone_url`
Repository SSH clone URL
`repository_url`
Repository URL
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 6.0` - `tls`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 6.0` - `tls`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`github_branch_default.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/branch_default) (resource) - [`github_branch_protection.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/branch_protection) (resource) - [`github_repository.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository) (resource) - [`github_repository_deploy_key.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_deploy_key) (resource) - [`github_repository_file.application_set`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file) (resource) - [`github_repository_file.codeowners_file`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file) (resource) - [`github_repository_file.gitignore`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file) (resource) - [`github_repository_file.pull_request_template`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file) (resource) - [`github_repository_file.readme`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_file) (resource) - [`github_team_repository.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/team_repository) (resource) - [`tls_private_key.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.github_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`github_repository.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository) (data source) - [`github_team.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/team) (data source) - [`github_user.automation_user`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/user) (data source) --- ## athena This component is responsible for provisioning an Amazon Athena workgroup, databases, and related resources. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/catalog/athena/defaults.yaml` file (base component for all Athena deployments with default settings): ```yaml components: terraform: athena/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true tags: Team: sre Service: athena create_s3_bucket: true create_kms_key: true athena_kms_key_deletion_window: 7 bytes_scanned_cutoff_per_query: null enforce_workgroup_configuration: true publish_cloudwatch_metrics_enabled: true encryption_option: "SSE_KMS" s3_output_path: "" workgroup_state: "ENABLED" database: [] ``` ```yaml import: - catalog/athena/defaults components: terraform: athena/example: metadata: component: athena inherits: - athena/defaults vars: enabled: true name: athena-example workgroup_description: "My Example Athena Workgroup" database: - example_db_1 - example_db_2 ``` ### CloudTrail Integration Using Athena with CloudTrail logs is a powerful way to enhance your analysis of AWS service activity. This component supports creating a CloudTrail table for each account and setting up queries to read CloudTrail logs from a centralized location. To set up the CloudTrail Integration, first create the `create` and `alter` queries in Athena with this component. When `var.cloudtrail_database` is defined, this component will create these queries. ```yaml import: - catalog/athena/defaults components: terraform: athena/audit: metadata: component: athena inherits: - athena/defaults vars: enabled: true name: athena-audit workgroup_description: "Athena Workgroup for Auditing" cloudtrail_database: audit databases: audit: comment: "Auditor database for Athena" properties: {} named_queries: platform_dev: database: audit description: "example query against CloudTrail logs" query: | SELECT useridentity.arn, eventname, sourceipaddress, eventtime FROM %s.platform_dev_cloudtrail_logs LIMIT 100; ``` Once those are created, run the `create` and then the `alter` queries in the AWS Console to create and then fill the tables in Athena. :::info Athena runs queries with the permissions of the user executing the query. In order to be able to query CloudTrail logs, the `audit` account must have access to the KMS key used to encrypt CloudTrails logs. Set `var.audit_access_enabled` to `true` in the `cloudtrail` component ::: ## Variables ### Required Variables
`databases` (`map(any)`) required
Map of Athena databases and related configuration.
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`athena_kms_key` (`string`) optional
Use an existing KMS key for Athena if `create_kms_key` is `false`. **Default value:** `null`
`athena_kms_key_deletion_window` (`number`) optional
KMS key deletion window (in days). **Default value:** `7`
`athena_s3_bucket_id` (`string`) optional
Use an existing S3 bucket for Athena query results if `create_s3_bucket` is `false`. **Default value:** `null`
`bytes_scanned_cutoff_per_query` (`number`) optional
Integer for the upper data usage limit (cutoff) for the amount of bytes a single query in a workgroup is allowed to scan. Must be at least 10485760. **Default value:** `null`
`cloudtrail_bucket_component_name` (`string`) optional
The name of the CloudTrail bucket component **Default value:** `"cloudtrail-bucket"`
`cloudtrail_database` (`string`) optional
The name of the Athena Database to use for CloudTrail logs. If set, an Athena table will be created for the CloudTrail trail. **Default value:** `""`
`create_kms_key` (`bool`) optional
Enable the creation of a KMS key used by Athena workgroup. **Default value:** `true`
`create_s3_bucket` (`bool`) optional
Enable the creation of an S3 bucket to use for Athena query results **Default value:** `true`
`data_catalogs` (`map(any)`) optional
Map of Athena data catalogs and parameters **Default value:** `{ }`
`enforce_workgroup_configuration` (`bool`) optional
Boolean whether the settings for the workgroup override client-side settings. **Default value:** `true`
`named_queries` (`map(map(string))`) optional
Map of Athena named queries and parameters **Default value:** `{ }`
`publish_cloudwatch_metrics_enabled` (`bool`) optional
Boolean whether Amazon CloudWatch metrics are enabled for the workgroup. **Default value:** `true`
`s3_output_path` (`string`) optional
The S3 bucket path used to store query results. **Default value:** `""`
`workgroup_description` (`string`) optional
Description of the Athena workgroup. **Default value:** `""`
`workgroup_encryption_option` (`string`) optional
Indicates whether Amazon S3 server-side encryption with Amazon S3-managed keys (SSE_S3), server-side encryption with KMS-managed keys (SSE_KMS), or client-side encryption with KMS-managed keys (CSE_KMS) is used. **Default value:** `"SSE_KMS"`
`workgroup_force_destroy` (`bool`) optional
The option to delete the workgroup and its contents even if the workgroup contains any named queries. **Default value:** `false`
`workgroup_state` (`string`) optional
State of the workgroup. Valid values are `DISABLED` or `ENABLED`. **Default value:** `"ENABLED"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`data_catalogs`
List of newly created Athena data catalogs.
`databases`
List of newly created Athena databases.
`kms_key_arn`
ARN of KMS key used by Athena.
`named_queries`
List of newly created Athena named queries.
`s3_bucket_id`
ID of S3 bucket used for Athena query results.
`workgroup_id`
ID of newly created Athena workgroup.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `athena` | 0.2.1 | [`cloudposse/athena/aws`](https://registry.terraform.io/modules/cloudposse/athena/aws/0.2.1) | n/a `cloudtrail_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_athena_named_query.cloudtrail_query_alter_tables`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_named_query) (resource) - [`aws_athena_named_query.cloudtrail_query_create_tables`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_named_query) (resource) ## Data Sources The following data sources are used by this module: --- ## aurora-mysql This component provisions Amazon Aurora MySQL RDS clusters and seeds relevant database information (hostnames, username, password, etc.) into AWS SSM Parameter Store. ## Usage **Stack Level**: Regional Here's an example for how to use this component. `stacks/catalog/aurora-mysql/defaults.yaml` file (base component for all Aurora MySQL clusters with default settings): ```yaml components: terraform: aurora-mysql/defaults: metadata: type: abstract vars: enabled: false name: rds mysql_deletion_protection: false mysql_storage_encrypted: true aurora_mysql_engine: "aurora-mysql" allowed_cidr_blocks: # all automation - 10.128.0.0/22 # all corp - 10.128.16.0/22 eks_component_names: - eks/eks # https://docs.aws.amazon.com/AmazonRDS/latest/AuroraMySQLReleaseNotes/AuroraMySQL.Updates.3020.html # aws rds describe-db-engine-versions --engine aurora-mysql --query 'DBEngineVersions[].EngineVersion' aurora_mysql_engine_version: "8.0.mysql_aurora.3.02.0" # engine and cluster family are notoriously hard to find. # If you know the engine version (example here is "8.0.mysql_aurora.3.02.0"), use Engine and DBParameterGroupFamily from: # aws rds describe-db-engine-versions --engine aurora-mysql --query "DBEngineVersions[]" | \ # jq '.[] | select(.EngineVersion == "8.0.mysql_aurora.3.02.0") | # { Engine: .Engine, EngineVersion: .EngineVersion, DBParameterGroupFamily: .DBParameterGroupFamily }' # # Returns: # { # "Engine": "aurora-mysql", # "EngineVersion": "8.0.mysql_aurora.3.02.0", # "DBParameterGroupFamily": "aurora-mysql8.0" # } aurora_mysql_cluster_family: "aurora-mysql8.0" mysql_name: shared # 1 writer, 1 reader mysql_cluster_size: 2 mysql_admin_user: "" # generate random username mysql_admin_password: "" # generate random password mysql_db_name: "" # generate random db name # https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html mysql_instance_type: "db.t3.medium" mysql_skip_final_snapshot: false ``` Example configuration for a dev cluster. Import this file into the primary region. `stacks/catalog/aurora-mysql/dev.yaml` file (override the default settings for the cluster in the `dev` account): ```yaml import: - catalog/aurora-mysql/defaults components: terraform: aurora-mysql/dev: metadata: component: aurora-mysql inherits: - aurora-mysql/defaults vars: instance_type: db.r5.large mysql_cluster_size: 1 mysql_name: main mysql_db_name: main ``` Example deployment with primary cluster deployed to us-east-1 in a `platform-dev` account: `atmos terraform apply aurora-mysql/dev -s platform-use1-dev` ## Disaster Recovery with Cross-Region Replication This component is designed to support cross-region replication with continuous replication. If enabled and deployed, a secondary cluster will be deployed in a different region than the primary cluster. This approach is highly aggressive and costly, but in a disaster scenario where the primary cluster fails, the secondary cluster can be promoted to take its place. Follow these steps to handle a Disaster Recovery. ### Usage To deploy a secondary cluster for cross-region replication, add the following catalog entries to an alternative region: Default settings for a secondary, replica cluster. For this example, this file is saved as `stacks/catalog/aurora-mysql/replica/defaults.yaml` ```yaml import: - catalog/aurora-mysql/defaults components: terraform: aurora-mysql/replica/defaults: metadata: component: aurora-mysql inherits: - aurora-mysql/defaults vars: eks_component_names: [] allowed_cidr_blocks: # all automation in primary region (where Spacelift is deployed) - 10.128.0.0/22 # all corp in the same region as this cluster - 10.132.16.0/22 mysql_instance_type: "db.t3.medium" mysql_name: "replica" primary_cluster_region: use1 is_read_replica: true is_promoted_read_replica: false # False by default, added for visibility ``` Environment specific settings for `dev` as an example: ```yaml import: - catalog/aurora-mysql/replica/defaults components: terraform: aurora-mysql/dev: metadata: component: aurora-mysql inherits: - aurora-mysql/defaults - aurora-mysql/replica/defaults vars: enabled: true primary_cluster_component: aurora-mysql/dev ``` ### Promoting the Read Replica Promoting an existing RDS Replicate cluster to a fully standalone cluster is not currently supported by Terraform: https://github.com/hashicorp/terraform-provider-aws/issues/6749 Instead, promote the Replicate cluster with the AWS CLI command: `aws rds promote-read-replica-db-cluster --db-cluster-identifier ` After promoting the replica, update the stack configuration to prevent future Terraform runs from re-enabling replication. In this example, modify `stacks/catalog/aurora-mysql/replica/defaults.yaml` ```yaml is_promoted_read_replica: true ``` Re-deploying the component should show no changes. For example, `atmos terraform apply aurora-mysql/dev -s platform-use2-dev` ## Variables ### Required Variables
`aurora_mysql_cluster_family` (`string`) required
DBParameterGroupFamily (e.g. `aurora5.6`, `aurora-mysql5.7` for Aurora MySQL databases). See https://stackoverflow.com/a/55819394 for help finding the right one to use.
`aurora_mysql_engine` (`string`) required
Engine for Aurora database: `aurora` for MySQL 5.6, `aurora-mysql` for MySQL 5.7
`region` (`string`) required
AWS Region
### Optional Variables
`allow_ingress_from_vpc_accounts` optional
List of account contexts to pull VPC ingress CIDR and add to cluster security group. e.g. \{ environment = "ue2", stage = "auto", tenant = "core" \} Defaults to the "vpc" component in the given account **Type:** ```hcl list(object({ vpc = optional(string, "vpc") environment = optional(string) stage = optional(string) tenant = optional(string) })) ``` **Default value:** `[ ]`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the RDS cluster **Default value:** `[ ]`
`aurora_mysql_cluster_parameters` optional
List of DB cluster parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`aurora_mysql_engine_version` (`string`) optional
Engine Version for Aurora database. **Default value:** `""`
`aurora_mysql_instance_parameters` optional
List of DB instance parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`auto_minor_version_upgrade` (`bool`) optional
Automatically update the cluster when a new minor version is released **Default value:** `false`
`eks_component_names` (`set(string)`) optional
The names of the eks components **Default value:** ```hcl [ "eks/cluster" ] ```
`is_promoted_read_replica` (`bool`) optional
If `true`, do not assign a Replication Source to the Cluster. Set to `true` after manually promoting the cluster from a replica to a standalone cluster. **Default value:** `false`
`is_read_replica` (`bool`) optional
If `true`, create this DB cluster as a Read Replica. **Default value:** `false`
`mysql_admin_password` (`string`) optional
MySQL password for the admin user **Default value:** `""`
`mysql_admin_user` (`string`) optional
MySQL admin user name **Default value:** `""`
`mysql_backup_retention_period` (`number`) optional
Number of days for which to retain backups **Default value:** `3`
`mysql_backup_window` (`string`) optional
Daily time range during which the backups happen **Default value:** `"07:00-09:00"`
`mysql_cluster_size` (`string`) optional
MySQL cluster size **Default value:** `2`
`mysql_db_name` (`string`) optional
Database name (default is not to create a database) **Default value:** `""`
`mysql_db_port` (`number`) optional
Database port **Default value:** `3306`
`mysql_deletion_protection` (`string`) optional
Set to `true` to protect the database from deletion **Default value:** `true`
`mysql_enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to export to cloudwatch. The following log types are supported: audit, error, general, slowquery **Default value:** ```hcl [ "audit", "error", "general", "slowquery" ] ```
`mysql_instance_type` (`string`) optional
EC2 instance type for RDS MySQL cluster **Default value:** `"db.t3.medium"`
`mysql_maintenance_window` (`string`) optional
Weekly time range during which system maintenance can occur, in UTC **Default value:** `"sat:10:00-sat:10:30"`
`mysql_name` (`string`) optional
MySQL solution name (part of cluster identifier) **Default value:** `""`
`mysql_skip_final_snapshot` (`string`) optional
Determines whether a final DB snapshot is created before the DB cluster is deleted **Default value:** `false`
`mysql_storage_encrypted` (`string`) optional
Set to `true` to keep the database contents encrypted **Default value:** `true`
`performance_insights_enabled` (`bool`) optional
Set `true` to enable Performance Insights **Default value:** `false`
`primary_cluster_component` (`string`) optional
If this cluster is a read replica and no replication source is explicitly given, the component name for the primary cluster **Default value:** `"aurora-mysql"`
`primary_cluster_region` (`string`) optional
If this cluster is a read replica and no replication source is explicitly given, the region to look for a matching cluster **Default value:** `""`
`publicly_accessible` (`bool`) optional
Set to true to create the cluster in a public subnet **Default value:** `false`
`replication_source_identifier` (`string`) optional
ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica. If this value is empty and replication is enabled, remote state will attempt to find a matching cluster in the Primary DB Cluster's region **Default value:** `""`
`secrets_store_type` (`string`) optional
Secret Store type to save database credentials. Valid values: `SSM`, `ASM` **Default value:** `"SSM"`
`ssm_password_source` (`string`) optional
If `var.ssm_passwords_enabled` is `true`, DB user passwords will be retrieved from SSM using `var.ssm_password_source` and the database username. If this value is not set, a default path will be created using the SSM path prefix and ID of the associated Aurora Cluster. **Default value:** `""`
`ssm_path_prefix` (`string`) optional
SSM path prefix **Default value:** `"rds"`
`vpc_component_name` (`string`) optional
The name of the VPC component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aurora_mysql_cluster_arn`
The ARN of Aurora cluster
`aurora_mysql_cluster_id`
The ID of Aurora cluster
`aurora_mysql_cluster_name`
Aurora MySQL cluster identifier
`aurora_mysql_endpoint`
Aurora MySQL endpoint
`aurora_mysql_master_hostname`
Aurora MySQL DB master hostname
`aurora_mysql_master_password`
Location of admin password
`aurora_mysql_master_password_asm_key`
ASM key for admin password
`aurora_mysql_master_password_ssm_key`
SSM key for admin password
`aurora_mysql_master_username`
Aurora MySQL username for the master DB user
`aurora_mysql_reader_endpoint`
Aurora MySQL reader endpoint
`aurora_mysql_replicas_hostname`
Aurora MySQL replicas hostname
`cluster_domain`
Cluster DNS name
`kms_key_arn`
KMS key ARN for Aurora MySQL
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aurora_mysql` | 2.2.0 | [`cloudposse/rds-cluster/aws`](https://registry.terraform.io/modules/cloudposse/rds-cluster/aws/2.2.0) | n/a `cluster` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `dns-delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_rds` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `parameter_store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `primary_cluster` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpc_ingress` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_secretsmanager_secret.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) (resource) - [`aws_secretsmanager_secret_version.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) (resource) - [`random_password.mysql_admin_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.mysql_admin_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) - [`random_pet.mysql_db_name`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## aurora-mysql-resources This component provisions Aurora MySQL resources: additional databases, users, permissions, and grants. NOTE: Creating additional users (including read-only users) and databases requires Spacelift, since that action must be done via the MySQL provider, and by default only the automation account is whitelisted by the Aurora cluster. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. `stacks/catalog/aurora-mysql/resources/defaults.yaml` file (base component for Aurora MySQL Resources with default settings): ```yaml components: terraform: aurora-mysql-resources/defaults: metadata: type: abstract vars: enabled: true ``` Example (not actual): `stacks/uw2-dev.yaml` file (override the default settings for the cluster resources in the `dev` account, create an additional database and user): ```yaml import: - catalog/aurora-mysql/resources/defaults components: terraform: aurora-mysql-resources/dev: metadata: component: aurora-mysql-resources inherits: - aurora-mysql-resources/defaults vars: aurora_mysql_component_name: aurora-mysql/dev additional_users: example: db_user: example db_password: "" grants: - grant: ["ALL"] db: example object_type: database schema: null ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`additional_databases` (`set(string)`) optional
Additional databases to be created with the cluster **Default value:** `[ ]`
`additional_grants` optional
Create additional database user with specified grants. If `var.ssm_password_source` is set, passwords will be retrieved from SSM parameter store, otherwise, passwords will be generated and stored in SSM parameter store under the service's key. **Type:** ```hcl map(list(object({ grant : list(string) db : string }))) ``` **Default value:** `{ }`
`additional_users` optional
Create additional database user for a service, specifying username, grants, and optional password. If no password is specified, one will be generated. Username and password will be stored in SSM parameter store under the service's key. **Type:** ```hcl map(object({ db_user : string db_password : string grants : list(object({ grant : list(string) db : string })) })) ``` **Default value:** `{ }`
`aurora_mysql_component_name` (`string`) optional
Aurora MySQL component name to read the remote state from **Default value:** `"aurora-mysql"`
`mysql_admin_password` (`string`) optional
MySQL password for the admin user. If not provided, the password will be pulled from SSM **Default value:** `""`
`mysql_cluster_enabled` (`string`) optional
Set to `false` to prevent the module from creating any resources **Default value:** `true`
`mysql_db_name` (`string`) optional
Database name (default is not to create a database **Default value:** `""`
`read_passwords_from_ssm` (`bool`) optional
When `true`, fetch user passwords from SSM **Default value:** `true`
`ssm_password_source` (`string`) optional
If var.read_passwords_from_ssm is true, DB user passwords will be retrieved from SSM using `var.ssm_password_source` and the database username. If this value is not set, a default path will be created using the SSM path prefix and ID of the associated Aurora Cluster. **Default value:** `""`
`ssm_path_prefix` (`string`) optional
SSM path prefix **Default value:** `"rds"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`additional_grants`
Additional DB users created
`additional_users`
Additional DB users created
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `mysql`, version: `>= 3.0.22` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `mysql`, version: `>= 3.0.22` ### Modules Name | Version | Source | Description --- | --- | --- | --- `additional_grants` | latest | [`./modules/mysql-user`](https://registry.terraform.io/modules/./modules/mysql-user/) | n/a `additional_users` | latest | [`./modules/mysql-user`](https://registry.terraform.io/modules/./modules/mysql-user/) | n/a `aurora_mysql` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`mysql_database.additional`](https://registry.terraform.io/providers/petoju/mysql/latest/docs/resources/database) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.admin_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## aurora-postgres This component is responsible for provisioning Aurora Postgres RDS clusters. It seeds relevant database information (hostnames, username, password, etc.) into AWS SSM Parameter Store. ## Usage **Stack Level**: Regional Here's an example for how to use this component. `stacks/catalog/aurora-postgres/defaults.yaml` file (base component for all Aurora Postgres clusters with default settings): ```yaml components: terraform: aurora-postgres/defaults: metadata: type: abstract vars: enabled: true name: aurora-postgres tags: Team: sre Service: aurora-postgres cluster_name: shared deletion_protection: false storage_encrypted: true engine: aurora-postgresql # Provisioned configuration engine_mode: provisioned engine_version: "15.3" cluster_family: aurora-postgresql15 # 1 writer, 1 reader cluster_size: 2 # https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html instance_type: db.t3.medium admin_user: postgres admin_password: "" # generate random password database_name: postgres database_port: 5432 skip_final_snapshot: false # Enhanced Monitoring # A boolean flag to enable/disable the creation of the enhanced monitoring IAM role. # If set to false, the module will not create a new role and will use rds_monitoring_role_arn for enhanced monitoring enhanced_monitoring_role_enabled: true # The interval, in seconds, between points when enhanced monitoring metrics are collected for the DB instance. # To disable collecting Enhanced Monitoring metrics, specify 0. The default is 0. Valid Values: 0, 1, 5, 10, 15, 30, 60 rds_monitoring_interval: 15 # Allow ingress from the following accounts # If any of tenant, stage, or environment aren't given, this will be taken allow_ingress_from_vpc_accounts: - tenant: core stage: auto ``` Example (not actual): `stacks/uw2-dev.yaml` file (override the default settings for the cluster in the `dev` account, create an additional database and user): ```yaml import: - catalog/aurora-postgres/defaults components: terraform: aurora-postgres: metadata: component: aurora-postgres inherits: - aurora-postgres/defaults vars: enabled: true ``` ### Finding Aurora Engine Version Use the following to query the AWS API by `engine-mode`. Both provisioned and Serverless v2 use the `privisoned` engine mode, whereas only Serverless v1 uses the `serverless` engine mode. ```bash aws rds describe-db-engine-versions \ --engine aurora-postgresql \ --query 'DBEngineVersions[].EngineVersion' \ --filters 'Name=engine-mode,Values=serverless' ``` Use the following to query AWS API by `db-instance-class`. Use this query to find supported versions for a specific instance class, such as `db.serverless` with Serverless v2. ```bash aws rds describe-orderable-db-instance-options \ --engine aurora-postgresql \ --db-instance-class db.serverless \ --query 'OrderableDBInstanceOptions[].[EngineVersion]' ``` Once a version has been selected, use the following to find the cluster family. ```bash aws rds describe-db-engine-versions --engine aurora-postgresql --query "DBEngineVersions[]" | \ jq '.[] | select(.EngineVersion == "15.3") | { Engine: .Engine, EngineVersion: .EngineVersion, DBParameterGroupFamily: .DBParameterGroupFamily }' ``` ## Examples Generally there are three different engine configurations for Aurora: provisioned, Serverless v1, and Serverless v2. ### Provisioned Aurora Postgres [See the default usage example above](#usage) ### Serverless v1 Aurora Postgres Serverless v1 requires `engine-mode` set to `serverless` uses `scaling_configuration` to configure scaling options. For valid values, see [ModifyCurrentDBClusterCapacity](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_ModifyCurrentDBClusterCapacity.html). ```yaml components: terraform: aurora-postgres: vars: enabled: true name: aurora-postgres eks_component_names: - eks/cluster allow_ingress_from_vpc_accounts: # Allows Spacelift - tenant: core stage: auto environment: use2 # Allows VPN - tenant: core stage: network environment: use2 cluster_name: shared engine: aurora-postgresql # Serverless v1 configuration engine_mode: serverless instance_type: "" # serverless engine_mode ignores `var.instance_type` engine_version: "13.9" # Latest supported version as of 08/28/2023 cluster_family: aurora-postgresql13 cluster_size: 0 # serverless scaling_configuration: - auto_pause: true max_capacity: 4 min_capacity: 2 seconds_until_auto_pause: 300 timeout_action: null admin_user: postgres admin_password: "" # generate random password database_name: postgres database_port: 5432 storage_encrypted: true deletion_protection: true skip_final_snapshot: false # Creating read-only users or additional databases requires Spacelift read_only_users_enabled: false # Enhanced Monitoring # A boolean flag to enable/disable the creation of the enhanced monitoring IAM role. # If set to false, the module will not create a new role and will use rds_monitoring_role_arn for enhanced monitoring enhanced_monitoring_role_enabled: true enhanced_monitoring_attributes: ["monitoring"] # The interval, in seconds, between points when enhanced monitoring metrics are collected for the DB instance. # To disable collecting Enhanced Monitoring metrics, specify 0. The default is 0. Valid Values: 0, 1, 5, 10, 15, 30, 60 rds_monitoring_interval: 15 iam_database_authentication_enabled: false additional_users: {} ``` ### Serverless v2 Aurora Postgres Aurora Postgres Serverless v2 uses the `provisioned` engine mode with `db.serverless` instances. In order to configure scaling with Serverless v2, use `var.serverlessv2_scaling_configuration`. For more on valid scaling configurations, see [Performance and scaling for Aurora Serverless v2](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.setting-capacity.html). ```yaml components: terraform: aurora-postgres: vars: enabled: true name: aurora-postgres eks_component_names: - eks/cluster allow_ingress_from_vpc_accounts: # Allows Spacelift - tenant: core stage: auto environment: use2 # Allows VPN - tenant: core stage: network environment: use2 cluster_name: shared engine: aurora-postgresql # Serverless v2 configuration engine_mode: provisioned instance_type: "db.serverless" engine_version: "15.3" cluster_family: aurora-postgresql15 cluster_size: 2 serverlessv2_scaling_configuration: min_capacity: 2 max_capacity: 64 admin_user: postgres admin_password: "" # generate random password database_name: postgres database_port: 5432 storage_encrypted: true deletion_protection: true skip_final_snapshot: false # Creating read-only users or additional databases requires Spacelift read_only_users_enabled: false # Enhanced Monitoring # A boolean flag to enable/disable the creation of the enhanced monitoring IAM role. # If set to false, the module will not create a new role and will use rds_monitoring_role_arn for enhanced monitoring enhanced_monitoring_role_enabled: true enhanced_monitoring_attributes: ["monitoring"] # The interval, in seconds, between points when enhanced monitoring metrics are collected for the DB instance. # To disable collecting Enhanced Monitoring metrics, specify 0. The default is 0. Valid Values: 0, 1, 5, 10, 15, 30, 60 rds_monitoring_interval: 15 iam_database_authentication_enabled: false additional_users: {} ``` ## Variables ### Required Variables
`cluster_name` (`string`) required
Short name for this cluster
`cluster_size` (`number`) required
Postgres cluster size
`engine_mode` (`string`) required
The database engine mode. Valid values: `global`, `multimaster`, `parallelquery`, `provisioned`, `serverless`
`instance_type` (`string`) required
EC2 instance type for Postgres cluster
`region` (`string`) required
AWS Region
### Optional Variables
`admin_password` (`string`) optional
Postgres password for the admin user **Default value:** `""`
`admin_user` (`string`) optional
Postgres admin user name **Default value:** `""`
`allow_ingress_from_vpc_accounts` optional
List of account contexts to pull VPC ingress CIDR and add to cluster security group. e.g. \{ environment = "ue2", stage = "auto", tenant = "core" \} Defaults to the "vpc" component in the given account **Type:** ```hcl list(object({ vpc = optional(string, "vpc") environment = optional(string) stage = optional(string) tenant = optional(string) })) ``` **Default value:** `[ ]`
`allow_major_version_upgrade` (`bool`) optional
Enable to allow major engine version upgrades when changing engine versions. Defaults to false. **Default value:** `false`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDRs allowed to access the database (in addition to security groups and subnets) **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
List of security group ids that should be allowed access to the database **Default value:** `[ ]`
`allowed_security_group_names` (`list(string)`) optional
List of security group names (tags) that should be allowed access to the database **Default value:** `[ ]`
`autoscaling_enabled` (`bool`) optional
Whether to enable cluster autoscaling **Default value:** `false`
`autoscaling_max_capacity` (`number`) optional
Maximum number of instances to be maintained by the autoscaler **Default value:** `5`
`autoscaling_min_capacity` (`number`) optional
Minimum number of instances to be maintained by the autoscaler **Default value:** `1`
`autoscaling_policy_type` (`string`) optional
Autoscaling policy type. `TargetTrackingScaling` and `StepScaling` are supported **Default value:** `"TargetTrackingScaling"`
`autoscaling_scale_in_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling down activity can start. Default is 300s **Default value:** `300`
`autoscaling_scale_out_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling up activity can start. Default is 300s **Default value:** `300`
`autoscaling_target_metrics` (`string`) optional
The metrics type to use. If this value isn't provided the default is CPU utilization **Default value:** `"RDSReaderAverageCPUUtilization"`
`autoscaling_target_value` (`number`) optional
The target value to scale with respect to target metrics **Default value:** `75`
`backup_window` (`string`) optional
Daily time range during which the backups happen, UTC **Default value:** `"07:00-09:00"`
`ca_cert_identifier` (`string`) optional
The identifier of the CA certificate for the DB instance **Default value:** `null`
`cluster_dns_name_part` (`string`) optional
Part of DNS name added to module and cluster name for DNS for cluster endpoint **Default value:** `"writer"`
`cluster_family` (`string`) optional
Family of the DB parameter group. Valid values for Aurora PostgreSQL: `aurora-postgresql9.6`, `aurora-postgresql10`, `aurora-postgresql11`, `aurora-postgresql12` **Default value:** `"aurora-postgresql13"`
`cluster_parameters` optional
List of DB cluster parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`database_name` (`string`) optional
Name for an automatically created database on cluster creation. An empty name will generate a db name. **Default value:** `""`
`database_port` (`number`) optional
Database port **Default value:** `5432`
`deletion_protection` (`bool`) optional
Specifies whether the Cluster should have deletion protection enabled. The database can't be deleted when this value is set to `true` **Default value:** `false`
`dns_gbl_delegated_environment_name` (`string`) optional
The name of the environment where global `dns_delegated` is provisioned **Default value:** `"gbl"`
`eks_component_names` (`set(string)`) optional
The names of the eks components **Default value:** ```hcl [ "eks/cluster" ] ```
`eks_security_group_enabled` (`bool`) optional
Use the eks default security group **Default value:** `false`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to export to cloudwatch. The following log types are supported: audit, error, general, slowquery **Default value:** `[ ]`
`engine` (`string`) optional
Name of the database engine to be used for the DB cluster **Default value:** `"postgresql"`
`engine_version` (`string`) optional
Engine version of the Aurora global database **Default value:** `"13.4"`
`enhanced_monitoring_attributes` (`list(string)`) optional
Attributes used to format the Enhanced Monitoring IAM role. If this role hits IAM role length restrictions (max 64 characters), consider shortening these strings. **Default value:** ```hcl [ "enhanced-monitoring" ] ```
`enhanced_monitoring_role_enabled` (`bool`) optional
A boolean flag to enable/disable the creation of the enhanced monitoring IAM role. If set to `false`, the module will not create a new role and will use `rds_monitoring_role_arn` for enhanced monitoring **Default value:** `true`
`iam_database_authentication_enabled` (`bool`) optional
Specifies whether or mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled **Default value:** `false`
`intra_security_group_traffic_enabled` (`bool`) optional
Whether to allow traffic between resources inside the database's security group. **Default value:** `false`
`maintenance_window` (`string`) optional
Weekly time range during which system maintenance can occur, in UTC **Default value:** `"wed:03:00-wed:04:00"`
`manage_admin_user_password` (`bool`) optional
Set to true to allow RDS to manage the master user password in Secrets Manager. Cannot be set if admin_password is provided **Default value:** `false`
`performance_insights_enabled` (`bool`) optional
Whether to enable Performance Insights **Default value:** `false`
`promotion_tier` (`number`) optional
Failover Priority setting on instance level. The reader who has lower tier has higher priority to get promoted to writer. Readers in promotion tiers 0 and 1 scale at the same time as the writer. Readers in promotion tiers 2–15 scale independently from the writer. For more information, see: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.how-it-works.html#aurora-serverless-v2.how-it-works.scaling **Default value:** `0`
`publicly_accessible` (`bool`) optional
Set true to make this database accessible from the public internet **Default value:** `false`
`rds_monitoring_interval` (`number`) optional
The interval, in seconds, between points when enhanced monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. The default is 0. Valid Values: 0, 1, 5, 10, 15, 30, 60 **Default value:** `60`
`reader_dns_name_part` (`string`) optional
Part of DNS name added to module and cluster name for DNS for cluster reader **Default value:** `"reader"`
`restore_to_point_in_time` optional
List of point-in-time recovery options. Valid parameters are: `source_cluster_identifier` Identifier of the source database cluster from which to restore. `restore_type`: Type of restore to be performed. Valid options are "full-copy" and "copy-on-write". `use_latest_restorable_time`: Set to true to restore the database cluster to the latest restorable backup time. Conflicts with `restore_to_time`. `restore_to_time`: Date and time in UTC format to restore the database cluster to. Conflicts with `use_latest_restorable_time`. **Type:** ```hcl list(object({ source_cluster_identifier = string restore_type = optional(string, "copy-on-write") use_latest_restorable_time = optional(bool, true) restore_to_time = optional(string, null) })) ``` **Default value:** `[ ]`
`retention_period` (`number`) optional
Number of days to retain backups for **Default value:** `5`
`scaling_configuration` optional
List of nested attributes with scaling properties. Only valid when `engine_mode` is set to `serverless`. This is required for Serverless v1 **Type:** ```hcl list(object({ auto_pause = bool max_capacity = number min_capacity = number seconds_until_auto_pause = number timeout_action = string })) ``` **Default value:** `[ ]`
`serverlessv2_scaling_configuration` optional
Nested attribute with scaling properties for ServerlessV2. Only valid when `engine_mode` is set to `provisioned.` This is required for Serverless v2 **Type:** ```hcl object({ min_capacity = number max_capacity = number }) ``` **Default value:** `null`
`skip_final_snapshot` (`bool`) optional
Normally AWS makes a snapshot of the database before deleting it. Set this to `true` in order to skip this. NOTE: The final snapshot has a name derived from the cluster name. If you delete a cluster, get a final snapshot, then create a cluster of the same name, its final snapshot will fail with a name collision unless you delete the previous final snapshot first. **Default value:** `false`
`snapshot_identifier` (`string`) optional
Specifies whether or not to create this cluster from a snapshot **Default value:** `null`
`ssm_cluster_name_override` (`string`) optional
Set a cluster name into the ssm path prefix **Default value:** `""`
`ssm_path_prefix` (`string`) optional
Top level SSM path prefix (without leading or trailing slash) **Default value:** `"aurora-postgres"`
`storage_encrypted` (`bool`) optional
Specifies whether the DB cluster is encrypted **Default value:** `true`
`vpc_component_name` (`string`) optional
The name of the VPC component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_username`
Postgres admin username
`allowed_security_groups`
The resulting list of security group IDs that are allowed to connect to the Aurora Postgres cluster.
`cluster_endpoint`
Postgres cluster endpoint
`cluster_identifier`
Postgres cluster identifier
`config_map`
Map containing information pertinent to a PostgreSQL client configuration.
`database_name`
Postgres database name
`instance_endpoints`
List of Postgres instance endpoints
`kms_key_arn`
KMS key ARN for Aurora Postgres
`master_hostname`
Postgres master hostname
`reader_endpoint`
Postgres reader endpoint
`replicas_hostname`
Postgres replicas hostname
`security_group_id`
The security group ID of the Aurora Postgres cluster
`ssm_key_paths`
Names (key paths) of all SSM parameters stored for this cluster
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `postgresql`, version: `>= 1.17.1` - `random`, version: `>= 2.3` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 2.3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aurora_postgres_cluster` | 2.2.0 | [`cloudposse/rds-cluster/aws`](https://registry.terraform.io/modules/cloudposse/rds-cluster/aws/2.2.0) | https://www.terraform.io/docs/providers/aws/r/rds_cluster.html `cluster` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_rds` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `parameter_store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpc_ingress` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`random_password.admin_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.admin_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) - [`random_pet.database_name`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_security_groups.allowed`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/security_groups) (data source) --- ## aurora-postgres-resources This component is responsible for provisioning Aurora Postgres resources: additional databases, users, permissions, grants, etc. ## PostgreSQL Quick Reference on Grants GRANTS can be on database, schema, role, table, and other database objects (e.g. columns in a table for fine control). Database and schema do not have much to grant. The `object_type` field in the input determines which kind of object the grant is being applied to. The `db` field is always required. The `schema` field is required unless the `object_type` is `db`, in which case it should be set to the empty string (`""`). The keyword PUBLIC indicates that the privileges are to be granted to all roles, including those that might be created later. PUBLIC can be thought of as an implicitly defined group that always includes all roles. Any particular role will have the sum of privileges granted directly to it, privileges granted to any role it is presently a member of, and privileges granted to PUBLIC. When an object is created, it is assigned an owner. The owner is normally the role that executed the creation statement. For most kinds of objects, the initial state is that only the owner (or a superuser) can do anything with the object. To allow other roles to use it, privileges must be granted. (When using AWS managed RDS, you cannot have access to any superuser roles; superuser is reserved for AWS to use to manage the cluster.) PostgreSQL grants privileges on some types of objects to PUBLIC by default when the objects are created. No privileges are granted to PUBLIC by default on tables, table columns, sequences, foreign data wrappers, foreign servers, large objects, schemas, or tablespaces. For other types of objects, the default privileges granted to PUBLIC are as follows: CONNECT and TEMPORARY (create temporary tables) privileges for databases; EXECUTE privilege for functions and procedures; and USAGE privilege for languages and data types (including domains). The object owner can, of course, REVOKE both default and expressly granted privileges. (For maximum security, issue the REVOKE in the same transaction that creates the object; then there is no window in which another user can use the object.) Also, these default privilege settings can be overridden using the ALTER DEFAULT PRIVILEGES command. The CREATE privilege: - For databases, allows new schemas and publications to be created within the database, and allows trusted extensions to be installed within the database. - For schemas, allows new objects to be created within the schema. To rename an existing object, you must own the object and have this privilege for the containing schema. For databases and schemas, there are not a lot of other privileges to grant, and all but CREATE are granted by default, so you might as well grant "ALL". For tables etc., the creator has full control. You grant access to other users via explicit grants. This component does not allow fine-grained grants. You have to specify the database, and unless the grant is on the database, you have to specify the schema. For any other object type (table, sequence, function, procedure, routine, foreign_data_wrapper, foreign_server, column), the component applies the grants to all objects of that type in the specified schema. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: aurora-postgres-resources: vars: aurora_postgres_component_name: aurora-postgres-example additional_users: example: db_user: example db_password: "" grants: - grant: ["ALL"] db: example object_type: database schema: "" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`additional_databases` (`set(string)`) optional
Additional databases to be created with the cluster **Default value:** `[ ]`
`additional_grants` optional
Create additional database user with specified grants. If `var.ssm_password_source` is set, passwords will be retrieved from SSM parameter store, otherwise, passwords will be generated and stored in SSM parameter store under the service's key. **Type:** ```hcl map(list(object({ grant : list(string) db : string }))) ``` **Default value:** `{ }`
`additional_schemas` optional
Create additional schemas for a given database. If no database is given, the schema will use the database used by the provider configuration **Type:** ```hcl map(object({ database : string })) ``` **Default value:** `{ }`
`additional_users` optional
Create additional database user for a service, specifying username, grants, and optional password. If no password is specified, one will be generated. Username and password will be stored in SSM parameter store under the service's key. **Type:** ```hcl map(object({ db_user : string db_password : string grants : list(object({ grant : list(string) db : string schema : string object_type : string })) })) ``` **Default value:** `{ }`
`admin_password` (`string`) optional
postgresql password for the admin user **Default value:** `""`
`aurora_postgres_component_name` (`string`) optional
Aurora Postgres component name to read the remote state from **Default value:** `"aurora-postgres"`
`cluster_enabled` (`string`) optional
Set to `false` to prevent the module from creating any resources **Default value:** `true`
`db_name` (`string`) optional
Database name (default is not to create a database) **Default value:** `""`
`kms_key_arn` (`string`) optional
The ARN for the KMS encryption key. **Default value:** `null`
`read_passwords_from_ssm` (`bool`) optional
When `true`, fetch user passwords from SSM **Default value:** `true`
`ssm_password_source` (`string`) optional
If var.read_passwords_from_ssm is true, DB user passwords will be retrieved from SSM using `var.ssm_password_source` and the database username. If this value is not set, a default path will be created using the SSM path prefix and ID of the associated Aurora Cluster. **Default value:** `""`
`ssm_path_prefix` (`string`) optional
SSM path prefix **Default value:** `"aurora-postgres"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`additional_databases`
Additional databases
`additional_grants`
Additional grants
`additional_schemas`
Additional schemas
`additional_users`
Additional users
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `postgresql`, version: `>= 1.17.1` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `postgresql`, version: `>= 1.17.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `additional_grants` | latest | [`./modules/postgresql-user`](https://registry.terraform.io/modules/./modules/postgresql-user/) | n/a `additional_users` | latest | [`./modules/postgresql-user`](https://registry.terraform.io/modules/./modules/postgresql-user/) | n/a `aurora_postgres` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`postgresql_database.additional`](https://registry.terraform.io/providers/cyrilgdn/postgresql/latest/docs/resources/database) (resource) - [`postgresql_schema.additional`](https://registry.terraform.io/providers/cyrilgdn/postgresql/latest/docs/resources/schema) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.admin_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## app Auth0 Application component. [Auth0](https://auth0.com/docs/) is a third-party service that provides authentication and authorization as a service. It is typically used to to authenticate users. An Auth0 application is a client that can request authentication and authorization from an Auth0 server. Auth0 applications can be of different types, such as regular web applications, single-page applications, machine-to-machine applications, and others. Each application has a set of allowed origins, allowed callback URLs, and allowed web origins. ## Usage Before deploying this component, you need to deploy the `auth0/tenant` component. This components with authenticate with the [Auth0 Terraform provider](https://registry.terraform.io/providers/auth0/auth0/latest/) using the Auth0 tenant's client ID and client secret configured with the `auth0/tenant` component. **Stack Level**: Global Here's an example snippet for how to use this component. :::important Be sure that the context ID does not overlap with the context ID of other Auth0 components, such as `auth0/tenant`. We use this ID to generate the SSM parameter names. ::: ```yaml # stacks/catalog/auth0/app.yaml components: terraform: auth0/app: vars: enabled: true name: "auth0-app" # We can centralize plat-sandbox, plat-dev, and plat-staging all use a "nonprod" Auth0 tenant, which is deployed in plat-staging. auth0_tenant_stage_name: "plat-staging" # Common client configuration grant_types: - "authorization_code" - "refresh_token" - "implicit" - "client_credentials" # Stage-specific client configuration callbacks: - "https://auth.acme-dev.com/login/auth0/callback" allowed_origins: - "https://*.acme-dev.com" web_origins: - "https://portal.acme-dev.com" - "https://auth.acme-dev.com" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`allowed_origins` (`list(string)`) optional
Allowed Origins **Default value:** `[ ]`
`app_type` (`string`) optional
Auth0 Application Type **Default value:** `"regular_web"`
`auth0_debug` (`bool`) optional
Enable debug mode for the Auth0 provider **Default value:** `true`
`auth0_tenant_component_name` (`string`) optional
The name of the component **Default value:** `"auth0/tenant"`
`auth0_tenant_environment_name` (`string`) optional
The name of the environment where the Auth0 tenant component is deployed. Defaults to the environment of the current stack. **Default value:** `""`
`auth0_tenant_stage_name` (`string`) optional
The name of the stage where the Auth0 tenant component is deployed. Defaults to the stage of the current stack. **Default value:** `""`
`auth0_tenant_tenant_name` (`string`) optional
The name of the tenant where the Auth0 tenant component is deployed. Yes this is a bit redundant, since Auth0 also calls this resource a tenant. Defaults to the tenant of the current stack. **Default value:** `""`
`authentication_method` (`string`) optional
The authentication method for the client credentials **Default value:** `"client_secret_post"`
`callbacks` (`list(string)`) optional
Allowed Callback URLs **Default value:** `[ ]`
`cross_origin_auth` (`bool`) optional
Whether this client can be used to make cross-origin authentication requests (true) or it is not allowed to make such requests (false). **Default value:** `false`
`grant_types` (`list(string)`) optional
Allowed Grant Types **Default value:** `[ ]`
`jwt_alg` (`string`) optional
JWT Algorithm **Default value:** `"RS256"`
`jwt_lifetime_in_seconds` (`number`) optional
JWT Lifetime in Seconds **Default value:** `36000`
`logo_uri` (`string`) optional
Logo URI **Default value:** `"https://cloudposse.com/wp-content/uploads/2017/07/CloudPosse2-TRANSAPRENT.png"`
`oidc_conformant` (`bool`) optional
OIDC Conformant **Default value:** `true`
`ssm_base_path` (`string`) optional
The base path for the SSM parameters. If not defined, this is set to the module context ID. This is also required when `var.enabled` is set to `false` **Default value:** `""`
`sso` (`bool`) optional
Single Sign-On for the Auth0 app **Default value:** `true`
`web_origins` (`list(string)`) optional
Allowed Web Origins **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`auth0_client_id`
The Auth0 Application Client ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `auth0_ssm_parameters` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `auth0_tenant` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `iam_roles_auth0_provider` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`auth0_client.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/client) (resource) - [`auth0_client_credentials.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/client_credentials) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.auth0_client_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_client_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## connection Auth 0 Connection component. [Auth0](https://auth0.com/docs/) is a third-party service that provides authentication and authorization as a service. It is typically used to to authenticate users. An Auth0 connection is a bridge between Auth0 and an identity provider (IdP) that allows your application to authenticate users. Auth0 supports many types of connections, including social identity providers such as Google, Facebook, and Twitter, enterprise identity providers such as Microsoft Azure AD, and passwordless authentication methods such as email and SMS. ## Usage Before deploying this component, you need to deploy the `auth0/tenant` component. This components with authenticate with the [Auth0 Terraform provider](https://registry.terraform.io/providers/auth0/auth0/latest/) using the Auth0 tenant's client ID and client secret configured with the `auth0/tenant` component. **Stack Level**: Global Here's an example snippet for how to use this component. ```yaml # stacks/catalog/auth0/connection.yaml components: terraform: auth0/connection: vars: enabled: true name: "auth0" # These must all be specified for the connection to be created strategy: "email" connection_name: "email" options_name: "email" email_from: "{{`{{ application.name }}`}} " email_subject: "Welcome to {{`{{ application.name }}`}}" syntax: "liquid" auth_params: scope: "openid profile" response_type: "code" totp: time_step: 895 length: 6 template_file: "templates/email.html" # Stage-specific configuration auth0_app_connections: - stage: sandbox - stage: dev - stage: staging ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`auth0_app_connections` optional
The list of Auth0 apps to add to this connection **Type:** ```hcl list(object({ component = optional(string, "auth0/app") environment = optional(string, "") stage = optional(string, "") tenant = optional(string, "") })) ``` **Default value:** `[ ]`
`auth0_debug` (`bool`) optional
Enable debug mode for the Auth0 provider **Default value:** `true`
`auth0_tenant_component_name` (`string`) optional
The name of the component **Default value:** `"auth0/tenant"`
`auth0_tenant_environment_name` (`string`) optional
The name of the environment where the Auth0 tenant component is deployed. Defaults to the environment of the current stack. **Default value:** `""`
`auth0_tenant_stage_name` (`string`) optional
The name of the stage where the Auth0 tenant component is deployed. Defaults to the stage of the current stack. **Default value:** `""`
`auth0_tenant_tenant_name` (`string`) optional
The name of the tenant where the Auth0 tenant component is deployed. Yes this is a bit redundant, since Auth0 also calls this resource a tenant. Defaults to the tenant of the current stack. **Default value:** `""`
`auth_params` optional
Query string parameters to be included as part of the generated passwordless email link. **Type:** ```hcl object({ scope = optional(string, null) response_type = optional(string, null) }) ``` **Default value:** `{ }`
`brute_force_protection` (`bool`) optional
Indicates whether to enable brute force protection, which will limit the number of signups and failed logins from a suspicious IP address. **Default value:** `true`
`connection_name` (`string`) optional
The name of the connection **Default value:** `""`
`disable_signup` (`bool`) optional
Indicates whether to allow user sign-ups to your application. **Default value:** `false`
`email_from` (`string`) optional
When using an email strategy, the address to use as the sender **Default value:** `null`
`email_subject` (`string`) optional
When using an email strategy, the subject of the email **Default value:** `null`
`non_persistent_attrs` (`list(string)`) optional
If there are user fields that should not be stored in Auth0 databases due to privacy reasons, you can add them to the DenyList here. **Default value:** `[ ]`
`options_name` (`string`) optional
The name of the connection options. Required for the email strategy. **Default value:** `""`
`set_user_root_attributes` (`string`) optional
Determines whether to sync user profile attributes at each login or only on the first login. Options include: `on_each_login`, `on_first_login`. **Default value:** `null`
`strategy` (`string`) optional
The strategy to use for the connection **Default value:** `"auth0"`
`syntax` (`string`) optional
The syntax of the template body **Default value:** `null`
`template` (`string`) optional
The template to use for the connection. If not provided, the `template_file` variable must be set. **Default value:** `""`
`template_file` (`string`) optional
The path to the template file. If not provided, the `template` variable must be set. **Default value:** `""`
`totp` optional
The TOTP settings for the connection **Type:** ```hcl object({ time_step = optional(number, 900) length = optional(number, 6) }) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`auth0_connection_id`
The Auth0 Connection ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `auth0_apps` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `auth0_tenant` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `iam_roles_auth0_provider` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`auth0_connection.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/connection) (resource) - [`auth0_connection_clients.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/connection_clients) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.auth0_client_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_client_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## tenant This component configures an [Auth0](https://auth0.com/docs/) tenant. This component is used to configure authentication for the Terraform provider for Auth0 and to configure the Auth0 tenant itself. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. ```yaml # catalog/auth0/tenant.yaml components: terraform: auth0/tenant: vars: enabled: true # Make sure this name does not conflict with other Auth0 components, such as `auth0/app` name: auth0 support_email: "tech@acme.com" support_url: "https://acme.com" ``` ### Auth0 Tenant Creation Chicken before the egg... The Auth0 tenant must exist before we can manage it with Terraform. In order to create the Auth0 application used by the [Auth0 Terraform provider](https://registry.terraform.io/providers/auth0/auth0/latest/), we must first create the Auth0 tenant. Then once we have the Auth0 provider configured, we can import the tenant into Terraform. However, the tenant is not a resource identifiable by an ID within the Auth0 Management API! We can nevertheless import it using a random string. On first run, we import the existing tenant using a random string. It does not matter what this value is. Terraform will use the same tenant as the Auth0 application for the Terraform Auth0 Provider. Create the Auth0 tenant now using the Auth0 Management API or the Auth0 Dashboard following [the Auth0 create tenants documentation](https://auth0.com/docs/get-started/auth0-overview/create-tenants). ### Provider Pre-requisites Once the Auth0 tenant is created or you've been given access to an existing tenant, you can configure the Auth0 provider in Terraform. Follow the [Auth0 provider documentation](https://registry.terraform.io/providers/auth0/auth0/latest/docs/guides/quickstart) to create a Machine to Machine application. :::tip #### Machine to Machine App Name Use the Context Label format for the machine name for consistency. For example, `acme-plat-gbl-prod-auth0-provider`. ::: After creating the Machine to Machine application, add the app's domain, client ID, and client secret to AWS Systems Manager Parameter Store in the same account and region as this component deployment. The path for the parameters are defined by the component deployment's Null Label context ID as follows: ```hcl auth0_domain_ssm_path = "/${module.this.id}/domain" auth0_client_id_ssm_path = "/${module.this.id}/client_id" auth0_client_secret_ssm_path = "/${module.this.id}/client_secret" ``` For example, if we're deploying `auth0/tenant` into `plat-gbl-prod` and my default region is `us-west-2`, then I would add the following parameters to the `plat-prod` account in `us-west-2`: :::important Be sure that this AWS SSM parameter path does not conflict with SSM parameters used by other Auth0 components, such as `auth0/app`. In both components, the SSM parameter paths are defined by the component deployment's context ID. ::: ``` /acme-plat-gbl-prod-auth0/domain /acme-plat-gbl-prod-auth0/client_id /acme-plat-gbl-prod-auth0/client_secret ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`support_email` (`string`) required
The email address to be displayed in the Auth0 Universal Login page.
`support_url` (`string`) required
The URL to be displayed in the Auth0 Universal Login page.
### Optional Variables
`allowed_logout_urls` (`list(string)`) optional
The URLs that Auth0 can redirect to after logout. **Default value:** `[ ]`
`auth0_debug` (`bool`) optional
Enable debug mode for the Auth0 provider **Default value:** `true`
`auth0_prompt_experience` (`string`) optional
Which prompt login experience to use. Options include classic and new. **Default value:** `"new"`
`default_redirection_uri` (`string`) optional
The default redirection URI. **Default value:** `""`
`disable_clickjack_protection_headers` (`bool`) optional
Whether to disable clickjack protection headers. **Default value:** `true`
`disable_fields_map_fix` (`bool`) optional
Whether to disable fields map fix. **Default value:** `false`
`disable_management_api_sms_obfuscation` (`bool`) optional
Whether to disable management API SMS obfuscation. **Default value:** `false`
`email_provider_default_from_address` (`string`) optional
The default from address for the email provider. **Default value:** `""`
`email_provider_name` (`string`) optional
The name of the email provider. If not defined, no email provider will be created. **Default value:** `""`
`enable_public_signup_user_exists_error` (`bool`) optional
Whether to enable public signup user exists error. **Default value:** `true`
`enabled_locales` (`list(string)`) optional
The enabled locales. **Default value:** ```hcl [ "en" ] ```
`friendly_name` (`string`) optional
The friendly name of the Auth0 tenant. If not provided, the module context ID will be used. **Default value:** `""`
`idle_session_lifetime` (`number`) optional
The idle session lifetime in hours. **Default value:** `72`
`no_disclose_enterprise_connections` (`bool`) optional
Whether to disclose enterprise connections. **Default value:** `false`
`oidc_logout_prompt_enabled` (`bool`) optional
Whether the OIDC logout prompt is enabled. **Default value:** `false`
`picture_url` (`string`) optional
The URL of the picture to be displayed in the Auth0 Universal Login page. **Default value:** `"https://cloudposse.com/wp-content/uploads/2017/07/CloudPosse2-TRANSAPRENT.png"`
`provider_ssm_base_path` (`string`) optional
The base path for the SSM parameters. If not defined, this is set to the module context ID. This is also required when `var.enabled` is set to `false` **Default value:** `""`
`sandbox_version` (`string`) optional
The sandbox version. **Default value:** `"18"`
`sendgrid_api_key_ssm_path` (`string`) optional
The SSM path to the SendGrid API key. Only required if `email_provider_name` is `sendgrid`. **Default value:** `""`
`session_cookie_mode` (`string`) optional
The session cookie mode. **Default value:** `"persistent"`
`session_lifetime` (`number`) optional
The session lifetime in hours. **Default value:** `168`
`use_scope_descriptions_for_consent` (`bool`) optional
Whether to use scope descriptions for consent. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`auth0_domain`
The Auth0 custom domain
`client_id_ssm_path`
The SSM parameter path for the Auth0 client ID
`client_secret_ssm_path`
The SSM parameter path for the Auth0 client secret
`domain_ssm_path`
The SSM parameter path for the Auth0 domain
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `auth0`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`auth0_custom_domain.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/custom_domain) (resource) - [`auth0_custom_domain_verification.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/custom_domain_verification) (resource) - [`auth0_email_provider.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/email_provider) (resource) - [`auth0_prompt.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/prompt) (resource) - [`auth0_tenant.this`](https://registry.terraform.io/providers/auth0/auth0/latest/docs/resources/tenant) (resource) - [`aws_route53_record.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.auth0_client_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_client_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.auth0_domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.sendgrid_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## aws-backup This component is responsible for provisioning an AWS Backup Plan. It creates a schedule for backing up given ARNs. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ### Component Abstraction and Separation By separating the "common" settings from the component, we can first provision the IAM Role and AWS Backup Vault to prepare resources for future use without incuring cost. For example, `stacks/catalog/aws-backup/common`: ```yaml # This configuration creates the AWS Backup Vault and IAM Role, and does not incur any cost on its own. # See: https://aws.amazon.com/backup/pricing/ components: terraform: aws-backup: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: {} aws-backup/common: metadata: component: aws-backup inherits: - aws-backup vars: enabled: true iam_role_enabled: true # this will be reused vault_enabled: true # this will be reused plan_enabled: false ## Please be careful when enabling backup_vault_lock_configuration, # backup_vault_lock_configuration: ## `changeable_for_days` enables compliance mode and once the lock is set, the retention policy cannot be changed unless through account deletion! # changeable_for_days: 36500 # max_retention_days: 365 # min_retention_days: 1 ``` Then if we would like to deploy the component into a given stacks we can import the following to deploy our backup plans. Since most of these values are shared and common, we can put them in a `catalog/aws-backup/` yaml file and share them across environments. This makes deploying the same configuration to multiple environments easy. `stacks/catalog/aws-backup/defaults`: ```yaml import: - catalog/aws-backup/common components: terraform: aws-backup/plan-defaults: metadata: component: aws-backup type: abstract settings: spacelift: workspace_enabled: true depends_on: - aws-backup/common vars: enabled: true iam_role_enabled: false # reuse from aws-backup-vault vault_enabled: false # reuse from aws-backup-vault plan_enabled: true plan_name_suffix: aws-backup-defaults aws-backup/daily-plan: metadata: component: aws-backup inherits: - aws-backup/plan-defaults vars: plan_name_suffix: aws-backup-daily # https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html rules: - name: "plan-daily" schedule: "cron(0 5 ? * * *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes lifecycle: delete_after: 35 # 7 * 5 # days selection_tags: - type: STRINGEQUALS key: aws-backup/efs value: daily - type: STRINGEQUALS key: aws-backup/rds value: daily aws-backup/weekly-plan: metadata: component: aws-backup inherits: - aws-backup/plan-defaults vars: plan_name_suffix: aws-backup-weekly # https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html rules: - name: "plan-weekly" schedule: "cron(0 5 ? * SAT *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes lifecycle: delete_after: 90 # 30 * 3 # days selection_tags: - type: STRINGEQUALS key: aws-backup/efs value: weekly - type: STRINGEQUALS key: aws-backup/rds value: weekly aws-backup/monthly-plan: metadata: component: aws-backup inherits: - aws-backup/plan-defaults vars: plan_name_suffix: aws-backup-monthly # https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html rules: - name: "plan-monthly" schedule: "cron(0 5 1 * ? *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes lifecycle: delete_after: 2555 # 365 * 7 # days cold_storage_after: 90 # 30 * 3 # days selection_tags: - type: STRINGEQUALS key: aws-backup/efs value: monthly - type: STRINGEQUALS key: aws-backup/rds value: monthly ``` Deploying to a new stack (environment) then only requires: ```yaml import: - catalog/aws-backup/defaults ``` The above configuration can be used to deploy a new backup to a new region. --- ### Adding Resources to the Backup - Adding Tags Once an `aws-backup` with a plan and `selection_tags` has been established we can begin adding resources for it to backup by using the tagging method. This only requires that we add tags to the resources we wish to backup, which can be done with the following snippet: ```yaml components: terraform: vars: tags: aws-backup/resource_schedule: "daily-14day-backup" ``` Just ensure the tag key-value pair matches what was added to your backup plan and aws will take care of the rest. ### Copying across regions If we want to create a backup vault in another region that we can copy to, then we need to create another vault, and then specify that we want to copy to it. To create a vault in a region simply: ```yaml components: terraform: aws-backup: vars: plan_enabled: false # disables the plan (which schedules resource backups) ``` This will output an ARN - which you can then use as the destination in the rule object's `copy_action` (it will be specific to that particular plan), as seen in the following snippet: ```yaml components: terraform: aws-backup/plan-with-cross-region-replication: metadata: component: aws-backup inherits: - aws-backup/plan-defaults vars: plan_name_suffix: aws-backup-cross-region # https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html rules: - name: "plan-cross-region" schedule: "cron(0 5 ? * * *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes lifecycle: delete_after: 35 # 7 * 5 # days copy_action: destination_vault_arn: "arn:aws:backup::111111111111:backup-vault:--" lifecycle: delete_after: 35 ``` ### Backup Lock Configuration To enable backup lock configuration, you can use the following snippet: #### Compliance Mode Vaults locked in compliance mode cannot be deleted once the cooling-off period ("grace time") expires. During grace time, you can still remove the vault lock and change the lock configuration. To enable **Compliance Mode**, set `changeable_for_days` to a value greater than 0. Once the lock is set, the retention policy cannot be changed unless through account deletion! ```yaml # Please be careful when enabling backup_vault_lock_configuration, backup_vault_lock_configuration: # `changeable_for_days` enables compliance mode and once the lock is set, the retention policy cannot be changed unless through account deletion! changeable_for_days: 36500 max_retention_days: 365 min_retention_days: 1 ``` #### Governance Mode Vaults locked in governance mode can have the lock removed by users with sufficient IAM permissions. To enable **governance mode** ```yaml backup_vault_lock_configuration: max_retention_days: 365 min_retention_days: 1 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`advanced_backup_setting` optional
An object that specifies backup options for each resource type. **Type:** ```hcl object({ backup_options = map(string) resource_type = string }) ``` **Default value:** `null`
`backup_resources` (`list(string)`) optional
An array of strings that either contain Amazon Resource Names (ARNs) or match patterns of resources to assign to a backup plan **Default value:** `[ ]`
`backup_vault_lock_configuration` optional
The backup vault lock configuration, each vault can have one vault lock in place. This will enable Backup Vault Lock on an AWS Backup vault it prevents the deletion of backup data for the specified retention period. During this time, the backup data remains immutable and cannot be deleted or modified." `changeable_for_days` - The number of days before the lock date. If omitted creates a vault lock in `governance` mode, otherwise it will create a vault lock in `compliance` mode. **Type:** ```hcl object({ changeable_for_days = optional(number) max_retention_days = optional(number) min_retention_days = optional(number) }) ``` **Default value:** `null`
`iam_role_enabled` (`bool`) optional
Whether or not to create a new IAM Role and Policy Attachment **Default value:** `true`
`kms_key_arn` (`string`) optional
The server-side encryption key that is used to protect your backups **Default value:** `null`
`plan_enabled` (`bool`) optional
Whether or not to create a new Plan **Default value:** `true`
`plan_name_suffix` (`string`) optional
The string appended to the plan name **Default value:** `null`
`rules` optional
An array of rule maps used to define schedules in a backup plan **Type:** ```hcl list(object({ name = string schedule = optional(string) enable_continuous_backup = optional(bool) start_window = optional(number) completion_window = optional(number) lifecycle = optional(object({ cold_storage_after = optional(number) delete_after = optional(number) opt_in_to_archive_for_supported_resources = optional(bool) })) copy_action = optional(object({ destination_vault_arn = optional(string) lifecycle = optional(object({ cold_storage_after = optional(number) delete_after = optional(number) opt_in_to_archive_for_supported_resources = optional(bool) })) })) })) ``` **Default value:** `[ ]`
`selection_tags` (`list(map(string))`) optional
An array of tag condition objects used to filter resources based on tags for assigning to a backup plan **Default value:** `[ ]`
`vault_enabled` (`bool`) optional
Whether or not a new Vault should be created **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`backup_plan_arn`
Backup Plan ARN
`backup_plan_version`
Unique, randomly generated, Unicode, UTF-8 encoded string that serves as the version ID of the backup plan
`backup_selection_id`
Backup Selection ID
`backup_vault_arn`
Backup Vault ARN
`backup_vault_id`
Backup Vault ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `backup` | 1.1.0 | [`cloudposse/backup/aws`](https://registry.terraform.io/modules/cloudposse/backup/aws/1.1.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## aws-config This component is responsible for configuring AWS Config. AWS Config service enables you to track changes to your AWS resources over time. It continuously monitors and records configuration changes to your AWS resources and provides you with a detailed view of the relationships between those resources. With AWS Config, you can assess, audit, and evaluate the configurations of your AWS resources for compliance, security, and governance purposes. Some of the key features of AWS Config include: - Configuration history: AWS Config maintains a detailed history of changes to your AWS resources, allowing you to see when changes were made, who made them, and what the changes were. - Configuration snapshots: AWS Config can take periodic snapshots of your AWS resources configurations, giving you a point-in-time view of their configuration. - Compliance monitoring: AWS Config provides a range of pre-built rules and checks to monitor your resources for compliance with best practices and industry standards. - Relationship mapping: AWS Config can map the relationships between your AWS resources, enabling you to see how changes to one resource can impact others. - Notifications and alerts: AWS Config can send notifications and alerts when changes are made to your AWS resources that could impact their compliance or security posture. :::warning #### AWS Config Limitations You'll also want to be aware of some limitations with AWS Config: - The maximum number of AWS Config rules that can be evaluated in a single account is 1000. - This can be mitigated by removing rules that are duplicated across packs. You'll have to manually search for these duplicates. - You can also look for rules that do not apply to any resources and remove those. You'll have to manually click through rules in the AWS Config interface to see which rules are not being evaluated. - If you end up still needing more than 1000 rules, one recommendation is to only run packs on a schedule with a lambda that removes the pack after results are collected. If you had different schedule for each day of the week, that would mean 7000 rules over the week. The aggregators would not be able to handle this, so you would need to make sure to store them somewhere else (i.e. S3) so the findings are not lost. - See the [Audit Manager docs](https://aws.amazon.com/blogs/mt/integrate-across-the-three-lines-model-part-2-transform-aws-config-conformance-packs-into-aws-audit-manager-assessments/) if you think you would like to convert conformance packs to custom Audit Manager assessments. - The maximum number of AWS Config conformance packs that can be created in a single account is 50. ::: Overall, AWS Config provides you with a powerful toolset to help you monitor and manage the configurations of your AWS resources, ensuring that they remain compliant, secure, and properly configured over time. ## Usage ## Prerequisites As part of [CIS AWS Foundations 1.20](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-cis-controls.html#securityhub-cis-controls-1.20), this component assumes that a designated support IAM role with the following permissions has been deployed to every account in the organization: ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "AllowSupport", "Effect": "Allow", "Action": ["support:*"], "Resource": "*" }, { "Sid": "AllowTrustedAdvisor", "Effect": "Allow", "Action": "trustedadvisor:Describe*", "Resource": "*" } ] } ``` Before deploying this AWS Config component `config-bucket` and `cloudtrail-bucket` should be deployed first. ## Usage **Stack Level**: Regional or Global This component has a `default_scope` variable for configuring if it will be an organization-wide or account-level component by default. Note that this can be overridden by the `scope` variable in the `conformance_packs` items. :::tip #### Using the account default_scope If default_scope == `account`, AWS Config is regional AWS service, so this component needs to be deployed to all regions. If an individual `conformance_packs` item has `scope` set to `organization`, that particular pack will be deployed to the organization level. ::: :::tip #### Using the organization default_scope If default_scope == `organization`, AWS Config is global unless overridden in the `conformance_packs` items. You will need to update your org to allow the `config-multiaccountsetup.amazonaws.com` service access principal for this to work. If you are using our `account` component, just add that principal to the `aws_service_access_principals` variable. ::: At the AWS Organizational level, the Components designate an account to be the `central collection account` and a single region to be the `central collection region` so that compliance information can be aggregated into a central location. Logs are typically written to the `audit` account and AWS Config deployed into to the `security` account. Here's an example snippet for how to use this component: ```yaml components: terraform: aws-config: vars: enabled: true account_map_tenant: core az_abbreviation_type: fixed # In each AWS account, an IAM role should be created in the main region. # If the main region is set to us-east-1, the value of the var.create_iam_role variable should be true. # For all other regions, the value of var.create_iam_role should be false. create_iam_role: false central_resource_collector_account: core-security global_resource_collector_region: us-east-1 config_bucket_env: ue1 config_bucket_stage: audit config_bucket_tenant: core conformance_packs: - name: Operational-Best-Practices-for-CIS-AWS-v1.4-Level2 conformance_pack: https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-CIS-AWS-v1.4-Level2.yaml parameter_overrides: AccessKeysRotatedParamMaxAccessKeyAge: '45' - name: Operational-Best-Practices-for-HIPAA-Security.yaml conformance_pack: https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-HIPAA-Security.yaml parameter_overrides: ... (etc) managed_rules: access-keys-rotated: identifier: ACCESS_KEYS_ROTATED description: "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is NON_COMPLIANT if the access keys have not been rotated for more than maxAccessKeyAge number of days." input_parameters: maxAccessKeyAge: "30" enabled: true tags: { } ``` ## Deployment Apply to your central region security account ```sh atmos terraform plan aws-config-{central-region} --stack core-{central-region}-security -var=create_iam_role=true ``` For example when central region is `us-east-1`: ```sh atmos terraform plan aws-config-ue1 --stack core-ue1-security -var=create_iam_role=true ``` Apply aws-config to all stacks in all stages. ```sh atmos terraform plan aws-config-{each region} --stack {each region}-{each stage} ``` ## Variables ### Required Variables
`central_resource_collector_account` (`string`) required
The name of the account that is the centralized aggregation account.
`config_bucket_env` (`string`) required
The environment of the AWS Config S3 Bucket
`config_bucket_stage` (`string`) required
The stage of the AWS Config S3 Bucket
`global_resource_collector_region` (`string`) required
The region that collects AWS Config data for global resources such as IAM
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_tenant` (`string`) optional
(Optional) The tenant where the account_map component required by remote-state is deployed. **Default value:** `""`
`az_abbreviation_type` (`string`) optional
AZ abbreviation type, `fixed` or `short` **Default value:** `"fixed"`
`config_bucket_component_name` (`string`) optional
The name of the config-bucket component **Default value:** `"config-bucket"`
`config_bucket_tenant` (`string`) optional
(Optional) The tenant of the AWS Config S3 Bucket **Default value:** `""`
`config_component_name` (`string`) optional
The name of the aws config component (i.e., this component) **Default value:** `"aws-config"`
`conformance_packs` optional
List of conformance packs. Each conformance pack is a map with the following keys: name, conformance_pack, parameter_overrides. For example: conformance_packs = [ \{ name = "Operational-Best-Practices-for-CIS-AWS-v1.4-Level1" conformance_pack = "https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-CIS-AWS-v1.4-Level1.yaml" parameter_overrides = \{ "AccessKeysRotatedParamMaxAccessKeyAge" = "45" \} \}, \{ name = "Operational-Best-Practices-for-CIS-AWS-v1.4-Level2" conformance_pack = "https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-CIS-AWS-v1.4-Level2.yaml" parameter_overrides = \{ "IamPasswordPolicyParamMaxPasswordAge" = "45" \} \} ] Complete list of AWS Conformance Packs managed by AWSLabs can be found here: https://github.com/awslabs/aws-config-rules/tree/master/aws-config-conformance-packs **Type:** ```hcl list(object({ name = string conformance_pack = string parameter_overrides = map(string) scope = optional(string, null) })) ``` **Default value:** `[ ]`
`create_iam_role` (`bool`) optional
Flag to indicate whether an IAM Role should be created to grant the proper permissions for AWS Config **Default value:** `false`
`default_scope` (`string`) optional
The default scope of the conformance pack. Valid values are `account` and `organization`. **Default value:** `"account"`
`delegated_accounts` (`set(string)`) optional
The account IDs of other accounts that will send their AWS Configuration or Security Hub data to this account **Default value:** `null`
`global_collector_component_name_pattern` (`string`) optional
A string formatting pattern used to construct or look up the name of the global AWS Config collector region component. This pattern should align with the regional naming convention of the aws-config component. For example, if the pattern is "%s-%s" and you pass ("aws-config", "use1"), the resulting component name will be "aws-config-use1". Adjust this pattern if your environment uses a different naming convention for regional AWS Config components. **Default value:** `"%s-%s"`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`iam_role_arn` (`string`) optional
The ARN for an IAM Role AWS Config uses to make read or write requests to the delivery channel and to describe the AWS resources associated with the account. This is only used if create_iam_role is false. If you want to use an existing IAM Role, set the variable to the ARN of the existing role and set create_iam_role to `false`. See the AWS Docs for further information: http://docs.aws.amazon.com/config/latest/developerguide/iamrole-permissions.html **Default value:** `null`
`iam_roles_environment_name` (`string`) optional
The name of the environment where the IAM roles are provisioned **Default value:** `"gbl"`
`managed_rules` optional
A list of AWS Managed Rules that should be enabled on the account. See the following for a list of possible rules to enable: https://docs.aws.amazon.com/config/latest/developerguide/managed-rules-by-aws-config.html Example: ``` managed_rules = { access-keys-rotated = { identifier = "ACCESS_KEYS_ROTATED" description = "Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge. The rule is NON_COMPLIANT if the access keys have not been rotated for more than maxAccessKeyAge number of days." input_parameters = { maxAccessKeyAge : "90" } enabled = true tags = {} } } ``` **Type:** ```hcl map(object({ description = string identifier = string input_parameters = any tags = map(string) enabled = bool })) ``` **Default value:** `{ }`
`privileged` (`bool`) optional
True if the default provider already has access to the backend **Default value:** `false`
`root_account_stage` (`string`) optional
The stage name for the Organization root (master) account **Default value:** `"root"`
`team_roles_component_name` (`string`) optional
The name of the team-roles component **Default value:** `"aws-team-roles"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_config_configuration_recorder_id`
The ID of the AWS Config Recorder
`aws_config_iam_role`
The ARN of the IAM Role used for AWS Config
`storage_bucket_arn`
Storage Config bucket ARN
`storage_bucket_id`
Storage Config bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `aws_config` | 1.5.3 | [`cloudposse/config/aws`](https://registry.terraform.io/modules/cloudposse/config/aws/1.5.3) | n/a `aws_config_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `aws_team_roles` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `config_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `conformance_pack` | 1.5.3 | [`cloudposse/config/aws//modules/conformance-pack`](https://registry.terraform.io/modules/cloudposse/config/aws/modules/conformance-pack/1.5.3) | n/a `global_collector_region` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `org_conformance_pack` | latest | [`./modules/org-conformance-pack`](https://registry.terraform.io/modules/./modules/org-conformance-pack/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_partition.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## org-conformance-pack # AWS Config Conformance Pack This module deploys a [Conformance Pack](https://docs.aws.amazon.com/config/latest/developerguide/conformance-packs.html). A conformance pack is a collection of AWS Config rules and remediation actions that can be easily deployed as a single entity in an account and a Region or across an organization in AWS Organizations. Conformance packs are created by authoring a YAML template that contains the list of AWS Config managed or custom rules and remediation actions. The Conformance Pack cannot be deployed until AWS Config is deployed, which can be deployed using the [aws-config](../../) component. ## Usage First, make sure your root `account` allows the service access principal `config-multiaccountsetup.amazonaws.com` to update child organizations. You can see the docs on the account module here: [aws_service_access_principals](/components/library/aws/account/#aws_service_access_principals) Then you have two options: - Set the `default_scope` of the parent `aws-config` component to be `organization` (can be overridden by the `scope` of each `conformance_packs` item) - Set the `scope` of the `conformance_packs` item to be `organization` An example YAML stack config for Atmos follows. Note, that both options are shown for demonstration purposes. In practice you should only have one `aws-config` per account: ```yaml components: terraform: account: vars: aws_service_access_principals: - config-multiaccountsetup.amazonaws.com aws-config/cis/level-1: vars: conformance_packs: - name: Operational-Best-Practices-for-CIS-AWS-v1.4-Level1 conformance_pack: https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-CIS-AWS-v1.4-Level1.yaml scope: organization aws-config/cis/level-2: vars: default_scope: organization conformance_packs: - name: Operational-Best-Practices-for-CIS-AWS-v1.4-Level2 conformance_pack: https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-CIS-AWS-v1.4-Level2.yaml ``` ## Variables ### Required Variables
`conformance_pack` (`string`) required
The URL to a Conformance Pack
### Optional Variables
`parameter_overrides` (`map(any)`) optional
A map of parameters names to values to override from the template **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN for the AWS Config Organization Conformance Pack
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `http`, version: `>= 2.1.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `http`, version: `>= 2.1.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_config_organization_conformance_pack.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_organization_conformance_pack) (resource) ## Data Sources The following data sources are used by this module: - [`http_http.conformance_pack`](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) (data source) None --- ## aws-inspector This component is responsible for provisioning an [AWS Inspector](https://docs.aws.amazon.com/inspector/latest/user/what-is-inspector.html) by installing the [Inspector agent](https://repost.aws/knowledge-center/set-up-amazon-inspector) across all EC2 instances and applying the Inspector rules. AWS Inspector is a security assessment service offered by Amazon Web Services (AWS). It helps you analyze and evaluate the security and compliance of your applications and infrastructure deployed on AWS. AWS Inspector automatically assesses the resources within your AWS environment, such as Amazon EC2 instances, for potential security vulnerabilities and deviations from security best practices. Here are some key features and functionalities of AWS Inspector: - **Security Assessments:** AWS Inspector performs security assessments by analyzing the behavior of your resources and identifying potential security vulnerabilities. It examines the network configuration, operating system settings, and installed software to detect common security issues. - **Vulnerability Detection:** AWS Inspector uses a predefined set of rules to identify common vulnerabilities, misconfigurations, and security exposures. It leverages industry-standard security best practices and continuously updates its knowledge base to stay current with emerging threats. - **Agent-Based Architecture:** AWS Inspector utilizes an agent-based approach, where you install an Inspector agent on your EC2 instances. The agent collects data about the system and its configuration, securely sends it to AWS Inspector, and allows for more accurate and detailed assessments. - **Security Findings:** After performing an assessment, AWS Inspector generates detailed findings that highlight security vulnerabilities, including their severity level, impact, and remediation steps. These findings can help you prioritize and address security issues within your AWS environment. - **Integration with AWS Services:** AWS Inspector seamlessly integrates with other AWS services, such as AWS CloudFormation, AWS Systems Manager, and AWS Security Hub. This allows you to automate security assessments, manage findings, and centralize security information across your AWS infrastructure. ## Usage Stack Level: Regional Example stack snippet: ```yaml components: terraform: aws-inspector: vars: enabled: true enabled_rules: - cis ``` The `aws-inspector` component can be included in a Terraform stack configuration. In the example, it is enabled with `enabled: true`. The `enabled_rules` variable specifies a list of rules to enable and uses short forms (e.g., `cis`) that automatically resolve to the correct rule package ARN for the target region. See the `var.enabled_rules` input for available short forms. For a comprehensive list of rules and their corresponding ARNs, refer to the Amazon Inspector ARNs for rules packages documentation. Customize the configuration and enabled rules to tailor security assessments to your requirements and compliance standards. ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`enabled_rules` (`list(string)`) optional
A list of AWS Inspector rules that should run on a periodic basis. Valid values are `cve`, `cis`, `nr`, `sbp` which map to the appropriate [Inspector rule arns by region](https://docs.aws.amazon.com/inspector/latest/userguide/inspector_rules-arns.html). **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`inspector`
The AWS Inspector module outputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `inspector` | 0.4.0 | [`cloudposse/inspector/aws`](https://registry.terraform.io/modules/cloudposse/inspector/aws/0.4.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_association.install_agent`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_association) (resource) ## Data Sources The following data sources are used by this module: --- ## aws-inspector2 This component is responsible for configuring Inspector V2 within an AWS Organization. ## Usage **Stack Level**: Regional ## Deployment Overview The deployment of this component requires multiple runs with different variable settings to properly configure the AWS Organization. First, you delegate Inspector V2 central management to the Administrator account (usually `security` account). After the Administrator account is delegated, we configure it to manage Inspector V2 across all the Organization accounts and send all their findings to that account. In the examples below, we assume that the AWS Organization Management account is `root` and the AWS Organization Delegated Administrator account is `security`. ### Deploy to Organization Management Account First, the component is deployed to the AWS Organization Management account `root` in each region in order to configure the [AWS Delegated Administrator account](https://docs.aws.amazon.com/inspector/latest/user/designating-admin.html) that operates Amazon Inspector V2. ```yaml # ue1-root components: terraform: aws-inspector2/delegate-orgadmin/ue1: metadata: component: aws-inspector2 vars: enabled: true region: us-east-1 ``` ### Deploy Organization Settings in Delegated Administrator Account Now the component can be deployed to the Delegated Administrator Account `security` to create the organization-wide configuration for all the Organization accounts. Note that `var.admin_delegated` set to `true` indicates that the delegation has already been performed from the Organization Management account, and only the resources required for organization-wide configuration will be created. ```yaml # ue1-security components: terraform: aws-inspector2/orgadmin-configuration/ue1: metadata: component: aws-inspector2 vars: enabled: true region: us-east-1 admin_delegated: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_tenant` (`string`) optional
The tenant where the `account_map` component required by remote-state is deployed **Default value:** `"core"`
`admin_delegated` (`bool`) optional
A flag to indicate if the AWS Organization-wide settings should be created. This can only be done after the GuardDuty Administrator account has already been delegated from the AWS Org Management account (usually 'root'). See the Deployment section of the README for more information. **Default value:** `false`
`auto_enable_ec2` (`bool`) optional
Whether Amazon EC2 scans are automatically enabled for new members of the Amazon Inspector organization. **Default value:** `true`
`auto_enable_ecr` (`bool`) optional
Whether Amazon ECR scans are automatically enabled for new members of the Amazon Inspector organization. **Default value:** `true`
`auto_enable_lambda` (`bool`) optional
Whether Lambda Function scans are automatically enabled for new members of the Amazon Inspector organization. **Default value:** `true`
`delegated_administrator_account_name` (`string`) optional
The name of the account that is the AWS Organization Delegated Administrator account **Default value:** `"security"`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`member_association_excludes` (`list(string)`) optional
List of account names to exclude from Amazon Inspector member association **Default value:** `[ ]`
`organization_management_account_name` (`string`) optional
The name of the AWS Organization management account **Default value:** `null`
`privileged` (`bool`) optional
true if the default provider already has access to the backend **Default value:** `false`
`root_account_stage` (`string`) optional
The stage name for the Organization root (management) account. This is used to lookup account IDs from account names using the `account-map` component. **Default value:** `"root"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_inspector2_member_association`
The Inspector2 member association resource.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Providers - `aws`, version: `>= 5.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_inspector2_delegated_admin_account.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector2_delegated_admin_account) (resource) - [`aws_inspector2_enabler.delegated_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector2_enabler) (resource) - [`aws_inspector2_enabler.member_accounts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector2_enabler) (resource) - [`aws_inspector2_member_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector2_member_association) (resource) - [`aws_inspector2_organization_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector2_organization_configuration) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) --- ## aws-saml This component provisions SAML metadata into AWS IAM as new SAML providers. For Okta integrations (when `okta` is included in the key provided to the `saml_providers` input), it also creates an Okta API user and an associated Access Key pair, and stores the credentials in AWS SSM Parameter Store. ## Usage Stack Level: Global, in the account to which users will log in, typically only `identity`. Here's an example snippet for how to use this component. IMPORTANT: The given SAML metadata files must exist at the root of the module. ```yaml components: terraform: aws-saml: vars: enabled: true saml_providers: enabled: true example-okta: Okta_metadata_example.xml example-gsuite: GoogleIDPMetadata-example.com.xml ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`saml_providers` (`map(string)`) required
Map of provider names to XML data filenames
### Optional Variables
`attach_permissions_to_group` (`bool`) optional
If true, attach IAM permissions to a group rather than directly to the API user **Default value:** `false`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`okta_api_users`
Map of OKTA API Users
`saml_provider_arns`
Map of SAML provider names to provider ARNs
`saml_provider_assume_role_policy`
JSON "assume role" policy document to use for roles allowed to log in via SAML
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `okta_api_user` | latest | [`./modules/okta-user`](https://registry.terraform.io/modules/./modules/okta-user/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_saml_provider.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_saml_provider) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.saml_provider_assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## aws-shield This component is responsible for enabling AWS Shield Advanced Protection for the following resources: - Application Load Balancers (ALBs) - CloudFront Distributions - Elastic IPs - Route53 Hosted Zones This component assumes that resources it is configured to protect are not already protected by other components that have their `xxx_aws_shield_protection_enabled` variable set to `true`. This component also requires that the account where the component is being provisioned to has been [subscribed to AWS Shield Advanced](https://docs.aws.amazon.com/waf/latest/developerguide/enable-ddos-prem.html). ## Usage **Stack Level**: Global or Regional The following snippet shows how to use all of this component's features in a stack configuration: ```yaml components: terraform: aws-shield: settings: spacelift: workspace_enabled: true vars: enabled: true route53_zone_names: - test.ue1.example.net alb_names: - k8s-common-2c5f23ff99 cloudfront_distribution_ids: - EDFDVBD632BHDS5 eips: - 3.214.128.240 - 35.172.208.150 - 35.171.70.50 ``` A typical global configuration will only include the `route53_zone_names` and `cloudfront_distribution_ids` variables, as global Route53 Hosted Zones may exist in that account, and because CloudFront is a global AWS service. A global stack configuration will not have a VPC, and hence `alb_names` and `eips` should not be defined: ```yaml components: terraform: aws-shield: settings: spacelift: workspace_enabled: true vars: enabled: true route53_zone_names: - test.example.net cloudfront_distribution_ids: - EDFDVBD632BHDS5 ``` Regional stack configurations will typically make use of all resources except for `cloudfront_distribution_ids`: ```yaml components: terraform: aws-shield: settings: spacelift: workspace_enabled: true vars: route53_zone_names: - test.ue1.example.net alb_names: - k8s-common-2c5f23ff99 eips: - 3.214.128.240 - 35.172.208.150 - 35.171.70.50 ``` Stack configurations which rely on components with a `xxx_aws_shield_protection_enabled` variable should set that variable to `true` and leave the corresponding variable for this component as empty, relying on that component's AWS Shield Advanced functionality instead. This leads to more simplified inter-component dependencies and minimizes the need for maintaining the provisioning order during a cold-start. ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alb_names` (`list(string)`) optional
list of ALB names which will be protected with AWS Shield Advanced **Default value:** `[ ]`
`alb_protection_enabled` (`bool`) optional
Enable ALB protection. By default, ALB names are read from the EKS cluster ALB control group **Default value:** `false`
`cloudfront_distribution_ids` (`list(string)`) optional
list of CloudFront Distribution IDs which will be protected with AWS Shield Advanced **Default value:** `[ ]`
`eips` (`list(string)`) optional
List of Elastic IPs which will be protected with AWS Shield Advanced **Default value:** `[ ]`
`route53_zone_names` (`list(string)`) optional
List of Route53 Hosted Zone names which will be protected with AWS Shield Advanced **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`application_load_balancer_protections`
AWS Shield Advanced Protections for ALBs
`cloudfront_distribution_protections`
AWS Shield Advanced Protections for CloudFront Distributions
`elastic_ip_protections`
AWS Shield Advanced Protections for Elastic IPs
`route53_hosted_zone_protections`
AWS Shield Advanced Protections for Route53 Hosted Zones
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_shield_protection.alb_shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) - [`aws_shield_protection.cloudfront_shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) - [`aws_shield_protection.eip_shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) - [`aws_shield_protection.route53_zone_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) ## Data Sources The following data sources are used by this module: - [`aws_alb.alb`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/alb) (data source) - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_cloudfront_distribution.cloudfront_distribution`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/cloudfront_distribution) (data source) - [`aws_eip.eip`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eip) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_route53_zone.route53_zone`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## aws-ssosync # Component: `ssosync` Deploys [AWS ssosync](https://github.com/awslabs/ssosync) to sync Google Groups with AWS SSO. AWS `ssosync` is a Lambda application that regularly manages Identity Store users. This component requires manual deployment by a privileged user because it deploys a role in the root or identity management account. ## Usage You should be able to deploy the `ssosync` component to the same account as `aws-sso`. Typically that is the `core-gbl-root` or `gbl-root` stack. **Stack Level**: Global **Deployment**: Must be deployed by `managers` team-member or SuperAdmin using `atmos` CLI (since this is a root account deployment). This could also be deployed in an identity management account. The following is an example snippet for how to use this component: (`stacks/catalog/aws-ssosync.yaml`) ```yaml components: terraform: ssosync: vars: enabled: true name: ssosync google_admin_email: admin@acme.com log_format: text log_level: warn schedule_expression: "rate(15 minutes)" # Filter the groups that will be synced and is optional (default: all groups) # This supports wild cards `*` google_group_match: - "email='developer@acme.com'" - "email='aws@acme.com'" - "name='Acme Team'" ``` We recommend following a similar process to what the [AWS ssosync](https://github.com/awslabs/ssosync) documentation recommends. ### Deployment Overview of steps: 1. Configure AWS IAM Identity Center 1. Configure Google Cloud console 1. Configure Google Admin console 1. Deploy the `aws-ssosync` component 1. Deploy the `aws-sso` component #### 1. Configure AWS IAM Identity Center (AWS SSO) Follow [AWS documentation to configure SAML and SCIM with Google Workspace and IAM Identity Center](https://docs.aws.amazon.com/singlesignon/latest/userguide/gs-gwp.html). Do steps 1-4. **Step 5: Google Workspace: Configure auto provisioning is impossible.** As part of this process, save the SCIM endpoint token and URL. Then in AWS SSM Parameter Store, create two `SecureString` parameters in the same account used for AWS SSO. This is usually the root account in the primary region. These can be found by clicking `Enable Automatic provisioning` in the AWS IAM Identity Center console. ``` # Typically looks like `https://scim.us-east-2.amazonaws.com/.../scim/v2` /ssosync/scim_endpoint_url # Typically looks like a base64 encoded value /ssosync/scim_endpoint_access_token ``` Select `Settings`, under the `Identity Source` section, copy the `Identity Store ID` and create the following parameter: ``` # Typically looks like `d-000000aaaa` /ssosync/identity_store_id ``` #### 2. Configure Google Cloud console Within the [Google Cloud console](https://console.cloud.google.com), we need to create a new Google Project and Service Account and enable the Admin SDK API. Follow these steps: 2. Create a new project. Give the project a descriptive name such as `AWS SSO Sync` 3. Enable Admin SDK in APIs: `APIs & Services > Enabled APIs & Services > + ENABLE APIS AND SERVICES` ![Enable Admin SDK](https://raw.githubusercontent.com/cloudposse-terraform-components/aws-ssosync/main/src/docs/img/admin_sdk.png) 4. Create Service Account: `IAM & Admin > Service Accounts > Create Service Account` [(ref)](https://cloud.google.com/iam/docs/service-accounts-create). ![Create Service Account](https://raw.githubusercontent.com/cloudposse-terraform-components/aws-ssosync/main/src/docs/img/create_service_account.png) 5. Download credentials for the new Service Account: `IAM & Admin > Service Accounts > select Service Account > Keys > ADD KEY > Create new key > JSON` ![Download Credentials](https://raw.githubusercontent.com/cloudposse-terraform-components/aws-ssosync/main/src/docs/img/dl_service_account_creds.png) 6. Save the JSON credentials as a new `SecureString` AWS SSM parameter in the same account used for AWS SSO. Use the full JSON string as the value for the parameter. ``` /ssosync/google_credentials ``` #### 3. Configure Google Admin console - Open the [Google Admin console](https://admin.google.com/) - From your domain’s Admin console, go to `Main menu menu > Security > Access and data control > API controls` - In the Domain wide delegation pane, select `Manage Domain Wide Delegation`. - Click `Add new`. - In the Client ID field, enter the `Unique ID` of the Service Account created in step 2, this should be a 22 number string. - In the OAuth Scopes field, enter ```console https://www.googleapis.com/auth/admin.directory.group.readonly,https://www.googleapis.com/auth/admin.directory.group.member.readonly,https://www.googleapis.com/auth/admin.directory.user.readonly ``` #### 4. Deploy the `aws-ssosync` component Make sure that all four of the following SSM parameters exist in the target account and region: - `/ssosync/scim_endpoint_url` - `/ssosync/scim_endpoint_access_token` - `/ssosync/identity_store_id` - `/ssosync/google_credentials` If deployed successfully, Groups and Users should be programmatically copied from the Google Workspace into AWS IAM Identity Center on the given schedule. If these Groups are not showing up, check the CloudWatch logs for the new Lambda function and refer the [FAQs](#FAQ) included below. #### 5. Deploy the `aws-sso` component Use the names of the Groups now provisioned programmatically in the `aws-sso` component catalog. Follow the [aws-sso](https://github.com/cloudposse-terraform-components/aws-ssosync/tree/main/aws-ssosync/../aws-sso/) component documentation to deploy the `aws-sso` component. ### FAQ #### Why is the tool forked by `Benbentwo`? The `awslabs` tool requires AWS Secrets Managers for the Google Credentials. However, we would prefer to use AWS SSM to store all credentials consistency and not require AWS Secrets Manager. Therefore we've created a Pull Request and will point to a fork until the PR is merged. Ref: - https://github.com/awslabs/ssosync/pull/133 - https://github.com/awslabs/ssosync/issues/93 #### What should I use for the Google Admin Email Address? The Service Account created will assume the User given by `--google-admin` / `SSOSYNC_GOOGLE_ADMIN` / `var.google_admin_email`. Therefore, this user email must be a valid Google admin user in your organization. This is not the same email as the Service Account. If Google fails to query Groups, you may see the following error: ```console Notifying Lambda and mark this execution as Failure: googleapi: Error 404: Domain not found., notFound ``` #### Common Group Name Query Error If filtering group names using query strings, make sure the provided string is valid. For example, `google_group_match: "name:aws*"` is incorrect. Instead use `google_group_match: "Name:aws*"` If not, you may again see the same error message: ```console Notifying Lambda and mark this execution as Failure: googleapi: Error 404: Domain not found., notFound ``` Ref: > The specific error you are seeing is because the google api doesn't like the query string you provided for the -g > parameter. try -g "Name:Fuel\*" https://github.com/awslabs/ssosync/issues/91 ## Variables ### Required Variables
`google_admin_email` (`string`) required
Google Admin email
`region` (`string`) required
AWS Region where AWS SSO is enabled
### Optional Variables
`architecture` (`string`) optional
Architecture of the Lambda function **Default value:** `"x86_64"`
`google_credentials_ssm_path` (`string`) optional
SSM Path for `ssosync` secrets **Default value:** `"/ssosync"`
`google_group_match` (`list(string)`) optional
Google Workspace group filter query parameter, example: 'name:Admin* email:aws-*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-groups **Default value:** `[ ]`
`google_user_match` (`list(string)`) optional
Google Workspace user filter query parameter, example: 'name:John* email:admin*', see: https://developers.google.com/admin-sdk/directory/v1/guides/search-users **Default value:** `[ ]`
`ignore_groups` (`string`) optional
Ignore these Google Workspace groups **Default value:** `""`
`ignore_users` (`string`) optional
Ignore these Google Workspace users **Default value:** `""`
`include_groups` (`string`) optional
Include only these Google Workspace groups. (Only applicable for sync_method user_groups) **Default value:** `""`
`log_format` (`string`) optional
Log format for Lambda function logging **Default value:** `"json"`
`log_level` (`string`) optional
Log level for Lambda function logging **Default value:** `"warn"`
`schedule_expression` (`string`) optional
Schedule for trigger the execution of ssosync (see CloudWatch schedule expressions) **Default value:** `"rate(15 minutes)"`
`ssosync_url_prefix` (`string`) optional
URL prefix for ssosync binary **Default value:** `"https://github.com/cloudposse/ssosync/releases/download"`
`ssosync_version` (`string`) optional
Version of ssosync to use **Default value:** `"v3.0.0"`
`sync_method` (`string`) optional
Sync method to use **Default value:** `"groups"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the lambda function
`invoke_arn`
Invoke ARN of the lambda function
`qualified_arn`
ARN identifying your Lambda Function Version (if versioning is enabled via publish = true)
`ssosync_artifact_url`
URL of the ssosync artifact
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `archive`, version: `>= 2.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `null`, version: `>= 3.0` - `random`, version: `>= 1.4.1` ### Providers - `archive`, version: `>= 2.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `null`, version: `>= 3.0` - `random`, version: `>= 1.4.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `ssosync_artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | This module is the resource that actually downloads the artifact from GitHub as a tar.gz `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`archive_file.lambda`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/resources/file) (resource) - [`aws_cloudwatch_event_rule.ssosync`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.ssosync`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_lambda_function.ssosync`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.allow_cloudwatch_execution`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`null_resource.extract_my_tgz`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`random_pet.zip_recreator`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.ssosync_lambda_assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ssosync_lambda_identity_center`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_ssm_parameter.google_credentials`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.identity_store_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.scim_endpoint_access_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.scim_endpoint_url`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## aws-team-roles This component is responsible for provisioning user and system IAM roles outside the `identity` account. It sets them up to be assumed from the "team" roles defined in the `identity` account by [the `aws-teams` component](https://github.com/cloudposse-terraform-components/aws-team-roles/tree/main/aws-team-roles/../aws-teams) and/or the AWS SSO permission sets defined in [the `aws-sso` component](https://github.com/cloudposse-terraform-components/aws-team-roles/tree/main/aws-team-roles/../aws-sso), and/or be directly accessible via SAML logins. ### Privileges are Granted to Users via IAM Policies Each role is granted permissions by attaching a list of IAM policies to the IAM role via its `role_policy_arns` list. You can configure AWS managed policies by entering the ARNs of the policies directly into the list, or you can create a custom policy as follows: 1. Give the policy a name, e.g. `eks-admin`. We will use `NAME` as a placeholder for the name in the instructions below. 2. Create a file in the `aws-teams` directory with the name `policy-NAME.tf`. 3. In that file, create a policy as follows: ```hcl data "aws_iam_policy_document" "NAME" { # Define the policy here } resource "aws_iam_policy" "NAME" { name = format("%s-NAME", module.this.id) policy = data.aws_iam_policy_document.NAME.json tags = module.this.tags } ``` 4. Create a file named `additional-policy-map_override.tf` in the `aws-team-roles` directory (if it does not already exist). This is a [terraform override file](https://developer.hashicorp.com/terraform/language/files/override), meaning its contents will be merged with the main terraform file, and any locals defined in it will override locals defined in other files. Having your code in this separate override file makes it possible for the component to provide a placeholder local variable so that it works without customization, while allowing you to customize the component and still update it without losing your customizations. 5. In that file, redefine the local variable `overridable_additional_custom_policy_map` map as follows: ```hcl locals { overridable_additional_custom_policy_map = { NAME = aws_iam_policy.NAME.arn } } ``` If you have multiple custom policies, add each one to the map in the form `NAME = aws_iam_policy.NAME.arn`. 6. With that done, you can now attach that policy by adding the name to the `role_policy_arns` list. For example: ```yaml role_policy_arns: - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" - "NAME" ``` ## Usage **Stack Level**: Global **Deployment**: Must be deployed by _SuperAdmin_ using `atmos` CLI Here's an example snippet for how to use this component. This specific usage is an example only, and not intended for production use. You set the defaults in one YAML file, and import that file into each account's Global stack (except for the `identity` account itself). If desired, you can make account-specific changes by overriding settings, for example - Disable entire roles in the account by setting `enabled: false` - Limit who can access the role by setting a different value for `trusted_teams` - Change the permissions available to that role by overriding the `role_policy_arns` (not recommended, limit access to the role or create a different role with the desired set of permissions instead). Note that when overriding, **maps are deep merged, but lists are replaced**. This means, for example, that your setting of `trusted_primary_roles` in an override completely replaces the default, it does not add to it, so if you want to allow an extra "primary" role to have access to the role, you have to include all the default "primary" roles in the list, too, or they will lose access. ```yaml components: terraform: aws-team-roles: backend: s3: # Override the default Role for accessing the backend, because SuperAdmin is not allowed to assume that role role_arn: null vars: enabled: true roles: # `template` serves as the default configuration for other roles via the YAML anchor. # However, `atmos` does not support "import" of YAML anchors, so if you define a new role # in another file, you will not be able to reference this anchor. template: &user-template # If `enabled: false`, the role will not be created in this account enabled: false # `max_session_duration` set the maximum session duration (in seconds) for the IAM roles. # This setting can have a value from 3600 (1 hour) to 43200 (12 hours). # For roles people log into via SAML, a long duration is convenient to prevent them # from having to frequently re-authenticate. # For roles assumed from some other role, the setting is practically irrelevant, because # the AssumeRole API limits the duration to 1 hour in any case. # References: # - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html # - https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html max_session_duration: 3600 # 1 hour in seconds # role_policy_arns are the IAM Policy ARNs to attach to this policy. In addition to real ARNs, # you can use keys in the `custom_policy_map` in `main.tf` to select policies defined in the component. # If you are using keys from the map, plans look better if you put them after the real role ARNs. role_policy_arns: [] role_description: "Template role, should not exist" # If `aws_saml_login_enabled: true` then the role will be available via SAML logins, # but only via the SAML IDPs configured for this account. # Otherwise, it will only be accessible via `assume role`. aws_saml_login_enabled: false ## The following attributes control access to this role via `assume role`. ## `trusted_*` grants access, `denied_*` denies access. ## If a role is both trusted and denied, it will not be able to access this role. # Permission sets specify users operating from the given AWS SSO permission set in this account. trusted_permission_sets: [] denied_permission_sets: [] # Primary roles specify the short role names of roles in the primary (identity) # account that are allowed to assume this role. # BE CAREFUL: This is setting the default access for other roles. trusted_teams: [] denied_teams: [] # Role ARNs specify Role ARNs in any account that are allowed to assume this role. # BE CAREFUL: there is nothing limiting these Role ARNs to roles within our organization. trusted_role_arns: [] denied_role_arns: [] ## ## admin and terraform are the core team roles ## admin: <<: *user-template enabled: true role_policy_arns: - "arn:aws:iam::aws:policy/AdministratorAccess" role_description: "Full administration of this account" trusted_teams: ["admin"] terraform: <<: *user-template enabled: true # We require Terraform to be allowed to create and modify IAM roles # and policies (e.g. for EKS service accounts), so there is no use trying to restrict it. # For better security, we could segregate components that needed # administrative permissions and use a more restrictive role # for Terraform, such as PowerUser (further restricted to deny AWS SSO changes). role_policy_arns: - "arn:aws:iam::aws:policy/AdministratorAccess" role_description: "Role for Terraform administration of this account" trusted_teams: ["admin", "spacelift"] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`roles` required
A map of roles to configure the accounts. **Type:** ```hcl map(object({ enabled = bool denied_teams = list(string) denied_permission_sets = list(string) denied_role_arns = list(string) max_session_duration = number # in seconds 3600 <= max <= 43200 (12 hours) role_description = string role_policy_arns = list(string) aws_saml_login_enabled = bool trusted_teams = list(string) trusted_permission_sets = list(string) trusted_role_arns = list(string) })) ```
### Optional Variables
`aws_saml_component_name` (`string`) optional
The name of the aws-saml component **Default value:** `"aws-saml"`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`trusted_github_repos` (`map(list(string))`) optional
Map where keys are role names (same keys as `roles`) and values are lists of GitHub repositories allowed to assume those roles. See `account-map/modules/github-assume-role-policy.mixin.tf` for specifics about repository designations. **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role_name_role_arn_map`
Map of role names to role ARNs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `local`, version: `>= 1.3` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `local`, version: `>= 1.3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `aws_saml` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.eks_viewer`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.kms_planner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.vpn_planner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`local_file.account_info`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.eks_view_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.eks_viewer_access_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.kms_planner_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.kms_planner_access_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.vpn_planner_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.vpn_planner_access_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## aws-teams This component is responsible for provisioning all primary user and system roles into the centralized identity account. This is expected to be used alongside [the `aws-team-roles` component](https://github.com/cloudposse-terraform-components/aws-teams/tree/main/aws-teams/../aws-team-roles) to provide fine-grained role delegation across the account hierarchy. ### Teams Function Like Groups and are Implemented as Roles The "teams" created in the `identity` account by this module can be thought of as access control "groups": a user who is allowed access one of these teams gets access to a set of roles (and corresponding permissions) across a set of accounts. Generally, there is nothing else provisioned in the `identity` account, so the teams have limited access to resources in the `identity` account by design. Teams are implemented as IAM Roles in each account. Access to the "teams" in the `identity` account is controlled by the `aws-saml` and `aws-sso` components. Access to the roles in all the other accounts is controlled by the "assume role" policies of those roles, which allow the "team" or AWS SSO Permission set to assume the role (or not). ### Privileges are Defined for Each Role in Each Account by `aws-team-roles` Every account besides the `identity` account has a set of IAM roles created by the `aws-team-roles` component. In that component, the account's roles are assigned privileges, and those privileges ultimately determine what a user can do in that account. Access to the roles can be granted in a number of ways. One way is by listing "teams" created by this component as "trusted" (`trusted_teams`), meaning that users who have access to the team role in the `identity` account are allowed (trusted) to assume the role configured in the target account. Another is by listing an AWS SSO Permission Set in the account (`trusted_permission_sets`). ### Role Access is Enabled by SAML and/or AWS SSO configuration Users can again access to a role in the `identity` account through either (or both) of 2 mechanisms: #### SAML Access - SAML access is globally configured via the `aws-saml` component, enabling an external SAML Identity Provider (IdP) to control access to roles in the `identity` account. (SAML access can be separately configured for other accounts, see the `aws-saml` and `aws-team-roles` components for more on that.) - Individual roles are enabled for SAML access by setting `aws_saml_login_enabled: true` in the role configuration. - Individual users are granted access to these roles by configuration in the SAML IdP. #### AWS SSO Access The `aws-sso` component can create AWS Permission Sets that allow users to assume specific roles in the `identity` account. See the `aws-sso` component for details. ## Known Problems ### Error: `assume role policy: LimitExceeded: Cannot exceed quota for ACLSizePerRole: 2048` The `aws-teams` architecture, when enabling access to a role via lots of AWS SSO Profiles, can create large "assume role" policies, large enough to exceed the default quota of 2048 characters. If you run into this limitation, you will get an error like this: ``` Error: error updating IAM Role (acme-gbl-root-tfstate-backend-analytics-ro) assume role policy: LimitExceeded: Cannot exceed quota for ACLSizePerRole: 2048 ``` This can happen in either/both the `identity` and `root` accounts (for Terraform state access). So far, we have always been able to resolve this by requesting a quota increase, which is automatically granted a few minutes after making the request. To request the quota increase: - Log in to the AWS Web console as admin in the affected account - Set your region to N. Virginia `us-east-1` - Navigate to the Service Quotas page via the account dropdown menu - Click on AWS Services in the left sidebar - Search for "IAM" and select "AWS Identity and Access Management (IAM)". (If you don't find that option, make sure you have selected the `us-east-1` region. - Find and select "Role trust policy length" - Request an increase to 4096 characters - Wait for the request to be approved, usually less than a few minutes ## Usage **Stack Level**: Global **Deployment**: Must be deployed by SuperAdmin using `atmos` CLI Here's an example snippet for how to use this component. The component should only be applied once, which is typically done via the identity stack (e.g. `gbl-identity.yaml`). ```yaml components: terraform: aws-teams: backend: s3: role_arn: null vars: teams_config: # Viewer has the same permissions as Observer but only in this account. It is not allowed access to other accounts. # Viewer also serves as the default configuration for all roles via the YAML anchor. viewer: &user-template # `max_session_duration` set the maximum session duration (in seconds) for the IAM roles. # This setting can have a value from 3600 (1 hour) to 43200 (12 hours). # For roles people log into via SAML, a long duration is convenient to prevent them # from having to frequently re-authenticate. # For roles assumed from some other role, the setting is practically irrelevant, because # the AssumeRole API limits the duration to 1 hour in any case. # References: # - https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use.html # - https://docs.aws.amazon.com/STS/latest/APIReference/API_AssumeRole.html max_session_duration: 43200 # 12 hours in seconds # role_policy_arns are the IAM Policy ARNs to attach to this policy. In addition to real ARNs, # you can use keys in the `custom_policy_map` in `main.tf` to select policies defined in the component. # If you are using keys from the map, plans look better if you put them after the real role ARNs. role_policy_arns: - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" role_description: "Team restricted to viewing resources in the identity account" # If `aws_saml_login_enabled: true` then the role will be available via SAML logins. # Otherwise, it will only be accessible via `assume role`. aws_saml_login_enabled: false # The following attributes control access to this role via `assume role`. # `trusted_*` grants access, `denied_*` denies access. # If a role is both trusted and denied, it will not be able to access this role. # Permission sets specify users operating from the given AWS SSO permission set in this account. trusted_permission_sets: [] denied_permission_sets: [] # Primary roles specify the short role names of roles in the primary (identity) # account that are allowed to assume this role. trusted_teams: [] denied_teams: ["viewer"] # Role ARNs specify Role ARNs in any account that are allowed to assume this role. # BE CAREFUL: there is nothing limiting these Role ARNs to roles within our organization. trusted_role_arns: [] denied_role_arns: [] admin: <<: *user-template role_description: "Team with PowerUserAccess permissions in `identity` and AdministratorAccess to all other accounts except `root`" # Limit `admin` to Power User to prevent accidentally destroying the admin role itself # Use SuperAdmin to administer IAM access role_policy_arns: ["arn:aws:iam::aws:policy/PowerUserAccess"] # TODO Create a "security" team with AdministratorAccess to audit and security, remove "admin" write access to those accounts aws_saml_login_enabled: true # list of roles in primary that can assume into this role in delegated accounts # primary admin can assume delegated admin trusted_teams: ["admin"] # GH runner should be moved to its own `ghrunner` role trusted_permission_sets: ["IdentityAdminTeamAccess"] spacelift: <<: *user-template role_description: Team for our privileged Spacelift server role_policy_arns: - team_role_access aws_saml_login_enabled: false trusted_teams: - admin trusted_role_arns: ["arn:aws:iam::123456789012:role/eg-ue2-auto-spacelift-worker-pool-admin"] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`teams_config` required
A roles map to configure the accounts. **Type:** ```hcl map(object({ denied_teams = list(string) denied_permission_sets = list(string) denied_role_arns = list(string) max_session_duration = number # in seconds 3600 <= max <= 43200 (12 hours) role_description = string role_policy_arns = list(string) aws_saml_login_enabled = bool allowed_roles = optional(map(list(string)), {}) trusted_teams = list(string) trusted_permission_sets = list(string) trusted_role_arns = list(string) })) ```
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`aws_saml_component_name` (`string`) optional
The name of the aws-saml component **Default value:** `"aws-saml"`
`aws_saml_environment_name` (`string`) optional
The name of the environment where SSO is provisioned **Default value:** `"gbl"`
`aws_saml_stage_name` (`string`) optional
The name of the stage where SSO is provisioned **Default value:** `"identity"`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`trusted_github_repos` (`map(list(string))`) optional
Map where keys are role names (same keys as `teams_config`) and values are lists of GitHub repositories allowed to assume those roles. See `account-map/modules/github-assume-role-policy.mixin.tf` for specifics about repository designations. **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role_arns`
List of role ARNs
`team_name_role_arn_map`
Map of team names to role ARNs
`team_names`
List of team names
`teams_config`
Map of team config with name, target arn, and description
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `local`, version: `>= 1.3` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `local`, version: `>= 1.3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `aws_saml` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.team_role_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`local_file.account_info`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.team_role_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## Terraform Components(Aws) import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a library of reusable Terraform "root module" components. --- ## bastion This component provisions a generic Bastion host within an Auto Scaling Group (ASG) with parameterized `user_data` and supports AWS SSM Session Manager for remote access with IAM authentication. If a special `container.sh` script is desired to run, set `container_enabled` to `true`, and set the `image_repository` and `image_container` variables. By default, this component acts as an "SSM Bastion", which is deployed to a private subnet and has SSM enabled, allowing access via the AWS Console, AWS CLI, or SSM Session tools such as [aws-gate](https://github.com/xen0l/aws-gate). Alternatively, this component can be used as a regular SSH Bastion, deployed to a public subnet with Security Group rules allowing inbound traffic over port 22. ## Usage **Stack Level**: Regional By default, this component can be used as an "SSM Bastion" (deployed to a private subnet, accessed via SSM): ```yaml components: terraform: bastion: vars: enabled: true name: bastion-ssm # Your choice of availability zones. If not specified, all private subnets are used. availability_zones: ["us-east-1a", "us-east-1b", "us-east-1c"] instance_type: t3.micro image_container: infrastructure:latest image_repository: "111111111111.dkr.ecr.us-east-1.amazonaws.com/example/infrastructure" ``` The following is an example snippet for how to use this component as a traditional bastion: ```yaml components: terraform: bastion: vars: enabled: true name: bastion-traditional image_container: infrastructure:latest image_repository: "111111111111.dkr.ecr.us-east-1.amazonaws.com/example/infrastructure" associate_public_ip_address: true # deploy to public subnet and associate public IP with instance custom_bastion_hostname: bastion vanity_domain: example.com security_group_rules: - type: "ingress" from_port: 22 to_port: 22 protocol: tcp cidr_blocks: ["1.2.3.4/32"] - type: "egress" from_port: 0 to_port: 0 protocol: -1 cidr_blocks: ["0.0.0.0/0"] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`associate_public_ip_address` (`bool`) optional
Whether to associate public IP to the instance. **Default value:** `false`
`availability_zones` (`list(string)`) optional
AWS Availability Zones in which to deploy multi-AZ resources. If not provided, resources will be provisioned in every private subnet in the VPC. **Default value:** `[ ]`
`container_command` (`string`) optional
The container command passed in after `docker run --rm -it bash -c`. **Default value:** `"bash"`
`image_container` (`string`) optional
The image container to use in `container.sh`. **Default value:** `""`
`image_repository` (`string`) optional
The image repository to use in `container.sh`. **Default value:** `""`
`instance_type` (`string`) optional
Bastion instance type **Default value:** `"t2.micro"`
`kms_alias_name_ssm` (`string`) optional
KMS alias name for SSM **Default value:** `"alias/aws/ssm"`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully completed with `aws_security_group_rule` resource. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "from_port": 0, "protocol": -1, "to_port": 0, "type": "egress" }, { "cidr_blocks": [ "0.0.0.0/0" ], "from_port": 22, "protocol": "tcp", "to_port": 22, "type": "ingress" } ] ```
`vpc_component_name` (`string`) optional
Name of the VPC component to look up via remote state **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`autoscaling_group_id`
The AutoScaling Group ID
`iam_instance_profile`
Name of AWS IAM Instance Profile
`security_group_id`
ID on the AWS Security Group associated with the ASG
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `cloudinit`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `cloudinit`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `bastion_autoscale_group` | 0.42.0 | [`cloudposse/ec2-autoscale-group/aws`](https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.42.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `sg` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `ssm_tls_ssh_key_pair` | 0.10.2 | [`cloudposse/ssm-tls-ssh-key-pair/aws`](https://registry.terraform.io/modules/cloudposse/ssm-tls-ssh-key-pair/aws/0.10.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.bastion_image`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`cloudinit_config.config`](https://registry.terraform.io/providers/hashicorp/cloudinit/latest/docs/data-sources/config) (data source) --- ## cloudmap-namespace ## Usage ## Variables ### Required Variables
`description` (`string`) required
Description of the Cloud Map Namespace
`region` (`string`) required
AWS region
### Optional Variables
`type` (`string`) optional
Type of the Cloud Map Namespace **Default value:** `"http"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the namespace
`id`
The ID of the namespace
`name`
The name of the namespace
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_service_discovery_http_namespace.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_http_namespace) (resource) - [`aws_service_discovery_private_dns_namespace.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_private_dns_namespace) (resource) - [`aws_service_discovery_public_dns_namespace.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_public_dns_namespace) (resource) ## Data Sources The following data sources are used by this module: --- ## cloudtrail This component is responsible for provisioning CloudTrail auditing in an individual AWS account. It's expected to be used alongside [the `cloudtrail-bucket` component](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/cloudtrail-bucket) as it utilizes that bucket via remote state. This component can either be deployed selectively to various accounts with `is_organization_trail=false`, or alternatively created in all accounts if deployed to the management account with `is_organization_trail=true`. ## Usage **Stack Level**: Global The following is an example snippet for how to use this component: (`gbl-root.yaml`) ```yaml components: terraform: cloudtrail: vars: enabled: true cloudtrail_bucket_environment_name: "ue1" cloudtrail_bucket_stage_name: "audit" cloudwatch_logs_retention_in_days: 730 is_organization_trail: true ``` ## Variables ### Required Variables
`cloudtrail_bucket_environment_name` (`string`) required
The name of the environment where the CloudTrail bucket is provisioned
`cloudtrail_bucket_stage_name` (`string`) required
The stage name where the CloudTrail bucket is provisioned
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of a account-map component **Default value:** `"account-map"`
`audit_access_enabled` (`bool`) optional
If `true`, allows the Audit account access to read Cloudtrail logs directly from S3. This is a requirement for running Athena queries in the Audit account. **Default value:** `false`
`cloudtrail_bucket_component_name` (`string`) optional
The name of the CloudTrail bucket component **Default value:** `"cloudtrail-bucket"`
`cloudtrail_cloudwatch_logs_role_max_session_duration` (`number`) optional
The maximum session duration (in seconds) for the CloudTrail CloudWatch Logs role. Can have a value from 1 hour to 12 hours **Default value:** `43200`
`cloudwatch_log_group_class` (`string`) optional
Specifies the log class of the log group. Possible values are STANDARD or INFREQUENT_ACCESS. **Default value:** `"STANDARD"`
`cloudwatch_logs_retention_in_days` (`number`) optional
Number of days to retain logs for. CIS recommends 365 days. Possible values are: 0, 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. Set to 0 to keep logs indefinitely. **Default value:** `365`
`enable_log_file_validation` (`bool`) optional
Specifies whether log file integrity validation is enabled. Creates signed digest for validated contents of logs **Default value:** `true`
`enable_logging` (`bool`) optional
Enable logging for the trail **Default value:** `true`
`include_global_service_events` (`bool`) optional
Specifies whether the trail is publishing events from global services such as IAM to the log files **Default value:** `true`
`is_multi_region_trail` (`bool`) optional
Specifies whether the trail is created in the current region or in all regions **Default value:** `true`
`is_organization_trail` (`bool`) optional
Specifies whether the trail is created for all accounts in an organization in AWS Organizations, or only for the current AWS account. The default is false, and cannot be true unless the call is made on behalf of an AWS account that is the management account for an organization in AWS Organizations. **Default value:** `false`
`kms_abac_statements` optional
A list of ABAC statements which are placed in an IAM policy. Each statement must have the following attributes: - `sid` (optional): A unique identifier for the statement. - `effect`: The effect of the statement. Valid values are `Allow` and `Deny`. - `actions`: A list of actions to allow or deny. - `conditions`: A list of conditions to evaluate when the statement is applied. **Type:** ```hcl list(object({ sid = optional(string) effect = string actions = list(string) principals = map(list(string)) conditions = list(object({ test = string variable = string values = list(string) })) })) ``` **Default value:** `[ ]`
`kms_key_alias` (`string`) optional
The alias for the KMS key. If not set, the alias will be set to `alias/` **Default value:** `null`
`kms_key_enabled` (`bool`) optional
Toggle to enable/disable the encrypted log group feature that has not been extensively tested. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudtrail_arn`
CloudTrail ARN
`cloudtrail_home_region`
The region in which CloudTrail was created
`cloudtrail_id`
CloudTrail ID
`cloudtrail_logs_log_group_arn`
CloudTrail Logs log group ARN
`cloudtrail_logs_log_group_name`
CloudTrail Logs log group name
`cloudtrail_logs_role_arn`
CloudTrail Logs role ARN
`cloudtrail_logs_role_name`
CloudTrail Logs role name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.30.0, < 6.0.0` ### Providers - `aws`, version: `>= 5.30.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `cloudtrail` | 0.24.0 | [`cloudposse/cloudtrail/aws`](https://registry.terraform.io/modules/cloudposse/cloudtrail/aws/0.24.0) | n/a `cloudtrail_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_cloudtrail` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.cloudtrail_cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_iam_policy.cloudtrail_cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.cloudtrail_cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.cloudtrail_cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.cloudtrail_cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.cloudtrail_cloudwatch_logs_assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.kms_key_cloudtrail`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## cloudtrail-bucket This component is responsible for provisioning a bucket for storing CloudTrail logs for auditing purposes. It's expected to be used alongside [the `cloudtrail` component](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/cloudtrail). ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. It's suggested to apply this component to only the centralized `audit` account. ```yaml components: terraform: cloudtrail-bucket: vars: enabled: true name: "cloudtrail" noncurrent_version_expiration_days: 180 noncurrent_version_transition_days: 30 standard_transition_days: 60 glacier_transition_days: 180 expiration_days: 365 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_log_bucket_name` (`string`) optional
If var.create_access_log_bucket is false, this is the name of the S3 bucket where s3 access logs will be sent to. **Default value:** `""`
`acl` (`string`) optional
The canned ACL to apply. We recommend log-delivery-write for compatibility with AWS services. Valid values are private, public-read, public-read-write, aws-exec-read, authenticated-read, bucket-owner-read, bucket-owner-full-control, log-delivery-write. Due to https://docs.aws.amazon.com/AmazonS3/latest/userguide/create-bucket-faq.html, this will need to be set to 'private' during creation, but you can update normally after. **Default value:** `"log-delivery-write"`
`create_access_log_bucket` (`bool`) optional
Whether or not to create an access log bucket. **Default value:** `false`
`expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`force_destroy` (`bool`) optional
(Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`lifecycle_rule_enabled` (`bool`) optional
Enable lifecycle events on this bucket **Default value:** `true`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Specifies when noncurrent object versions transition to a different storage tier **Default value:** `30`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are AES256, aws:kms, or aws:kms:dsse **Default value:** `"AES256"`
`standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudtrail_bucket_arn`
CloudTrail S3 bucket ARN
`cloudtrail_bucket_domain_name`
CloudTrail S3 bucket domain name
`cloudtrail_bucket_id`
CloudTrail S3 bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudtrail_s3_bucket` | 0.28.0 | [`cloudposse/cloudtrail-s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/cloudtrail-s3-bucket/aws/0.28.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## cloudwatch-logs This component is responsible for creation of CloudWatch Log Streams and Log Groups. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: cloudwatch-logs: vars: enabled: true name: cloudwatch-logs retention_in_days: 15 stream_names: - app-1 - app-2 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`additional_permissions` (`list(string)`) optional
Additional permissions granted to assumed role **Default value:** ```hcl [ "logs:CreateLogStream", "logs:DeleteLogStream" ] ```
`principals` (`map(any)`) optional
Map of service name as key and a list of ARNs to allow assuming the role as value. (e.g. map(`AWS`, list(`arn:aws:iam:::role/admin`))) **Default value:** ```hcl { "Service": [ "ecs.amazonaws.com" ] } ```
`retention_in_days` (`string`) optional
Number of days you want to retain log events in the log group **Default value:** `"30"`
`stream_names` (`list(string)`) optional
Names of streams **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`log_group_arn`
ARN of the log group
`log_group_name`
Name of log group
`role_arn`
ARN of role to assume
`role_name`
Name of role to assume
`stream_arns`
ARN of the log stream
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0, < 6.0.0` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_logs` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `logs` | 0.6.9 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.9) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## cognito This component is responsible for provisioning and managing AWS Cognito resources. This component can provision the following resources: - [Cognito User Pools](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-identity-pools.html) - [Cognito User Pool Clients](https://docs.aws.amazon.com/cognito/latest/developerguide/user-pool-settings-client-apps.html) - [Cognito User Pool Domains](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-add-custom-domain.html) - [Cognito User Pool Identity Providers](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-identity-provider.html) - [Cognito User Pool Resource Servers](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-define-resource-servers.html) - [Cognito User Pool User Groups](https://docs.aws.amazon.com/cognito/latest/developerguide/cognito-user-pools-user-groups.html) ## Usage **Stack Level**: Global Here's an example snippet for how to use this component: ```yaml components: terraform: cognito: settings: spacelift: workspace_enabled: true vars: enabled: true # The full name of the User Pool will be: --- name: cognito schemas: - name: "email" attribute_data_type: "String" developer_only_attribute: false mutable: false required: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`admin_create_user_config` (`map(any)`) optional
The configuration for AdminCreateUser requests **Default value:** `{ }`
`admin_create_user_config_allow_admin_create_user_only` (`bool`) optional
Set to `true` if only the administrator is allowed to create user profiles. Set to `false` if users can sign themselves up via an app **Default value:** `true`
`admin_create_user_config_email_message` (`string`) optional
The message template for email messages. Must contain `{username}` and `{####}` placeholders, for username and temporary password, respectively **Default value:** `"{username}, your temporary password is `\{####\}`"`
`admin_create_user_config_email_subject` (`string`) optional
The subject line for email messages **Default value:** `"Your verification code"`
`admin_create_user_config_sms_message` (`string`) optional
The message template for SMS messages. Must contain `{username}` and `{####}` placeholders, for username and temporary password, respectively **Default value:** `"Your username is {username} and temporary password is `\{####\}`"`
`alias_attributes` (`list(string)`) optional
Attributes supported as an alias for this user pool. Possible values: phone_number, email, or preferred_username. Conflicts with `username_attributes` **Default value:** `null`
`auto_verified_attributes` (`list(string)`) optional
The attributes to be auto-verified. Possible values: email, phone_number **Default value:** `[ ]`
`client_access_token_validity` (`number`) optional
Time limit, between 5 minutes and 1 day, after which the access token is no longer valid and cannot be used. This value will be overridden if you have entered a value in `token_validity_units`. **Default value:** `60`
`client_allowed_oauth_flows` (`list(string)`) optional
List of allowed OAuth flows (code, implicit, client_credentials) **Default value:** `[ ]`
`client_allowed_oauth_flows_user_pool_client` (`bool`) optional
Whether the client is allowed to follow the OAuth protocol when interacting with Cognito user pools **Default value:** `true`
`client_allowed_oauth_scopes` (`list(string)`) optional
List of allowed OAuth scopes (phone, email, openid, profile, and aws.cognito.signin.user.admin) **Default value:** `[ ]`
`client_callback_urls` (`list(string)`) optional
List of allowed callback URLs for the identity providers **Default value:** `[ ]`
`client_default_redirect_uri` (`string`) optional
The default redirect URI. Must be in the list of callback URLs **Default value:** `""`
`client_explicit_auth_flows` (`list(string)`) optional
List of authentication flows (ADMIN_NO_SRP_AUTH, CUSTOM_AUTH_FLOW_ONLY, USER_PASSWORD_AUTH) **Default value:** `[ ]`
`client_generate_secret` (`bool`) optional
Should an application secret be generated **Default value:** `true`
`client_id_token_validity` (`number`) optional
Time limit, between 5 minutes and 1 day, after which the ID token is no longer valid and cannot be used. Must be between 5 minutes and 1 day. Cannot be greater than refresh token expiration. This value will be overridden if you have entered a value in `token_validity_units`. **Default value:** `60`
`client_logout_urls` (`list(string)`) optional
List of allowed logout URLs for the identity providers **Default value:** `[ ]`
`client_name` (`string`) optional
The name of the application client **Default value:** `null`
`client_prevent_user_existence_errors` (`string`) optional
Choose which errors and responses are returned by Cognito APIs during authentication, account confirmation, and password recovery when the user does not exist in the user pool. When set to ENABLED and the user does not exist, authentication returns an error indicating either the username or password was incorrect, and account confirmation and password recovery return a response indicating a code was sent to a simulated destination. When set to LEGACY, those APIs will return a UserNotFoundException exception if the user does not exist in the user pool. **Default value:** `null`
`client_read_attributes` (`list(string)`) optional
List of user pool attributes the application client can read from **Default value:** `[ ]`
`client_refresh_token_validity` (`number`) optional
The time limit in days refresh tokens are valid for. Must be between 60 minutes and 3650 days. This value will be overridden if you have entered a value in `token_validity_units` **Default value:** `30`
`client_supported_identity_providers` (`list(string)`) optional
List of provider names for the identity providers that are supported on this client **Default value:** `[ ]`
`client_token_validity_units` (`any`) optional
Configuration block for units in which the validity times are represented in. Valid values for the following arguments are: `seconds`, `minutes`, `hours` or `days`. **Default value:** ```hcl { "access_token": "minutes", "id_token": "minutes", "refresh_token": "days" } ```
`client_write_attributes` (`list(string)`) optional
List of user pool attributes the application client can write to **Default value:** `[ ]`
`clients` (`any`) optional
User Pool clients configuration **Default value:** `[ ]`
`deletion_protection` (`string`) optional
(Optional) When active, DeletionProtection prevents accidental deletion of your user pool. Before you can delete a user pool that you have protected against deletion, you must deactivate this feature. Valid values are ACTIVE and INACTIVE, Default value is INACTIVE. **Default value:** `"INACTIVE"`
`device_configuration` (`map(any)`) optional
The configuration for the user pool's device tracking **Default value:** `{ }`
`device_configuration_challenge_required_on_new_device` (`bool`) optional
Indicates whether a challenge is required on a new device. Only applicable to a new device **Default value:** `false`
`device_configuration_device_only_remembered_on_user_prompt` (`bool`) optional
If true, a device is only remembered on user prompt **Default value:** `false`
`domain` (`string`) optional
Cognito User Pool domain **Default value:** `null`
`domain_certificate_arn` (`string`) optional
The ARN of an ISSUED ACM certificate in `us-east-1` for a custom domain **Default value:** `null`
`email_configuration` (`map(any)`) optional
Email configuration **Default value:** `{ }`
`email_configuration_email_sending_account` (`string`) optional
Instruct Cognito to either use its built-in functionality or Amazon SES to send out emails. Allowed values: `COGNITO_DEFAULT` or `DEVELOPER` **Default value:** `"COGNITO_DEFAULT"`
`email_configuration_from_email_address` (`string`) optional
Sender’s email address or sender’s display name with their email address (e.g. `john@example.com`, `John Smith ` or `"John Smith Ph.D." )`. Escaped double quotes are required around display names that contain certain characters as specified in RFC 5322 **Default value:** `null`
`email_configuration_reply_to_email_address` (`string`) optional
The REPLY-TO email address **Default value:** `""`
`email_configuration_source_arn` (`string`) optional
The ARN of the email configuration source **Default value:** `""`
`email_verification_message` (`string`) optional
A string representing the email verification message **Default value:** `null`
`email_verification_subject` (`string`) optional
A string representing the email verification subject **Default value:** `null`
`identity_providers` (`list(any)`) optional
Cognito Identity Providers configuration **Default value:** `[ ]`
`lambda_config` (`any`) optional
Configuration for the AWS Lambda triggers associated with the User Pool **Default value:** `null`
`lambda_config_create_auth_challenge` (`string`) optional
The ARN of the lambda creating an authentication challenge **Default value:** `""`
`lambda_config_custom_email_sender` (`map(any)`) optional
A custom email sender AWS Lambda trigger **Default value:** `{ }`
`lambda_config_custom_message` (`string`) optional
AWS Lambda trigger custom message **Default value:** `""`
`lambda_config_custom_sms_sender` (`map(any)`) optional
A custom SMS sender AWS Lambda trigger **Default value:** `{ }`
`lambda_config_define_auth_challenge` (`string`) optional
Authentication challenge **Default value:** `""`
`lambda_config_kms_key_id` (`string`) optional
The Amazon Resource Name of Key Management Service Customer master keys. Amazon Cognito uses the key to encrypt codes and temporary passwords sent to CustomEmailSender and CustomSMSSender. **Default value:** `null`
`lambda_config_post_authentication` (`string`) optional
A post-authentication AWS Lambda trigger **Default value:** `""`
`lambda_config_post_confirmation` (`string`) optional
A post-confirmation AWS Lambda trigger **Default value:** `""`
`lambda_config_pre_authentication` (`string`) optional
A pre-authentication AWS Lambda trigger **Default value:** `""`
`lambda_config_pre_sign_up` (`string`) optional
A pre-registration AWS Lambda trigger **Default value:** `""`
`lambda_config_pre_token_generation` (`string`) optional
Allow to customize identity token claims before token generation **Default value:** `""`
`lambda_config_user_migration` (`string`) optional
The user migration Lambda config type **Default value:** `""`
`lambda_config_verify_auth_challenge_response` (`string`) optional
Verifies the authentication challenge response **Default value:** `""`
`mfa_configuration` (`string`) optional
Multi-factor authentication configuration. Must be one of the following values (ON, OFF, OPTIONAL) **Default value:** `"OFF"`
`number_schemas` (`list(any)`) optional
A container with the number schema attributes of a user pool. Maximum of 50 attributes **Default value:** `[ ]`
`password_policy` optional
User Pool password policy configuration **Type:** ```hcl object({ minimum_length = number, require_lowercase = bool, require_numbers = bool, require_symbols = bool, require_uppercase = bool, temporary_password_validity_days = number }) ``` **Default value:** `null`
`password_policy_minimum_length` (`number`) optional
The minimum password length **Default value:** `8`
`password_policy_require_lowercase` (`bool`) optional
Whether you have required users to use at least one lowercase letter in their password **Default value:** `true`
`password_policy_require_numbers` (`bool`) optional
Whether you have required users to use at least one number in their password **Default value:** `true`
`password_policy_require_symbols` (`bool`) optional
Whether you have required users to use at least one symbol in their password **Default value:** `true`
`password_policy_require_uppercase` (`bool`) optional
Whether you have required users to use at least one uppercase letter in their password **Default value:** `true`
`password_policy_temporary_password_validity_days` (`number`) optional
Password policy temporary password validity_days **Default value:** `7`
`recovery_mechanisms` (`list(any)`) optional
List of account recovery options **Default value:** `[ ]`
`resource_server_identifier` (`string`) optional
Resource server identifier **Default value:** `null`
`resource_server_name` (`string`) optional
Resource server name **Default value:** `null`
`resource_server_scope_description` (`string`) optional
Resource server scope description **Default value:** `null`
`resource_server_scope_name` (`string`) optional
Resource server scope name **Default value:** `null`
`resource_servers` (`list(any)`) optional
Resource servers configuration **Default value:** `[ ]`
`schemas` (`list(any)`) optional
A container with the schema attributes of a User Pool. Maximum of 50 attributes **Default value:** `[ ]`
`sms_authentication_message` (`string`) optional
A string representing the SMS authentication message **Default value:** `null`
`sms_configuration` (`map(any)`) optional
SMS configuration **Default value:** `{ }`
`sms_configuration_external_id` (`string`) optional
The external ID used in IAM role trust relationships **Default value:** `""`
`sms_configuration_sns_caller_arn` (`string`) optional
The ARN of the Amazon SNS caller. This is usually the IAM role that you've given Cognito permission to assume **Default value:** `""`
`sms_verification_message` (`string`) optional
A string representing the SMS verification message **Default value:** `null`
`software_token_mfa_configuration` (`map(any)`) optional
Configuration block for software token MFA. `mfa_configuration` must also be enabled for this to work **Default value:** `{ }`
`software_token_mfa_configuration_enabled` (`bool`) optional
If `true`, and if `mfa_configuration` is also enabled, multi-factor authentication by software TOTP generator will be enabled **Default value:** `false`
`string_schemas` (`list(any)`) optional
A container with the string schema attributes of a user pool. Maximum of 50 attributes **Default value:** `[ ]`
`user_group_description` (`string`) optional
The description of the user group **Default value:** `null`
`user_group_name` (`string`) optional
The name of the user group **Default value:** `null`
`user_group_precedence` (`number`) optional
The precedence of the user group **Default value:** `null`
`user_group_role_arn` (`string`) optional
The ARN of the IAM role to be associated with the user group **Default value:** `null`
`user_groups` (`list(any)`) optional
User groups configuration **Default value:** `[ ]`
`user_pool_add_ons` (`map(any)`) optional
Configuration block for user pool add-ons to enable user pool advanced security mode features **Default value:** `{ }`
`user_pool_add_ons_advanced_security_mode` (`string`) optional
The mode for advanced security, must be one of `OFF`, `AUDIT` or `ENFORCED` **Default value:** `null`
`user_pool_name` (`string`) optional
User pool name. If not provided, the name will be generated from the context **Default value:** `null`
`username_attributes` (`list(string)`) optional
Specifies whether email addresses or phone numbers can be specified as usernames when a user signs up. Conflicts with `alias_attributes` **Default value:** `null`
`username_configuration` (`map(any)`) optional
The Username Configuration. Setting `case_sensitive` specifies whether username case sensitivity will be applied for all users in the user pool through Cognito APIs **Default value:** `{ }`
`verification_message_template` (`map(any)`) optional
The verification message templates configuration **Default value:** `{ }`
`verification_message_template_default_email_option` (`string`) optional
The default email option. Must be either `CONFIRM_WITH_CODE` or `CONFIRM_WITH_LINK`. Defaults to `CONFIRM_WITH_CODE` **Default value:** `null`
`verification_message_template_email_message_by_link` (`string`) optional
The email message template for sending a confirmation link to the user, it must contain the `{##Click Here##}` placeholder **Default value:** `null`
`verification_message_template_email_subject_by_link` (`string`) optional
The subject line for the email message template for sending a confirmation link to the user **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the User Pool
`client_ids`
The ids of the User Pool clients
`client_ids_map`
The IDs map of the User Pool clients
`client_secrets`
The client secrets of the User Pool clients
`client_secrets_map`
The client secrets map of the User Pool clients
`creation_date`
The date the User Pool was created
`domain_app_version`
The app version for the domain
`domain_aws_account_id`
The AWS account ID for the User Pool domain
`domain_cloudfront_distribution_arn`
The URL of the CloudFront distribution
`domain_s3_bucket`
The S3 bucket where the static files for the domain are stored
`endpoint`
The endpoint name of the User Pool. Example format: cognito-idp.REGION.amazonaws.com/xxxx_yyyyy
`id`
The ID of the User Pool
`last_modified_date`
The date the User Pool was last modified
`resource_servers_scope_identifiers`
A list of all scopes configured in the format identifier/scope_name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.8.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.8.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cognito_identity_provider.identity_provider`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_identity_provider) (resource) - [`aws_cognito_resource_server.resource`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_resource_server) (resource) - [`aws_cognito_user_group.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_user_group) (resource) - [`aws_cognito_user_pool.pool`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_user_pool) (resource) - [`aws_cognito_user_pool_client.client`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_user_pool_client) (resource) - [`aws_cognito_user_pool_domain.domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cognito_user_pool_domain) (resource) ## Data Sources The following data sources are used by this module: --- ## config-bucket This module creates an S3 bucket suitable for storing `AWS Config` data. It implements a configurable log retention policy, which allows you to efficiently manage logs across different storage classes (_e.g._ `Glacier`) and ultimately expire the data altogether. It enables server-side encryption by default. It blocks public access to the bucket by default. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. It's suggested to apply this component to only the centralized `audit` account. ```yaml components: terraform: config-bucket: vars: enabled: true name: "config" noncurrent_version_expiration_days: 180 noncurrent_version_transition_days: 30 standard_transition_days: 60 glacier_transition_days: 180 expiration_days: 365 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`acl` (`string`) optional
The canned ACL to apply. We recommend log-delivery-write for compatibility with AWS services **Default value:** `"log-delivery-write"`
`enable_glacier_transition` (`bool`) optional
Enables the transition to AWS Glacier (note that this can incur unnecessary costs for huge amount of small files **Default value:** `true`
`expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`lifecycle_rule_enabled` (`bool`) optional
Enable lifecycle events on this bucket **Default value:** `true`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Specifies when noncurrent object versions transition to a different storage tier **Default value:** `30`
`standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`config_bucket_arn`
Config bucket ARN
`config_bucket_domain_name`
Config bucket FQDN
`config_bucket_id`
Config bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `config_bucket` | 1.0.2 | [`cloudposse/config-storage/aws`](https://registry.terraform.io/modules/cloudposse/config-storage/aws/1.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## datadog-child-organization Terraform component to provision a Datadog child organization using the Datadog provider. Datadog API/App keys and API URL are sourced via the `aws-datadog-credentials` component module; you only need to provide the child organization name and AWS region. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Regional or Global Example Atmos component configuration: ```yaml components: terraform: aws-datadog-child-organization: vars: enabled: true region: us-east-1 organization_name: your-child-organization-name ``` ## Variables ### Required Variables
`organization_name` (`string`) required
Datadog organization name
`region` (`string`) required
AWS Region
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`api_key`
Information about Datadog API key
`application_key`
Datadog application key with its associated metadata
`description`
Description of the organization
`id`
Organization ID
`public_id`
Public ID of the organization
`settings`
Organization settings
`user`
Information about organization users
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_child_organization` | 1.6.0 | [`cloudposse/platform/datadog//modules/child_organization`](https://registry.terraform.io/modules/cloudposse/platform/datadog/modules/child_organization/1.6.0) | n/a `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## datadog-credentials This component is responsible for provisioning SSM or ASM entries for Datadog API keys. It's required that the DataDog API and APP secret keys are available in the `var.datadog_secrets_source_store_account` account in AWS SSM Parameter Store at the `/datadog/%v/datadog_app_key` paths (where `%v` are the corresponding account names). This component copies keys from the source account (e.g. `auto`) to the destination account where this is being deployed. The purpose of using this formatted copying of keys handles a couple of problems. 1. The keys are needed in each account where datadog resources will be deployed. 1. The keys might need to be different per account or tenant, or any subset of accounts. 1. If the keys need to be rotated they can be rotated from a single management account. This module also has a submodule which allows other resources to quickly use it to create a datadog provider. See Datadog's [documentation about provisioning keys](https://docs.datadoghq.com/account_management/api-app-keys) for more information. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Global :::warning This is subject to change from a **Global** to a **Regional** stack level. This is because we need the keys in each region where we deploy datadog resources - so that we don't need to configure extra AWS Providers (which would need to be dynamic - which we cannot do). This is a limitation of Terraform. ::: This component should be deployed to every account where you want to provision datadog resources. This is usually every account except `root` and `identity` Here's an example snippet for how to use this component. It's suggested to apply this component to all accounts which you want to track AWS metrics with DataDog. In this example we use the key paths `/datadog/%v/datadog_api_key` and `/datadog/%v/datadog_app_key` where `%v` is `default`, this can be changed through `datadog_app_secret_key` & `datadog_api_secret_key` variables. The output Keys in the deployed account will be `/datadog/datadog_api_key` and `/datadog/datadog_app_key`. ```yaml components: terraform: datadog-configuration: settings: spacelift: workspace_enabled: true vars: enabled: true name: datadog-configuration datadog_secrets_store_type: SSM datadog_secrets_source_store_account_stage: auto datadog_secrets_source_store_account_region: "us-east-2" ``` Here is a snippet of using the `datadog_keys` submodule: ```terraform module "datadog_configuration" { source = "../datadog-configuration/modules/datadog_keys" enabled = true context = module.this.context } provider "datadog" { api_key = module.datadog_configuration.datadog_api_key app_key = module.datadog_configuration.datadog_app_key api_url = module.datadog_configuration.datadog_api_url validate = local.enabled } ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`datadog_api_secret_key` (`string`) optional
The name of the Datadog API secret **Default value:** `"default"`
`datadog_api_secret_key_source_pattern` (`string`) optional
The format string (%v will be replaced by the var.datadog_api_secret_key) for the key of the Datadog API secret in the source account **Default value:** `"/datadog/%v/datadog_api_key"`
`datadog_api_secret_key_target_pattern` (`string`) optional
The format string (%v will be replaced by the var.datadog_api_secret_key) for the key of the Datadog API secret in the target account **Default value:** `"/datadog/datadog_api_key"`
`datadog_app_secret_key` (`string`) optional
The name of the Datadog APP secret **Default value:** `"default"`
`datadog_app_secret_key_source_pattern` (`string`) optional
The format string (%v will be replaced by the var.datadog_app_secret_key) for the key of the Datadog APP secret in the source account **Default value:** `"/datadog/%v/datadog_app_key"`
`datadog_app_secret_key_target_pattern` (`string`) optional
The format string (%v will be replaced by the var.datadog_api_secret_key) for the key of the Datadog APP secret in the target account **Default value:** `"/datadog/datadog_app_key"`
`datadog_secrets_source_store_account_region` (`string`) optional
Region for holding Secret Store Datadog Keys, leave as null to use the same region as the stack **Default value:** `null`
`datadog_secrets_source_store_account_stage` (`string`) optional
Stage holding Secret Store for Datadog API and app keys. **Default value:** `"auto"`
`datadog_secrets_source_store_account_tenant` (`string`) optional
Tenant holding Secret Store for Datadog API and app keys. **Default value:** `"core"`
`datadog_secrets_store_type` (`string`) optional
Secret Store type for Datadog API and app keys. Valid values: `SSM`, `ASM` **Default value:** `"SSM"`
`datadog_site_url` (`string`) optional
The Datadog Site URL, https://docs.datadoghq.com/getting_started/site/ **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`datadog_api_key_location`
The Datadog API key in the secrets store
`datadog_api_url`
The URL of the Datadog API
`datadog_app_key_location`
The Datadog APP key location in the secrets store
`datadog_secrets_store_type`
The type of the secrets store to use for Datadog API and APP keys
`datadog_site`
The Datadog site to use
`region`
The region where the keys will be created
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `iam_roles_datadog_secrets` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_secretsmanager_secret.datadog_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret) (data source) - [`aws_secretsmanager_secret.datadog_app_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret) (data source) - [`aws_secretsmanager_secret_version.datadog_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret_version) (data source) - [`aws_secretsmanager_secret_version.datadog_app_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret_version) (data source) - [`aws_ssm_parameter.datadog_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.datadog_app_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## datadog_keys Useful submodule for other modules to quickly configure the datadog provider ## Usage ```hcl module "datadog_configuration" { source = "../datadog-configuration/modules/datadog_keys" enabled = true context = module.this.context } provider "datadog" { api_key = module.datadog_configuration.datadog_api_key app_key = module.datadog_configuration.datadog_app_key api_url = module.datadog_configuration.datadog_api_url validate = local.enabled } ``` ## Variables ### Required Variables
### Optional Variables
`global_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`api_key_ssm_arn`
Datadog API Key SSM ARN
`datadog_api_key`
Datadog API Key
`datadog_api_key_location`
The Datadog API key in the secrets store
`datadog_api_url`
Datadog API URL
`datadog_app_key`
Datadog APP Key
`datadog_app_key_location`
The Datadog APP key location in the secrets store
`datadog_secrets_store_type`
The type of the secrets store to use for Datadog API and APP keys
`datadog_site`
Datadog Site
`datadog_tags`
The Context Tags in datadog tag format (list of strings formatted as 'key:value')
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `always` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `datadog_configuration` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils_example_complete` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.datadog_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.datadog_app_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) None --- ## datadog-integration This component is responsible for provisioning Datadog AWS integrations. It depends on the `datadog-configuration` component to get the Datadog API keys. See Datadog's [documentation about provisioning keys](https://docs.datadoghq.com/account_management/api-app-keys) for more information. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. It's suggested to apply this component to all accounts which you want to track AWS metrics with DataDog. ```yaml components: terraform: datadog-integration: settings: spacelift: workspace_enabled: true vars: enabled: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_specific_namespace_rules` (`map(string)`) optional
An object, (in the form \{"namespace1":true/false, "namespace2":true/false\} ), that enables or disables metric collection for specific AWS namespaces for this AWS account only **Default value:** `{ }`
`context_host_and_filter_tags` (`list(string)`) optional
Automatically add host and filter tags for these context keys **Default value:** ```hcl [ "namespace", "tenant", "stage" ] ```
`cspm_resource_collection_enabled` (`bool`) optional
Enable Datadog Cloud Security Posture Management scanning of your AWS account. See [announcement](https://www.datadoghq.com/product/cloud-security-management/cloud-security-posture-management/) for details. **Default value:** `null`
`datadog_aws_account_id` (`string`) optional
The AWS account ID Datadog's integration servers use for all integrations **Default value:** `"464622532012"`
`excluded_regions` (`list(string)`) optional
An array of AWS regions to exclude from metrics collection **Default value:** `[ ]`
`filter_tags` (`list(string)`) optional
An array of EC2 tags (in the form `key:value`) that defines a filter that Datadog use when collecting metrics from EC2. Wildcards, such as ? (for single characters) and * (for multiple characters) can also be used **Default value:** `[ ]`
`host_tags` (`list(string)`) optional
An array of tags (in the form `key:value`) to add to all hosts and metrics reporting through this integration **Default value:** `[ ]`
`included_regions` (`list(string)`) optional
An array of AWS regions to include in metrics collection **Default value:** `[ ]`
`integrations` (`list(string)`) optional
List of AWS permission names to apply for different integrations (e.g. 'all', 'core') **Default value:** ```hcl [ "all" ] ```
`metrics_collection_enabled` (`bool`) optional
When enabled, a metric-by-metric crawl of the CloudWatch API pulls data and sends it to Datadog. New metrics are pulled every ten minutes, on average. **Default value:** `null`
`resource_collection_enabled` (`bool`) optional
Some Datadog products leverage information about how your AWS resources (such as S3 Buckets, RDS snapshots, and CloudFront distributions) are configured. When `resource_collection_enabled` is `true`, Datadog collects this information by making read-only API calls into your AWS account. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_account_id`
AWS Account ID of the IAM Role for the Datadog integration
`aws_role_name`
Name of the AWS IAM Role for the Datadog integration
`datadog_external_id`
Datadog integration external ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `datadog_integration` | 2.1.1 | [`cloudposse/datadog-integration/aws`](https://registry.terraform.io/modules/cloudposse/datadog-integration/aws/2.1.1) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_regions.all`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/regions) (data source) --- ## datadog-lambda-forwarder This component provisions all infrastructure required to deploy [Datadog Lambda forwarders](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring). It depends on the `datadog-configuration` component to obtain the Datadog API keys. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component: ```yaml components: terraform: datadog-lambda-forwarder: settings: spacelift: workspace_enabled: true vars: enabled: true name: datadog-lambda-forwarder # Set `forwarder_rds_enabled` to `true` and configure `rds-enhanced-monitoring` Log Group when: # 1. The account has RDS instances provisioned # 2. RDS Enhanced Monitoring is enabled # 3. CloudWatch Log Group `RDSOSMetrics` exists (it will be created by AWS automatically when RDS Enhanced Monitoring is enabled) forwarder_rds_enabled: true forwarder_log_enabled: true forwarder_vpc_logs_enabled: true cloudwatch_forwarder_log_groups: rds-enhanced-monitoring: name: "RDSOSMetrics" filter_pattern: "" eks-cluster: # Use either `name` or `name_prefix` with `name_suffix` # If `name_prefix` with `name_suffix` are used, the final `name` will be constructed using `name_prefix` + context + `name_suffix`, # e.g. "/aws/eks/eg-ue2-prod-eks-cluster/cluster" name_prefix: "/aws/eks/" name_suffix: "eks-cluster/cluster" filter_pattern: "" transfer-sftp: name: "/aws/transfer/s-xxxxxxxxxxxx" filter_pattern: "" ``` Note for other regions: you need to deploy the `datadog-configuration` component in the respective region — the Datadog configuration is moving to a regional implementation. For example, if you usually deploy to `us-west-2` (and DD Configuration is `gbl`), deploy it to the new region and then deploy the lambda forwarder. ```yaml import: - orgs/acme/plat/dev/_defaults - mixins/region/us-east-1 - catalog/datadog/configuration - catalog/datadog/lambda-forwarder components: terraform: datadog-configuration: vars: datadog_secrets_store_type: SSM datadog_secrets_source_store_account_stage: auto datadog_secrets_source_store_account_region: "us-west-2" datadog-lambda-forwarder: vars: datadog_configuration_environment: "use1" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`cloudwatch_forwarder_event_patterns` optional
Map of title to CloudWatch Event patterns to forward to Datadog. Event structure from here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html#CloudWatchEventsPatterns Example: ```hcl cloudwatch_forwarder_event_rules = { "guardduty" = { source = ["aws.guardduty"] detail-type = ["GuardDuty Finding"] } "ec2-terminated" = { source = ["aws.ec2"] detail-type = ["EC2 Instance State-change Notification"] detail = { state = ["terminated"] } } } ``` **Type:** ```hcl map(object({ version = optional(list(string)) id = optional(list(string)) detail-type = optional(list(string)) source = optional(list(string)) account = optional(list(string)) time = optional(list(string)) region = optional(list(string)) resources = optional(list(string)) detail = optional(map(list(string))) })) ``` **Default value:** `{ }`
`cloudwatch_forwarder_log_groups` (`map(map(string))`) optional
Map of CloudWatch Log Groups with a filter pattern that the Lambda forwarder will send logs from. For example: \{ mysql1 = \{ name = "/aws/rds/maincluster", filter_pattern = "" \} **Default value:** `{ }`
`context_tags` (`set(string)`) optional
List of context tags to add to each monitor **Default value:** ```hcl [ "namespace", "tenant", "environment", "stage" ] ```
`context_tags_enabled` (`bool`) optional
Whether to add context tags to add to each monitor **Default value:** `true`
`datadog_configuration_environment` (`string`) optional
AWS region where the Datadog configuration is deployed, useful for multi region setups, null uses default (gbl) **Default value:** `null`
`datadog_forwarder_lambda_environment_variables` (`map(string)`) optional
Map of environment variables to pass to the Lambda Function **Default value:** `{ }`
`dd_api_key_kms_ciphertext_blob` (`string`) optional
CiphertextBlob stored in environment variable DD_KMS_API_KEY used by the lambda function, along with the KMS key, to decrypt Datadog API key **Default value:** `""`
`dd_artifact_filename` (`string`) optional
The Datadog artifact filename minus extension **Default value:** `"aws-dd-forwarder"`
`dd_forwarder_version` (`string`) optional
Version tag of Datadog lambdas to use. https://github.com/DataDog/datadog-serverless-functions/releases **Default value:** `"3.66.0"`
`dd_module_name` (`string`) optional
The Datadog GitHub repository name **Default value:** `"datadog-serverless-functions"`
`dd_tags_map` (`map(string)`) optional
A map of Datadog tags to apply to all logs forwarded to Datadog **Default value:** `{ }`
`forwarder_lambda_debug_enabled` (`bool`) optional
Whether to enable or disable debug for the Lambda forwarder **Default value:** `false`
`forwarder_log_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for Logs. It can be a local file, URL or git repo **Default value:** `null`
`forwarder_log_enabled` (`bool`) optional
Flag to enable or disable Datadog log forwarder **Default value:** `false`
`forwarder_log_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog log forwarder lambda function **Default value:** `[ ]`
`forwarder_log_retention_days` (`number`) optional
Number of days to retain Datadog forwarder lambda execution logs. One of [0 1 3 5 7 14 30 60 90 120 150 180 365 400 545 731 1827 3653] **Default value:** `14`
`forwarder_rds_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for RDS. It can be a local file, url or git repo **Default value:** `null`
`forwarder_rds_enabled` (`bool`) optional
Flag to enable or disable Datadog RDS enhanced monitoring forwarder **Default value:** `false`
`forwarder_rds_filter_pattern` (`string`) optional
Filter pattern for Lambda forwarder RDS **Default value:** `""`
`forwarder_rds_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog RDS enhanced monitoring lambda function **Default value:** `[ ]`
`forwarder_vpc_logs_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for VPC Logs. It can be a local file, url or git repo **Default value:** `null`
`forwarder_vpc_logs_enabled` (`bool`) optional
Flag to enable or disable Datadog VPC flow log forwarder **Default value:** `false`
`forwarder_vpc_logs_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog VPC flow log forwarder lambda function **Default value:** `[ ]`
`forwarder_vpclogs_filter_pattern` (`string`) optional
Filter pattern for Lambda forwarder VPC Logs **Default value:** `""`
`kms_key_id` (`string`) optional
Optional KMS key ID to encrypt Datadog Lambda function logs **Default value:** `null`
`lambda_arn_enabled` (`bool`) optional
Enable adding the Lambda Arn to this account integration **Default value:** `true`
`lambda_policy_source_json` (`string`) optional
Additional IAM policy document that can optionally be passed and merged with the created policy document **Default value:** `""`
`lambda_reserved_concurrent_executions` (`number`) optional
Amount of reserved concurrent executions for the lambda function. A value of 0 disables Lambda from being triggered and -1 removes any concurrency limitations. Defaults to Unreserved Concurrency Limits -1 **Default value:** `-1`
`lambda_runtime` (`string`) optional
Runtime environment for Datadog Lambda **Default value:** `"python3.8"`
`log_collection_services` (`list(string)`) optional
List of log collection services to enable **Default value:** ```hcl [ "apigw-access-logs", "apigw-execution-logs", "elbv2", "elb", "cloudfront", "lambda", "redshift", "s3" ] ```
`s3_bucket_kms_arns` (`list(string)`) optional
List of KMS key ARNs for s3 bucket encryption **Default value:** `[ ]`
`s3_buckets` (`list(string)`) optional
The names of S3 buckets to forward logs to Datadog **Default value:** `[ ]`
`s3_buckets_with_prefixes` (`map(object({ bucket_name : string, bucket_prefix : string }))`) optional
The names S3 buckets and prefix to forward logs to Datadog **Default value:** `{ }`
`security_group_ids` (`list(string)`) optional
List of security group IDs to use when the Lambda Function runs in a VPC **Default value:** `null`
`subnet_ids` (`list(string)`) optional
List of subnet IDs to use when deploying the Lambda Function in a VPC **Default value:** `null`
`tracing_config_mode` (`string`) optional
Can be either PassThrough or Active. If PassThrough, Lambda will only trace the request from an upstream service if it contains a tracing header with 'sampled=1'. If Active, Lambda will respect any tracing header it receives from an upstream service **Default value:** `"PassThrough"`
`vpclogs_cloudwatch_log_group` (`string`) optional
The name of the CloudWatch Log Group for VPC flow logs **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lambda_forwarder_log_function_arn`
Datadog Lambda forwarder CloudWatch/S3 function ARN
`lambda_forwarder_log_function_name`
Datadog Lambda forwarder CloudWatch/S3 function name
`lambda_forwarder_rds_enhanced_monitoring_function_name`
Datadog Lambda forwarder RDS Enhanced Monitoring function name
`lambda_forwarder_rds_function_arn`
Datadog Lambda forwarder RDS Enhanced Monitoring function ARN
`lambda_forwarder_vpc_log_function_arn`
Datadog Lambda forwarder VPC Flow Logs function ARN
`lambda_forwarder_vpc_log_function_name`
Datadog Lambda forwarder VPC Flow Logs function name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Providers - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog-integration` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `datadog_lambda_forwarder` | 1.10.0 | [`cloudposse/datadog-lambda-forwarder/aws`](https://registry.terraform.io/modules/cloudposse/datadog-lambda-forwarder/aws/1.10.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `log_group_prefix` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`datadog_integration_aws_lambda_arn.log_collector`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_aws_lambda_arn) (resource) - [`datadog_integration_aws_lambda_arn.rds_collector`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_aws_lambda_arn) (resource) - [`datadog_integration_aws_lambda_arn.vpc_logs_collector`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_aws_lambda_arn) (resource) - [`datadog_integration_aws_log_collection.main`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_aws_log_collection) (resource) ## Data Sources The following data sources are used by this module: --- ## datadog-logs-archive This component provisions Datadog Log Archives. It creates a single log archive pipeline for each AWS account. If the `catchall` flag is set, it creates a catchall archive within the same S3 bucket. Each log archive filters for the tag `env:$env` where `$env` is the environment/account name (e.g. `sbx`, `prd`, `tools`), as well as any tags identified in the `additional_query_tags` key. The `catchall` archive, as the name implies, filters for `*`. A second bucket is created for CloudTrail, and a CloudTrail is configured to monitor the log archive bucket and log activity to the CloudTrail bucket. To forward these CloudTrail logs to Datadog, the CloudTrail bucket's ID must be added to the `s3_buckets` key for our `datadog-lambda-forwarder` component. Both buckets support object lock, with overridable defaults of COMPLIANCE mode and a duration of 7 days. Prerequisites - Datadog integration set up in the target environment - Relies on the Datadog API and App keys added by our Datadog integration component Issues, Gotchas, Good-to-Knows - Destroy/reprovision process - Because of the protections for S3 buckets, destroying/replacing the bucket may require two passes or a manual bucket delete followed by Terraform cleanup. If the bucket has a full day or more of logs, deleting it manually first helps avoid Terraform timeouts. - Two-step process to destroy via Terraform: 1) Set `s3_force_destroy` to `true` and apply 2) Set `enabled` to `false` and apply, or run `terraform destroy` ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage Stack Level: Global It's suggested to apply this component to all accounts from which Datadog receives logs. Example Atmos snippet: ```yaml components: terraform: datadog-logs-archive: settings: spacelift: workspace_enabled: true vars: enabled: true # additional_query_tags: # - "forwardername:*-dev-datadog-lambda-forwarder-logs" # - "account:123456789012" ``` ## Variables ### Required Variables
`query_override` (`string`) required
Override query for datadog archive. If null would be used query 'env:\{stage\} OR account:\{aws account id\} OR \{additional_query_tags\}'
`region` (`string`) required
AWS Region
### Optional Variables
`additional_query_tags` (`list(any)`) optional
Additional tags to be used in the query for this archive **Default value:** `[ ]`
`archive_lifecycle_config` optional
Lifecycle configuration for the archive S3 bucket **Type:** ```hcl object({ abort_incomplete_multipart_upload_days = optional(number, null) enable_glacier_transition = optional(bool, true) glacier_transition_days = optional(number, 365) noncurrent_version_glacier_transition_days = optional(number, 30) enable_deeparchive_transition = optional(bool, false) deeparchive_transition_days = optional(number, 0) noncurrent_version_deeparchive_transition_days = optional(number, 0) enable_standard_ia_transition = optional(bool, false) standard_transition_days = optional(number, 0) expiration_days = optional(number, 0) noncurrent_version_expiration_days = optional(number, 0) }) ``` **Default value:** `{ }`
`catchall_enabled` (`bool`) optional
Set to true to enable a catchall for logs unmatched by any queries. This should only be used in one environment/account **Default value:** `false`
`cloudtrail_lifecycle_config` optional
Lifecycle configuration for the cloudtrail S3 bucket **Type:** ```hcl object({ abort_incomplete_multipart_upload_days = optional(number, null) enable_glacier_transition = optional(bool, true) glacier_transition_days = optional(number, 365) noncurrent_version_glacier_transition_days = optional(number, 365) enable_deeparchive_transition = optional(bool, false) deeparchive_transition_days = optional(number, 0) noncurrent_version_deeparchive_transition_days = optional(number, 0) enable_standard_ia_transition = optional(bool, false) standard_transition_days = optional(number, 0) expiration_days = optional(number, 0) noncurrent_version_expiration_days = optional(number, 0) }) ``` **Default value:** `{ }`
`lifecycle_rules_enabled` (`bool`) optional
Enable/disable lifecycle management rules for log archive s3 objects **Default value:** `true`
`object_lock_days_archive` (`number`) optional
Object lock duration for archive buckets in days **Default value:** `7`
`object_lock_days_cloudtrail` (`number`) optional
Object lock duration for cloudtrail buckets in days **Default value:** `7`
`object_lock_mode_archive` (`string`) optional
Object lock mode for archive bucket. Possible values are COMPLIANCE or GOVERNANCE **Default value:** `"COMPLIANCE"`
`object_lock_mode_cloudtrail` (`string`) optional
Object lock mode for cloudtrail bucket. Possible values are COMPLIANCE or GOVERNANCE **Default value:** `"COMPLIANCE"`
`s3_force_destroy` (`bool`) optional
Set to true to delete non-empty buckets when enabled is set to false **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`archive_id`
The ID of the environment-specific log archive
`bucket_arn`
The ARN of the bucket used for log archive storage
`bucket_domain_name`
The FQDN of the bucket used for log archive storage
`bucket_id`
The ID (name) of the bucket used for log archive storage
`bucket_region`
The region of the bucket used for log archive storage
`catchall_id`
The ID of the catchall log archive
`cloudtrail_bucket_arn`
The ARN of the bucket used for access logging via cloudtrail
`cloudtrail_bucket_domain_name`
The FQDN of the bucket used for access logging via cloudtrail
`cloudtrail_bucket_id`
The ID (name) of the bucket used for access logging via cloudtrail
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.19` - `http`, version: `>= 2.1.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.19` - `http`, version: `>= 2.1.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `archive_bucket` | 4.10.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.10.0) | n/a `bucket_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `cloudtrail` | 0.24.0 | [`cloudposse/cloudtrail/aws`](https://registry.terraform.io/modules/cloudposse/cloudtrail/aws/0.24.0) | n/a `cloudtrail_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `cloudtrail_s3_bucket` | 4.10.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.10.0) | n/a `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`datadog_logs_archive.catchall_archive`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/logs_archive) (resource) - [`datadog_logs_archive.logs_archive`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/logs_archive) (resource) - [`datadog_logs_archive_order.archive_order`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/logs_archive_order) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.datadog_aws_role_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`http_http.current_order`](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) (data source) --- ## datadog-monitor This component provisions Datadog monitors and assigns Datadog roles to those monitors. It depends on the `datadog-configuration` component to obtain Datadog API keys. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component: ```yaml components: terraform: datadog-monitor: settings: spacelift: workspace_enabled: true vars: enabled: true local_datadog_monitors_config_paths: - "catalog/monitors/dev/*.yaml" ``` ## Conventions - Treat Datadog like a separate cloud provider with integrations ([datadog-integration](https://docs.cloudposse.com/components/library/aws/datadog-integration)) into your accounts. - Use the `catalog` convention to define a set of alerts. You can use ours or define your own: https://github.com/cloudposse/terraform-datadog-platform/tree/master/catalog/monitors - The monitors catalog for this component supports Datadog monitor exports. You can use [the status page of a monitor to export it from 'settings'](https://docs.datadoghq.com/monitors/manage/status/#settings). You can add the export to existing files or make new ones. Because the export is JSON formatted, it's also YAML compatible. If you prefer, you can convert the export to YAML using your text editor or a CLI tool like `yq`. ## Adjust Thresholds per Stack Since there are many parameters that may be adjusted for a given monitor, we define all monitors through YAML. By convention, we define the default monitors that should apply to all environments, and then adjust the thresholds per environment. This is accomplished using the `datadog-monitor` component variable `local_datadog_monitors_config_paths`, which defines the path to the YAML configuration files. By passing a path for `dev` and `prod`, we can define configurations that are different per environment. For example, you might have the following settings defined for `prod` and `dev` stacks that override the defaults. For the `dev` stack: ``` components: terraform: datadog-monitor: vars: # Located in the components/terraform/datadog-monitor directory local_datadog_monitors_config_paths: - catalog/monitors/*.yaml - catalog/monitors/dev/*.yaml # note this line ``` For the `prod` stack: ``` components: terraform: datadog-monitor: vars: # Located in the components/terraform/datadog-monitor directory local_datadog_monitors_config_paths: - catalog/monitors/*.yaml - catalog/monitors/prod/*.yaml # note this line ``` Behind the scenes (with `atmos`) we fetch all files from these glob patterns, template them, and merge them by key. If we peek into the `*.yaml` and `dev/*.yaml` files above you could see an example like this: **components/terraform/datadog-monitor/catalog/monitors/elb.yaml** ``` elb-lb-httpcode-5xx-notify: name: "(ELB) {{ env }} HTTP 5XX client error detected" type: query alert query: | avg(last_15m):max:aws.elb.httpcode_elb_5xx{${context_dd_tags}} by {env,host} > 20 message: | [${ dd_env }] [ {{ env }} ] lb:[ {{host}} ] {{#is_warning}} Number of HTTP 5XX client error codes generated by the load balancer > {{warn_threshold}}% {{/is_warning}} {{#is_alert}} Number of HTTP 5XX client error codes generated by the load balancer > {{threshold}}% {{/is_alert}} Check LB escalation_message: "" tags: {} options: renotify_interval: 60 notify_audit: false require_full_window: true include_tags: true timeout_h: 0 evaluation_delay: 60 new_host_delay: 300 new_group_delay: 0 groupby_simple_monitor: false renotify_occurrences: 0 renotify_statuses: [] validate: true notify_no_data: false no_data_timeframe: 5 priority: 3 threshold_windows: {} thresholds: critical: 50 warning: 20 priority: 3 restricted_roles: null ``` **components/terraform/datadog-monitor/catalog/monitors/dev/elb.yaml** ``` elb-lb-httpcode-5xx-notify: query: | avg(last_15m):max:aws.elb.httpcode_elb_5xx{${context_dd_tags}} by {env,host} > 30 priority: 2 options: thresholds: critical: 30 warning: 10 ``` ## Key Notes ### Inheritance The default YAML is applied to every stage that it's deployed to. For `dev`, we override the thresholds and priority for this monitor. This merging is done by key of the monitor, in this case `elb-lb-httpcode-5xx-notify`. ### Templating The `${ dd_env }` syntax is Terraform templating. While double braces (`{{ env }}`) refer to Datadog templating, `${ dd_env }` is a template variable we pass into our monitors. In this example we use it to specify a grouping in the message. This value is passed in and can be overridden via stacks. We pass a value via: ``` components: terraform: datadog-monitor: vars: # Located in the components/terraform/datadog-monitor directory local_datadog_monitors_config_paths: - catalog/monitors/*.yaml - catalog/monitors/dev/*.yaml # templatefile() is used for all yaml config paths with these variables. datadog_monitors_config_parameters: dd_env: "dev" ``` This allows us to further use inheritance from stack configuration to keep our monitors DRY but configurable. Another available option is to use our catalog as base monitors and then override them with your specific fine tuning. ``` components: terraform: datadog-monitor: vars: local_datadog_monitors_config_paths: - https://raw.githubusercontent.com/cloudposse/terraform-datadog-platform/0.27.0/catalog/monitors/ec2.yaml - catalog/monitors/ec2.yaml ``` ## Other Gotchas Our integration action that checks for `'source_type_name' equals 'Monitor Alert'` will also be true for synthetics. Whereas if we check for `'event_type' equals 'query_alert_monitor'`, that's only true for monitors, because synthetics will only be picked up by an integration action when `event_type` is `synthetics_alert`. This is important if we need to distinguish between monitors and synthetics in OpsGenie, which is the case when we want to ensure clean messaging on OpsGenie incidents in Statuspage. ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alert_tags` (`list(string)`) optional
List of alert tags to add to all alert messages, e.g. `["@opsgenie"]` or `["@devops", "@opsgenie"]` **Default value:** `null`
`alert_tags_separator` (`string`) optional
Separator for the alert tags. All strings from the `alert_tags` variable will be joined into one string using the separator and then added to the alert message **Default value:** `"\n"`
`datadog_monitor_context_tags` (`set(string)`) optional
List of context tags to add to each monitor **Default value:** ```hcl [ "namespace", "tenant", "environment", "stage" ] ```
`datadog_monitor_context_tags_enabled` (`bool`) optional
Whether to add context tags to each monitor **Default value:** `true`
`datadog_monitor_globals` (`any`) optional
Global parameters to add to each monitor **Default value:** `{ }`
`datadog_monitors_config_parameters` (`map(any)`) optional
Map of parameters to Datadog monitor configurations **Default value:** `{ }`
`local_datadog_monitors_config_paths` (`list(string)`) optional
List of paths to local Datadog monitor configurations **Default value:** `[ ]`
`message_postfix` (`string`) optional
Additional information to put after each monitor message **Default value:** `""`
`message_prefix` (`string`) optional
Additional information to put before each monitor message **Default value:** `""`
`remote_datadog_monitors_base_path` (`string`) optional
Base path to remote Datadog monitor configurations **Default value:** `""`
`remote_datadog_monitors_config_paths` (`list(string)`) optional
List of paths to remote Datadog monitor configurations **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`datadog_monitor_names`
Names of the created Datadog monitors
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `datadog_monitors` | 1.6.0 | [`cloudposse/platform/datadog//modules/monitors`](https://registry.terraform.io/modules/cloudposse/platform/datadog/modules/monitors/1.6.0) | n/a `datadog_monitors_merge` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `local_datadog_monitors_yaml_config` | 1.0.2 | [`cloudposse/config/yaml`](https://registry.terraform.io/modules/cloudposse/config/yaml/1.0.2) | n/a `remote_datadog_monitors_yaml_config` | 1.0.2 | [`cloudposse/config/yaml`](https://registry.terraform.io/modules/cloudposse/config/yaml/1.0.2) | Convert all Datadog Monitors from YAML config to Terraform map with token replacement using `parameters` `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## datadog-private-location-ecs This component creates a Datadog Private Location and deploys it to ECS (EC2 or Fargate). ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Note** The app key required for this component requires admin level permissions if you are using the default roles. Admin's have permissions to Write to private locations, which is needed for this component. **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml # stacks/catalog/datadog/private-location.yaml components: terraform: datadog-private-location: metadata: component: datadog-private-location-ecs settings: spacelift: workspace_enabled: true vars: enabled: true name: datadog-private-location task: task_memory: 512 task_cpu: 256 launch_type: FARGATE # capacity_provider_strategies takes precedence over launch_type capacity_provider_strategies: - capacity_provider: FARGATE_SPOT weight: 100 base: null network_mode: awsvpc desired_count: 1 ignore_changes_desired_count: true ignore_changes_task_definition: false use_alb_security_group: false assign_public_ip: false propagate_tags: SERVICE wait_for_steady_state: true circuit_breaker_deployment_enabled: true circuit_breaker_rollback_enabled: true containers: datadog: name: datadog-private-location image: public.ecr.aws/datadog/synthetics-private-location-worker:latest compatibilities: - EC2 - FARGATE - FARGATE_SPOT log_configuration: logDriver: awslogs options: {} port_mappings: [] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alb_configuration` (`string`) optional
The configuration to use for the ALB, specifying which cluster alb configuration to use **Default value:** `"default"`
`containers` (`any`) optional
Feed inputs into container definition module **Default value:** `{ }`
`private_location_description` (`string`) optional
The description of the private location. **Default value:** `null`
`task` (`any`) optional
Feed inputs into ecs_alb_service_task module **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ecs_cluster_arn`
Selected ECS cluster ARN
`lb_arn`
Selected LB ARN
`lb_listener_https`
Selected LB HTTPS Listener
`lb_sg_id`
Selected LB SG ID
`subnet_ids`
Selected subnet IDs
`vpc_id`
Selected VPC ID
`vpc_sg_id`
Selected VPC SG ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Providers - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `container_definition` | 0.61.2 | [`cloudposse/ecs-container-definition/aws`](https://registry.terraform.io/modules/cloudposse/ecs-container-definition/aws/0.61.2) | n/a `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `ecs_alb_service_task` | 0.78.0 | [`cloudposse/ecs-alb-service-task/aws`](https://registry.terraform.io/modules/cloudposse/ecs-alb-service-task/aws/0.78.0) | n/a `ecs_cluster` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `roles_to_principals` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/roles-to-principals`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/roles-to-principals/v1.536.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`datadog_synthetics_private_location.private_location`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/synthetics_private_location) (resource) ## Data Sources The following data sources are used by this module: --- ## datadog-synthetics This component provides the ability to implement [Datadog synthetic tests](https://docs.datadoghq.com/synthetics/guide/). Synthetic tests allow you to observe how your systems and applications are performing using simulated requests and actions from the AWS managed locations around the globe, and to monitor internal endpoints from [Private Locations](https://docs.datadoghq.com/synthetics/private_locations). ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component: ### Stack Configuration ```yaml components: terraform: datadog-synthetics: metadata: component: "datadog-synthetics" settings: spacelift: workspace_enabled: true vars: enabled: true name: "datadog-synthetics" locations: - "all" # List of paths to Datadog synthetic test configurations synthetics_paths: - "catalog/synthetics/examples/*.yaml" synthetics_private_location_component_name: "datadog-synthetics-private-location" private_location_test_enabled: true ``` ### Synthetics Configuration Examples Below are examples of Datadog browser and API synthetic tests. The synthetic tests are defined in YAML using either the [Datadog Terraform provider](https://registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/synthetics_test) schema or the [Datadog Synthetics API](https://docs.datadoghq.com/api/latest/synthetics) schema. See the `terraform-datadog-platform` Terraform module [README](https://github.com/cloudposse/terraform-datadog-platform/blob/main/modules/synthetics/README.md) for more details. We recommend using the API schema so you can more create and edit tests using the Datadog web API and then import them into this module by downloading the test using the Datadog REST API. (See the Datadog API documentation for the appropriate `curl` commands to use.) ```yaml # API schema my-browser-test: name: My Browser Test status: live type: browser config: request: method: GET headers: {} url: https://example.com/login setCookie: |- DatadogTest=true message: "My Browser Test Failed" options: device_ids: - chrome.laptop_large - edge.tablet - firefox.mobile_small ignoreServerCertificateError: false disableCors: false disableCsp: false noScreenshot: false tick_every: 86400 min_failure_duration: 0 min_location_failed: 1 retry: count: 0 interval: 300 monitor_options: renotify_interval: 0 ci: executionRule: non_blocking rumSettings: isEnabled: false enableProfiling: false enableSecurityTesting: false locations: - aws:us-east-1 - aws:us-west-2 # Terraform schema my-api-test: name: "API Test" message: "API Test Failed" type: api subtype: http tags: - "managed-by:Terraform" status: "live" request_definition: url: "CHANGEME" method: GET request_headers: Accept-Charset: "utf-8, iso-8859-1;q=0.5" Accept: "text/json" options_list: tick_every: 1800 no_screenshot: false follow_redirects: true retry: count: 2 interval: 10 monitor_options: renotify_interval: 300 assertion: - type: statusCode operator: is target: "200" - type: body operator: validatesJSONPath targetjsonpath: operator: is targetvalue: true jsonpath: foo.bar ``` These configuration examples are defined in the YAML files in the [catalog/synthetics/examples](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/datadog-synthetics/catalog/synthetics/examples) folder. You can use different subfolders for your use-case. For example, you can have `dev` and `prod` subfolders to define different synthetic tests for the `dev` and `prod` environments. Then use the `synthetic_paths` variable to point the component to the synthetic test configuration files. The configuration files are processed and transformed in the following order: - The `datadog-synthetics` component loads the YAML configuration files from the filesystem paths specified by the `synthetics_paths` variable - Then, in the [synthetics](https://github.com/cloudposse/terraform-datadog-platform/blob/master/modules/synthetics/main.tf) module, the YAML configuration files are merged and transformed from YAML into the [Datadog Terraform provider](https://registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/synthetics_test) schema - And finally, the Datadog Terraform provider uses the [Datadog Synthetics API](https://docs.datadoghq.com/api/latest/synthetics) specifications to call the Datadog API and provision the synthetic tests ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`synthetics_paths` (`list(string)`) required
List of paths to Datadog synthetic test configurations
### Optional Variables
`alert_tags` (`list(string)`) optional
List of alert tags to add to all alert messages, e.g. `["@opsgenie"]` or `["@devops", "@opsgenie"]` **Default value:** `null`
`alert_tags_separator` (`string`) optional
Separator for the alert tags. All strings from the `alert_tags` variable will be joined into one string using the separator and then added to the alert message **Default value:** `"\n"`
`config_parameters` (`map(any)`) optional
Map of parameter values to interpolate into Datadog Synthetic configurations **Default value:** `{ }`
`context_tags` (`set(string)`) optional
List of context tags to add to each synthetic check **Default value:** ```hcl [ "namespace", "tenant", "environment", "stage" ] ```
`context_tags_enabled` (`bool`) optional
Whether to add context tags to add to each synthetic check **Default value:** `true`
`datadog_synthetics_globals` (`any`) optional
Map of keys to add to every monitor **Default value:** `{ }`
`locations` (`list(string)`) optional
Array of locations used to run synthetic tests **Default value:** `[ ]`
`private_location_test_enabled` (`bool`) optional
Use private locations or the public locations provided by datadog **Default value:** `false`
`synthetics_private_location_component_name` (`string`) optional
The name of the Datadog synthetics private location component **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`datadog_synthetics_test_ids`
IDs of the created Datadog synthetic tests
`datadog_synthetics_test_maps`
Map (name: id) of the created Datadog synthetic tests
`datadog_synthetics_test_monitor_ids`
IDs of the monitors associated with the Datadog synthetics tests
`datadog_synthetics_test_names`
Names of the created Datadog synthetic tests
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `datadog_synthetics` | 1.6.0 | [`cloudposse/platform/datadog//modules/synthetics`](https://registry.terraform.io/modules/cloudposse/platform/datadog/modules/synthetics/1.6.0) | n/a `datadog_synthetics_merge` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `datadog_synthetics_private_location` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `datadog_synthetics_yaml_config` | 1.0.2 | [`cloudposse/config/yaml`](https://registry.terraform.io/modules/cloudposse/config/yaml/1.0.2) | Convert all Datadog synthetics from YAML config to Terraform map `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## datadog-synthetics-private-location This component provisions a Datadog synthetics private location on Datadog and a private location agent on EKS cluster. Private locations allow you to monitor internal-facing applications or any private URLs that are not accessible from the public internet. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: datadog-synthetics-private-location: settings: spacelift: workspace_enabled: true vars: enabled: true name: "datadog-synthetics-private-location" description: "Datadog Synthetics Private Location Agent" kubernetes_namespace: "monitoring" create_namespace: true # https://github.com/DataDog/helm-charts/tree/main/charts/synthetics-private-location repository: "https://helm.datadoghq.com" chart: "synthetics-private-location" chart_version: "0.15.15" timeout: 180 wait: true atomic: true cleanup_on_fail: true ``` ## Synthetics Private Location Config ```shell docker run --rm datadog/synthetics-private-location-worker --help ``` ``` The Datadog Synthetics Private Location Worker runs tests on privately accessible websites and brings results to Datadog Access keys: --accessKey Access Key for Datadog API authentication [string] --secretAccessKey Secret Access Key for Datadog API authentication [string] --datadogApiKey Datadog API key to send browser tests artifacts (e.g. screenshots) [string] --privateKey Private Key used to decrypt test configurations [array] --publicKey Public Key used by Datadog to encrypt test results. Composed of --publicKey.pem and --publicKey.fingerprint Worker configuration: --site Datadog site (datadoghq.com, us3.datadoghq.com, datadoghq.eu or ddog-gov.com) [string] [required] [default: "datadoghq.com"] --concurrency Maximum number of tests executed in parallel [number] [default: 10] --maxNumberMessagesToFetch Maximum number of tests that can be fetched at the same time [number] [default: 10] --proxyDatadog Proxy URL used to send requests to Datadog [string] [default: none] --dumpConfig Display non-secret worker configuration parameters [boolean] --enableStatusProbes Enable the probes system for Kubernetes [boolean] [default: false] --statusProbesPort The port for the probes server to listen on [number] [default: 8080] --config Path to JSON config file [default: "/etc/datadog/synthetics-check-runner.json"] Tests configuration: --maxTimeout Maximum test execution duration, in milliseconds [number] [default: 60000] --proxyTestRequests Proxy URL used to send test requests [string] [default: none] --proxyIgnoreSSLErrors Discard SSL errors when using a proxy [boolean] [default: false] --dnsUseHost Use local DNS config for API tests and HTTP steps in browser tests (currently ["192.168.65.5"]) [boolean] [default: true] --dnsServer DNS server IPs used in given order for API tests and HTTP steps in browser tests (--dnsServer="1.0.0.1" --dnsServer="9.9.9.9") and after local DNS config, if --dnsUseHost is present [array] [default: ["8.8.8.8","1.1.1.1"]] Network filtering: --allowedIPRanges Grant access to IP ranges (has precedence over --blockedIPRanges) [default: none] --blockedIPRanges Deny access to IP ranges (e.g. --blockedIPRanges.4="127.0.0.0/8" --blockedIPRanges.6="::1/128") [default: none] --enableDefaultBlockedIpRanges Deny access to all reserved IP ranges, except for those explicitly set in --allowedIPRanges [boolean] [default: false] --allowedDomainNames Grant access to domain names for API tests (has precedence over --blockedDomainNames, e.g. --allowedDomainNames="*.example.com") [array] [default: none] --blockedDomainNames Deny access to domain names for API tests (e.g. --blockedDomainNames="example.org" --blockedDomainNames="*.com") [array] [default: none] Options: --enableIPv6 Use IPv6 to perform tests. (Warning: IPv6 in Docker is only supported with Linux host) [boolean] [default: false] --version Show version number [boolean] -f, --logFormat Format log output [choices: "pretty", "pretty-compact", "json"] [default: "pretty"] -h, --help Show help [boolean] Volumes: /etc/datadog/certs/ .pem certificates present in this directory will be imported and trusted as certificate authorities for API and browser tests Environment variables: Command options can also be set via environment variables (DATADOG_API_KEY="...", DATADOG_WORKER_CONCURRENCY="15", DATADOG_DNS_USE_HOST="true") For options that accept multiple arguments, JSON string array notation should be used (DATADOG_TESTS_DNS_SERVER='["8.8.8.8", "1.1.1.1"]') Supported environment variables: DATADOG_ACCESS_KEY, DATADOG_API_KEY, DATADOG_PRIVATE_KEY, DATADOG_PUBLIC_KEY_FINGERPRINT, DATADOG_PUBLIC_KEY_PEM, DATADOG_SECRET_ACCESS_KEY, DATADOG_SITE, DATADOG_WORKER_CONCURRENCY, DATADOG_WORKER_LOG_FORMAT, DATADOG_WORKER_MAX_NUMBER_MESSAGES_TO_FETCH, DATADOG_WORKER_PROXY, DATADOG_TESTS_DNS_SERVER, DATADOG_TESTS_DNS_USE_HOST, DATADOG_TESTS_PROXY, DATADOG_TESTS_PROXY_IGNORE_SSL_ERRORS, DATADOG_TESTS_TIMEOUT, DATADOG_ALLOWED_IP_RANGES_4, DATADOG_ALLOWED_IP_RANGES_6, DATADOG_BLOCKED_IP_RANGES_4, DATADOG_BLOCKED_IP_RANGES_6, DATADOG_ENABLE_DEFAULT_WINDOWS_FIREWALL_RULES, DATADOG_ALLOWED_DOMAIN_NAMES, DATADOG_BLOCKED_DOMAIN_NAMES, DATADOG_WORKER_ENABLE_STATUS_PROBES, DATADOG_WORKER_STATUS_PROBES_PORT ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended
`kubernetes_namespace` (`string`) required
Kubernetes namespace to install the release into
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used **Default value:** `true`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`description` (`string`) optional
Release description attribute (visible in the history) **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`private_location_tags` (`set(string)`) optional
List of static tags to associate with the synthetics private location **Default value:** `[ ]`
`repository` (`string`) optional
Repository URL where to locate the requested chart **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
`synthetics_private_location_id`
Synthetics private location ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` - `helm`, version: `>= 2.3.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` - `local`, version: `>= 1.3` - `template`, version: `>= 2.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `datadog_synthetics_private_location` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`datadog_synthetics_private_location.this`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/synthetics_private_location) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## delegated-administrator Description of this component 55 ## Usage **Stack Level**: Regional or Global Here's an example snippet for how to use this component. ```yaml components: terraform: foo: vars: enabled: true ``` ## Variables ### Required Variables
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`mock`
Mock output example for the Cloud Posse Terraform component template
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## endpoint This component provisions DMS endpoints. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: dms/endpoint/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true autodeploy: false vars: enabled: true dms-endpoint-source-example: metadata: component: dms/endpoint inherits: - dms/endpoint/defaults vars: name: source-example endpoint_type: source engine_name: aurora-postgresql server_name: "" database_name: "" port: 5432 extra_connection_attributes: "" secrets_manager_access_role_arn: "" secrets_manager_arn: "" ssl_mode: none attributes: - source dms-endpoint-target-example: metadata: component: dms/endpoint inherits: - dms/endpoint/defaults vars: name: target-example endpoint_type: target engine_name: s3 extra_connection_attributes: "" s3_settings: bucket_name: "" bucket_folder: null cdc_inserts_only: false csv_row_delimiter: " " csv_delimiter: "," data_format: parquet compression_type: GZIP date_partition_delimiter: NONE date_partition_enabled: true date_partition_sequence: YYYYMMDD include_op_for_full_load: true parquet_timestamp_in_millisecond: true timestamp_column_name: timestamp service_access_role_arn: "" attributes: - target ``` ## Variables ### Required Variables
`endpoint_type` (`string`) required
Type of endpoint. Valid values are `source`, `target`
`engine_name` (`string`) required
Type of engine for the endpoint. Valid values are `aurora`, `aurora-postgresql`, `azuredb`, `db2`, `docdb`, `dynamodb`, `elasticsearch`, `kafka`, `kinesis`, `mariadb`, `mongodb`, `mysql`, `opensearch`, `oracle`, `postgres`, `redshift`, `s3`, `sqlserver`, `sybase`
`region` (`string`) required
AWS Region
### Optional Variables
`certificate_arn` (`string`) optional
Certificate ARN **Default value:** `null`
`database_name` (`string`) optional
Name of the endpoint database **Default value:** `null`
`elasticsearch_settings` (`map(any)`) optional
Configuration block for OpenSearch settings **Default value:** `null`
`extra_connection_attributes` (`string`) optional
Additional attributes associated with the connection to the source database **Default value:** `""`
`kafka_settings` (`map(any)`) optional
Configuration block for Kafka settings **Default value:** `null`
`kinesis_settings` (`map(any)`) optional
Configuration block for Kinesis settings **Default value:** `null`
`kms_key_arn` (`string`) optional
(Required when engine_name is `mongodb`, optional otherwise). ARN for the KMS key that will be used to encrypt the connection parameters. If you do not specify a value for `kms_key_arn`, then AWS DMS will use your default encryption key **Default value:** `null`
`mongodb_settings` (`map(any)`) optional
Configuration block for MongoDB settings **Default value:** `null`
`password` (`string`) optional
Password to be used to login to the endpoint database **Default value:** `""`
`password_path` (`string`) optional
If set, the path in AWS SSM Parameter Store to fetch the password for the DMS admin user **Default value:** `""`
`port` (`number`) optional
Port used by the endpoint database **Default value:** `null`
`redshift_settings` (`map(any)`) optional
Configuration block for Redshift settings **Default value:** `null`
`s3_settings` (`map(any)`) optional
Configuration block for S3 settings **Default value:** `null`
`secrets_manager_access_role_arn` (`string`) optional
ARN of the IAM role that specifies AWS DMS as the trusted entity and has the required permissions to access the value in SecretsManagerSecret **Default value:** `null`
`secrets_manager_arn` (`string`) optional
Full ARN, partial ARN, or friendly name of the SecretsManagerSecret that contains the endpoint connection details. Supported only for engine_name as aurora, aurora-postgresql, mariadb, mongodb, mysql, oracle, postgres, redshift or sqlserver **Default value:** `null`
`server_name` (`string`) optional
Host name of the database server **Default value:** `null`
`service_access_role` (`string`) optional
ARN used by the service access IAM role for DynamoDB endpoints **Default value:** `null`
`ssl_mode` (`string`) optional
The SSL mode to use for the connection. Can be one of `none`, `require`, `verify-ca`, `verify-full` **Default value:** `"none"`
`username` (`string`) optional
User name to be used to login to the endpoint database **Default value:** `""`
`username_path` (`string`) optional
If set, the path in AWS SSM Parameter Store to fetch the username for the DMS admin user **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dms_endpoint_arn`
DMS endpoint ARN
`dms_endpoint_id`
DMS endpoint ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.26.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.26.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dms_endpoint` | 1.3.1 | [`cloudposse/dms/aws//modules/dms-endpoint`](https://registry.terraform.io/modules/cloudposse/dms/aws/modules/dms-endpoint/1.3.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.username`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## iam This component provisions IAM roles required for DMS. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: dms/iam: metadata: component: dms/iam settings: spacelift: workspace_enabled: true autodeploy: false vars: enabled: true name: dms ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dms_cloudwatch_logs_role_arn`
DMS CloudWatch Logs role ARN
`dms_redshift_s3_role_arn`
DMS Redshift S3 role ARN
`dms_vpc_management_role_arn`
DMS VPC management role ARN
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.26.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dms_iam` | 1.3.1 | [`cloudposse/dms/aws//modules/dms-iam`](https://registry.terraform.io/modules/cloudposse/dms/aws/modules/dms-iam/1.3.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## replication-instance This component provisions DMS replication instances. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: dms/replication-instance/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true autodeploy: false vars: enabled: true allocated_storage: 50 apply_immediately: true auto_minor_version_upgrade: true allow_major_version_upgrade: false availability_zone: null engine_version: "3.4" multi_az: false preferred_maintenance_window: "sun:10:30-sun:14:30" publicly_accessible: false dms-replication-instance-t2-small: metadata: component: dms/replication-instance inherits: - dms/replication-instance/defaults vars: # Replication instance name must start with a letter, only contain alphanumeric characters and hyphens name: "t2-small" replication_instance_class: "dms.t2.small" allocated_storage: 50 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`allocated_storage` (`number`) optional
The amount of storage (in gigabytes) to be initially allocated for the replication instance. Default: 50, Min: 5, Max: 6144 **Default value:** `50`
`allow_major_version_upgrade` (`bool`) optional
Indicates that major version upgrades are allowed **Default value:** `false`
`apply_immediately` (`bool`) optional
Indicates whether the changes should be applied immediately or during the next maintenance window. Only used when updating an existing resource **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Indicates that major version upgrades are allowed **Default value:** `true`
`availability_zone` (`any`) optional
The EC2 Availability Zone that the replication instance will be created in **Default value:** `null`
`engine_version` (`string`) optional
The engine version number of the replication instance **Default value:** `"3.5.4"`
`multi_az` (`bool`) optional
Specifies if the replication instance is a multi-az deployment. You cannot set the `availability_zone` parameter if the `multi_az` parameter is set to true **Default value:** `false`
`preferred_maintenance_window` (`string`) optional
The weekly time range during which system maintenance can occur, in Universal Coordinated Time (UTC) **Default value:** `"sun:10:30-sun:14:30"`
`publicly_accessible` (`bool`) optional
Specifies the accessibility options for the replication instance. A value of true represents an instance with a public IP address. A value of false represents an instance with a private IP address **Default value:** `false`
`replication_instance_class` (`string`) optional
The compute and memory capacity of the replication instance as specified by the replication instance class **Default value:** `"dms.t3.small"`
`security_group_allow_all_egress` (`bool`) optional
A convenience that adds to the rules a rule that allows all egress. If this is false and no egress rules are specified via `rules` or `rule-matrix`, then no egress will be allowed. **Default value:** `true`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_ingress_cidr_blocks` (`list(string)`) optional
A list of CIDR blocks for the the cluster Security Group to allow ingress to the cluster security group. **Default value:** `[ ]`
`security_group_ingress_from_port` (`number`) optional
Start port on which the Glue connection accepts incoming connections. **Default value:** `0`
`security_group_ingress_to_port` (`number`) optional
End port on which the Glue connection accepts incoming connections. **Default value:** `65535`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dms_replication_instance_arn`
DMS replication instance ARN
`dms_replication_instance_id`
DMS replication instance ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `aws`, version: `>= 4.26.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dms_replication_instance` | 1.3.1 | [`cloudposse/dms/aws//modules/dms-replication-instance`](https://registry.terraform.io/modules/cloudposse/dms/aws/modules/dms-replication-instance/1.3.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## replication-task This component provisions DMS replication tasks. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: dms/replication-task/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true autodeploy: false vars: enabled: true start_replication_task: true migration_type: full-load-and-cdc dms-replication-task-example: metadata: component: dms/replication-task inherits: - dms/replication-task/defaults vars: name: example replication_instance_component_name: dms-replication-instance-t2-small source_endpoint_component_name: dms-endpoint-source-example target_endpoint_component_name: dms-endpoint-target-example replication_task_settings_file: "config/replication-task-settings-example.json" table_mappings_file: "config/replication-task-table-mappings-example.json" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`replication_instance_component_name` (`string`) required
DMS replication instance component name (used to get the ARN of the DMS replication instance)
`source_endpoint_component_name` (`string`) required
DMS source endpoint component name (used to get the ARN of the DMS source endpoint)
`table_mappings_file` (`string`) required
Path to the JSON file that contains the table mappings. See https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.html for more details
`target_endpoint_component_name` (`string`) required
DMS target endpoint component name (used to get the ARN of the DMS target endpoint)
### Optional Variables
`cdc_start_position` (`string`) optional
Indicates when you want a change data capture (CDC) operation to start. The value can be in date, checkpoint, or LSN/SCN format depending on the source engine, Conflicts with `cdc_start_time` **Default value:** `null`
`cdc_start_time` (`string`) optional
The Unix timestamp integer for the start of the Change Data Capture (CDC) operation. Conflicts with `cdc_start_position` **Default value:** `null`
`migration_type` (`string`) optional
The migration type. Can be one of `full-load`, `cdc`, `full-load-and-cdc` **Default value:** `"full-load-and-cdc"`
`replication_task_settings_file` (`string`) optional
Path to the JSON file that contains the task settings. See https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.html for more details **Default value:** `null`
`start_replication_task` (`bool`) optional
If set to `true`, the created replication tasks will be started automatically **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dms_replication_task_arn`
DMS replication task ARN
`dms_replication_task_id`
DMS replication task ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `aws`, version: `>= 4.26.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dms_endpoint_source` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dms_endpoint_target` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dms_replication_instance` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dms_replication_task` | 0.2.0 | [`cloudposse/dms/aws//modules/dms-replication-task`](https://registry.terraform.io/modules/cloudposse/dms/aws/modules/dms-replication-task/0.2.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## dns-delegated This component provisions a delegated DNS zone for managing subdomains delegated from a primary DNS account. The primary DNS zone must already exist via the dns-primary component. Use dns-delegated when you need a subdomain (e.g. prod.example.com) managed in a different account from the primary root zone (e.g. example.com). If you are deploying a root zone (e.g. example.com) and only a single account needs to manage or update the zone, use the dns-primary component instead. See “Why not use dns-delegated for all vanity domains?” for details. This component also provisions a wildcard ACM certificate for the delegated subdomain. Deploy it globally (once per account) rather than regionally; see “Why should the dns-delegated component be deployed globally rather than regionally?” for rationale. Note: After delegating a subdomain (e.g. prod.example.com) to an account, that account can create deeper subdomains (e.g. api.use1.prod.example.com) without additional delegation, but additional TLS certificates may be required because a wildcard certificate only matches a single level. Use the acm component for additional certs. Limitations Switching a hosted zone from public to private can cause issues because the provider will try to perform an update instead of a ForceNew. It is not possible to toggle between public and private. If changing from public to private and downtime is acceptable, delete records and the hosted zone, destroy the Terraform component, and re-deploy with new settings. If downtime is acceptable (workaround): 1. Delete anything using ACMs connected to previous hosted zones 2. Delete ACMs 3. Delete entries in the public hosted zone 4. Delete the hosted zone 5. Use atmos to destroy dns-delegated to remove the public hosted zone 6. Use atmos to deploy dns-delegated for the private hosted zone 7. Re-deploy dependent components (aurora-postgres, msk, external-dns, echo-server, etc.) to the new hosted zone If downtime is not acceptable (workaround): 1. Create a new virtual component of dns-delegated with the correct private inputs 2. Deploy the new dns-delegated-private component 3. Re-deploy dependent components to the new hosted zone Caveats - Do not create an NS delegation for a subdomain within a zone that is not authoritative for that subdomain (e.g. if a parent subdomain is already delegated). Route 53 Public DNS allows conflicting delegations, which can cause inconsistent resolution depending on the resolver’s strategy (see RFC7816 “QName Minimization”). Verify proper resolution with multiple resolvers (e.g. 8.8.8.8 and 1.1.1.1). ## Usage Stack Level: Global Use this component in global stacks for any accounts where you host services that need DNS records on a delegated subdomain of the root domain. Public hosted zone example: devplatform.example.net is created and the example.net zone in the primary DNS account contains a record delegating DNS to the new hosted zone. This also creates an ACM record. ```yaml components: terraform: dns-delegated: vars: zone_config: - subdomain: devplatform zone_name: example.net request_acm_certificate: true dns_private_zone_enabled: false # dns_soa_config configures the SOA record for the zone:: # - awsdns-hostmaster.amazon.com. ; AWS default value for administrator email address # - 1 ; serial number, not used by AWS # - 7200 ; refresh time in seconds for secondary DNS servers to refresh SOA record # - 900 ; retry time in seconds for secondary DNS servers to retry failed SOA record update # - 1209600 ; expire time in seconds (1209600 is 2 weeks) for secondary DNS servers to remove SOA record if they cannot refresh it # - 60 ; nxdomain TTL, or time in seconds for secondary DNS servers to cache negative responses # See SOA Record Documentation for more information. dns_soa_config: "awsdns-hostmaster.amazon.com. 1 7200 900 1209600 60" ``` Private hosted zone example: devplatform.example.net is created and the example.net zone in the primary DNS account contains a record delegating DNS to the new hosted zone. This creates an ACM record using a Private CA. ```yaml components: terraform: dns-delegated: vars: zone_config: - subdomain: devplatform zone_name: example.net request_acm_certificate: true dns_private_zone_enabled: true vpc_region_abbreviation_type: short vpc_primary_environment_name: use2 certificate_authority_component_name: private-ca-subordinate certificate_authority_stage_name: pca certificate_authority_environment_name: use2 certificate_authority_component_key: subordinate ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`zone_config` required
Zone config **Type:** ```hcl list(object({ subdomain = string zone_name = string })) ```
### Optional Variables
`aws_shield_protection_enabled` (`bool`) optional
Enable or disable AWS Shield Advanced protection for Route53 Zones. If set to 'true', a subscription to AWS Shield Advanced must exist in this account. **Default value:** `false`
`certificate_authority_component_key` (`string`) optional
Use this component key e.g. `root` or `mgmt` to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_component_name` (`string`) optional
Use this component name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_enabled` (`bool`) optional
Whether to use the certificate authority or not **Default value:** `false`
`certificate_authority_environment_name` (`string`) optional
Use this environment name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`certificate_authority_stage_name` (`string`) optional
Use this stage name to read from the remote state to get the certificate_authority_arn if using an authority type of SUBORDINATE **Default value:** `null`
`dns_private_zone_enabled` (`bool`) optional
Whether to set the zone to public or private **Default value:** `false`
`dns_soa_config` (`string`) optional
Root domain name DNS SOA record: - awsdns-hostmaster.amazon.com. ; AWS default value for administrator email address - 1 ; serial number, not used by AWS - 7200 ; refresh time in seconds for secondary DNS servers to refresh SOA record - 900 ; retry time in seconds for secondary DNS servers to retry failed SOA record update - 1209600 ; expire time in seconds (1209600 is 2 weeks) for secondary DNS servers to remove SOA record if they cannot refresh it - 60 ; nxdomain TTL, or time in seconds for secondary DNS servers to cache negative responses See [SOA Record Documentation](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/SOA-NSrecords.html) for more information. **Default value:** `"awsdns-hostmaster.amazon.com. 1 7200 900 1209600 60"`
`request_acm_certificate` (`bool`) optional
Whether or not to create an ACM certificate **Default value:** `true`
`vpc_component_name` (`string`) optional
The name of a VPC component **Default value:** `"vpc"`
`vpc_primary_environment_name` (`string`) optional
The name of the environment where primary VPC is deployed **Default value:** `null`
`vpc_region_abbreviation_type` (`string`) optional
Type of VPC abbreviation (either `fixed` or `short`) to use in names. See https://github.com/cloudposse/terraform-aws-utils for details. **Default value:** `"fixed"`
`vpc_secondary_environment_names` (`list(string)`) optional
The names of the environments where secondary VPCs are deployed **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`acm_ssm_parameter`
The SSM parameter for the ACM cert.
`certificate`
The ACM certificate information.
`default_dns_zone_id`
Default root DNS zone ID for the cluster
`default_domain_name`
Default root domain name (e.g. dev.example.net) for the cluster
`route53_hosted_zone_protections`
List of AWS Shield Advanced Protections for Route53 Hosted Zones.
`zones`
Subdomain and zone config
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 0.18.0 | [`cloudposse/acm-request-certificate/aws`](https://registry.terraform.io/modules/cloudposse/acm-request-certificate/aws/0.18.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `private_ca` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_record.root_ns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.soa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_zone) (resource) - [`aws_route53_zone.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_zone) (resource) - [`aws_route53_zone_association.secondary`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_zone_association) (resource) - [`aws_shield_protection.shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) - [`aws_ssm_parameter.acm_arn`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_route53_zone.root_zone`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## dns-primary This component is responsible for provisioning the primary DNS zones into an AWS account. By convention, we typically provision the primary DNS zones in the `dns` account. The primary account for branded zones (e.g. `example.com`), however, would be in the `prod` account, while staging zone (e.g. `example.qa`) might be in the `staging` account. The zones from the primary DNS zone are then expected to be delegated to other accounts via [the `dns-delegated` component](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/dns-delegated). Additionally, external records can be created on the primary DNS zones via the `record_config` variable. ## Architecture ### Summary The `dns` account gets a single `dns-primary` component deployed. Every other account that needs DNS entries gets a single `dns-delegated` component, chaining off the domains in the `dns` account. Optionally, accounts can have a single `dns-primary` component of their own, to have apex domains (which Cloud Posse calls "vanity domains"). Typically, these domains are configured with CNAME (or apex alias) records to point to service domain entries. ### Details The purpose of the `dns` account is to host root domains shared by several accounts (with each account being delegated its own subdomain) and to be the owner of domain registrations purchased from Amazon. The purpose of the `dns-primary` component is to provision AWS Route53 zones for the root domains. These zones, once provisioned, must be manually configured into the Domain Name Registrar's records as name servers. A single component can provision multiple domains and, optionally, associated ACM (SSL) certificates in a single account. Cloud Posse's architecture expects root domains shared by several accounts to be provisioned in the `dns` account with `dns-primary` and delegated to other accounts using the `dns-delegated` component, with each account getting its own subdomain corresponding to a Route 53 zone in the delegated account. Cloud Posse's architecture requires at least one such domain, called "the service domain", be provisioned. The service domain is not customer facing, and is provisioned to allow fully automated construction of host names without any concerns about how they look. Although they are not secret, the public will never see them. Root domains used by a single account are provisioned with the `dns-primary` component directly in that account. Cloud Posse calls these "vanity domains". These can be whatever the marketing or PR or other stakeholders want to be. After a domain is provisioned in the `dns` account, the `dns-delegated` component can provision one or more subdomains for each account, and, optionally, associated ACM certificates. For the service domain, Cloud Posse recommends using the account name as the delegated subdomain (either directly, e.g. "plat-dev", or as multiple subdomains, e.g. "dev.plat") because that allows `dns-delegated` to automatically provision any required host name in that zone. There is no automated support for `dns-primary` to provision root domains outside of the `dns` account that are to be shared by multiple accounts, and such usage is not recommended. If you must, `dns-primary` can provision a subdomain of a root domain that is provisioned in another account (not `dns`). In this case, the delegation of the subdomain must be done manually by entering the name servers into the parent domain's records (instead of in the Registrar's records). The architecture does not support other configurations, or non-standard component names. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. This component should only be applied once as the DNS zones it creates are global. This is typically done via the DNS stack (e.g. `gbl-dns.yaml`). ```yaml components: terraform: dns-primary: vars: domain_names: - example.net record_config: - root_zone: example.net name: "" type: A ttl: 60 records: - 53.229.170.215 # using a period at the end of a name - root_zone: example.net name: www. type: CNAME ttl: 60 records: - example.net # using numbers as name requires quotes - root_zone: example.net name: "123456." type: CNAME ttl: 60 records: - example.net # strings that are very long, this could be a DKIM key - root_zone: example.net name: service._domainkey. type: CNAME ttl: 60 records: - !!str |- YourVeryLongStringGoesHere ``` :::tip Use the [acm](https://docs.cloudposse.com/components/library/aws/acm) component for more advanced certificate requirements. ::: ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alias_record_config` optional
DNS Alias Record config **Type:** ```hcl list(object({ root_zone = string name = string type = string zone_id = string record = string evaluate_target_health = bool })) ``` **Default value:** `[ ]`
`dns_soa_config` (`string`) optional
Root domain name DNS SOA record: - awsdns-hostmaster.amazon.com. ; AWS default value for administrator email address - 1 ; serial number, not used by AWS - 7200 ; refresh time in seconds for secondary DNS servers to refresh SOA record - 900 ; retry time in seconds for secondary DNS servers to retry failed SOA record update - 1209600 ; expire time in seconds (1209600 is 2 weeks) for secondary DNS servers to remove SOA record if they cannot refresh it - 60 ; nxdomain TTL, or time in seconds for secondary DNS servers to cache negative responses See [SOA Record Documentation](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/SOA-NSrecords.html) for more information. **Default value:** `"awsdns-hostmaster.amazon.com. 1 7200 900 1209600 60"`
`domain_names` (`list(string)`) optional
Root domain name list, e.g. `["example.net"]` **Default value:** `null`
`record_config` optional
DNS Record config **Type:** ```hcl list(object({ root_zone = string name = string type = string ttl = string records = list(string) })) ``` **Default value:** `[ ]`
`request_acm_certificate` (`bool`) optional
Whether or not to request an ACM certificate for each domain **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`acms`
ACM certificates for domains
`zones`
DNS zones
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 0.18.0 | [`cloudposse/acm-request-certificate/aws`](https://registry.terraform.io/modules/cloudposse/acm-request-certificate/aws/0.18.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_record.aliasrec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.dnsrec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.soa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_zone.root`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_zone) (resource) ## Data Sources The following data sources are used by this module: --- ## documentdb This component is responsible for provisioning DocumentDB clusters. ## Usage **Stack Level**: Regional Here is an example snippet for how to use this component: ```yaml components: terraform: documentdb: backend: s3: workspace_key_prefix: documentdb vars: enabled: true cluster_size: 3 engine: docdb engine_version: 3.6.0 cluster_family: docdb3.6 retention_period: 35 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`apply_immediately` (`bool`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Specifies whether any minor engine upgrades will be applied automatically to the DB instance during the maintenance window or not **Default value:** `true`
`cluster_family` (`string`) optional
The family of the DocumentDB cluster parameter group. For more details, see https://docs.aws.amazon.com/documentdb/latest/developerguide/db-cluster-parameter-group-create.html **Default value:** `"docdb3.6"`
`cluster_parameters` optional
List of DB parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`cluster_size` (`number`) optional
Number of DB instances to create in the cluster **Default value:** `3`
`db_port` (`number`) optional
DocumentDB port **Default value:** `27017`
`deletion_protection_enabled` (`bool`) optional
A value that indicates whether the DB cluster has deletion protection enabled **Default value:** `false`
`dns_gbl_delegated_environment_name` (`string`) optional
The name of the environment where global `dns_delegated` is provisioned **Default value:** `"gbl"`
`eks_component_name` (`string`) optional
The name of the EKS component **Default value:** `"eks"`
`eks_security_group_ingress_enabled` (`bool`) optional
Whether to add the Security Group managed by the EKS cluster in the same regional stack to the ingress allowlist of the DocumentDB cluster. **Default value:** `true`
`enable_performance_insights` (`bool`) optional
Specifies whether to enable Performance Insights for the DB Instance. **Default value:** `false`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to export to cloudwatch. The following log types are supported: `audit`, `error`, `general`, `slowquery` **Default value:** `[ ]`
`encryption_enabled` (`bool`) optional
Specifies whether the DB cluster is encrypted **Default value:** `true`
`engine` (`string`) optional
The name of the database engine to be used for this DB cluster. Defaults to `docdb`. Valid values: `docdb` **Default value:** `"docdb"`
`engine_version` (`string`) optional
The version number of the database engine to use **Default value:** `"3.6.0"`
`instance_class` (`string`) optional
The instance class to use. For more details, see https://docs.aws.amazon.com/documentdb/latest/developerguide/db-instance-classes.html#db-instance-class-specs **Default value:** `"db.r4.large"`
`manage_master_user_password` (`bool`) optional
Whether to manage the master user password using AWS Secrets Manager. **Default value:** `null`
`master_password` (`string`) optional
(Required unless a snapshot_identifier is provided) Password for the master DB user. Note that this may show up in logs, and it will be stored in the state file. Please refer to the DocumentDB Naming Constraints **Default value:** `null`
`master_username` (`string`) optional
(Required unless a snapshot_identifier is provided) Username for the master DB user **Default value:** `"admin1"`
`preferred_backup_window` (`string`) optional
Daily time range during which the backups happen **Default value:** `"07:00-09:00"`
`preferred_maintenance_window` (`string`) optional
The window to perform maintenance in. Syntax: `ddd:hh24:mi-ddd:hh24:mi`. **Default value:** `"Mon:22:00-Mon:23:00"`
`retention_period` (`number`) optional
Number of days to retain backups for **Default value:** `5`
`skip_final_snapshot` (`bool`) optional
Determines whether a final DB snapshot is created before the DB cluster is deleted **Default value:** `true`
`snapshot_identifier` (`string`) optional
Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amazon Resource Name (ARN) of the cluster
`cluster_name`
Cluster Identifier
`endpoint`
Endpoint of the DocumentDB cluster
`master_host`
DB master hostname
`master_username`
Username for the master DB user
`reader_endpoint`
A read-only endpoint of the DocumentDB cluster, automatically load-balanced across replicas
`replicas_host`
DB replicas hostname
`security_group_arn`
ARN of the DocumentDB cluster Security Group
`security_group_id`
ID of the DocumentDB cluster Security Group
`security_group_name`
Name of the DocumentDB cluster Security Group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.29.0, < 6.0.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 5.29.0, < 6.0.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `documentdb_cluster` | 0.30.2 | [`cloudposse/documentdb-cluster/aws`](https://registry.terraform.io/modules/cloudposse/documentdb-cluster/aws/0.30.2) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.master_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.master_username`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.master_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: --- ## dynamodb This component is responsible for provisioning a DynamoDB table. ## Usage **Stack Level**: Regional Here is an example snippet for how to use this component: ```yaml components: terraform: dynamodb: backend: s3: workspace_key_prefix: dynamodb vars: enabled: true hash_key: HashKey range_key: RangeKey billing_mode: PAY_PER_REQUEST autoscaler_enabled: false encryption_enabled: true point_in_time_recovery_enabled: true streams_enabled: false ttl_enabled: false ``` ## Variables ### Required Variables
`hash_key` (`string`) required
DynamoDB table Hash Key
`region` (`string`) required
AWS Region.
### Optional Variables
`autoscale_max_read_capacity` (`number`) optional
DynamoDB autoscaling max read capacity **Default value:** `20`
`autoscale_max_write_capacity` (`number`) optional
DynamoDB autoscaling max write capacity **Default value:** `20`
`autoscale_min_read_capacity` (`number`) optional
DynamoDB autoscaling min read capacity **Default value:** `5`
`autoscale_min_write_capacity` (`number`) optional
DynamoDB autoscaling min write capacity **Default value:** `5`
`autoscale_read_target` (`number`) optional
The target value (in %) for DynamoDB read autoscaling **Default value:** `50`
`autoscale_write_target` (`number`) optional
The target value (in %) for DynamoDB write autoscaling **Default value:** `50`
`autoscaler_attributes` (`list(string)`) optional
Additional attributes for the autoscaler module **Default value:** `[ ]`
`autoscaler_enabled` (`bool`) optional
Flag to enable/disable DynamoDB autoscaling **Default value:** `false`
`autoscaler_tags` (`map(string)`) optional
Additional resource tags for the autoscaler module **Default value:** `{ }`
`billing_mode` (`string`) optional
DynamoDB Billing mode. Can be PROVISIONED or PAY_PER_REQUEST **Default value:** `"PROVISIONED"`
`deletion_protection_enabled` (`bool`) optional
Enable/disable DynamoDB table deletion protection **Default value:** `false`
`dynamodb_attributes` optional
Additional DynamoDB attributes in the form of a list of mapped values **Type:** ```hcl list(object({ name = string type = string })) ``` **Default value:** `[ ]`
`encryption_enabled` (`bool`) optional
Enable DynamoDB server-side encryption **Default value:** `true`
`global_secondary_index_map` optional
Additional global secondary indexes in the form of a list of mapped values **Type:** ```hcl list(object({ hash_key = string name = string non_key_attributes = list(string) projection_type = string range_key = string read_capacity = number write_capacity = number })) ``` **Default value:** `[ ]`
`hash_key_type` (`string`) optional
Hash Key type, which must be a scalar type: `S`, `N`, or `B` for String, Number or Binary data, respectively. **Default value:** `"S"`
`import_table` optional
Import Amazon S3 data into a new table. **Type:** ```hcl object({ # Valid values are GZIP, ZSTD and NONE input_compression_type = optional(string, null) # Valid values are CSV, DYNAMODB_JSON, and ION. input_format = string input_format_options = optional(object({ csv = object({ delimiter = string header_list = list(string) }) }), null) s3_bucket_source = object({ bucket = string bucket_owner = optional(string) key_prefix = optional(string) }) }) ``` **Default value:** `null`
`local_secondary_index_map` optional
Additional local secondary indexes in the form of a list of mapped values **Type:** ```hcl list(object({ name = string non_key_attributes = list(string) projection_type = string range_key = string })) ``` **Default value:** `[ ]`
`point_in_time_recovery_enabled` (`bool`) optional
Enable DynamoDB point in time recovery **Default value:** `true`
`range_key` (`string`) optional
DynamoDB table Range Key **Default value:** `""`
`range_key_type` (`string`) optional
Range Key type, which must be a scalar type: `S`, `N`, or `B` for String, Number or Binary data, respectively. **Default value:** `"S"`
`replicas` (`list(string)`) optional
List of regions to create a replica table in **Default value:** `[ ]`
`server_side_encryption_kms_key_arn` (`string`) optional
The ARN of the CMK that should be used for the AWS KMS encryption. This attribute should only be specified if the key is different from the default DynamoDB CMK, alias/aws/dynamodb. **Default value:** `null`
`stream_view_type` (`string`) optional
When an item in the table is modified, what information is written to the stream **Default value:** `""`
`streams_enabled` (`bool`) optional
Enable DynamoDB streams **Default value:** `false`
`table_name` (`string`) optional
Table name. If provided, the bucket will be created with this name instead of generating the name from the context **Default value:** `null`
`ttl_attribute` (`string`) optional
DynamoDB table TTL attribute **Default value:** `""`
`ttl_enabled` (`bool`) optional
Set to false to disable DynamoDB table TTL **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`global_secondary_index_names`
DynamoDB global secondary index names
`hash_key`
DynamoDB table hash key
`local_secondary_index_names`
DynamoDB local secondary index names
`range_key`
DynamoDB table range key
`table_arn`
DynamoDB table ARN
`table_id`
DynamoDB table ID
`table_name`
DynamoDB table name
`table_stream_arn`
DynamoDB table stream ARN
`table_stream_label`
DynamoDB table stream label
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dynamodb_table` | 0.37.0 | [`cloudposse/dynamodb/aws`](https://registry.terraform.io/modules/cloudposse/dynamodb/aws/0.37.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## ec2-client-vpn This component is responsible for provisioning VPN Client Endpoints. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. This component should only be applied once as the resources it creates are regional. This is typically done via the corp stack (e.g. `uw2-corp.yaml`). This is because a vpc endpoint requires a vpc and the network stack does not have a vpc. ```yaml components: terraform: ec2-client-vpn: settings: spacelift: workspace_enabled: true vars: enabled: true client_cidr: 10.100.0.0/10 logging_stream_name: client_vpn logging_enabled: true retention_in_days: 7 organization_name: acme split_tunnel: true availability_zones: - us-west-2a - us-west-2b - us-west-2c associated_security_group_ids: [] additional_routes: - destination_cidr_block: 0.0.0.0/0 description: Internet Route authorization_rules: - name: Internet Rule authorize_all_groups: true description: Allows routing to the internet" target_network_cidr: 0.0.0.0/0 ``` ### Deploying NOTE: This module uses the `aws_ec2_client_vpn_route` resource which throws an error if too many API calls come from a single host. Ignore this error and repeat the terraform command. It usually takes 3 deploys (or destroys) to complete. Error on create (See issue https://github.com/hashicorp/terraform-provider-aws/issues/19750) ``` ConcurrentMutationLimitExceeded: Cannot initiate another change for this endpoint at this time. Please try again later. ``` Error on destroy (See issue https://github.com/hashicorp/terraform-provider-aws/issues/16645) ``` timeout while waiting for resource to be gone (last state: 'deleting', timeout: 1m0s) ``` ### Testing NOTE: The `GoogleIDPMetadata-cloudposse.com.xml` in this repo is equivalent to the one in the `sso` component and is used for testing. This component can only specify a single SAML document. The customer SAML xml should be placed in this directory side-by-side the CloudPosse SAML xml. Prior to testing, the component needs to be deployed and the AWS client app needs to be setup by the IdP admin otherwise the following steps will result in an error similar to `app_not_configured_for_user`. 1. Deploy the component in a regional account with a VPC like `ue2-corp`. 1. Copy the contents of `client_configuration` into a file called `client_configuration.ovpn` 1. Download AWS client VPN `brew install --cask aws-vpn-client` 1. Launch the VPN 1. File > Manage Profiles to open the Manage Profiles window 1. Click Add Profile to open the Add Profile window 1. Set the display name e.g. `--` 1. Click the folder icon and find the file that was saved in a previous step 1. Click Add Profile to save the profile 1. Click Done to close to Manage Profiles window 1. Under "Ready to connect.", choose the profile, and click Connect A browser will launch and allow you to connect to the VPN. 1. Make a note of where this component is deployed 1. Ensure that the resource to connect to is in a VPC that is connected by the transit gateway 1. Ensure that the resource to connect to contains a security group with a rule that allows ingress from where the client vpn is deployed (e.g. `ue2-corp`) 1. Use `nmap` to test if the port is `open`. If the port is `filtered` then it's not open. ```console nmap -p ``` Successful tests have been seen with MSK and RDS. ## Variables ### Required Variables
`authorization_rules` required
List of objects describing the authorization rules for the Client VPN. Each Target Network CIDR range given will be used to create an additional route attached to the Client VPN endpoint with the same Description. **Type:** ```hcl list(object({ name = string access_group_id = string authorize_all_groups = bool description = string target_network_cidr = string })) ```
`client_cidr` (`string`) required
Network CIDR to use for clients
`logging_stream_name` (`string`) required
Names of stream used for logging
`organization_name` (`string`) required
Name of organization to use in private certificate
`region` (`string`) required
VPN Endpoints are region-specific. This identifies the region. AWS Region
### Optional Variables
`associated_security_group_ids` (`list(string)`) optional
List of security groups to attach to the client vpn network associations **Default value:** `[ ]`
`authentication_type` (`string`) optional
One of `certificate-authentication` or `federated-authentication` **Default value:** `"certificate-authentication"`
`ca_common_name` (`string`) optional
Unique Common Name for CA self-signed certificate **Default value:** `null`
`dns_servers` (`list(string)`) optional
Information about the DNS servers to be used for DNS resolution. A Client VPN endpoint can have up to two DNS servers. If no DNS server is specified, the DNS address of the VPC that is to be associated with Client VPN endpoint is used as the DNS server. **Default value:** `[ ]`
`export_client_certificate` (`bool`) optional
Flag to determine whether to export the client certificate with the VPN configuration **Default value:** `true`
`logging_enabled` (`bool`) optional
Enables or disables Client VPN Cloudwatch logging. **Default value:** `false`
`retention_in_days` (`number`) optional
Number of days you want to retain log events in the log group **Default value:** `30`
`root_common_name` (`string`) optional
Unique Common Name for Root self-signed certificate **Default value:** `null`
`saml_metadata_document` (`string`) optional
Optional SAML metadata document. Must include this or `saml_provider_arn` **Default value:** `null`
`saml_provider_arn` (`string`) optional
Optional SAML provider ARN. Must include this or `saml_metadata_document` **Default value:** `null`
`server_common_name` (`string`) optional
Unique Common Name for Server self-signed certificate **Default value:** `null`
`split_tunnel` (`bool`) optional
Indicates whether split-tunnel is enabled on VPN endpoint. Default value is false. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`client_configuration`
VPN Client Configuration file (.ovpn) contents that can be imported into AWS client vpn
`full_client_configuration`
Client configuration including client certificate and private key for mutual authentication
`vpn_endpoint_arn`
The ARN of the Client VPN Endpoint Connection.
`vpn_endpoint_dns_name`
The DNS Name of the Client VPN Endpoint Connection.
`vpn_endpoint_id`
The ID of the Client VPN Endpoint Connection.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0, < 6.0.0` - `awsutils`, version: `>= 0.11.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ec2_client_vpn` | 0.16.0 | [`cloudposse/ec2-client-vpn/aws`](https://registry.terraform.io/modules/cloudposse/ec2-client-vpn/aws/0.16.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## ec2-instance This component is responsible for provisioning a single EC2 instance. ## Usage **Stack Level**: Regional The typical stack configuration for this component is as follows: ```yaml components: terraform: ec2-instance: vars: enabled: true name: ec2 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`ami_filters` optional
A list of AMI filters for finding the latest AMI **Type:** ```hcl list(object({ name = string values = list(string) })) ``` **Default value:** ```hcl [ { "name": "architecture", "values": [ "x86_64" ] }, { "name": "virtualization-type", "values": [ "hvm" ] } ] ```
`ami_name_regex` (`string`) optional
The regex used to match the latest AMI to be used for the EC2 instance. **Default value:** `"^amzn2-ami-hvm.*"`
`ami_owner` (`string`) optional
The owner of the AMI used for the ZScaler EC2 instances. **Default value:** `"amazon"`
`instance_type` (`string`) optional
The instance family to use for the EC2 instance **Default value:** `"t3a.micro"`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully completed with `aws_security_group_rule` resource. To get more info see [security_group_rule](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule). **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "from_port": 0, "protocol": "-1", "to_port": 65535, "type": "egress" } ] ```
`user_data` (`string`) optional
User data to be included with this EC2 instance **Default value:** `"echo \"hello user data\""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`instance_id`
Instance ID
`private_ip`
Private IP of the instance
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `template`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `template`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ec2_instance` | 2.0.0 | [`cloudposse/ec2-instance/aws`](https://registry.terraform.io/modules/cloudposse/ec2-instance/aws/2.0.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ami.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`template_file.userdata`](https://registry.terraform.io/providers/cloudposse/template/latest/docs/data-sources/file) (data source) --- ## ecr This component is responsible for provisioning repositories, lifecycle rules, and permissions for streamlined ECR usage. This utilizes [the roles-to-principals submodule](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/account-map/modules/roles-to-principals) to assign accounts to various roles. It is also compatible with the [GitHub Actions IAM Role mixin](https://github.com/cloudposse-terraform-components/mixins/blob/main/src/mixins/github-actions-iam-role/README-github-action-iam-role.md).
Warning (Older) regarding eks-iam component :::warning Older versions of our reference architecture have an`eks-iam` component that needs to be updated to provide sufficient IAM roles to allow pods to pull from ECR repos :::
## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. This component is normally only applied once as the resources it creates are globally accessible, but you may want to create ECRs in multiple regions for redundancy. This is typically provisioned via the stack for the "artifact" account (typically `auto`, `artifact`, or `corp`) in the primary region. ```yaml components: terraform: ecr: vars: ecr_user_enabled: false enable_lifecycle_policy: true max_image_count: 500 scan_images_on_push: true protected_tags: - prod # Tag mutability supports: MUTABLE, IMMUTABLE, IMMUTABLE_WITH_EXCLUSION, MUTABLE_WITH_EXCLUSION image_tag_mutability: IMMUTABLE_WITH_EXCLUSION # When using *_WITH_EXCLUSION, specify exclusions to allow certain tags to be mutable image_tag_mutability_exclusion_filter: - filter: "latest" filter_type: "WILDCARD" - filter: "dev-" filter_type: "WILDCARD" images: - infrastructure - microservice-a - microservice-b - microservice-c read_write_account_role_map: identity: - admin - cicd automation: - admin read_only_account_role_map: corp: ["*"] dev: ["*"] prod: ["*"] stage: ["*"] ``` ## Variables ### Required Variables
`enable_lifecycle_policy` (`bool`) required
Enable/disable image lifecycle policy
`images` (`list(string)`) required
List of image names (ECR repo names) to create repos for
`max_image_count` (`number`) required
Max number of images to store. Old ones will be deleted to make room for new ones.
`read_write_account_role_map` (`map(list(string))`) required
Map of `account:[role, role...]` for write access. Use `*` for role to grant access to entire account
`region` (`string`) required
AWS Region
### Optional Variables
`custom_lifecycle_rules` optional
Custom lifecycle rules to override or complement the default ones **Type:** ```hcl list(object({ description = optional(string) selection = object({ tagStatus = string countType = string countNumber = number countUnit = optional(string) tagPrefixList = optional(list(string)) tagPatternList = optional(list(string)) }) action = object({ type = string }) })) ``` **Default value:** `[ ]`
`default_lifecycle_rules_settings` optional
Default lifecycle rules settings **Type:** ```hcl object({ untagged_image_rule = optional(object({ enabled = optional(bool, true) }), { enabled = true }) remove_old_image_rule = optional(object({ enabled = optional(bool, true) }), { enabled = true }) }) ``` **Default value:** ```hcl { "remove_old_image_rule": { "enabled": true }, "untagged_image_rule": { "enabled": true } } ```
`ecr_user_enabled` (`bool`) optional
Enable/disable the provisioning of the ECR user (for CI/CD systems that don't support assuming IAM roles to access ECR, e.g. Codefresh) **Default value:** `false`
`image_tag_mutability` (`string`) optional
The tag mutability setting for the repository. Must be one of: `MUTABLE`, `IMMUTABLE`, `IMMUTABLE_WITH_EXCLUSION`, or `MUTABLE_WITH_EXCLUSION` **Default value:** `"MUTABLE"`
`image_tag_mutability_exclusion_filter` optional
List of exclusion filters for image tag mutability. Each filter object must contain 'filter' and 'filter_type' attributes. Requires AWS provider >= 6.8.0 **Type:** ```hcl list(object({ filter = string filter_type = optional(string, "WILDCARD") })) ``` **Default value:** `[ ]`
`principals_lambda` (`list(string)`) optional
Principal account IDs of Lambdas allowed to consume ECR **Default value:** `[ ]`
`protected_tags` (`list(string)`) optional
Tags to refrain from deleting **Default value:** `[ ]`
`protected_tags_keep_count` (`number`) optional
Number of Image versions to keep for protected tags **Default value:** `999999`
`pull_through_cache_rules` optional
Map of pull through cache rules to configure **Type:** ```hcl map(object({ registry = string secret = optional(string, "") })) ``` **Default value:** `{ }`
`read_only_account_role_map` (`map(list(string))`) optional
Map of `account:[role, role...]` for read-only access. Use `*` for role to grant access to entire account **Default value:** `{ }`
`replication_configurations` optional
Replication configuration for a registry. See [Replication Configuration](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_replication_configuration#replication-configuration). **Type:** ```hcl list(object({ rules = list(object({ destinations = list(object({ region = string registry_id = string })) repository_filters = list(object({ filter = string filter_type = string })) })) })) ``` **Default value:** `[ ]`
`scan_images_on_push` (`bool`) optional
Indicates whether images are scanned after being pushed to the repository **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ecr_repo_arn_map`
Map of image names to ARNs
`ecr_repo_url_map`
Map of image names to URLs
`ecr_user_arn`
ECR user ARN
`ecr_user_name`
ECR user name
`ecr_user_unique_id`
ECR user unique ID assigned by AWS
`repository_host`
ECR repository name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 6.8.0, < 7.0.0` ### Providers - `aws`, version: `>= 6.8.0, < 7.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ecr` | 1.0.0 | [`cloudposse/ecr/aws`](https://registry.terraform.io/modules/cloudposse/ecr/aws/1.0.0) | n/a `full_access` | latest | [`../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../account-map/modules/roles-to-principals/) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `readonly_access` | latest | [`../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../account-map/modules/roles-to-principals/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ecr_pull_through_cache_rule.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_pull_through_cache_rule) (resource) - [`aws_iam_policy.ecr_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_user.ecr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) (resource) - [`aws_iam_user_policy_attachment.ecr_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.ecr_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_secretsmanager_secret.cache_credentials`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret) (data source) --- ## ecs This component is responsible for provisioning an ECS Cluster and associated load balancer. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. The following will create - ecs cluster - load balancer with an ACM cert placed on example.com - r53 record on all \*.example.com which will point to the load balancer ```yaml components: terraform: ecs: settings: spacelift: workspace_enabled: true vars: name: ecs enabled: true acm_certificate_domain: example.com route53_record_name: "*" # Create records will be created in each zone zone_names: - example.com capacity_providers_fargate: true capacity_providers_fargate_spot: true capacity_providers_ec2: default: instance_type: t3.medium max_size: 2 alb_configuration: public: internal_enabled: false # resolves to *.public-platform..... route53_record_name: "*.public-platform" additional_certs: - "my-vanity-domain.com" private: internal_enabled: true route53_record_name: "*.private-platform" additional_certs: - "my-vanity-domain.com" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`acm_certificate_domain` (`string`) optional
Domain to get the ACM cert to use on the ALB. **Default value:** `null`
`acm_certificate_domain_suffix` (`string`) optional
Domain suffix to use with dns delegated HZ to get the ACM cert to use on the ALB **Default value:** `null`
`alb_configuration` (`map(any)`) optional
Map of multiple ALB configurations. **Default value:** `{ }`
`alb_ingress_cidr_blocks_http` (`list(string)`) optional
List of CIDR blocks allowed to access environment over HTTP **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`alb_ingress_cidr_blocks_https` (`list(string)`) optional
List of CIDR blocks allowed to access environment over HTTPS **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the ECS cluster **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
List of Security Group IDs to be allowed to connect to the ECS cluster **Default value:** `[ ]`
`capacity_providers_ec2` optional
EC2 autoscale groups capacity providers **Type:** ```hcl map(object({ instance_type = string max_size = number security_group_ids = optional(list(string), []) min_size = optional(number, 0) image_id = optional(string) instance_initiated_shutdown_behavior = optional(string, "terminate") key_name = optional(string, "") user_data = optional(string, "") enable_monitoring = optional(bool, true) instance_warmup_period = optional(number, 300) maximum_scaling_step_size = optional(number, 1) minimum_scaling_step_size = optional(number, 1) target_capacity_utilization = optional(number, 100) ebs_optimized = optional(bool, false) block_device_mappings = optional(list(object({ device_name = string no_device = bool virtual_name = string ebs = object({ delete_on_termination = bool encrypted = bool iops = number throughput = number kms_key_id = string snapshot_id = string volume_size = number volume_type = string }) })), []) instance_market_options = optional(object({ market_type = string spot_options = object({ block_duration_minutes = number instance_interruption_behavior = string max_price = number spot_instance_type = string valid_until = string }) })) instance_refresh = optional(object({ strategy = string preferences = optional(object({ instance_warmup = optional(number, null) min_healthy_percentage = optional(number, null) skip_matching = optional(bool, null) auto_rollback = optional(bool, null) scale_in_protected_instances = optional(string) standby_instances = optional(string) }), null) triggers = optional(list(string), []) })) mixed_instances_policy = optional(object({ instances_distribution = object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string }) }), { instances_distribution = null }) placement = optional(object({ affinity = string availability_zone = string group_name = string host_id = string tenancy = string })) credit_specification = optional(object({ cpu_credits = string })) elastic_gpu_specifications = optional(object({ type = string })) disable_api_termination = optional(bool, false) default_cooldown = optional(number, 300) health_check_grace_period = optional(number, 300) force_delete = optional(bool, false) termination_policies = optional(list(string), ["Default"]) suspended_processes = optional(list(string), []) placement_group = optional(string, "") metrics_granularity = optional(string, "1Minute") enabled_metrics = optional(list(string), [ "GroupMinSize", "GroupMaxSize", "GroupDesiredCapacity", "GroupInServiceInstances", "GroupPendingInstances", "GroupStandbyInstances", "GroupTerminatingInstances", "GroupTotalInstances", "GroupInServiceCapacity", "GroupPendingCapacity", "GroupStandbyCapacity", "GroupTerminatingCapacity", "GroupTotalCapacity", "WarmPoolDesiredCapacity", "WarmPoolWarmedCapacity", "WarmPoolPendingCapacity", "WarmPoolTerminatingCapacity", "WarmPoolTotalCapacity", "GroupAndWarmPoolDesiredCapacity", "GroupAndWarmPoolTotalCapacity", ]) wait_for_capacity_timeout = optional(string, "10m") service_linked_role_arn = optional(string, "") metadata_http_endpoint_enabled = optional(bool, true) metadata_http_put_response_hop_limit = optional(number, 2) metadata_http_tokens_required = optional(bool, true) metadata_http_protocol_ipv6_enabled = optional(bool, false) tag_specifications_resource_types = optional(set(string), ["instance", "volume"]) max_instance_lifetime = optional(number, null) capacity_rebalance = optional(bool, false) warm_pool = optional(object({ pool_state = string min_size = number max_group_prepared_capacity = number })) })) ``` **Default value:** `{ }`
`capacity_providers_fargate` (`bool`) optional
Use FARGATE capacity provider **Default value:** `true`
`capacity_providers_fargate_spot` (`bool`) optional
Use FARGATE_SPOT capacity provider **Default value:** `false`
`container_insights_enabled` (`bool`) optional
Whether or not to enable container insights **Default value:** `true`
`dns_delegated_component_name` (`string`) optional
Use this component name to read from the remote state to get the dns_delegated zone ID **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
Use this environment name to read from the remote state to get the dns_delegated zone ID **Default value:** `"gbl"`
`dns_delegated_stage_name` (`string`) optional
Use this stage name to read from the remote state to get the dns_delegated zone ID **Default value:** `null`
`internal_enabled` (`bool`) optional
Whether to create an internal load balancer for services in this cluster **Default value:** `false`
`maintenance_page_path` (`string`) optional
The path from this directory to the text/html page to use as the maintenance page. Must be within 1024 characters **Default value:** `"templates/503_example.html"`
`route53_enabled` (`bool`) optional
Whether or not to create a route53 record for the ALB **Default value:** `true`
`route53_record_name` (`string`) optional
The route53 record name **Default value:** `"*"`
`vpc_component_name` (`string`) optional
The name of a VPC component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`alb`
ALB outputs
`cluster_arn`
ECS cluster ARN
`cluster_name`
ECS Cluster Name
`private_subnet_ids`
Private subnet ids
`public_subnet_ids`
Public subnet ids
`records`
Record names
`security_group_id`
Security group id
`vpc_id`
VPC ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>=6.0.0` ### Providers - `aws`, version: `>=6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb` | 1.12.0 | [`cloudposse/alb/aws`](https://registry.terraform.io/modules/cloudposse/alb/aws/1.12.0) | n/a `cluster` | 1.2.0 | [`cloudposse/ecs-cluster/aws`](https://registry.terraform.io/modules/cloudposse/ecs-cluster/aws/1.2.0) | n/a `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `target_group_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | This is used due to the short limit on target group names i.e. 32 characters `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_lb_listener_certificate.additional_certs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_certificate) (resource) - [`aws_route53_record.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_acm_certificate.additional_certs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/acm_certificate) (data source) - [`aws_acm_certificate.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/acm_certificate) (data source) --- ## ecs-service This component is responsible for creating an ECS service. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml # stacks/catalog/ecs-service/defaults.yaml components: terraform: ecs-service/defaults: metadata: component: ecs-service type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true public_lb_enabled: false ecr_stage_name: mgmt-automation task: launch_type: FARGATE network_mode: awsvpc desired_count: 1 ignore_changes_desired_count: true ignore_changes_task_definition: false assign_public_ip: false propagate_tags: SERVICE wait_for_steady_state: true circuit_breaker_deployment_enabled: true circuit_breaker_rollback_enabled: true ``` This will launch google's `echoserver` using an external image from gcr NOTE: Usage of `image` instead of `ecr_image`. ```yaml # stacks/catalog/ecs-service/echoserver.yaml import: - catalog/ecs-service/defaults components: terraform: ecs/platform/echoserver/service: metadata: component: ecs-service inherits: - ecs-service/defaults vars: enabled: true name: echoserver public_lb_enabled: false cluster_attributes: [platform] ## Example task_exec_iam_policy # task_exec_iam_policy: # - policy_id: "EcsServiceEchoServer" # statements: # - sid: "EcsServiceEchoServer" # effect: "Allow" # actions: # - "kms:Decrypt" # resources: # - "*" containers: service: name: "echoserver" image: gcr.io/google_containers/echoserver:1.10 port_mappings: - containerPort: 8080 hostPort: 8080 protocol: tcp task: desired_count: 1 task_memory: 512 task_cpu: 256 ``` This will launch a `kong` service using an ecr image from `mgmt-automation` account. NOTE: Usage of `ecr_image` instead of `image`. ```yaml import: - catalog/ecs-service/defaults components: terraform: ecs/b2b/kong/service: metadata: component: ecs-service inherits: - ecs-service/defaults vars: name: kong public_lb_enabled: true cluster_attributes: [b2b] containers: service: name: "kong-gateway" ecr_image: kong:latest map_environment: KONG_DECLARATIVE_CONFIG: /home/kong/production.yml port_mappings: - containerPort: 8000 hostPort: 8000 protocol: tcp task: desired_count: 1 task_memory: 512 task_cpu: 256 ``` This will launch a `httpd` service using an external image from dockerhub NOTE: Usage of `image` instead of `ecr_image`. ```yaml # stacks/catalog/ecs-service/httpd.yaml import: - catalog/ecs-service/defaults components: terraform: ecs/platform/httpd/service: metadata: component: ecs-service inherits: - ecs-service/defaults vars: enabled: true name: httpd public_lb_enabled: true cluster_attributes: [platform] containers: service: name: "Hello" image: httpd:2.4 port_mappings: - containerPort: 80 hostPort: 80 protocol: tcp command: - '/bin/sh -c "echo '' Amazon ECS Sample App Amazon ECS Sample App Congratulations! Your application is now running on a container in Amazon ECS. '' > /usr/local/apache2/htdocs/index.html && httpd-foreground"' entrypoint: ["sh", "-c"] task: desired_count: 1 task_memory: 512 task_cpu: 256 ``` #### Other Domains This component supports alternate service names for your ECS Service through a couple of variables: - `vanity_domain` & `vanity_alias` - This will create a route to the service in the listener rules of the ALB. This will also create a Route 53 alias record in the hosted zone in this account. The hosted zone is looked up by the `vanity_domain` input. - `additional_targets` - This will create a route to the service in the listener rules of the ALB. This will not create a Route 53 alias record. Examples: ```yaml ecs/platform/service/echo-server: vars: vanity_domain: "dev-acme.com" vanity_alias: - "echo-server.dev-acme.com" additional_targets: - "echo.acme.com" ``` This then creates the following listener rules: ```text HTTP Host Header is echo-server.public-platform.use2.dev.plat.service-discovery.com OR echo-server.dev-acme.com OR echo.acme.com ``` It will also create the record in Route53 to point `"echo-server.dev-acme.com"` to the ALB. Thus `"echo-server.dev-acme.com"` should resolve. We can then create a pointer to this service in the `acme.come` hosted zone. ```yaml dns-primary: vars: domain_names: - acme.com record_config: - root_zone: acme.com name: echo. type: CNAME ttl: 60 records: - echo-server.dev-acme.com ``` This will create a CNAME record in the `acme.com` hosted zone that points `echo.acme.com` to `echo-server.dev-acme.com`. ### EFS EFS is supported by this ecs service, you can use either `efs_volumes` or `efs_component_volumes` in your task definition. This example shows how to use `efs_component_volumes` which remote looks up efs component and uses the `efs_id` to mount the volume. And how to use `efs_volumes` ```yaml components: terraform: ecs-services/my-service: metadata: component: ecs-service inherits: - ecs-services/defaults vars: containers: service: name: app image: my-image:latest log_configuration: logDriver: awslogs options: {} port_mappings: - containerPort: 8080 hostPort: 8080 protocol: tcp mount_points: - containerPath: "/var/lib/" sourceVolume: "my-volume-mount" task: efs_component_volumes: - name: "my-volume-mount" host_path: null efs_volume_configuration: - component: efs/my-volume-mount root_directory: "/var/lib/" transit_encryption: "ENABLED" transit_encryption_port: 2999 authorization_config: [] efs_volumes: - name: "my-volume-mount-2" host_path: null efs_volume_ configuration: - file_system_id: "fs-1234" root_directory: "/var/lib/" transit_encryption: "ENABLED" transit_encryption_port: 2998 authorization_config: [] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`additional_lb_target_groups` optional
Additional load balancer target group configurations for registering multiple container ports. This allows you to register sidecar containers to separate target groups. Each entry requires: - container_name: Name of the container to register - container_port: Port on the container to register - target_group_arn: ARN of the target group. Each additional port must specify a unique target group ARN **Type:** ```hcl list(object({ container_name = string container_port = number target_group_arn = string })) ``` **Default value:** `[ ]`
`additional_security_groups` (`list(string)`) optional
A list of additional security group IDs to add to the service **Default value:** `[ ]`
`additional_targets` (`list(string)`) optional
Additional target routes to add to the ALB that point to this service. The only difference between this and `var.vanity_alias` is `var.vanity_alias` will create an alias record in Route 53 in the hosted zone in this account as well. `var.additional_targets` only adds the listener route to this service's target group. **Default value:** `[ ]`
`alb_configuration` (`string`) optional
The configuration to use for the ALB, specifying which cluster alb configuration to use **Default value:** `"default"`
`alb_name` (`string`) optional
The name of the ALB this service should attach to **Default value:** `null`
`autoscaling_dimension` (`string`) optional
The dimension to use to decide to autoscale **Default value:** `"cpu"`
`autoscaling_enabled` (`bool`) optional
Should this service autoscale using SNS alarams **Default value:** `true`
`chamber_service` (`string`) optional
SSM parameter service name for use with chamber. This is used in chamber_format where /$chamber_service/$name/$container_name/$parameter would be the default. **Default value:** `"ecs-service"`
`cluster_attributes` (`list(string)`) optional
The attributes of the cluster name e.g. if the full name is `namespace-tenant-environment-dev-ecs-b2b` then the `cluster_name` is `ecs` and this value should be `b2b`. **Default value:** `[ ]`
`containers` optional
Inputs for the container definition module. `user`: The user to run as inside the container. Can be any of these formats: user, user:group, uid, uid:gid, user:gid, uid:group. The default (null) will use the container's configured `USER` directive or root if not set." **Type:** ```hcl map(object({ name = string ecr_image = optional(string) image = optional(string) memory = optional(number) memory_reservation = optional(number) cpu = optional(number) essential = optional(bool, true) readonly_root_filesystem = optional(bool, null) privileged = optional(bool, null) user = optional(string, null) container_depends_on = optional(list(object({ containerName = string condition = string # START, COMPLETE, SUCCESS, HEALTHY })), null) port_mappings = optional(list(object({ containerPort = number hostPort = optional(number) protocol = optional(string) name = optional(string) appProtocol = optional(string) })), []) command = optional(list(string), null) entrypoint = optional(list(string), null) healthcheck = optional(object({ command = list(string) interval = number retries = number startPeriod = number timeout = number }), null) ulimits = optional(list(object({ name = string softLimit = number hardLimit = number })), null) log_configuration = optional(object({ logDriver = string options = optional(map(string), {}) })) docker_labels = optional(map(string), null) map_environment = optional(map(string), {}) map_secrets = optional(map(string), {}) volumes_from = optional(list(object({ sourceContainer = string readOnly = bool })), null) mount_points = optional(list(object({ sourceVolume = optional(string) containerPath = optional(string) readOnly = optional(bool) })), []) })) ``` **Default value:** `{ }`
`cpu_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High Alarm action **Default value:** `[ ]`
`cpu_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`cpu_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High OK action **Default value:** `[ ]`
`cpu_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`cpu_utilization_high_threshold` (`number`) optional
The maximum percentage of CPU utilization average **Default value:** `80`
`cpu_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low Alarm action **Default value:** `[ ]`
`cpu_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`cpu_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low OK action **Default value:** `[ ]`
`cpu_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`cpu_utilization_low_threshold` (`number`) optional
The minimum percentage of CPU utilization average **Default value:** `20`
`custom_security_group_rules` optional
The list of custom security group rules to add to the service security group **Type:** ```hcl list(object({ type = string from_port = number to_port = number protocol = string cidr_blocks = optional(list(string)) description = optional(string) source_security_group_id = optional(string) prefix_list_ids = optional(list(string)) security_group_id = optional(string) })) ``` **Default value:** `[ ]`
`datadog_agent_sidecar_enabled` (`bool`) optional
Enable the Datadog Agent Sidecar **Default value:** `false`
`datadog_api_key_ssm_parameter_name` (`string`) optional
The SSM Parameter Name containing the Datadog API Key **Default value:** `null`
`datadog_log_method_is_firelens` (`bool`) optional
Datadog logs can be sent via cloudwatch logs (and lambda) or firelens, set this to true to enable firelens via a sidecar container for fluentbit **Default value:** `false`
`datadog_logging_default_tags_enabled` (`bool`) optional
Add Default tags to all logs sent to Datadog **Default value:** `true`
`datadog_logging_tags` (`map(string)`) optional
Tags to add to all logs sent to Datadog **Default value:** `null`
`datadog_sidecar_containers_logs_enabled` (`bool`) optional
Enable the Datadog Agent Sidecar to send logs to aws cloudwatch group, requires `datadog_agent_sidecar_enabled` to be true **Default value:** `true`
`datadog_site` (`string`) optional
The Datadog Site to send logs to **Default value:** `"us5.datadoghq.com"`
`ecr_region` (`string`) optional
The region to use for the fully qualified ECR image URL. Defaults to the current region. **Default value:** `""`
`ecr_stage_name` (`string`) optional
The ecr stage (account) name to use for the fully qualified ECR image URL. **Default value:** `"auto"`
`ecs_cluster_name` (`any`) optional
The name of the ECS Cluster this belongs to **Default value:** `"ecs"`
`enable_all_egress_rule` (`bool`) optional
A flag to enable/disable adding the all ports egress rule to the service security group **Default value:** `true`
`exec_enabled` (`bool`) optional
Specifies whether to enable Amazon ECS Exec for the tasks within the service **Default value:** `false`
`github_actions_allowed_repos` (`list(string)`) optional
A list of the GitHub repositories that are allowed to assume this role from GitHub Actions. For example, ["cloudposse/infra-live"]. Can contain "*" as wildcard. If org part of repo name is omitted, "cloudposse" will be assumed. **Default value:** `[ ]`
`github_actions_ecspresso_enabled` (`bool`) optional
Create IAM policies required for deployments with Ecspresso **Default value:** `false`
`github_actions_iam_role_attributes` (`list(string)`) optional
Additional attributes to add to the role name **Default value:** `[ ]`
`github_actions_iam_role_enabled` (`bool`) optional
Flag to toggle creation of an IAM Role that GitHub Actions can assume to access AWS resources **Default value:** `false`
`github_oidc_trusted_role_arns` (`list(string)`) optional
A list of IAM Role ARNs allowed to assume this cluster's GitHub OIDC role **Default value:** `[ ]`
`health_check_healthy_threshold` (`number`) optional
The number of consecutive health checks successes required before healthy **Default value:** `2`
`health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `15`
`health_check_matcher` (`string`) optional
The HTTP response codes to indicate a healthy check **Default value:** `"200-404"`
`health_check_path` (`string`) optional
The destination for the health check request **Default value:** `"/health"`
`health_check_port` (`string`) optional
The port to use to connect with the target. Valid values are either ports 1-65536, or `traffic-port`. Defaults to `traffic-port` **Default value:** `"traffic-port"`
`health_check_protocol` (`string`) optional
The protocol to use to connect with the target. Defaults to HTTP. Not applicable when target_type is lambda **Default value:** `"HTTP"`
`health_check_timeout` (`number`) optional
The amount of time to wait in seconds before failing a health check request **Default value:** `10`
`health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before unhealthy **Default value:** `2`
`http_protocol` (`string`) optional
Which http protocol to use in outputs and SSM url params. This value is ignored if a load balancer is not used. If it is `null`, the redirect value from the ALB determines the protocol. **Default value:** `null`
`iam_policy_enabled` (`bool`) optional
If set to true will create IAM policy in AWS **Default value:** `false`
`iam_policy_statements` (`any`) optional
Map of IAM policy statements to use in the policy. This can be used with or instead of the `var.iam_source_json_url`. **Default value:** `{ }`
`kinesis_enabled` (`bool`) optional
Enable Kinesis **Default value:** `false`
`kms_alias_name_ssm` (`string`) optional
KMS alias name for SSM **Default value:** `"alias/aws/ssm"`
`kms_key_alias` (`string`) optional
ID of KMS key **Default value:** `"default"`
`lb_catch_all` (`bool`) optional
Should this service act as catch all for all subdomain hosts of the vanity domain **Default value:** `false`
`logs` (`any`) optional
Feed inputs into cloudwatch logs module **Default value:** `{ }`
`memory_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High Alarm action **Default value:** `[ ]`
`memory_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`memory_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High OK action **Default value:** `[ ]`
`memory_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`memory_utilization_high_threshold` (`number`) optional
The maximum percentage of Memory utilization average **Default value:** `80`
`memory_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low Alarm action **Default value:** `[ ]`
`memory_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`memory_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low OK action **Default value:** `[ ]`
`memory_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`memory_utilization_low_threshold` (`number`) optional
The minimum percentage of Memory utilization average **Default value:** `20`
`nlb_name` (`string`) optional
The name of the NLB this service should attach to **Default value:** `null`
`port` (`number`) optional
The port for the created ALB target group. Defaults to 80 **Default value:** `80`
`protocol` (`string`) optional
The protocol for the created ALB target group. Defaults to HTTP **Default value:** `"HTTP"`
`rds_name` (`any`) optional
The name of the RDS database this service should allow access to **Default value:** `null`
`retention_period` (`number`) optional
Length of time data records are accessible after they are added to the stream **Default value:** `48`
`s3_mirror_name` (`string`) optional
The name of the S3 mirror component **Default value:** `null`
`scale_down_step_adjustments` optional
List of step adjustments for scale down policy **Type:** ```hcl list(object({ metric_interval_lower_bound = optional(number) metric_interval_upper_bound = optional(number) scaling_adjustment = number })) ``` **Default value:** ```hcl [ { "metric_interval_lower_bound": null, "metric_interval_upper_bound": 0, "scaling_adjustment": -1 } ] ```
`scale_up_step_adjustments` optional
List of step adjustments for scale up policy **Type:** ```hcl list(object({ metric_interval_lower_bound = optional(number) metric_interval_upper_bound = optional(number) scaling_adjustment = number })) ``` **Default value:** ```hcl [ { "metric_interval_lower_bound": 0, "metric_interval_upper_bound": null, "scaling_adjustment": 1 } ] ```
`service_connect_configurations` optional
The list of Service Connect configurations. See `service_connect_configuration` docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#service_connect_configuration **Type:** ```hcl list(object({ enabled = bool namespace = optional(string, null) log_configuration = optional(object({ log_driver = string options = optional(map(string), null) secret_option = optional(list(object({ name = string value_from = string })), []) }), null) service = optional(list(object({ client_alias = list(object({ dns_name = string port = number })) discovery_name = optional(string, null) ingress_port_override = optional(number, null) port_name = string })), []) })) ``` **Default value:** `[ ]`
`service_registries` optional
The list of Service Registries. See `service_registries` docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#service_registries **Type:** ```hcl list(object({ namespace = string registry_arn = optional(string) port = optional(number) container_name = optional(string) container_port = optional(number) })) ``` **Default value:** `[ ]`
`shard_count` (`number`) optional
Number of shards that the stream will use **Default value:** `1`
`shard_level_metrics` (`list(string)`) optional
List of shard-level CloudWatch metrics which can be enabled for the stream **Default value:** ```hcl [ "IncomingBytes", "IncomingRecords", "IteratorAgeMilliseconds", "OutgoingBytes", "OutgoingRecords", "ReadProvisionedThroughputExceeded", "WriteProvisionedThroughputExceeded" ] ```
`ssm_enabled` (`bool`) optional
If `true` create SSM keys for the database user and password. **Default value:** `false`
`ssm_key_format` (`string`) optional
SSM path format. The values will will be used in the following order: `var.ssm_key_prefix`, `var.name`, `var.ssm_key_*` **Default value:** `"/%v/%v/%v"`
`ssm_key_prefix` (`string`) optional
SSM path prefix. Omit the leading forward slash `/`. **Default value:** `"ecs-service"`
`stickiness_cookie_duration` (`number`) optional
The time period, in seconds, during which requests from a client should be routed to the same target. After this time period expires, the load balancer-generated cookie is considered stale. The range is 1 second to 1 week (604800 seconds). The default value is 1 day (86400 seconds) **Default value:** `86400`
`stickiness_enabled` (`bool`) optional
Boolean to enable / disable `stickiness`. Default is `true` **Default value:** `true`
`stickiness_type` (`string`) optional
The type of sticky sessions. The only current possible value is `lb_cookie` **Default value:** `"lb_cookie"`
`stream_mode` (`string`) optional
Stream mode details for the Kinesis stream **Default value:** `"PROVISIONED"`
`task` optional
Feed inputs into ecs_alb_service_task module **Type:** ```hcl object({ task_cpu = optional(number) task_memory = optional(number) task_role_arn = optional(string, "") pid_mode = optional(string, null) ipc_mode = optional(string, null) network_mode = optional(string) propagate_tags = optional(string) assign_public_ip = optional(bool, false) use_alb_security_groups = optional(bool, true) launch_type = optional(string, "FARGATE") scheduling_strategy = optional(string, "REPLICA") capacity_provider_strategies = optional(list(object({ capacity_provider = string weight = number base = number })), []) deployment_minimum_healthy_percent = optional(number, null) deployment_maximum_percent = optional(number, null) desired_count = optional(number, 0) min_capacity = optional(number, 1) max_capacity = optional(number, 2) wait_for_steady_state = optional(bool, true) circuit_breaker_deployment_enabled = optional(bool, true) circuit_breaker_rollback_enabled = optional(bool, true) ignore_changes_task_definition = optional(bool, true) ignore_changes_desired_count = optional(bool, false) ecs_service_enabled = optional(bool, true) bind_mount_volumes = optional(list(object({ name = string host_path = string })), []) efs_volumes = optional(list(object({ host_path = string name = string efs_volume_configuration = list(object({ file_system_id = string root_directory = string transit_encryption = string transit_encryption_port = string authorization_config = list(object({ access_point_id = string iam = string })) })) })), []) efs_component_volumes = optional(list(object({ host_path = string name = string efs_volume_configuration = list(object({ component = optional(string, "efs") tenant = optional(string, null) environment = optional(string, null) stage = optional(string, null) root_directory = string transit_encryption = string transit_encryption_port = string authorization_config = list(object({ access_point_id = string iam = string })) })) })), []) docker_volumes = optional(list(object({ host_path = string name = string docker_volume_configuration = list(object({ autoprovision = bool driver = string driver_opts = map(string) labels = map(string) scope = string })) })), []) fsx_volumes = optional(list(object({ host_path = string name = string fsx_windows_file_server_volume_configuration = list(object({ file_system_id = string root_directory = string authorization_config = list(object({ credentials_parameter = string domain = string })) })) })), []) }) ``` **Default value:** `{ }`
`task_enabled` (`bool`) optional
Whether or not to use the ECS task module **Default value:** `true`
`task_exec_iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`task_exec_policy_arns_map` (`map(string)`) optional
A map of name to IAM Policy ARNs to attach to the generated task execution role. The names are arbitrary, but must be known at plan time. The purpose of the name is so that changes to one ARN do not cause a ripple effect on the other ARNs. If you cannot provide unique names known at plan time, use `task_exec_policy_arns` instead. **Default value:** `{ }`
`task_iam_role_component` (`string`) optional
A component that outputs an iam_role module as 'role' for adding to the service as a whole. **Default value:** `null`
`task_policy_arns` (`list(string)`) optional
The IAM policy ARNs to attach to the ECS task IAM role **Default value:** ```hcl [ "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly", "arn:aws:iam::aws:policy/AmazonSSMReadOnlyAccess" ] ```
`task_security_group_component` (`string`) optional
A component that outputs security_group_id for adding to the service as a whole. **Default value:** `null`
`unauthenticated_paths` (`list(string)`) optional
Unauthenticated path pattern to match **Default value:** `[ ]`
`unauthenticated_priority` (`string`) optional
The priority for the rules without authentication, between 1 and 50000 (1 being highest priority). Must be different from `authenticated_priority` since a listener can't have multiple rules with the same priority **Default value:** `0`
`use_lb` (`bool`) optional
Whether use load balancer for the service **Default value:** `false`
`use_rds_client_sg` (`bool`) optional
Use the RDS client security group **Default value:** `false`
`vanity_alias` (`list(string)`) optional
The vanity aliases to use for the public LB. **Default value:** `[ ]`
`vanity_domain` (`string`) optional
Whether to use the vanity domain alias for the service **Default value:** `null`
`vpc_component_name` (`string`) optional
The name of a VPC component **Default value:** `"vpc"`
`zone_component` (`string`) optional
The component name to look up service domain remote-state on **Default value:** `"dns-delegated"`
`zone_component_output` (`string`) optional
A json query to use to get the zone domain from the remote state. See **Default value:** `".default_domain_name"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ecs_cluster_arn`
Selected ECS cluster ARN
`environment_map`
Environment variables to pass to the container, this is a map of key/value pairs, where the key is `containerName,variableName`
`full_domain`
Domain to respond to GET requests
`github_actions_iam_role_arn`
ARN of IAM role for GitHub Actions
`github_actions_iam_role_name`
Name of IAM role for GitHub Actions
`lb_arn`
Selected LB ARN
`lb_listener_https`
Selected LB HTTPS Listener
`lb_sg_id`
Selected LB SG ID
`logs`
Output of cloudwatch logs module
`service_arn`
The ECS service ARN
`service_image`
The image of the service container
`service_name`
The ECS service name
`ssm_key_prefix`
SSM prefix
`ssm_parameters`
SSM parameters for the ECS Service
`subnet_ids`
Selected subnet IDs
`task_definition_arn`
The task definition ARN
`task_definition_revision`
The task definition revision
`task_template`
The task template rendered
`vpc_id`
Selected VPC ID
`vpc_sg_id`
Selected VPC SG ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.66.1, < 6.0.0` - `jq`, version: `>=0.2.0` - `template`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.66.1, < 6.0.0` - `jq`, version: `>=0.2.0` - `template`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `alb_ingress` | 0.30.0 | [`cloudposse/alb-ingress/aws`](https://registry.terraform.io/modules/cloudposse/alb-ingress/aws/0.30.0) | n/a `cloudmap_namespace` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `cloudmap_namespace_service_discovery` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `container_definition` | 0.61.2 | [`cloudposse/ecs-container-definition/aws`](https://registry.terraform.io/modules/cloudposse/ecs-container-definition/aws/0.61.2) | n/a `datadog_container_definition` | 0.61.2 | [`cloudposse/ecs-container-definition/aws`](https://registry.terraform.io/modules/cloudposse/ecs-container-definition/aws/0.61.2) | n/a `datadog_fluent_bit_container_definition` | 0.61.2 | [`cloudposse/ecs-container-definition/aws`](https://registry.terraform.io/modules/cloudposse/ecs-container-definition/aws/0.61.2) | n/a `datadog_sidecar_logs` | 0.6.9 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.9) | n/a `ecs_alb_service_task` | 0.78.0 | [`cloudposse/ecs-alb-service-task/aws`](https://registry.terraform.io/modules/cloudposse/ecs-alb-service-task/aws/0.78.0) | n/a `ecs_cloudwatch_autoscaling` | 1.0.0 | [`cloudposse/ecs-cloudwatch-autoscaling/aws`](https://registry.terraform.io/modules/cloudposse/ecs-cloudwatch-autoscaling/aws/1.0.0) | n/a `ecs_cloudwatch_sns_alarms` | 0.13.2 | [`cloudposse/ecs-cloudwatch-sns-alarms/aws`](https://registry.terraform.io/modules/cloudposse/ecs-cloudwatch-sns-alarms/aws/0.13.2) | n/a `ecs_cluster` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `efs` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `gha_assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `gha_role_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `iam_role` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `logs` | 0.6.9 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.9) | n/a `nlb` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `rds` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `roles_to_principals` | latest | [`../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../account-map/modules/roles-to-principals/) | n/a `s3` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `security_group` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `service_domain` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `task_exec_iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vanity_alias` | 0.13.0 | [`cloudposse/route53-alias/aws`](https://registry.terraform.io/modules/cloudposse/route53-alias/aws/0.13.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.github_actions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.github_actions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.additional_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.custom_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_kinesis_stream.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream) (resource) - [`aws_s3_object.task_definition_template`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_object) (resource) - [`aws_security_group_rule.custom_sg_rules`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_service_discovery_service.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/service_discovery_service) (resource) - [`aws_ssm_parameter.full_urls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_ecs_task_definition.created_task`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ecs_task_definition) (data source) - [`aws_iam_policy_document.github_actions_iam_ecspresso_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.github_actions_iam_platform_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.github_actions_iam_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_kms_alias.selected`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_alias) (data source) - [`aws_route53_zone.selected`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) - [`aws_route53_zone.selected_vanity`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) - [`aws_s3_object.task_definition`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_object) (data source) - [`aws_s3_objects.mirror`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_objects) (data source) - [`aws_ssm_parameter.datadog_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameters_by_path.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameters_by_path) (data source) - [`jq_query.service_domain_query`](https://registry.terraform.io/providers/massdriver-cloud/jq/latest/docs/data-sources/query) (data source) - [`template_file.envs`](https://registry.terraform.io/providers/cloudposse/template/latest/docs/data-sources/file) (data source) --- ## efs This component is responsible for provisioning an [EFS](https://aws.amazon.com/efs/) Network File System with KMS encryption-at-rest. EFS is an excellent choice as the default block storage for EKS clusters so that volumes are not zone-locked. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: efs: vars: enabled: true name: shared-files dns_name: shared-files provisioned_throughput_in_mibps: 10 # additional_security_group_rules: # - key: "fargate_efs" # type: "ingress" # from_port: 2049 # to_port: 2049 # protocol: "tcp" # description: "Allow Fargate EFS Volume mounts" # cidr_blocks: ["0.0.0.0/0"] ``` ## Variables ### Required Variables
`hostname_template` (`string`) required
The `format()` string to use to generate the hostname via `format(var.hostname_template, var.tenant, var.stage, var.environment)`" Typically something like `"echo.%[3]v.%[2]v.example.com"`.
`region` (`string`) required
AWS Region
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`efs_backup_policy_enabled` (`bool`) optional
If `true`, automatic backups will be enabled. **Default value:** `false`
`eks_component_names` (`set(string)`) optional
The names of the eks components **Default value:** ```hcl [ "eks/cluster" ] ```
`eks_security_group_enabled` (`bool`) optional
Use the eks default security group **Default value:** `false`
`performance_mode` (`string`) optional
The file system performance mode. Can be either `generalPurpose` or `maxIO` **Default value:** `"generalPurpose"`
`provisioned_throughput_in_mibps` (`number`) optional
The throughput, measured in MiB/s, that you want to provision for the file system. Only applicable with `throughput_mode` set to provisioned **Default value:** `0`
`throughput_mode` (`string`) optional
Throughput mode for the file system. Defaults to bursting. Valid values: `bursting`, `provisioned`. When using `provisioned`, also set `provisioned_throughput_in_mibps` **Default value:** `"bursting"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`efs_arn`
EFS ARN
`efs_dns_name`
EFS DNS name
`efs_host`
DNS hostname for the EFS
`efs_id`
EFS ID
`efs_mount_target_dns_names`
List of EFS mount target DNS names
`efs_mount_target_ids`
List of EFS mount target IDs (one per Availability Zone)
`efs_mount_target_ips`
List of EFS mount target IPs (one per Availability Zone)
`efs_network_interface_ids`
List of mount target network interface IDs
`security_group_arn`
EFS Security Group ARN
`security_group_id`
EFS Security Group ID
`security_group_name`
EFS Security Group name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `efs` | 1.2.1 | [`cloudposse/efs/aws`](https://registry.terraform.io/modules/cloudposse/efs/aws/1.2.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `gbl_dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_efs` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_efs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## actions-runner-controller This component creates a Helm release for [actions-runner-controller](https://github.com/actions-runner-controller/actions-runner-controller) on an EKS cluster. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/actions-runner-controller ... ``` The default catalog values `e.g. stacks/catalog/eks/actions-runner-controller.yaml` ```yaml components: terraform: eks/actions-runner-controller: vars: enabled: true name: "actions-runner" # avoids hitting name length limit on IAM role chart: "actions-runner-controller" chart_repository: "https://actions-runner-controller.github.io/actions-runner-controller" chart_version: "0.23.7" kubernetes_namespace: "actions-runner-system" create_namespace: true kubeconfig_exec_auth_api_version: "client.authentication.k8s.io/v1beta1" # helm_manifest_experiment_enabled feature causes inconsistent final plans with charts that have CRDs # see https://github.com/hashicorp/terraform-provider-helm/issues/711#issuecomment-836192991 helm_manifest_experiment_enabled: false ssm_github_secret_path: "/github_runners/controller_github_app_secret" github_app_id: "REPLACE_ME_GH_APP_ID" github_app_installation_id: "REPLACE_ME_GH_INSTALLATION_ID" # use to enable docker config json secret, which can login to dockerhub for your GHA Runners docker_config_json_enabled: true # The content of this param should look like: # { # "auths": { # "https://index.docker.io/v1/": { # "username": "your_username", # "password": "your_password # "email": "your_email", # "auth": "$(echo "your_username:your_password" | base64)" # } # } # } | base64 ssm_docker_config_json_path: "/github_runners/docker/config-json" # ssm_github_webhook_secret_token_path: "/github_runners/github_webhook_secret_token" # The webhook based autoscaler is much more efficient than the polling based autoscaler webhook: enabled: true hostname_template: "gha-webhook.%[3]v.%[2]v.%[1]v.acme.com" eks_component_name: "eks/cluster" resources: limits: cpu: 500m memory: 256Mi requests: cpu: 250m memory: 128Mi runners: infra-runner: node_selector: kubernetes.io/os: "linux" kubernetes.io/arch: "amd64" type: "repository" # can be either 'organization' or 'repository' dind_enabled: true # If `true`, a Docker daemon will be started in the runner Pod. # To run Docker in Docker (dind), change image to summerwind/actions-runner-dind # If not running Docker, change image to summerwind/actions-runner use a smaller image image: summerwind/actions-runner-dind # `scope` is org name for Organization runners, repo name for Repository runners scope: "org/infra" min_replicas: 0 # Default, overridden by scheduled_overrides below max_replicas: 20 # Scheduled overrides. See https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#scheduled-overrides # Order is important. The earlier entry is prioritized higher than later entries. So you usually define # one-time overrides at the top of your list, then yearly, monthly, weekly, and lastly daily overrides. scheduled_overrides: # Override the daily override on the weekends - start_time: "2024-07-06T00:00:00-08:00" # Start of Saturday morning Pacific Standard Time end_time: "2024-07-07T23:59:59-07:00" # End of Sunday night Pacific Daylight Time min_replicas: 0 recurrence_rule: frequency: "Weekly" # Keep a warm pool of runners during normal working hours - start_time: "2024-07-01T09:00:00-08:00" # 9am Pacific Standard Time (8am PDT), start of workday end_time: "2024-07-01T17:00:00-07:00" # 5pm Pacific Daylight Time (6pm PST), end of workday min_replicas: 2 recurrence_rule: frequency: "Daily" scale_down_delay_seconds: 100 resources: limits: cpu: 200m memory: 512Mi requests: cpu: 100m memory: 128Mi webhook_driven_scaling_enabled: true # max_duration is the duration after which a job will be considered completed, # (and the runner killed) even if the webhook has not received a "job completed" event. # This is to ensure that if an event is missed, it does not leave the runner running forever. # Set it long enough to cover the longest job you expect to run and then some. # See https://github.com/actions/actions-runner-controller/blob/9afd93065fa8b1f87296f0dcdf0c2753a0548cb7/docs/automatically-scaling-runners.md?plain=1#L264-L268 max_duration: "90m" # Pull-driven scaling is obsolete and should not be used. pull_driven_scaling_enabled: false # Labels are not case-sensitive to GitHub, but *are* case-sensitive # to the webhook based autoscaler, which requires exact matches # between the `runs-on:` label in the workflow and the runner labels. labels: - "Linux" - "linux" - "Ubuntu" - "ubuntu" - "X64" - "x64" - "x86_64" - "amd64" - "AMD64" - "core-auto" - "common" # Uncomment this additional runner if you want to run a second # runner pool for `arm64` architecture #infra-runner-arm64: # node_selector: # kubernetes.io/os: "linux" # kubernetes.io/arch: "arm64" # # Add the corresponding taint to the Kubernetes nodes running `arm64` architecture # # to prevent Kubernetes pods without node selectors from being scheduled on them. # tolerations: # - key: "kubernetes.io/arch" # operator: "Equal" # value: "arm64" # effect: "NoSchedule" # type: "repository" # can be either 'organization' or 'repository' # dind_enabled: false # If `true`, a Docker sidecar container will be deployed # # To run Docker in Docker (dind), change image to summerwind/actions-runner-dind # # If not running Docker, change image to summerwind/actions-runner use a smaller image # image: summerwind/actions-runner-dind # # `scope` is org name for Organization runners, repo name for Repository runners # scope: "org/infra" # group: "ArmRunners" # # Tell Karpenter not to evict this pod while it is running a job. # # If we do not set this, Karpenter will feel free to terminate the runner while it is running a job, # # as part of its consolidation efforts, even when using "on demand" instances. # running_pod_annotations: # karpenter.sh/do-not-disrupt: "true" # min_replicas: 0 # Set to so that no ARM instance is running idle, set to 1 for faster startups # max_replicas: 20 # scale_down_delay_seconds: 100 # resources: # limits: # cpu: 200m # memory: 512Mi # requests: # cpu: 100m # memory: 128Mi # webhook_driven_scaling_enabled: true # max_duration: "90m" # pull_driven_scaling_enabled: false # # Labels are not case-sensitive to GitHub, but *are* case-sensitive # # to the webhook based autoscaler, which requires exact matches # # between the `runs-on:` label in the workflow and the runner labels. # # Leave "common" off the list so that "common" jobs are always # # scheduled on the amd64 runners. This is because the webhook # # based autoscaler will not scale a runner pool if the # # `runs-on:` labels in the workflow match more than one pool. # labels: # - "Linux" # - "linux" # - "Ubuntu" # - "ubuntu" # - "amd64" # - "AMD64" # - "core-auto" ``` ### Generating Required Secrets AWS SSM is used to store and retrieve secrets. Decide on the SSM path for the GitHub secret (PAT or Application private key) and GitHub webhook secret. Since the secret is automatically scoped by AWS to the account and region where the secret is stored, we recommend the secret be stored at `/github_runners/controller_github_app_secret` unless you plan on running multiple instances of the controller. If you plan on running multiple instances of the controller, and want to give them different access (otherwise they could share the same secret), then you can add a path component to the SSM path. For example `/github_runners/cicd/controller_github_app_secret`. ``` ssm_github_secret_path: "/github_runners/controller_github_app_secret" ``` The preferred way to authenticate is by _creating_ and _installing_ a GitHub App. This is the recommended approach as it allows for more much more restricted access than using a personal access token, at least until [fine-grained personal access token permissions](https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/) are generally available. Follow the instructions [here](https://github.com/actions-runner-controller/actions-runner-controller/blob/master/docs/detailed-docs.md#deploying-using-github-app-authentication) to create and install the GitHub App. At the creation stage, you will be asked to generate a private key. This is the private key that will be used to authenticate the Action Runner Controller. Download the file and store the contents in SSM using the following command, adjusting the profile and file name. The profile should be the `admin` role in the account to which you are deploying the runner controller. The file name should be the name of the private key file you downloaded. ``` AWS_PROFILE=acme-mgmt-use2-auto-admin chamber write github_runners controller_github_app_secret -- "$(cat APP_NAME.DATE.private-key.pem)" ``` You can verify the file was correctly written to SSM by matching the private key fingerprint reported by GitHub with: ``` AWS_PROFILE=acme-mgmt-use2-auto-admin chamber read -q github_runners controller_github_app_secret | openssl rsa -in - -pubout -outform DER | openssl sha256 -binary | openssl base64 ``` At this stage, record the Application ID and the private key fingerprint in your secrets manager (e.g. 1Password). You will need the Application ID to configure the runner controller, and want the fingerprint to verify the private key. Proceed to install the GitHub App in the organization or repository you want to use the runner controller for, and record the Installation ID (the final numeric part of the URL, as explained in the instructions linked above) in your secrets manager. You will need the Installation ID to configure the runner controller. In your stack configuration, set the following variables, making sure to quote the values so they are treated as strings, not numbers. ``` github_app_id: "12345" github_app_installation_id: "12345" ``` OR (obsolete) - A PAT with the scope outlined in [this document](https://github.com/actions-runner-controller/actions-runner-controller#deploying-using-pat-authentication). Save this to the value specified by `ssm_github_token_path` using the following command, adjusting the AWS_PROFILE to refer to the `admin` role in the account to which you are deploying the runner controller: ``` AWS_PROFILE=acme-mgmt-use2-auto-admin chamber write github_runners controller_github_app_secret -- "" ``` 2. If using the Webhook Driven autoscaling (recommended), generate a random string to use as the Secret when creating the webhook in GitHub. Generate the string using 1Password (no special characters, length 45) or by running ```bash dd if=/dev/random bs=1 count=33 2>/dev/null | base64 ``` Store this key in AWS SSM under the same path specified by `ssm_github_webhook_secret_token_path` ``` ssm_github_webhook_secret_token_path: "/github_runners/github_webhook_secret" ``` ### Dockerhub Authentication Authenticating with Dockerhub is optional but when enabled can ensure stability by increasing the number of pulls allowed from your runners. To get started set `docker_config_json_enabled` to `true` and `ssm_docker_config_json_path` to the SSM path where the credentials are stored, for example `github_runners/docker`. To create the credentials file, fill out a JSON file locally with the following content: ```json { "auths": { "https://index.docker.io/v1/": { "username": "your_username", "password": "your_password", "email": "your_email", "auth": "$(echo "your_username: your_password" | base64)" } } } ``` Then write the file to SSM with the following Atmos Workflow: ```yaml save/docker-config-json: description: Prompt for uploading Docker Config JSON to the AWS SSM Parameter Store steps: - type: shell command: |- echo "Please enter the Docker Config JSON file path" echo "See https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry for information on how to create the file" read -p "Docker Config JSON file path: " -r DOCKER_CONFIG_JSON_FILE_PATH if [ -z "DOCKER_CONFIG_JSON_FILE_PATH" ] then echo 'Inputs cannot be blank please try again!' exit 0 fi DOCKER_CONFIG_JSON=$(<$DOCKER_CONFIG_JSON_FILE_PATH); ENCODED_DOCKER_CONFIG_JSON=$(echo "$DOCKER_CONFIG_JSON" | base64 -w 0 ); echo $DOCKER_CONFIG_JSON echo $ENCODED_DOCKER_CONFIG_JSON AWS_PROFILE=acme-core-gbl-auto-admin set -e chamber write github_runners/docker config-json -- "$ENCODED_DOCKER_CONFIG_JSON" echo 'Saved Docker Config JSON to the AWS SSM Parameter Store' ``` Don't forget to update the AWS Profile in the script. ### Using Runner Groups GitHub supports grouping runners into distinct [Runner Groups](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups), which allow you to have different access controls for different runners. Read the linked documentation about creating and configuring Runner Groups, which you must do through the GitHub Web UI. If you choose to create Runner Groups, you can assign one or more Runner pools (from the `runners` map) to groups (only one group per runner pool) by including `group: ` in the runner configuration. We recommend including it immediately after `scope`. ### Using Webhook Driven Autoscaling (recommended) We recommend using Webhook Driven Autoscaling until GitHub's own autoscaling solution is as capable as the Summerwind solution this component deploys. See [this discussion](https://github.com/actions/actions-runner-controller/discussions/3340) for some perspective on why the Summerwind solution is currently (summer 2024) considered superior. To use the Webhook Driven Autoscaling, in addition to setting `webhook_driven_scaling_enabled` to `true`, you must also install the GitHub organization-level webhook after deploying the component (specifically, the webhook server). The URL for the webhook is determined by the `webhook.hostname_template` and where it is deployed. Recommended URL is `https://gha-webhook.[environment].[stage].[tenant].[service-discovery-domain]`. As a GitHub organization admin, go to `https://github.com/organizations/[organization]/settings/hooks`, and then: - Click"Add webhook" and create a new webhook with the following settings: - Payload URL: copy from Terraform output `webhook_payload_url` - Content type: `application/json` - Secret: whatever you configured in the `sops` secret above - Which events would you like to trigger this webhook: - Select "Let me select individual events" - Uncheck everything ("Pushes" is likely the only thing already selected) - Check "Workflow jobs" - Ensure that "Active" is checked (should be checked by default) - Click "Add webhook" at the bottom of the settings page After the webhook is created, select "edit" for the webhook and go to the "Recent Deliveries" tab and verify that there is a delivery (of a "ping" event) with a green check mark. If not, verify all the settings and consult the logs of the `actions-runner-controller-github-webhook-server` pod. ### Configuring Webhook Driven Autoscaling The `HorizontalRunnerAutoscaler scaleUpTriggers.duration` (see [Webhook Driven Scaling documentation](https://github. com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#webhook-driven-scaling)) is controlled by the `max_duration` setting for each Runner. The purpose of this timeout is to ensure, in case a job cancellation or termination event gets missed, that the resulting idle runner eventually gets terminated. #### How the Autoscaler Determines the Desired Runner Pool Size When a job is queued, a `capacityReservation` is created for it. The HRA (Horizontal Runner Autoscaler) sums up all the capacity reservations to calculate the desired size of the runner pool, subject to the limits of `minReplicas` and `maxReplicas`. The idea is that a `capacityReservation` is deleted when a job is completed or canceled, and the pool size will be equal to `jobsStarted - jobsFinished`. However, it can happen that a job will finish without the HRA being successfully notified about it, so as a safety measure, the `capacityReservation` will expire after a configurable amount of time, at which point it will be deleted without regard to the job being finished. This ensures that eventually an idle runner pool will scale down to `minReplicas`. If it happens that the capacity reservation expires before the job is finished, the Horizontal Runner Autoscaler (HRA) will scale down the pool by 2 instead of 1: once because the capacity reservation expired, and once because the job finished. This will also cause starvation of waiting jobs, because the next in line will have its timeout timer started but will not actually start running because no runner is available. And if `minReplicas` is set to zero, the pool will scale down to zero before finishing all the jobs, leaving some waiting indefinitely. This is why it is important to set the `max_duration` to a time long enough to cover the full time a job may have to wait between the time it is queued and the time it finishes, assuming that the HRA scales up the pool by 1 and runs the job on the new runner. :::tip If there are more jobs queued than there are runners allowed by `maxReplicas`, the timeout timer does not start on the capacity reservation until enough reservations ahead of it are removed for it to be considered as representing and active job. Although there are some edge cases regarding `max_duration` that seem not to be covered properly (see [actions-runner-controller issue #2466](https://github.com/actions/actions-runner-controller/issues/2466)), they only merit adding a few extra minutes to the timeout. ::: ### Recommended `max_duration` Duration #### Consequences of Too Short of a `max_duration` Duration If you set `max_duration` to too short a duration, the Horizontal Runner Autoscaler will cancel capacity reservations for jobs that have not yet finished, and the pool will become too small. This will be most serious if you have set `minReplicas = 0` because in this case, jobs will be left in the queue indefinitely. With a higher value of `minReplicas`, the pool will eventually make it through all the queued jobs, but not as quickly as intended due to the incorrectly reduced capacity. #### Consequences of Too Long of a `max_duration` Duration If the Horizontal Runner Autoscaler misses a scale-down event (which can happen because events do not have delivery guarantees), a runner may be left running idly for as long as the `max_duration` duration. The only problem with this is the added expense of leaving the idle runner running. #### Recommendation As a result, we recommend setting `max_duration` to a period long enough to cover: - The time it takes for the HRA to scale up the pool and make a new runner available - The time it takes for the runner to pick up the job from GitHub - The time it takes for the job to start running on the new runner - The maximum time a job might take Because the consequences of expiring a capacity reservation before the job is finished can be severe, we recommend setting `max_duration` to a period at least 30 minutes longer than you expect the longest job to take. Remember, when everything works properly, the HRA will scale down the pool as jobs finish, so there is little cost to setting a long duration, and the cost looks even smaller by comparison to the cost of having too short a duration. For lightly used runner pools expecting only short jobs, you can set `max_duration` to `"30m"`. As a rule of thumb, we recommend setting `maxReplicas` high enough that jobs never wait on the queue more than an hour. ### Interaction with Karpenter or other EKS autoscaling solutions Kubernetes cluster autoscaling solutions generally expect that a Pod runs a service that can be terminated on one Node and restarted on another with only a short duration needed to finish processing any in-flight requests. When the cluster is resized, the cluster autoscaler will do just that. However, GitHub Action Runner Jobs do not fit this model. If a Pod is terminated in the middle of a job, the job is lost. The likelihood of this happening is increased by the fact that the Action Runner Controller Autoscaler is expanding and contracting the size of the Runner Pool on a regular basis, causing the cluster autoscaler to more frequently want to scale up or scale down the EKS cluster, and, consequently, to move Pods around. To handle these kinds of situations, Karpenter respects an annotation on the Pod: ```yaml spec: template: metadata: annotations: karpenter.sh/do-not-disrupt: "true" ``` When you set this annotation on the Pod, Karpenter will not evict it. This means that the Pod will stay on the Node it is on, and the Node it is on will not be considered for eviction. This is good because it means that the Pod will not be terminated in the middle of a job. However, it also means that the Node the Pod is on will not be considered for termination, which means that the Node will not be removed from the cluster, which means that the cluster will not shrink in size when you would like it to. Since the Runner Pods terminate at the end of the job, this is not a problem for the Pods actually running jobs. However, if you have set `minReplicas > 0`, then you have some Pods that are just idling, waiting for jobs to be assigned to them. These Pods are exactly the kind of Pods you want terminated and moved when the cluster is underutilized. Therefore, when you set `minReplicas > 0`, you should **NOT** set `karpenter.sh/do-not-evict: "true"` on the Pod via the `pod_annotations` attribute of the `runners` input. (**But wait**, _there is good news_!) We have [requested a feature](https://github.com/actions/actions-runner-controller/issues/2562) that will allow you to set `karpenter.sh/do-not-disrupt: "true"` and `minReplicas > 0` at the same time by only annotating Pods running jobs. Meanwhile, **we have implemented this for you** using a job startup hook. This hook will set annotations on the Pod when the job starts. When the job finishes, the Pod will be deleted by the controller, so the annotations will not need to be removed. Configure annotations that apply only to Pods running jobs in the `running_pod_annotations` attribute of the `runners` input. ### Updating CRDs When updating the chart or application version of `actions-runner-controller`, it is possible you will need to install new CRDs. Such a requirement should be indicated in the `actions-runner-controller` release notes and may require some adjustment to our custom chart or configuration. This component uses `helm` to manage the deployment, and `helm` will not auto-update CRDs. If new CRDs are needed, install them manually via a command like ``` kubectl create -f https://raw.githubusercontent.com/actions-runner-controller/actions-runner-controller/master/charts/actions-runner-controller/crds/actions.summerwind.dev_horizontalrunnerautoscalers.yaml ``` ### Useful Reference Consult [actions-runner-controller](https://github.com/actions-runner-controller/actions-runner-controller) documentation for further details. ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
`resources` required
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
`runners` required
Map of Action Runner configurations, with the key being the name of the runner. Please note that the name must be in kebab-case. For example: ```hcl organization_runner = { type = "organization" # can be either 'organization' or 'repository' dind_enabled: true # A Docker daemon will be started in the runner Pod image: summerwind/actions-runner-dind # If dind_enabled=false, set this to 'summerwind/actions-runner' scope = "ACME" # org name for Organization runners, repo name for Repository runners group = "core-automation" # Optional. Assigns the runners to a runner group, for access control. scale_down_delay_seconds = 300 min_replicas = 1 max_replicas = 5 labels = [ "Ubuntu", "core-automation", ] } ``` **Type:** ```hcl map(object({ type = string scope = string group = optional(string, null) image = optional(string, "summerwind/actions-runner-dind") auto_update_enabled = optional(bool, true) dind_enabled = optional(bool, true) node_selector = optional(map(string), {}) pod_annotations = optional(map(string), {}) # running_pod_annotations are only applied to the pods once they start running a job running_pod_annotations = optional(map(string), {}) # affinity is too complex to model. Whatever you assigned affinity will be copied # to the runner Pod spec. affinity = optional(any) tolerations = optional(list(object({ key = string operator = string value = optional(string, null) effect = string })), []) scale_down_delay_seconds = optional(number, 300) min_replicas = number max_replicas = number # Scheduled overrides. See https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md#scheduled-overrides # Order is important. The earlier entry is prioritized higher than later entries. So you usually define # one-time overrides at the top of your list, then yearly, monthly, weekly, and lastly daily overrides. scheduled_overrides = optional(list(object({ start_time = string # ISO 8601 format, eg, "2021-06-01T00:00:00+09:00" end_time = string # ISO 8601 format, eg, "2021-06-01T00:00:00+09:00" min_replicas = optional(number) max_replicas = optional(number) recurrence_rule = optional(object({ frequency = string # One of Daily, Weekly, Monthly, Yearly until_time = optional(string) # ISO 8601 format time after which the schedule will no longer apply })) })), []) busy_metrics = optional(object({ scale_up_threshold = string scale_down_threshold = string scale_up_adjustment = optional(string) scale_down_adjustment = optional(string) scale_up_factor = optional(string) scale_down_factor = optional(string) })) webhook_driven_scaling_enabled = optional(bool, true) # max_duration is the duration after which a job will be considered completed, # even if the webhook has not received a "job completed" event. # This is to ensure that if an event is missed, it does not leave the runner running forever. # Set it long enough to cover the longest job you expect to run and then some. # See https://github.com/actions/actions-runner-controller/blob/9afd93065fa8b1f87296f0dcdf0c2753a0548cb7/docs/automatically-scaling-runners.md?plain=1#L264-L268 # Defaults to 1 hour programmatically (to be able to detect if both max_duration and webhook_startup_timeout are set). max_duration = optional(string) # The name `webhook_startup_timeout` was misleading and has been deprecated. # It has been renamed `max_duration`. webhook_startup_timeout = optional(string) # Adjust the time (in seconds) to wait for the Docker in Docker daemon to become responsive. wait_for_docker_seconds = optional(string, "") pull_driven_scaling_enabled = optional(bool, false) labels = optional(list(string), []) # If not null, `docker_storage` specifies the size (as `go` string) of # an ephemeral (default storage class) Persistent Volume to allocate for the Docker daemon. # Takes precedence over `tmpfs_enabled` for the Docker daemon storage. docker_storage = optional(string, null) # storage is deprecated in favor of docker_storage, since it is only storage for the Docker daemon storage = optional(string, null) # If `pvc_enabled` is true, a Persistent Volume Claim will be created for the runner # and mounted at /home/runner/work/shared. This is useful for sharing data between runners. pvc_enabled = optional(bool, false) # If `tmpfs_enabled` is `true`, both the runner and the docker daemon will use a tmpfs volume, # meaning that all data will be stored in RAM rather than on disk, bypassing disk I/O limitations, # but what would have been disk usage is now additional memory usage. You must specify memory # requests and limits when using tmpfs or else the Pod will likely crash the Node. tmpfs_enabled = optional(bool) resources = optional(object({ limits = optional(object({ cpu = optional(string, "1") memory = optional(string, "1Gi") # ephemeral-storage is the Kubernetes name, but `ephemeral_storage` is the gomplate name, # so allow either. If both are specified, `ephemeral-storage` takes precedence. ephemeral-storage = optional(string) ephemeral_storage = optional(string, "10Gi") }), {}) requests = optional(object({ cpu = optional(string, "500m") memory = optional(string, "256Mi") # ephemeral-storage is the Kubernetes name, but `ephemeral_storage` is the gomplate name, # so allow either. If both are specified, `ephemeral-storage` takes precedence. ephemeral-storage = optional(string) ephemeral_storage = optional(string, "1Gi") }), {}) }), {}) })) ```
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`context_tags_enabled` (`bool`) optional
Whether or not to include all context tags as labels for each runner **Default value:** `false`
`controller_replica_count` (`number`) optional
The number of replicas of the runner-controller to run. **Default value:** `2`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`docker_config_json_enabled` (`bool`) optional
Whether the Docker config JSON is enabled **Default value:** `false`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`existing_kubernetes_secret_name` (`string`) optional
If you are going to create the Kubernetes Secret the runner-controller will use by some means (such as SOPS) outside of this component, set the name of the secret here and it will be used. In this case, this component will not create a secret and you can leave the secret-related inputs with their default (empty) values. The same secret will be used by both the runner-controller and the webhook-server. **Default value:** `""`
`github_app_id` (`string`) optional
The ID of the GitHub App to use for the runner controller. **Default value:** `""`
`github_app_installation_id` (`string`) optional
The "Installation ID" of the GitHub App to use for the runner controller. **Default value:** `""`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`s3_bucket_arns` (`list(string)`) optional
List of ARNs of S3 Buckets to which the runners will have read-write access to. **Default value:** `[ ]`
`ssm_docker_config_json_path` (`string`) optional
SSM path to the Docker config JSON **Default value:** `null`
`ssm_github_secret_path` (`string`) optional
The path in SSM to the GitHub app private key file contents or GitHub PAT token. **Default value:** `""`
`ssm_github_webhook_secret_token_path` (`string`) optional
The path in SSM to the GitHub Webhook Secret token. **Default value:** `""`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
`webhook` optional
Configuration for the GitHub Webhook Server. `hostname_template` is the `format()` string to use to generate the hostname via `format(var.hostname_template, var.tenant, var.stage, var.environment)`" Typically something like `"echo.%[3]v.%[2]v.example.com"`. `queue_limit` is the maximum number of webhook events that can be queued up for processing by the autoscaler. When the queue gets full, webhook events will be dropped (status 500). **Type:** ```hcl object({ enabled = bool hostname_template = string queue_limit = optional(number, 1000) }) ``` **Default value:** ```hcl { "enabled": false, "hostname_template": null, "queue_limit": 1000 } ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
`metadata_action_runner_releases`
Block statuses of the deployed actions-runner chart releases
`webhook_payload_url`
Payload URL for GitHub webhook
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `actions_runner` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `actions_runner_controller` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_ssm_parameter.docker_config_json`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.github_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.github_webhook_secret_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## addon This component is responsible for installing and managing addons for EKS clusters. You may want to use this component rather than `var.addons` with `eks/cluster` to deploy addons that require additional prerequisites or configuration before they can be installed. For example, if you need to install a priority class before installing an addon, you can use this component to install the priority class first. ## Usage **Stack Level**: Regional For example, to install the CloudWatch Observability addon for EKS: ```yaml components: terraform: # https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Observability-EKS-addon.html eks/addon/cloudwatch: metadata: component: eks/addon vars: addon_name: amazon-cloudwatch-observability addon_version: "v2.5.0-eksbuild.1" kubernetes_namespace: amazon-cloudwatch # this namespace is defined by the addon resolve_conflicts_on_create: OVERWRITE resolve_conflicts_on_update: OVERWRITE priority_class_enabled: true additional_policy_arns: - "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy" ``` ## Variables ### Required Variables
`addon_name` (`string`) required
The name of the EKS addon
`region` (`string`) required
AWS Region
### Optional Variables
`additional_policy_arns` (`list(string)`) optional
Additional policy ARNs to attach to the EKS addon service account **Default value:** `[ ]`
`addon_version` (`string`) optional
The version of the EKS addon **Default value:** `null`
`configuration_values` (`map(string)`) optional
The configuration values for the EKS addon **Default value:** `{ }`
`create_timeout` (`string`) optional
The timeout to create the EKS addon **Default value:** `"15m"`
`delete_timeout` (`string`) optional
The timeout to delete the EKS addon **Default value:** `"15m"`
`eks_component_name` (`string`) optional
The name of the EKS component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
The Kubernetes namespace for the EKS addon **Default value:** `"kube-system"`
`priority_class_enabled` (`bool`) optional
Whether to enable the priority class for the EKS addon **Default value:** `false`
`resolve_conflicts_on_create` (`string`) optional
How to resolve conflicts on addon creation **Default value:** `null`
`resolve_conflicts_on_update` (`string`) optional
How to resolve conflicts on addon update **Default value:** `null`
`update_timeout` (`string`) optional
The timeout to update the EKS addon **Default value:** `"15m"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`addon_arn`
The Amazon Resource Name (ARN) of the EKS addon
`addon_version`
The version of the EKS addon
`priority_class_name`
The name of the Kubernetes priority class (if enabled)
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks_iam_role` | 2.2.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.2.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eks_addon.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) (resource) - [`aws_iam_role_policy_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`kubernetes_priority_class.this`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/priority_class) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## alb-controller This component creates a Helm release for [alb-controller](https://github.com/kubernetes-sigs/aws-load-balancer-controller) on an EKS cluster. [alb-controller](https://github.com/kubernetes-sigs/aws-load-balancer-controller) is a Kubernetes addon that, in the context of AWS, provisions and manages ALBs and NLBs based on Service and Ingress annotations. This module also can (and is recommended to) provision a default IngressClass. ### Special note about upgrading When upgrading the chart version, check to see if the IAM policy for the service account needs to be updated. If it does, update the policy in the `distributed-iam-policy.tf` file. Probably the easiest way to check if it needs updating is to simply download the policy from https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/main/docs/install/iam_policy.json and compare it to the policy in `distributed-iam-policy.tf`. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/alb-controller ... ``` The default catalog values `e.g. stacks/catalog/eks/alb-controller.yaml` ```yaml components: terraform: eks/alb-controller: vars: chart: aws-load-balancer-controller chart_repository: https://aws.github.io/eks-charts # IMPORTANT: When updating the chart version, check to see if the IAM policy for the service account. # needs to be updated, and if it does, update the policy in the `distributed-iam-policy.tf` file. chart_version: "1.7.1" create_namespace: true kubernetes_namespace: alb-controller # this feature causes inconsistent final plans # see https://github.com/hashicorp/terraform-provider-helm/issues/711#issuecomment-836192991 helm_manifest_experiment_enabled: false default_ingress_class_name: default default_ingress_group: common default_ingress_ip_address_type: ipv4 default_ingress_scheme: internet-facing # You can use `chart_values` to set any other chart options. Treat `chart_values` as the root of the doc. # # # For example # --- # chart_values: # enableShield: false chart_values: {} resources: limits: cpu: 200m memory: 256Mi requests: cpu: 100m memory: 128Mi ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
`resources` required
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`default_ingress_additional_tags` (`map(string)`) optional
Additional tags to apply to the ingress load balancer. **Default value:** `{ }`
`default_ingress_class_name` (`string`) optional
Class name for default ingress **Default value:** `"default"`
`default_ingress_enabled` (`bool`) optional
Set `true` to deploy a default IngressClass. There should only be one default per cluster. **Default value:** `true`
`default_ingress_group` (`string`) optional
Group name for default ingress **Default value:** `"common"`
`default_ingress_ip_address_type` (`string`) optional
IP address type for default ingress, one of `ipv4` or `dualstack`. **Default value:** `"ipv4"`
`default_ingress_load_balancer_attributes` (`list(object({ key = string, value = string }))`) optional
A list of load balancer attributes to apply to the default ingress load balancer. See [Load Balancer Attributes](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#load-balancer-attributes). **Default value:** `[ ]`
`default_ingress_scheme` (`string`) optional
Scheme for default ingress, one of `internet-facing` or `internal`. **Default value:** `"internet-facing"`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`vpc_component_name` (`string`) optional
The name of the vpc component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb_controller` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## alb-controller-ingress-class This component deploys a Kubernetes `IngressClass` resource for the AWS Load Balancer Controller. This is not often needed, as the default IngressClass deployed by the `eks/alb-controller` component is sufficient for most use cases, and when it is not, a service can deploy its own IngressClass. This is for the rare case where you want to deploy an additional IngressClass deploying an additional ALB that you nevertheless want to be shared by some services, with none of them explicitly owning it. ## Usage **Stack Level**: Regional ```yaml components: terraform: eks/alb-controller-ingress-class: vars: class_name: special group: special ip_address_type: ipv4 scheme: internet-facing ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`additional_tags` (`map(string)`) optional
Additional tags to apply to the ingress load balancer. **Default value:** `{ }`
`class_name` (`string`) optional
Class name for default ingress **Default value:** `"default"`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`group` (`string`) optional
Group name for default ingress **Default value:** `"common"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`ip_address_type` (`string`) optional
IP address type for default ingress, one of `ipv4` or `dualstack`. **Default value:** `"dualstack"`
`is_default` (`bool`) optional
Set `true` to make this the default IngressClass. There should only be one default per cluster. **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`load_balancer_attributes` (`list(object({ key = string, value = string }))`) optional
A list of load balancer attributes to apply to the default ingress load balancer. See [Load Balancer Attributes](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/application-load-balancers.html#load-balancer-attributes). **Default value:** `[ ]`
`scheme` (`string`) optional
Scheme for default ingress, one of `internet-facing` or `internal`. **Default value:** `"internet-facing"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_ingress_class_v1.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/ingress_class_v1) (resource) - [`kubernetes_manifest.alb_controller_class_params`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## alb-controller-ingress-group This component provisions a Kubernetes Service that creates an AWS Application Load Balancer (ALB) for a specific IngressGroup (https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/#ingressgroup). An IngressGroup is a feature of the AWS Load Balancer Controller (https://github.com/kubernetes-sigs/aws-load-balancer-controller) which allows multiple Kubernetes Ingresses to share the same Application Load Balancer. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/alb-controller-ingress-group ``` The default catalog values (e.g. `stacks/catalog/eks/alb-controller-ingress-group.yaml`) will create a Kubernetes Service in the `default` namespace with an IngressGroup (https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/#ingressgroup) named `alb-controller-ingress-group`. ```yaml components: terraform: eks/alb-controller-ingress-group: metadata: component: eks/alb-controller-ingress-group settings: spacelift: workspace_enabled: true vars: enabled: true # change the name of the Ingress Group name: alb-controller-ingress-group ``` ## Variables ### Required Variables
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
### Optional Variables
`additional_annotations` (`map(any)`) optional
Additional annotations to add to the Kubernetes ingress **Default value:** `{ }`
`alb_access_logs_enabled` (`bool`) optional
Whether or not to enable access logs for the ALB **Default value:** `false`
`alb_access_logs_s3_bucket_name` (`string`) optional
The name of the S3 bucket to store the access logs in **Default value:** `null`
`alb_access_logs_s3_bucket_prefix` (`string`) optional
The prefix to use when storing the access logs **Default value:** `"echo-server"`
`alb_group_name` (`string`) optional
The name of the alb group **Default value:** `null`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `false`
`default_annotations` (`map(any)`) optional
Default annotations to add to the Kubernetes ingress **Default value:** ```hcl { "alb.ingress.kubernetes.io/listen-ports": "[{\"HTTP\": 80}, {\"HTTPS\": 443}]", "alb.ingress.kubernetes.io/scheme": "internet-facing", "alb.ingress.kubernetes.io/ssl-policy": "ELBSecurityPolicy-TLS13-1-2-2021-06", "alb.ingress.kubernetes.io/target-type": "ip", "kubernetes.io/ingress.class": "alb" } ```
`dns_delegated_component_name` (`string`) optional
The name of the `dns_delegated` component **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
`eks_component_name` (`string`) optional
The name of the `eks` component **Default value:** `"eks/cluster"`
`fixed_response_config` (`map(any)`) optional
Configuration to overwrite the defaults such as `contentType`, `statusCode`, and `messageBody` **Default value:** `{ }`
`fixed_response_template` (`string`) optional
Fixed response template to service as a default backend **Default value:** `"resources/default-backend.html.tpl"`
`fixed_response_vars` (`map(any)`) optional
The templatefile vars to use for the fixed response template **Default value:** ```hcl { "email": "hello@cloudposse.com" } ```
`global_accelerator_component_name` (`string`) optional
The name of the `global_accelerator` component **Default value:** `"global-accelerator"`
`global_accelerator_enabled` (`bool`) optional
Whether or not Global Accelerator Endpoint Group should be provisioned for the load balancer **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes kube config file **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_service_enabled` (`bool`) optional
Whether or not to enable a default kubernetes service **Default value:** `false`
`kubernetes_service_path` (`string`) optional
The kubernetes default service's path if enabled **Default value:** `"/*"`
`kubernetes_service_port` (`number`) optional
The kubernetes default service's port if enabled **Default value:** `8080`
`waf_component_name` (`string`) optional
The name of the `waf` component **Default value:** `"waf"`
`waf_enabled` (`bool`) optional
Whether or not WAF ACL annotation should be provisioned for the load balancer **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`annotations`
The annotations of the Ingress
`group_name`
The value of `alb.ingress.kubernetes.io/group.name` of the Ingress
`host`
The name of the host used by the Ingress
`ingress_class`
The value of the `kubernetes.io/ingress.class` annotation of the Kubernetes Ingress
`load_balancer_name`
The name of the load balancer created by the Ingress
`load_balancer_scheme`
The value of the `alb.ingress.kubernetes.io/scheme` annotation of the Kubernetes Ingress
`message_body_length`
The length of the message body to ensure it's lower than the maximum limit
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `global_accelerator` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `load_balancer_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `waf` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_globalaccelerator_endpoint_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_endpoint_group) (resource) - [`kubernetes_ingress_v1.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/ingress_v1) (resource) - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) - [`kubernetes_service.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/service) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_lb.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lb) (data source) --- ## argocd This component provisions [Argo CD](https://argoproj.github.io/cd/), a declarative GitOps continuous delivery tool for Kubernetes. Note: Argo CD CRDs must be installed separately from this component/Helm release. ## Usage ### Install Argo CD CRDs Install the Argo CD CRDs prior to deploying this component: ```shell kubectl apply -k "https://github.com/argoproj/argo-cd/manifests/crds?ref=" # Eg. version v2.4.9 kubectl apply -k "https://github.com/argoproj/argo-cd/manifests/crds?ref=v2.4.9" ``` ### Preparing AppProject repos First, make sure you have a GitHub repo ready to go. We have a component for this called the `argocd-repo` component. It will create a GitHub repo and adds some secrets and code owners. Most importantly, it configures an `applicationset.yaml` that includes all the details for helm to create ArgoCD CRDs. These CRDs let ArgoCD know how to fulfill changes to its repo. ```yaml components: terraform: argocd-repo-defaults: metadata: type: abstract vars: enabled: true github_user: acme_admin github_user_email: infra@acme.com github_organization: ACME github_codeowner_teams: - "@ACME/acme-admins" - "@ACME/CloudPosse" - "@ACME/developers" gitignore_entries: - "**/.DS_Store" - ".DS_Store" - "**/.vscode" - "./vscode" - ".idea/" - ".vscode/" permissions: - team_slug: acme-admins permission: admin - team_slug: CloudPosse permission: admin - team_slug: developers permission: push ``` ### Injecting infrastructure details into applications Second, your application repos could use values to best configure their helm releases. We have an `eks/platform` component for exposing various infra outputs. It takes remote state lookups and stores them into SSM. We demonstrate how to pull the platform SSM parameters later. Here's an example `eks/platform` config: ```yaml components: terraform: eks/platform: metadata: type: abstract component: eks/platform backend: s3: workspace_key_prefix: platform deps: - catalog/eks/cluster - catalog/eks/alb-controller-ingress-group - catalog/acm vars: enabled: true name: "platform" eks_component_name: eks/cluster ssm_platform_path: /platform/%s/%s references: default_alb_ingress_group: component: eks/alb-controller-ingress-group output: .group_name default_ingress_domain: component: dns-delegated environment: gbl output: "[.zones[].name][-1]" eks/platform/acm: metadata: component: eks/platform inherits: - eks/platform vars: eks_component_name: eks/cluster references: default_ingress_domain: component: acm environment: use2 output: .domain_name eks/platform/dev: metadata: component: eks/platform inherits: - eks/platform vars: platform_environment: dev acm/qa2: settings: spacelift: workspace_enabled: true metadata: component: acm vars: enabled: true name: acm-qa2 tags: Team: sre Service: acm process_domain_validation_options: true validation_method: DNS dns_private_zone_enabled: false certificate_authority_enabled: false ``` In the previous sample we create platform settings for a `dev` platform and a `qa2` platform. Understand that these are arbitrary titles that are used to separate the SSM parameters so that if, say, a particular hostname is needed, we can safely select the right hostname using a moniker such as `qa2`. These otherwise are meaningless and do not need to align with any particular stage or tenant. ### ArgoCD on SAML / AWS Identity Center (formerly aws-sso) Here's an example snippet for how to use this component: ```yaml components: terraform: eks/argocd: settings: spacelift: workspace_enabled: true depends_on: - argocd-applicationset - tenant-gbl-corp-argocd-depoy-non-prod vars: enabled: true alb_group_name: argocd alb_name: argocd alb_logs_prefix: argocd certificate_issuer: selfsigning-issuer github_organization: MyOrg oidc_enabled: false saml_enabled: true ssm_store_account: corp ssm_store_account_region: us-west-2 argocd_repo_name: argocd-deploy-non-prod argocd_rbac_policies: - "p, role:org-admin, applications, *, */*, allow" - "p, role:org-admin, clusters, get, *, allow" - "p, role:org-admin, repositories, get, *, allow" - "p, role:org-admin, repositories, create, *, allow" - "p, role:org-admin, repositories, update, *, allow" - "p, role:org-admin, repositories, delete, *, allow" # Note: the IDs for AWS Identity Center groups will change if you alter/replace them: argocd_rbac_groups: - group: deadbeef-dead-beef-dead-beefdeadbeef role: admin - group: badca7sb-add0-65ba-dca7-sbadd065badc role: reader chart_values: global: logging: format: json level: warn sso-saml/aws-sso: settings: spacelift: workspace_enabled: true metadata: component: sso-saml-provider vars: enabled: true ssm_path_prefix: "/sso/saml/aws-sso" usernameAttr: email emailAttr: email groupsAttr: groups ``` Note, if you set up `sso-saml-provider`, you will need to restart DEX on your EKS cluster manually: ```bash kubectl delete pod -n argocd ``` The configuration above will work for AWS Identity Center if you have the following attributes in a [Custom SAML 2.0 application](https://docs.aws.amazon.com/singlesignon/latest/userguide/samlapps.html): | attribute name | value | type | | :------------- | :-------------- | :---------- | | Subject | $\{user:subject\} | persistent | | email | $\{user:email\} | unspecified | | groups | $\{user:groups\} | unspecified | You will also need to assign AWS Identity Center groups to your Custom SAML 2.0 application. Make a note of each group and replace the IDs in the `argocd_rbac_groups` var accordingly. ### Google Workspace OIDC To use Google OIDC: ```yaml oidc_enabled: true saml_enabled: false oidc_providers: google: uses_dex: true type: google id: google name: Google serviceAccountAccess: enabled: true key: googleAuth.json value: /sso/oidc/google/serviceaccount admin_email: an_actual_user@acme.com config: # This filters emails when signing in with Google to only this domain. helpful for picking the right one. hostedDomains: - acme.com clientID: /sso/saml/google/clientid clientSecret: /sso/saml/google/clientsecret ``` ### Working with ArgoCD and GitHub Here's a simple GitHub action that will trigger a deployment in ArgoCD: ```yaml # NOTE: Example will show dev, and qa2 name: argocd-deploy on: push: branches: - main jobs: ci: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v2.1.0 with: aws-region: us-east-2 role-to-assume: arn:aws:iam::123456789012:role/github-action-worker - name: Build shell: bash run: docker build -t some.docker.repo/acme/app . & docker push some.docker.repo/acmo/app - name: Checkout Argo Configuration uses: actions/checkout@v3 with: repository: acme/argocd-deploy-non-prod ref: main path: argocd-deploy-non-prod - name: Deploy to dev shell: bash run: | echo Rendering helmfile: helmfile \ --namespace acme-app \ --environment dev \ --file deploy/app/release.yaml \ --state-values-file <(aws ssm get-parameter --name /platform/dev),<(docker image inspect some.docker.repo/acme/app) \ template > argocd-deploy-non-prod/plat/use2-dev/apps/my-preview-acme-app/manifests/resources.yaml echo Updating sha for app: yq e '' -i argocd-deploy-non-prod/plat/use2-dev/apps/my-preview-acme-app/config.yaml echo Committing new helmfile pushd argocd-deploy-non-prod git add --all git commit --message 'Updating acme-app' git push popd ``` In the above example, we make a few assumptions: - You've already made the app in ArgoCD by creating a YAML file in your non-prod ArgoCD repo at the path `plat/use2-dev/apps/my-preview-acme-app/config.yaml` with contents: ```yaml app_repository: acme/app app_commit: deadbeefdeadbeef app_hostname: https://some.app.endpoint/landing_page name: my-feature-branch.acme-app namespace: my-feature-branch manifests: plat/use2-dev/apps/my-preview-acme-app/manifests ``` - you have set up `ecr` with permissions for github to push docker images to it - you already have your `ApplicationSet` and `AppProject` crd's in `plat/use2-dev/argocd/applicationset.yaml`, which should be generated by our `argocd-repo` component. - your app has a [helmfile template](https://helmfile.readthedocs.io/en/latest/#templating) in `deploy/app/release.yaml` - that helmfile template can accept both the `eks/platform` config which is pulled from ssm at the path configured in `eks/platform/defaults` - the helmfile template can update container resources using the output of `docker image inspect` ### Notifications Here's a configuration for letting argocd send notifications back to GitHub: 1. [Create GitHub PAT](https://docs.github.com/en/enterprise-server@3.6/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token) with scope `repo:status` 2. Save the PAT to SSM `/argocd/notifications/notifiers/common/github-token` 3. Use this atmos stack configuration ```yaml components: terraform: eks/argocd/notifications: metadata: component: eks/argocd vars: github_default_notifications_enabled: true ``` ### Webhook Here's a configuration Github notify ArgoCD on commit: 1. [Create GitHub PAT](https://docs.github.com/en/enterprise-server@3.6/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token) with scope `admin:repo_hook` 2. Save the PAT to SSM `/argocd/github/api_key` 3. Use this atmos stack configuration ```yaml components: terraform: eks/argocd/notifications: metadata: component: eks/argocd vars: github_webhook_enabled: true ``` #### Creating Webhooks with `github-webhook` If you are creating webhooks for ArgoCD deployment repos in multiple GitHub Organizations, you cannot use the same Terraform GitHub provider. Instead, we can use Atmos to deploy multiple component. To do this, disable the webhook creation in this component and deploy the webhook with the `github-webhook` component as such: ```yaml components: terraform: eks/argocd: metadata: component: eks/argocd inherits: - eks/argocd/defaults vars: github_webhook_enabled: true # create webhook value; required for argo-cd chart create_github_webhook: false # created with github-webhook argocd_repositories: "argocd-deploy-non-prod/org1": # this is the name of the `argocd-repo` component for "org1" environment: ue2 stage: auto tenant: core "argocd-deploy-non-prod/org2": environment: ue2 stage: auto tenant: core webhook/org1/argocd: metadata: component: github-webhook vars: github_organization: org1 github_repository: argocd-deploy-non-prod webhook_url: "https://argocd.ue2.dev.plat.acme.org/api/webhook" ssm_github_webhook: "/argocd/github/webhook" webhook/org2/argocd: metadata: component: github-webhook vars: github_organization: org2 github_repository: argocd-deploy-non-prod webhook_url: "https://argocd.ue2.dev.plat.acme.org/api/webhook" ssm_github_webhook: "/argocd/github/webhook" ``` ### Slack Notifications ArgoCD supports Slack notifications on application deployments. 1. In order to enable Slack notifications, first create a Slack Application following the [ArgoCD documentation](https://argocd-notifications.readthedocs.io/en/stable/services/slack/). 1. Create an OAuth token for the new Slack App 1. Save the OAuth token to AWS SSM Parameter Store in the same account and region as Github tokens. For example, `core-use2-auto` 1. Add the app to the chosen Slack channel. _If not added, notifications will not work_ 1. For this component, enable Slack integrations for each Application with `var.slack_notifications_enabled` and `var.slack_notifications`: ```yaml slack_notifications_enabled: true slack_notifications: channel: argocd-updates ``` 6. In the `argocd-repo` component, set `var.slack_notifications_channel` to the name of the Slack notification channel to add the relevant ApplicationSet annotations ### Troubleshooting #### Login to ArgoCD admin UI For ArgoCD v1.9 and later, the initial admin password is available from a Kubernetes secret named `argocd-initial-admin-secret`. To get the initial password, execute the following command: ```shell kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 --decode ``` Then open the ArgoCD admin UI and use the username `admin` and the password obtained in the previous step to log in to the ArgoCD admin. #### Error "server.secretkey is missing" If you provision a new version of the `eks/argocd` component, and some Helm Chart values get updated, you might encounter the error "server.secretkey is missing" in the ArgoCD admin UI. To fix the error, execute the following commands: ```shell # Download `kubeconfig` and set EKS cluster set-eks-cluster cluster-name # Restart the `argocd-server` Pods kubectl rollout restart deploy/argocd-server -n argocd # Get the new admin password from the Kubernetes secret kubectl get secret -n argocd argocd-initial-admin-secret -o jsonpath='{.data.password}' | base64 --decode ``` Reference: https://stackoverflow.com/questions/75046330/argo-cd-error-server-secretkey-is-missing ## Variables ### Required Variables
`github_organization` (`string`) required
GitHub Organization
`region` (`string`) required
AWS Region.
`ssm_store_account` (`string`) required
Account storing SSM parameters
`ssm_store_account_region` (`string`) required
AWS region storing SSM parameters
### Optional Variables
`admin_enabled` (`bool`) optional
Toggles Admin user creation the deployed chart **Default value:** `false`
`alb_group_name` (`string`) optional
A name used in annotations to reuse an ALB (e.g. `argocd`) or to generate a new one **Default value:** `null`
`alb_logs_bucket` (`string`) optional
The name of the bucket for ALB access logs. The bucket must have policy allowing the ELB logging principal **Default value:** `""`
`alb_logs_prefix` (`string`) optional
`alb_logs_bucket` s3 bucket prefix **Default value:** `""`
`alb_name` (`string`) optional
The name of the ALB (e.g. `argocd`) provisioned by `alb-controller`. Works together with `var.alb_group_name` **Default value:** `null`
`anonymous_enabled` (`bool`) optional
Toggles anonymous user access using default RBAC setting (Defaults to read-only) **Default value:** `false`
`argocd_apps_chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"argocd-apps"`
`argocd_apps_chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"A Helm chart for managing additional Argo CD Applications and Projects"`
`argocd_apps_chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://argoproj.github.io/argo-helm"`
`argocd_apps_chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values for the argocd_apps chart **Default value:** `{ }`
`argocd_apps_chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"0.0.3"`
`argocd_apps_enabled` (`bool`) optional
Enable argocd apps **Default value:** `true`
`argocd_create_namespaces` (`bool`) optional
ArgoCD create namespaces policy **Default value:** `false`
`argocd_rbac_default_policy` (`string`) optional
Default ArgoCD RBAC default role. See https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/#basic-built-in-roles for more information. **Default value:** `"role:readonly"`
`argocd_rbac_groups` optional
List of ArgoCD Group Role Assignment strings to be added to the argocd-rbac configmap policy.csv item. e.g. [ \{ group: idp-group-name, role: argocd-role-name \}, ] becomes: `g, idp-group-name, role:argocd-role-name` See https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/ for more information. **Type:** ```hcl list(object({ group = string, role = string })) ``` **Default value:** `[ ]`
`argocd_rbac_policies` (`list(string)`) optional
List of ArgoCD RBAC Permission strings to be added to the argocd-rbac configmap policy.csv item. See https://argo-cd.readthedocs.io/en/stable/operator-manual/rbac/ for more information. **Default value:** `[ ]`
`argocd_repositories` optional
Map of objects defining an `argocd_repo` to configure. The key is the name of the ArgoCD repository. **Type:** ```hcl map(object({ environment = string # The environment where the `argocd_repo` component is deployed. stage = string # The stage where the `argocd_repo` component is deployed. tenant = string # The tenant where the `argocd_repo` component is deployed. })) ``` **Default value:** `{ }`
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`certificate_issuer` (`string`) optional
Certificate manager cluster issuer **Default value:** `"letsencrypt-staging"`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"argo-cd"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://argoproj.github.io/argo-helm"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"5.55.0"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_github_webhook` (`bool`) optional
Enable GitHub webhook creation Use this to create the GitHub Webhook for the given ArgoCD repo using the value created when `var.github_webhook_enabled` is `true`. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `false`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`forecastle_enabled` (`bool`) optional
Toggles Forecastle integration in the deployed chart **Default value:** `false`
`github_app_enabled` (`bool`) optional
Whether to use GitHub App authentication instead of PAT **Default value:** `false`
`github_app_id` (`string`) optional
The ID of the GitHub App to use for authentication **Default value:** `null`
`github_app_installation_id` (`string`) optional
The Installation ID of the GitHub App to use for authentication **Default value:** `null`
`github_base_url` (`string`) optional
This is the target GitHub base API endpoint. Providing a value is a requirement when working with GitHub Enterprise. It is optional to provide this value and it can also be sourced from the `GITHUB_BASE_URL` environment variable. The value must end with a slash, for example: `https://terraformtesting-ghe.westus.cloudapp.azure.com/` **Default value:** `null`
`github_default_notifications_enabled` (`bool`) optional
Enable default GitHub commit statuses notifications (required for CD sync mode) **Default value:** `true`
`github_deploy_keys_enabled` (`bool`) optional
Enable GitHub deploy keys for the repository. These are used for Argo CD application syncing. Alternatively, you can use a GitHub App to access this desired state repository configured with `var.github_app_enabled`, `var.github_app_id`, and `var.github_app_installation_id`. **Default value:** `true`
`github_notifications_app_enabled` (`bool`) optional
Whether to use GitHub App authentication for notifications instead of PAT **Default value:** `false`
`github_notifications_app_id` (`string`) optional
The ID of the GitHub App to use for notifications authentication **Default value:** `null`
`github_notifications_app_installation_id` (`string`) optional
The Installation ID of the GitHub App to use for notifications authentication **Default value:** `null`
`github_token_override` (`string`) optional
Use the value of this variable as the GitHub token instead of reading it from SSM **Default value:** `null`
`github_webhook_enabled` (`bool`) optional
Enable GitHub webhook integration Use this to create a secret value and pass it to the argo-cd chart **Default value:** `true`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`host` (`string`) optional
Host name to use for ingress and ALB **Default value:** `""`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
The namespace to install the release into. **Default value:** `"argocd"`
`notifications_notifiers` optional
Notification Triggers to configure. See: https://argocd-notifications.readthedocs.io/en/stable/triggers/ See: [Example value in argocd-notifications Helm Chart](https://github.com/argoproj/argo-helm/blob/a0a74fb43d147073e41aadc3d88660b312d6d638/charts/argocd-notifications/values.yaml#L352) **Type:** ```hcl object({ ssm_path_prefix = optional(string, "/argocd/notifications/notifiers") # service.webhook.: webhook = optional(map( object({ url = string headers = optional(list( object({ name = string value = string }) ), []) insecureSkipVerify = optional(bool, false) }) )) }) ``` **Default value:** `{ }`
`notifications_templates` optional
Notification Templates to configure. See: https://argocd-notifications.readthedocs.io/en/stable/templates/ See: [Example value in argocd-notifications Helm Chart](https://github.com/argoproj/argo-helm/blob/a0a74fb43d147073e41aadc3d88660b312d6d638/charts/argocd-notifications/values.yaml#L158) **Type:** ```hcl map(object({ message = string alertmanager = optional(object({ labels = map(string) annotations = map(string) generatorURL = string })) webhook = optional(map( object({ method = optional(string) path = optional(string) body = optional(string) }) )) })) ``` **Default value:** `{ }`
`notifications_triggers` optional
Notification Triggers to configure. See: https://argocd-notifications.readthedocs.io/en/stable/triggers/ See: [Example value in argocd-notifications Helm Chart](https://github.com/argoproj/argo-helm/blob/a0a74fb43d147073e41aadc3d88660b312d6d638/charts/argocd-notifications/values.yaml#L352) **Type:** ```hcl map(list( object({ oncePer = optional(string) send = list(string) when = string }) )) ``` **Default value:** `{ }`
`oidc_enabled` (`bool`) optional
Toggles OIDC integration in the deployed chart **Default value:** `false`
`oidc_issuer` (`string`) optional
OIDC issuer URL **Default value:** `""`
`oidc_name` (`string`) optional
Name of the OIDC resource **Default value:** `""`
`oidc_rbac_scopes` (`string`) optional
OIDC RBAC scopes to request **Default value:** `"[argocd_realm_access]"`
`oidc_requested_scopes` (`string`) optional
Set of OIDC scopes to request **Default value:** `"[\"openid\", \"profile\", \"email\", \"groups\"]"`
`rbac_enabled` (`bool`) optional
Enable Service Account for pods. **Default value:** `true`
`resources` optional
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ``` **Default value:** `null`
`saml_enabled` (`bool`) optional
Toggles SAML integration in the deployed chart **Default value:** `false`
`saml_rbac_scopes` (`string`) optional
SAML RBAC scopes to request **Default value:** `"[email,groups]"`
`saml_sso_providers` optional
SAML SSO providers components **Type:** ```hcl map(object({ component = string environment = optional(string, null) })) ``` **Default value:** `{ }`
`service_type` (`string`) optional
Service type for exposing the ArgoCD service. The available type values and their behaviors are: ClusterIP: Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). LoadBalancer: Exposes the Service externally using a cloud provider's load balancer. **Default value:** `"NodePort"`
`slack_notifications` optional
ArgoCD Slack notification configuration. Requires Slack Bot created with token stored at the given SSM Parameter path. See: https://argocd-notifications.readthedocs.io/en/stable/services/slack/ **Type:** ```hcl object({ token_ssm_path = optional(string, "/argocd/notifications/notifiers/slack/token") api_url = optional(string, null) username = optional(string, "ArgoCD") icon = optional(string, null) }) ``` **Default value:** `{ }`
`slack_notifications_enabled` (`bool`) optional
Whether or not to enable Slack notifications. See `var.slack_notifications. **Default value:** `false`
`ssm_github_api_key` (`string`) optional
SSM path to the GitHub API key **Default value:** `"/argocd/github/api_key"`
`ssm_github_app_private_key` (`string`) optional
SSM path to the GitHub App private key **Default value:** `"/argocd/github/app_private_key"`
`ssm_github_notifications_app_private_key` (`string`) optional
SSM path to the GitHub App private key for notifications **Default value:** `"/argocd/github_notifications/app_private_key"`
`ssm_oidc_client_id` (`string`) optional
The SSM Parameter Store path for the ID of the IdP client **Default value:** `"/argocd/oidc/client_id"`
`ssm_oidc_client_secret` (`string`) optional
The SSM Parameter Store path for the secret of the IdP client **Default value:** `"/argocd/oidc/client_secret"`
`ssm_store_account_tenant` (`string`) optional
Tenant of the account storing SSM parameters. If the tenant label is not used, leave this as null. **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `300`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`github_webhook_value`
The value of the GitHub webhook secret used for ArgoCD
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 4.0` - `helm`, version: `>= 2.6.0, < 3.0.0` - `kubernetes`, version: `>= 2.9.0, != 2.21.0` - `random`, version: `>= 3.5` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 4.0` - `kubernetes`, version: `>= 2.9.0, != 2.21.0` - `random`, version: `>= 3.5` ### Modules Name | Version | Source | Description --- | --- | --- | --- `argocd` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `argocd_apps` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `argocd_repo` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `iam_roles_config_secrets` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `notifications_notifiers` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `notifications_templates` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `saml_sso_providers` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`github_repository_webhook.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_webhook) (resource) - [`random_password.webhook`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_ssm_parameter.github_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.github_app_private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.github_deploy_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.github_notifications_app_private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.oidc_client_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.oidc_client_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.slack_notifications`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameters_by_path.argocd_notifications`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameters_by_path) (data source) - [`kubernetes_resources.crd`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/resources) (data source) --- ## cert-manager This component creates a Helm release for [cert-manager](https://github.com/jetstack/cert-manager) on a Kubernetes cluster. [cert-manager](https://github.com/jetstack/cert-manager) is a Kubernetes addon that provisions X.509 certificates. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/cert-manager ... ``` The default catalog values `e.g. stacks/catalog/eks/cert-manager.yaml` ```yaml enabled: true name: cert-manager kubernetes_namespace: cert-manager # `helm_manifest_experiment_enabled` does not work with cert-manager or any Helm chart that uses CRDs helm_manifest_experiment_enabled: false # Use the cert-manager as a private CA (Certificate Authority) # to issue certificates for use within the Kubernetes cluster. # Something like this is required for the ALB Ingress Controller. cert_manager_issuer_selfsigned_enabled: true # Use Let's Encrypt to issue certificates for use outside the Kubernetes cluster, # ones that will be trusted by browsers. # These do not (yet) work with the ALB Ingress Controller, # which require ACM certificates, so we have no use for them. letsencrypt_enabled: true # cert_manager_issuer_support_email_template is only used if letsencrypt_enabled is true. # If it were true, we would want to set it at the organization level. cert_manager_issuer_support_email_template: "aws+%s@acme.com" cert_manager_repository: https://charts.jetstack.io cert_manager_chart: cert-manager cert_manager_chart_version: v1.5.4 # use a local chart to provision Certificate Issuers cert_manager_issuer_chart: ./cert-manager-issuer/ cert_manager_resources: limits: cpu: 200m memory: 256Mi requests: cpu: 100m memory: 128Mi ``` ## Variables ### Required Variables
`cert_manager_issuer_support_email_template` (`string`) required
The support email template format.
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If `true`, if any part of the installation process fails, all parts are treated as failed. Highly recommended to prevent cert-manager from getting into a wedged state. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`cart_manager_rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`cert_manager_chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"cert-manager"`
`cert_manager_chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cert_manager_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`cert_manager_issuer_chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"./cert-manager-issuer/"`
`cert_manager_issuer_chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cert_manager_issuer_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`cert_manager_issuer_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `null`
`cert_manager_issuer_selfsigned_enabled` (`bool`) optional
Whether or not to use selfsigned issuer. **Default value:** `true`
`cert_manager_issuer_values` (`any`) optional
Additional values to yamlencode as `helm_release` values for cert-manager-issuer. **Default value:** `{ }`
`cert_manager_metrics_enabled` (`bool`) optional
Whether or not to enable metrics for cert-manager. **Default value:** `false`
`cert_manager_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://charts.jetstack.io"`
`cert_manager_resources` optional
The cpu and memory of the cert manager's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ``` **Default value:** ```hcl { "limits": { "cpu": "200m", "memory": "256Mi" }, "requests": { "cpu": "100m", "memory": "128Mi" } } ```
`cert_manager_values` (`any`) optional
Additional values to yamlencode as `helm_release` values for cert-manager. **Default value:** `{ }`
`cleanup_on_fail` (`bool`) optional
If `true`, resources created in this deploy will be deleted when deploy fails. Highly recommended to prevent cert-manager from getting into a wedeged state. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `true`. **Default value:** `true`
`dns_gbl_delegated_environment_name` (`string`) optional
The name of the environment where global `dns_delegated` is provisioned **Default value:** `"gbl"`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
The namespace to install the release into. **Default value:** `"cert-manager"`
`letsencrypt_enabled` (`bool`) optional
Whether or not to use letsencrypt issuer and manager. If this is enabled, it will also provision an IAM role. **Default value:** `false`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Set `true` to wait until all resources are in a ready state before marking the release as successful. Ignored if provisioning Issuers. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cert_manager_issuer_metadata`
Block status of the deployed release
`cert_manager_metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cert_manager` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `cert_manager_issuer` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## cloudwatch This component installs the CloudWatch Observability chart for EKS. You may want to use this chart rather than the addon if you need to install a priority class with the CloudWatch Observability chart. The addon at this time does not support priority classes with configuration (see References for details). ## Usage **Stack Level**: Regional For example, to install the CloudWatch Observability chart for EKS: ```yaml components: terraform: eks/cloudwatch: vars: name: eks-cloudwatch # We need to create a priority class for the CloudWatch agent to use # to ensure the cloudwatch-agent and fluent-bit pods are scheduled on all nodes priority_class_enabled: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used **Default value:** `true`
`chart` (`string`) optional
The Helm chart to install **Default value:** `"amazon-cloudwatch-observability"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history) **Default value:** `"Amazon CloudWatch Observability for EKS"`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://aws-observability.github.io/helm-charts"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values **Default value:** `{ }`
`chart_version` (`string`) optional
The version of the Helm chart to install **Default value:** `"v3.0.0"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the EKS component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Name of the Kubernetes Namespace this pod is deployed in to **Default value:** `"amazon-cloudwatch"`
`priority_class_enabled` (`bool`) optional
Whether to enable the priority class for the EKS addon **Default value:** `false`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks) **Default value:** `900`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudwatch_log_group_names`
List of CloudWatch log group names created by the agent
`metadata`
Block status of the deployed release
`priority_class_name`
Name of the priority class
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `local` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_role_policy_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`kubernetes_priority_class.this`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/priority_class) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## cluster This component is responsible for provisioning an end-to-end EKS Cluster, including managed node groups and Fargate profiles. :::note #### Windows not supported This component has not been tested with Windows worker nodes of any launch type. Although upstream modules support Windows nodes, there are likely issues around incorrect or insufficient IAM permissions or other configuration that would need to be resolved for this component to properly configure the upstream modules for Windows nodes. If you need Windows nodes, please experiment and be on the lookout for issues, and then report any issues to Cloud Posse. ::: ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. This example expects the [Cloud Posse Reference Architecture](https://docs.cloudposse.com/) Identity and Network designs deployed for mapping users to EKS service roles and granting access in a private network. In addition, this example has the GitHub OIDC integration added and makes use of Karpenter to dynamically scale cluster nodes. For more on these requirements, see [Identity Reference Architecture](https://docs.cloudposse.com/layers/identity/), [Network Reference Architecture](https://docs.cloudposse.com/layers/network/), the [GitHub OIDC component](https://docs.cloudposse.com/components/library/aws/github-oidc-provider/), and the [Karpenter component](https://docs.cloudposse.com/components/library/aws/eks/karpenter/). ### Mixin pattern for Kubernetes version We recommend separating out the Kubernetes and related addons versions into a separate mixin (one per Kubernetes minor version), to make it easier to run different versions in different environments, for example while testing a new version. We also recommend leaving "resolve conflicts" settings unset and therefore using the default "OVERWRITE" setting because any custom configuration that you would want to preserve should be managed by Terraform configuring the add-ons directly. For example, create `catalog/eks/cluster/mixins/k8s-1-29.yaml` with the following content: ```yaml components: terraform: eks/cluster: vars: cluster_kubernetes_version: "1.29" # You can set all the add-on versions to `null` to use the latest version, # but that introduces drift as new versions are released. As usual, we recommend # pinning the versions to a specific version and upgrading when convenient. # Determine the latest version of the EKS add-ons for the specified Kubernetes version # EKS_K8S_VERSION=1.29 # replace with your cluster version # ADD_ON=vpc-cni # replace with the add-on name # echo "${ADD_ON}:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name $ADD_ON \ # --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table # To see versions for all the add-ons, wrap the above command in a for loop: # for ADD_ON in vpc-cni kube-proxy coredns aws-ebs-csi-driver aws-efs-csi-driver; do # echo "${ADD_ON}:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name $ADD_ON \ # --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table # done # To see the custom configuration schema for an add-on, run the following command: # aws eks describe-addon-configuration --addon-name aws-ebs-csi-driver \ # --addon-version v1.20.0-eksbuild.1 | jq '.configurationSchema | fromjson' # See the `coredns` configuration below for an example of how to set a custom configuration. # https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html # https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html#creating-an-add-on addons: # https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html # https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html # https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html#cni-iam-role-create-role # https://aws.github.io/aws-eks-best-practices/networking/vpc-cni/#deploy-vpc-cni-managed-add-on vpc-cni: addon_version: "v1.16.0-eksbuild.1" # set `addon_version` to `null` to use the latest version # https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html kube-proxy: addon_version: "v1.29.0-eksbuild.1" # set `addon_version` to `null` to use the latest version # https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html coredns: addon_version: "v1.11.1-eksbuild.4" # set `addon_version` to `null` to use the latest version ## override default replica count of 2. In very large clusters, you may want to increase this. configuration_values: '{"replicaCount": 3}' # https://docs.aws.amazon.com/eks/latest/userguide/csi-iam-role.html # https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons # https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#csi-iam-role # https://github.com/kubernetes-sigs/aws-ebs-csi-driver aws-ebs-csi-driver: addon_version: "v1.27.0-eksbuild.1" # set `addon_version` to `null` to use the latest version # If you are not using [volume snapshots](https://kubernetes.io/blog/2020/12/10/kubernetes-1.20-volume-snapshot-moves-to-ga/#how-to-use-volume-snapshots) # (and you probably are not), disable the EBS Snapshotter # See https://github.com/aws/containers-roadmap/issues/1919 configuration_values: '{"sidecars":{"snapshotter":{"forceEnable":false}}}' aws-efs-csi-driver: addon_version: "v1.7.7-eksbuild.1" # set `addon_version` to `null` to use the latest version # Set a short timeout in case of conflict with an existing efs-controller deployment create_timeout: "7m" ``` ### Common settings for all Kubernetes versions In your main stack configuration, you can then set the Kubernetes version by importing the appropriate mixin: ```yaml # import: - catalog/eks/cluster/mixins/k8s-1-29 components: terraform: eks/cluster: vars: enabled: true name: eks vpc_component_name: "vpc" eks_component_name: "eks/cluster" # Your choice of availability zones or availability zone ids # availability_zones: ["us-east-1a", "us-east-1b", "us-east-1c"] aws_ssm_agent_enabled: true allow_ingress_from_vpc_accounts: - tenant: core stage: auto - tenant: core stage: corp - tenant: core stage: network public_access_cidrs: [] allowed_cidr_blocks: [] allowed_security_groups: [] enabled_cluster_log_types: # Caution: enabling `api` log events may lead to a substantial increase in Cloudwatch Logs expenses. - api - audit - authenticator - controllerManager - scheduler oidc_provider_enabled: true # Allows GitHub OIDC role github_actions_iam_role_enabled: true github_actions_iam_role_attributes: ["eks"] github_actions_allowed_repos: - acme/infra # We recommend, at a minimum, deploying 1 managed node group, # with the same number of instances as availability zones (typically 3). managed_node_groups_enabled: true node_groups: # for most attributes, setting null here means use setting from node_group_defaults main: # availability_zones = null will create one autoscaling group # in every private subnet in the VPC availability_zones: null # Tune the desired and minimum group size according to your baseload requirements. # We recommend no autoscaling for the main node group, so it will # stay at the specified desired group size, with additional # capacity provided by Karpenter. Nevertheless, we recommend # deploying enough capacity in the node group to handle your # baseload requirements, and in production, we recommend you # have a large enough node group to handle 3/2 (1.5) times your # baseload requirements, to handle the loss of a single AZ. desired_group_size: 3 # number of instances to start with, should be >= number of AZs min_group_size: 3 # must be >= number of AZs max_group_size: 3 # Can only set one of ami_release_version or kubernetes_version # Leave both null to use latest AMI for Cluster Kubernetes version kubernetes_version: null # use cluster Kubernetes version ami_release_version: null # use latest AMI for Kubernetes version attributes: [] create_before_destroy: true cluster_autoscaler_enabled: true instance_types: # Tune the instance type according to your baseload requirements. - c7a.medium ami_type: AL2_x86_64 # use "AL2_x86_64" for standard instances, "AL2_x86_64_GPU" for GPU instances node_userdata: # WARNING: node_userdata is alpha status and will likely change in the future. # Also, it is only supported for AL2 and some Windows AMIs, not BottleRocket or AL2023. # Kubernetes docs: https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/ kubelet_extra_args: >- --kube-reserved cpu=100m,memory=0.6Gi,ephemeral-storage=1Gi --system-reserved cpu=100m,memory=0.2Gi,ephemeral-storage=1Gi --eviction-hard memory.available<200Mi,nodefs.available<10%,imagefs.available<15% block_device_map: # EBS volume for local ephemeral storage # IGNORED if legacy `disk_encryption_enabled` or `disk_size` are set! # Use "/dev/xvda" for most of the instances (without local NVMe) # using most of the Linuxes, "/dev/xvdb" for BottleRocket "/dev/xvda": ebs: volume_size: 100 # number of GB volume_type: gp3 kubernetes_labels: {} kubernetes_taints: {} resources_to_tag: - instance - volume tags: null # The abbreviation method used for Availability Zones in your project. # Used for naming resources in managed node groups. # Either "short" or "fixed". availability_zone_abbreviation_type: fixed cluster_private_subnets_only: true cluster_encryption_config_enabled: true cluster_endpoint_private_access: true cluster_endpoint_public_access: false cluster_log_retention_period: 90 # List of `aws-team-roles` (in the account where the EKS cluster is deployed) to map to Kubernetes RBAC groups # You cannot set `system:*` groups here, except for `system:masters`. # The `idp:*` roles referenced here are created by the `eks/idp-roles` component. # While set here, the `idp:*` roles will have no effect until after # the `eks/idp-roles` component is applied, which must be after the # `eks/cluster` component is deployed. aws_team_roles_rbac: - aws_team_role: admin groups: - system:masters - aws_team_role: poweruser groups: - idp:poweruser - aws_team_role: observer groups: - idp:observer - aws_team_role: planner groups: - idp:observer - aws_team_role: terraform groups: - system:masters # Permission sets from AWS SSO allowing cluster access # See `aws-sso` component. aws_sso_permission_sets_rbac: - aws_sso_permission_set: PowerUserAccess groups: - idp:poweruser # Set to false if you are not using Karpenter karpenter_iam_role_enabled: true # All Fargate Profiles will use the same IAM Role when `legacy_fargate_1_role_per_profile_enabled` is set to false. # Recommended for all new clusters, but will damage existing clusters provisioned with the legacy component. legacy_fargate_1_role_per_profile_enabled: false # While it is possible to deploy add-ons to Fargate Profiles, it is not recommended. Use a managed node group instead. deploy_addons_to_fargate: false ``` ### Amazon EKS End-of-Life Dates When picking a Kubernetes version, be sure to review the [end-of-life dates for Amazon EKS](https://endoflife.date/amazon-eks). Refer to the chart below: | cycle | release | latest | latest release | eol | extended support | | :---- | :--------: | :---------- | :------------: | :--------: | :--------------: | | 1.29 | 2024-01-23 | 1.29-eks-6 | 2024-04-18 | 2025-03-23 | 2026-03-23 | | 1.28 | 2023-09-26 | 1.28-eks-12 | 2024-04-18 | 2024-11-26 | 2025-11-26 | | 1.27 | 2023-05-24 | 1.27-eks-16 | 2024-04-18 | 2024-07-24 | 2025-07-24 | | 1.26 | 2023-04-11 | 1.26-eks-17 | 2024-04-18 | 2024-06-11 | 2025-06-11 | | 1.25 | 2023-02-21 | 1.25-eks-18 | 2024-04-18 | 2024-05-01 | 2025-05-01 | | 1.24 | 2022-11-15 | 1.24-eks-21 | 2024-04-18 | 2024-01-31 | 2025-01-31 | | 1.23 | 2022-08-11 | 1.23-eks-23 | 2024-04-18 | 2023-10-11 | 2024-10-11 | | 1.22 | 2022-04-04 | 1.22-eks-14 | 2023-06-30 | 2023-06-04 | 2024-09-01 | | 1.21 | 2021-07-19 | 1.21-eks-18 | 2023-06-09 | 2023-02-16 | 2024-07-15 | | 1.20 | 2021-05-18 | 1.20-eks-14 | 2023-05-05 | 2022-11-01 | False | | 1.19 | 2021-02-16 | 1.19-eks-11 | 2022-08-15 | 2022-08-01 | False | | 1.18 | 2020-10-13 | 1.18-eks-13 | 2022-08-15 | 2022-08-15 | False | \* This Chart was generated 2024-05-12 with [the `eol` tool](https://github.com/hugovk/norwegianblue). Install it with `python3 -m pip install --upgrade norwegianblue` and create a new table by running `eol --md amazon-eks` locally, or view the information by visiting [the endoflife website](https://endoflife.date/amazon-eks). You can also view the release and support timeline for [the Kubernetes project itself](https://endoflife.date/kubernetes). ### Using Addons EKS clusters support “Addons” that can be automatically installed on a cluster. Install these addons with the [`var.addons` input](https://docs.cloudposse.com/components/library/aws/eks/cluster/#input_addons). :::tip Run the following command to see all available addons, their type, and their publisher. You can also see the URL for addons that are available through the AWS Marketplace. Replace 1.27 with the version of your cluster. See [Creating an addon](https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html#creating-an-add-on) for more details. ::: ```shell EKS_K8S_VERSION=1.29 # replace with your cluster version aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION \ --query 'addons[].{MarketplaceProductUrl: marketplaceInformation.productUrl, Name: addonName, Owner: owner Publisher: publisher, Type: type}' --output table ``` :::tip You can see which versions are available for each addon by executing the following commands. Replace 1.29 with the version of your cluster. ::: ```shell EKS_K8S_VERSION=1.29 # replace with your cluster version echo "vpc-cni:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name vpc-cni \ --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table echo "kube-proxy:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name kube-proxy \ --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table echo "coredns:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name coredns \ --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table echo "aws-ebs-csi-driver:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name aws-ebs-csi-driver \ --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table echo "aws-efs-csi-driver:" && aws eks describe-addon-versions --kubernetes-version $EKS_K8S_VERSION --addon-name aws-efs-csi-driver \ --query 'addons[].addonVersions[].{Version: addonVersion, Defaultversion: compatibilities[0].defaultVersion}' --output table ``` Some add-ons accept additional configuration. For example, the `vpc-cni` addon accepts a `disableNetworking` parameter. View the available configuration options (as JSON Schema) via the `aws eks describe-addon-configuration` command. For example: ```shell aws eks describe-addon-configuration \ --addon-name aws-ebs-csi-driver \ --addon-version v1.20.0-eksbuild.1 | jq '.configurationSchema | fromjson' ``` You can then configure the add-on via the `configuration_values` input. For example: ```yaml aws-ebs-csi-driver: configuration_values: '{"node": {"loggingFormat": "json"}}' ``` Configure the addons like the following example: ```yaml # https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html # https://docs.aws.amazon.com/eks/latest/userguide/managing-add-ons.html#creating-an-add-on # https://aws.amazon.com/blogs/containers/amazon-eks-add-ons-advanced-configuration/ addons: # https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html # https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html # https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html#cni-iam-role-create-role # https://aws.github.io/aws-eks-best-practices/networking/vpc-cni/#deploy-vpc-cni-managed-add-on vpc-cni: addon_version: "v1.12.2-eksbuild.1" # set `addon_version` to `null` to use the latest version # https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html kube-proxy: addon_version: "v1.25.6-eksbuild.1" # set `addon_version` to `null` to use the latest version # https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html coredns: addon_version: "v1.9.3-eksbuild.2" # set `addon_version` to `null` to use the latest version # Override default replica count of 2, to have one in each AZ configuration_values: '{"replicaCount": 3}' # https://docs.aws.amazon.com/eks/latest/userguide/csi-iam-role.html # https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons # https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#csi-iam-role # https://github.com/kubernetes-sigs/aws-ebs-csi-driver aws-ebs-csi-driver: addon_version: "v1.19.0-eksbuild.2" # set `addon_version` to `null` to use the latest version # If you are not using [volume snapshots](https://kubernetes.io/blog/2020/12/10/kubernetes-1.20-volume-snapshot-moves-to-ga/#how-to-use-volume-snapshots) # (and you probably are not), disable the EBS Snapshotter with: configuration_values: '{"sidecars":{"snapshotter":{"forceEnable":false}}}' ``` Some addons, such as CoreDNS, require at least one node to be fully provisioned first. See [issue #170](https://github.com/cloudposse/terraform-aws-eks-cluster/issues/170) for more details. Set `var.addons_depends_on` to `true` to require the Node Groups to be provisioned before addons. ```yaml addons_depends_on: true addons: coredns: addon_version: "v1.8.7-eksbuild.1" ``` :::warning Addons may not be suitable for all use-cases! For example, if you are deploying Karpenter to Fargate and using Karpenter to provision all nodes, these nodes will never be available before the cluster component is deployed if you are using the CoreDNS addon (for example). This is one of the reasons we recommend deploying a managed node group: to ensure that the addons will become fully functional during deployment of the cluster. ::: For more information on upgrading EKS Addons, see ["How to Upgrade EKS Cluster Addons"](https://docs.cloudposse.com/learn/maintenance/upgrades/how-to-upgrade-eks-cluster-addons/) ### Adding and Configuring a new EKS Addon The component already supports all the EKS addons shown in the configurations above. To add a new EKS addon, not supported by the cluster, add it to the `addons` map (`addons` variable): ```yaml addons: my-addon: addon_version: "..." ``` If the new addon requires an EKS IAM Role for Kubernetes Service Account, perform the following steps: - Add a file `addons-custom.tf` to the `eks/cluster` folder if not already present - In the file, add an IAM policy document with the permissions required for the addon, and use the `eks-iam-role` module to provision an IAM Role for Kubernetes Service Account for the addon: ```hcl data "aws_iam_policy_document" "my_addon" { statement { sid = "..." effect = "Allow" resources = ["..."] actions = [ "...", "..." ] } } module "my_addon_eks_iam_role" { source = "cloudposse/eks-iam-role/aws" version = "2.1.0" eks_cluster_oidc_issuer_url = local.eks_cluster_oidc_issuer_url service_account_name = "..." service_account_namespace = "..." aws_iam_policy_document = [one(data.aws_iam_policy_document.my_addon[*].json)] context = module.this.context } ``` For examples of how to configure the IAM role and IAM permissions for EKS addons, see [addons.tf](https://github.com/cloudposse-terraform-components/aws-eks-cluster/tree/main/cluster/addons.tf). - Add a file `additional-addon-support_override.tf` to the `eks/cluster` folder if not already present - In the file, add the IAM Role for Kubernetes Service Account for the addon to the `overridable_additional_addon_service_account_role_arn_map` map: ```hcl locals { overridable_additional_addon_service_account_role_arn_map = { my-addon = module.my_addon_eks_iam_role.service_account_role_arn } } ``` - This map will override the default map in the [additional-addon-support.tf](https://github.com/cloudposse-terraform-components/aws-eks-cluster/tree/main/cluster/additional-addon-support.tf) file, and will be merged into the final map together with the default EKS addons `vpc-cni` and `aws-ebs-csi-driver` (which this component configures and creates IAM Roles for Kubernetes Service Accounts) - Follow the instructions in the [additional-addon-support.tf](https://github.com/cloudposse-terraform-components/aws-eks-cluster/tree/main/cluster/additional-addon-support.tf) file if the addon may need to be deployed to Fargate, or has dependencies that Terraform cannot detect automatically. ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_config` optional
Access configuration for the EKS cluster **Type:** ```hcl object({ authentication_mode = optional(string, "API") bootstrap_cluster_creator_admin_permissions = optional(bool, false) }) ``` **Default value:** `{ }`
`addons` optional
Manages [EKS addons](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) resources **Type:** ```hcl map(object({ enabled = optional(bool, true) addon_version = optional(string, null) # configuration_values is a JSON string, such as '{"computeType": "Fargate"}'. configuration_values = optional(string, null) # Set default resolve_conflicts to OVERWRITE because it is required on initial installation of # add-ons that have self-managed versions installed by default (e.g. vpc-cni, coredns), and # because any custom configuration that you would want to preserve should be managed by Terraform. resolve_conflicts_on_create = optional(string, "OVERWRITE") resolve_conflicts_on_update = optional(string, "OVERWRITE") service_account_role_arn = optional(string, null) create_timeout = optional(string, null) update_timeout = optional(string, null) delete_timeout = optional(string, null) })) ``` **Default value:** `{ }`
`addons_depends_on` (`bool`) optional
If set `true` (recommended), all addons will depend on managed node groups provisioned by this component and therefore not be installed until nodes are provisioned. See [issue #170](https://github.com/cloudposse/terraform-aws-eks-cluster/issues/170) for more details. **Default value:** `true`
`allow_ingress_from_vpc_accounts` (`any`) optional
List of account contexts to pull VPC ingress CIDR and add to cluster security group. e.g. \{ environment = "ue2", stage = "auto", tenant = "core" \} **Default value:** `[ ]`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the EKS cluster **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
List of Security Group IDs to be allowed to connect to the EKS cluster **Default value:** `[ ]`
`availability_zone_abbreviation_type` (`string`) optional
Type of Availability Zone abbreviation (either `fixed` or `short`) to use in names. See https://github.com/cloudposse/terraform-aws-utils for details. **Default value:** `"fixed"`
`availability_zone_ids` (`list(string)`) optional
List of Availability Zones IDs where subnets will be created. Overrides `availability_zones`. Can be the full name, e.g. `use1-az1`, or just the part after the AZ ID region code, e.g. `-az1`, to allow reusable values across regions. Consider contention for resources and spot pricing in each AZ when selecting. Useful in some regions when using only some AZs and you want to use the same ones across multiple accounts. **Default value:** `[ ]`
`availability_zones` (`list(string)`) optional
AWS Availability Zones in which to deploy multi-AZ resources. Ignored if `availability_zone_ids` is set. Can be the full name, e.g. `us-east-1a`, or just the part after the region, e.g. `a` to allow reusable values across regions. If not provided, resources will be provisioned in every zone with a private subnet in the VPC. **Default value:** `[ ]`
`aws_ssm_agent_enabled` (`bool`) optional
Set true to attach the required IAM policy for AWS SSM agent to each EC2 instance's IAM Role **Default value:** `false`
`aws_sso_permission_sets_rbac` optional
(Not Recommended): AWS SSO (IAM Identity Center) permission sets in the EKS deployment account to add to `aws-auth` ConfigMap. Unfortunately, `aws-auth` ConfigMap does not support SSO permission sets, so we map the generated IAM Role ARN corresponding to the permission set at the time Terraform runs. This is subject to change when any changes are made to the AWS SSO configuration, invalidating the mapping, and requiring a `terraform apply` in this project to update the `aws-auth` ConfigMap and restore access. **Type:** ```hcl list(object({ aws_sso_permission_set = string groups = list(string) })) ``` **Default value:** `[ ]`
`aws_team_roles_rbac` optional
List of `aws-team-roles` (in the target AWS account) to map to Kubernetes RBAC groups. **Type:** ```hcl list(object({ aws_team_role = string groups = list(string) })) ``` **Default value:** `[ ]`
`cluster_encryption_config_enabled` (`bool`) optional
Set to `true` to enable Cluster Encryption Configuration **Default value:** `true`
`cluster_encryption_config_kms_key_deletion_window_in_days` (`number`) optional
Cluster Encryption Config KMS Key Resource argument - key deletion windows in days post destruction **Default value:** `10`
`cluster_encryption_config_kms_key_enable_key_rotation` (`bool`) optional
Cluster Encryption Config KMS Key Resource argument - enable kms key rotation **Default value:** `true`
`cluster_encryption_config_kms_key_id` (`string`) optional
KMS Key ID to use for cluster encryption config **Default value:** `""`
`cluster_encryption_config_kms_key_policy` (`string`) optional
Cluster Encryption Config KMS Key Resource argument - key policy **Default value:** `null`
`cluster_encryption_config_resources` (`list(string)`) optional
Cluster Encryption Config Resources to encrypt, e.g. `["secrets"]` **Default value:** ```hcl [ "secrets" ] ```
`cluster_endpoint_private_access` (`bool`) optional
Indicates whether or not the Amazon EKS private API server endpoint is enabled. Default to AWS EKS resource and it is `false` **Default value:** `false`
`cluster_endpoint_public_access` (`bool`) optional
Indicates whether or not the Amazon EKS public API server endpoint is enabled. Default to AWS EKS resource and it is `true` **Default value:** `true`
`cluster_kubernetes_version` (`string`) optional
Desired Kubernetes master version. If you do not specify a value, the latest available version is used **Default value:** `null`
`cluster_log_retention_period` (`number`) optional
Number of days to retain cluster logs. Requires `enabled_cluster_log_types` to be set. See https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. **Default value:** `0`
`cluster_private_subnets_only` (`bool`) optional
Whether or not to enable private subnets or both public and private subnets **Default value:** `false`
`color` (`string`) optional
The cluster stage represented by a color; e.g. blue, green **Default value:** `""`
`deploy_addons_to_fargate` (`bool`) optional
Set to `true` (not recommended) to deploy addons to Fargate instead of initial node pool **Default value:** `false`
`enabled_cluster_log_types` (`list(string)`) optional
A list of the desired control plane logging to enable. For more information, see https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. Possible values [`api`, `audit`, `authenticator`, `controllerManager`, `scheduler`] **Default value:** `[ ]`
`fargate_profile_iam_role_kubernetes_namespace_delimiter` (`string`) optional
Delimiter for the Kubernetes namespace in the IAM Role name for Fargate Profiles **Default value:** `"-"`
`fargate_profile_iam_role_permissions_boundary` (`string`) optional
If provided, all Fargate Profiles IAM roles will be created with this permissions boundary attached **Default value:** `null`
`fargate_profiles` optional
Fargate Profiles config **Type:** ```hcl map(object({ kubernetes_namespace = string kubernetes_labels = map(string) })) ``` **Default value:** `{ }`
`karpenter_iam_role_enabled` (`bool`) optional
Flag to enable/disable creation of IAM role for EC2 Instance Profile that is attached to the nodes launched by Karpenter **Default value:** `false`
`legacy_do_not_create_karpenter_instance_profile` (`bool`) optional
**Obsolete:** The issues this was meant to mitigate were fixed in AWS Terraform Provider v5.43.0 and Karpenter v0.33.0. This variable will be removed in a future release. Remove this input from your configuration and leave it at default. **Old description:** When `true` (the default), suppresses creation of the IAM Instance Profile for nodes launched by Karpenter, to preserve the legacy behavior of the `eks/karpenter` component creating it. Set to `false` to enable creation of the IAM Instance Profile, which ensures that both the role and the instance profile have the same lifecycle, and avoids AWS Provider issue [#32671](https://github.com/hashicorp/terraform-provider-aws/issues/32671). Use in conjunction with `eks/karpenter` component `legacy_create_karpenter_instance_profile`. **Default value:** `true`
`legacy_fargate_1_role_per_profile_enabled` (`bool`) optional
Set to `false` for new clusters to create a single Fargate Pod Execution role for the cluster. Set to `true` for existing clusters to preserve the old behavior of creating a Fargate Pod Execution role for each Fargate Profile. **Default value:** `true`
`managed_node_groups_enabled` (`bool`) optional
Set false to prevent the creation of EKS managed node groups. **Default value:** `true`
`map_additional_iam_roles` optional
Additional IAM roles to grant access to the cluster. *WARNING*: Full Role ARN, including path, is required for `rolearn`. In earlier versions (with `aws-auth` ConfigMap), only the path had to be removed from the Role ARN. The path is now required. `username` is now ignored. This input is planned to be replaced in a future release with a more flexible input structure that consolidates `map_additional_iam_roles` and `map_additional_iam_users`. **Type:** ```hcl list(object({ rolearn = string username = optional(string) groups = list(string) })) ``` **Default value:** `[ ]`
`map_additional_iam_users` optional
Additional IAM roles to grant access to the cluster. `username` is now ignored. This input is planned to be replaced in a future release with a more flexible input structure that consolidates `map_additional_iam_roles` and `map_additional_iam_users`. **Type:** ```hcl list(object({ userarn = string username = optional(string) groups = list(string) })) ``` **Default value:** `[ ]`
`map_additional_worker_roles` (`list(string)`) optional
(Deprecated) AWS IAM Role ARNs of unmanaged Linux worker nodes to grant access to the EKS cluster. In earlier versions, this could be used to grant access to worker nodes of any type that were not managed by the EKS cluster. Now EKS requires that unmanaged worker nodes be classified as Linux or Windows servers, in this input is temporarily retained with the assumption that all worker nodes are Linux servers. (It is likely that earlier versions did not work properly with Windows worker nodes anyway.) This input is deprecated and will be removed in a future release. In the future, this component will either have a way to separate Linux and Windows worker nodes, or drop support for unmanaged worker nodes entirely. **Default value:** `[ ]`
`node_group_defaults` optional
Defaults for node groups in the cluster **Type:** ```hcl object({ ami_release_version = optional(string, null) ami_type = optional(string, null) attributes = optional(list(string), null) availability_zones = optional(list(string)) # set to null to use var.availability_zones cluster_autoscaler_enabled = optional(bool, null) create_before_destroy = optional(bool, null) desired_group_size = optional(number, null) instance_types = optional(list(string), null) kubernetes_labels = optional(map(string), {}) kubernetes_taints = optional(list(object({ key = string value = string effect = string })), []) node_userdata = optional(object({ before_cluster_joining_userdata = optional(string) bootstrap_extra_args = optional(string) kubelet_extra_args = optional(string) after_cluster_joining_userdata = optional(string) }), {}) kubernetes_version = optional(string, null) # set to null to use cluster_kubernetes_version max_group_size = optional(number, null) min_group_size = optional(number, null) resources_to_tag = optional(list(string), null) tags = optional(map(string), null) # block_device_map copied from cloudposse/terraform-aws-eks-node-group # Keep in sync via copy and paste, but make optional # Most of the time you want "/dev/xvda". For BottleRocket, use "/dev/xvdb". block_device_map = optional(map(object({ no_device = optional(bool, null) virtual_name = optional(string, null) ebs = optional(object({ delete_on_termination = optional(bool, true) encrypted = optional(bool, true) iops = optional(number, null) kms_key_id = optional(string, null) snapshot_id = optional(string, null) throughput = optional(number, null) # for gp3, MiB/s, up to 1000 volume_size = optional(number, 50) # disk size in GB volume_type = optional(string, "gp3") # Catch common camel case typos. These have no effect, they just generate better errors. # It would be nice to actually use these, but volumeSize in particular is a number here # and in most places it is a string with a unit suffix (e.g. 20Gi) # Without these defined, they would be silently ignored and the default values would be used instead, # which is difficult to debug. deleteOnTermination = optional(any, null) kmsKeyId = optional(any, null) snapshotId = optional(any, null) volumeSize = optional(any, null) volumeType = optional(any, null) })) })), null) # DEPRECATED: disk_encryption_enabled is DEPRECATED, use `block_device_map` instead. disk_encryption_enabled = optional(bool, null) # DEPRECATED: disk_size is DEPRECATED, use `block_device_map` instead. disk_size = optional(number, null) }) ``` **Default value:** ```hcl { "block_device_map": { "/dev/xvda": { "ebs": { "encrypted": true, "volume_size": 20, "volume_type": "gp2" } } }, "desired_group_size": 1, "instance_types": [ "t3.medium" ], "kubernetes_version": null, "max_group_size": 100 } ```
`node_groups` optional
List of objects defining a node group for the cluster **Type:** ```hcl map(object({ # EKS AMI version to use, e.g. "1.16.13-20200821" (no "v"). ami_release_version = optional(string, null) # Type of Amazon Machine Image (AMI) associated with the EKS Node Group ami_type = optional(string, null) # Additional attributes (e.g. `1`) for the node group attributes = optional(list(string), null) # will create 1 auto scaling group in each specified availability zone # or all AZs with subnets if none are specified anywhere availability_zones = optional(list(string), null) # Whether to enable Node Group to scale its AutoScaling Group cluster_autoscaler_enabled = optional(bool, null) # True to create new node_groups before deleting old ones, avoiding a temporary outage create_before_destroy = optional(bool, null) # Desired number of worker nodes when initially provisioned desired_group_size = optional(number, null) # Set of instance types associated with the EKS Node Group. Terraform will only perform drift detection if a configuration value is provided. instance_types = optional(list(string), null) # Key-value mapping of Kubernetes labels. Only labels that are applied with the EKS API are managed by this argument. Other Kubernetes labels applied to the EKS Node Group will not be managed kubernetes_labels = optional(map(string), null) # List of objects describing Kubernetes taints. kubernetes_taints = optional(list(object({ key = string value = string effect = string })), null) node_userdata = optional(object({ before_cluster_joining_userdata = optional(string) bootstrap_extra_args = optional(string) kubelet_extra_args = optional(string) after_cluster_joining_userdata = optional(string) }), {}) # Desired Kubernetes master version. If you do not specify a value, the latest available version is used kubernetes_version = optional(string, null) # The maximum size of the AutoScaling Group max_group_size = optional(number, null) # The minimum size of the AutoScaling Group min_group_size = optional(number, null) # List of auto-launched resource types to tag resources_to_tag = optional(list(string), null) tags = optional(map(string), null) # block_device_map copied from cloudposse/terraform-aws-eks-node-group # Keep in sync via copy and paste, but make optional. # Most of the time you want "/dev/xvda". For BottleRocket, use "/dev/xvdb". block_device_map = optional(map(object({ no_device = optional(bool, null) virtual_name = optional(string, null) ebs = optional(object({ delete_on_termination = optional(bool, true) encrypted = optional(bool, true) iops = optional(number, null) kms_key_id = optional(string, null) snapshot_id = optional(string, null) throughput = optional(number, null) # for gp3, MiB/s, up to 1000 volume_size = optional(number, 20) # Disk size in GB volume_type = optional(string, "gp3") # Catch common camel case typos. These have no effect, they just generate better errors. # It would be nice to actually use these, but volumeSize in particular is a number here # and in most places it is a string with a unit suffix (e.g. 20Gi) # Without these defined, they would be silently ignored and the default values would be used instead, # which is difficult to debug. deleteOnTermination = optional(any, null) kmsKeyId = optional(any, null) snapshotId = optional(any, null) volumeSize = optional(any, null) volumeType = optional(any, null) })) })), null) # DEPRECATED: # Enable disk encryption for the created launch template (if we aren't provided with an existing launch template) # DEPRECATED: disk_encryption_enabled is DEPRECATED, use `block_device_map` instead. disk_encryption_enabled = optional(bool, null) # Disk size in GiB for worker nodes. Terraform will only perform drift detection if a configuration value is provided. # DEPRECATED: disk_size is DEPRECATED, use `block_device_map` instead. disk_size = optional(number, null) })) ``` **Default value:** `{ }`
`oidc_provider_enabled` (`bool`) optional
Create an IAM OIDC identity provider for the cluster, then you can create IAM roles to associate with a service account in the cluster, instead of using kiam or kube2iam. For more information, see https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html **Default value:** `true`
`public_access_cidrs` (`list(string)`) optional
Indicates which CIDR blocks can access the Amazon EKS public API server endpoint when enabled. EKS defaults this to a list with 0.0.0.0/0. **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`subnet_type_tag_key` (`string`) optional
The tag used to find the private subnets to find by availability zone. If null, will be looked up in vpc outputs. **Default value:** `null`
`vpc_component_name` (`string`) optional
The name of the vpc component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`availability_zones`
Availability Zones in which the cluster is provisioned
`eks_addons_versions`
Map of enabled EKS Addons names and versions
`eks_auth_worker_roles`
List of worker IAM roles that were included in the `auth-map` ConfigMap.
`eks_cluster_arn`
The Amazon Resource Name (ARN) of the cluster
`eks_cluster_certificate_authority_data`
The Kubernetes cluster certificate authority data
`eks_cluster_endpoint`
The endpoint for the Kubernetes API server
`eks_cluster_id`
The name of the cluster
`eks_cluster_identity_oidc_issuer`
The OIDC Identity issuer for the cluster
`eks_cluster_managed_security_group_id`
Security Group ID that was created by EKS for the cluster. EKS creates a Security Group and applies it to ENI that is attached to EKS Control Plane master nodes and to any managed workloads
`eks_cluster_version`
The Kubernetes server version of the cluster
`eks_managed_node_workers_role_arns`
List of ARNs for workers in managed node groups
`eks_node_group_arns`
List of all the node group ARNs in the cluster
`eks_node_group_count`
Count of the worker nodes
`eks_node_group_ids`
EKS Cluster name and EKS Node Group name separated by a colon
`eks_node_group_role_names`
List of worker nodes IAM role names
`eks_node_group_statuses`
Status of the EKS Node Group
`fargate_profile_role_arns`
Fargate Profile Role ARNs
`fargate_profile_role_names`
Fargate Profile Role names
`fargate_profiles`
Fargate Profiles
`karpenter_iam_role_arn`
Karpenter IAM Role ARN
`karpenter_iam_role_name`
Karpenter IAM Role name
`vpc_cidr`
The CIDR of the VPC where this cluster is deployed.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_ebs_csi_driver_eks_iam_role` | 2.2.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.2.1) | n/a `aws_ebs_csi_driver_fargate_profile` | 1.3.0 | [`cloudposse/eks-fargate-profile/aws`](https://registry.terraform.io/modules/cloudposse/eks-fargate-profile/aws/1.3.0) | n/a `aws_efs_csi_driver_eks_iam_role` | 2.2.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.2.1) | n/a `coredns_fargate_profile` | 1.3.0 | [`cloudposse/eks-fargate-profile/aws`](https://registry.terraform.io/modules/cloudposse/eks-fargate-profile/aws/1.3.0) | n/a `eks_cluster` | 4.8.0 | [`cloudposse/eks-cluster/aws`](https://registry.terraform.io/modules/cloudposse/eks-cluster/aws/4.8.0) | n/a `fargate_pod_execution_role` | 1.3.0 | [`cloudposse/eks-fargate-profile/aws`](https://registry.terraform.io/modules/cloudposse/eks-fargate-profile/aws/1.3.0) | n/a `fargate_profile` | 1.3.0 | [`cloudposse/eks-fargate-profile/aws`](https://registry.terraform.io/modules/cloudposse/eks-fargate-profile/aws/1.3.0) | ############################################################################## ## Both New and Legacy behavior, use caution when modifying ############################################################################## `iam_arns` | latest | [`../../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../../account-map/modules/roles-to-principals/) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `karpenter_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `region_node_group` | latest | [`./modules/node_group_by_region`](https://registry.terraform.io/modules/./modules/node_group_by_region/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpc_cni_eks_iam_role` | 2.2.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.2.1) | n/a `vpc_ingress` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_policy.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.karpenter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.amazon_ec2_container_registry_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_worker_node_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_ssm_managed_instance_core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.aws_ebs_csi_driver`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.aws_efs_csi_driver`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.vpc_cni`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`random_pet.camel_case_warning`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_availability_zones.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.vpc_cni_ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_roles.sso_roles`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_roles) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## datadog-agent This component installs the `datadog-agent` for EKS clusters. ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage **Stack Level**: Regional Use this in the catalog as default values. ```yaml components: terraform: datadog-agent: settings: spacelift: workspace_enabled: true vars: enabled: true eks_component_name: eks/cluster name: "datadog" description: "Datadog Kubernetes Agent" kubernetes_namespace: "monitoring" create_namespace: true repository: "https://helm.datadoghq.com" chart: "datadog" chart_version: "3.29.2" timeout: 1200 wait: true atomic: true cleanup_on_fail: true cluster_checks_enabled: false helm_manifest_experiment_enabled: false secrets_store_type: SSM tags: team: sre service: datadog-agent app: monitoring # datadog-agent shouldn't be deployed to the Fargate nodes values: agents: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: eks.amazonaws.com/compute-type operator: NotIn values: - fargate datadog: env: - name: DD_EC2_PREFER_IMDSV2 # this merges ec2 instances and the node in the hostmap section value: "true" ``` Deploy this to a particular environment such as dev, prod, etc. This will add cluster checks to a specific environment. ```yaml components: terraform: datadog-agent: vars: # Order affects merge order. Later takes priority. We append lists though. datadog_cluster_check_config_paths: - catalog/cluster-checks/defaults/*.yaml - catalog/cluster-checks/dev/*.yaml datadog_cluster_check_config_parameters: {} # add additional tags to all data coming in from this agent. datadog_tags: - "env:dev" - "region:us-west-2" - "stage:dev" ``` ## Cluster Checks Cluster Checks are configurations that allow us to setup external URLs to be monitored. They can be configured through the datadog agent or annotations on kubernetes services. Cluster Checks are similar to synthetics checks, they are not as indepth, but significantly cheaper. Use Cluster Checks when you need a simple health check beyond the kubernetes pod health check. Public addresses that test endpoints must use the agent configuration, whereas service addresses internal to the cluster can be tested by annotations. ### Adding Cluster Checks Cluster Checks can be enabled or disabled via the `cluster_checks_enabled` variable. We recommend this be set to true. New Cluster Checks can be added to defaults to be applied in every account. Alternatively they can be placed in an individual stage folder which will be applied to individual stages. This is controlled by the `datadog_cluster_check_config_parameters` variable, which determines the paths of yaml files to look for cluster checks per stage. Once they are added, and properly configured, the new checks show up in the network monitor creation under `ssl` and `Http` **Please note:** the yaml file name doesn't matter, but the root key inside which is `something.yaml` does matter. this is following [datadogs docs](https://docs.datadoghq.com/agent/cluster_agent/clusterchecks/?tab=helm#configuration-from-static-configuration-files) for `.yaml`. #### Sample Yaml :::warning The key of a filename must match datadog docs, which is `.yaml` > [Datadog Cluster Checks](https://docs.datadoghq.com/agent/cluster_agent/clusterchecks/?tab=helm#configuration-from-static-configuration-files) ::: Cluster Checks **can** be used for external URL testing (loadbalancer endpoints), whereas annotations **must** be used for kubernetes services. ``` http_check.yaml: cluster_check: true init_config: instances: - name: "[${stage}] Echo Server" url: "https://echo.${stage}.uw2.acme.com" - name: "[${stage}] Portal" url: "https://portal.${stage}.uw2.acme.com" - name: "[${stage}] ArgoCD" url: "https://argocd.${stage}.uw2.acme.com" ``` ### Monitoring Cluster Checks Using Cloudposse's `datadog-monitor` component. The following yaml snippet will monitor all HTTP Cluster Checks, this can be added to each stage (usually via a defaults folder). ```yaml https-checks: name: "(Network Check) ${stage} - HTTPS Check" type: service check query: | "http.can_connect".over("stage:${stage}").by("instance").last(2).count_by_status() message: | HTTPS Check failed on {{instance.name}} in Stage: {{stage.name}} escalation_message: "" tags: managed-by: Terraform notify_no_data: false notify_audit: false require_full_window: true enable_logs_sample: false force_delete: true include_tags: true locked: false renotify_interval: 0 timeout_h: 0 evaluation_delay: 0 new_host_delay: 0 new_group_delay: 0 no_data_timeframe: 2 threshold_windows: {} thresholds: critical: 1 warning: 1 ok: 1 ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended
`kubernetes_namespace` (`string`) required
Kubernetes namespace to install the release into
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used **Default value:** `true`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails **Default value:** `true`
`cluster_checks_enabled` (`bool`) optional
Enable Cluster Checks for the Datadog Agent **Default value:** `false`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`datadog_cluster_check_auto_added_tags` (`list(string)`) optional
List of tags to add to Datadog Cluster Check **Default value:** ```hcl [ "stage", "environment" ] ```
`datadog_cluster_check_config_parameters` (`map(any)`) optional
Map of parameters to Datadog Cluster Check configurations **Default value:** `{ }`
`datadog_cluster_check_config_paths` (`list(string)`) optional
List of paths to Datadog Cluster Check configurations **Default value:** `[ ]`
`datadog_tags` (`set(string)`) optional
List of static tags to attach to every metric, event and service check collected by the agent **Default value:** `[ ]`
`description` (`string`) optional
Release description attribute (visible in the history) **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the EKS component. Used to get the remote state **Default value:** `"eks/eks"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`repository` (`string`) optional
Repository URL where to locate the requested chart **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cluster_checks`
Cluster Checks for the cluster
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.7` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` - `utils`, version: `>= 1.10.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_agent` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `datadog_cluster_check_yaml_config` | 1.0.2 | [`cloudposse/config/yaml`](https://registry.terraform.io/modules/cloudposse/config/yaml/1.0.2) | n/a `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `values_merge` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## echo-server This is copied from [cloudposse/terraform-aws-components](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/echo-server). This component installs the [Ealenn/Echo-Server](https://github.com/Ealenn/Echo-Server) to EKS clusters. The echo server is a server that sends it back to the client a JSON representation of all the data the server received, which is a combination of information sent by the client and information sent by the web server infrastructure. For further details, please consult the [Echo-Server documentation](https://ealenn.github.io/Echo-Server/). ## Prerequisites Echo server is intended to provide end-to-end testing of everything needed to deploy an application or service with a public HTTPS endpoint. It uses defaults where possible, such as using the default IngressClass, in order to verify that the defaults are sufficient for a typical application. In order to minimize the impact of the echo server on the rest of the cluster, it does not set any configuration that would affect other ingresses, such as WAF rules, logging, or redirecting HTTP to HTTPS. Those settings should be configured in the IngressClass where possible. Therefore, it requires several other components. At the moment, it supports 2 configurations: 1. ALB with ACM Certificate - AWS Load Balancer Controller (ALB) version 2.2.0 or later, with ACM certificate auto-discovery enabled - A default IngressClass, which can be provisioned by the `alb-controller` component as part of deploying the controller, or can be provisioned separately, for example by the `alb-controller-ingress-class` component. - Pre-provisioned ACM TLS certificate covering the provisioned host name (typically a wildcard certificate covering all hosts in the domain) 2. Nginx with Cert Manager Certificate - Nginx (via `kubernetes/ingress-nginx` controller). We recommend `ingress-nginx` v1.1.0 or later, but `echo-server` should work with any version that supports Ingress API version `networking.k8s.io/v1`. - `jetstack/cert-manager` configured to automatically (via Ingress Shim, installed by default) generate TLS certificates via a Cluster Issuer (by default, named `letsEncrypt-prod`). In both configurations, it has these common requirements: - EKS component deployed, with component name specified in `eks_component_name` (defaults to "eks/cluster") - Kubernetes version 1.19 or later - Ingress API version `networking.k8s.io/v1` - [kubernetes-sigs/external-dns](https://github.com/kubernetes-sigs/external-dns) - A default IngressClass, either explicitly provisioned or supported without provisioning by the Ingress controller. ## Warnings A Terraform plan may fail to apply, giving a Kubernetes authentication failure. This is due to a known issue with Terraform and the Kubernetes provider. During the "plan" phase Terraform gets a short-lived Kubernetes authentication token and caches it, and then tries to use it during "apply". If the token has expired by the time you try to run "apply", the "apply" will fail. The workaround is to run `terraform apply -auto-approve` without a "plan" file. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. Set `ingress_type` to "alb" if using `alb-controller` or "nginx" if using `ingress-nginx`. Normally, you should not set the IngressClass or IngressGroup, as this component is intended to test the defaults. However, if you need to, set them in `chart_values`: ```yaml chart_values: ingress: class: "other-ingress-class" alb: # IngressGroup is specific to alb-controller group_name: "other-ingress-group" ``` Note that if you follow recommendations and do not set the ingress class name, the deployed Ingress will have the ingressClassName setting injected by the Ingress controller, set to the then-current default. This means that if later you change the default IngressClass, the Ingress will be NOT be updated to use the new default. Furthermore, because of limitations in the Helm provider, this will not be detected as drift. You will need to destroy and re-deploy the echo server to update the Ingress to the new default. ```yaml components: terraform: echo-server: settings: spacelift: workspace_enabled: true vars: enabled: true name: "echo-server" kubernetes_namespace: "echo" description: "Echo server, for testing purposes" create_namespace: true timeout: 180 wait: true atomic: true cleanup_on_fail: true ingress_type: "alb" # or "nginx" # %[1]v is the tenant name, %[2]v is the stage name, %[3]v is the region name hostname_template: "echo.%[3]v.%[2]v.%[1]v.sample-domain.net" ``` In rare cases where some ingress controllers do not support the `ingressClassName` field, you can restore the old `kubernetes.io/ingress.class` annotation by setting `ingress.use_ingress_class_annotation: true` in `chart_values`. ## Variables ### Required Variables
`hostname_template` (`string`) required
The `format()` string to use to generate the hostname via `format(var.hostname_template, var.tenant, var.stage, var.environment)`" Typically something like `"echo.%[3]v.%[2]v.example.com"`.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_values` (`any`) optional
Addition map values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`ingress_type` (`string`) optional
Set to 'nginx' to create an ingress resource relying on an NGiNX backend for the echo-server service. Set to 'alb' to create an ingress resource relying on an AWS ALB backend for the echo-server service. Leave blank to not create any ingress for the echo-server service. **Default value:** `null`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`hostname`
Hostname of the deployed echo server
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `echo_server` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## external-dns This component creates a Helm deployment for [external-dns](https://github.com/kubernetes-sigs/external-dns) on a Kubernetes cluster. [external-dns](https://github.com/kubernetes-sigs/external-dns) is a Kubernetes addon that configures public DNS servers with information about exposed Kubernetes services to make them discoverable. ## Usage **Stack Level**: Regional Once the catalog is created, the file can be imported as follows. ```yaml import: - catalog/eks/external-dns ... ``` The default catalog values `e.g. stacks/catalog/eks/external-dns.yaml` ```yaml components: terraform: external-dns: vars: enabled: true name: external-dns chart: external-dns chart_repository: https://kubernetes-sigs.github.io/external-dns/ chart_version: "1.18.0" create_namespace: true kubernetes_namespace: external-dns resources: limits: cpu: 200m memory: 256Mi requests: cpu: 100m memory: 128Mi # Set this to a unique value to avoid conflicts with other external-dns instances managing the same zones. # For example, when using blue-green deployment pattern to update EKS cluster. txt_prefix: "" # You can use `chart_values` to set any other chart options. Treat `chart_values` as the root of the doc. # See documentation for latest chart version and list of chart_values: https://artifacthub.io/packages/helm/external-dns/external-dns # # # For example # --- # chart_values: # provider: # name: aws # extraArgs: # - --aws-batch-change-size=1000 chart_values: {} # Extra hosted zones to lookup and support by component name dns_components: - component: dns-primary - component: dns-delegated - component: dns-delegated/abc - component: dns-delegated/123 environment: "gbl" # Optional (default "gbl") ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Addition map values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`crd_enabled` (`bool`) optional
Install and use the integrated DNSEndpoint CRD. **Default value:** `false`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`dns_components` optional
A list of additional DNS components to search for ZoneIDs **Type:** ```hcl list(object({ component = string, environment = optional(string) })) ``` **Default value:** `[ ]`
`dns_gbl_delegated_environment_name` (`string`) optional
The name of the environment where global `dns_delegated` is provisioned **Default value:** `"gbl"`
`dns_gbl_primary_environment_name` (`string`) optional
The name of the environment where global `dns_primary` is provisioned **Default value:** `"gbl"`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`istio_enabled` (`bool`) optional
Add istio gateways to monitored sources. **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`metrics_enabled` (`bool`) optional
Whether or not to enable metrics in the helm chart. **Default value:** `false`
`policy` (`string`) optional
Modify how DNS records are synchronized between sources and providers (options: sync, upsert-only) **Default value:** `"sync"`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`resources` optional
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ``` **Default value:** ```hcl { "limits": { "cpu": "200m", "memory": "256Mi" }, "requests": { "cpu": "100m", "memory": "128Mi" } } ```
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`txt_prefix` (`string`) optional
Prefix to create a TXT record with a name following the pattern prefix.``. **Default value:** `"external-dns"`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `additional_dns_components` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dns_gbl_primary` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `external_dns` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## external-secrets-operator This component (ESO) is used to create an external `SecretStore` configured to synchronize secrets from AWS SSM Parameter store as Kubernetes Secrets within the cluster. Per the operator pattern, the `external-secret-operator` pods will watch for any `ExternalSecret` resources which reference the `SecretStore` to pull secrets from. In practice, this means apps will define an `ExternalSecret` that pulls all env into a single secret as part of a helm chart; e.g.: ``` # Part of the charts in `/releases apiVersion: external-secrets.io/v1beta1 kind: ExternalSecret metadata: name: app-secrets spec: refreshInterval: 30s secretStoreRef: name: "secret-store-parameter-store" # Must match name of the Cluster Secret Store created by this component kind: ClusterSecretStore target: creationPolicy: Owner name: app-secrets dataFrom: - find: name: regexp: "^/app/" # Match the path prefix of your service rewrite: - regexp: source: "/app/(.*)" # Remove the path prefix of your service from the name before creating the envars target: "$1" ``` This component assumes secrets are prefixed by "service" in parameter store (e.g. `/app/my_secret`). The `SecretStore`. The component is designed to pull secrets from a `path` prefix (defaulting to `"app"`). This should work nicely along `chamber` which uses this same path (called a "service" in Chamber). For example, developers should store keys like so. ```bash assume-role acme-platform-gbl-sandbox-admin chamber write app MY_KEY my-value ``` See `docs/recipes.md` for more information on managing secrets. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: eks/external-secrets-operator: settings: spacelift: workspace_enabled: true vars: enabled: true name: "external-secrets-operator" helm_manifest_experiment_enabled: false chart: "external-secrets" chart_repository: "https://charts.external-secrets.io" chart_version: "0.8.3" kubernetes_namespace: "secrets" create_namespace: true timeout: 90 wait: true atomic: true cleanup_on_fail: true tags: Team: sre Service: external-secrets-operator resources: limits: cpu: "100m" memory: "300Mi" requests: cpu: "20m" memory: "60Mi" parameter_store_paths: - app - rds # You can use `chart_values` to set any other chart options. Treat `chart_values` as the root of the doc. # # # For example # --- # chart_values: # installCRDs: true chart_values: {} kms_aliases_allow_decrypt: [] # - "alias/foo/bar" ``` ## Variables ### Required Variables
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
`resources` required
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"external-secrets"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"External Secrets Operator is a Kubernetes operator that integrates external secret management systems including AWS SSM, Parameter Store, Hasicorp Vault, 1Password Secrets Automation, etc. It reads values from external vaults and injects values as a Kubernetes Secret"`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://charts.external-secrets.io"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"0.6.0-rc1"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kms_aliases_allow_decrypt` (`list(string)`) optional
A list of KMS aliases that the SecretStore is allowed to decrypt. **Default value:** `[ ]`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`parameter_store_paths` (`set(string)`) optional
A list of path prefixes that the SecretStore is allowed to access via IAM. This should match the convention 'service' that Chamber uploads keys under. **Default value:** ```hcl [ "app" ] ```
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `external_secrets_operator` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | CRDs are automatically installed by "cloudposse/helm-release/aws" https://external-secrets.io/v0.5.9/guides-getting-started/ `external_ssm_secrets` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_kms_alias.kms_aliases`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_alias) (data source) - [`kubernetes_resources.crd`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/data-sources/resources) (data source) --- ## github-actions-runner This component deploys self-hosted GitHub Actions Runners and a [Controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller#introduction) on an EKS cluster, using "[runner scale sets](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#runner-scale-set)". This solution is supported by GitHub and supersedes the [actions-runner-controller](https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md) developed by Summerwind and deployed by Cloud Posse's [actions-runner-controller](https://docs.cloudposse.com/components/library/aws/eks/actions-runner-controller/) component. ### Current limitations The runner image used by Runner Sets contains [no more packages than are necessary](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#about-the-runner-container-image) to run the runner. This is in contrast to the Summerwind implementation, which contains some commonly needed packages like `build-essential`, `curl`, `wget`, `git`, and `jq`, and the GitHub hosted images which contain a robust set of tools. (This is a limitation of the official Runner Sets implementation, not this component per se.) You will need to install any tools you need in your workflows, either as part of your workflow (recommended), by maintaining a [custom runner image](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#creating-your-own-runner-image), or by running such steps in a [separate container](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container) that has the tools pre-installed. Many tools have publicly available actions to install them, such as `actions/setup-node` to install NodeJS or `dcarbone/install-jq-action` to install `jq`. You can also install packages using `awalsh128/cache-apt-pkgs-action`, which has the advantage of being able to skip the installation if the package is already installed, so you can more efficiently run the same workflow on GitHub hosted as well as self-hosted runners. :::info There are (as of this writing) open feature requests to add some commonly needed packages to the official Runner Sets runner image. You can upvote these requests [here](https://github.com/actions/actions-runner-controller/discussions/3168) and [here](https://github.com/orgs/community/discussions/80868) to help get them implemented. ::: In the current version of this component, only "dind" (Docker in Docker) mode has been tested. Support for "kubernetes" mode is provided, but has not been validated. Many elements in the Controller chart are not directly configurable by named inputs. To configure them, you can use the `controller.chart_values` input or create a `resources/values-controller.yaml` file in the component to supply values. Almost all the features of the Runner Scale Set chart are configurable by named inputs. The exceptions are: - There is no specific input for specifying an outbound HTTP proxy. - There is no specific input for supplying a [custom certificate authority (CA) certificate](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#custom-tls-certificates) to use when connecting to GitHub Enterprise Server. You can specify these values by creating a `resources/values-runner.yaml` file in the component and setting values as shown by the default Helm [values.yaml](https://github.com/actions/actions-runner-controller/blob/master/charts/gha-runner-scale-set/values.yaml), and they will be applied to all runners. Currently, this component has some additional limitations. In particular: - The controller and all runners and listeners share the Image Pull Secrets. You cannot use different ones for different runners. - All the runners use the same GitHub secret (app or PAT). Using a GitHub app is preferred anyway, and the single GitHub app serves the entire organization. - Only one controller is supported per cluster, though it can have multiple replicas. These limitations could be addressed if there is demand. Contact [Cloud Posse Professional Services](https://cloudposse.com/professional-services/) if you would be interested in sponsoring the development of any of these features. ### Ephemeral work storage The runners are configured to use ephemeral storage for workspaces, but the details and defaults can be a bit confusing. When running in "dind" ("Docker in Docker") mode, the default is to use `emptyDir`, which means space on the `kubelet` base directory, which is usually the root disk. You can manage the amount of storage allowed to be used with `ephemeral_storage` requests and limits, or you can just let it use whatever free space there is on the root disk. When running in `kubernetes` mode, the only supported local disk storage is an ephemeral `PersistentVolumeClaim`, which causes a separate disk to be allocated for the runner pod. This disk is ephemeral, and will be deleted when the runner pod is deleted. When combined with the recommended ephemeral runner configuration, this means that a new disk will be created for each job, and deleted when the job is complete. That is a lot of overhead and will slow things down somewhat. The size of the attached PersistentVolume is controlled by `ephemeral_pvc_storage` (a Kubernetes size string like "1G") and the kind of storage is controlled by `ephemeral_pvc_storage_class` (which can be omitted to use the cluster default storage class). This mode is also optionally available when using `dind`. To enable it, set `ephemeral_pvc_storage` to the desired size. Leave `ephemeral_pvc_storage` at the default value of `null` to use `emptyDir` storage (recommended). Beware that using a PVC may significantly increase the startup of the runner. If you are using a PVC, you may want to keep idle runners available so that jobs can be started without waiting for a new runner to start. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/github-actions-runner ... ``` The default catalog values `e.g. stacks/catalog/eks/github-actions-runner.yaml` ```yaml components: terraform: eks/github-actions-runner: vars: enabled: true ssm_region: "us-east-2" name: "gha-runner-controller" charts: controller: chart_version: "0.7.0" runner_sets: chart_version: "0.7.0" controller: kubernetes_namespace: "gha-runner-controller" create_namespace: true create_github_kubernetes_secret: true ssm_github_secret_path: "/github-action-runners/github-auth-secret" github_app_id: "123456" github_app_installation_id: "12345678" runners: config-default: &runner-default enabled: false github_url: https://github.com/cloudposse # group: "default" # kubernetes_namespace: "gha-runner-private" create_namespace: true # If min_replicas > 0 and you also have do-not-evict: "true" set # then the idle/waiting runner will keep Karpenter from deprovisioning the node # until a job runs and the runner is deleted. # override by setting `pod_annotations: {}` pod_annotations: karpenter.sh/do-not-evict: "true" min_replicas: 0 max_replicas: 8 resources: limits: cpu: 1100m memory: 1024Mi ephemeral-storage: 5Gi requests: cpu: 500m memory: 256Mi ephemeral-storage: 1Gi self-hosted-default: <<: *runner-default enabled: true kubernetes_namespace: "gha-runner-private" # If min_replicas > 0 and you also have do-not-evict: "true" set # then the idle/waiting runner will keep Karpenter from deprovisioning the node # until a job runs and the runner is deleted. So we override the default. pod_annotations: {} min_replicas: 1 max_replicas: 12 resources: limits: cpu: 1100m memory: 1024Mi ephemeral-storage: 5Gi requests: cpu: 500m memory: 256Mi ephemeral-storage: 1Gi self-hosted-large: <<: *runner-default enabled: true resources: limits: cpu: 6000m memory: 7680Mi ephemeral-storage: 90G requests: cpu: 4000m memory: 7680Mi ephemeral-storage: 40G ``` ### Authentication and Secrets The GitHub Action Runners need to authenticate to GitHub in order to do such things as register runners and pickup jobs. You can authenticate using either a [GitHub App](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app) or a [Personal Access Token (classic)](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-personal-access-token-classic). The preferred way to authenticate is by _creating_ and _installing_ a GitHub App. This is the recommended approach as it allows for much more restricted access than using a Personal Access Token (classic), and the Action Runners do not currently support using a fine-grained Personal Access Token. #### Site note about SSM and Regions This component supports using AWS SSM to store and retrieve secrets. SSM parameters are regional, so if you want to deploy to multiple regions you have 2 choices: 1. Create the secrets in each region. This is the most robust approach, but requires you to create the secrets in each region and keep them in sync. 2. Create the secrets in one region and use the `ssm_region` input to specify the region where they are stored. This is the easiest approach, but does add some obstacles to managing deployments during a region outage. If the region where the secrets are stored goes down, there will be no impact on runners in other regions, but you will not be able to deploy new runners or modify existing runners until the SSM region is restored or until you set up SSM parameters in a new region. Alternatively, you can create Kubernetes secrets outside of this component (perhaps using [SOPS](https://github.com/getsops/sops)) and reference them by name. We describe here how to save the secrets to SSM, but you can save the secrets wherever and however you want to, as long as you deploy them as Kubernetes secret the runners can reference. If you store them in SSM, this component will take care of the rest, but the standard Terraform caveat applies: any secrets referenced by Terraform will be stored unencrypted in the Terraform state file. #### Creating and Using a GitHub App Follow the instructions [here](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-github-app) to create and install a GitHub App for the runners to use for authentication. At the App creation stage, you will be asked to generate a private key. This is the private key that will be used to authenticate the Action Runner. Download the file and store the contents in SSM using the following command, adjusting the profile, region, and file name. The profile should be the `terraform` role in the account to which you are deploying the runner controller. The region should be the region where you are deploying the primary runner controller. If you are deploying runners to multiple regions, they can all reference the same SSM parameter by using the `ssm_region` input to specify the region where they are stored. The file name (argument to `cat`) should be the name of the private key file you downloaded. ``` # Adjust profile name and region to suit your environment, use file name you chose for key AWS_PROFILE=acme-core-gbl-auto-terraform AWS_REGION=us-west-2 chamber write github-action-runners github-auth-secret -- "$(cat APP_NAME.DATE.private-key.pem)" ``` You can verify the file was correctly written to SSM by matching the private key fingerprint reported by GitHub with: ``` AWS_PROFILE=acme-core-gbl-auto-terraform AWS_REGION=us-west-2 chamber read -q github-action-runners github-auth-secret | openssl rsa -in - -pubout -outform DER | openssl sha256 -binary | openssl base64 ``` At this stage, record the Application ID and the private key fingerprint in your secrets manager (e.g. 1Password). You may want to record the private key as well, or you may consider it sufficient to have it in SSM. You will need the Application ID to configure the runner controller, and want the fingerprint to verify the private key. (You can see the fingerprint in the GitHub App settings, under "Private keys".) Proceed to install the GitHub App in the organization or repository you want to use the runner controller for, and record the Installation ID (the final numeric part of the URL, as explained in the instructions linked above) in your secrets manager. You will need the Installation ID to configure the runner controller. In your stack configuration, set the following variables, making sure to quote the values so they are treated as strings, not numbers. ``` github_app_id: "12345" github_app_installation_id: "12345" ``` #### OR (obsolete): Creating and Using a Personal Access Token (classic) Though not recommended, you can use a Personal Access Token (classic) to authenticate the runners. To do so, create a PAT (classic) as described in the [GitHub Documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/authenticating-to-the-github-api#authenticating-arc-with-a-personal-access-token-classic). Save this to the value specified by `ssm_github_token_path` using the following command, adjusting the AWS profile and region as explained above: ``` AWS_PROFILE=acme-core-gbl-auto-terraform AWS_REGION=us-west-2 chamber write github-action-runners github-auth-secret -- "" ``` ### Using Runner Groups GitHub supports grouping runners into distinct [Runner Groups](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups), which allow you to have different access controls for different runners. Read the linked documentation about creating and configuring Runner Groups, which you must do through the GitHub Web UI. If you choose to create Runner Groups, you can assign one or more Runner Sets (from the `runners` map) to groups (only one group per runner set, but multiple sets can be in the same group) by including `group: ` in the runner configuration. We recommend including it immediately after `github_url`. ### Interaction with Karpenter or other EKS autoscaling solutions Kubernetes cluster autoscaling solutions generally expect that a Pod runs a service that can be terminated on one Node and restarted on another with only a short duration needed to finish processing any in-flight requests. When the cluster is resized, the cluster autoscaler will do just that. However, GitHub Action Runner Jobs do not fit this model. If a Pod is terminated in the middle of a job, the job is lost. The likelihood of this happening is increased by the fact that the Action Runner Controller Autoscaler is expanding and contracting the size of the Runner Pool on a regular basis, causing the cluster autoscaler to more frequently want to scale up or scale down the EKS cluster, and, consequently, to move Pods around. To handle these kinds of situations, Karpenter respects an annotation on the Pod: ```yaml spec: template: metadata: annotations: karpenter.sh/do-not-evict: "true" ``` When you set this annotation on the Pod, Karpenter will not voluntarily evict it. This means that the Pod will stay on the Node it is on, and the Node it is on will not be considered for deprovisioning (scale down). This is good because it means that the Pod will not be terminated in the middle of a job. However, it also means that the Node the Pod is on will remain running until the Pod is terminated, even if the node is underutilized and Karpenter would like to get rid of it. Since the Runner Pods terminate at the end of the job, this is not a problem for the Pods actually running jobs. However, if you have set `minReplicas > 0`, then you have some Pods that are just idling, waiting for jobs to be assigned to them. These Pods are exactly the kind of Pods you want terminated and moved when the cluster is underutilized. Therefore, when you set `minReplicas > 0`, you should **NOT** set `karpenter.sh/do-not-evict: "true"` on the Pod. ### Updating CRDs When updating the chart or application version of `gha-runner-scale-set-controller`, it is possible you will need to install new CRDs. Such a requirement should be indicated in the `gha-runner-scale-set-controller` release notes and may require some adjustment to this component. This component uses `helm` to manage the deployment, and `helm` will not auto-update CRDs. If new CRDs are needed, follow the instructions in the release notes for the Helm chart or `gha-runner-scale-set-controller` itself. ### Useful Reference - Runner Scale Set Controller's Helm chart [values.yaml](https://github.com/actions/actions-runner-controller/blob/master/charts/gha-runner-scale-set-controller/values.yaml) - Runner Scale Set's Helm chart [values.yaml](https://github.com/actions/actions-runner-controller/blob/master/charts/gha-runner-scale-set/values.yaml) - Runner Scale Set's [Docker image](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#about-the-runner-container-image) and [how to create your own](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#creating-your-own-runner-image) When reviewing documentation, code, issues, etc. for self-hosted GitHub action runners or the Actions Runner Controller (ARC), keep in mind that there are 2 implementations going by that name. The original implementation, which is now deprecated, uses the `actions.summerwind.dev` API group, and is at times called the Summerwind or Legacy implementation. It is primarily described by documentation in the [actions/actions-runner-controller](https://github.com/actions/actions-runner-controller) GitHub repository itself. The new implementation, which is the one this component uses, uses the `actions.github.com` API group, and is at times called the GitHub implementation or "Runner Scale Sets" implementation. The new implementation is described in the official [GitHub documentation](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller). Feature requests about the new implementation are officially directed to the [Actions category of GitHub community discussion](https://github.com/orgs/community/discussions/categories/actions). However, Q&A and community support is directed to the `actions/actions-runner-controller` repo's [Discussion section](https://github.com/actions/actions-runner-controller/discussions), though beware that discussions about the old implementation are mixed in with discussions about the new implementation. Bug reports for the new implementation are still filed under the `actions/actions-runner-controller` repo's [Issues](https://github.com/actions/actions-runner-controller/issues) tab, though again, these are mixed in with bug reports for the old implementation. Look for the `gha-runner-scale-set` label to find issues specific to the new implementation. ## Variables ### Required Variables
`charts` required
Map of Helm charts to install. Keys are "controller" and "runner_sets". **Type:** ```hcl map(object({ chart_version = string chart = optional(string, null) # defaults according to the key to "gha-runner-scale-set-controller" or "gha-runner-scale-set" chart_description = optional(string, null) # visible in Helm history chart_repository = optional(string, "oci://ghcr.io/actions/actions-runner-controller-charts") wait = optional(bool, true) atomic = optional(bool, true) cleanup_on_fail = optional(bool, true) timeout = optional(number, null) })) ```
`controller` required
Configuration for the controller. **Type:** ```hcl object({ image = optional(object({ repository = optional(string, null) tag = optional(string, null) # Defaults to the chart appVersion pull_policy = optional(string, null) }), null) replicas = optional(number, 1) kubernetes_namespace = string create_namespace = optional(bool, true) chart_values = optional(any, null) affinity = optional(map(string), {}) labels = optional(map(string), {}) node_selector = optional(map(string), {}) priority_class_name = optional(string, "") resources = optional(object({ limits = optional(object({ cpu = optional(string, null) memory = optional(string, null) }), null) requests = optional(object({ cpu = optional(string, null) memory = optional(string, null) }), null) }), null) tolerations = optional(list(object({ key = string operator = string value = optional(string, null) effect = string })), []) log_level = optional(string, "info") log_format = optional(string, "json") update_strategy = optional(string, "immediate") }) ```
`region` (`string`) required
AWS Region.
### Optional Variables
`create_github_kubernetes_secret` (`bool`) optional
If `true`, this component will create the Kubernetes Secret that will be used to get the GitHub App private key or GitHub PAT token, based on the value retrieved from SSM at the `var.ssm_github_secret_path`. WARNING: This will cause the secret to be stored in plaintext in the Terraform state. If `false`, this component will not create a secret and you must create it (with the name given by `var.github_kubernetes_secret_name`) in every namespace where you are deploying runners (the controller does not need it). **Default value:** `true`
`create_image_pull_kubernetes_secret` (`bool`) optional
If `true` and `image_pull_secret_enabled` is `true`, this component will create the Kubernetes image pull secret resource, using the value in SSM at the path specified by `ssm_image_pull_secret_path`. WARNING: This will cause the secret to be stored in plaintext in the Terraform state. If `false`, this component will not create a secret and you must create it (with the name given by `var.github_kubernetes_secret_name`) in every namespace where you are deploying controllers or runners. **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`github_app_id` (`string`) optional
The ID of the GitHub App to use for the runner controller. Leave empty if using a GitHub PAT. **Default value:** `null`
`github_app_installation_id` (`string`) optional
The "Installation ID" of the GitHub App to use for the runner controller. Leave empty if using a GitHub PAT. **Default value:** `null`
`github_kubernetes_secret_name` (`string`) optional
Name of the Kubernetes Secret that will be used to get the GitHub App private key or GitHub PAT token. **Default value:** `"gha-github-secret"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`image_pull_kubernetes_secret_name` (`string`) optional
Name of the Kubernetes Secret that will be used as the imagePullSecret. **Default value:** `"gha-image-pull-secret"`
`image_pull_secret_enabled` (`bool`) optional
Whether to configure the controller and runners with an image pull secret. **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`runners` optional
Map of Runner Scale Set configurations, with the key being the name of the runner set. Please note that the name must be in kebab-case (no underscores). For example: ```hcl organization-runner = { # Specify the scope (organization or repository) and the target # of the runner via the `github_url` input. # ex: https://github.com/myorg/myrepo or https://github.com/myorg github_url = https://github.com/myorg group = "core-automation" # Optional. Assigns the runners to a runner group, for access control. min_replicas = 1 max_replicas = 5 } ``` **Type:** ```hcl map(object({ # we allow a runner to be disabled because Atmos cannot delete an inherited map object enabled = optional(bool, true) github_url = string group = optional(string, null) kubernetes_namespace = optional(string, null) # defaults to the controller's namespace create_namespace = optional(bool, true) image = optional(string, "ghcr.io/actions/actions-runner:latest") # repo and tag mode = optional(string, "dind") # Optional. Can be "dind" or "kubernetes". pod_labels = optional(map(string), {}) pod_annotations = optional(map(string), {}) affinity = optional(map(string), {}) node_selector = optional(map(string), {}) tolerations = optional(list(object({ key = string operator = string value = optional(string, null) effect = string # tolerationSeconds is not supported, because Terraform requires all objects in a list to have the same keys, # but tolerationSeconds must be omitted to get the default behavior of "tolerate forever". # If really needed, could use a default value of 1,000,000,000 (one billion seconds = about 32 years). })), []) min_replicas = number max_replicas = number # ephemeral_pvc_storage and _class are ignored for "dind" mode but required for "kubernetes" mode ephemeral_pvc_storage = optional(string, null) # ex: 10Gi ephemeral_pvc_storage_class = optional(string, null) kubernetes_mode_service_account_annotations = optional(map(string), {}) resources = optional(object({ limits = optional(object({ cpu = optional(string, null) memory = optional(string, null) ephemeral-storage = optional(string, null) }), null) requests = optional(object({ cpu = optional(string, null) memory = optional(string, null) ephemeral-storage = optional(string, null) }), null) }), null) })) ``` **Default value:** `{ }`
`ssm_github_secret_path` (`string`) optional
The path in SSM to the GitHub app private key file contents or GitHub PAT token. **Default value:** `"/github-action-runners/github-auth-secret"`
`ssm_image_pull_secret_path` (`string`) optional
SSM path to the base64 encoded `dockercfg` image pull secret. **Default value:** `"/github-action-runners/image-pull-secrets"`
`ssm_region` (`string`) optional
AWS Region where SSM secrets are stored. Defaults to `var.region`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
`runners`
Human-readable summary of the deployed runners
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `gha_runner_controller` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `gha_runners` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_namespace.controller`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) - [`kubernetes_namespace.runner`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) - [`kubernetes_secret_v1.controller_image_pull_secret`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret_v1) (resource) - [`kubernetes_secret_v1.controller_ns_github_secret`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret_v1) (resource) - [`kubernetes_secret_v1.github_secret`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret_v1) (resource) - [`kubernetes_secret_v1.image_pull_secret`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret_v1) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_ssm_parameter.github_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.image_pull_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## idp-roles This component installs the `idp-roles` for EKS clusters. These identity provider roles specify several pre-determined permission levels for cluster users and come with bindings that make them easy to assign to Users and Groups. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: eks/idp-roles: settings: spacelift: workspace_enabled: true vars: enabled: true name: "idp-roles" kubeconfig_exec_auth_api_version: "client.authentication.k8s.io/v1beta1" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"Identity provider roles and role bindings"`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Kubernetes namespace to install the release into **Default value:** `"kube-system"`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `300`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `idp_roles` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## karpenter-controller This component provisions [Karpenter](https://karpenter.sh) on an EKS cluster. It requires at least version 0.32.0 of Karpenter, though using the latest version is recommended. ## Usage **Stack Level**: Regional These instructions assume you are provisioning 2 EKS clusters in the same account and region, named "blue" and "green", and alternating between them. If you are only using a single cluster, you can ignore the "blue" and "green" references and remove the `metadata` block from the `karpenter` module. ```yaml components: terraform: # Base component of all `karpenter` components eks/karpenter: metadata: type: abstract vars: enabled: true eks_component_name: "eks/cluster" name: "karpenter" # https://github.com/aws/karpenter/tree/main/charts/karpenter chart_repository: "oci://public.ecr.aws/karpenter" chart: "karpenter" chart_version: "1.6.0" # Enable Karpenter to get advance notice of spot instances being terminated # See https://karpenter.sh/docs/concepts/#interruption interruption_handler_enabled: true resources: limits: cpu: "300m" memory: "1Gi" requests: cpu: "100m" memory: "512Mi" cleanup_on_fail: true atomic: true wait: true # "karpenter-crd" can be installed as an independent helm chart to manage the lifecycle of Karpenter CRDs crd_chart_enabled: true crd_chart: "karpenter-crd" # replicas set the number of Karpenter controller replicas to run replicas: 2 # "settings" controls a subset of the settings for the Karpenter controller regarding batch idle and max duration. # you can read more about these settings here: https://karpenter.sh/docs/reference/settings/ settings: batch_idle_duration: "1s" batch_max_duration: "10s" # (Optional) "settings" which do not have an explicit mapping and may be subject to change between helm chart versions additional_settings: featureGates: nodeRepair: false reservedCapacity: true spotToSpotConsolidation: true # The logging settings for the Karpenter controller logging: enabled: true level: controller: "info" global: "info" webhook: "error" ``` ## Provision Karpenter on EKS cluster Here we describe how to provision Karpenter on an EKS cluster. We will be using the `plat-ue2-dev` stack as an example. ### Provision Service-Linked Roles for EC2 Spot and EC2 Spot Fleet Note: If you want to use EC2 Spot for the instances launched by Karpenter, you may need to provision the following Service-Linked Role for EC2 Spot: - Service-Linked Role for EC2 Spot This is only necessary if this is the first time you're using EC2 Spot in the account. Since this is a one-time operation, we recommend you do this manually via the AWS CLI: ```bash aws --profile --gbl--admin iam create-service-linked-role --aws-service-name spot.amazonaws.com ``` Note that if the Service-Linked Roles already exist in the AWS account (if you used EC2 Spot or Spot Fleet before), and you try to provision them again, you will see the following errors: ```text An error occurred (InvalidInput) when calling the CreateServiceLinkedRole operation: Service role name AWSServiceRoleForEC2Spot has been taken in this account, please try a different suffix ``` For more details, see: - https://docs.aws.amazon.com/batch/latest/userguide/spot_fleet_IAM_role.html - https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html The process of provisioning Karpenter on an EKS cluster consists of 3 steps. ### 1. Provision EKS IAM Role for Nodes Launched by Karpenter :::note #### VPC assumptions being made We assume you've already created a VPC using our [VPC component](https://github.com/cloudposse-terraform-components/aws-eks-karpenter-controller/tree/main/modules/vpc) and have private subnets already set up. The Karpenter node pools will be launched in the private subnets. ::: EKS IAM Role for Nodes launched by Karpenter are provisioned by the `eks/cluster` component. (EKS can also provision a Fargate Profile for Karpenter, but deploying Karpenter to Fargate is not recommended.): ```yaml components: terraform: eks/cluster-blue: metadata: component: eks/cluster inherits: - eks/cluster vars: karpenter_iam_role_enabled: true ``` :::note The AWS Auth API for EKS is used to authorize the Karpenter controller to interact with the EKS cluster. ::: Karpenter is installed using a Helm chart. The Helm chart installs the Karpenter controller and a webhook pod as a Deployment that needs to run before the controller can be used for scaling your cluster. We recommend a minimum of one small node group with at least one worker node. As an alternative, you can run these pods on EKS Fargate by creating a Fargate profile for the karpenter namespace. Doing so will cause all pods deployed into this namespace to run on EKS Fargate. Do not run Karpenter on a node that is managed by Karpenter. See [Run Karpenter Controller...](https://aws.github.io/aws-eks-best-practices/karpenter/#run-the-karpenter-controller-on-eks-fargate-or-on-a-worker-node-that-belongs-to-a-node-group) for more details. We provision IAM Role for Nodes launched by Karpenter because they must run with an Instance Profile that grants permissions necessary to run containers and configure networking. We define the IAM role for the Instance Profile in `components/terraform/eks/cluster/controller-policy.tf`. Note that we provision the EC2 Instance Profile for the Karpenter IAM role in the `components/terraform/eks/karpenter` component (see the next step). Run the following commands to provision the EKS Instance Profile for Karpenter and the IAM role for instances launched by Karpenter on the blue EKS cluster and add the role ARNs to the EKS Auth API: ```bash atmos terraform plan eks/cluster-blue -s plat-ue2-dev atmos terraform apply eks/cluster-blue -s plat-ue2-dev ``` For more details, refer to: - [Getting started with Terraform](https://aws-ia.github.io/terraform-aws-eks-blueprints/getting-started/) - [Getting started with `eksctl`](https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/) ### 2. Provision `karpenter` component In this step, we provision the `components/terraform/eks/karpenter` component, which deploys the following resources: - Karpenter CustomerResourceDefinitions (CRDs) using the Karpenter CRD Chart and the `helm_release` Terraform resource - Karpenter Kubernetes controller using the Karpenter Helm Chart and the `helm_release` Terraform resource - EKS IAM role for Kubernetes Service Account for the Karpenter controller (with all the required permissions) - An SQS Queue and Event Bridge rules for handling Node Interruption events (i.e. Spot) Create a stack config for the blue Karpenter component in `stacks/catalog/eks/clusters/blue.yaml`: ```yaml eks/karpenter-blue: metadata: component: eks/karpenter inherits: - eks/karpenter vars: eks_component_name: eks/cluster-blue ``` Run the following commands to provision the Karpenter component on the blue EKS cluster: ```bash atmos terraform plan eks/karpenter-blue -s plat-ue2-dev atmos terraform apply eks/karpenter-blue -s plat-ue2-dev ``` ### 3. Provision `karpenter-node-pool` component In this step, we provision the `components/terraform/eks/karpenter-node-pool` component, which deploys Karpenter [NodePools](https://karpenter.sh/v0.36/getting-started/getting-started-with-karpenter/#5-create-nodepool) using the `kubernetes_manifest` resource. :::tip #### Why use a separate component for NodePools? We create the NodePools as a separate component since the CRDs for the NodePools are created by the Karpenter component. This helps manage dependencies. ::: First, create an abstract component for the `eks/karpenter-node-pool` component: ```yaml components: terraform: eks/karpenter-node-pool: metadata: type: abstract vars: enabled: true # Disabling Manifest Experiment disables stored metadata with Terraform state # Otherwise, the state will show changes on all plans helm_manifest_experiment_enabled: false node_pools: default: # Whether to place EC2 instances launched by Karpenter into VPC private subnets. Set it to `false` to use public subnets private_subnets_enabled: true # You can use disruption to set the maximum instance lifetime for the EC2 instances launched by Karpenter. # You can also configure how fast or slow Karpenter should add/remove nodes. # See more: https://karpenter.sh/v0.36/concepts/disruption/ disruption: max_instance_lifetime: "336h" # 14 days # Taints can be used to prevent pods without the right tolerations from running on this node pool. # See more: https://karpenter.sh/v0.36/concepts/nodepools/#taints taints: [] total_cpu_limit: "1k" # Karpenter node pool total memory limit for all pods running on the EC2 instances launched by Karpenter total_memory_limit: "1200Gi" # Set acceptable (In) and unacceptable (Out) Kubernetes and Karpenter values for node provisioning based on # Well-Known Labels and cloud-specific settings. These can include instance types, zones, computer architecture, # and capacity type (such as AWS spot or on-demand). # See https://karpenter.sh/v0.36/concepts/nodepools/#spectemplatespecrequirements for more details requirements: - key: "karpenter.sh/capacity-type" operator: "In" # See https://karpenter.sh/docs/concepts/nodepools/#capacity-type # Allow fallback to on-demand instances when spot instances are unavailable # By default, Karpenter uses the "price-capacity-optimized" allocation strategy # https://aws.amazon.com/blogs/compute/introducing-price-capacity-optimized-allocation-strategy-for-ec2-spot-instances/ # It is currently not configurable, but that may change in the future. # See https://github.com/aws/karpenter-provider-aws/issues/1240 values: - "on-demand" - "spot" - key: "kubernetes.io/os" operator: "In" values: - "linux" - key: "kubernetes.io/arch" operator: "In" values: - "amd64" # The following two requirements pick instances such as c3 or m5 - key: karpenter.k8s.aws/instance-category operator: In values: ["c", "m", "r"] - key: karpenter.k8s.aws/instance-generation operator: Gt values: ["2"] ``` Now, create the stack config for the blue Karpenter NodePool component in `stacks/catalog/eks/clusters/blue.yaml`: ```yaml eks/karpenter-node-pool/blue: metadata: component: eks/karpenter-node-pool inherits: - eks/karpenter-node-pool vars: eks_component_name: eks/cluster-blue ``` Finally, run the following commands to deploy the Karpenter NodePools on the blue EKS cluster: ```bash atmos terraform plan eks/karpenter-node-pool/blue -s plat-ue2-dev atmos terraform apply eks/karpenter-node-pool/blue -s plat-ue2-dev ``` ## Node Interruption Karpenter also supports listening for and responding to Node Interruption events. If interruption handling is enabled, Karpenter will watch for upcoming involuntary interruption events that would cause disruption to your workloads. These interruption events include: - Spot Interruption Warnings - Scheduled Change Health Events (Maintenance Events) - Instance Terminating Events - Instance Stopping Events :::tip #### Interruption Handler vs. Termination Handler The Node Interruption Handler is not the same as the Node Termination Handler. The latter is always enabled and cleanly shuts down the node in 2 minutes in response to a Node Termination event. The former gets advance notice that a node will soon be terminated, so it can have 5-10 minutes to shut down a node. ::: For more details, see refer to the [Karpenter docs](https://karpenter.sh/v0.32/concepts/disruption/#interruption) and [FAQ](https://karpenter.sh/v0.32/faq/#interruption-handling) To enable Node Interruption handling, set `var.interruption_handler_enabled` to `true`. This will create an SQS queue and a set of Event Bridge rules to deliver interruption events to Karpenter. ## Custom Resource Definition (CRD) Management Karpenter ships with a few Custom Resource Definitions (CRDs). In earlier versions of this component, when installing a new version of the `karpenter` helm chart, CRDs were not be upgraded at the same time, requiring manual steps to upgrade CRDs after deploying the latest chart. However Karpenter now supports an additional, independent helm chart for CRD management. This helm chart, `karpenter-crd`, can be installed alongside the `karpenter` helm chart to automatically manage the lifecycle of these CRDs. To deploy the `karpenter-crd` helm chart, set `var.crd_chart_enabled` to `true`. (Installing the `karpenter-crd` chart is recommended. `var.crd_chart_enabled` defaults to `false` to preserve backward compatibility with older versions of this component.) ## Troubleshooting For Karpenter issues, checkout the [Karpenter Troubleshooting Guide](https://karpenter.sh/docs/troubleshooting/) ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended
`chart_repository` (`string`) required
Repository URL where to locate the requested chart
`region` (`string`) required
AWS Region
`resources` required
The CPU and memory of the deployment's limits and requests **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
### Optional Variables
`additional_settings` (`any`) optional
Additional settings to merge into the Karpenter controller settings. This is useful for setting featureGates or other advanced settings that may vary by chart version. These settings will be merged with the base settings and take precedence over any conflicting keys. Example: additional_settings = \{ featureGates = \{ nodeRepair = false reservedCapacity = true spotToSpotConsolidation = false \} \} **Default value:** `{ }`
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history) **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails **Default value:** `true`
`crd_chart` (`string`) optional
The name of the Karpenter CRD chart to be installed, if `var.crd_chart_enabled` is set to `true`. **Default value:** `"karpenter-crd"`
`crd_chart_enabled` (`bool`) optional
`karpenter-crd` can be installed as an independent helm chart to manage the lifecycle of Karpenter CRDs. Set to `true` to install this CRD helm chart before the primary karpenter chart. **Default value:** `false`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`interruption_handler_enabled` (`bool`) optional
If `true`, deploy a SQS queue and Event Bridge rules to enable interruption handling by Karpenter. https://karpenter.sh/docs/concepts/disruption/#interruption **Default value:** `true`
`interruption_queue_message_retention` (`number`) optional
The message retention in seconds for the interruption handler SQS queue. **Default value:** `300`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`logging` optional
A subset of the logging settings for the Karpenter controller **Type:** ```hcl object({ enabled = optional(bool, true) level = optional(object({ controller = optional(string, "info") global = optional(string, "info") webhook = optional(string, "error") }), {}) }) ``` **Default value:** `{ }`
`metrics_enabled` (`bool`) optional
Whether to expose the Karpenter's Prometheus metric **Default value:** `true`
`metrics_port` (`number`) optional
Container port to use for metrics **Default value:** `8080`
`replicas` (`number`) optional
The number of Karpenter controller replicas to run **Default value:** `2`
`settings` optional
A subset of the settings for the Karpenter controller. Some settings are implicitly set by this component, such as `clusterName` and `interruptionQueue`. All settings can be overridden by providing a `settings` section in the `chart_values` variable. The settings provided here are the ones mostly likely to be set to other than default values, and are provided here for convenience. **Type:** ```hcl object({ batch_idle_duration = optional(string, "1s") batch_max_duration = optional(string, "10s") }) ``` **Default value:** `{ }`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `karpenter` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | Deploy Karpenter helm chart `karpenter_crd` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | Deploy karpenter-crd helm chart "karpenter-crd" can be installed as an independent helm chart to manage the lifecycle of Karpenter CRDs `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.interruption_handler`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.interruption_handler`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_policy.v1alpha`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role_policy_attachment.v1alpha`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_sqs_queue.interruption_handler`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) (resource) - [`aws_sqs_queue_policy.interruption_handler`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_iam_policy_document.interruption_handler`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## karpenter-node-pool This component deploys [Karpenter NodePools](https://karpenter.sh/docs/concepts/nodepools/) to an EKS cluster. Karpenter is still in v0 and rapidly evolving. At this time, this component only supports a subset of the features available in Karpenter. Support could be added for additional features as needed. Not supported: - Elements of NodePool: - [`template.spec.kubelet`](https://karpenter.sh/docs/concepts/nodepools/#spectemplatespeckubelet) - [`limits`](https://karpenter.sh/docs/concepts/nodepools/#limits) currently only supports `cpu` and `memory`. Other limits such as `nvidia.com/gpu` are not supported. - Elements of NodeClass: - `subnetSelectorTerms`. This component only supports selecting all public or all private subnets of the referenced EKS cluster. - `securityGroupSelectorTerms`. This component only supports selecting the security group of the referenced EKS cluster. - `amiSelectorTerms`. Such terms override the `amiFamily` setting, which is the only AMI selection supported by this component. - `instanceStorePolicy` - `userData` - `detailedMonitoring` - `associatePublicIPAddress` ## Usage **Stack Level**: Regional If provisioning more than one NodePool, it is [best practice](https://aws.github.io/aws-eks-best-practices/karpenter/#creating-nodepools) to create NodePools that are mutually exclusive or weighted. ```yaml components: terraform: eks/karpenter-node-pool: settings: spacelift: workspace_enabled: true vars: enabled: true eks_component_name: eks/cluster name: "karpenter-node-pool" # https://karpenter.sh/v0.36.0/docs/concepts/nodepools/ node_pools: default: name: default # Whether to place EC2 instances launched by Karpenter into VPC private subnets. Set it to `false` to use public subnets private_subnets_enabled: true disruption: consolidation_policy: WhenUnderutilized consolidate_after: 1h max_instance_lifetime: 336h budgets: # This budget allows 0 disruptions during business hours (from 9am to 5pm) on weekdays - schedule: "0 9 * * mon-fri" duration: 8h nodes: "0" # The total cpu of the cluster. Maps to spec.limits.cpu in the Karpenter NodeClass total_cpu_limit: "100" # The total memory of the cluster. Maps to spec.limits.memory in the Karpenter NodeClass total_memory_limit: "1000Gi" # The weight of the node pool. See https://karpenter.sh/docs/concepts/scheduling/#weighted-nodepools weight: 50 # Taints to apply to the nodes in the node pool. See https://karpenter.sh/docs/concepts/nodeclasses/#spectaints taints: - key: "node.kubernetes.io/unreachable" effect: "NoExecute" value: "true" # Taints to apply to the nodes in the node pool at startup. See https://karpenter.sh/docs/concepts/nodeclasses/#specstartuptaints startup_taints: - key: "node.kubernetes.io/unreachable" effect: "NoExecute" value: "true" # Metadata options for the node pool. See https://karpenter.sh/docs/concepts/nodeclasses/#specmetadataoptions metadata_options: httpEndpoint: "enabled" # allows the node to call the AWS metadata service httpProtocolIPv6: "disabled" httpPutResponseHopLimit: 2 httpTokens: "required" # The AMI used by Karpenter provisioner when provisioning nodes. Based on the value set for amiFamily, Karpenter will automatically query for the appropriate EKS optimized AMI via AWS Systems Manager (SSM) # Bottlerocket, AL2, Ubuntu # https://karpenter.sh/v0.18.0/aws/provisioning/#amazon-machine-image-ami-family ami_family: AL2 # Karpenter provisioner block device mappings. block_device_mappings: - deviceName: /dev/xvda ebs: volumeSize: 200Gi volumeType: gp3 encrypted: true deleteOnTermination: true # Set acceptable (In) and unacceptable (Out) Kubernetes and Karpenter values for node provisioning based on # Well-Known Labels and cloud-specific settings. These can include instance types, zones, computer architecture, # and capacity type (such as AWS spot or on-demand). # See https://karpenter.sh/v0.18.0/provisioner/#specrequirements for more details requirements: - key: "karpenter.sh/capacity-type" operator: "In" values: - "on-demand" - "spot" - key: "node.kubernetes.io/instance-type" operator: "In" # See https://aws.amazon.com/ec2/instance-explorer/ and https://aws.amazon.com/ec2/instance-types/ # Values limited by DenyEC2InstancesWithoutEncryptionInTransit service control policy # See https://github.com/cloudposse/terraform-aws-service-control-policies/blob/master/catalog/ec2-policies.yaml # Karpenter recommends allowing at least 20 instance types to ensure availability. values: - "c5n.2xlarge" - "c5n.xlarge" - "c5n.large" - "c6i.2xlarge" - "c6i.xlarge" - "c6i.large" - "m5n.2xlarge" - "m5n.xlarge" - "m5n.large" - "m5zn.2xlarge" - "m5zn.xlarge" - "m5zn.large" - "m6i.2xlarge" - "m6i.xlarge" - "m6i.large" - "r5n.2xlarge" - "r5n.xlarge" - "r5n.large" - "r6i.2xlarge" - "r6i.xlarge" - "r6i.large" - key: "kubernetes.io/arch" operator: "In" values: - "amd64" ``` ## Variables ### Required Variables
`node_pools` required
Configuration for node pools. See code for details. **Type:** ```hcl map(object({ # The name of the Karpenter provisioner. The map key is used if this is not set. name = optional(string) # Whether to place EC2 instances launched by Karpenter into VPC private subnets. Set it to `false` to use public subnets. private_subnets_enabled = bool # The Disruption spec controls how Karpenter scales down the node group. # See the example (sadly not the specific `spec.disruption` documentation) at https://karpenter.sh/docs/concepts/nodepools/ for details disruption = optional(object({ # Describes which types of Nodes Karpenter should consider for consolidation. # If using 'WhenUnderutilized', Karpenter will consider all nodes for consolidation and attempt to remove or # replace Nodes when it discovers that the Node is underutilized and could be changed to reduce cost. # If using `WhenEmpty`, Karpenter will only consider nodes for consolidation that contain no workload pods. consolidation_policy = optional(string, "WhenUnderutilized") # The amount of time Karpenter should wait after discovering a consolidation decision (`go` duration string, s, m, or h). # This value can currently (v0.36.0) only be set when the consolidationPolicy is 'WhenEmpty'. # You can choose to disable consolidation entirely by setting the string value 'Never' here. # Earlier versions of Karpenter called this field `ttl_seconds_after_empty`. consolidate_after = optional(string) # The amount of time a Node can live on the cluster before being removed (`go` duration string, s, m, or h). # You can choose to disable expiration entirely by setting the string value 'Never' here. # This module sets a default of 336 hours (14 days), while the Karpenter default is 720 hours (30 days). # Note that Karpenter calls this field "expiresAfter", and earlier versions called it `ttl_seconds_until_expired`, # but we call it "max_instance_lifetime" to match the corresponding field in EC2 Auto Scaling Groups. max_instance_lifetime = optional(string, "336h") # Budgets control the the maximum number of NodeClaims owned by this NodePool that can be terminating at once. # See https://karpenter.sh/docs/concepts/disruption/#disruption-budgets for details. # A percentage is the percentage of the total number of active, ready nodes not being deleted, rounded up. # If there are multiple active budgets, Karpenter uses the most restrictive value. # If left undefined, this will default to one budget with a value of nodes: 10%. # Note that budgets do not prevent or limit involuntary terminations. # Example: # On Weekdays during business hours, don't do any deprovisioning. # budgets = { # schedule = "0 9 * * mon-fri" # duration = 8h # nodes = "0" # } budgets = optional(list(object({ # The schedule specifies when a budget begins being active, using extended cronjob syntax. # See https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#schedule-syntax for syntax details. # Timezones are not supported. This field is required if Duration is set. schedule = optional(string) # Duration determines how long a Budget is active after each Scheduled start. # If omitted, the budget is always active. This is required if Schedule is set. # Must be a whole number of minutes and hours, as cron does not work in seconds, # but since Go's `duration.String()` always adds a "0s" at the end, that is allowed. duration = optional(string) # The percentage or number of nodes that Karpenter can scale down during the budget. nodes = string # Reasons can be one of Drifted, Underutilized, or Empty # If omitted, it’s assumed that the budget applies to all reasons. # See https://karpenter.sh/v1.1/concepts/disruption/#reasons reasons = optional(list(string)) })), []) }), {}) # Karpenter provisioner total CPU limit for all pods running on the EC2 instances launched by Karpenter total_cpu_limit = string # Karpenter provisioner total memory limit for all pods running on the EC2 instances launched by Karpenter total_memory_limit = string # Set a weight for this node pool. # See https://karpenter.sh/docs/concepts/scheduling/#weighted-nodepools weight = optional(number, 50) labels = optional(map(string)) annotations = optional(map(string)) # Karpenter provisioner taints configuration. See https://aws.github.io/aws-eks-best-practices/karpenter/#create-provisioners-that-are-mutually-exclusive for more details taints = optional(list(object({ key = string effect = string value = optional(string) }))) startup_taints = optional(list(object({ key = string effect = string value = optional(string) }))) # Karpenter node metadata options. See https://karpenter.sh/docs/concepts/nodeclasses/#specmetadataoptions for more details metadata_options = optional(object({ httpEndpoint = optional(string, "enabled") httpProtocolIPv6 = optional(string, "disabled") httpPutResponseHopLimit = optional(number, 2) # httpTokens can be either "required" or "optional" httpTokens = optional(string, "required") }), {}) # ami_family dictates the default bootstrapping logic. # It is only required if you do not specify amiSelectorTerms.alias ami_family = optional(string, null) # Selectors for the AMI used by Karpenter provisioner when provisioning nodes. # Usually use { alias = "@latest" } but version can be pinned instead of "latest". # Based on the ami_selector_terms, Karpenter will automatically query for the appropriate EKS optimized AMI via AWS Systems Manager (SSM) ami_selector_terms = list(any) # Karpenter nodes block device mappings. Controls the Elastic Block Storage volumes that Karpenter attaches to provisioned nodes. # Karpenter uses default block device mappings for the AMI Family specified. # For example, the Bottlerocket AMI Family defaults with two block device mappings, # and normally you only want to scale `/dev/xvdb` where Containers and there storage are stored. # Most other AMIs only have one device mapping at `/dev/xvda`. # See https://karpenter.sh/docs/concepts/nodeclasses/#specblockdevicemappings for more details block_device_mappings = list(object({ deviceName = string ebs = optional(object({ volumeSize = string volumeType = string deleteOnTermination = optional(bool, true) encrypted = optional(bool, true) iops = optional(number) kmsKeyID = optional(string, "alias/aws/ebs") snapshotID = optional(string) throughput = optional(number) })) })) # Set acceptable (In) and unacceptable (Out) Kubernetes and Karpenter values for node provisioning based on Well-Known Labels and cloud-specific settings. These can include instance types, zones, computer architecture, and capacity type (such as AWS spot or on-demand). See https://karpenter.sh/v0.18.0/provisioner/#specrequirements for more details requirements = list(object({ key = string operator = string # Operators like "Exists" and "DoesNotExist" do not require a value values = optional(list(string)) })) # Any values for spec.template.spec.kubelet allowed by Karpenter. # Not fully specified, because they are subject to change. # See: # https://karpenter.sh/docs/concepts/nodepools/#spectemplatespeckubelet # https://kubernetes.io/docs/reference/config-api/kubelet-config.v1beta1/ kubelet = optional(any, {}) })) ```
`region` (`string`) required
AWS Region
### Optional Variables
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`import_profile_name` (`string`) optional
AWS Profile name to use when importing a resource **Default value:** `null`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ec2_node_classes`
Deployed Karpenter EC2NodeClass
`node_pools`
Deployed Karpenter NodePool
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_manifest.ec2_node_class`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) (resource) - [`kubernetes_manifest.node_pool`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## keda This component is used to install the KEDA operator. See this overview of how KEDA works with triggers with a `ScaledObject`, which is a light wrapper around HPAs: https://keda.sh/docs/2.9/concepts/scaling-deployments/#overview ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: eks/keda: vars: enabled: true name: keda create_namespace: true kubernetes_namespace: "keda" chart_repository: "https://kedacore.github.io/charts" chart: "keda" chart_version: "2.13.2" chart_values: {} timeout: 180 ``` ## Variables ### Required Variables
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"keda"`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"2.8"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"Used for autoscaling from external metrics configured as triggers."`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://kedacore.github.io/charts"`
`resources` (`any`) optional
A sub-nested map of deployment to resources. e.g. \{ operator = \{ requests = \{ cpu = 100m, memory = 100Mi \}, limits = \{ cpu = 200m, memory = 200Mi \} \} \} **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release.
`service_account_name`
Kubernetes Service Account name
`service_account_namespace`
Kubernetes Service Account namespace
`service_account_policy_arn`
IAM policy ARN
`service_account_policy_id`
IAM policy ID
`service_account_policy_name`
IAM policy name
`service_account_role_arn`
IAM role ARN
`service_account_role_name`
IAM role name
`service_account_role_unique_id`
IAM role unique ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.6.0` - `kubernetes`, version: `>= 2.9.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `keda` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## loki Grafana Loki is a set of resources that can be combined into a fully featured logging stack. Unlike other logging systems, Loki is built around the idea of only indexing metadata about your logs: labels (just like Prometheus labels). Log data itself is then compressed and stored in chunks in object stores such as S3 or GCS, or even locally on a filesystem. This component deploys the [grafana/loki](https://github.com/grafana/loki/tree/main/production/helm/loki) helm chart. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: eks/loki: vars: enabled: true name: loki alb_controller_ingress_group_component_name: eks/alb-controller-ingress-group/internal ``` :::important We recommend using an internal ALB for logging services. You must connect to the private network to access the Loki endpoint. ::: ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`additional_schema_config` optional
A list of additional `configs` for the `schemaConfig` for the Loki chart. This list will be merged with the default schemaConfig.config defined by `var.default_schema_config` **Type:** ```hcl list(object({ from = string object_store = string schema = string store = string index = object({ prefix = string period = string }) })) ``` **Default value:** `[ ]`
`alb_controller_ingress_group_component_name` (`string`) optional
The name of the eks/alb-controller-ingress-group component. This should be an internal facing ALB **Default value:** `"eks/alb-controller-ingress-group"`
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`basic_auth_enabled` (`bool`) optional
If `true`, enabled Basic Auth for the Ingress service. A user and password will be created and stored in AWS SSM. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"loki"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus."`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://grafana.github.io/helm-charts"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`default_schema_config` optional
A list of default `configs` for the `schemaConfig` for the Loki chart. For new installations, the default schema config doesn't change. See https://grafana.com/docs/loki/latest/operations/storage/schema/#new-loki-installs **Type:** ```hcl list(object({ from = string object_store = string schema = string store = string index = object({ prefix = string period = string }) })) ``` **Default value:** ```hcl [ { "from": "2024-04-01", "index": { "period": "24h", "prefix": "index_" }, "object_store": "s3", "schema": "v13", "store": "tsdb" } ] ```
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes kube config file **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Kubernetes namespace to install the release into **Default value:** `"monitoring"`
`ssm_path_template` (`string`) optional
A string template to be used to create paths in AWS SSM to store basic auth credentials for this service **Default value:** `"/%s/basic-auth/%s"`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `300`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`basic_auth_username`
If enabled, the username for basic auth
`id`
The ID of this deployment
`metadata`
Block status of the deployed release
`ssm_path_basic_auth_password`
If enabled, the path in AWS SSM to find the password for basic auth
`url`
The hostname used for this Loki deployment
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` - `random`, version: `>= 2.3` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb_controller_ingress_group` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `basic_auth_ssm_parameters` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `loki` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `loki_storage` | 4.10.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.10.0) | n/a `loki_tls_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`random_pet.basic_auth_username`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) - [`random_string.basic_auth_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## metrics-server This component creates a Helm release for [metrics-server](https://github.com/kubernetes-sigs/metrics-server) is a Kubernetes addon that provides resource usage metrics used in particular by other addons such Horizontal Pod Autoscaler. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/metrics-server ... ``` The default catalog values `e.g. stacks/catalog/eks/metrics-server.yaml` ```yaml components: terraform: metrics-server: backend: s3: workspace_key_prefix: metrics-server vars: enabled: true chart_version: 5.10.4 rbac_enabled: true # You can use `chart_values` to set any other chart options. Treat `chart_values` as the root of the doc. # # # For example # --- # chart_values: # enableShield: false chart_values: {} ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"metrics-server"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://kubernetes-sigs.github.io/metrics-server/"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"3.11.0"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `true`. **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the EKS component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
The namespace to install the release into. **Default value:** `"metrics-server"`
`metrics_server_component` (`string`) optional
The name of the Metrics Server component **Default value:** `"eks-metrics-server"`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`resources` optional
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ``` **Default value:** ```hcl { "limits": { "cpu": "100m", "memory": "300Mi" }, "requests": { "cpu": "20m", "memory": "60Mi" } } ```
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.14.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `metrics_server` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## node-termination-handler This component creates a Helm release for [aws-node-termination-handler](https://github.com/aws/aws-node-termination-handler) on a Kubernetes cluster. [aws-node-termination-handler](https://github.com/aws/aws-node-termination-handler) is a Kubernetes addon that (by default) monitors the EC2 IMDS endpoint for scheduled maintenance events, spot instance termination events, and rebalance recommendation events, and drains and/or cordons nodes upon such events. This ensures that workloads on Kubernetes are evicted gracefully when a node needs to be terminated. ## Usage **Stack Level**: Regional Once the catalog file is created, the file can be imported as follows. ```yaml import: - catalog/eks/aws-node-termination-handler ... ``` The default catalog values ```yaml components: terraform: aws-node-termination-handler: backend: s3: workspace_key_prefix: aws-node-termination-handler vars: enabled: true chart_version: 0.15.3 rbac_enabled: true # You can use `chart_values` to set any other chart options. Treat `chart_values` as the root of the doc. # # # For example # --- # chart_values: # enableShield: false chart_values: {} ``` ## Variables ### Required Variables
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"aws-node-termination-handler"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://aws.github.io/eks-charts"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `"0.15.3"`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`resources` optional
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ``` **Default value:** ```hcl { "limits": { "cpu": "100m", "memory": "128Mi" }, "requests": { "cpu": "50m", "memory": "64Mi" } } ```
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_node_termination_handler` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## prometheus-scraper This component provisions the an Amazon Managed collector or scraper to connect Amazon Managed Prometheus (AMP) with an EKS cluster. A common use case for Amazon Managed Service for Prometheus is to monitor Kubernetes clusters managed by Amazon Elastic Kubernetes Service (Amazon EKS). Kubernetes clusters, and many applications that run within Amazon EKS, automatically export their metrics for Prometheus-compatible scrapers to access. Amazon Managed Service for Prometheus provides a fully managed, agentless scraper, or collector, that automatically discovers and pulls Prometheus-compatible metrics. You don't have to manage, install, patch, or maintain agents or scrapers. An Amazon Managed Service for Prometheus collector provides reliable, stable, highly available, automatically scaled collection of metrics for your Amazon EKS cluster. Amazon Managed Service for Prometheus managed collectors work with Amazon EKS clusters, including EC2 and Fargate. An Amazon Managed Service for Prometheus collector creates an Elastic Network Interface (ENI) per subnet specified when creating the scraper. The collector scrapes the metrics through these ENIs, and uses remote_write to push the data to your Amazon Managed Service for Prometheus workspace using a VPC endpoint. The scraped data never travels on the public internet. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: eks/prometheus-scraper: vars: enabled: true name: prometheus-scraper # This refers to the `managed-prometheus/workspace` Terraform component, # but the component name can be whatever you choose to name the stack component prometheus_component_name: prometheus ``` ### Authenticating with EKS In order for this managed collector to authenticate with the EKS cluster, update auth map after deploying. Note the `scraper_role_arn` and `clusterrole_username` outputs and set them to `rolearn` and `username` respectively with the `map_additional_iam_roles` input for `eks/cluster`. ```yaml components: terraform: eks/cluster: vars: map_additional_iam_roles: # this role is used to grant the Prometheus scraper access to this cluster. See eks/prometheus-scraper - rolearn: "arn:aws:iam::111111111111:role/AWSServiceRoleForAmazonPrometheusScraper_111111111111111" username: "acme-plat-ue2-sandbox-prometheus-scraper" groups: [] ``` Then reapply the given cluster component. ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"AWS Managed Prometheus (AMP) scrapper roles and role bindings"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`eks_scrape_configuration` (`string`) optional
Scrape configuration for the agentless scraper that will installed with EKS integrations **Default value:** `"global:\n scrape_interval: 30s\nscrape_configs:\n # pod metrics\n - job_name: pod_exporter\n kubernetes_sd_configs:\n - role: pod\n # container metrics\n - job_name: cadvisor\n scheme: https\n authorization:\n credentials_file: /var/run/secrets/kubernetes.io/serviceaccount/token\n kubernetes_sd_configs:\n - role: node\n relabel_configs:\n - action: labelmap\n regex: __meta_kubernetes_node_label_(.+)\n - replacement: kubernetes.default.svc:443\n target_label: __address__\n - source_labels: [__meta_kubernetes_node_name]\n regex: (.+)\n target_label: __metrics_path__\n replacement: /api/v1/nodes/$1/proxy/metrics/cadvisor\n # apiserver metrics\n - bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token\n job_name: kubernetes-apiservers\n kubernetes_sd_configs:\n - role: endpoints\n relabel_configs:\n - action: keep\n regex: default;kubernetes;https\n source_labels:\n - __meta_kubernetes_namespace\n - __meta_kubernetes_service_name\n - __meta_kubernetes_endpoint_port_name\n scheme: https\n # kube proxy metrics\n - job_name: kube-proxy\n honor_labels: true\n kubernetes_sd_configs:\n - role: pod\n relabel_configs:\n - action: keep\n source_labels:\n - __meta_kubernetes_namespace\n - __meta_kubernetes_pod_name\n separator: '/'\n regex: 'kube-system/kube-proxy.+'\n - source_labels:\n - __address__\n action: replace\n target_label: __address__\n regex: (.+?)(\\\\:\\\\d+)?\n replacement: $1:10249\n - job_name: 'kubernetes-pods'\n honor_labels: true\n kubernetes_sd_configs:\n - role: pod\n relabel_configs:\n - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]\n action: keep\n regex: true\n - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]\n action: replace\n target_label: __metrics_path__\n regex: (.+)\n - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]\n action: replace\n regex: ([^:]+)(?::\\d+)?;(\\d+)\n replacement: $1:$2\n target_label: __address__\n - action: labelmap\n regex: __meta_kubernetes_pod_label_(.+)\n - source_labels: [__meta_kubernetes_namespace]\n action: replace\n target_label: kubernetes_namespace\n - source_labels: [__meta_kubernetes_pod_name]\n action: replace\n target_label: kubernetes_pod_name\n"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes kube config file **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Kubernetes namespace to install the release into **Default value:** `"kube-system"`
`prometheus_component_name` (`string`) optional
The name of the Amazon Managed Prometheus workspace component **Default value:** `"managed-prometheus/workspace"`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `300`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`vpc_component_name` (`string`) optional
The name of the vpc component **Default value:** `"vpc"`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`clusterrole_username`
The username of the ClusterRole used to give the scraper in-cluster permissions
`scraper_role_arn`
The Amazon Resource Name (ARN) of the IAM role that provides permissions for the scraper to discover, collect, and produce metrics
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `prometheus` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `scraper_access` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_prometheus_scraper.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_scraper) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## promtail Promtail is an agent which ships the contents of local logs to a Loki instance. This component deploys the [grafana/promtail](https://github.com/grafana/helm-charts/tree/main/charts/promtail) Helm chart and expects `eks/loki` to be deployed. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: eks/promtail: vars: enabled: true name: promtail ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alb_controller_ingress_group_component_name` (`string`) optional
The name of the eks/alb-controller-ingress-group component. This should be an internal facing ALB **Default value:** `"eks/alb-controller-ingress-group"`
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart` (`string`) optional
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended. **Default value:** `"promtail"`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `"Promtail is an agent which ships the contents of local logs to a Loki instance"`
`chart_repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `"https://grafana.github.io/helm-charts"`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes kube config file **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Kubernetes namespace to install the release into **Default value:** `"monitoring"`
`loki_component_name` (`string`) optional
The name of the eks/loki component **Default value:** `"eks/loki"`
`push_api` optional
Describes and configures Promtail to expose a Loki push API server with an Ingress configuration. - enabled: Set this to `true` to enable this feature - scrape_config: Optional. This component includes a basic configuration by default, or override the default configuration here. **Type:** ```hcl object({ enabled = optional(bool, false) scrape_config = optional(string, "") }) ``` **Default value:** `{ }`
`scrape_configs` (`list(string)`) optional
A list of local path paths starting with this component's base path for Promtail Scrape Configs **Default value:** ```hcl [ "scrape_config/default_kubernetes_pods.yaml" ] ```
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `300`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart **Default value:** `false`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb_controller_ingress_group` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `chart_values` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `loki` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `promtail` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_ssm_parameter.basic_auth_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## redis This component installs `redis` for EKS clusters. This is a Self Hosted Redis Cluster installed on EKS. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. `stacks/catalog/eks/redis/defaults` file (base component for default Redis settings): ```yaml components: terraform: eks/redis/defaults: metadata: component: eks/redis type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true name: redis tags: Team: sre Service: redis create_namespace: true kubernetes_namespace: "redis" # https://github.com/bitnami/charts/tree/master/bitnami/redis chart_repository: https://charts.bitnami.com/bitnami chart_version: "17.1.0" chart: "redis" timeout: 180 resources: limits: cpu: 200m memory: 256Mi requests: cpu: 100m memory: 128Mi # Set a specific version of Redis using the image.tag and image.repository values. # Documentation: https://docs.bitnami.com/kubernetes/infrastructure/redis/configuration/change-image-version/ # Defaults: https://github.com/bitnami/charts/blob/master/bitnami/redis/values.yaml#L81-L82 chart_values: image: tag: 7.0.4-debian-11-r11 repository: bitnami/redis # Disabling Manifest Experiment disables stored metadata with Terraform state # Otherwise, the state will show changes on all plans helm_manifest_experiment_enabled: false ``` `stacks/catalog/eks/redis/dev` file (derived component for "dev" specific settings): ```yaml import: - catalog/eks/redis/defaults components: terraform: eks/redis/dev: metadata: component: eks/redis inherits: - eks/redis/defaults vars: {} ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
`resources` required
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `redis` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## redis-operator This component installs `redis-operator` for EKS clusters. Redis Operator creates/configures/manages high availability redis with sentinel automatic failover atop Kubernetes. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. `stacks/catalog/eks/redis-operator/defaults` file (base component for default redis-operator settings): ```yaml components: terraform: eks/redis-operator/defaults: metadata: component: eks/redis-operator type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true name: redis-operator tags: Team: sre Service: redis-operator create_namespace: true kubernetes_namespace: "redis-operator" # https://github.com/spotahome/redis-operator chart_repository: https://spotahome.github.io/redis-operator chart_version: "3.1.4" chart: "redis-operator" timeout: 180 resources: limits: cpu: 200m memory: 256Mi requests: cpu: 100m memory: 128Mi # Set a specific version of Redis using the image.tag and image.repository values. # Defaults: https://github.com/spotahome/redis-operator/blob/master/charts/redisoperator/values.yaml#L6 chart_values: image: repository: quay.io/spotahome/redis-operator tag: v1.1.1 ``` `stacks/catalog/eks/redis-operator/dev` file (derived component for "dev" specific settings): ```yaml import: - catalog/eks/redis-operator/defaults components: terraform: eks/redis-operator/dev: metadata: component: eks/redis-operator inherits: - eks/redis-operator/defaults vars: {} ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region.
`resources` required
The cpu and memory of the deployment's limits and requests. **Type:** ```hcl object({ limits = object({ cpu = string memory = string }) requests = object({ cpu = string memory = string }) }) ```
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values. **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`rbac_enabled` (`bool`) optional
Service Account for pods. **Default value:** `true`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.0, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `redis_operator` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## reloader This component installs the [Stakater Reloader](https://github.com/stakater/Reloader) for EKS clusters. `reloader` can watch `ConfigMap`s and `Secret`s for changes and use these to trigger rolling upgrades on pods and their associated `DeploymentConfig`s, `Deployment`s, `Daemonset`s `Statefulset`s and `Rollout`s. ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: eks/reloader: settings: spacelift: workspace_enabled: true vars: enabled: true name: reloader create_namespace: true kubernetes_namespace: "reloader" repository: "https://stakater.github.io/stakater-charts" chart: "reloader" chart_version: "v0.0.124" timeout: 180 ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. **Default value:** `true`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. **Default value:** `true`
`create_namespace` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`values` (`any`) optional
YAML-valid specification of values to be passed to the helm_release resource **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.7.1, != 2.21.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`helm_release.this`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## spacelift-worker-pool This component provisions the `WorkerPool` part of the [Kubernetes Operator](https://docs.spacelift.io/concepts/worker-pools/kubernetes-workers#kubernetes-workers) for [Spacelift Worker Pools](https://docs.spacelift.io/concepts/worker-pools#kubernetes) into an EKS cluster. You can provision this component multiple times to create multiple worker pools in a single EKS cluster. ## Usage :::note Before provisioning the `eks/spacelift-worker-pool` component, the `eks/spacelift-worker-pool-controller` component must be provisioned first into an EKS cluster to enable the [Spacelift Worker Pool Kubernetes Controller](https://docs.spacelift.io/concepts/worker-pools#kubernetes). The `eks/spacelift-worker-pool-controller` component must be provisioned only once per EKS cluster. ::: The Spacelift worker needs to pull a Docker image from an ECR repository. It will run the Terraform commands inside the Docker container. In the Cloud Posse reference architecture, this image is the "infra" or "infrastructure" image derived from [Geodesic](https://github.com/cloudposse/geodesic). The worker service account needs permission to pull the image from the ECR repository, and the details of where to find the image are configured in the various `ecr_*` variables. **Stack Level**: Regional ```yaml # stacks/catalog/eks/spacelift-worker-pool/defaults.yaml components: terraform: eks/spacelift-worker-pool: enabled: true name: "spacelift-worker-pool" space_name: root # aws_config_file is the path in the Docker container to the AWS_CONFIG_FILE. # "/etc/aws-config/aws-config-spacelift" is the usual path in the "infrastructure" image. aws_config_file: "/etc/aws-config/aws-config-spacelift" spacelift_api_endpoint: "https://yourcompany.app.spacelift.io" eks_component_name: "eks/cluster" worker_pool_size: 40 kubernetes_namespace: "spacelift-worker-pool" kubernetes_service_account_enabled: true kubernetes_service_account_name: "spacelift-worker-pool" keep_successful_pods: false kubernetes_role_api_groups: [""] kubernetes_role_resources: ["*"] kubernetes_role_resource_names: null kubernetes_role_verbs: ["get", "list"] ecr_component_name: ecr ecr_environment_name: use1 ecr_stage_name: artifacts ecr_tenant_name: core ecr_repo_name: infra ``` ## Variables ### Required Variables
`aws_config_file` (`string`) required
The AWS_CONFIG_FILE used by the worker. Can be overridden by `/.spacelift/config.yml`.
`ecr_repo_name` (`string`) required
ECR repository name
`kubernetes_namespace` (`string`) required
Name of the Kubernetes Namespace the Spacelift worker pool is deployed in to
`region` (`string`) required
AWS Region
`spacelift_api_endpoint` (`string`) required
The Spacelift API endpoint URL (e.g. https://example.app.spacelift.io)
### Optional Variables
`aws_profile` (`string`) optional
The AWS_PROFILE used by the worker. If not specified, `"${var.namespace}-identity"` will be used. Can be overridden by `/.spacelift/config.yml`. **Default value:** `null`
`ecr_component_name` (`string`) optional
ECR component name **Default value:** `"ecr"`
`ecr_environment_name` (`string`) optional
The name of the environment where `ecr` is provisioned **Default value:** `""`
`ecr_stage_name` (`string`) optional
The name of the stage where `ecr` is provisioned **Default value:** `"artifacts"`
`ecr_tenant_name` (`string`) optional
The name of the tenant where `ecr` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`grpc_server_resources` optional
Resources for the gRPC server part of the worker pool deployment. The default values are usually sufficient. **Type:** ```hcl object({ requests = optional(object({ memory = optional(string, "50Mi") cpu = optional(string, "50m") }), {}) limits = optional(object({ memory = optional(string, "500Mi") cpu = optional(string, "500m") }), {}) }) ``` **Default value:** `{ }`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`iam_attributes` (`list(string)`) optional
Additional attributes to add to the IDs of the IAM role and policy **Default value:** `[ ]`
`iam_override_policy_documents` (`list(string)`) optional
List of IAM policy documents that are merged together into the exported document with higher precedence. In merging, statements with non-blank SIDs will override statements with the same SID from earlier documents in the list and from other "source" documents. **Default value:** `null`
`iam_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the IAM Role **Default value:** `null`
`iam_source_json_url` (`string`) optional
IAM source JSON policy to download **Default value:** `null`
`iam_source_policy_documents` (`list(string)`) optional
List of IAM policy documents that are merged together into the exported document. Statements defined in `iam_source_policy_documents` must have unique SIDs. Statements with the same SID as in statements in documents assigned to the `iam_override_policy_documents` arguments will be overridden. **Default value:** `null`
`keep_successful_pods` (`bool`) optional
Indicates whether run Pods should automatically be removed as soon as they complete successfully, or be kept so that they can be inspected later. By default run Pods are removed as soon as they complete successfully. Failed Pods are not automatically removed to allow debugging. **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_role_api_groups` (`list(string)`) optional
List of APIGroups for the Kubernetes Role created for the Kubernetes Service Account **Default value:** ```hcl [ "" ] ```
`kubernetes_role_resource_names` (`list(string)`) optional
List of resource names for the Kubernetes Role created for the Kubernetes Service Account **Default value:** `null`
`kubernetes_role_resources` (`list(string)`) optional
List of resources for the Kubernetes Role created for the Kubernetes Service Account **Default value:** ```hcl [ "*" ] ```
`kubernetes_role_verbs` (`list(string)`) optional
List of verbs that apply to ALL the ResourceKinds for the Kubernetes Role created for the Kubernetes Service Account **Default value:** ```hcl [ "get", "list" ] ```
`kubernetes_service_account_enabled` (`bool`) optional
Flag to enable/disable Kubernetes service account **Default value:** `false`
`kubernetes_service_account_name` (`string`) optional
Kubernetes service account name **Default value:** `null`
`space_name` (`string`) optional
The name of the Spacelift Space to create the worker pool in **Default value:** `"root"`
`worker_pool_description` (`string`) optional
Spacelift worker pool description. The default dynamically includes EKS cluster ID and Spacelift Space name. **Default value:** `null`
`worker_pool_size` (`number`) optional
Worker pool size. The number of workers registered with Spacelift. **Default value:** `1`
`worker_spec` optional
Configuration for the Workers in the worker pool **Type:** ```hcl object({ tmpfs_enabled = optional(bool, false) resources = optional(object({ limits = optional(object({ cpu = optional(string, "1") memory = optional(string, "4500Mi") ephemeral-storage = optional(string, "2G") }), {}) requests = optional(object({ cpu = optional(string, "750m") memory = optional(string, "4Gi") ephemeral-storage = optional(string, "1G") }), {}) }), {}) annotations = optional(map(string), {}) node_selector = optional(map(string), {}) tolerations = optional(list(object({ key = optional(string) operator = optional(string) value = optional(string) effect = optional(string) toleration_seconds = optional(number) })), []) # activeDeadlineSeconds defines the length of time in seconds before which the Pod will # be marked as failed. This can be used to set a time limit for your runs. active_deadline_seconds = optional(number, 4200) # 4200 seconds = 70 minutes termination_grace_period_seconds = optional(number, 50) }) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`service_account_name`
Kubernetes Service Account name
`service_account_namespace`
Kubernetes Service Account namespace
`service_account_policy_arn`
IAM policy ARN
`service_account_policy_id`
IAM policy ID
`service_account_policy_name`
IAM policy name
`service_account_role_arn`
IAM role ARN
`service_account_role_name`
IAM role name
`service_account_role_unique_id`
IAM role unique ID
`spacelift_worker_pool_manifest`
Spacelift worker pool Kubernetes manifest
`worker_pool_id`
Spacelift worker pool ID
`worker_pool_name`
Spacelift worker pool name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.18.1, != 2.21.0` - `spacelift`, version: `>= 0.1.2` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.18.1, != 2.21.0` - `spacelift`, version: `>= 0.1.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `ecr` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks_iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `eks_iam_role` | 2.2.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.2.1) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_manifest.spacelift_worker_pool`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/manifest) (resource) - [`kubernetes_role_binding_v1.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role_binding_v1) (resource) - [`kubernetes_role_v1.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role_v1) (resource) - [`kubernetes_secret.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret) (resource) - [`kubernetes_service_account_v1.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/service_account_v1) (resource) - [`spacelift_worker_pool.default`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/worker_pool) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.spacelift_key_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.spacelift_key_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`spacelift_spaces.default`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/data-sources/spaces) (data source) --- ## spacelift-worker-pool-controller This component provisions the controller part of the [Kubernetes Operator](https://docs.spacelift.io/concepts/worker-pools/kubernetes-workers#kubernetes-workers) for [Spacelift Worker Pools](https://docs.spacelift.io/concepts/worker-pools#kubernetes) into an EKS cluster. It must be installed in the cluster before installing the `eks/spacelift-worker-pool` component. The `eks/spacelift-worker-pool-controller` component must be provisioned only once per EKS cluster. You can deploy the `eks/spacelift-worker-pool` component multiple times. ## Usage **Stack Level**: Regional ```yaml # stacks/catalog/eks/spacelift-worker-pool-controller/defaults.yaml components: terraform: eks/spacelift-worker-pool-controller: vars: enabled: true name: "spacelift-controller" eks_component_name: eks/cluster # https://github.com/spacelift-io/spacelift-helm-charts/tree/main/spacelift-workerpool-controller # https://docs.spacelift.io/concepts/worker-pools#kubernetes chart: "spacelift-workerpool-controller" chart_repository: "https://downloads.spacelift.io/helm" chart_version: "0.19.0" chart_description: "Helm chart for deploying Spacelift worker pool controller and WorkerPool CRD" create_namespace_with_kubernetes: true kubernetes_namespace: "spacelift-worker-pool" timeout: 180 cleanup_on_fail: true atomic: true wait: true chart_values: {} ``` ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`chart_repository` (`string`) required
Repository URL where to locate the requested chart.
`region` (`string`) required
AWS Region
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used **Default value:** `true`
`chart_description` (`string`) optional
Set release description attribute (visible in the history). **Default value:** `null`
`chart_values` (`any`) optional
Additional values to yamlencode as `helm_release` values **Default value:** `{ }`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails **Default value:** `true`
`create_namespace_with_kubernetes` (`bool`) optional
Create the Kubernetes namespace if it does not yet exist **Default value:** `true`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`kubernetes_namespace` (`string`) optional
Name of the Kubernetes Namespace this pod is deployed in to **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`spacelift_worker_pool_controller_metadata`
Block status of the deployed Spacelift worker pool Kubernetes controller
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.18.1, != 2.21.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `spacelift_worker_pool_controller` | 0.10.1 | [`cloudposse/helm-release/aws`](https://registry.terraform.io/modules/cloudposse/helm-release/aws/0.10.1) | Deploy Spacelift worker pool Kubernetes controller Helm chart https://docs.spacelift.io/concepts/worker-pools#installation https://github.com/spacelift-io/spacelift-helm-charts/tree/main/spacelift-workerpool-controller https://github.com/spacelift-io/spacelift-helm-charts/blob/main/spacelift-workerpool-controller/values.yaml `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## storage-class This component is responsible for provisioning `StorageClasses` in an EKS cluster. See the list of guides and references linked at the bottom of this README for more information. A StorageClass provides part of the configuration for a PersistentVolumeClaim, which copies the configuration when it is created. Thus, you can delete a StorageClass without affecting existing PersistentVolumeClaims, and changes to a StorageClass do not propagate to existing PersistentVolumeClaims. ## Usage **Stack Level**: Regional, per cluster This component can create storage classes backed by EBS or EFS, and is intended to be used with the corresponding EKS add-ons `aws-ebs-csi-driver` and `aws-efs-csi-driver` respectively. In the case of EFS, this component also requires that you have provisioned an EFS filesystem in the same region as your cluster, and expects you have used the `efs` (previously `eks/efs`) component to do so. The EFS storage classes will get the file system ID from the EFS component's output. ### Note: Default Storage Class Exactly one StorageClass can be designated as the default StorageClass for a cluster. This default StorageClass is then used by PersistentVolumeClaims that do not specify a storage class. Prior to Kubernetes 1.26, if more than one StorageClass is marked as default, a PersistentVolumeClaim without `storageClassName` explicitly specified cannot be created. In Kubernetes 1.26 and later, if more than one StorageClass is marked as default, the last one created will be used, which means you can get by with just ignoring the default "gp2" StorageClass that EKS creates for you. EKS always creates a default storage class for the cluster, typically an EBS backed class named `gp2`. Find out what the default storage class is for your cluster by running this command: ```bash # You only need to run `set-cluster` when you are changing target clusters set-cluster admin # replace admin with other role name if desired kubectl get storageclass ``` This will list the available storage classes, with the default one marked with `(default)` next to its name. If you want to change the default, you can unset the existing default manually, like this: ```bash SC_NAME=gp2 # Replace with the name of the storage class you want to unset as default # You only need to run `set-cluster` when you are changing target clusters set-cluster admin # replace admin with other role name if desired kubectl patch storageclass $SC_NAME -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"false"}}}' ``` Or you can import the existing default storage class into Terraform and manage or delete it entirely, like this: ```bash SC_NAME=gp2 # Replace with the name of the storage class you want to unset as default atmos terraform import eks/storage-class 'kubernetes_storage_class_v1.ebs["'${SC_NAME}'"]' $SC_NAME -s=core-usw2-dev ``` View the parameters of a storage class by running this command: ```bash SC_NAME=gp2 # Replace with the name of the storage class you want to view # You only need to run `set-cluster` when you are changing target clusters set-cluster admin # replace admin with other role name if desired kubectl get storageclass $SC_NAME -o yaml ``` You can then match that configuration, except that you cannot omit `allow_volume_exansion`. ```yaml ebs_storage_classes: gp2: make_default_storage_class: true include_tags: false # Preserve values originally set by eks/cluster. # Set to "" to omit. provisioner: kubernetes.io/aws-ebs parameters: type: gp2 encrypted: "" ``` Here's an example snippet for how to use this component. ```yaml eks/storage-class: vars: ebs_storage_classes: gp2: make_default_storage_class: false include_tags: false # Preserve values originally set by eks/cluster. # Set to "" to omit. provisioner: kubernetes.io/aws-ebs parameters: type: gp2 encrypted: "" gp3: make_default_storage_class: true parameters: type: gp3 efs_storage_classes: efs-sc: make_default_storage_class: false efs_component_name: "efs" # Replace with the name of the EFS component, previously "eks/efs" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`ebs_storage_classes` optional
A map of storage class name to EBS parameters to create **Type:** ```hcl map(object({ enabled = optional(bool, true) make_default_storage_class = optional(bool, false) include_tags = optional(bool, true) # If true, StorageClass will set our tags on created EBS volumes labels = optional(map(string), null) reclaim_policy = optional(string, "Delete") volume_binding_mode = optional(string, "WaitForFirstConsumer") mount_options = optional(list(string), null) # Allowed topologies are poorly documented, and poorly implemented. # According to the API spec https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.26/#storageclass-v1-storage-k8s-io # it should be a list of objects with a `matchLabelExpressions` key, which is a list of objects with `key` and `values` keys. # However, the Terraform resource only allows a single object in a matchLabelExpressions block, not a list, # the EBS driver appears to only allow a single matchLabelExpressions block, and it is entirely unclear # what should happen if either of the lists has more than one element. # So we simplify it here to be singletons, not lists, and allow for a future change to the resource to support lists, # and a future replacement for this flattened object which can maintain backward compatibility. allowed_topologies_match_label_expressions = optional(object({ key = optional(string, "topology.ebs.csi.aws.com/zone") values = list(string) }), null) allow_volume_expansion = optional(bool, true) # parameters, see https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/parameters.md parameters = object({ fstype = optional(string, "ext4") # "csi.storage.k8s.io/fstype" type = optional(string, "gp3") iopsPerGB = optional(string, null) allowAutoIOPSPerGBIncrease = optional(string, null) # "true" or "false" iops = optional(string, null) throughput = optional(string, null) encrypted = optional(string, "true") kmsKeyId = optional(string, null) # ARN of the KMS key to use for encryption. If not specified, the default key is used. blockExpress = optional(string, null) # "true" or "false" blockSize = optional(string, null) }) provisioner = optional(string, "ebs.csi.aws.com") # TODO: support tags # https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/docs/tagging.md })) ``` **Default value:** `{ }`
`efs_storage_classes` optional
A map of storage class name to EFS parameters to create **Type:** ```hcl map(object({ enabled = optional(bool, true) make_default_storage_class = optional(bool, false) labels = optional(map(string), null) efs_component_name = optional(string, "eks/efs") reclaim_policy = optional(string, "Delete") volume_binding_mode = optional(string, "Immediate") # Mount options are poorly documented. # TLS is now the default and need not be specified. https://github.com/kubernetes-sigs/aws-efs-csi-driver/tree/master/docs#encryption-in-transit # Other options include `lookupcache` and `iam`. mount_options = optional(list(string), null) parameters = optional(object({ basePath = optional(string, "/efs_controller") directoryPerms = optional(string, "700") provisioningMode = optional(string, "efs-ap") gidRangeStart = optional(string, null) gidRangeEnd = optional(string, null) # Support for cross-account EFS mounts # See https://github.com/kubernetes-sigs/aws-efs-csi-driver/tree/master/examples/kubernetes/cross_account_mount # and for gritty details on secrets: https://kubernetes-csi.github.io/docs/secrets-and-credentials-storage-class.html az = optional(string, null) provisioner-secret-name = optional(string, null) # "csi.storage.k8s.io/provisioner-secret-name" provisioner-secret-namespace = optional(string, null) # "csi.storage.k8s.io/provisioner-secret-namespace" }), {}) provisioner = optional(string, "efs.csi.aws.com") })) ``` **Default value:** `{ }`
`eks_component_name` (`string`) optional
The name of the EKS component for the cluster in which to create the storage classes **Default value:** `"eks/cluster"`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `false`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes config file. If supplied, `kubeconfig_context_format` will be ignored. **Default value:** `""`
`kubeconfig_context_format` (`string`) optional
A format string to use for creating the `kubectl` context name when `kubeconfig_file_enabled` is `true` and `kubeconfig_context` is not supplied. Must include a single `%s` which will be replaced with the cluster name. **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`storage_classes`
Storage classes created by this module
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `helm`, version: `>= 2.0.0, < 3.0.0` - `kubernetes`, version: `>= 2.22.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `kubernetes`, version: `>= 2.22.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `efs` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_storage_class_v1.ebs`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class_v1) (resource) - [`kubernetes_storage_class_v1.efs`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/storage_class_v1) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) --- ## tailscale ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: eks/tailscale: vars: enabled: true name: tailscale create_namespace: true kubernetes_namespace: "tailscale" image_repo: tailscale/k8s-operator image_tag: unstable ``` ## Variables ### Required Variables
`kubernetes_namespace` (`string`) required
The namespace to install the release into.
`region` (`string`) required
AWS Region
### Optional Variables
`chart_values` (`any`) optional
Addition map values to yamlencode as `helm_release` values. **Default value:** `{ }`
`create_namespace` (`bool`) optional
Create the namespace if it does not yet exist. Defaults to `false`. **Default value:** `false`
`deployment_name` (`string`) optional
Name of the tailscale deployment, defaults to `tailscale` if this is null **Default value:** `null`
`eks_component_name` (`string`) optional
The name of the eks component **Default value:** `"eks/cluster"`
`env` (`map(string)`) optional
Map of ENV vars in the format `key=value`. These ENV vars will be set in the `utils` provider before executing the data source **Default value:** `null`
`helm_manifest_experiment_enabled` (`bool`) optional
Enable storing of the rendered manifest for helm_release so the full diff of what is changing can been seen in the plan **Default value:** `true`
`image_repo` (`string`) optional
Image repository for the deployment **Default value:** `"ghcr.io/tailscale/tailscale"`
`image_tag` (`string`) optional
Image Tag for the deployment. **Default value:** `"latest"`
`import_profile_name` (`string`) optional
AWS Profile name to use when importing a resource **Default value:** `null`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`kube_data_auth_enabled` (`bool`) optional
If `true`, use an `aws_eks_cluster_auth` data source to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled` or `kube_exec_auth_enabled`. **Default value:** `false`
`kube_exec_auth_aws_profile` (`string`) optional
The AWS config profile for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_aws_profile_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_aws_profile` as the `profile` to `aws eks get-token` **Default value:** `false`
`kube_exec_auth_enabled` (`bool`) optional
If `true`, use the Kubernetes provider `exec` feature to execute `aws eks get-token` to authenticate to the EKS cluster. Disabled by `kubeconfig_file_enabled`, overrides `kube_data_auth_enabled`. **Default value:** `true`
`kube_exec_auth_role_arn` (`string`) optional
The role ARN for `aws eks get-token` to use **Default value:** `""`
`kube_exec_auth_role_arn_enabled` (`bool`) optional
If `true`, pass `kube_exec_auth_role_arn` as the role ARN to `aws eks get-token` **Default value:** `true`
`kube_secret` (`string`) optional
Kube Secret Name for tailscale **Default value:** `"tailscale"`
`kubeconfig_context` (`string`) optional
Context to choose from the Kubernetes kube config file **Default value:** `""`
`kubeconfig_exec_auth_api_version` (`string`) optional
The Kubernetes API version of the credentials returned by the `exec` auth plugin **Default value:** `"client.authentication.k8s.io/v1beta1"`
`kubeconfig_file` (`string`) optional
The Kubernetes provider `config_path` setting to use when `kubeconfig_file_enabled` is `true` **Default value:** `""`
`kubeconfig_file_enabled` (`bool`) optional
If `true`, configure the Kubernetes provider with `kubeconfig_file` and use that kubeconfig file for authenticating to the EKS cluster **Default value:** `false`
`routes` (`list(string)`) optional
List of CIDR Ranges or IPs to allow Tailscale to connect to **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`deployment`
Tail scale operator deployment K8S resource
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `kubernetes`, version: `>= 2.7.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `store_read` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`kubernetes_cluster_role.tailscale_operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/cluster_role) (resource) - [`kubernetes_cluster_role_binding.tailscale_operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/cluster_role_binding) (resource) - [`kubernetes_deployment.operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/deployment) (resource) - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) - [`kubernetes_role.operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role) (resource) - [`kubernetes_role.proxies`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role) (resource) - [`kubernetes_role_binding.operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role_binding) (resource) - [`kubernetes_role_binding.proxies`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/role_binding) (resource) - [`kubernetes_secret.operator_oauth`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret) (resource) - [`kubernetes_service_account.operator`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/service_account) (resource) - [`kubernetes_service_account.proxies`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/service_account) (resource) ## Data Sources The following data sources are used by this module: - [`aws_eks_cluster.kubernetes`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) (data source) - [`aws_eks_cluster_auth.eks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster_auth) (data source) - [`aws_subnet.vpc_subnets`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) --- ## vertical-pod-autoscaler Description of this component 55 ## Usage **Stack Level**: Regional or Global Here's an example snippet for how to use this component. ```yaml components: terraform: foo: vars: enabled: true ``` ## Variables ### Required Variables
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`mock`
Mock output example for the Cloud Posse Terraform component template
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## elasticache-redis This component provisions AWS [ElastiCache Redis](https://aws.amazon.com/elasticache/redis/) clusters. The `engine` can either be `redis` or `valkey`. For more information, see [why aws supports valkey](https://aws.amazon.com/blogs/opensource/why-aws-supports-valkey/). ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. `stacks/catalog/elasticache/elasticache-redis/defaults.yaml` file (default settings for all Redis clusters): ```yaml components: terraform: elasticache-redis: vars: enabled: true name: "elasticache-redis" family: redis6.x egress_cidr_blocks: ["0.0.0.0/0"] port: 6379 at_rest_encryption_enabled: true transit_encryption_enabled: false apply_immediately: false automatic_failover_enabled: false cloudwatch_metric_alarms_enabled: false snapshot_retention_limit: 1 redis_clusters: redis-main: num_replicas: 1 num_shards: 0 replicas_per_shard: 0 engine: "redis" engine_version: 6.0.5 instance_type: cache.t2.small parameters: - name: notify-keyspace-events value: "lK" ``` `stacks/org/ou/account/region.yaml` file (imports and overrides the default settings for a specific cluster): ```yaml import: - catalog/elasticache/elasticache-redis/defaults.yaml components: terraform: elasticache-redis: vars: enabled: true redis_clusters: redis-main: num_replicas: 1 num_shards: 0 replicas_per_shard: 0 engine_version: 6.0.5 instance_type: cache.t2.small parameters: - name: notify-keyspace-events value: lK ``` ## Variables ### Required Variables
`apply_immediately` (`bool`) required
Apply changes immediately
`at_rest_encryption_enabled` (`bool`) required
Enable encryption at rest
`automatic_failover_enabled` (`bool`) required
Enable automatic failover
`cloudwatch_metric_alarms_enabled` (`bool`) required
Boolean flag to enable/disable CloudWatch metrics alarms
`family` (`string`) required
Redis family
`port` (`number`) required
Port number
`redis_clusters` (`map(any)`) required
Redis cluster configuration
`region` (`string`) required
AWS region
`transit_encryption_enabled` (`bool`) required
Enable TLS
### Optional Variables
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP address. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allow_ingress_from_this_vpc` (`bool`) optional
If set to `true`, allow ingress from the VPC CIDR for this account **Default value:** `true`
`allow_ingress_from_vpc_stages` (`list(string)`) optional
List of stages to pull VPC ingress cidr and add to security group **Default value:** `[ ]`
`auth_token_enabled` (`bool`) optional
Enable auth token **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Specifies whether minor version engine upgrades will be applied automatically to the underlying Cache Cluster instances during the maintenance window. Only supported if the engine version is 6 or higher. **Default value:** `false`
`availability_zones` (`list(string)`) optional
Availability zone IDs **Default value:** `[ ]`
`dns_delegated_component_name` (`string`) optional
The name of the Delegated DNS component **Default value:** `"dns-delegated"`
`eks_component_names` (`set(string)`) optional
The names of the eks components **Default value:** `[ ]`
`eks_security_group_enabled` (`bool`) optional
Use the eks default security group **Default value:** `false`
`ingress_cidr_blocks` (`list(string)`) optional
CIDR blocks for permitted ingress **Default value:** `[ ]`
`multi_az_enabled` (`bool`) optional
Multi AZ (Automatic Failover must also be enabled. If Cluster Mode is enabled, Multi AZ is on by default, and this setting is ignored) **Default value:** `false`
`snapshot_retention_limit` (`number`) optional
The number of days for which ElastiCache will retain automatic cache cluster snapshots before deleting them. **Default value:** `0`
`transit_encryption_mode` (`string`) optional
Transit encryption mode. Valid values are 'preferred' and 'required' **Default value:** `null`
`vpc_component_name` (`string`) optional
The name of a VPC component **Default value:** `"vpc"`
`vpc_ingress_component_name` (`string`) optional
The name of a Ingress VPC component **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`redis_clusters`
Redis cluster objects
`security_group_id`
The security group ID of the ElastiCache Redis cluster
`transit_encryption_mode`
TLS in-transit encryption mode for Redis cluster
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `redis_clusters` | latest | [`./modules/redis_cluster`](https://registry.terraform.io/modules/./modules/redis_cluster/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpc_ingress` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## elasticsearch This component is responsible for provisioning an Elasticsearch cluster with built-in integrations with Kibana and Logstash. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: elasticsearch: vars: enabled: true name: foobar instance_type: "t3.medium.elasticsearch" elasticsearch_version: "7.9" encrypt_at_rest_enabled: true dedicated_master_enabled: false elasticsearch_subdomain_name: "es" kibana_subdomain_name: "kibana" ebs_volume_size: 40 create_iam_service_linked_role: true kibana_hostname_enabled: true domain_hostname_enabled: true ``` ## Variables ### Required Variables
`create_iam_service_linked_role` (`bool`) required
Whether to create `AWSServiceRoleForAmazonElasticsearchService` service-linked role. Set this to `false` if you already have an ElasticSearch cluster created in the AWS account and `AWSServiceRoleForAmazonElasticsearchService` already exists. See https://github.com/terraform-providers/terraform-provider-aws/issues/5218 for more information.
`dedicated_master_enabled` (`bool`) required
Indicates whether dedicated master nodes are enabled for the cluster
`domain_hostname_enabled` (`bool`) required
Explicit flag to enable creating a DNS hostname for ES. If `true`, then `var.dns_zone_id` is required.
`ebs_volume_size` (`number`) required
EBS volumes for data storage in GB
`elasticsearch_subdomain_name` (`string`) required
The name of the subdomain for Elasticsearch in the DNS zone (_e.g._ `elasticsearch`, `ui`, `ui-es`, `search-ui`)
`elasticsearch_version` (`string`) required
Version of Elasticsearch to deploy (_e.g._ `7.1`, `6.8`, `6.7`, `6.5`, `6.4`, `6.3`, `6.2`, `6.0`, `5.6`, `5.5`, `5.3`, `5.1`, `2.3`, `1.5`
`encrypt_at_rest_enabled` (`bool`) required
Whether to enable encryption at rest
`instance_type` (`string`) required
The type of the instance
`kibana_hostname_enabled` (`bool`) required
Explicit flag to enable creating a DNS hostname for Kibana. If `true`, then `var.dns_zone_id` is required.
`kibana_subdomain_name` (`string`) required
The name of the subdomain for Kibana in the DNS zone (_e.g._ `kibana`, `ui`, `ui-es`, `search-ui`, `kibana.elasticsearch`)
`region` (`string`) required
AWS region
### Optional Variables
`dedicated_master_count` (`number`) optional
Number of dedicated master nodes in the cluster **Default value:** `0`
`dedicated_master_type` (`string`) optional
Instance type of the dedicated master nodes in the cluster **Default value:** `"t2.small.elasticsearch"`
`dns_delegated_environment_name` (`string`) optional
The name of the environment where the `dns-delegated` component is deployed **Default value:** `"gbl"`
`elasticsearch_iam_actions` (`list(string)`) optional
List of actions to allow for the IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` **Default value:** ```hcl [ "es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost", "es:ESHttpHead", "es:Describe*", "es:List*" ] ```
`elasticsearch_iam_role_arns` (`list(string)`) optional
List of additional IAM role ARNs to permit access to the Elasticsearch domain **Default value:** `[ ]`
`elasticsearch_password` (`string`) optional
Password for the elasticsearch user **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`domain_arn`
ARN of the Elasticsearch domain
`domain_endpoint`
Domain-specific endpoint used to submit index, search, and data upload requests
`domain_hostname`
Elasticsearch domain hostname to submit index, search, and data upload requests
`domain_id`
Unique identifier for the Elasticsearch domain
`elasticsearch_user_iam_role_arn`
The ARN of the IAM role to allow access to Elasticsearch cluster
`elasticsearch_user_iam_role_name`
The name of the IAM role to allow access to Elasticsearch cluster
`kibana_endpoint`
Domain-specific endpoint for Kibana without https scheme
`kibana_hostname`
Kibana hostname
`master_password_ssm_key`
SSM key of Elasticsearch master password
`security_group_id`
Security Group ID to control access to the Elasticsearch domain
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `elasticsearch` | 1.2.0 | [`cloudposse/elasticsearch/aws`](https://registry.terraform.io/modules/cloudposse/elasticsearch/aws/1.2.0) | n/a `elasticsearch_log_cleanup` | 0.14.1 | [`cloudposse/lambda-elasticsearch-cleanup/aws`](https://registry.terraform.io/modules/cloudposse/lambda-elasticsearch-cleanup/aws/0.14.1) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.admin_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.elasticsearch_domain_endpoint`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.elasticsearch_kibana_endpoint`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.elasticsearch_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: --- ## eventbridge The `eventbridge` component is a Terraform module that defines a CloudWatch EventBridge rule. The rule is pointed at cloudwatch by default. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: eventbridge/ecs-alerts: metadata: component: eventbridge vars: name: ecs-faults enabled: true cloudwatch_event_rule_description: "ECS failures and warnings" cloudwatch_event_rule_pattern: source: - aws.ecs detail: $or: - eventType: - WARN - ERROR - agentConnected: - false - containers: exitCode: - anything-but: - 0 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`cloudwatch_event_rule_description` (`string`) optional
Description of the CloudWatch Event Rule. If empty, will default to `module.this.id` **Default value:** `""`
`cloudwatch_event_rule_pattern` (`any`) optional
Pattern of the CloudWatch Event Rule **Default value:** ```hcl { "source": [ "aws.ec2" ] } ```
`event_log_retention_in_days` (`number`) optional
Number of days to retain the event logs **Default value:** `3`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudwatch_event_rule_arn`
The ARN of the CloudWatch Event Rule
`cloudwatch_event_rule_name`
The name of the CloudWatch Event Rule
`cloudwatch_logs_log_group_arn`
The ARN of the CloudWatch Log Group
`cloudwatch_logs_log_group_name`
The name of the CloudWatch Log Group
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_event` | 0.9.1 | [`cloudposse/cloudwatch-events/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-events/aws/0.9.1) | n/a `cloudwatch_logs` | 0.6.9 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.9) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_resource_policy.eventbridge_cloudwatch_logs_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_resource_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.eventbridge_cloudwatch_logs_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## github-action-token-rotator This component provisions the [Github Action Token Rotator](https://github.com/cloudposse/terraform-aws-github-action-token-rotator). It creates an AWS Lambda function that rotates GitHub Action tokens stored in AWS Systems Manager Parameter Store. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. This is generally deployed once and to the automation account's primary region. `stacks/catalog/github-action-token-rotator.yaml` file: ```yaml components: terraform: github-action-token-rotator: vars: enabled: true github_org_name: my-org github_app_installation_id: 11111111 github_app_id: 222222 parameter_store_private_key_path: /github/runners/my-org/privateKey parameter_store_token_path: /github/runners/my-org/registrationToken ``` Follow the manual steps using the [guide in the upstream module](https://github.com/cloudposse/terraform-aws-github-action-token-rotator#quick-start) and use `chamber` to add the secrets to the appropriate stage. ## Variables ### Required Variables
`github_app_id` (`string`) required
GitHub App ID
`github_app_installation_id` (`string`) required
GitHub App Installation ID
`github_org_name` (`string`) required
SSM parameter name format
`parameter_store_private_key_path` (`string`) required
Path to read the GitHub App private key from parameter store
`parameter_store_token_path` (`string`) required
Path to store the token in parameter store
`region` (`string`) required
AWS Region
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`github_action_token_rotator`
GitHub action token rotator module outputs.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `github_action_token_rotator` | 0.1.0 | [`cloudposse/github-action-token-rotator/aws`](https://registry.terraform.io/modules/cloudposse/github-action-token-rotator/aws/0.1.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## github-oidc-provider This component authorizes the GitHub OIDC provider as an identity provider for an AWS account. It is intended to be used with `aws-teams`, `aws-team-roles`, and/or `github-actions-iam-role.mixin.tf`. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. - This must be installed in the `identity` account in order to use standard SAML roles with role chaining. - This must be installed in each individual account where you want to provision a service role for a GitHub action that will be assumed directly by the action. For security, since this component adds an identity provider, only SuperAdmin can install it. ```yaml components: terraform: github-oidc-provider: vars: enabled: true ``` ## Configuring the GitHub OIDC Provider This component was created to add the GitHub OIDC provider so that GitHub Actions can safely assume roles without the need to store static credentials in the environment. The details of the GitHub OIDC provider are hard coded in the component, however at some point the provider's thumbprint may change, at which point you can use [get_github_oidc_thumbprint.sh](https://github.com/cloudposse/terraform-aws-components/blob/main/modules/github-oidc-provider/scripts/get_github_oidc_thumbprint.sh) to get the new thumbprint and add it to the list in `var.thumbprint_list`. This script will pull one of two thumbprints. There are two possible intermediary certificates for the Actions SSL certificate and either can be returned by the GitHub servers, requiring customers to trust both. This is a known behavior when the intermediary certificates are cross-signed by the CA. Therefore, run this script until both values are retrieved. Add both to `var.thumbprint_list`. For more, see https://github.blog/changelog/2023-06-27-github-actions-update-on-oidc-integration-with-aws/ ## FAQ ### I cannot assume the role from GitHub Actions after deploying The following error is very common if the GitHub workflow is missing proper permission. ```bash Error: User: arn:aws:sts::***:assumed-role/acme-core-use1-auto-actions-runner@actions-runner-system/token-file-web-identity is not authorized to perform: sts:TagSession on resource: arn:aws:iam::999999999999:role/acme-plat-use1-dev-gha ``` In order to use a web identity, GitHub Action pipelines must have the following permission. See [GitHub Action documentation for more](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#adding-permissions-settings). ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`superadmin` (`bool`) optional
Set `true` if running as the SuperAdmin user **Default value:** `false`
`thumbprint_list` (`list(string)`) optional
List of OIDC provider certificate thumbprints **Default value:** ```hcl [ "6938fd4d98bab03faadb97b34396831e3780aea1", "1c58a3a8518e8759bf075b76b750d4f2df264fcd" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`oidc_provider_arn`
GitHub OIDC provider ARN
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_openid_connect_provider.oidc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_openid_connect_provider) (resource) ## Data Sources The following data sources are used by this module: --- ## github-oidc-role This component is responsible for creating IAM roles for GitHub Actions to assume. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. ```yaml # stacks/catalog/github-oidc-role/defaults.yaml components: terraform: github-oidc-role/defaults: metadata: type: abstract vars: enabled: true name: gha-iam # Note: inherited lists are not merged, they are replaced github_actions_allowed_repos: - MyOrg/* ## allow all repos in MyOrg ``` Example using for gitops (predefined policy): ```yaml # stacks/catalog/github-oidc-role/gitops.yaml import: - catalog/github-oidc-role/defaults components: terraform: github-oidc-role/gitops: metadata: component: github-oidc-role inherits: - github-oidc-role/defaults vars: enabled: true # Note: inherited lists are not merged, they are replaced github_actions_allowed_repos: - "MyOrg/infrastructure" attributes: ["gitops"] iam_policies: - gitops gitops_policy_configuration: s3_bucket_component_name: gitops/s3-bucket dynamodb_component_name: gitops/dynamodb ``` Example using for lambda-cicd (predefined policy): ```yaml # stacks/catalog/github-oidc-role/lambda-cicd.yaml import: - catalog/github-oidc-role/defaults components: terraform: github-oidc-role/lambda-cicd: metadata: component: github-oidc-role inherits: - github-oidc-role/defaults vars: enabled: true github_actions_allowed_repos: - MyOrg/example-app-on-lambda-with-gha attributes: ["lambda-cicd"] iam_policies: - lambda-cicd lambda_cicd_policy_configuration: enable_ssm_access: true enable_s3_access: true s3_bucket_component_name: s3-bucket/github-action-artifacts s3_bucket_environment_name: gbl s3_bucket_stage_name: artifacts s3_bucket_tenant_name: core ``` Example Using an AWS Managed policy and a custom inline policy: ```yaml # stacks/catalog/github-oidc-role/custom.yaml import: - catalog/github-oidc-role/defaults components: terraform: github-oidc-role/custom: metadata: component: github-oidc-role inherits: - github-oidc-role/defaults vars: enabled: true github_actions_allowed_repos: - MyOrg/example-app-on-lambda-with-gha attributes: ["custom"] iam_policies: - arn:aws:iam::aws:policy/AdministratorAccess iam_policy: - version: "2012-10-17" statements: - effect: "Allow" actions: - "ec2:*" resources: - "*" ``` ### Adding Custom Policies There are two methods for adding custom policies to the IAM role. 1. Through the `iam_policy` input which you can use to add inline policies to the IAM role. 2. By defining policies in Terraform and then attaching them to roles by name. #### Defining Custom Policies in Terraform 1. Give the policy a unique name, e.g. `docker-publish`. We will use `NAME` as a placeholder for the name in the instructions below. 2. Create a file in the component directory (i.e. `github-oidc-role`) with the name `policy_NAME.tf`. 3. In that file, conditionally (based on need) create a policy document as follows: ```hcl locals { NAME_policy_enabled = contains(var.iam_policies, "NAME") NAME_policy = local.NAME_policy_enabled ? one(data.aws_iam_policy_document.NAME.*.json) : null } data "aws_iam_policy_document" "NAME" { count = local.NAME_policy_enabled ? 1 : 0 # Define the policy here } ``` Note that you can also add input variables and outputs to this file if desired. Just make sure that all inputs are optional. 4. Create a file named `additional-policy-map_override.tf` in the component directory (if it does not already exist). This is a [terraform override file](https://developer.hashicorp.com/terraform/language/files/override), meaning its contents will be merged with the main terraform file, and any locals defined in it will override locals defined in other files. Having your code in this separate override file makes it possible for the component to provide a placeholder local variable so that it works without customization, while allowing you to customize the component and still update it without losing your customizations. 5. In that file, redefine the local variable `overridable_additional_custom_policy_map` map as follows: ```hcl locals { overridable_additional_custom_policy_map = { "NAME" = local.NAME_policy } } ``` If you have multiple custom policies, using just this one file, add each policy document to the map in the form `NAME = local.NAME_policy`. 6. With that done, you can now attach that policy by adding the name to the `iam_policies` list. For example: ```yaml iam_policies: - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" - "NAME" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`github_actions_allowed_repos` (`list(string)`) optional
A list of the GitHub repositories that are allowed to assume this role from GitHub Actions. For example, ["cloudposse/infra-live"]. Can contain "*" as wildcard. If org part of repo name is omitted, "cloudposse" will be assumed. **Default value:** `[ ]`
`gitops_policy_configuration` optional
Configuration for the GitOps IAM Policy, valid keys are - `s3_bucket_component_name` - Component Name of where to store the TF Plans in S3, defaults to `gitops/s3-bucket` - `dynamodb_component_name` - Component Name of where to store the TF Plans in Dynamodb, defaults to `gitops/dynamodb` - `s3_bucket_environment_name` - Environment name for the S3 Bucket, defaults to current environment - `dynamodb_environment_name` - Environment name for the Dynamodb Table, defaults to current environment **Type:** ```hcl object({ s3_bucket_component_name = optional(string, "gitops/s3-bucket") dynamodb_component_name = optional(string, "gitops/dynamodb") s3_bucket_environment_name = optional(string) dynamodb_environment_name = optional(string) }) ``` **Default value:** `{ }`
`iam_policies` (`list(string)`) optional
List of policies to attach to the IAM role, should be either an ARN of an AWS Managed Policy or a name of a custom policy e.g. `gitops` **Default value:** `[ ]`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`lambda_cicd_policy_configuration` optional
Configuration for the lambda-cicd policy. The following keys are supported: - `enable_kms_access` - (bool) - Whether to allow access to KMS. Defaults to false. - `enable_ssm_access` - (bool) - Whether to allow access to SSM. Defaults to false. - `enable_s3_access` - (bool) - Whether to allow access to S3. Defaults to false. - `s3_bucket_component_name` - (string) - The name of the component to use for the S3 bucket. Defaults to `s3-bucket/github-action-artifacts`. - `s3_bucket_environment_name` - (string) - The name of the environment to use for the S3 bucket. Defaults to the environment of the current module. - `s3_bucket_tenant_name` - (string) - The name of the tenant to use for the S3 bucket. Defaults to the tenant of the current module. - `s3_bucket_stage_name` - (string) - The name of the stage to use for the S3 bucket. Defaults to the stage of the current module. - `enable_lambda_update` - (bool) - Whether to allow access to update lambda functions. Defaults to false. **Type:** ```hcl object({ enable_kms_access = optional(bool, false) enable_ssm_access = optional(bool, false) enable_s3_access = optional(bool, false) s3_bucket_component_name = optional(string, "s3-bucket/github-action-artifacts") s3_bucket_environment_name = optional(string) s3_bucket_tenant_name = optional(string) s3_bucket_stage_name = optional(string) enable_lambda_update = optional(bool, false) }) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`github_actions_iam_role_arn`
ARN of IAM role for GitHub Actions
`github_actions_iam_role_name`
Name of IAM role for GitHub Actions
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dynamodb` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `gha_assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `s3_artifacts_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `s3_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_role.github_actions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.gitops_iam_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.lambda_cicd_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## github-repository :lock: Managing GitHub repos in a compliant way just got way easier. One of the most common requirements for companies operating in regulated spaces is having consistent, enforceable repository configurations. Some teams have hacked together homegrown solutions, while others know it's something they need to do but never quite got to. Our `aws-github-repository` component changes that by providing comprehensive GitHub repository management entirely through infrastructure as code. **Manage repos and all settings declaratively:** - Repository configuration (visibility, topics, features, merge settings) - Environments with deployment protection and reviewers - Variables and secrets pulling from AWS Secrets Manager and Parameter Store - Branch protection rules and rule sets - Webhooks, deploy keys, labels, and custom properties - Team and user permissions **Import existing repos and bring them under management:** - Import existing repositories and their configurations - Bring legacy repos under consistent management - Maintain existing settings while adding compliance controls **Define repository archetypes once:** - Create abstract components for common configurations - Use Atmos inheritance to automatically apply baseline compliance - Every new repo inherits your security and compliance standards - DRY up infrastructure with reusable templates All powered by Atmos + a GitHub App for secure, auditable rollout across your organization. ## Usage **Stack Level**: Regional This component provides comprehensive GitHub repository management with support for advanced features like environments, webhooks, deploy keys, custom properties, and more. ## Basic Usage Here's a simple example for creating a basic repository: ```yaml components: terraform: my-basic-repo: vars: enabled: true owner: "my-organization" repository: name: "my-basic-repo" description: "A basic repository with standard settings" homepage_url: "https://github.com/my-organization/my-basic-repo" visibility: "private" default_branch: "main" topics: - terraform - github - infrastructure teams: devops: admin developers: push variables: ENVIRONMENT: "production" REGION: "us-east-1" secrets: AWS_ACCESS_KEY_ID: "nacl:dGVzdC1hY2Nlc3Mta2V5LWlkCg==" DATABASE_URL: "ssm:///my-basic-repo/database-url" ``` ## Advanced Usage with Atmos Inheritance and Imports The component supports Atmos inheritance and imports to create DRY (Don't Repeat Yourself) configurations. This allows you to define common settings once and reuse them across multiple repositories. :::tip For complete working examples of inheritance and imports, see the [`/examples`](https://github.com/cloudposse-terraform-components/aws-github-repository/tree/main/github-repository/examples) folder in this repository. ::: ### Creating Abstract Components First, create abstract components that define common configurations: ```yaml # catalog/github/repo/defaults.yaml components: terraform: github/repo/defaults: metadata: component: github-repository type: abstract vars: enabled: true # Common team permissions teams: devops: admin # Default repository settings repository: homepage_url: "https://github.com/{{ .vars.owner }}/{{ .vars.repository.name }}" topics: - terraform - github default_branch: "main" visibility: "private" # Common features auto_init: true gitignore_template: "TeX" license_template: "GPL-3.0" # Merge settings allow_merge_commit: true allow_squash_merge: true allow_rebase_merge: true allow_auto_merge: true # Branch protection delete_branch_on_merge: true web_commit_signoff_required: true ``` ### Creating a Master Defaults Component Combine abstract components into a master defaults component: ```yaml # catalog/github/defaults.yaml import: - catalog/github/repo/* - catalog/github/environment/* - catalog/github/ruleset/* components: terraform: github/defaults: metadata: type: abstract inherits: - github/repo/defaults - github/ruleset/branch-protection - github/environment/defaults ``` ### Using Inheritance in Your Stacks Now you can create repositories that inherit from these defaults: ```yaml # orgs/mycompany/myteam.yaml import: - ./_defaults - catalog/github/defaults components: terraform: # Simple repository inheriting all defaults my-simple-app: metadata: component: github-repository inherits: - github/defaults vars: repository: name: my-simple-app description: "My simple application" topics: - application - nodejs # Repository with custom overrides my-custom-app: metadata: component: github-repository inherits: - github/defaults vars: repository: name: my-custom-app description: "My custom application" topics: - application - python - api # Override default team permissions teams: devops: admin backend: push frontend: push # Add custom variables and secrets variables: LANGUAGE: "python" FRAMEWORK: "fastapi" secrets: PYTHON_API_KEY: "asm://my-custom-app-api-key" ``` ## Secrets and Variables Management The component supports setting repository and environment secrets and variables. Secrets and variables can be set using the following methods: - Raw values (unencrypted string) (example: `my-secret-value`) - AWS Secrets Manager (SM) (example: `asm://secret-name`) - AWS Systems Manager Parameter Store (SSM) (example: `ssm:///my/secret/path`) In addition to that secrets supports base64 encoded values [encrypted](https://docs.github.com/en/rest/guides/encrypting-secrets-for-the-rest-api?apiVersion=2022-11-28) with [repository key](https://docs.github.com/en/rest/actions/secrets?apiVersion=2022-11-28#get-a-repository-public-key). The value should be prefixed with `nacl:` (example: `nacl:dGVzdC12YWx1ZS0yCg==`). ### Environment-Specific Secrets and Variables ```yaml environments: production: variables: ENVIRONMENT: "production" CLUSTER_NAME: "prod-cluster" LOG_LEVEL: "warn" secrets: PROD_DATABASE_URL: "ssm:///my-repo/prod/database-url" PROD_API_KEY: "asm://my-repo-prod-api-key" PROD_JWT_SECRET: "nacl:cHJvZC1qd3Qtc2VjcmV0Cg==" staging: variables: ENVIRONMENT: "staging" CLUSTER_NAME: "staging-cluster" LOG_LEVEL: "info" secrets: STAGING_DATABASE_URL: "ssm:///my-repo/staging/database-url" STAGING_API_KEY: "asm://my-repo-staging-api-key" ``` ## Import Mode The component supports importing existing repository and its configurations: - collaborators - variables - environments - environment variables - labels - custom properties values - autolink references - deploy keys Import mode is enabled by setting `import` input variable to `true`. ```yaml components: terraform: my-imported-repo: vars: enabled: true import: true owner: "my-organization" repository: name: "existing-repo-to-import" ``` The following configurations are not supported for import: - secrets - environment secrets - branch protection policies - rulesets ## Variables ### Required Variables
`owner` (`string`) required
Owner of the repository
`region` (`string`) required
AWS Region
`repository` required
Repository configuration **Type:** ```hcl object({ name = string description = optional(string, null) visibility = optional(string, "public") homepage_url = optional(string, null) archived = optional(bool, false) has_issues = optional(bool, false) has_projects = optional(bool, false) has_discussions = optional(bool, false) has_wiki = optional(bool, false) has_downloads = optional(bool, false) is_template = optional(bool, false) allow_auto_merge = optional(bool, false) allow_squash_merge = optional(bool, true) squash_merge_commit_title = optional(string, "PR_TITLE") squash_merge_commit_message = optional(string, "PR_BODY") allow_merge_commit = optional(bool, true) merge_commit_title = optional(string, "PR_TITLE") merge_commit_message = optional(string, "PR_BODY") allow_rebase_merge = optional(bool, true) delete_branch_on_merge = optional(bool, false) default_branch = optional(string, "main") web_commit_signoff_required = optional(bool, false) topics = optional(list(string), []) license_template = optional(string, null) gitignore_template = optional(string, null) auto_init = optional(bool, false) ignore_vulnerability_alerts_during_read = optional(bool, false) enable_vulnerability_alerts = optional(bool, true) allow_update_branch = optional(bool, false) security_and_analysis = optional(object({ advanced_security = bool secret_scanning = bool secret_scanning_push_protection = bool }), null) archive_on_destroy = optional(bool, false) }) ```
### Optional Variables
`autolink_references` optional
Autolink references **Type:** ```hcl map(object({ key_prefix = string target_url_template = string is_alphanumeric = optional(bool, false) })) ``` **Default value:** `{ }`
`custom_properties` optional
Custom properties for the repository **Type:** ```hcl map(object({ string = optional(string, null) boolean = optional(bool, null) single_select = optional(string, null) multi_select = optional(list(string), null) })) ``` **Default value:** `{ }`
`deploy_keys` optional
Deploy keys for the repository **Type:** ```hcl map(object({ title = string key = string read_only = optional(bool, false) })) ``` **Default value:** `{ }`
`environments` optional
Environments for the repository. Enviroment secrets should be encrypted using the GitHub public key in Base64 format if prefixed with nacl:. Read more: https://docs.github.com/en/actions/security-for-github-actions/encrypted-secrets **Type:** ```hcl map(object({ wait_timer = optional(number, 0) can_admins_bypass = optional(bool, false) prevent_self_review = optional(bool, false) reviewers = optional(object({ teams = optional(list(string), []) users = optional(list(string), []) }), null) deployment_branch_policy = optional(object({ protected_branches = optional(bool, false) custom_branches = optional(object({ branches = optional(list(string), null) tags = optional(list(string), null) }), null) }), null) variables = optional(map(string), null) secrets = optional(map(string), null) })) ``` **Default value:** `{ }`
`import` (`bool`) optional
Import repository **Default value:** `false`
`labels` optional
A map of labels to configure for the repository **Type:** ```hcl map(object({ color = string description = string })) ``` **Default value:** `{ }`
`rulesets` optional
A map of rulesets to configure for the repository **Type:** ```hcl map(object({ name = string # disabled, active enforcement = string # branch, tag target = string bypass_actors = optional(list(object({ # always, pull_request bypass_mode = string actor_id = optional(string, null) # RepositoryRole, Team, Integration, OrganizationAdmin actor_type = string })), []) conditions = object({ ref_name = object({ # Supports ~DEFAULT_BRANCH or ~ALL include = optional(list(string), []) exclude = optional(list(string), []) }) }) rules = object({ branch_name_pattern = optional(object({ # starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), commit_author_email_pattern = optional(object({ # starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), creation = optional(bool, false), deletion = optional(bool, false), non_fast_forward = optional(bool, false), required_pull_request_reviews = optional(object({ dismiss_stale_reviews = bool required_approving_review_count = number }), null), commit_message_pattern = optional(object({ # starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), committer_email_pattern = optional(object({ # starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), merge_queue = optional(object({ check_response_timeout_minutes = optional(number, 60) # ALLGREEN, HEADGREEN grouping_strategy = string max_entries_to_build = optional(number, 5) max_entries_to_merge = optional(number, 5) # MERGE, SQUASH, REBASE merge_method = optional(string, "MERGE") min_entries_to_merge = optional(number, 1) min_entries_to_merge_wait_minutes = optional(number, 5) }), null), pull_request = optional(object({ dismiss_stale_reviews_on_push = optional(bool, false) require_code_owner_review = optional(bool, false) require_last_push_approval = optional(bool, false) required_approving_review_count = optional(number, 0) required_review_thread_resolution = optional(bool, false) }), null), required_deployments = optional(object({ required_deployment_environments = optional(list(string), []) }), null), required_status_checks = optional(object({ required_check = list(object({ context = string integration_id = optional(number, null) })) strict_required_status_checks_policy = optional(bool, false) do_not_enforce_on_create = optional(bool, false) }), null), tag_name_pattern = optional(object({ # starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), # Unsupported due to drift. # https://github.com/integrations/terraform-provider-github/pull/2701 # required_code_scanning = optional(object({ # required_code_scanning_tool = list(object({ # // none, errors, errors_and_warnings, all # alerts_threshold = string # // none, critical, high_or_higher, medium_or_higher, all # security_alerts_threshold = string # tool = string # })) # }), null), }), })) ``` **Default value:** `{ }`
`secrets` (`map(string)`) optional
Secrets for the repository (if prefixed with nacl: it should be encrypted value using the GitHub public key in Base64 format. Read more: https://docs.github.com/en/actions/security-for-github-actions/encrypted-secrets) **Default value:** `{ }`
`teams` (`map(string)`) optional
A map of teams and their permissions for the repository **Default value:** `{ }`
`template` optional
Template repository **Type:** ```hcl object({ owner = string name = string include_all_branches = optional(bool, false) }) ``` **Default value:** `null`
`users` (`map(string)`) optional
A map of users and their permissions for the repository **Default value:** `{ }`
`variables` (`map(string)`) optional
Environment variables for the repository **Default value:** `{ }`
`webhooks` optional
A map of webhooks to configure for the repository **Type:** ```hcl map(object({ active = optional(bool, true) events = list(string) url = string content_type = optional(string, "json") insecure_ssl = optional(bool, false) secret = optional(string, null) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`collaborators_invitation_ids`
Collaborators invitation IDs
`full_name`
Full name of the created repository
`git_clone_url`
Git clone URL of the created repository
`html_url`
HTML URL of the created repository
`http_clone_url`
HTTP clone URL of the created repository
`node_id`
Node ID of the created repository
`primary_language`
Primary language of the created repository
`repo_id`
Repository ID of the created repository
`rulesets_etags`
Rulesets etags
`rulesets_node_ids`
Rulesets node IDs
`rulesets_rules_ids`
Rulesets rules IDs
`ssh_clone_url`
SSH clone URL of the created repository
`svn_url`
SVN URL of the created repository
`webhooks_urls`
Webhooks URLs
## Dependencies ### Requirements - `terraform`, version: `>= 1.7.0` - `aws`, version: `>= 5.0.0` - `github`, version: `>= 6.6.0` ### Providers - `aws`, version: `>= 5.0.0` - `github`, version: `>= 6.6.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `repository` | 0.2.1 | [`cloudposse/repository/github`](https://registry.terraform.io/modules/cloudposse/repository/github/0.2.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_secretsmanager_secret.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret) (data source) - [`aws_secretsmanager_secret_version.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/secretsmanager_secret_version) (data source) - [`aws_ssm_parameter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`github_actions_environment_variables.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/actions_environment_variables) (data source) - [`github_actions_variables.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/actions_variables) (data source) - [`github_issue_labels.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/issue_labels) (data source) - [`github_repository.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository) (data source) - [`github_repository_autolink_references.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository_autolink_references) (data source) - [`github_repository_custom_properties.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository_custom_properties) (data source) - [`github_repository_deploy_keys.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository_deploy_keys) (data source) - [`github_repository_environments.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/repository_environments) (data source) --- ## github-runners This component is responsible for provisioning EC2 instances for GitHub runners. :::tip We also have a similar component based on [actions-runner-controller](https://github.com/actions-runner-controller/actions-runner-controller) for Kubernetes. ::: ## Requirements ## Configuration ### API Token Prior to deployment, the API Token must exist in SSM. To generate the token, please follow [these instructions](https://cloudposse.atlassian.net/l/c/N4dH05ud). Once generated, write the API token to the SSM key store at the following location within the same AWS account and region where the GitHub Actions runner pool will reside. ``` assume-role chamber write github/runners/ registration-token ghp_secretstring ``` ## Background ### Registration Github Actions Self-Hosted runners can be scoped to the Github Organization, a Single Repository, or a group of Repositories (Github Enterprise-Only). Upon startup, each runner uses a `REGISTRATION_TOKEN` to call the Github API to register itself with the Organization, Repository, or Runner Group (Github Enterprise). ### Running Workflows Once a Self-Hosted runner is registered, you will have to update your workflow with the `runs-on` attribute specify it should run on a self-hosted runner: ``` name: Test Self Hosted Runners on: push: branches: [main] jobs: build: runs-on: [self-hosted] ``` ### Workflow Github Permissions (GITHUB_TOKEN) Each run of the Github Actions Workflow is assigned a GITHUB_TOKEN, which allows your workflow to perform actions against Github itself such as cloning a repo, updating the checks API status, etc., and expires at the end of the workflow run. The GITHUB_TOKEN has two permission "modes" it can operate in `Read and write permissions` ("Permissive" or "Full Access") and `Read repository contents permission` ("Restricted" or "Read-Only"). By default, the GITHUB_TOKEN is granted Full Access permissions, but you can change this via the Organization or Repo settings. If you opt for the Read-Only permissions, you can optionally grant or revoke access to specific APIs via the workflow `yaml` file and a full list of APIs that can be accessed can be found in the [documentation](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token) and is shown below in the table. It should be noted that the downside to this permissions model is that any user with write access to the repository can escalate permissions for the workflow by updating the `yaml` file, however, the APIs available via this token are limited. Most notably the GITHUB_TOKEN does not have access to the `users`, `repos`, `apps`, `billing`, or `collaborators` APIs, so the tokens do not have access to modify sensitive settings or add/remove users from the Organization/Repository. > Example of using escalated permissions for the entire workflow ``` name: Pull request labeler on: [ pull_request_target ] permissions: contents: read pull-requests: write jobs: triage: runs-on: ubuntu-latest steps: - uses: actions/labeler@v2 with: repo-token: ${{ secrets.GITHUB_TOKEN }} ``` > Example of using escalated permissions for a job ``` name: Create issue on commit on: [ push ] jobs: create_commit: runs-on: ubuntu-latest permissions: issues: write steps: - name: Create issue using REST API run: | curl --request POST \ --url https://api.github.com/repos/${{ github.repository }}/issues \ --header 'authorization: Bearer ${{ secrets.GITHUB_TOKEN }}' \ --header 'content-type: application/json' \ --data '{ "title": "Automated issue for commit: ${{ github.sha }}", "body": "This issue was automatically created by the GitHub Action workflow **${{ github.workflow }}**. \n\n The commit hash was: _${{ github.sha }}_." }' \ --fail ``` ### Pre-Requisites for Using This Component In order to use this component, you will have to obtain the `REGISTRATION_TOKEN` mentioned above from your Github Organization or Repository and store it in SSM Parameter store. In addition, it is recommended that you set the permissions “mode” for Self-hosted runners to Read-Only. The instructions for doing both are below. #### Workflow Permissions 1. Browse to [https://github.com/organizations/\{Org\}/settings/actions](https://github.com/organizations/\{Org\}/settings/actions) (Organization) or [https://github.com/\{Org\}/\{Repo\}/settings/actions](https://github.com/\{Org\}/\{Repo\}/settings/actions) (Repository) 2. Set the default permissions for the GITHUB_TOKEN to Read Only ### Creating Registration Token :::tip We highly recommend using a GitHub Application with the github-action-token-rotator module to generate the Registration Token. This will ensure that the token is rotated and that the token is stored in SSM Parameter Store encrypted with KMS. ::: #### GitHub Application Follow the quickstart with the upstream module, [cloudposse/terraform-aws-github-action-token-rotator](https://github.com/cloudposse/terraform-aws-github-action-token-rotator#quick-start), or follow the steps below. 1. Create a new GitHub App 1. Add the following permission: ```diff # Required Permissions for Repository Runners: ## Repository Permissions + Actions (read) + Administration (read / write) + Metadata (read) # Required Permissions for Organization Runners: ## Repository Permissions + Actions (read) + Metadata (read) ## Organization Permissions + Self-hosted runners (read / write) ``` 1. Generate a Private Key If you are working with Cloud Posse, upload this Private Key, GitHub App ID, and Github App Installation ID to 1Password and skip the rest. Otherwise, complete the private key setup in `core--auto`. 1. Convert the private key to a PEM file using the following command: `openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in {DOWNLOADED_FILE_NAME}.pem -out private-key-pkcs8.key` 1. Upload PEM file key to the specified ssm path: `/github/runners/acme/private-key` in `core--auto` 1. Create another sensitive SSM parameter `/github/runners/acme/registration-token` in `core--auto` with any basic value, such as "foo". This will be overwritten by the rotator. 1. Update the GitHub App ID and Installation ID in the `github-action-token-rotator` catalog. :::tip If you change the Private Key saved in SSM, redeploy `github-action-token-rotator` ::: #### (ClickOps) Obtain the Runner Registration Token 1. Browse to [https://github.com/organizations/\{Org\}/settings/actions/runners](https://github.com/organizations/\{Org\}/settings/actions/runners) (Organization) or [https://github.com/\{Org\}/\{Repo\}/settings/actions/runners](https://github.com/\{Org\}/\{Repo\}/settings/actions/runners) (Repository) 2. Click the **New Runner** button (Organization) or **New Self Hosted Runner** button (Repository) 3. Copy the Github Runner token from the next screen. Note that this is the only time you will see this token. Note that if you exit the `New {Self Hosted} Runner` screen and then later return by clicking the `New {Self Hosted} Runner` button again, the registration token will be invalidated and a new token will be generated. 4. Add the `REGISTRATION_TOKEN` to the `/github/token` SSM parameter in the account where Github runners are hosted (usually `automation`), encrypted with KMS. ``` chamber write github token ``` # FAQ ## The GitHub Registration Token is not updated in SSM The `github-action-token-rotator` runs an AWS Lambda function every 30 minutes. This lambda will attempt to use a private key in its environment configuration to generate a GitHub Registration Token, and then store that token to AWS SSM Parameter Store. If the GitHub Registration Token parameter, `/github/runners/acme/registration-token`, is not updated, read through the following tips: 1. The private key is stored at the given parameter path: `parameter_store_private_key_path: /github/runners/acme/private-key` 1. The private key is Base 64 encoded. If you pull the key from SSM and decode it, it should begin with `-----BEGIN PRIVATE KEY-----` 1. If the private key has changed, you must _redeploy_ `github-action-token-rotator`. Run a plan against the component to make sure there are not changes required. ## The GitHub Registration Token is valid, but the Runners are not registering with GitHub If you first deployed the `github-action-token-rotator` component initially with an invalid configuration and then deployed the `github-runners` component, the instance runners will have failed to register with GitHub. After you correct `github-action-token-rotator` and have a valid GitHub Registration Token in SSM, _destroy and recreate_ the `github-runners` component. If you cannot see the runners registered in GitHub, check the system logs on one of EC2 Instances in AWS in `core--auto`. ## I cannot assume the role from GitHub Actions after deploying The following error is very common if the GitHub workflow is missing proper permission. ```bash Error: User: arn:aws:sts::***:assumed-role/acme-core-use1-auto-actions-runner@actions-runner-system/token-file-web-identity is not authorized to perform: sts:TagSession on resource: arn:aws:iam::999999999999:role/acme-plat-use1-dev-gha ``` In order to use a web identity, GitHub Action pipelines must have the following permission. See [GitHub Action documentation for more](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#adding-permissions-settings). ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` ## FAQ ### Can we scope it to a github org with both private and public repos ? Yes but this requires Github Enterprise Cloud and the usage of runner groups to scope permissions of runners to specific repos. If you set the scope to the entire org without runner groups and if the org has both public and private repos, then the risk of using a self-hosted runner incorrectly is a vulnerability within public repos. [https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups) If you do not have github enterprise cloud and runner groups cannot be utilized, then it’s best to create new github runners per repo or use the summerwind action-runners-controller via a Github App to set the scope to specific repos. ### How can we see the current spot pricing? Go to [ec2instances.info](http://ec2instances.info/) ### If we don’t use mixed at all does that mean we can’t do spot? It’s possible to do spot without using mixed instances but you leave yourself open to zero instance availability with a single instance type. For example, if you wanted to use spot and use `t3.xlarge` in `us-east-2` and for some reason, AWS ran out of `t3.xlarge`, you wouldn't have the option to choose another instance type and so all the GitHub Action runs would stall until availability returned. If you use on-demand pricing, it’s more expensive, but you’re more likely to get scheduling priority. For guaranteed availability, reserved instances are required. ### Do the overrides apply to both the on-demand and the spot instances, or only the spot instances? Since the overrides affect the launch template, I believe they will affect both spot instances and override since weighted capacity can be set for either or. The override terraform option is on the ASG’s `launch_template` > List of nested arguments provides the ability to specify multiple instance types. This will override the same > parameter in the launch template. For on-demand instances, Auto Scaling considers the order of preference of instance > types to launch based on the order specified in the overrides list. Defined below. And in the terraform resource for > `instances_distribution` > `spot_max_price` - (Optional) Maximum price per unit hour that the user is willing to pay for the Spot instances. > Default: an empty string which means the on-demand price. For a `mixed_instances_policy`, this will do purely > on-demand ``` mixed_instances_policy: instances_distribution: on_demand_allocation_strategy: "prioritized" on_demand_base_capacity: 1 on_demand_percentage_above_base_capacity: 0 spot_allocation_strategy: "capacity-optimized" spot_instance_pools: null spot_max_price: [] ``` This will always do spot unless instances are unavailable, then switch to on-demand. ``` mixed_instances_policy: instances_distribution: # ... spot_max_price: 0.05 ``` If you want a single instance type, you could still use the mixed instances policy to define that like above, or you can use these other inputs and comment out the `mixed_instances_policy` ``` instance_type: "t3.xlarge" # the below is optional in order to set the spot max price instance_market_options: market_type = "spot" spot_options: block_duration_minutes: 6000 instance_interruption_behavior: terminate max_price: 0.05 spot_instance_type = persistent valid_until: null ``` The `overrides` will override the `instance_type` above. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: github-runners: vars: cpu_utilization_high_threshold_percent: 5 cpu_utilization_low_threshold_percent: 1 default_cooldown: 300 github_scope: company instance_type: "t3.small" max_size: 10 min_size: 1 runner_group: default scale_down_cooldown_seconds: 2700 wait_for_capacity_timeout: 10m mixed_instances_policy: instances_distribution: on_demand_allocation_strategy: "prioritized" on_demand_base_capacity: 1 on_demand_percentage_above_base_capacity: 0 spot_allocation_strategy: "capacity-optimized" spot_instance_pools: null spot_max_price: null override: - instance_type: "t4g.large" weighted_capacity: null - instance_type: "m5.large" weighted_capacity: null - instance_type: "m5a.large" weighted_capacity: null - instance_type: "m5n.large" weighted_capacity: null - instance_type: "m5zn.large" weighted_capacity: null - instance_type: "m4.large" weighted_capacity: null - instance_type: "c5.large" weighted_capacity: null - instance_type: "c5a.large" weighted_capacity: null - instance_type: "c5n.large" weighted_capacity: null - instance_type: "c4.large" weighted_capacity: null ``` ## Variables ### Required Variables
`github_scope` (`string`) required
Scope of the runner (e.g. `cloudposse/example` for repo or `cloudposse` for org)
`max_size` (`number`) required
The maximum size of the autoscale group
`min_size` (`number`) required
The minimum size of the autoscale group
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`ami_filter` (`map(list(string))`) optional
Map of lists used to look up the AMI which will be used for the GitHub Actions Runner. **Default value:** ```hcl { "name": [ "amzn2-ami-hvm-2.*-x86_64-ebs" ] } ```
`ami_owners` (`list(string)`) optional
The list of owners used to select the AMI of action runner instances. **Default value:** ```hcl [ "amazon" ] ```
`block_device_mappings` optional
Specify volumes to attach to the instance besides the volumes specified by the AMI **Type:** ```hcl list(object({ device_name = string no_device = bool virtual_name = string ebs = object({ delete_on_termination = bool encrypted = bool iops = number kms_key_id = string snapshot_id = string volume_size = number volume_type = string }) })) ``` **Default value:** `[ ]`
`cpu_utilization_high_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_high_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_high_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `90`
`cpu_utilization_low_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_low_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_low_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `10`
`default_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes before another scaling activity can start **Default value:** `300`
`docker_compose_version` (`string`) optional
The version of docker-compose to install **Default value:** `"1.29.2"`
`instance_type` (`string`) optional
Default instance type for the action runner. **Default value:** `"m5.large"`
`max_instance_lifetime` (`number`) optional
The maximum amount of time, in seconds, that an instance can be in service, values must be either equal to 0 or between 604800 and 31536000 seconds **Default value:** `null`
`mixed_instances_policy` optional
Policy to use a mixed group of on-demand/spot of differing types. Launch template is automatically generated. https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#mixed_instances_policy-1 **Type:** ```hcl object({ instances_distribution = object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string }) override = list(object({ instance_type = string weighted_capacity = number })) }) ``` **Default value:** `null`
`runner_group` (`string`) optional
GitHub runner group **Default value:** `"default"`
`runner_labels` (`list(string)`) optional
List of labels to add to the GitHub Runner (e.g. 'Amazon Linux 2'). **Default value:** `[ ]`
`runner_role_additional_policy_arns` (`list(string)`) optional
List of policy ARNs that will be attached to the runners' default role on creation in addition to the defaults **Default value:** `[ ]`
`runner_version` (`string`) optional
GitHub runner release version **Default value:** `"2.288.1"`
`scale_down_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`ssm_parameter_name_format` (`string`) optional
SSM parameter name format **Default value:** `"/%s/%s"`
`ssm_path` (`string`) optional
GitHub token SSM path **Default value:** `"github"`
`ssm_path_key` (`string`) optional
GitHub token SSM path key **Default value:** `"registration-token"`
`userdata_post_install` (`string`) optional
Shell script to run post installation of github action runner **Default value:** `""`
`userdata_pre_install` (`string`) optional
Shell script to run before installation of github action runner **Default value:** `""`
`wait_for_capacity_timeout` (`string`) optional
A maximum duration that Terraform should wait for ASG instances to be healthy before timing out. (See also Waiting for Capacity below.) Setting this to '0' causes Terraform to skip all Capacity Waiting behavior **Default value:** `"10m"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`autoscaling_group_arn`
The Amazon Resource Name (ARN) of the Auto Scaling Group.
`autoscaling_group_name`
The name of the Auto Scaling Group.
`autoscaling_lifecycle_hook_name`
The name of the Lifecycle Hook for the Auto Scaling Group.
`eventbridge_rule_arn`
The ARN of the Eventbridge rule for the EC2 lifecycle transition.
`eventbridge_target_arn`
The ARN of the Eventbridge target corresponding to the Eventbridge rule for the EC2 lifecycle transition.
`iam_role_arn`
The ARN of the IAM role associated with the Autoscaling Group
`ssm_document_arn`
The ARN of the SSM document.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `cloudinit`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `cloudinit`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `autoscale_group` | 0.42.0 | [`cloudposse/ec2-autoscale-group/aws`](https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.42.0) | n/a `graceful_scale_in` | latest | [`./modules/graceful_scale_in`](https://registry.terraform.io/modules/./modules/graceful_scale_in/) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `sg` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.github_action_runner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_policy.github_action_runner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.github_action_runner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.runner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_iam_policy_document.github_action_runner`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.instance_assume_role_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.github_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`cloudinit_config.config`](https://registry.terraform.io/providers/hashicorp/cloudinit/latest/docs/data-sources/config) (data source) --- ## github-webhook This component provisions a GitHub webhook for a single GitHub repository. You may want to use this component if you are provisioning webhooks for multiple ArgoCD deployment repositories across GitHub organizations. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. This example pulls the value of the webhook from `remote-state` ```yaml components: terraform: webhook/cloudposse/argocd: metadata: component: github-webhook vars: github_organization: cloudposse github_repository: argocd-deploy-non-prod webhook_url: "https://argocd.ue2.dev.plat.cloudposse.org/api/webhook" remote_state_github_webhook_enabled: true # default value added for visibility remote_state_component_name: eks/argocd ``` ### SSM Stored Value Example Here's an example snippet for how to use this component with a value stored in SSM ```yaml components: terraform: webhook/cloudposse/argocd: metadata: component: github-webhook vars: github_organization: cloudposse github_repository: argocd-deploy-non-prod webhook_url: "https://argocd.ue2.dev.plat.cloudposse.org/api/webhook" remote_state_github_webhook_enabled: false ssm_github_webhook_enabled: true ssm_github_webhook: "/argocd/github/webhook" ``` ### Input Value Example Here's an example snippet for how to use this component with a value stored in Terraform variables. ```yaml components: terraform: webhook/cloudposse/argocd: metadata: component: github-webhook vars: github_organization: cloudposse github_repository: argocd-deploy-non-prod webhook_url: "https://argocd.ue2.dev.plat.cloudposse.org/api/webhook" remote_state_github_webhook_enabled: false ssm_github_webhook_enabled: false webhook_github_secret: "abcdefg" ``` ### ArgoCD Webhooks For usage with the `eks/argocd` component, see [Creating Webhooks with `github-webhook`](https://github.com/cloudposse/terraform-aws-components/blob/main/modules/eks/argocd/README.md#creating-webhooks-with-github-webhook) in that component's README. ## Variables ### Required Variables
`github_organization` (`string`) required
The name of the GitHub Organization where the repository lives
`github_repository` (`string`) required
The name of the GitHub repository where the webhook will be created
`region` (`string`) required
AWS Region.
`webhook_url` (`string`) required
The URL for the webhook
### Optional Variables
`github_base_url` (`string`) optional
This is the target GitHub base API endpoint. Providing a value is a requirement when working with GitHub Enterprise. It is optional to provide this value and it can also be sourced from the `GITHUB_BASE_URL` environment variable. The value must end with a slash, for example: `https://terraformtesting-ghe.westus.cloudapp.azure.com/` **Default value:** `null`
`github_token_override` (`string`) optional
Use the value of this variable as the GitHub token instead of reading it from SSM **Default value:** `null`
`remote_state_component_name` (`string`) optional
If fetching the Github Webhook value from remote-state, set this to the source component name. For example, `eks/argocd`. **Default value:** `""`
`remote_state_github_webhook_enabled` (`bool`) optional
If `true`, pull the GitHub Webhook value from remote-state **Default value:** `true`
`ssm_github_api_key` (`string`) optional
SSM path to the GitHub API key **Default value:** `"/argocd/github/api_key"`
`ssm_github_webhook` (`string`) optional
Format string of the SSM parameter path where the webhook will be pulled from. Only used if `var.webhook_github_secret` is not given. **Default value:** `"/github/webhook"`
`ssm_github_webhook_enabled` (`bool`) optional
If `true`, pull the GitHub Webhook value from AWS SSM Parameter Store using `var.ssm_github_webhook` **Default value:** `false`
`webhook_github_secret` (`string`) optional
The value to use as the GitHub webhook secret. Set both `var.ssm_github_webhook_enabled` and `var.remote_state_github_webhook_enabled` to `false` in order to use this value **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `github`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `source` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | This can be any component that has the required output, `github-webhook-value` This is typically eks/argocd `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`github_repository_webhook.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_webhook) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.github_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.webhook`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## global-accelerator This component provisions AWS Global Accelerator and its listeners. ## Usage **Stack Level**: Global Here are some example snippets for how to use this component: ```yaml global-accelerator: vars: enabled: true flow_logs_enabled: true flow_logs_s3_bucket: examplecorp-ue1-devplatform-global-accelerator-flow-logs flow_logs_s3_prefix: logs/ listeners: - client_affinity: NONE protocol: TCP port_ranges: - from_port: 80 to_port: 80 - from_port: 443 to_port: 443 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`flow_logs_enabled` (`bool`) optional
Enable or disable flow logs for the Global Accelerator. **Default value:** `false`
`flow_logs_s3_bucket_component` (`string`) optional
The component that deploys the S3 Bucket for the Accelerator Flow Logs. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`flow_logs_s3_bucket_environment` (`string`) optional
The environment where the S3 Bucket for the Accelerator Flow Logs exists. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`flow_logs_s3_bucket_stage` (`string`) optional
The stage where the S3 Bucket for the Accelerator Flow Logs exists. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`flow_logs_s3_bucket_tenant` (`string`) optional
The tenant where the S3 Bucket for the Accelerator Flow Logs exists. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`flow_logs_s3_prefix` (`string`) optional
The Object Prefix within the S3 Bucket for the Accelerator Flow Logs. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`listeners` optional
list of listeners to configure for the global accelerator. For more information, see: [aws_globalaccelerator_listener](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_listener). **Type:** ```hcl list(object({ client_affinity = string port_ranges = list(object({ from_port = number to_port = number })) protocol = string })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dns_name`
DNS name of the Global Accelerator.
`listener_ids`
Global Accelerator Listener IDs.
`name`
Name of the Global Accelerator.
`static_ips`
Global Static IPs owned by the Global Accelerator.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `flow_logs_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `global_accelerator` | 0.6.0 | [`cloudposse/global-accelerator/aws`](https://registry.terraform.io/modules/cloudposse/global-accelerator/aws/0.6.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## global-accelerator-endpoint-group This component is responsible for provisioning a Global Accelerator Endpoint Group. This component assumes that the `global-accelerator` component has already been deployed to the same account in the environment specified by `var.global_accelerator_environment_name`. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: global-accelerator-endpoint-group: vars: enabled: true config: endpoint_configuration: - endpoint_lb_name: my-load-balancer ``` ## Variables ### Required Variables
`config` (`any`) required
Endpoint Group configuration. This object needs to be fully compliant with the `aws_globalaccelerator_endpoint_group` resource, except for the following differences: * `listener_arn`, which is specified separately, is omitted. * The values for `endpoint_configuration` and `port_override` within each object in `endpoint_groups` should be lists. * Inside the `endpoint_configuration` block, `endpoint_lb_name` can be supplied in place of `endpoint_id` as long as it is a valid unique name for an existing ALB or NLB. For more information, see: [aws_globalaccelerator_endpoint_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_endpoint_group).
`region` (`string`) required
AWS Region.
### Optional Variables
`global_accelerator_environment_name` (`string`) optional
The name of the environment where the global component `global_accelerator` is provisioned **Default value:** `"gbl"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
The ID of the Global Accelerator Endpoint Group.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `endpoint_group` | 0.6.0 | [`cloudposse/global-accelerator/aws//modules/endpoint-group`](https://registry.terraform.io/modules/cloudposse/global-accelerator/aws/modules/endpoint-group/0.6.0) | n/a `global_accelerator` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## catalog-database This component provisions Glue catalog databases. ## Usage ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/catalog-database/example: metadata: component: glue/catalog-database vars: enabled: true name: example catalog_database_description: Glue catalog database example location_uri: "s3://awsglue-datasets/examples/medicare/Medicare_Hospital_Provider.csv" glue_iam_component_name: "glue/iam" lakeformation_permissions_enabled: true lakeformation_permissions: - "ALL" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`catalog_database_description` (`string`) optional
Glue catalog database description **Default value:** `null`
`catalog_database_name` (`string`) optional
Glue catalog database name. The acceptable characters are lowercase letters, numbers, and the underscore character **Default value:** `null`
`catalog_id` (`string`) optional
ID of the Glue Catalog to create the database in. If omitted, this defaults to the AWS Account ID **Default value:** `null`
`create_table_default_permission` (`any`) optional
Creates a set of default permissions on the table for principals **Default value:** `null`
`glue_iam_component_name` (`string`) optional
Glue IAM component name. Used to get the Glue IAM role from the remote state **Default value:** `"glue/iam"`
`lakeformation_permissions` (`list(string)`) optional
List of permissions granted to the principal. Refer to https://docs.aws.amazon.com/lake-formation/latest/dg/lf-permissions-reference.html for more details **Default value:** ```hcl [ "ALL" ] ```
`lakeformation_permissions_enabled` (`bool`) optional
Whether to enable adding Lake Formation permissions to the IAM role that is used to access the Glue database **Default value:** `true`
`location_uri` (`string`) optional
Location of the database (for example, an HDFS path) **Default value:** `null`
`parameters` (`map(string)`) optional
Map of key-value pairs that define parameters and properties of the database **Default value:** `null`
`target_database` optional
Configuration block for a target database for resource linking **Type:** ```hcl object({ # If `target_database` is provided (not `null`), all these fields are required catalog_id = string database_name = string }) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`catalog_database_arn`
Catalog database ARN
`catalog_database_id`
Catalog database ID
`catalog_database_name`
Catalog database name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_catalog_database` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-catalog-database`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-catalog-database/0.4.0) | n/a `glue_iam_role` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lakeformation_permissions.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_permissions) (resource) ## Data Sources The following data sources are used by this module: --- ## catalog-table This component provisions Glue catalog tables. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/catalog-table/example: metadata: component: glue/catalog-table vars: enabled: true name: example catalog_table_description: Glue catalog table example glue_iam_component_name: glue/iam glue_catalog_database_component_name: glue/catalog-database/example lakeformation_permissions_enabled: true lakeformation_permissions: - "ALL" storage_descriptor: location: "s3://awsglue-datasets/examples/medicare/Medicare_Hospital_Provider.csv" ``` ## Variables ### Required Variables
`glue_catalog_database_component_name` (`string`) required
Glue catalog database component name where the table metadata resides. Used to get the Glue catalog database from the remote state
`region` (`string`) required
AWS Region
### Optional Variables
`catalog_id` (`string`) optional
ID of the Glue Catalog and database to create the table in. If omitted, this defaults to the AWS Account ID plus the database name **Default value:** `null`
`catalog_table_description` (`string`) optional
Description of the table **Default value:** `null`
`catalog_table_name` (`string`) optional
Name of the table **Default value:** `null`
`glue_iam_component_name` (`string`) optional
Glue IAM component name. Used to get the Glue IAM role from the remote state **Default value:** `"glue/iam"`
`lakeformation_permissions` (`list(string)`) optional
List of permissions granted to the principal. Refer to https://docs.aws.amazon.com/lake-formation/latest/dg/lf-permissions-reference.html for more details **Default value:** ```hcl [ "ALL" ] ```
`lakeformation_permissions_enabled` (`bool`) optional
Whether to enable adding Lake Formation permissions to the IAM role that is used to access the Glue table **Default value:** `true`
`owner` (`string`) optional
Owner of the table **Default value:** `null`
`parameters` (`map(string)`) optional
Properties associated with this table, as a map of key-value pairs **Default value:** `null`
`partition_index` optional
Configuration block for a maximum of 3 partition indexes **Type:** ```hcl object({ index_name = string keys = list(string) }) ``` **Default value:** `null`
`partition_keys` (`map(string)`) optional
Configuration block of columns by which the table is partitioned. Only primitive types are supported as partition keys **Default value:** `null`
`retention` (`number`) optional
Retention time for the table **Default value:** `null`
`storage_descriptor` (`any`) optional
Configuration block for information about the physical storage of this table **Default value:** `null`
`table_type` (`string`) optional
Type of this table (`EXTERNAL_TABLE`, `VIRTUAL_VIEW`, etc.). While optional, some Athena DDL queries such as `ALTER TABLE` and `SHOW CREATE TABLE` will fail if this argument is empty **Default value:** `null`
`target_table` optional
Configuration block of a target table for resource linking **Type:** ```hcl object({ catalog_id = string database_name = string name = string }) ``` **Default value:** `null`
`view_expanded_text` (`string`) optional
If the table is a view, the expanded text of the view; otherwise null **Default value:** `null`
`view_original_text` (`string`) optional
If the table is a view, the original text of the view; otherwise null **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`catalog_table_arn`
Catalog table ARN
`catalog_table_id`
Catalog table ID
`catalog_table_name`
Catalog table name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_catalog_database` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_catalog_table` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-catalog-table`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-catalog-table/0.4.0) | n/a `glue_iam_role` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lakeformation_permissions.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_permissions) (resource) ## Data Sources The following data sources are used by this module: --- ## connection(Connection) This component provisions Glue connections. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/connection/example/redshift: metadata: component: glue/connection vars: connection_name: "jdbc-redshift" connection_description: "Glue Connection for Redshift" connection_type: "JDBC" db_type: "redshift" connection_db_name: "analytics" ssm_path_username: "/glue/redshift/admin_user" ssm_path_password: "/glue/redshift/admin_password" ssm_path_endpoint: "/glue/redshift/endpoint" physical_connection_enabled: true vpc_component_name: "vpc" ``` ## Variables ### Required Variables
`connection_type` (`string`) required
The type of the connection. Supported are: JDBC, MONGODB, KAFKA, and NETWORK. Defaults to JDBC
`region` (`string`) required
AWS Region
`vpc_component_name` (`string`) required
VPC component name
### Optional Variables
`catalog_id` (`string`) optional
The ID of the Data Catalog in which to create the connection. If none is supplied, the AWS account ID is used by default **Default value:** `null`
`connection_db_name` (`string`) optional
Database name that the Glue connector will reference **Default value:** `null`
`connection_description` (`string`) optional
Connection description **Default value:** `null`
`connection_name` (`string`) optional
Connection name. If not provided, the name will be generated from the context **Default value:** `null`
`connection_properties` (`map(string)`) optional
A map of key-value pairs used as parameters for this connection **Default value:** `null`
`db_type` (`string`) optional
Database type for the connection URL: `postgres` or `redshift` **Default value:** `"redshift"`
`match_criteria` (`list(string)`) optional
A list of criteria that can be used in selecting this connection **Default value:** `null`
`physical_connection_enabled` (`bool`) optional
Flag to enable/disable physical connection **Default value:** `false`
`security_group_allow_all_egress` (`bool`) optional
A convenience that adds to the rules a rule that allows all egress. If this is false and no egress rules are specified via `rules` or `rule-matrix`, then no egress will be allowed. **Default value:** `true`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_ingress_cidr_blocks` (`list(string)`) optional
A list of CIDR blocks for the the cluster Security Group to allow ingress to the cluster security group **Default value:** `[ ]`
`security_group_ingress_from_port` (`number`) optional
Start port on which the Glue connection accepts incoming connections **Default value:** `0`
`security_group_ingress_to_port` (`number`) optional
End port on which the Glue connection accepts incoming connections **Default value:** `0`
`ssm_path_endpoint` (`string`) optional
Database endpoint SSM path **Default value:** `null`
`ssm_path_password` (`string`) optional
Database password SSM path **Default value:** `null`
`ssm_path_username` (`string`) optional
Database username SSM path **Default value:** `null`
`target_security_group_rules` (`list(any)`) optional
Additional Security Group rules that allow Glue to communicate with the target database **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`connection_arn`
Glue connection ARN
`connection_id`
Glue connection ID
`connection_name`
Glue connection name
`security_group_arn`
The ARN of the Security Group associated with the Glue connection
`security_group_id`
The ID of the Security Group associated with the Glue connection
`security_group_name`
The name of the Security Group and associated with the Glue connection
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_connection` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-connection`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-connection/0.4.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `target_security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | This allows adding the necessary Security Group rules for Glue to communicate with Redshift `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.endpoint`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_subnet.selected`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) --- ## crawler This component provisions Glue crawlers. ## Usage **Stack Level**: Regional ```yaml components: terraform: # The crawler crawls the data in an S3 bucket and puts the results into a table in the Glue Catalog. # The crawler will read the first 2 MB of data from the file, and recognize the schema. # After that, the crawler will sync the table. glue/crawler/example: metadata: component: glue/crawler vars: enabled: true name: example crawler_description: "Glue crawler example" glue_iam_component_name: "glue/iam" glue_catalog_database_component_name: "glue/catalog-database/example" glue_catalog_table_component_name: "glue/catalog-table/example" schedule: "cron(0 1 * * ? *)" schema_change_policy: delete_behavior: LOG update_behavior: null ``` ## Variables ### Required Variables
`glue_catalog_database_component_name` (`string`) required
Glue catalog database component name where metadata resides. Used to get the Glue catalog database from the remote state
`region` (`string`) required
AWS Region
### Optional Variables
`catalog_target` optional
List of nested Glue catalog target arguments **Type:** ```hcl list(object({ database_name = string tables = list(string) })) ``` **Default value:** `null`
`classifiers` (`list(string)`) optional
List of custom classifiers. By default, all AWS classifiers are included in a crawl, but these custom classifiers always override the default classifiers for a given classification **Default value:** `null`
`configuration` (`string`) optional
JSON string of configuration information **Default value:** `null`
`crawler_description` (`string`) optional
Glue crawler description **Default value:** `null`
`crawler_name` (`string`) optional
Glue crawler name. If not provided, the name will be generated from the context **Default value:** `null`
`delta_target` optional
List of nested Delta target arguments **Type:** ```hcl list(object({ connection_name = string delta_tables = list(string) write_manifest = bool })) ``` **Default value:** `null`
`dynamodb_target` (`list(any)`) optional
List of nested DynamoDB target arguments **Default value:** `null`
`glue_catalog_table_component_name` (`string`) optional
Glue catalog table component name where metadata resides. Used to get the Glue catalog table from the remote state **Default value:** `null`
`glue_iam_component_name` (`string`) optional
Glue IAM component name. Used to get the Glue IAM role from the remote state **Default value:** `"glue/iam"`
`jdbc_target` (`list(any)`) optional
List of nested JBDC target arguments **Default value:** `null`
`lineage_configuration` optional
Specifies data lineage configuration settings for the crawler **Type:** ```hcl object({ crawler_lineage_settings = string }) ``` **Default value:** `null`
`mongodb_target` (`list(any)`) optional
List of nested MongoDB target arguments **Default value:** `null`
`recrawl_policy` optional
A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run **Type:** ```hcl object({ recrawl_behavior = string }) ``` **Default value:** `null`
`s3_target` (`list(any)`) optional
List of nested Amazon S3 target arguments **Default value:** `null`
`schedule` (`string`) optional
A cron expression for the schedule **Default value:** `null`
`schema_change_policy` (`map(string)`) optional
Policy for the crawler's update and deletion behavior **Default value:** `null`
`security_configuration` (`string`) optional
The name of Security Configuration to be used by the crawler **Default value:** `null`
`table_prefix` (`string`) optional
The table prefix used for catalog tables that are created **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`crawler_arn`
Crawler ARN
`crawler_id`
Crawler ID
`crawler_name`
Crawler name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_catalog_database` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_catalog_table` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_crawler` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-crawler`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-crawler/0.4.0) | n/a `glue_iam_role` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## iam(Iam) This component provisions IAM roles for AWS Glue. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/iam: metadata: component: glue/iam vars: enabled: true name: glue iam_role_description: "Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs" iam_policy_description: "Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs" iam_managed_policy_arns: - "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`iam_managed_policy_arns` (`list(string)`) optional
IAM managed policy ARNs **Default value:** ```hcl [ "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ] ```
`iam_policy_description` (`string`) optional
Glue IAM policy description **Default value:** `"Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs"`
`iam_role_description` (`string`) optional
Glue IAM role description **Default value:** `"Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role_arn`
The ARN of the Glue role
`role_id`
The ID of the Glue role
`role_name`
The name of the Glue role
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_role` | 0.22.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.22.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## job This component provisions Glue jobs. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/job/example: metadata: component: glue/job vars: enabled: true name: example job_description: Glue job example glue_version: "2.0" worker_type: Standard number_of_workers: 2 max_retries: 2 timeout: 20 glue_iam_component_name: "glue/iam" glue_job_s3_bucket_component_name: "s3/datalake" glue_job_s3_bucket_script_path: "glue/glue_job.py" glue_job_command_name: glueetl glue_job_command_python_version: 3 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`command` (`map(any)`) optional
The command of the job **Default value:** `null`
`connections` (`list(string)`) optional
The list of connections used for this job **Default value:** `null`
`default_arguments` (`map(string)`) optional
The map of default arguments for the job. You can specify arguments here that your own job-execution script consumes, as well as arguments that AWS Glue itself consumes **Default value:** `null`
`execution_property` optional
Execution property of the job **Type:** ```hcl object({ # The maximum number of concurrent runs allowed for the job. The default is 1. max_concurrent_runs = number }) ``` **Default value:** `null`
`glue_iam_component_name` (`string`) optional
Glue IAM component name. Used to get the Glue IAM role from the remote state **Default value:** `"glue/iam"`
`glue_job_command_name` (`string`) optional
The name of the job command. Defaults to glueetl. Use pythonshell for Python Shell Job Type, or gluestreaming for Streaming Job Type. max_capacity needs to be set if pythonshell is chosen **Default value:** `"glueetl"`
`glue_job_command_python_version` (`number`) optional
The Python version being used to execute a Python shell job. Allowed values are 2, 3 or 3.9. Version 3 refers to Python 3.6 **Default value:** `3`
`glue_job_s3_bucket_component_name` (`string`) optional
Glue job S3 bucket component name. Used to get the remote state of the S3 bucket where the Glue job script is located **Default value:** `null`
`glue_job_s3_bucket_script_path` (`string`) optional
Glue job script path in the S3 bucket **Default value:** `null`
`glue_version` (`string`) optional
The version of Glue to use **Default value:** `"2.0"`
`job_description` (`string`) optional
Glue job description **Default value:** `null`
`job_name` (`string`) optional
Glue job name. If not provided, the name will be generated from the context **Default value:** `null`
`max_capacity` (`number`) optional
The maximum number of AWS Glue data processing units (DPUs) that can be allocated when the job runs. Required when `pythonshell` is set, accept either 0.0625 or 1.0. Use `number_of_workers` and `worker_type` arguments instead with `glue_version` 2.0 and above **Default value:** `null`
`max_retries` (`number`) optional
The maximum number of times to retry the job if it fails **Default value:** `null`
`non_overridable_arguments` (`map(string)`) optional
Non-overridable arguments for this job, specified as name-value pairs **Default value:** `null`
`notification_property` optional
Notification property of the job **Type:** ```hcl object({ # After a job run starts, the number of minutes to wait before sending a job run delay notification notify_delay_after = number }) ``` **Default value:** `null`
`number_of_workers` (`number`) optional
The number of workers of a defined `worker_type` that are allocated when a job runs **Default value:** `null`
`security_configuration` (`string`) optional
The name of the Security Configuration to be associated with the job **Default value:** `null`
`timeout` (`number`) optional
The job timeout in minutes. The default is 2880 minutes (48 hours) for `glueetl` and `pythonshell` jobs, and `null` (unlimited) for `gluestreaming` jobs **Default value:** `2880`
`worker_type` (`string`) optional
The type of predefined worker that is allocated when a job runs. Accepts a value of `Standard`, `G.1X`, or `G.2X` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`job_arn`
Glue job ARN
`job_id`
Glue job ID
`job_name`
Glue job name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_iam_role` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_job` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-job`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-job/0.4.0) | n/a `glue_job_s3_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.glue_job_aws_tools_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role_policy_attachment.glue_jobs_aws_tools_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.glue_redshift_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.glue_job_aws_tools_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## registry This component provisions Glue registries. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/registry/example: metadata: component: glue/registry vars: enabled: true name: example registry_name: example registry_description: "Glue registry example" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`registry_description` (`string`) optional
Glue registry description **Default value:** `null`
`registry_name` (`string`) optional
Glue registry name. If not provided, the name will be generated from the context **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`registry_arn`
Glue registry ARN
`registry_id`
Glue registry ID
`registry_name`
Glue registry name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_registry` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-registry`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-registry/0.4.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## schema This component provisions Glue schemas. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/schema/example: metadata: component: glue/schema vars: enabled: true name: example schema_name: example schema_description: "Glue schema example" data_format: JSON glue_registry_component_name: "glue/registry/example" ``` ## Variables ### Required Variables
`glue_registry_component_name` (`string`) required
Glue registry component name. Used to get the Glue registry from the remote state
`region` (`string`) required
AWS Region
### Optional Variables
`compatibility` (`string`) optional
The compatibility mode of the schema. Valid values are NONE, DISABLED, BACKWARD, BACKWARD_ALL, FORWARD, FORWARD_ALL, FULL, and FULL_ALL **Default value:** `"NONE"`
`data_format` (`string`) optional
The data format of the schema definition. Valid values are `AVRO`, `JSON` and `PROTOBUF` **Default value:** `"JSON"`
`schema_definition` (`string`) optional
The schema definition using the `data_format` setting **Default value:** `null`
`schema_description` (`string`) optional
Glue schema description **Default value:** `null`
`schema_name` (`string`) optional
Glue schema name. If not provided, the name will be generated from the context **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`latest_schema_version`
The latest version of the schema associated with the returned schema definition
`next_schema_version`
The next version of the schema associated with the returned schema definition
`registry_name`
Glue registry name
`schema_arn`
Glue schema ARN
`schema_checkpoint`
The version number of the checkpoint (the last time the compatibility mode was changed)
`schema_id`
Glue schema ID
`schema_name`
Glue schema name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_registry` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_schema` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-schema`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-schema/0.4.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## trigger This component provisions Glue triggers. ## Usage **Stack Level**: Regional ```yaml components: terraform: glue/trigger/example: metadata: component: glue/trigger vars: enabled: true name: example trigger_name: example trigger_description: "Glue trigger example" glue_workflow_component_name: "glue/workflow/example" glue_job_component_name: "glue/job/example" glue_job_timeout: 10 trigger_enabled: true start_on_creation: true schedule: "cron(15 12 * * ? *)" type: SCHEDULED ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`actions` (`list(any)`) optional
List of actions initiated by the trigger when it fires **Default value:** `null`
`event_batching_condition` (`map(number)`) optional
Batch condition that must be met (specified number of events received or batch time window expired) before EventBridge event trigger fires **Default value:** `null`
`glue_job_component_name` (`string`) optional
Glue workflow job name. Used to get the Glue job from the remote state **Default value:** `null`
`glue_job_timeout` (`number`) optional
The job run timeout in minutes. It overrides the timeout value of the job **Default value:** `null`
`glue_workflow_component_name` (`string`) optional
Glue workflow component name. Used to get the Glue workflow from the remote state **Default value:** `null`
`predicate` (`any`) optional
A predicate to specify when the new trigger should fire. Required when trigger type is `CONDITIONAL` **Default value:** `null`
`schedule` (`string`) optional
Cron formatted schedule. Required for triggers with type `SCHEDULED` **Default value:** `null`
`start_on_creation` (`bool`) optional
Set to `true` to start `SCHEDULED` and `CONDITIONAL` triggers when created. `true` is not supported for `ON_DEMAND` triggers **Default value:** `true`
`trigger_description` (`string`) optional
Glue trigger description **Default value:** `null`
`trigger_enabled` (`bool`) optional
Whether to start the created trigger **Default value:** `true`
`trigger_name` (`string`) optional
Glue trigger name. If not provided, the name will be generated from the context **Default value:** `null`
`type` (`string`) optional
The type of trigger. Options are CONDITIONAL, SCHEDULED or ON_DEMAND **Default value:** `"CONDITIONAL"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`trigger_arn`
Glue trigger ARN
`trigger_id`
Glue trigger ID
`trigger_name`
Glue trigger name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_job` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `glue_trigger` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-trigger`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-trigger/0.4.0) | n/a `glue_workflow` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## workflow This component provisions Glue workflows. Stack Level: Regional ## Usage ```yaml components: terraform: glue/workflow/example: metadata: component: "glue/workflow" vars: enabled: true name: example workflow_name: example workflow_description: "Glue workflow example" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`default_run_properties` (`map(string)`) optional
A map of default run properties for this workflow. These properties are passed to all jobs associated to the workflow **Default value:** `null`
`max_concurrent_runs` (`number`) optional
Maximum number of concurrent runs. If you leave this parameter blank, there is no limit to the number of concurrent workflow runs **Default value:** `null`
`workflow_description` (`string`) optional
Glue workflow description **Default value:** `null`
`workflow_name` (`string`) optional
Glue workflow name. If not provided, the name will be generated from the context **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`workflow_arn`
Glue workflow ARN
`workflow_id`
Glue workflow ID
`workflow_name`
Glue workflow name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `utils`, version: `>= 1.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `glue_workflow` | 0.4.0 | [`cloudposse/glue/aws//modules/glue-workflow`](https://registry.terraform.io/modules/cloudposse/glue/aws/modules/glue-workflow/0.4.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## guardduty This component is responsible for configuring GuardDuty within an AWS Organization. AWS GuardDuty is a managed threat detection service. It is designed to help protect AWS accounts and workloads by continuously monitoring for malicious activities and unauthorized behaviors. To detect potential security threats, GuardDuty analyzes various data sources within your AWS environment, such as AWS CloudTrail logs, VPC Flow Logs, and DNS logs. Key features and components of AWS GuardDuty include: - Threat detection: GuardDuty employs machine learning algorithms, anomaly detection, and integrated threat intelligence to identify suspicious activities, unauthorized access attempts, and potential security threats. It analyzes event logs and network traffic data to detect patterns, anomalies, and known attack techniques. - Threat intelligence: GuardDuty leverages threat intelligence feeds from AWS, trusted partners, and the global community to enhance its detection capabilities. It uses this intelligence to identify known malicious IP addresses, domains, and other indicators of compromise. - Real-time alerts: When GuardDuty identifies a potential security issue, it generates real-time alerts that can be delivered through AWS CloudWatch Events. These alerts can be integrated with other AWS services like Amazon SNS or AWS Lambda for immediate action or custom response workflows. - Multi-account support: GuardDuty can be enabled across multiple AWS accounts, allowing centralized management and monitoring of security across an entire organization's AWS infrastructure. This helps to maintain consistent security policies and practices. - Automated remediation: GuardDuty integrates with other AWS services, such as AWS Macie, AWS Security Hub, and AWS Systems Manager, to facilitate automated threat response and remediation actions. This helps to minimize the impact of security incidents and reduces the need for manual intervention. - Security findings and reports: GuardDuty provides detailed security findings and reports that include information about detected threats, affected AWS resources, and recommended remediation actions. These findings can be accessed through the AWS Management Console or retrieved via APIs for further analysis and reporting. GuardDuty offers a scalable and flexible approach to threat detection within AWS environments, providing organizations with an additional layer of security to proactively identify and respond to potential security risks. ## Usage **Stack Level**: Regional ## Deployment Overview This component is complex in that it must be deployed multiple times with different variables set to configure the AWS Organization successfully. It is further complicated by the fact that you must deploy each of the the component instances described below to every region that existed before March 2019 and to any regions that have been opted-in as described in the [AWS Documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions). In the examples below, we assume that the AWS Organization Management account is `root` and the AWS Organization Delegated Administrator account is `security`, both in the `core` tenant. ### Deploy to Delegated Administrator Account First, the component is deployed to the [Delegated Administrator](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_organizations.html) account in each region in order to configure the central GuardDuty detector that each account will send its findings to. ```yaml # core-ue1-security components: terraform: guardduty/delegated-administrator/ue1: metadata: component: guardduty vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 ``` ```bash atmos terraform apply guardduty/delegated-administrator/ue1 -s core-ue1-security atmos terraform apply guardduty/delegated-administrator/ue2 -s core-ue2-security atmos terraform apply guardduty/delegated-administrator/uw1 -s core-uw1-security # ... other regions ``` ### Deploy to Organization Management (root) Account Next, the component is deployed to the AWS Organization Management, a/k/a `root`, Account in order to set the AWS Organization Designated Administrator account. Note that you must use the `SuperAdmin` permissions as we are deploying to the AWS Organization Management account. Since we are using the `SuperAdmin` user, it will already have access to the state bucket, so we set the `role_arn` of the backend config to null and set `var.privileged` to `true`. ```yaml # core-ue1-root components: terraform: guardduty/root/ue1: metadata: component: guardduty backend: s3: role_arn: null vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 privileged: true ``` ```bash atmos terraform apply guardduty/root/ue1 -s core-ue1-root atmos terraform apply guardduty/root/ue2 -s core-ue2-root atmos terraform apply guardduty/root/uw1 -s core-uw1-root # ... other regions ``` ### Deploy Organization Settings in Delegated Administrator Account Finally, the component is deployed to the Delegated Administrator Account again in order to create the organization-wide configuration for the AWS Organization, but with `var.admin_delegated` set to `true` to indicate that the delegation has already been performed from the Organization Management account. ```yaml # core-ue1-security components: terraform: guardduty/org-settings/ue1: metadata: component: guardduty vars: enabled: true delegated_administrator_account_name: core-security environment: use1 region: us-east-1 admin_delegated: true ``` ```bash atmos terraform apply guardduty/org-settings/ue1 -s core-ue1-security atmos terraform apply guardduty/org-settings/ue2 -s core-ue2-security atmos terraform apply guardduty/org-settings/uw1 -s core-uw1-security # ... other regions ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_tenant` (`string`) optional
The tenant where the `account_map` component required by remote-state is deployed **Default value:** `"core"`
`admin_delegated` (`bool`) optional
A flag to indicate if the AWS Organization-wide settings should be created. This can only be done after the GuardDuty Administrator account has already been delegated from the AWS Org Management account (usually 'root'). See the Deployment section of the README for more information. **Default value:** `false`
`auto_enable_organization_members` (`string`) optional
Indicates the auto-enablement configuration of GuardDuty for the member accounts in the organization. Valid values are `ALL`, `NEW`, `NONE`. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_organization_configuration#auto_enable_organization_members **Default value:** `"NEW"`
`cloudwatch_enabled` (`bool`) optional
Flag to indicate whether CloudWatch logging should be enabled for GuardDuty **Default value:** `false`
`cloudwatch_event_rule_pattern_detail_type` (`string`) optional
The detail-type pattern used to match events that will be sent to SNS. For more information, see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html https://docs.aws.amazon.com/eventbridge/latest/userguide/event-types.html https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html **Default value:** `"GuardDuty Finding"`
`create_sns_topic` (`bool`) optional
Flag to indicate whether an SNS topic should be created for notifications. If you want to send findings to a new SNS topic, set this to true and provide a valid configuration for subscribers. **Default value:** `false`
`delegated_admininstrator_component_name` (`string`) optional
The name of the component that created the GuardDuty detector. **Default value:** `"guardduty/delegated-administrator"`
`delegated_administrator_account_name` (`string`) optional
The name of the account that is the AWS Organization Delegated Administrator account **Default value:** `"core-security"`
`detector_features` optional
A map of detector features for streaming foundational data sources to detect communication with known malicious domains and IP addresses and identify anomalous behavior. For more information, see: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty-features-activation-model.html#guardduty-features feature_name: The name of the detector feature. Possible values include: S3_DATA_EVENTS, EKS_AUDIT_LOGS, EBS_MALWARE_PROTECTION, RDS_LOGIN_EVENTS, EKS_RUNTIME_MONITORING, LAMBDA_NETWORK_LOGS, RUNTIME_MONITORING. Specifying both EKS Runtime Monitoring (EKS_RUNTIME_MONITORING) and Runtime Monitoring (RUNTIME_MONITORING) will cause an error. You can add only one of these two features because Runtime Monitoring already includes the threat detection for Amazon EKS resources. For more information, see: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_DetectorFeatureConfiguration.html. status: The status of the detector feature. Valid values include: ENABLED or DISABLED. additional_configuration: Optional information about the additional configuration for a feature in your GuardDuty account. For more information, see: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_DetectorAdditionalConfiguration.html. addon_name: The name of the add-on for which the configuration applies. Possible values include: EKS_ADDON_MANAGEMENT, ECS_FARGATE_AGENT_MANAGEMENT, and EC2_AGENT_MANAGEMENT. For more information, see: https://docs.aws.amazon.com/guardduty/latest/APIReference/API_DetectorAdditionalConfiguration.html. status: The status of the add-on. Valid values include: ENABLED or DISABLED. **Type:** ```hcl map(object({ feature_name = string status = string additional_configuration = optional(object({ addon_name = string status = string }), null) })) ``` **Default value:** `{ }`
`finding_publishing_frequency` (`string`) optional
The frequency of notifications sent for finding occurrences. If the detector is a GuardDuty member account, the value is determined by the GuardDuty master account and cannot be modified, otherwise it defaults to SIX_HOURS. For standalone and GuardDuty master accounts, it must be configured in Terraform to enable drift detection. Valid values for standalone and master accounts: FIFTEEN_MINUTES, ONE_HOUR, SIX_HOURS." For more information, see: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html#guardduty_findings_cloudwatch_notification_frequency **Default value:** `null`
`findings_notification_arn` (`string`) optional
The ARN for an SNS topic to send findings notifications to. This is only used if create_sns_topic is false. If you want to send findings to an existing SNS topic, set this to the ARN of the existing topic and set create_sns_topic to false. **Default value:** `null`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`kubernetes_audit_logs_enabled` (`bool`) optional
If `true`, enables Kubernetes audit logs as a data source for Kubernetes protection. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector#audit_logs **Default value:** `false`
`malware_protection_scan_ec2_ebs_volumes_enabled` (`bool`) optional
Configure whether Malware Protection is enabled as data source for EC2 instances EBS Volumes in GuardDuty. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector#malware-protection **Default value:** `false`
`organization_management_account_name` (`string`) optional
The name of the AWS Organization management account **Default value:** `null`
`privileged` (`bool`) optional
true if the default provider already has access to the backend **Default value:** `false`
`root_account_stage` (`string`) optional
The stage name for the Organization root (management) account. This is used to lookup account IDs from account names using the `account-map` component. **Default value:** `"root"`
`s3_protection_enabled` (`bool`) optional
If `true`, enables S3 protection. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector#s3-logs **Default value:** `true`
`subscribers` optional
A map of subscription configurations for SNS topics For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription#argument-reference protocol: The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see link) (email is an option but is unsupported in terraform, see link). endpoint: The endpoint to send data to, the contents will vary with the protocol. (see link for more information) endpoint_auto_confirms: Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty. Default is false. raw_message_delivery: Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property). Default is false. **Type:** ```hcl map(object({ protocol = string endpoint = string endpoint_auto_confirms = bool raw_message_delivery = bool })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`delegated_administrator_account_id`
The AWS Account ID of the AWS Organization delegated administrator account
`guardduty_detector_arn`
The ARN of the GuardDuty detector created by the component
`guardduty_detector_id`
The ID of the GuardDuty detector created by the component
`sns_topic_name`
The name of the SNS topic created by the component
`sns_topic_subscriptions`
The SNS topic subscriptions created by the component
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Providers - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `guardduty` | 0.6.0 | [`cloudposse/guardduty/aws`](https://registry.terraform.io/modules/cloudposse/guardduty/aws/0.6.0) | If we are are in the AWS Org designated administrator account, enable the GuardDuty detector and optionally create an SNS topic for notifications and CloudWatch event rules for findings `guardduty_delegated_detector` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_guardduty_detector_feature.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector_feature) (resource) - [`aws_guardduty_organization_admin_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_organization_admin_account) (resource) - [`aws_guardduty_organization_configuration.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_organization_configuration) (resource) - [`awsutils_guardduty_organization_settings.this`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/resources/guardduty_organization_settings) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) --- ## iam-policy Terraform component that composes IAM policy documents and creates an AWS IAM policy. - Uses the Cloud Posse `cloudposse/iam-policy/aws` module to merge multiple policy document sources with override semantics. - Exports the rendered policy JSON (`output.json`) and the ARN of the created policy (`output.policy_arn`). - Supports adding base (`iam_source_policy_documents`) and override (`iam_override_policy_documents`) JSON documents, plus first-class typed `iam_policy` statements. Note: Description of this component 55 ## Usage **Stack Level**: Regional or Global Minimal Atmos stack example showing inline statements plus merged JSON documents: ```yaml components: terraform: policy: vars: description: "Example IAM policy" # Optional: typed statements (compatible with aws_iam_policy_document) iam_policy: - policy_id: "EC2DescribeInstances" statements: - sid: "EC2DescribeInstances" effect: "Allow" actions: ["ec2:DescribeInstances"] resources: ["*"] # Optional: base source policy documents (JSON strings) iam_source_policy_documents: - | {"Version":"2012-10-17","Statement":[{"Sid":"KMS","Effect":"Allow","Action":["kms:*"] ,"Resource":"*"}]} # Optional: higher-precedence override documents (JSON strings) iam_override_policy_documents: - | {"Version":"2012-10-17","Statement":[{"Sid":"S3ReadWrite","Effect":"Allow","Action":["s3:GetObject","s3:ListBucket","s3:ListBucketMultipartUploads","s3:ListBucketVersions","s3:ListMultipartUploadParts","s3:PutObject","s3:HeadObject"],"Resource":"*"}]} ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`description` (`string`) optional
Description of created IAM policy **Default value:** `null`
`iam_override_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document with higher precedence. In merging, statements with non-blank SIDs will override statements with the same SID from earlier documents in the list and from other "source" documents. **Default value:** `null`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`iam_source_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document. Statements defined in `iam_source_policy_documents` must have unique SIDs and be distinct from SIDs in `iam_policy`. Statements in these documents will be overridden by statements with the same SID in `iam_override_policy_documents`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`json`
JSON body of the IAM policy document
`policy_arn`
ARN of created IAM policy
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.22.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## iam-role This component is responsible for provisioning simple IAM roles. If a more complicated IAM role and policy are desired then it is better to use a separate component specific to that role. ## Usage **Stack Level**: Global Abstract ```yaml # stacks/catalog/iam-role.yaml components: terraform: iam-role/defaults: metadata: type: abstract component: iam-role settings: spacelift: workspace_enabled: true vars: enabled: true ``` Use-case: An IAM role for AWS Workspaces Directory since this service does not have a service linked role. ```yaml # stacks/catalog/aws-workspaces/directory/iam-role.yaml import: - catalog/iam-role components: terraform: aws-workspaces/directory/iam-role: metadata: component: iam-role inherits: - iam-role/defaults vars: name: workspaces_DefaultRole # Added _ here to allow the _ character regex_replace_chars: /[^a-zA-Z0-9-_]/ # Keep the current name casing label_value_case: none # Use the "name" without the other context inputs i.e. namespace, tenant, environment, attributes use_fullname: false role_description: | Used with AWS Workspaces Directory. The name of the role does not match the normal naming convention because this name is a requirement to work with the service. This role has to be used until AWS provides the respective service linked role. principals: Service: - workspaces.amazonaws.com # This will prevent the creation of a managed IAM policy policy_document_count: 0 managed_policy_arns: - arn:aws:iam::aws:policy/AmazonWorkSpacesServiceAccess - arn:aws:iam::aws:policy/AmazonWorkSpacesSelfServiceAccess ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`role_description` (`string`) required
The description of the IAM role that is visible in the IAM role manager
### Optional Variables
`assume_role_actions` (`list(string)`) optional
The IAM action to be granted by the AssumeRole policy **Default value:** ```hcl [ "sts:AssumeRole", "sts:SetSourceIdentity", "sts:TagSession" ] ```
`assume_role_conditions` optional
List of conditions for the assume role policy **Type:** ```hcl list(object({ test = string variable = string values = list(string) })) ``` **Default value:** `[ ]`
`assume_role_policy` (`string`) optional
A JSON assume role policy document. If set, this will be used as the assume role policy and the principals, assume_role_conditions, and assume_role_actions variables will be ignored. **Default value:** `null`
`instance_profile_enabled` (`bool`) optional
Create EC2 Instance Profile for the role **Default value:** `false`
`managed_policy_arns` (`set(string)`) optional
List of managed policies to attach to created role **Default value:** `[ ]`
`max_session_duration` (`number`) optional
The maximum session duration (in seconds) for the role. Can have a value from 1 hour to 12 hours **Default value:** `3600`
`path` (`string`) optional
Path to the role and policy. See [IAM Identifiers](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html) for more information. **Default value:** `"/"`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role **Default value:** `""`
`policy_description` (`string`) optional
The description of the IAM policy that is visible in the IAM policy manager **Default value:** `""`
`policy_document_count` (`number`) optional
Number of policy documents (length of policy_documents list) **Default value:** `1`
`policy_documents` (`list(string)`) optional
List of JSON IAM policy documents **Default value:** `[ ]`
`policy_name` (`string`) optional
The name of the IAM policy that is visible in the IAM policy manager **Default value:** `null`
`principals` (`map(list(string))`) optional
Map of service name as key and a list of ARNs to allow assuming the role as value (e.g. map(`AWS`, list(`arn:aws:iam:::role/admin`))) **Default value:** `{ }`
`use_fullname` (`bool`) optional
If set to 'true' then the full ID for the IAM role name (e.g. `[var.namespace]-[var.environment]-[var.stage]`) will be used. Otherwise, `var.name` will be used for the IAM role name. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role`
IAM role module outputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `role` | 0.22.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.22.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## iam-service-linked-roles This component is responsible for provisioning [IAM Service-Linked Roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html). ## Usage **Stack Level**: Global ```yaml components: terraform: iam-service-linked-roles: settings: spacelift: workspace_enabled: true vars: enabled: true service_linked_roles: spot_amazonaws_com: aws_service_name: "spot.amazonaws.com" description: "AWSServiceRoleForEC2Spot Service-Linked Role for EC2 Spot" spotfleet_amazonaws_com: aws_service_name: "spotfleet.amazonaws.com" description: "AWSServiceRoleForEC2SpotFleet Service-Linked Role for EC2 Spot Fleet" ``` ## Service-Linked Roles for EC2 Spot and EC2 Spot Fleet If you want to use EC2 Spot or Spot Fleet, you will need to provision the following Service-Linked Roles: - Service-Linked Role for EC2 Spot - Service-Linked Role for EC2 Spot Fleet This is only necessary if this is the first time you're using EC2 Spot and Spot Fleet in the account. Note that if the Service-Linked Roles already exist in the AWS account (for example, if you used EC2 Spot or Spot Fleet before), and you try to provision them again, you will see errors like the following: ```text An error occurred (InvalidInput) when calling the CreateServiceLinkedRole operation: Service role name AWSServiceRoleForEC2Spot has been taken in this account, please try a different suffix An error occurred (InvalidInput) when calling the CreateServiceLinkedRole operation: Service role name AWSServiceRoleForEC2SpotFleet has been taken in this account, please try a different suffix ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`service_linked_roles` required
Service-Linked roles config **Type:** ```hcl map(object({ aws_service_name = string description = string })) ```
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`service_linked_roles`
Provisioned Service-Linked roles
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0, < 6.0.0` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_service_linked_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) (resource) ## Data Sources The following data sources are used by this module: --- ## identity-center This component is responsible for creating [AWS SSO Permission Sets][1] and creating AWS SSO Account Assignments, that is, assigning IdP (Okta) groups and/or users to AWS SSO permission sets in specific AWS Accounts. This component assumes that AWS SSO has already been enabled via the AWS Console (there isn't terraform or AWS CLI support for this currently) and that the IdP has been configured to sync users and groups to AWS SSO. ## Usage ### Clickops 1. Go to root admin account 1. Select primary region 1. Go to AWS SSO 1. Enable AWS SSO #### Delegation no longer recommended Previously, Cloud Posse recommended delegating SSO to the identity account by following the next 2 steps: 1. Click Settings > Management 1. Delegate Identity as an administrator. This can take up to 30 minutes to take effect. However, this is no longer recommended. Because the delegated SSO administrator cannot make changes in the `root` account and this component needs to be able to make changes in the `root` account, any purported security advantage achieved by delegating SSO to the `identity` account is lost. Nevertheless, it is also not worth the effort to remove the delegation. If you have already delegated SSO to the `identity`, continue on, leaving the stack configuration in the `gbl-identity` stack rather than the currently recommended `gbl-root` stack. ### Google Workspace :::important > Your identity source is currently configured as 'External identity provider'. To add new groups or edit their > memberships, you must do this using your external identity provider. Groups _cannot_ be created with ClickOps in the AWS console and instead must be created with AWS API. ::: Google Workspace is now supported by AWS Identity Center, but Group creation is not automatically handled. After [configuring SAML and SCIM with Google Workspace and IAM Identity Center following the AWS documentation](https://docs.aws.amazon.com/singlesignon/latest/userguide/gs-gwp.html), add any Group name to `var.groups` to create the Group with Terraform. Once the setup steps as described in the AWS documentation have been completed and the Groups are created with Terraform, Users should automatically populate each created Group. ```yaml components: terraform: aws-sso: vars: groups: - "Developers" - "Dev Ops" ``` ### Atmos **Stack Level**: Global **Deployment**: Must be deployed by root-admin using `atmos` CLI Add catalog to `gbl-root` root stack. #### `account_assignments` The `account_assignments` setting configures access to permission sets for users and groups in accounts, in the following structure: ```yaml : groups: : permission_sets: - users: : permission_sets: - ``` - The account names (a.k.a. "stages") must already be configured via the `accounts` component. - The user and group names must already exist in AWS SSO. Usually this is accomplished by configuring them in Okta and syncing Okta with AWS SSO. - The permission sets are defined (by convention) in files names `policy-.tf` in the `aws-sso` component. The definition includes the name of the permission set. See `components/terraform/aws-sso/policy-AdminstratorAccess.tf` for an example. #### `identity_roles_accessible` The `identity_roles_accessible` element provides a list of role names corresponding to roles created in the `iam-primary-roles` component. For each named role, a corresponding permission set will be created which allows the user to assume that role. The permission set name is generated in Terraform from the role name using a statement like this one: ``` format("Identity%sTeamAccess", replace(title(replace(team, "_", "-")), "-", "")) ``` ### Defining a new permission set 1. Give the permission set a name, capitalized, in CamelCase, e.g. `AuditManager`. We will use `NAME` as a placeholder for the name in the instructions below. In Terraform, convert the name to lowercase snake case, e.g. `audit_manager`. 2. Create a file in the `aws-sso` directory with the name `policy-NAME.tf`. 3. In that file, create a policy as follows: ```hcl data "aws_iam_policy_document" "TerraformUpdateAccess" { # Define the custom policy here } locals { NAME_permission_set = { # e.g. audit_manager_permission_set name = "NAME", # e.g. AuditManager description = "", relay_state = "", session_duration = "PT1H", # One hour, maximum allowed for chained assumed roles tags = {}, inline_policy = data.aws_iam_policy_document.NAME.json, policy_attachments = [] # ARNs of AWS managed IAM policies to attach, e.g. arn:aws:iam::aws:policy/ReadOnlyAccess customer_managed_policy_attachments = [] # ARNs of customer managed IAM policies to attach } } ``` 4. Create a file named `additional-permission-sets-list_override.tf` in the `aws-sso` directory (if it does not already exist). This is a [terraform override file](https://developer.hashicorp.com/terraform/language/files/override), meaning its contents will be merged with the main terraform file, and any locals defined in it will override locals defined in other files. Having your code in this separate override file makes it possible for the component to provide a placeholder local variable so that it works without customization, while allowing you to customize the component and still update it without losing your customizations. 5. In that file, redefine the local variable `overridable_additional_permission_sets` as follows: ```hcl locals { overridable_additional_permission_sets = [ local.NAME_permission_set, ] } ``` If you have multiple custom policies, add each one to the list. 6. With that done, the new permission set will be created when the changes are applied. You can then use it just like the others. 7. If you want the permission set to be able to use Terraform, enable access to the Terraform state read/write (default) role in `tfstate-backend`. #### Example The example snippet below shows how to use this module with various combinations (plain YAML, YAML Anchors and a combination of the two): ```yaml prod-cloud-engineers: &prod-cloud-engineers Production Cloud Infrastructure Engineers: permission_sets: - AdministratorAccess - ReadOnlyAccess components: terraform: aws-sso: vars: account_assignments: audit: groups: <<: *prod-cloud-engineers Production Cloud Engineers: permission_sets: - ReadOnlyAccess corp: groups: *prod-cloud-engineers prod: groups: Administrators: permission_sets: - AdministratorAccess - ReadOnlyAccess Developers: permission_sets: - ReadOnlyAccess dev: groups: Administrators: permission_sets: - AdministratorAccess - ReadOnlyAccess Developers: permission_sets: - AdministratorAccess - ReadOnlyAccess aws_teams_accessible: - "developers" - "devops" - "managers" - "support" ``` [][40] [1]: https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html [2]: #requirement%5C_terraform [3]: #requirement%5C_aws [4]: #requirement%5C_external [5]: #requirement%5C_local [6]: #requirement%5C_template [7]: #requirement%5C_utils [8]: #provider%5C_aws [9]: #module%5C_account%5C_map [10]: #module%5C_permission%5C_sets [11]: #module%5C_role%5C_prefix [12]: #module%5C_sso%5C_account%5C_assignments [13]: #module%5C_this [14]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document [15]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document [16]: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document [17]: #input%5C_account%5C_assignments [18]: #input%5C_additional%5C_tag%5C_map [19]: #input%5C_attributes [20]: #input%5C_context [21]: #input%5C_delimiter [22]: #input%5C_enabled [23]: #input%5C_environment [24]: #input%5C_global%5C_environment%5C_name [25]: #input%5C_iam%5C_primary%5C_roles%5C_stage%5C_name [26]: #input%5C_id%5C_length%5C_limit [27]: #input%5C_identity%5C_roles%5C_accessible [28]: #input%5C_label%5C_key%5C_case [29]: #input%5C_label%5C_order [30]: #input%5C_label%5C_value%5C_case [31]: #input%5C_name [32]: #input%5C_namespace [33]: #input%5C_privileged [34]: #input%5C_regex%5C_replace%5C_chars [35]: #input%5C_region [36]: #input%5C_root%5C_account%5C_stage%5C_name [37]: #input%5C_stage [38]: #input%5C_tags [39]: https://github.com/cloudposse/terraform-aws-sso [40]: https://cpco.io/component ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_assignments` optional
Enables access to permission sets for users and groups in accounts, in the following structure: ```yaml : groups: : permission_sets: - users: : permission_sets: - ``` **Type:** ```hcl map(map(map(object({ permission_sets = list(string) } )))) ``` **Default value:** `{ }`
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`aws_teams_accessible` (`set(string)`) optional
List of IAM roles (e.g. ["admin", "terraform"]) for which to create permission sets that allow the user to assume that role. Named like admin -> IdentityAdminTeamAccess **Default value:** `[ ]`
`groups` (`list(string)`) optional
List of AWS Identity Center Groups to be created with the AWS API. When provisioning the Google Workspace Integration with AWS, Groups need to be created with API in order for automatic provisioning to work as intended. **Default value:** `[ ]`
`overridable_team_permission_set_name_pattern` (`string`) optional
The pattern used to generate the AWS SSO PermissionSet name for each team **Default value:** `"Identity%sTeamAccess"`
`privileged` (`bool`) optional
True if the user running the Terraform command already has access to the Terraform backend **Default value:** `false`
`session_duration` (`string`) optional
The default duration of the session in seconds for all permission sets. If not set, fallback to the default value in the module, which is 1 hour. **Default value:** `""`
`tfstate_backend_component_name` (`string`) optional
The name of the tfstate-backend component **Default value:** `"tfstate-backend"`
`tfstate_environment_name` (`string`) optional
The name of the environment where `tfstate-backend` is provisioned. If not set, the TerraformUpdateAccess permission set will not be created. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`group_ids`
Group IDs created for Identity Center
`permission_sets`
Permission sets
`sso_account_assignments`
SSO account assignments
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `iam_roles_root` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `permission_sets` | 1.2.0 | [`cloudposse/sso/aws//modules/permission-sets`](https://registry.terraform.io/modules/cloudposse/sso/aws/modules/permission-sets/1.2.0) | n/a `role_map` | latest | [`../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../account-map/modules/roles-to-principals/) | n/a `sso_account_assignments` | 1.2.0 | [`cloudposse/sso/aws//modules/account-assignments`](https://registry.terraform.io/modules/cloudposse/sso/aws/modules/account-assignments/1.2.0) | n/a `sso_account_assignments_root` | 1.2.0 | [`cloudposse/sso/aws//modules/account-assignments`](https://registry.terraform.io/modules/cloudposse/sso/aws/modules/account-assignments/1.2.0) | n/a `tfstate` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_identitystore_group.manual`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/identitystore_group) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_aws_team`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.dns_administrator_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.eks_read_only`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.terraform_update_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssoadmin_instances.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssoadmin_instances) (data source) --- ## ipam This component is responsible for provisioning IPAM per region in a centralized account. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: ipam: vars: enabled: true top_cidr: [10.96.0.0/11] pool_configurations: core: cidr: [10.96.0.0/12] locale: us-east-2 sub_pools: network: cidr: [10.96.0.0/16] ram_share_accounts: [core-network] auto: cidr: [10.97.0.0/16] ram_share_accounts: [core-auto] corp: cidr: [10.98.0.0/16] ram_share_accounts: [core-corp] plat: cidr: [10.112.0.0/12] locale: us-east-2 sub_pools: dev: cidr: [10.112.0.0/16] ram_share_accounts: [plat-dev] staging: cidr: [10.113.0.0/16] ram_share_accounts: [plat-staging] prod: cidr: [10.114.0.0/16] ram_share_accounts: [plat-prod] sandbox: cidr: [10.115.0.0/16] ram_share_accounts: [plat-sandbox] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`top_cidr` (`list(string)`) required
Top-level CIDR blocks.
### Optional Variables
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`address_family` (`string`) optional
IPv4/6 address family. **Default value:** `"ipv4"`
`ipam_scope_id` (`string`) optional
(Optional) Required if `var.ipam_id` is set. Determines which scope to deploy pools into. **Default value:** `null`
`ipam_scope_type` (`string`) optional
Which scope type to use. Valid inputs include `public` or `private`. You can alternatively provide your own scope ID. **Default value:** `"private"`
`pool_configurations` (`any`) optional
A multi-level, nested map describing nested IPAM pools. Can nest up to three levels with the top level being outside the `pool_configurations`. This attribute is quite complex, see README.md for further explanation. **Default value:** `{ }`
`top_auto_import` (`bool`) optional
`auto_import` setting for top-level pool. **Default value:** `null`
`top_cidr_authorization_context` (`any`) optional
A signed document that proves that you are authorized to bring the specified IP address range to Amazon using BYOIP. Document is not stored in the state file. For more information, refer to https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_ipam_pool_cidr#cidr_authorization_context. **Default value:** `null`
`top_description` (`string`) optional
Description of top-level pool. **Default value:** `""`
`top_ram_share_principals` (`list(string)`) optional
Principals to create RAM shares for top-level pool. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`pool_configurations`
Pool configurations
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `ipam` | 2.1.2 | [`aws-ia/ipam/aws`](https://registry.terraform.io/modules/aws-ia/ipam/aws/2.1.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## kinesis-firehose-stream This component provisions a Kinesis Firehose delivery stream and at this time supports CloudWatch to S3 delivery. It enables you to stream logs from EKS CloudWatch to an S3 bucket for long-term storage and analysis. ## Usage **Stack Level**: Regional Here's an example of how to set up a Firehose stream to capture EKS CloudWatch logs and deliver them to an S3 bucket: ```yaml components: terraform: # First, ensure you have the required dependencies: eks/cluster: vars: name: eks-cluster # ... other EKS cluster configuration eks/cloudwatch: vars: name: eks-cloudwatch # ... other CloudWatch configuration s3-bucket/cloudwatch: vars: name: cloudwatch-logs-bucket # ... other S3 bucket configuration # Then configure the Firehose stream: kinesis-firehose-stream/basic: metadata: component: kinesis-firehose-stream vars: name: cloudwatch-logs # Source CloudWatch component name source_cloudwatch_component_name: eks/cloudwatch # Destination S3 bucket component name destination_bucket_component_name: s3-bucket/cloudwatch # Optional: Enable encryption for the Firehose stream encryption_enabled: true ``` This configuration will: 1. Create a Kinesis Firehose delivery stream 2. Configure it to receive logs from the specified EKS CloudWatch component 3. Deliver the logs to the specified S3 bucket 4. Optionally enable encryption for the stream ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`destination_bucket_component_name` (`string`) optional
The name of the component that will be using the destination bucket **Default value:** `"s3-bucket/cloudwatch"`
`encryption_enabled` (`bool`) optional
Enable encryption for the Kinesis Firehose Delivery Stream **Default value:** `true`
`source_cloudwatch_component_name` (`string`) optional
The name of the component that will be using the source cloudwatch **Default value:** `"eks/cloudwatch"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`kinesis_firehose_stream_arn`
The ARN of the Kinesis Firehose stream
`kinesis_firehose_stream_id`
The ID of the Kinesis Firehose stream
`kinesis_firehose_stream_name`
The name of the Kinesis Firehose stream
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Providers - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `cloudwatch_subscription_role` | 0.22.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.22.0) | n/a `firehose_role` | 0.22.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.22.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `s3_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `stream_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_subscription_filter.firehose_delivery`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) (resource) - [`aws_kinesis_firehose_delivery_stream.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.cloudwatch_to_firehose`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.firehose_to_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_kms_alias.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_alias) (data source) --- ## kinesis-stream This component is responsible for provisioning an Amazon Kinesis data stream. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/catalog/kinesis-stream/defaults.yaml` file (base component for all kinesis deployments with default settings): ```yaml components: terraform: kinesis-stream/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true tags: Team: sre Service: kinesis-stream ``` ```yaml import: - catalog/kinesis-stream/defaults components: terraform: kinesis-example: metadata: component: kinesis-stream inherits: - kinesis-stream/defaults vars: name: kinesis-stream-example stream_mode: ON_DEMAND # shard_count: 2 # This does nothing if `stream_mode` is set to `ON_DEMAND` kms_key_id: "alias/aws/kinesis" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`consumer_count` (`number`) optional
Number of consumers to register with the Kinesis stream. **Default value:** `0`
`encryption_type` (`string`) optional
The encryption type to use. Acceptable values are `NONE` and `KMS`. **Default value:** `"KMS"`
`enforce_consumer_deletion` (`bool`) optional
Forcefully delete stream consumers before destroying the stream. **Default value:** `true`
`kms_key_id` (`string`) optional
The name of the KMS key to use for encryption. **Default value:** `"alias/aws/kinesis"`
`retention_period` (`number`) optional
Length of time data records are accessible after they are added to the stream. The maximum value is 168 hours. Minimum value is 24. **Default value:** `24`
`shard_count` (`number`) optional
The number of shards to provision for the stream. **Default value:** `1`
`shard_level_metrics` (`list(string)`) optional
A list of shard-level CloudWatch metrics to enabled for the stream. **Default value:** ```hcl [ "IncomingBytes", "OutgoingBytes" ] ```
`stream_mode` (`string`) optional
Specifies the capacity mode of the stream. Must be either `PROVISIONED` or `ON_DEMAND`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`name`
Name of the Kinesis stream.
`shard_count`
Number of shards provisioned.
`stream_arn`
ARN of the the Kinesis stream.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kinesis` | 0.4.0 | [`cloudposse/kinesis-stream/aws`](https://registry.terraform.io/modules/cloudposse/kinesis-stream/aws/0.4.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## kms This component is responsible for provisioning a KMS Key. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: kms: settings: spacelift: workspace_enabled: true vars: enabled: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`alias` (`string`) optional
The display name of the alias. The name must start with the word alias followed by a forward slash. If not specified, the alias name will be auto-generated. **Default value:** `null`
`allowed_principal_arns` (`list(string)`) optional
List of AWS principal ARNs allowed to assume the role. **Default value:** `[ ]`
`allowed_roles` (`map(list(string))`) optional
Map of account:[role, role...] specifying roles allowed to assume the role. Roles are symbolic names like `ops` or `terraform`. Use `*` as role for entire account. **Default value:** `{ }`
`customer_master_key_spec` (`string`) optional
Specifies whether the key contains a symmetric key or an asymmetric key pair and the encryption algorithms or signing algorithms that the key supports. Valid values: `SYMMETRIC_DEFAULT`, `RSA_2048`, `RSA_3072`, `RSA_4096`, `ECC_NIST_P256`, `ECC_NIST_P384`, `ECC_NIST_P521`, or `ECC_SECG_P256K1`. **Default value:** `"SYMMETRIC_DEFAULT"`
`deletion_window_in_days` (`number`) optional
Duration in days after which the key is deleted after destruction of the resource **Default value:** `10`
`description` (`string`) optional
The description for the KMS Key. **Default value:** `"Parameter Store KMS master key"`
`enable_key_rotation` (`bool`) optional
Specifies whether key rotation is enabled **Default value:** `true`
`key_usage` (`string`) optional
Specifies the intended use of the key. Valid values: `ENCRYPT_DECRYPT` or `SIGN_VERIFY`. **Default value:** `"ENCRYPT_DECRYPT"`
`multi_region` (`bool`) optional
Indicates whether the KMS key is a multi-Region (true) or regional (false) key. **Default value:** `false`
`policy` (`string`) optional
A valid KMS policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`kms_key`
Output for KMS module
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `allowed_role_map` | latest | [`../account-map/modules/roles-to-principals`](https://registry.terraform.io/modules/../account-map/modules/roles-to-principals/) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.key_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## lakeformation This component is responsible for provisioning Amazon Lake Formation resources. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/catalog/lakeformation/defaults.yaml` file (base component for all lakeformation deployments with default settings): ```yaml components: terraform: lakeformation/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true tags: Team: sre Service: lakeformation ``` ```yaml import: - catalog/lakeformation/defaults components: terraform: lakeformation-example: metadata: component: lakeformation inherits: - lakeformation/defaults vars: enabled: true name: lakeformation-example s3_bucket_arn: arn:aws:s3:::some-test-bucket create_service_linked_role: true admin_arn_list: - arn:aws:iam::012345678912:role/my-admin-role lf_tags: left: ["test1", "test2"] right: ["test3", "test4"] resources: database: name: example_db_1 tags: left: test1 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`s3_bucket_arn` (`string`) required
Amazon Resource Name (ARN) of the Lake Formation resource, an S3 path.
### Optional Variables
`admin_arn_list` (`list(string)`) optional
(Optional) Set of ARNs of AWS Lake Formation principals (IAM users or roles). **Default value:** `[ ]`
`catalog_id` (`string`) optional
(Optional) Identifier for the Data Catalog. If not provided, the account ID will be used. **Default value:** `null`
`create_service_linked_role` (`bool`) optional
Set to 'true' to create service-linked role for Lake Formation (can only be done once!) **Default value:** `false`
`database_default_permissions` (`list(map(any))`) optional
(Optional) Up to three configuration blocks of principal permissions for default create database permissions. **Default value:** `[ ]`
`lf_tags` (`map(list(string))`) optional
A map of key-value pairs to be used as Lake Formation tags. **Default value:** `{ }`
`resources` (`map(any)`) optional
A map of Lake Formation resources to create, with related attributes. **Default value:** `{ }`
`role_arn` (`string`) optional
(Optional) Role that has read/write access to the Lake Formation resource. If not provided, the Lake Formation service-linked role must exist and is used. **Default value:** `null`
`table_default_permissions` (`list(map(any))`) optional
(Optional) Up to three configuration blocks of principal permissions for default create table permissions. **Default value:** `[ ]`
`trusted_resource_owners` (`list(string)`) optional
(Optional) List of the resource-owning account IDs that the caller's account can use to share their user access details (user ARNs). **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lf_tags`
List of LF tags created.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `lakeformation` | 1.0.0 | [`cloudposse/lakeformation/aws`](https://registry.terraform.io/modules/cloudposse/lakeformation/aws/1.0.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_service_linked_role.lakeformation`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_role.lakeformation`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_role) (data source) --- ## lambda This component is responsible for provisioning Lambda functions. ## Usage **Stack Level**: Regional Stack configuration for defaults: ```yaml components: terraform: lambda-defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true ``` Sample App Yaml Entry: ```yaml import: - catalog/lambda/defaults components: terraform: lambda/hello-world-py: metadata: component: lambda inherits: - lambda/defaults vars: name: hello-world-py function_name: main description: Hello Lambda from Python! handler: lambda.lambda_handler # in go this is the compiled binary, python it's filename.function memory_size: 256 # https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html runtime: python3.9 package_type: Zip # `Zip` or `Image` policy_json: | { "Version": "2012-10-17", "Statement": [ { "Sid": "ListAllBuckets", "Effect": "Allow", "Action": "s3:ListAllMyBuckets", "Resource": "*" } ] } iam_policy: statements: - sid: AllowSQSWorkerWriteAccess effect: Allow actions: - sqs:SendMessage - sqs:SendMessageBatch resources: - arn:aws:sqs:*:111111111111:worker-queue # Filename example filename: lambdas/hello-world-python/output.zip # generated by zip variable. zip: enabled: true input_dir: hello-world-python output: hello-world-python/output.zip # S3 Source Example # s3_bucket_name: lambda-source # lambda main.tf calculates the rest of the bucket_name # s3_key: hello-world-go.zip ``` ### Notifications: #### SQS ```yaml sqs_notifications: my-service-a: sqs_component: component: sqs-queue/my-service-a my-service-b: sqs_arn: arn:aws:sqs:us-west-2:111111111111:my-service-b ``` #### S3 ```yaml s3_notifications: my-service-a: bucket_component: component: s3-bucket/my-service-a events: ["s3:ObjectCreated:*"] my-service-b: bucket_name: my-service-b events: ["s3:ObjectCreated:*", "s3:ObjectRemoved:*"] ``` #### Cron (CloudWatch Event) ```yaml cloudwatch_event_rules: schedule-a: schedule_expression: "rate(5 minutes)" schedule-b: schedule_expression: "cron(0 20 * * ? *)" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`architectures` (`list(string)`) optional
Instruction set architecture for your Lambda function. Valid values are ["x86_64"] and ["arm64"]. Default is ["x86_64"]. Removing this attribute, function's architecture stay the same. **Default value:** `null`
`cicd_s3_key_format` (`string`) optional
The format of the S3 key to store the latest version/sha of the Lambda function. This is used with cicd_ssm_param_name. Defaults to 'stage/\{stage\}/lambda/\{function_name\}/%s.zip' **Default value:** `null`
`cicd_ssm_param_name` (`string`) optional
The name of the SSM parameter to store the latest version/sha of the Lambda function. This is used with cicd_s3_key_format **Default value:** `null`
`cloudwatch_event_rules` optional
Creates EventBridge (CloudWatch Events) rules for invoking the Lambda Function along with the required permissions. **Type:** ```hcl map(object({ description = optional(string) event_bus_name = optional(string) event_pattern = optional(string) is_enabled = optional(bool) name_prefix = optional(string) role_arn = optional(string) schedule_expression = optional(string) })) ``` **Default value:** `{ }`
`cloudwatch_lambda_insights_enabled` (`bool`) optional
Enable CloudWatch Lambda Insights for the Lambda Function. **Default value:** `false`
`cloudwatch_logs_kms_key_arn` (`string`) optional
The ARN of the KMS Key to use when encrypting log data. **Default value:** `null`
`cloudwatch_logs_retention_in_days` (`number`) optional
Specifies the number of days you want to retain log events in the specified log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. **Default value:** `null`
`custom_iam_policy_arns` (`set(string)`) optional
ARNs of IAM policies to be attached to the Lambda role **Default value:** `[ ]`
`dead_letter_config_target_arn` (`string`) optional
ARN of an SNS topic or SQS queue to notify when an invocation fails. If this option is used, the function's IAM role must be granted suitable access to write to the target object, which means allowing either the sns:Publish or sqs:SendMessage action on this ARN, depending on which service is targeted." **Default value:** `null`
`description` (`string`) optional
Description of what the Lambda Function does. **Default value:** `null`
`filename` (`string`) optional
The path to the function's deployment package within the local filesystem. Works well with the `zip` variable. If defined, The s3_-prefixed options and image_uri cannot be used. **Default value:** `null`
`function_name` (`string`) optional
Unique name for the Lambda Function. **Default value:** `null`
`function_url_enabled` (`bool`) optional
Create a aws_lambda_function_url resource to expose the Lambda function **Default value:** `false`
`handler` (`string`) optional
The function entrypoint in your code. **Default value:** `null`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`iam_policy_description` (`string`) optional
Description of the IAM policy for the Lambda IAM role **Default value:** `"Minimum SSM read permissions for Lambda IAM Role"`
`image_config` (`any`) optional
The Lambda OCI [image configurations](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function#image_config) block with three (optional) arguments: - *entry_point* - The ENTRYPOINT for the docker image (type `list(string)`). - *command* - The CMD for the docker image (type `list(string)`). - *working_directory* - The working directory for the docker image (type `string`). **Default value:** `{ }`
`image_uri` (`string`) optional
The ECR image URI containing the function's deployment package. Conflicts with `filename`, `s3_bucket_name`, `s3_key`, and `s3_object_version`. **Default value:** `null`
`kms_key_arn` (`string`) optional
Amazon Resource Name (ARN) of the AWS Key Management Service (KMS) key that is used to encrypt environment variables. If this configuration is not provided when environment variables are in use, AWS Lambda uses a default service key. If this configuration is provided when environment variables are not in use, the AWS Lambda API does not save this configuration and Terraform will show a perpetual difference of adding the key. To fix the perpetual difference, remove this configuration. **Default value:** `""`
`lambda_at_edge_enabled` (`bool`) optional
Enable Lambda@Edge for your Node.js or Python functions. The required trust relationship and publishing of function versions will be configured in this module. **Default value:** `false`
`lambda_environment` optional
Environment (e.g. ENV variables) configuration for the Lambda function enable you to dynamically pass settings to your function code and libraries. **Type:** ```hcl object({ variables = map(string) }) ``` **Default value:** `null`
`layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to the Lambda Function. **Default value:** `[ ]`
`memory_size` (`number`) optional
Amount of memory in MB the Lambda Function can use at runtime. **Default value:** `128`
`package_type` (`string`) optional
The Lambda deployment package type. Valid values are `Zip` and `Image`. **Default value:** `"Zip"`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role **Default value:** `""`
`policy_json` (`string`) optional
IAM policy to attach to the Lambda role, specified as JSON. This can be used with or instead of `var.iam_policy`. **Default value:** `null`
`publish` (`bool`) optional
Whether to publish creation/change as new Lambda Function Version. **Default value:** `false`
`reserved_concurrent_executions` (`number`) optional
The amount of reserved concurrent executions for this lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations. **Default value:** `-1`
`runtime` (`string`) optional
The runtime environment for the Lambda function you are uploading. **Default value:** `null`
`s3_bucket_component` optional
The bucket component to use for the S3 bucket containing the function's deployment package. Conflicts with `s3_bucket_name`, `filename` and `image_uri`. **Type:** ```hcl object({ component = string tenant = optional(string) stage = optional(string) environment = optional(string) }) ``` **Default value:** `null`
`s3_bucket_name` (`string`) optional
The name suffix of the S3 bucket containing the function's deployment package. Conflicts with filename and image_uri. This bucket must reside in the same AWS region where you are creating the Lambda function. **Default value:** `null`
`s3_key` (`string`) optional
The S3 key of an object containing the function's deployment package. Conflicts with filename and image_uri. **Default value:** `null`
`s3_notifications` optional
A map of S3 bucket notifications to trigger the Lambda function **Type:** ```hcl map(object({ bucket_name = optional(string) bucket_component = optional(object({ component = string environment = optional(string) tenant = optional(string) stage = optional(string) })) events = optional(list(string)) filter_prefix = optional(string) filter_suffix = optional(string) source_account = optional(string) })) ``` **Default value:** `{ }`
`s3_object_version` (`string`) optional
The object version containing the function's deployment package. Conflicts with filename and image_uri. **Default value:** `null`
`source_code_hash` (`string`) optional
Used to trigger updates. Must be set to a base64-encoded SHA256 hash of the package file specified with either filename or s3_key. The usual way to set this is filebase64sha256('file.zip') where 'file.zip' is the local filename of the lambda function source archive. **Default value:** `""`
`sqs_notifications` optional
A map of SQS queue notifications to trigger the Lambda function **Type:** ```hcl map(object({ sqs_arn = optional(string) sqs_component = optional(object({ component = string environment = optional(string) tenant = optional(string) stage = optional(string) })) batch_size = optional(number) source_account = optional(string) on_failure_arn = optional(string) maximum_concurrency = optional(number) })) ``` **Default value:** `{ }`
`ssm_parameter_names` (`list(string)`) optional
List of AWS Systems Manager Parameter Store parameter names. The IAM role of this Lambda function will be enhanced with read permissions for those parameters. Parameters must start with a forward slash and can be encrypted with the default KMS key. **Default value:** `null`
`timeout` (`number`) optional
The amount of time the Lambda Function has to run in seconds. **Default value:** `3`
`tracing_config_mode` (`string`) optional
Tracing config mode of the Lambda function. Can be either PassThrough or Active. **Default value:** `null`
`vpc_config` optional
Provide this to allow your function to access your VPC (if both 'subnet_ids' and 'security_group_ids' are empty then vpc_config is considered to be empty or unset, see https://docs.aws.amazon.com/lambda/latest/dg/vpc.html for details). **Type:** ```hcl object({ security_group_ids = list(string) subnet_ids = list(string) }) ``` **Default value:** `null`
`zip` optional
Zip Configuration for local file deployments **Type:** ```hcl object({ enabled = optional(bool, false) output = optional(string, "output.zip") input_dir = optional(string, null) }) ``` **Default value:** ```hcl { "enabled": false, "output": "output.zip" } ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the lambda function
`function_name`
Lambda function name
`invoke_arn`
Invoke ARN of the lambda function
`qualified_arn`
ARN identifying your Lambda Function Version (if versioning is enabled via publish = true)
`role_arn`
Lambda IAM role ARN
`role_name`
Lambda IAM role name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `archive`, version: `>= 2.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0.0` ### Providers - `archive`, version: `>= 2.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `random`, version: `>= 3.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_event_rules_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `lambda` | 0.6.1 | [`cloudposse/lambda-function/aws`](https://registry.terraform.io/modules/cloudposse/lambda-function/aws/0.6.1) | n/a `s3_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `s3_bucket_notifications_component` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `sqs_iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `sqs_queue` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.event_rules`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.sns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.sqs_default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_event_source_mapping.event_source_mapping`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_event_source_mapping) (resource) - [`aws_lambda_function_url.lambda_url`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function_url) (resource) - [`aws_lambda_permission.allow_cloudwatch_to_call_lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.s3_notification`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_s3_bucket_notification.s3_notifications`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_notification) (resource) - [`random_pet.zip_recreator`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`archive_file.lambdazip`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`aws_ssm_parameter.cicd_ssm_param`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## macie This component is responsible for configuring Macie within an AWS Organization. Amazon Macie is a data security service that discovers sensitive data by using machine learning and pattern matching, provides visibility into data security risks, and enables automated protection against those risks. To help you manage the security posture of your organization's Amazon Simple Storage Service (Amazon S3) data estate, Macie provides you with an inventory of your S3 buckets, and automatically evaluates and monitors the buckets for security and access control. If Macie detects a potential issue with the security or privacy of your data, such as a bucket that becomes publicly accessible, Macie generates a finding for you to review and remediate as necessary. Macie also automates discovery and reporting of sensitive data to provide you with a better understanding of the data that your organization stores in Amazon S3. To detect sensitive data, you can use built-in criteria and techniques that Macie provides, custom criteria that you define, or a combination of the two. If Macie detects sensitive data in an S3 object, Macie generates a finding to notify you of the sensitive data that Macie found. In addition to findings, Macie provides statistics and other data that offer insight into the security posture of your Amazon S3 data, and where sensitive data might reside in your data estate. The statistics and data can guide your decisions to perform deeper investigations of specific S3 buckets and objects. You can review and analyze findings, statistics, and other data by using the Amazon Macie console or the Amazon Macie API. You can also leverage Macie integration with Amazon EventBridge and AWS Security Hub to monitor, process, and remediate findings by using other services, applications, and systems. ## Usage **Stack Level**: Regional ## Deployment Overview This component is complex in that it must be deployed multiple times with different variables set to configure the AWS Organization successfully. In the examples below, we assume that the AWS Organization Management account is `root` and the AWS Organization Delegated Administrator account is `security`, both in the `core` tenant. ### Deploy to Delegated Administrator Account First, the component is deployed to the [Delegated Administrator](https://docs.aws.amazon.com/macie/latest/user/accounts-mgmt-ao-integrate.html) account to configure the central Macie account∑. ```yaml # core-ue1-security components: terraform: macie/delegated-administrator: metadata: component: macie vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 ``` ```bash atmos terraform apply macie/delegated-administrator -s core-ue1-security ``` ### Deploy to Organization Management (root) Account Next, the component is deployed to the AWS Organization Management, a/k/a `root`, Account in order to set the AWS Organization Designated Administrator account. Note that you must `SuperAdmin` permissions as we are deploying to the AWS Organization Management account. Since we are using the `SuperAdmin` user, it will already have access to the state bucket, so we set the `role_arn` of the backend config to null and set `var.privileged` to `true`. ```yaml # core-ue1-root components: terraform: guardduty/root: metadata: component: macie backend: s3: role_arn: null vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 privileged: true ``` ```bash atmos terraform apply macie/root -s core-ue1-root ``` ### Deploy Organization Settings in Delegated Administrator Account Finally, the component is deployed to the Delegated Administrator Account again in order to create the organization-wide configuration for the AWS Organization, but with `var.admin_delegated` set to `true` to indicate that the delegation has already been performed from the Organization Management account. ```yaml # core-ue1-security components: terraform: macie/org-settings: metadata: component: macie vars: enabled: true delegated_administrator_account_name: core-security environment: use1 region: us-east-1 admin_delegated: true ``` ```bash atmos terraform apply macie/org-settings/ue1 -s core-ue1-security ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_tenant` (`string`) optional
The tenant where the `account_map` component required by remote-state is deployed **Default value:** `"core"`
`admin_delegated` (`bool`) optional
A flag to indicate if the AWS Organization-wide settings should be created. This can only be done after the GuardDuty Administrator account has already been delegated from the AWS Org Management account (usually 'root'). See the Deployment section of the README for more information. **Default value:** `false`
`delegated_admininstrator_component_name` (`string`) optional
The name of the component that created the Macie account. **Default value:** `"macie/delegated-administrator"`
`delegated_administrator_account_name` (`string`) optional
The name of the account that is the AWS Organization Delegated Administrator account **Default value:** `"core-security"`
`finding_publishing_frequency` (`string`) optional
Specifies how often to publish updates to policy findings for the account. This includes publishing updates to AWS Security Hub and Amazon EventBridge (formerly called Amazon CloudWatch Events). For more information, see: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html#guardduty_findings_cloudwatch_notification_frequency **Default value:** `"FIFTEEN_MINUTES"`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`member_accounts` (`list(string)`) optional
List of member account names to enable Macie on **Default value:** `[ ]`
`organization_management_account_name` (`string`) optional
The name of the AWS Organization management account **Default value:** `null`
`privileged` (`bool`) optional
true if the default provider already has access to the backend **Default value:** `false`
`root_account_stage` (`string`) optional
The stage name for the Organization root (management) account. This is used to lookup account IDs from account names using the `account-map` component. **Default value:** `"root"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`delegated_administrator_account_id`
The AWS Account ID of the AWS Organization delegated administrator account
`macie_account_id`
The ID of the Macie account created by the component
`macie_service_role_arn`
The Amazon Resource Name (ARN) of the service-linked role that allows Macie to monitor and analyze data in AWS resources for the account.
`member_account_ids`
The AWS Account IDs of the member accounts
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.17.0, < 6.0.0` ### Providers - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.17.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_macie2_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_account) (resource) - [`aws_macie2_organization_admin_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_organization_admin_account) (resource) - [`awsutils_macie2_organization_settings.this`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/resources/macie2_organization_settings) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) --- ## api-key This component provisions an API Key for an Amazon Managed Grafana workspace. It is intended for use with the [Grafana Terraform provider](https://registry.terraform.io/providers/grafana/grafana/latest) in other `managed-grafana` sub-components. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: grafana/api-key: metadata: component: managed-grafana/api-key vars: enabled: true grafana_component_name: grafana ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`grafana_component_name` (`string`) optional
The name of the Grafana component **Default value:** `"managed-grafana/workspace"`
`key_role` (`string`) optional
Specifies the permission level of the API key. Valid values are VIEWER, EDITOR, or ADMIN. **Default value:** `"ADMIN"`
`minutes_to_live` (`number`) optional
Specifies the time in minutes until the API key expires. Keys can be valid for up to 30 days. **Default value:** `43200`
`ssm_path_format_api_key` (`string`) optional
The path in AWS SSM to the Grafana API Key provisioned with this component **Default value:** `"/grafana/%s/api_key"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ssm_path_grafana_api_key`
The path in AWS SSM to the Grafana API Key provisioned with this component
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `time`, version: `>= 0.11.1` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `time`, version: `>= 0.11.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `managed_grafana` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `ssm_parameters` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_grafana_workspace_api_key.key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_workspace_api_key) (resource) - [`time_rotating.ttl`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/rotating) (resource) - [`time_static.ttl`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/static) (resource) ## Data Sources The following data sources are used by this module: --- ## dashboard This component is responsible for provisioning a dashboard in an Amazon Managed Grafana workspace. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: grafana/dashboard/prometheus: metadata: component: managed-grafana/dashboard vars: enabled: true name: "prometheus-dashboard" grafana_component_name: grafana grafana_api_key_component_name: grafana/api-key dashboard_url: "https://grafana.com/api/dashboards/315/revisions/3/download" config_input: "${DS_PROMETHEUS}": "acme-plat-ue2-sandbox-prometheus" # Input Value : Data source UID ``` ## Variables ### Required Variables
`dashboard_name` (`string`) required
The name to use for the dashboard. This must be unique.
`dashboard_url` (`string`) required
The marketplace URL of the dashboard to be created
`region` (`string`) required
AWS Region
### Optional Variables
`additional_config` (`map(any)`) optional
Additional dashboard configuration to be merged with the provided dashboard JSON **Default value:** `{ }`
`config_input` (`map(string)`) optional
A map of string replacements used to supply input for the dashboard config JSON **Default value:** `{ }`
`grafana_api_key_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana API key **Default value:** `"managed-grafana/api-key"`
`grafana_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana workspace **Default value:** `"managed-grafana/workspace"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` - `http`, version: `>= 3.4.2` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` - `http`, version: `>= 3.4.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `config_json` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `grafana` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `grafana_api_key` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`grafana_dashboard.this`](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/dashboard) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.grafana_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`http_http.grafana_dashboard_json`](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) (data source) --- ## workspace This component provisions an Amazon Managed Grafana workspace. Amazon Managed Grafana is a fully managed service for Grafana, a popular open-source analytics platform that enables you to query, visualize, and alert on your metrics, logs, and traces. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: grafana: metadata: component: managed-grafana/workspace vars: enabled: true name: grafana private_network_access_enabled: true sso_role_associations: - role: "ADMIN" group_ids: - "11111111-2222-3333-4444-555555555555" # This grafana workspace will be allowed to assume the cross # account access role from these prometheus components prometheus_source_accounts: - component: prometheus tenant: plat stage: sandbox - component: prometheus tenant: plat stage: dev ``` :::note We would prefer to have a custom URL for the provisioned Grafana workspace, but at the moment it's not supported natively and implementation would be non-trivial. We will continue to monitor that Issue and consider alternatives, such as using CloudFront. [Issue #6: Support for Custom Domains](https://github.com/aws/amazon-managed-grafana-roadmap/issues/6) ::: ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`private_network_access_enabled` (`bool`) optional
If set to `true`, enable the VPC Configuration to allow this workspace to access the private network using outputs from the vpc component **Default value:** `false`
`prometheus_policy_enabled` (`bool`) optional
Set this to `true` to allow this Grafana workspace to access Amazon Managed Prometheus in this account **Default value:** `false`
`prometheus_source_accounts` optional
A list of objects that describe an account where Amazon Managed Prometheus is deployed. This component grants this Grafana IAM role permission to assume the Prometheus access role in that target account. Use this for cross-account access **Type:** ```hcl list(object({ component = optional(string, "managed-prometheus/workspace") stage = string tenant = optional(string, "") environment = optional(string, "") })) ``` **Default value:** `[ ]`
`sso_role_associations` optional
A list of role to group ID list associations for granting Amazon Grafana access **Type:** ```hcl list(object({ role = string group_ids = list(string) })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`workspace_endpoint`
The returned URL of the Amazon Managed Grafana workspace
`workspace_id`
The ID of the Amazon Managed Grafana workspace
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `managed_grafana` | 0.5.0 | [`cloudposse/managed-grafana/aws`](https://registry.terraform.io/modules/cloudposse/managed-grafana/aws/0.5.0) | n/a `prometheus` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_grafana_role_association.sso`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_role_association) (resource) ## Data Sources The following data sources are used by this module: --- ## loki(Loki) This component is responsible for provisioning a Loki data source for an Amazon Managed Grafana workspace. Use this component alongside the `eks/loki` component. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: grafana/datasource/defaults: metadata: component: managed-grafana/data-source/managed-prometheus type: abstract vars: enabled: true grafana_component_name: grafana grafana_api_key_component_name: grafana/api-key grafana/datasource/plat-sandbox-loki: metadata: component: managed-grafana/data-source/loki inherits: - grafana/datasource/defaults vars: name: plat-sandbox-loki loki_tenant_name: plat loki_stage_name: sandbox grafana/datasource/plat-dev-loki: metadata: component: managed-grafana/data-source/loki inherits: - grafana/datasource/defaults vars: name: plat-dev-loki loki_tenant_name: plat loki_stage_name: dev grafana/datasource/plat-prod-loki: metadata: component: managed-grafana/data-source/loki inherits: - grafana/datasource/defaults vars: name: plat-prod-loki loki_tenant_name: plat loki_stage_name: prod ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`grafana_api_key_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana API key **Default value:** `"managed-grafana/api-key"`
`grafana_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana workspace **Default value:** `"managed-grafana/workspace"`
`loki_component_name` (`string`) optional
The name of the loki component **Default value:** `"eks/loki"`
`loki_environment_name` (`string`) optional
The environment where the loki component is deployed **Default value:** `""`
`loki_stage_name` (`string`) optional
The stage where the loki component is deployed **Default value:** `""`
`loki_tenant_name` (`string`) optional
The tenant where the loki component is deployed **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`uid`
The UID of this dashboard
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `grafana` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `grafana_api_key` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `loki` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `source_account_role` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`grafana_data_source.loki`](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.basic_auth_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.grafana_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## managed-prometheus This component provisions an Amazon Managed Prometheus data source for an Amazon Managed Grafana workspace. Use this component alongside the `managed-prometheus/workspace` component. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: grafana/datasource/defaults: metadata: component: managed-grafana/data-source/managed-prometheus type: abstract vars: enabled: true grafana_component_name: grafana grafana_api_key_component_name: grafana/api-key prometheus_component_name: prometheus grafana/datasource/plat-sandbox-prometheus: metadata: component: managed-grafana/data-source/managed-prometheus inherits: - grafana/datasource/defaults vars: name: plat-sandbox-prometheus prometheus_tenant_name: plat prometheus_stage_name: sandbox grafana/datasource/plat-dev-prometheus: metadata: component: managed-grafana/data-source/managed-prometheus inherits: - grafana/datasource/defaults vars: name: plat-dev-prometheus prometheus_tenant_name: plat prometheus_stage_name: dev grafana/datasource/plat-prod-prometheus: metadata: component: managed-grafana/data-source/managed-prometheus inherits: - grafana/datasource/defaults vars: name: plat-prod-prometheus prometheus_tenant_name: plat prometheus_stage_name: prod ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`grafana_api_key_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana API key **Default value:** `"managed-grafana/api-key"`
`grafana_component_name` (`string`) optional
The name of the component used to provision an Amazon Managed Grafana workspace **Default value:** `"managed-grafana/workspace"`
`prometheus_component_name` (`string`) optional
The name of the Amazon Managed Prometheus component to be added as a Grafana data source **Default value:** `"managed-prometheus/workspace"`
`prometheus_environment_name` (`string`) optional
The environment where the Amazon Managed Prometheus component is deployed **Default value:** `""`
`prometheus_stage_name` (`string`) optional
The stage where the Amazon Managed Prometheus component is deployed **Default value:** `""`
`prometheus_tenant_name` (`string`) optional
The tenant where the Amazon Managed Prometheus component is deployed **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`uid`
The UID of this dashboard
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `grafana`, version: `>= 2.18.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `grafana` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `grafana_api_key` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | v1.536.0 | [`github.com/cloudposse-terraform-components/aws-account-map//src/modules/iam-roles`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-account-map/src/modules/iam-roles/v1.536.0) | n/a `prometheus` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`grafana_data_source.managed_prometheus`](https://registry.terraform.io/providers/grafana/grafana/latest/docs/resources/data_source) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.grafana_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## workspace(Workspace) This component is responsible for provisioning a workspace for Amazon Managed Service for Prometheus, also known as Amazon Managed Prometheus (AMP). This component is intended to be deployed alongside Grafana. For example, use our `managed-grafana/workspace` component. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. We prefer to name the stack component with a simpler name, whereas the Terraform component should remain descriptive. ```yaml components: terraform: prometheus: metadata: component: managed-prometheus/workspace vars: enabled: true name: prometheus # Create cross-account role for core-auto to access AMP grafana_account_name: core-auto ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned **Default value:** `"core"`
`alert_manager_definition` (`string`) optional
The alert manager definition that you want to be applied. **Default value:** `""`
`grafana_account_name` (`string`) optional
The name of the account allowed to access AMP in this account. If defined, this module will create a cross-account IAM role for accessing AMP. Use this for cross-account Grafana. If not defined, no roles will be created. **Default value:** `""`
`rule_group_namespaces` optional
A list of name, data objects for each Amazon Managed Service for Prometheus (AMP) Rule Group Namespace **Type:** ```hcl list(object({ name = string data = string })) ``` **Default value:** `[ ]`
`vpc_endpoint_enabled` (`string`) optional
If set to `true`, restrict traffic through a VPC endpoint **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_role_arn`
If enabled with `var.allowed_account_id`, the Role ARN used for accessing Amazon Managed Prometheus in this account
`id`
The ID of this component deployment
`workspace_arn`
The ARN of this Amazon Managed Prometheus workspace
`workspace_endpoint`
The endpoint URL of this Amazon Managed Prometheus workspace
`workspace_id`
The ID of this Amazon Managed Prometheus workspace
`workspace_region`
The region where this workspace is deployed
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `managed_prometheus` | 0.1.2 | [`cloudposse/managed-prometheus/aws`](https://registry.terraform.io/modules/cloudposse/managed-prometheus/aws/0.1.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## memorydb This component provisions an AWS MemoryDB cluster. MemoryDB is a fully managed, Redis-compatible, in-memory database service. While Redis is commonly used as a cache, MemoryDB is designed to also function well as a [vector database](https://docs.aws.amazon.com/memorydb/latest/devguide/vector-search.html). This makes it appropriate for AI model backends. ## Usage **Stack Level**: Regional ### Example Here's an example snippet for how to use this component: ```yaml components: terraform: vpc: vars: availability_zones: - "a" - "b" - "c" ipv4_primary_cidr_block: "10.111.0.0/18" memorydb: vars: {} ``` ## Variables ### Required Variables
### Optional Variables
`admin_username` (`string`) optional
The username for the MemoryDB admin **Default value:** `"admin"`
`auto_minor_version_upgrade` (`bool`) optional
Indicates that minor engine upgrades will be applied automatically to the cluster during the maintenance window **Default value:** `true`
`engine_version` (`string`) optional
The version of the Redis engine to use **Default value:** `"6.2"`
`maintenance_window` (`string`) optional
The weekly time range during which system maintenance can occur **Default value:** `null`
`node_type` (`string`) optional
The compute and memory capacity of the nodes in the cluster **Default value:** `"db.r6g.large"`
`num_replicas_per_shard` (`number`) optional
The number of replicas per shard **Default value:** `1`
`num_shards` (`number`) optional
The number of shards in the cluster **Default value:** `1`
`parameter_group_family` (`string`) optional
The name of the parameter group family **Default value:** `"memorydb_redis6"`
`parameters` (`map(string)`) optional
Key-value mapping of parameters to apply to the parameter group **Default value:** `{ }`
`port` (`number`) optional
The port on which the cluster accepts connections **Default value:** `6379`
`security_group_ids` (`list(string)`) optional
List of security group IDs to associate with the MemoryDB cluster **Default value:** `[ ]`
`snapshot_arns` (`list(string)`) optional
List of ARNs for the snapshots to be restored. NOTE: destroys the existing cluster. Use for restoring. **Default value:** `[ ]`
`snapshot_retention_limit` (`number`) optional
The number of days for which MemoryDB retains automatic snapshots before deleting them **Default value:** `null`
`snapshot_window` (`string`) optional
The daily time range during which MemoryDB begins taking daily snapshots **Default value:** `null`
`sns_topic_arn` (`string`) optional
The ARN of the SNS topic to send notifications to **Default value:** `null`
`ssm_kms_key_id` (`string`) optional
The KMS key ID to use for SSM parameter encryption. If not specified, the default key will be used. **Default value:** `null`
`ssm_parameter_name` (`string`) optional
The name of the SSM parameter to store the password in. If not specified, the password will be stored in `/{context.id}/admin_password` **Default value:** `""`
`tls_enabled` (`bool`) optional
Indicates whether Transport Layer Security (TLS) encryption is enabled for the cluster **Default value:** `true`
`vpc_component_name` (`string`) optional
The name of the VPC component. This is used to pick out subnets for the MemoryDB cluster **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_acl_arn`
The ARN of the MemoryDB user's ACL
`admin_arn`
The ARN of the MemoryDB user
`admin_password_ssm_parameter_name`
The name of the SSM parameter storing the password for the MemoryDB user
`admin_username`
The username for the MemoryDB user
`arn`
The ARN of the MemoryDB cluster
`cluster_endpoint`
The endpoint of the MemoryDB cluster
`engine_patch_version`
The Redis engine version
`id`
The name of the MemoryDB cluster
`parameter_group_arn`
The ARN of the MemoryDB parameter group
`parameter_group_id`
The name of the MemoryDB parameter group
`shards`
The shard details for the MemoryDB cluster
`subnet_group_arn`
The ARN of the MemoryDB subnet group
`subnet_group_id`
The name of the MemoryDB subnet group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `memorydb` | 0.2.0 | [`cloudposse/memorydb/aws`](https://registry.terraform.io/modules/cloudposse/memorydb/aws/0.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## mq-broker This component is responsible for provisioning an AmazonMQ broker and the corresponding security group. ### Migrate `v1` to `v2` `EKS` component dependency removed. Instead of pulling security groups from EKS remote state, pass it as `var.allowed_security_groups`. If you are using Atmos, read [atmos shared data](https://atmos.tools/core-concepts/share-data/) manual. ```yaml components: terraform: mq-broker: vars: enabled: true apply_immediately: true auto_minor_version_upgrade: true deployment_mode: "ACTIVE_STANDBY_MULTI_AZ" engine_type: "ActiveMQ" engine_version: "5.15.14" host_instance_type: "mq.t3.micro" publicly_accessible: false general_log_enabled: true audit_log_enabled: true encryption_enabled: true use_aws_owned_key: true allowed_security_groups: - '{{ (atmos.Component "eks" .stack).outputs.eks_cluster_managed_security_group_id }}' ``` ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: mq-broker: vars: enabled: true apply_immediately: true auto_minor_version_upgrade: true deployment_mode: "ACTIVE_STANDBY_MULTI_AZ" engine_type: "ActiveMQ" engine_version: "5.15.14" host_instance_type: "mq.t3.micro" publicly_accessible: false general_log_enabled: true audit_log_enabled: true encryption_enabled: true use_aws_owned_key: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks that are allowed ingress to the broker's Security Group created in the module **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
List of security groups to be allowed to connect to the broker instance **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `false`
`audit_log_enabled` (`bool`) optional
Enables audit logging. User management action made using JMX or the ActiveMQ Web Console is logged **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Enables automatic upgrades to new minor versions for brokers, as Apache releases the versions **Default value:** `false`
`deployment_mode` (`string`) optional
The deployment mode of the broker. Supported: SINGLE_INSTANCE and ACTIVE_STANDBY_MULTI_AZ **Default value:** `"ACTIVE_STANDBY_MULTI_AZ"`
`encryption_enabled` (`bool`) optional
Flag to enable/disable Amazon MQ encryption at rest **Default value:** `true`
`engine_type` (`string`) optional
Type of broker engine, `ActiveMQ` or `RabbitMQ` **Default value:** `"ActiveMQ"`
`engine_version` (`string`) optional
The version of the broker engine. See https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/broker-engine.html for more details **Default value:** `"5.15.14"`
`existing_security_groups` (`list(string)`) optional
List of existing Security Group IDs to place the broker into. Set `use_existing_security_groups` to `true` to enable using `existing_security_groups` as Security Groups for the broker **Default value:** `[ ]`
`general_log_enabled` (`bool`) optional
Enables general logging via CloudWatch **Default value:** `true`
`host_instance_type` (`string`) optional
The broker's instance type. e.g. mq.t2.micro or mq.m4.large **Default value:** `"mq.t3.micro"`
`kms_mq_key_arn` (`string`) optional
ARN of the AWS KMS key used for Amazon MQ encryption **Default value:** `null`
`kms_ssm_key_arn` (`string`) optional
ARN of the AWS KMS key used for SSM encryption **Default value:** `"alias/aws/ssm"`
`maintenance_day_of_week` (`string`) optional
The maintenance day of the week. e.g. MONDAY, TUESDAY, or WEDNESDAY **Default value:** `"SUNDAY"`
`maintenance_time_of_day` (`string`) optional
The maintenance time, in 24-hour format. e.g. 02:00 **Default value:** `"03:00"`
`maintenance_time_zone` (`string`) optional
The maintenance time zone, in either the Country/City format, or the UTC offset format. e.g. CET **Default value:** `"UTC"`
`mq_admin_password` (`string`) optional
Admin password **Default value:** `null`
`mq_admin_password_ssm_parameter_name` (`string`) optional
SSM parameter name for Admin password **Default value:** `"mq_admin_password"`
`mq_admin_user` (`string`) optional
Admin username **Default value:** `null`
`mq_admin_user_ssm_parameter_name` (`string`) optional
SSM parameter name for Admin username **Default value:** `"mq_admin_username"`
`mq_application_password` (`string`) optional
Application password **Default value:** `null`
`mq_application_password_ssm_parameter_name` (`string`) optional
SSM parameter name for Application password **Default value:** `"mq_application_password"`
`mq_application_user` (`string`) optional
Application username **Default value:** `null`
`mq_application_user_ssm_parameter_name` (`string`) optional
SSM parameter name for Application username **Default value:** `"mq_application_username"`
`overwrite_ssm_parameter` (`bool`) optional
Whether to overwrite an existing SSM parameter **Default value:** `true`
`publicly_accessible` (`bool`) optional
Whether to enable connections from applications outside of the VPC that hosts the broker's subnets **Default value:** `false`
`ssm_parameter_name_format` (`string`) optional
SSM parameter name format **Default value:** `"/%s/%s"`
`ssm_path` (`string`) optional
SSM path **Default value:** `"mq"`
`use_aws_owned_key` (`bool`) optional
Boolean to enable an AWS owned Key Management Service (KMS) Customer Master Key (CMK) for Amazon MQ encryption that is not in your account **Default value:** `true`
`use_existing_security_groups` (`bool`) optional
Flag to enable/disable creation of Security Group in the module. Set to `true` to disable Security Group creation and provide a list of existing security Group IDs in `existing_security_groups` to place the broker into **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_username`
AmazonMQ admin username
`application_username`
AmazonMQ application username
`broker_arn`
AmazonMQ broker ARN
`broker_id`
AmazonMQ broker ID
`primary_amqp_ssl_endpoint`
AmazonMQ primary AMQP+SSL endpoint
`primary_console_url`
AmazonMQ active web console URL
`primary_ip_address`
AmazonMQ primary IP address
`primary_mqtt_ssl_endpoint`
AmazonMQ primary MQTT+SSL endpoint
`primary_ssl_endpoint`
AmazonMQ primary SSL endpoint
`primary_stomp_ssl_endpoint`
AmazonMQ primary STOMP+SSL endpoint
`primary_wss_endpoint`
AmazonMQ primary WSS endpoint
`secondary_amqp_ssl_endpoint`
AmazonMQ secondary AMQP+SSL endpoint
`secondary_console_url`
AmazonMQ secondary web console URL
`secondary_ip_address`
AmazonMQ secondary IP address
`secondary_mqtt_ssl_endpoint`
AmazonMQ secondary MQTT+SSL endpoint
`secondary_ssl_endpoint`
AmazonMQ secondary SSL endpoint
`secondary_stomp_ssl_endpoint`
AmazonMQ secondary STOMP+SSL endpoint
`secondary_wss_endpoint`
AmazonMQ secondary WSS endpoint
`security_group_arn`
The ARN of the created security group
`security_group_id`
AmazonMQ security group id
`security_group_name`
The name of the created security group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `local`, version: `>= 2.4` - `template`, version: `>= 2.2` - `utils`, version: `>= 1.10.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `mq_broker` | 3.6.0 | [`cloudposse/mq-broker/aws`](https://registry.terraform.io/modules/cloudposse/mq-broker/aws/3.6.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## msk This component is responsible for provisioning [Amazon Managed Streaming](https://aws.amazon.com/msk/) clusters for [Apache Kafka](https://aws.amazon.com/msk/what-is-kafka/). ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: msk: metadata: component: "msk" vars: enabled: true name: "msk" vpc_component_name: "vpc" dns_delegated_component_name: "dns-delegated" dns_delegated_environment_name: "gbl" # https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html kafka_version: "3.4.0" public_access_enabled: false # https://aws.amazon.com/msk/pricing/ broker_instance_type: "kafka.m5.large" # Number of brokers per AZ broker_per_zone: 1 # `broker_dns_records_count` specifies how many DNS records to create for the broker endpoints in the DNS zone provided in the `zone_id` variable. # This corresponds to the total number of broker endpoints created by the module. # Calculate this number by multiplying the `broker_per_zone` variable by the subnet count. broker_dns_records_count: 3 broker_volume_size: 500 client_broker: "TLS_PLAINTEXT" encryption_in_cluster: true encryption_at_rest_kms_key_arn: "" enhanced_monitoring: "DEFAULT" certificate_authority_arns: [] # Authentication methods client_allow_unauthenticated: true client_sasl_scram_enabled: false client_sasl_scram_secret_association_enabled: false client_sasl_scram_secret_association_arns: [] client_sasl_iam_enabled: false client_tls_auth_enabled: false jmx_exporter_enabled: false node_exporter_enabled: false cloudwatch_logs_enabled: false firehose_logs_enabled: false firehose_delivery_stream: "" s3_logs_enabled: false s3_logs_bucket: "" s3_logs_prefix: "" properties: {} autoscaling_enabled: true storage_autoscaling_target_value: 60 storage_autoscaling_max_capacity: null storage_autoscaling_disable_scale_in: false create_security_group: true security_group_rule_description: "Allow inbound %s traffic" # A list of IDs of Security Groups to allow access to the cluster security group allowed_security_group_ids: [] # A list of IPv4 CIDRs to allow access to the cluster security group allowed_cidr_blocks: [] ``` ## Variables ### Required Variables
`broker_instance_type` (`string`) required
The instance type to use for the Kafka brokers
`kafka_version` (`string`) required
The desired Kafka software version. Refer to https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html for more details
`region` (`string`) required
AWS region
`vpc_component_name` (`string`) required
The name of the Atmos VPC component
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group(s) via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. For more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule and https://github.com/cloudposse/terraform-aws-security-group. **Default value:** `[ ]`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP addresses. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`autoscaling_enabled` (`bool`) optional
To automatically expand your cluster's storage in response to increased usage, you can enable this. [More info](https://docs.aws.amazon.com/msk/latest/developerguide/msk-autoexpand.html) **Default value:** `true`
`broker_dns_records_count` (`number`) optional
This variable specifies how many DNS records to create for the broker endpoints in the DNS zone provided in the `zone_id` variable. This corresponds to the total number of broker endpoints created by the module. Calculate this number by multiplying the `broker_per_zone` variable by the subnet count. This variable is necessary to prevent the Terraform error: The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. **Default value:** `0`
`broker_per_zone` (`number`) optional
Number of Kafka brokers per zone **Default value:** `1`
`broker_volume_size` (`number`) optional
The size in GiB of the EBS volume for the data drive on each broker node **Default value:** `1000`
`certificate_authority_arns` (`list(string)`) optional
List of ACM Certificate Authority Amazon Resource Names (ARNs) to be used for TLS client authentication **Default value:** `[ ]`
`client_allow_unauthenticated` (`bool`) optional
Enable unauthenticated access **Default value:** `false`
`client_broker` (`string`) optional
Encryption setting for data in transit between clients and brokers. Valid values: `TLS`, `TLS_PLAINTEXT`, and `PLAINTEXT` **Default value:** `"TLS"`
`client_sasl_iam_enabled` (`bool`) optional
Enable client authentication via IAM policies. Cannot be set to `true` at the same time as `client_tls_auth_enabled` **Default value:** `false`
`client_sasl_scram_enabled` (`bool`) optional
Enable SCRAM client authentication via AWS Secrets Manager. Cannot be set to `true` at the same time as `client_tls_auth_enabled` **Default value:** `false`
`client_sasl_scram_secret_association_arns` (`list(string)`) optional
List of AWS Secrets Manager secret ARNs for SCRAM authentication **Default value:** `[ ]`
`client_sasl_scram_secret_association_enabled` (`bool`) optional
Enable the list of AWS Secrets Manager secret ARNs for SCRAM authentication **Default value:** `true`
`client_tls_auth_enabled` (`bool`) optional
Set `true` to enable the Client TLS Authentication **Default value:** `false`
`cloudwatch_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to Cloudwatch Logs **Default value:** `false`
`cloudwatch_logs_log_group` (`string`) optional
Name of the Cloudwatch Log Group to deliver logs to **Default value:** `null`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`custom_broker_dns_name` (`string`) optional
Custom Route53 DNS hostname for MSK brokers. Use `%%ID%%` key to specify brokers index in the hostname. Example: `kafka-broker%%ID%%.example.com` **Default value:** `null`
`dns_delegated_component_name` (`string`) optional
The component name of `dns-delegated` **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
The environment name of `dns-delegated` **Default value:** `"gbl"`
`encryption_at_rest_kms_key_arn` (`string`) optional
You may specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest **Default value:** `""`
`encryption_in_cluster` (`bool`) optional
Whether data communication among broker nodes is encrypted **Default value:** `true`
`enhanced_monitoring` (`string`) optional
Specify the desired enhanced MSK CloudWatch monitoring level. Valid values: `DEFAULT`, `PER_BROKER`, and `PER_TOPIC_PER_BROKER` **Default value:** `"DEFAULT"`
`firehose_delivery_stream` (`string`) optional
Name of the Kinesis Data Firehose delivery stream to deliver logs to **Default value:** `""`
`firehose_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to Kinesis Data Firehose **Default value:** `false`
`inline_rules_enabled` (`bool`) optional
NOT RECOMMENDED. Create rules "inline" instead of as separate `aws_security_group_rule` resources. See [#20046](https://github.com/hashicorp/terraform-provider-aws/issues/20046) for one of several issues with inline rules. See [this post](https://github.com/hashicorp/terraform-provider-aws/pull/9032#issuecomment-639545250) for details on the difference between inline rules and rule resources. **Default value:** `false`
`jmx_exporter_enabled` (`bool`) optional
Set `true` to enable the JMX Exporter **Default value:** `false`
`node_exporter_enabled` (`bool`) optional
Set `true` to enable the Node Exporter **Default value:** `false`
`preserve_security_group_id` (`bool`) optional
When `false` and `security_group_create_before_destroy` is `true`, changes to security group rules cause a new security group to be created with the new rules, and the existing security group is then replaced with the new one, eliminating any service interruption. When `true` or when changing the value (from `false` to `true` or from `true` to `false`), existing security group rules will be deleted before new ones are created, resulting in a service interruption, but preserving the security group itself. **NOTE:** Setting this to `true` does not guarantee the security group will never be replaced, it only keeps changes to the security group rules from triggering a replacement. See the [terraform-aws-security-group README](https://github.com/cloudposse/terraform-aws-security-group) for further discussion. **Default value:** `false`
`properties` (`map(string)`) optional
Contents of the server.properties file. Supported properties are documented in the [MSK Developer Guide](https://docs.aws.amazon.com/msk/latest/developerguide/msk-configuration-properties.html) **Default value:** `{ }`
`public_access_enabled` (`bool`) optional
Enable public access to MSK cluster (given that all of the requirements are met) **Default value:** `false`
`s3_logs_bucket` (`string`) optional
Name of the S3 bucket to deliver logs to **Default value:** `""`
`s3_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to S3 **Default value:** `false`
`s3_logs_prefix` (`string`) optional
Prefix to append to the S3 folder name logs are delivered to **Default value:** `""`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Managed by Terraform"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`security_group_rule_description` (`string`) optional
The description to place on each security group rule. The %s will be replaced with the protocol name **Default value:** `"Allow inbound %s traffic"`
`storage_autoscaling_disable_scale_in` (`bool`) optional
If the value is true, scale in is disabled and the target tracking policy won't remove capacity from the scalable resource **Default value:** `false`
`storage_autoscaling_max_capacity` (`number`) optional
Maximum size the autoscaling policy can scale storage. Defaults to `broker_volume_size` **Default value:** `null`
`storage_autoscaling_target_value` (`number`) optional
Percentage of storage used to trigger autoscaled storage increase **Default value:** `60`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bootstrap_brokers`
Comma separated list of one or more hostname:port pairs of Kafka brokers suitable to bootstrap connectivity to the Kafka cluster
`bootstrap_brokers_public_sasl_iam`
Comma separated list of one or more DNS names (or IP addresses) and SASL IAM port pairs for public access to the Kafka cluster using SASL/IAM
`bootstrap_brokers_public_sasl_scram`
Comma separated list of one or more DNS names (or IP addresses) and SASL SCRAM port pairs for public access to the Kafka cluster using SASL/SCRAM
`bootstrap_brokers_public_tls`
Comma separated list of one or more DNS names (or IP addresses) and TLS port pairs for public access to the Kafka cluster using TLS
`bootstrap_brokers_sasl_iam`
Comma separated list of one or more DNS names (or IP addresses) and SASL IAM port pairs for access to the Kafka cluster using SASL/IAM
`bootstrap_brokers_sasl_scram`
Comma separated list of one or more DNS names (or IP addresses) and SASL SCRAM port pairs for access to the Kafka cluster using SASL/SCRAM
`bootstrap_brokers_tls`
Comma separated list of one or more DNS names (or IP addresses) and TLS port pairs for access to the Kafka cluster using TLS
`broker_endpoints`
List of broker endpoints
`cluster_arn`
Amazon Resource Name (ARN) of the MSK cluster
`cluster_name`
The cluster name of the MSK cluster
`config_arn`
Amazon Resource Name (ARN) of the MSK configuration
`current_version`
Current version of the MSK Cluster
`hostnames`
List of MSK Cluster broker DNS hostnames
`latest_revision`
Latest revision of the MSK configuration
`security_group_arn`
The ARN of the created security group
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
`storage_mode`
Storage mode for supported storage tiers
`zookeeper_connect_string`
Comma separated list of one or more hostname:port pairs to connect to the Apache Zookeeper cluster
`zookeeper_connect_string_tls`
Comma separated list of one or more hostname:port pairs to connect to the Apache Zookeeper cluster via TLS
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kafka` | 2.5.0 | [`cloudposse/msk-apache-kafka-cluster/aws`](https://registry.terraform.io/modules/cloudposse/msk-apache-kafka-cluster/aws/2.5.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## mwaa This component provisions Amazon managed workflows for Apache Airflow. The s3 bucket `dag_bucket` stores DAGs to be executed by MWAA. ## Access Modes ### Public Allows the Airflow UI to be access over the public internet to users granted access by an IAM policy. ### Private Limits access to users within the VPC to users granted access by an IAM policy. - MWAA creates a VPC interface endpoint for the Airflow webserver and an interface endpoint for the pgsql metadatabase. - the endpoints are created in the AZs mapped to your private subnets - MWAA binds an IP address from your private subnet to the interface endpoint ### Managing access to VPC endpoings on MWAA MWAA creates a VPC endpoint in each of the private subnets. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: mwaa: vars: enabled: true name: app dag_processing_logs_enabled: true dag_processing_logs_level: INFO environment_class: mw1.small airflow_version: 2.0.2 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`airflow_configuration_options` (`map(string)`) optional
The Airflow override options **Default value:** `{ }`
`airflow_version` (`string`) optional
Airflow version of the MWAA environment, will be set by default to the latest version that MWAA supports. **Default value:** `""`
`allow_ingress_from_vpc_stages` (`list(string)`) optional
List of stages to pull VPC ingress cidr and add to security group **Default value:** ```hcl [ "auto", "corp" ] ```
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the MWAA cluster **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. **Default value:** `[ ]`
`allowed_web_access_role_arns` (`list(string)`) optional
List of role ARNs to allow airflow web access **Default value:** `[ ]`
`allowed_web_access_role_names` (`list(string)`) optional
List of role names to allow airflow web access **Default value:** `[ ]`
`create_iam_role` (`bool`) optional
Enabling or disabling the creation of a default IAM Role for AWS MWAA **Default value:** `true`
`create_s3_bucket` (`bool`) optional
Enabling or disabling the creation of an S3 bucket for AWS MWAA **Default value:** `true`
`dag_processing_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for processing DAGs **Default value:** `false`
`dag_processing_logs_level` (`string`) optional
DAG processing logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`dag_s3_path` (`string`) optional
Path to dags in s3 **Default value:** `"dags"`
`environment_class` (`string`) optional
Environment class for the cluster. Possible options are mw1.small, mw1.medium, mw1.large. **Default value:** `"mw1.small"`
`execution_role_arn` (`string`) optional
If `create_iam_role` is `false` then set this to the target MWAA execution role **Default value:** `""`
`max_workers` (`number`) optional
The maximum number of workers that can be automatically scaled up. Value need to be between 1 and 25. **Default value:** `10`
`min_workers` (`number`) optional
The minimum number of workers that you want to run in your environment. **Default value:** `1`
`plugins_s3_object_version` (`string`) optional
The plugins.zip file version you want to use. **Default value:** `null`
`plugins_s3_path` (`string`) optional
The relative path to the plugins.zip file on your Amazon S3 storage bucket. For example, plugins.zip. If a relative path is provided in the request, then plugins_s3_object_version is required **Default value:** `null`
`requirements_s3_object_version` (`string`) optional
The requirements.txt file version you **Default value:** `null`
`requirements_s3_path` (`string`) optional
The relative path to the requirements.txt file on your Amazon S3 storage bucket. For example, requirements.txt. If a relative path is provided in the request, then requirements_s3_object_version is required **Default value:** `null`
`scheduler_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the schedulers **Default value:** `false`
`scheduler_logs_level` (`string`) optional
Schedulers logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`source_bucket_arn` (`string`) optional
Set this to the Amazon Resource Name (ARN) of your Amazon S3 storage bucket. **Default value:** `null`
`task_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for DAG tasks **Default value:** `false`
`task_logs_level` (`string`) optional
DAG tasks logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`webserver_access_mode` (`string`) optional
Specifies whether the webserver is accessible over the internet, PUBLIC_ONLY or PRIVATE_ONLY **Default value:** `"PRIVATE_ONLY"`
`webserver_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the webservers **Default value:** `false`
`webserver_logs_level` (`string`) optional
Webserver logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`weekly_maintenance_window_start` (`string`) optional
Specifies the start date for the weekly maintenance window. **Default value:** `null`
`worker_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the workers **Default value:** `false`
`worker_logs_level` (`string`) optional
Workers logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of MWAA environment.
`created_at`
The Created At date of the Amazon MWAA Environment
`execution_role_arn`
IAM Role ARN for Amazon MWAA Execution Role
`logging_configuration`
The Logging Configuration of the MWAA Environment
`s3_bucket_arn`
ID of S3 bucket.
`security_group_id`
ID of the MWAA Security Group(s)
`service_role_arn`
The Service Role ARN of the Amazon MWAA Environment
`status`
The status of the Amazon MWAA Environment
`tags_all`
A map of tags assigned to the resource, including those inherited from the provider for the Amazon MWAA Environment
`webserver_url`
The webserver URL of the Amazon MWAA Environment
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_policy` | 0.5.0 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/0.5.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `mwaa_environment` | 0.15.0 | [`cloudposse/mwaa/aws`](https://registry.terraform.io/modules/cloudposse/mwaa/aws/0.15.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpc_ingress` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.mwaa_web_server_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role_policy_attachment.mwaa_web_server_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.secrets_manager_read_write`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: --- ## network-firewall This component is responsible for provisioning [AWS Network Firewall](https://aws.amazon.com/network-firewal) resources, including Network Firewall, firewall policy, rule groups, and logging configuration. ## Usage **Stack Level**: Regional Example of a Network Firewall with stateful 5-tuple rules: :::tip The "5-tuple" means the five items (columns) that each rule (row, or tuple) in a firewall policy uses to define whether to block or allow traffic: source and destination IP, source and destination port, and protocol. Refer to [Standard stateful rule groups in AWS Network Firewall](https://docs.aws.amazon.com/network-firewall/latest/developerguide/stateful-rule-groups-basic.html) for more details. ::: ```yaml components: terraform: network-firewall: settings: spacelift: workspace_enabled: true vars: enabled: true name: network-firewall # The name of a VPC component where the Network Firewall is provisioned vpc_component_name: vpc firewall_subnet_name: "firewall" stateful_default_actions: - "aws:alert_strict" stateless_default_actions: - "aws:forward_to_sfe" stateless_fragment_default_actions: - "aws:forward_to_sfe" stateless_custom_actions: [] delete_protection: false firewall_policy_change_protection: false subnet_change_protection: false logging_config: [] rule_group_config: stateful-packet-inspection: capacity: 50 name: stateful-packet-inspection description: "Stateful inspection of packets" type: "STATEFUL" rule_group: stateful_rule_options: rule_order: "STRICT_ORDER" rules_source: stateful_rule: - action: "DROP" header: destination: "124.1.1.24/32" destination_port: 53 direction: "ANY" protocol: "TCP" source: "1.2.3.4/32" source_port: 53 rule_option: keyword: "sid:1" - action: "PASS" header: destination: "ANY" destination_port: "ANY" direction: "ANY" protocol: "TCP" source: "10.10.192.0/19" source_port: "ANY" rule_option: keyword: "sid:2" - action: "PASS" header: destination: "ANY" destination_port: "ANY" direction: "ANY" protocol: "TCP" source: "10.10.224.0/19" source_port: "ANY" rule_option: keyword: "sid:3" ``` Example of a Network Firewall with [Suricata](https://suricata.readthedocs.io/en/suricata-6.0.0/rules/) rules: :::tip For [Suricata](https://suricata.io/) rule group type, you provide match and action settings in a string, in a Suricata compatible specification. The specification fully defines what the stateful rules engine looks for in a traffic flow and the action to take on the packets in a flow that matches the inspection criteria. Refer to [Suricata compatible rule strings in AWS Network Firewall](https://docs.aws.amazon.com/network-firewall/latest/developerguide/stateful-rule-groups-suricata.html) for more details. ::: ```yaml components: terraform: network-firewall: metadata: component: "network-firewall" settings: spacelift: workspace_enabled: true vars: enabled: true name: "network-firewall" # The name of a VPC component where the Network Firewall is provisioned vpc_component_name: "vpc" firewall_subnet_name: "firewall" delete_protection: false firewall_policy_change_protection: false subnet_change_protection: false # Logging config logging_enabled: true flow_logs_bucket_component_name: "network-firewall-logs-bucket-flow" alert_logs_bucket_component_name: "network-firewall-logs-bucket-alert" # https://docs.aws.amazon.com/network-firewall/latest/developerguide/stateless-default-actions.html # https://docs.aws.amazon.com/network-firewall/latest/APIReference/API_FirewallPolicy.html # https://docs.aws.amazon.com/network-firewall/latest/developerguide/rule-action.html#rule-action-stateless stateless_default_actions: - "aws:forward_to_sfe" stateless_fragment_default_actions: - "aws:forward_to_sfe" stateless_custom_actions: [] # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-rule-evaluation-order.html#suricata-strict-rule-evaluation-order.html # https://github.com/aws-samples/aws-network-firewall-strict-rule-ordering-terraform policy_stateful_engine_options_rule_order: "STRICT_ORDER" # https://docs.aws.amazon.com/network-firewall/latest/developerguide/stateful-default-actions.html # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-rule-evaluation-order.html#suricata-default-rule-evaluation-order # https://docs.aws.amazon.com/network-firewall/latest/APIReference/API_FirewallPolicy.html stateful_default_actions: - "aws:alert_established" # - "aws:alert_strict" # - "aws:drop_established" # - "aws:drop_strict" # https://docs.aws.amazon.com/network-firewall/latest/developerguide/rule-groups.html rule_group_config: stateful-inspection: # https://docs.aws.amazon.com/network-firewall/latest/developerguide/rule-group-managing.html#nwfw-rule-group-capacity # For stateful rules, `capacity` means the max number of rules in the rule group capacity: 1000 name: "stateful-inspection" description: "Stateful inspection of packets" type: "STATEFUL" rule_group: rule_variables: port_sets: [] ip_sets: - key: "CIDR_1" definition: - "10.10.0.0/11" - key: "CIDR_2" definition: - "10.11.0.0/11" - key: "SCANNER" definition: - "10.12.48.186/32" # bad actors - key: "BLOCKED_LIST" definition: - "193.142.146.35/32" - "69.40.195.236/32" - "125.17.153.207/32" - "185.220.101.4/32" - "195.219.212.151/32" - "162.247.72.199/32" - "147.185.254.17/32" - "179.60.147.101/32" - "157.230.244.66/32" - "192.99.4.116/32" - "62.102.148.69/32" - "185.129.62.62/32" stateful_rule_options: # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-rule-evaluation-order.html#suricata-strict-rule-evaluation-order.html # All the stateful rule groups are provided to the rule engine as Suricata compatible strings # Suricata can evaluate stateful rule groups by using the default rule group ordering method, # or you can set an exact order using the strict ordering method. # The settings for your rule groups must match the settings for the firewall policy that they belong to. # With strict ordering, the rule groups are evaluated by order of priority, starting from the lowest number, # and the rules in each rule group are processed in the order in which they're defined. rule_order: "STRICT_ORDER" # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-how-to-provide-rules.html rules_source: # Suricata rules for the rule group # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-examples.html # https://docs.aws.amazon.com/network-firewall/latest/developerguide/suricata-rule-evaluation-order.html # https://github.com/aws-samples/aws-network-firewall-terraform/blob/main/firewall.tf#L66 # https://docs.aws.amazon.com/network-firewall/latest/developerguide/stateful-rule-groups-suricata.html # https://coralogix.com/blog/writing-effective-suricata-rules-for-the-sta/ # https://suricata.readthedocs.io/en/suricata-6.0.10/rules/intro.html # https://suricata.readthedocs.io/en/suricata-6.0.0/rules/header-keywords.html # https://docs.aws.amazon.com/network-firewall/latest/developerguide/rule-action.html # # With Strict evaluation order, the rules in each rule group are processed in the order in which they're defined # # Pass – Discontinue inspection of the matching packet and permit it to go to its intended destination # # Drop or Alert – Evaluate the packet against all rules with drop or alert action settings. # If the firewall has alert logging configured, send a message to the firewall's alert logs for each matching rule. # The first log entry for the packet will be for the first rule that matched the packet. # After all rules have been evaluated, handle the packet according to the action setting in the first rule that matched the packet. # If the first rule has a drop action, block the packet. If it has an alert action, continue evaluation. # # Reject – Drop traffic that matches the conditions of the stateful rule and send a TCP reset packet back to sender of the packet. # A TCP reset packet is a packet with no payload and a RST bit contained in the TCP header flags. # Reject is available only for TCP traffic. This option doesn't support FTP and IMAP protocols. rules_string: | alert ip $BLOCKED_LIST any <> any any ( msg:"Alert on blocked traffic"; sid:100; rev:1; ) drop ip $BLOCKED_LIST any <> any any ( msg:"Blocked blocked traffic"; sid:200; rev:1; ) pass ip $SCANNER any -> any any ( msg: "Allow scanner"; sid:300; rev:1; ) alert ip $CIDR_1 any -> $CIDR_2 any ( msg:"Alert on CIDR_1 to CIDR_2 traffic"; sid:400; rev:1; ) drop ip $CIDR_1 any -> $CIDR_2 any ( msg:"Blocked CIDR_1 to CIDR_2 traffic"; sid:410; rev:1; ) pass ip any any <> any any ( msg: "Allow general traffic"; sid:10000; rev:1; ) ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`rule_group_config` (`any`) required
Rule group configuration. Refer to [networkfirewall_rule_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_rule_group) for configuration details
`vpc_component_name` (`string`) required
The name of a VPC component where the Network Firewall is provisioned
### Optional Variables
`alert_logs_bucket_component_name` (`string`) optional
Alert logs bucket component name **Default value:** `null`
`delete_protection` (`bool`) optional
A boolean flag indicating whether it is possible to delete the firewall **Default value:** `false`
`firewall_policy_change_protection` (`bool`) optional
A boolean flag indicating whether it is possible to change the associated firewall policy **Default value:** `false`
`firewall_subnet_name` (`string`) optional
Firewall subnet name **Default value:** `"firewall"`
`flow_logs_bucket_component_name` (`string`) optional
Flow logs bucket component name **Default value:** `null`
`logging_enabled` (`bool`) optional
Flag to enable/disable Network Firewall Flow and Alert Logs **Default value:** `false`
`network_firewall_description` (`string`) optional
AWS Network Firewall description. If not provided, the Network Firewall name will be used **Default value:** `null`
`network_firewall_name` (`string`) optional
Friendly name to give the Network Firewall. If not provided, the name will be derived from the context. Changing the name will cause the Firewall to be deleted and recreated. **Default value:** `null`
`network_firewall_policy_name` (`string`) optional
Friendly name to give the Network Firewall policy. If not provided, the name will be derived from the context. Changing the name will cause the policy to be deleted and recreated. **Default value:** `null`
`policy_stateful_engine_options_rule_order` (`string`) optional
Indicates how to manage the order of stateful rule evaluation for the policy. Valid values: DEFAULT_ACTION_ORDER, STRICT_ORDER **Default value:** `null`
`stateful_default_actions` (`list(string)`) optional
Default stateful actions **Default value:** ```hcl [ "aws:alert_strict" ] ```
`stateless_custom_actions` optional
Set of configuration blocks describing the custom action definitions that are available for use in the firewall policy's `stateless_default_actions` **Type:** ```hcl list(object({ action_name = string dimensions = list(string) })) ``` **Default value:** `[ ]`
`stateless_default_actions` (`list(string)`) optional
Default stateless actions **Default value:** ```hcl [ "aws:forward_to_sfe" ] ```
`stateless_fragment_default_actions` (`list(string)`) optional
Default stateless actions for fragmented packets **Default value:** ```hcl [ "aws:forward_to_sfe" ] ```
`subnet_change_protection` (`bool`) optional
A boolean flag indicating whether it is possible to change the associated subnet(s) **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`az_subnet_endpoint_stats`
List of objects with each object having three items: AZ, subnet ID, VPC endpoint ID
`network_firewall_arn`
Network Firewall ARN
`network_firewall_name`
Network Firewall name
`network_firewall_policy_arn`
Network Firewall policy ARN
`network_firewall_policy_name`
Network Firewall policy name
`network_firewall_status`
Nested list of information about the current status of the Network Firewall
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alert_logs_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `flow_logs_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `network_firewall` | 0.3.2 | [`cloudposse/network-firewall/aws`](https://registry.terraform.io/modules/cloudposse/network-firewall/aws/0.3.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## nlb This component provisions an AWS Network Load Balancer (NLB) using the upstream Cloud Posse Terraform module. It supports internet-facing or internal NLBs, TCP/TLS/UDP listeners, optional default target group, cross-zone load balancing, access logs, subnet EIP mappings, and deletion protection. It integrates with other Atmos components via remote state: - `vpc`: to automatically source the VPC ID and appropriate subnets (public or private) when not provided explicitly. - `acm` or `dns-delegated`: to automatically discover an ACM certificate ARN for TLS listeners when `certificate_arn` is not provided. Behavior is controlled by `dns_acm_enabled`. You can also override any of these via input variables, including providing a specific `certificate_arn` directly. ## Usage **Stack Level**: Regional ### Example Here's an example snippet showing how to use this component in an Atmos stack. ```yaml components: terraform: vpc: vars: availability_zones: ["a", "b", "c"] ipv4_primary_cidr_block: "10.100.0.0/18" nlb: vars: # Core settings internal: false tcp_enabled: true tcp_port: 80 tls_enabled: true tls_port: 443 # Optional: discover cert from other components via remote state # Toggle behavior with `dns_acm_enabled` or provide `certificate_arn` dns_acm_enabled: true acm_component_name: acm dns_delegated_component_name: dns-delegated # certificate_arn: "arn:aws:acm:..." # Optional: map EIPs to subnets subnet_mapping_enabled: false eip_allocation_ids: [] # Other common options cross_zone_load_balancing_enabled: true deletion_protection_enabled: false access_logs_enabled: false ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_logs_enabled` (`bool`) optional
A boolean flag to enable/disable access_logs **Default value:** `false`
`acm_component_name` (`string`) optional
Atmos `acm` component name **Default value:** `"acm"`
`certificate_arn` (`string`) optional
ARN of the certificate for the TLS listener **Default value:** `null`
`cross_zone_load_balancing_enabled` (`bool`) optional
Enable cross zone load balancing **Default value:** `true`
`deletion_protection_enabled` (`bool`) optional
Enable deletion protection for the NLB **Default value:** `false`
`deregistration_delay` (`number`) optional
Time to wait before deregistering targets **Default value:** `15`
`dns_acm_enabled` (`bool`) optional
If `true`, use the ACM ARN created by the given `dns-delegated` component. Otherwise, use the ACM ARN created by the given `acm` component. Overridden by `certificate_arn` **Default value:** `false`
`dns_delegated_component_name` (`string`) optional
Atmos `dns-delegated` component name **Default value:** `"dns-delegated"`
`dns_delegated_environment_name` (`string`) optional
`dns-delegated` component environment name **Default value:** `null`
`eip_allocation_ids` (`list(string)`) optional
Allocation IDs for EIPs when subnet mapping is enabled **Default value:** `[ ]`
`health_check_enabled` (`bool`) optional
Enable health checks **Default value:** `true`
`health_check_port` (`number`) optional
Port for health checks **Default value:** `null`
`internal` (`bool`) optional
Whether the NLB is internal **Default value:** `false`
`load_balancer_name` (`string`) optional
Name for the load balancer **Default value:** `""`
`load_balancer_name_max_length` (`number`) optional
Max length for the load balancer name **Default value:** `32`
`security_group_ids` (`list(string)`) optional
Additional security group IDs to allow access to the NLB **Default value:** `[ ]`
`stickiness_enabled` (`bool`) optional
Enable stickiness on the default target group **Default value:** `false`
`subnet_ids` (`list(string)`) optional
List of subnet IDs to associate with NLB **Default value:** `null`
`subnet_mapping_enabled` (`bool`) optional
Create EIPs for the provided subnet IDs **Default value:** `false`
`target_group_enabled` (`bool`) optional
Whether to create the default target group and listener **Default value:** `true`
`target_group_ip_address_type` (`string`) optional
IP address type for the target group **Default value:** `"ipv4"`
`target_group_name` (`string`) optional
Name of the default target group **Default value:** `""`
`target_group_name_max_length` (`number`) optional
Max length for the target group name **Default value:** `32`
`target_group_port` (`number`) optional
Port for the default target group **Default value:** `80`
`target_group_target_type` (`string`) optional
Target type for the default target group **Default value:** `"ip"`
`tcp_enabled` (`bool`) optional
Enable the TCP listener **Default value:** `true`
`tcp_port` (`number`) optional
Port for the TCP listener **Default value:** `80`
`tls_enabled` (`bool`) optional
Enable the TLS listener **Default value:** `false`
`tls_port` (`number`) optional
Port for the TLS listener **Default value:** `443`
`udp_enabled` (`bool`) optional
Enable the UDP listener **Default value:** `false`
`udp_port` (`number`) optional
Port for the UDP listener **Default value:** `53`
`vpc_component_name` (`string`) optional
Name of the VPC component **Default value:** `"vpc"`
`vpc_id` (`string`) optional
VPC ID to associate with NLB **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`nlb`
The NLB of the Component
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `nlb` | 0.18.2 | [`cloudposse/nlb/aws`](https://registry.terraform.io/modules/cloudposse/nlb/aws/0.18.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## opsgenie-team ## Deprecated This module is deprecated and will be archived on January 17, 2026. Please see the [pinned issue](https://github.com/cloudposse-terraform-components/aws-opsgenie-team/issues/57) for details and migration guidance. ### Historical Description This component provisions Opsgenie teams and related services, rules, and schedules. ## Usage #### Pre-requisites You need an API Key stored in `/opsgenie/opsgenie_api_key` of SSM; this is configurable using the `ssm_parameter_name_format` and `ssm_path` variables. Opsgenie is now part of Atlassian, so you need to make sure you are creating an Opsgenie API Key, which looks like `abcdef12-3456-7890-abcd-ef0123456789` and not an Atlassian API key, which looks like: ```shell ATAfT3xFfGF0VFXAfl8EmQNPVv1Hlazp3wsJgTmM8Ph7iP-RtQyiEfw-fkDS2LvymlyUOOhc5XiSx46vQWnznCJolq-GMX4KzdvOSPhEWr-BF6LEkJQC4CSjDJv0N7d91-0gVekNmCD2kXY9haUHUSpO4H7X6QxyImUb9VmOKIWTbQi8rf4CF28=63CB21B9 ``` Generate an API Key by going to Settings -> API key management on your Opsgenie control panel, which will have an address like `https://.app.opsgenie.com/settings/api-key-management`, and click the "Add new API key" button. Once you have the key, test it with curl to verify that you are at least on a Standard plan with OpsGenie: ``` curl -X GET 'https://api.opsgenie.com/v2/account' \ --header "Authorization: GenieKey $API_KEY" ``` The result should be something similar to below: ``` { "data": { "name": "opsgenie", "plan": { "maxUserCount": 1500, "name": "Enterprise", ... } ``` If you see `Free` or `Essentials` in the plan, then you won't be able to use this component. #### Getting Started - Stack Level: Global This component should only be applied once as the resources it creates are regional, but it works with integrations. This is typically done via the auto or corp stack (e.g. `gbl-auto.yaml`). ```yaml # 9-5 Mon-Fri business_hours: &business_hours type: "weekday-and-time-of-day" restrictions: - start_hour: 9 start_min: 00 start_day: "monday" end_hour: 17 end_min: 00 end_day: "friday" # 9-5 Every Day waking_hours: &waking_hours type: "time-of-day" restrictions: - start_hour: 9 start_min: 00 end_hour: 17 end_min: 00 # This is a partial incident mapping, we use this as a base to add P1 & P2 below. This is not a complete mapping as there is no P0 priority_level_to_incident: &priority_level_to_incident enabled: true type: incident priority: P1 order: 1 notify: # if omitted, this will default to the default schedule type: schedule name: default criteria: type: "match-all-conditions" conditions: - field: priority operation: equals expected_value: P0 p1: &p1_is_incident <<: *priority_level_to_incident priority: P1 criteria: type: "match-all-conditions" conditions: - field: priority operation: equals expected_value: P1 p2: &p2_is_incident <<: *priority_level_to_incident priority: P2 criteria: type: "match-all-conditions" conditions: - field: priority operation: equals expected_value: P2 components: terraform: # defaults opsgenie-team-defaults: metadata: type: abstract component: opsgenie-team vars: schedules: london_schedule: enabled: false description: "London Schedule" timezone: "Europe/London" # Routing Rules determine how alerts are routed to the team, # this includes priority changes, incident mappings, and schedules. routing_rules: london_schedule: enabled: false type: alert # https://support.atlassian.com/opsgenie/docs/supported-timezone-ids/ timezone: Europe/London notify: type: schedule # could be escalation, could be none name: london_schedule time_restriction: *waking_hours criteria: type: "match-all-conditions" conditions: - field: priority operation: greater-than expected_value: P2 # Since Incidents require a service, we create a rule for every `routing_rule` type `incident` for every service on the team. # This is done behind the scenes by the `opsgenie-team` component. # These rules below map P1 & P2 to incidents, using yaml anchors from above. p1: *p1_is_incident p2: *p2_is_incident # New team opsgenie-team-sre: metadata: type: real component: opsgenie-team inherits: - opsgenie-team-defaults vars: enabled: true name: sre # These members will be added with an opsgenie_user # To clickops members, set this key to an empty list `[]` members: - user: user@example.com role: owner escalations: otherteam_escalation: enabled: true name: otherteam_escalation description: Other team escalation rules: condition: if-not-acked notify_type: default delay: 60 recipients: - type: team name: otherteam yaep_escalation: enabled: true name: yaep_escalation description: Yet another escalation policy rules: condition: if-not-acked notify_type: default delay: 90 recipients: - type: user name: user@example.com schedule_escalation: enabled: true name: schedule_escalation description: Schedule escalation policy rules: condition: if-not-acked notify_type: default delay: 30 recipients: - type: schedule name: secondary_on_call ``` The API keys relating to the Opsgenie Integrations are stored in SSM Parameter Store and can be accessed via chamber. ``` AWS_PROFILE=foo chamber list opsgenie-team/ ``` ### ClickOps Work - After deploying the opsgenie-team component the created team will have a schedule named after the team. This is purposely left to be clickOps’d so the UI can be used to set who is on call, as that is the usual way (not through code). Additionally, we do not want a re-apply of the Terraform to delete or shuffle who is planned to be on call, thus we left who is on-call on a schedule out of the component. ### Known Issues #### Different API Endpoints in Use The problem is there are 3 different api endpoints in use - `/webapp` - the most robust - only exposed to the UI (that we've seen) - `/v2/` - robust with some differences from `webapp` - `/v1/` - the oldest and furthest from the live UI. #### Cannot create users This module does not create users. Users must have already been created to be added to a team. #### Cannot Add dependent Services - Api Currently doesn't support Multiple ServiceIds for incident Rules #### Cannot Add Stakeholders - Track the issue: https://github.com/opsgenie/terraform-provider-opsgenie/issues/278 #### No Resource to create Slack Integration - Track the issue: https://github.com/DataDog/terraform-provider-datadog/issues/67 #### Out of Date Terraform Docs Another Problem is the terraform docs are not always up to date with the provider code. The OpsGenie Provider uses a mix of `/v1` and `/v2`. This means there are many things you can only do from the UI. Listed below in no particular order - Incident Routing cannot add dependent services - in `v1` and `v2` a `service_incident_rule` object has `serviceId` as type string, in webapp this becomes `serviceIds` of type `list(string)` - Opsgenie Provider appears to be inconsistent with how it uses `time_restriction`: - `restrictions` for type `weekday-and-time-of-day` - `restriction` for type `time-of-day` Unfortunately none of this is in the terraform docs, and was found via errors and digging through source code. Track the issue: https://github.com/opsgenie/terraform-provider-opsgenie/issues/282 #### GMT Style Timezones We recommend to use the human readable timezone such as `Europe/London`. - Setting a schedule to a GMT-style timezone with offsets can cause inconsistent plans. Setting the timezone to `Etc/GMT+1` instead of `Europe/London`, will lead to permadrift as OpsGenie converts the GMT offsets to regional timezones at deploy-time. In the previous deploy, the GMT style get converted to `Atlantic/Cape_Verde`. ```hcl # module.routing["london_schedule"].module.team_routing_rule[0].opsgenie_team_routing_rule.this[0] will be updated in-place ~ resource "opsgenie_team_routing_rule" "this" { id = "4b4c4454-8ccf-41a9-b856-02bec6419ba7" name = "london_schedule" ~ timezone = "Atlantic/Cape_Verde" -> "Etc/GMT+1" # (2 unchanged attributes hidden) ``` Some GMT styles will not cause a timezone change on subsequent applies such as `Etc/GMT+8` for `Asia/Taipei`. - If the calendar date has crossed daylight savings time, the `Etc/GMT+` GMT style will need to be updated to reflect the correct timezone. Track the issue: https://github.com/opsgenie/terraform-provider-opsgenie/issues/258 ### Related How-to Guides [See OpsGenie in the Reference Architecture](https://docs.cloudposse.com/layers/alerting/opsgenie/) ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`create_only_integrations_enabled` (`bool`) optional
Whether to reuse all existing resources and only create new integrations **Default value:** `false`
`datadog_integration_enabled` (`bool`) optional
Whether to enable Datadog integration with opsgenie (datadog side) **Default value:** `true`
`escalations` (`map(any)`) optional
Escalations to configure and create for the team. **Default value:** `{ }`
`integrations` (`map(any)`) optional
API Integrations for the team. If not specified, `datadog` is assumed. **Default value:** `{ }`
`integrations_enabled` (`bool`) optional
Whether to enable the integrations submodule or not **Default value:** `true`
`kms_key_arn` (`string`) optional
AWS KMS key used for writing to SSM **Default value:** `"alias/aws/ssm"`
`members` (`set(any)`) optional
Members as objects with their role within the team. **Default value:** `[ ]`
`routing_rules` (`any`) optional
Routing Rules for the team **Default value:** `null`
`schedules` (`map(any)`) optional
Schedules to create for the team **Default value:** `{ }`
`services` (`map(any)`) optional
Services to create and register to the team. **Default value:** `{ }`
`ssm_parameter_name_format` (`string`) optional
SSM parameter name format **Default value:** `"/%s/%s"`
`ssm_path` (`string`) optional
SSM path **Default value:** `"opsgenie"`
`team_naming_format` (`string`) optional
OpsGenie Team Naming Format **Default value:** `"%s_%s"`
`team_options` optional
Configure the team options. See `opsgenie_team` Terraform resource [documentation](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/team#argument-reference) for more details. **Type:** ```hcl object({ description = optional(string) ignore_members = optional(bool, false) delete_default_resources = optional(bool, false) }) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`escalation`
Escalation rules created
`integration`
Integrations created
`routing`
Routing rules created
`team_id`
Team ID
`team_members`
Team members
`team_name`
Team Name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` - `opsgenie`, version: `>= 0.6.7` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `datadog`, version: `>= 3.3.0` - `opsgenie`, version: `>= 0.6.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `datadog_configuration` | v1.535.11 | [`github.com/cloudposse-terraform-components/aws-datadog-credentials//src/modules/datadog_keys`](https://registry.terraform.io/modules/github.com/cloudposse-terraform-components/aws-datadog-credentials/src/modules/datadog_keys/v1.535.11) | n/a `escalation` | latest | [`./modules/escalation`](https://registry.terraform.io/modules/./modules/escalation/) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `integration` | latest | [`./modules/integration`](https://registry.terraform.io/modules/./modules/integration/) | n/a `members_merge` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `routing` | latest | [`./modules/routing`](https://registry.terraform.io/modules/./modules/routing/) | n/a `schedule` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/schedule`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/schedule/0.16.0) | n/a `service` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/service`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/service/0.16.0) | n/a `team` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/team`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/team/0.16.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`datadog_integration_opsgenie_service_object.fake_service_name`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_opsgenie_service_object) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.opsgenie_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.opsgenie_team_api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`opsgenie_team.existing`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/team) (data source) - [`opsgenie_user.team_members`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/user) (data source) --- ## escalation ## Escalation Terraform module to configure [Opsgenie Escalation](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/escalation) ## Usage [Create Opsgenie Escalation example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/escalation) ```hcl module "escalation" { source = "cloudposse/incident-management/opsgenie//modules/escalation" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" escalation = { name = module.label.id owner_team_id = module.owner_team.team_id rule = { recipients = [{ type = "team" id = module.escalation_team.team_id }] } } } ``` ## Variables ### Required Variables
### Optional Variables
`escalation` (`any`) optional
Opsgenie Escalation configuration **Default value:** `{ }`
`team_name` (`string`) optional
Current OpsGenie Team Name **Default value:** `null`
`team_naming_format` (`string`) optional
OpsGenie Team Naming Format **Default value:** `"%s_%s"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`escalation_id`
The ID of the Opsgenie Escalation
`escalation_name`
Name of the Opsgenie Escalation
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `opsgenie`, version: `>= 0.6.7` ### Providers - `opsgenie`, version: `>= 0.6.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`opsgenie_escalation.this`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/escalation) (resource) ## Data Sources The following data sources are used by this module: - [`opsgenie_schedule.recipient`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/schedule) (data source) - [`opsgenie_team.recipient`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/team) (data source) - [`opsgenie_user.recipient`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/user) (data source) None --- ## integration ## Integration This module creates an OpsGenie integrations for a team. By Default, it creates a Datadog integration. ## Variables ### Required Variables
`team_name` (`string`) required
Name of the team to assign this integration to.
`type` (`string`) required
API Integration Type
### Optional Variables
`append_datadog_tags_enabled` (`bool`) optional
Add Datadog Tags to the Tags of alerts from this integration. **Default value:** `true`
`kms_key_arn` (`string`) optional
AWS KMS key used for writing to SSM **Default value:** `"alias/aws/ssm"`
`ssm_path_format` (`string`) optional
SSM parameter name format **Default value:** `"/opsgenie-team/%s"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ssm_path`
Full SSM path of the team integration key
`type`
Type of the team integration
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `opsgenie`, version: `>= 0.6.7` ### Providers - `opsgenie`, version: `>= 0.6.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `api_integration` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/api_integration`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/api_integration/0.16.0) | n/a `integration_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | Fully qualified integration name normalized `ssm_parameter_store` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | Populate SSM Parameter Store with API Keys for OpsGenie API Integrations. These keys can either be used when setting up OpsGenie integrations manually, Or they can be used programmatically, if their respective Terraform provider supports it. `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`opsgenie_integration_action.datadog`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/integration_action) (resource) ## Data Sources The following data sources are used by this module: - [`opsgenie_team.default`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/team) (data source) None --- ## routing ## Routing This module creates team routing rules, these are the initial rules that are applied to an alert to determine who gets notified. This module also creates incident service rules, which determine if an alert is considered a service incident or not. ## Variables ### Required Variables
`criteria` required
Criteria of the Routing Rule, rules to match or not **Type:** ```hcl object({ type = string, conditions = any }) ```
`incident_properties` (`map(any)`) required
Properties to override on the incident routing rule
`notify` (`map(any)`) required
Notification of team alerting rule
`order` (`number`) required
Order of the alerting rule
`priority` (`string`) required
Priority level of custom Incidents
### Optional Variables
`services` (`map(any)`) optional
Team services to associate with incident routing rules **Default value:** `null`
`team_name` (`string`) optional
Current OpsGenie Team Name **Default value:** `null`
`team_naming_format` (`string`) optional
OpsGenie Team Naming Format **Default value:** `"%s_%s"`
`time_restriction` (`any`) optional
Time restriction of alert routing rule **Default value:** `null`
`timezone` (`string`) optional
Timezone for this alerting route **Default value:** `null`
`type` (`string`) optional
Type of Routing Rule Alert or Incident **Default value:** `"alert"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`service_incident_rule`
Service incident rules for incidents
`team_routing_rule`
Team routing rules for alerts
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `opsgenie`, version: `>= 0.6.7` ### Providers - `opsgenie`, version: `>= 0.6.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `service_incident_rule` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/service_incident_rule`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/service_incident_rule/0.16.0) | n/a `serviceless_incident_rule` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/service_incident_rule`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/service_incident_rule/0.16.0) | n/a `team_routing_rule` | 0.16.0 | [`cloudposse/incident-management/opsgenie//modules/team_routing_rule`](https://registry.terraform.io/modules/cloudposse/incident-management/opsgenie/modules/team_routing_rule/0.16.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`opsgenie_schedule.notification_schedule`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/schedule) (data source) - [`opsgenie_service.incident_service`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/service) (data source) - [`opsgenie_team.default`](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/data-sources/team) (data source) None --- ## philips-labs-github-runners This component provisions the surrounding infrastructure for GitHub self-hosted runners. ## Prerequisites - GitHub App installed on the organization - For more details see [Philips Lab's Setting up a GitHub App](https://github.com/philips-labs/terraform-aws-github-runner/tree/main#setup-github-app-part-1) - Ensure you create a **PRIVATE KEY** and store it in SSM, **NOT** to be confused with a **Client Secret**. Private Keys are created in the GitHub App Configuration and scrolling to the bottom. - GitHub App ID and private key stored in SSM under `/pl-github-runners/id` (or the value of `var.github_app_id_ssm_path`) - GitHub App Private Key stored in SSM (base64 encoded) under `/pl-github-runners/key` (or the value of `var.github_app_key_ssm_path`) ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: philips-labs-github-runners: vars: enabled: true ``` The following will create - An API Gateway - Lambdas - SQS Queue - EC2 Launch Template instances The API Gateway is registered as a webhook within the GitHub app. Which scales up or down, via lambdas, the EC2 Launch Template by the number of messages in the SQS queue. ![Architecture](https://github.com/philips-labs/terraform-aws-github-runner/blob/main/docs/component-overview.svg) ## Modules ### `webhook-github-app` This is a fork of https://github.com/philips-labs/terraform-aws-github-runner/tree/main/modules/webhook-github-app. We customized it until this PR is resolved as it does not update the GitHub App webhook until this is merged. - https://github.com/philips-labs/terraform-aws-github-runner/pull/3625 This module also requires an environment variable: - `GH_TOKEN` — a GitHub token must be set This module also requires the `gh` CLI to be installed. Your Dockerfile can be updated to include the following to install it: ```dockerfile ARG GH_CLI_VERSION=2.39.1 # ... ARG GH_CLI_VERSION RUN apt-get update && apt-get install -y --allow-downgrades \ gh="${GH_CLI_VERSION}-*" ``` By default, we leave this disabled, as it requires a GitHub token to be set. You can enable it by setting `var.enable_update_github_app_webhook` to `true`. When enabled, it will update the GitHub App webhook to point to the API Gateway. This can occur if the API Gateway is deleted and recreated. When disabled, you will need to manually update the GitHub App webhook to point to the API Gateway. This is output by the component, and available via the `webhook` output under `endpoint`. ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`create_service_linked_role_spot` (`bool`) optional
(optional) create the service linked role for spot instances that is required by the scale-up lambda. **Default value:** `true`
`enable_update_github_app_webhook` (`bool`) optional
Enable updating the github app webhook **Default value:** `false`
`github_app_id_ssm_path` (`string`) optional
Path to the github app id in SSM **Default value:** `"/pl-github-runners/id"`
`github_app_key_ssm_path` (`string`) optional
Path to the github key in SSM **Default value:** `"/pl-github-runners/key"`
`instance_target_capacity_type` (`string`) optional
Default lifecycle used for runner instances, can be either `spot` or `on-demand`. **Default value:** `"spot"`
`release_version` (`string`) optional
Version of the application **Default value:** `"v5.4.0"`
`runner_extra_labels` (`list(string)`) optional
Extra (custom) labels for the runners (GitHub). Labels checks on the webhook can be enforced by setting `enable_workflow_job_labels_check`. GitHub read-only labels should not be provided. **Default value:** ```hcl [ "default" ] ```
`scale_up_reserved_concurrent_executions` (`number`) optional
Amount of reserved concurrent executions for the scale-up lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations. **Default value:** `-1`
`ssm_paths` optional
The root path used in SSM to store configuration and secrets. **Type:** ```hcl object({ root = optional(string, "github-action-runners") app = optional(string, "app") runners = optional(string, "runners") use_prefix = optional(bool, true) }) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`github_runners`
Information about the GitHub runners.
`queues`
Information about the GitHub runner queues. Such as `build_queue_arn` the ARN of the SQS queue to use for the build queue.
`ssm_parameters`
Information about the SSM parameters to use to register the runner.
`webhook`
Information about the webhook to use to register the runner.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `local`, version: `>= 2.4.0` - `random`, version: `>= 3.0` ### Providers - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `github_runner` | 6.1.0 | [`philips-labs/github-runner/aws`](https://registry.terraform.io/modules/philips-labs/github-runner/aws/6.1.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `module_artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `store_read` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `webhook_github_app` | 6.1.0 | [`philips-labs/github-runner/aws//modules/webhook-github-app`](https://registry.terraform.io/modules/philips-labs/github-runner/aws/modules/webhook-github-app/6.1.0) | n/a ## Resources The following resources are used by this module: - [`random_id.webhook_secret`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) (resource) ## Data Sources The following data sources are used by this module: --- ## private-link-service This component provisions AWS VPC Endpoint Services (**provider side**) to expose **YOUR services** to external consumers via AWS PrivateLink. ## What This Component Does **You are the PROVIDER** - This component creates the infrastructure to expose your services (EKS pods, RDS databases, APIs) to other AWS accounts or VPCs. ``` Your AWS Account (PROVIDER) Consumer's AWS Account ┌─────────────────────────────┐ ┌──────────────────────────┐ │ Your Services │ │ Their Applications │ │ - EKS pods │ │ - Airflow (Astronomer) │ │ - RDS databases │ │ - External systems │ │ - Internal APIs │ │ - Partner services │ │ ↓ │ │ ↑ │ │ Network Load Balancer ──────┼─────────┼─────────┘ │ │ ↓ │ AWS │ │ │ VPC Endpoint Service │ Private │ VPC Endpoint │ │ (this component) │ Link │ (they create) │ │ com.amazonaws.vpce... │ │ │ └─────────────────────────────┘ └──────────────────────────┘ ``` **Key Point**: The consumer (e.g., Astronomer) creates a VPC Endpoint in their account that connects to YOUR VPC Endpoint Service. Traffic flows privately over AWS's network, never touching the internet. ## Astronomer Integration This example shows the full workflow for exposing your EKS services to Astronomer's Airflow cluster via PrivateLink. ### Architecture ``` Astronomer's AWS Account YOUR AWS Account ┌──────────────────────────┐ ┌─────────────────────────────────┐ │ Airflow Workers │ │ EKS Cluster │ │ (run DAGs) │ │ │ │ ↓ │ │ Pods labeled: │ │ VPC Endpoint ────────────┼───Private─────┼→ astronomer: enabled │ │ (Astronomer creates) │ Link │ ↓ │ │ │ │ NLB (eks/nlb component) │ │ │ │ ↓ │ │ │ │ VPC Endpoint Service │ │ │ │ (this component) │ └──────────────────────────┘ └─────────────────────────────────┘ ``` ### Step 1: Label Your EKS Pods First, tag the pods you want to expose to Astronomer: ```yaml components: terraform: eks/echo-server: vars: # ... chart_values: labels: astronomer: enabled # ← This label exposes pods to Astronomer ``` ### Step 2: Create NLB via AWS Load Balancer Controller Deploy an NLB that targets your labeled pods: ```yaml components: terraform: eks/nlb/astronomer: metadata: component: eks/nlb vars: enabled: true name: "nlb" attributes: ["astronomer"] # Target pods with the astronomer label nlb_selector: astronomer: enabled ``` ### Step 3: Create VPC Endpoint Service Now expose the NLB via PrivateLink: ```yaml components: terraform: private-link-service/astronomer: metadata: component: private-link-service vars: enabled: true name: "private-link-service" attributes: ["astronomer"] # Reference the NLB created in Step 2 vpc_endpoint_service_network_load_balancer_arns: - !terraform.output eks/nlb/astronomer nlb_arn # Allow Astronomer's AWS account (get from their support) vpc_endpoint_service_allowed_principals: - "arn:aws:iam::ASTRONOMER-ACCOUNT-ID:role/astronomer-remote-management" ``` ### Step 4: Share Service Name with Astronomer Get the VPC Endpoint Service name: ```bash vpc_endpoint_service_name = "com.amazonaws.vpce.us-west-2.vpce-svc-0abc123def456789" ``` ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: private-link-service: vars: enabled: true name: "private-link-service" vpc_endpoint_service_network_load_balancer_arns: - !terraform.output eks/nlb nlb_arn # Get customer AWS account ID or role ARN from their support team # Example (get from Astronomer support): vpc_endpoint_service_allowed_principals: - "arn:aws:iam::ASTRONOMER-ACCOUNT-ID:role/astronomer-remote-management" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`vpc_endpoint_service_acceptance_required` (`bool`) optional
Whether or not VPC endpoint connection requests to the service must be accepted by the service owner **Default value:** `true`
`vpc_endpoint_service_allowed_principals` (`list(string)`) optional
List of ARNs of principals allowed to discover the VPC endpoint service **Default value:** `[ ]`
`vpc_endpoint_service_gateway_load_balancer_arns` (`list(string)`) optional
List of Gateway Load Balancer ARNs to associate with the VPC endpoint service **Default value:** `[ ]`
`vpc_endpoint_service_network_load_balancer_arns` (`list(string)`) optional
List of Network Load Balancer ARNs to associate with the VPC endpoint service **Default value:** `[ ]`
`vpc_endpoint_service_private_dns_name` (`string`) optional
Private DNS name for the VPC endpoint service **Default value:** `null`
`vpc_endpoint_service_supported_ip_address_types` (`list(string)`) optional
The supported IP address types. Valid values: ipv4, ipv6 **Default value:** ```hcl [ "ipv4" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`endpoint_events_sns_topic_arn`
The ARN of the SNS topic for endpoint connection events
`vpc_endpoint_service_arn`
The ARN of the VPC endpoint service
`vpc_endpoint_service_id`
The ID of the VPC endpoint service
`vpc_endpoint_service_name`
The service name that consumers use to connect
`vpc_endpoint_service_state`
The state of the VPC endpoint service
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_sns_topic.endpoint_events`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) (resource) - [`aws_vpc_endpoint_connection_notification.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_connection_notification) (resource) - [`aws_vpc_endpoint_service.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_service) (resource) - [`aws_vpc_endpoint_service_allowed_principal.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_service_allowed_principal) (resource) ## Data Sources The following data sources are used by this module: --- ## rds This component is responsible for provisioning an RDS instance. It seeds relevant database information (hostnames, username, password, etc.) into AWS SSM Parameter Store. Security Groups Guidance: By default this component creates a client security group and adds that security group id to the default attached security group. Ideally other AWS resources that require RDS access can be granted this client security group. Additionally you can grant access via specific CIDR blocks or security group ids. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ### PostgreSQL ```yaml components: terraform: rds/defaults: metadata: type: abstract vars: enabled: true use_fullname: false name: my-postgres-db instance_class: db.t3.micro database_name: my-postgres-db # database_user: admin # enable to specify something specific engine: postgres engine_version: "15.2" database_port: 5432 db_parameter_group: "postgres15" allocated_storage: 10 #GBs ssm_enabled: true client_security_group_enabled: true ## The following settings allow the database to be accessed from anywhere # publicly_accessible: true # use_private_subnets: false # allowed_cidr_blocks: # - 0.0.0.0/0 ``` ### Microsoft SQL ```yaml components: terraform: rds: vars: enabled: true name: mssql # SQL Server 2017 Enterprise engine: sqlserver-ee engine_version: "14.00.3356.20" db_parameter_group: "sqlserver-ee-14.0" license_model: license-included # Required for MSSQL database_name: null database_port: 1433 database_user: mssql instance_class: db.t3.xlarge # There are issues with enabling this multi_az: false allocated_storage: 20 publicly_accessible: false ssm_enabled: true # This does not seem to work correctly deletion_protection: false ``` ### Provisioning from a snapshot The snapshot identifier variable can be added to provision an instance from a snapshot HOWEVER- Keep in mind these instances are provisioned from a unique kms key per rds. For clean terraform runs, you must first provision the key for the destination instance, then copy the snapshot using that kms key. Example - I want a new instance `rds-example-new` to be provisioned from a snapshot of `rds-example-old`: 1. Use the console to manually make a snapshot of rds instance `rds-example-old` 1. provision the kms key for `rds-example-new` ``` atmos terraform plan rds-example-new -s ue1-staging '-target=module.kms_key_rds.aws_kms_key.default[0]' atmos terraform apply rds-example-new -s ue1-staging '-target=module.kms_key_rds.aws_kms_key.default[0]' ``` 1. Use the console to copy the snapshot to a new name using the above provisioned kms key 1. Add `snapshot_identifier` variable to `rds-example-new` catalog and specify the newly copied snapshot that used the above key 1. Post provisioning, remove the `snapshot_idenfier` variable and verify terraform runs clean for the copied instance ## Variables ### Required Variables
`allocated_storage` (`number`) required
The allocated storage in GBs
`database_name` (`string`) required
The name of the database to create when the DB instance is created
`database_port` (`number`) required
Database port (_e.g._ `3306` for `MySQL`). Used in the DB Security Group to allow access to the DB instance from the provided `security_group_ids`
`db_parameter_group` (`string`) required
The DB parameter group family name. The value depends on DB engine used. See [DBParameterGroupFamily](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBParameterGroup.html#API_CreateDBParameterGroup_RequestParameters) for instructions on how to retrieve applicable value.
`engine` (`string`) required
Database engine type
`engine_version` (`string`) required
Database engine version, depends on engine type
`instance_class` (`string`) required
Class of RDS instance
`region` (`string`) required
AWS Region
### Optional Variables
`allow_major_version_upgrade` (`bool`) optional
Allow major version upgrade **Default value:** `false`
`allowed_cidr_blocks` (`list(string)`) optional
The whitelisted CIDRs which to allow `ingress` traffic to the DB instance **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any database modifications are applied immediately, or during the next maintenance window **Default value:** `false`
`associate_security_group_ids` (`list(string)`) optional
The IDs of the existing security groups to associate with the DB instance **Default value:** `[ ]`
`auto_minor_version_upgrade` (`bool`) optional
Allow automated minor version upgrade (e.g. from Postgres 9.5.3 to Postgres 9.5.4) **Default value:** `true`
`availability_zone` (`string`) optional
The AZ for the RDS instance. Specify one of `subnet_ids`, `db_subnet_group_name` or `availability_zone`. If `availability_zone` is provided, the instance will be placed into the default VPC or EC2 Classic **Default value:** `null`
`backup_retention_period` (`number`) optional
Backup retention period in days. Must be > 0 to enable backups **Default value:** `0`
`backup_window` (`string`) optional
When AWS can perform DB snapshots, can't overlap with maintenance window **Default value:** `"22:00-03:00"`
`ca_cert_identifier` (`string`) optional
The identifier of the CA certificate for the DB instance **Default value:** `null`
`charset_name` (`string`) optional
The character set name to use for DB encoding. [Oracle & Microsoft SQL only](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#character_set_name). For other engines use `db_parameter` **Default value:** `null`
`client_security_group_enabled` (`bool`) optional
create a client security group and include in attached default security group **Default value:** `true`
`copy_tags_to_snapshot` (`bool`) optional
Copy tags from DB to a snapshot **Default value:** `true`
`database_password` (`string`) optional
Database password for the admin user **Default value:** `""`
`database_user` (`string`) optional
Database admin user name **Default value:** `""`
`db_options` optional
A list of DB options to apply with an option group. Depends on DB engine **Type:** ```hcl list(object({ db_security_group_memberships = list(string) option_name = string port = number version = string vpc_security_group_memberships = list(string) option_settings = list(object({ name = string value = string })) })) ``` **Default value:** `[ ]`
`db_parameter` optional
A list of DB parameters to apply. Note that parameters may differ from a DB family to another **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`db_subnet_group_name` (`string`) optional
Name of DB subnet group. DB instance will be created in the VPC associated with the DB subnet group. Specify one of `subnet_ids`, `db_subnet_group_name` or `availability_zone` **Default value:** `null`
`deletion_protection` (`bool`) optional
Set to true to enable deletion protection on the RDS instance **Default value:** `false`
`dns_delegated_component_name` (`string`) optional
The name of the Delegated DNS component **Default value:** `"dns-delegated"`
`dns_gbl_delegated_environment_name` (`string`) optional
The name of the environment where global `dns_delegated` is provisioned **Default value:** `"gbl"`
`dns_zone_id` (`string`) optional
The ID of the DNS Zone in Route53 where a new DNS record will be created for the DB host name **Default value:** `""`
`eks_component_name` (`string`) optional
The name of the EKS component **Default value:** `"eks"`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to enable for exporting to CloudWatch logs. If omitted, no logs will be exported. Valid values (depending on engine): alert, audit, error, general, listener, slowquery, trace, postgresql (PostgreSQL), upgrade (PostgreSQL). **Default value:** `[ ]`
`final_snapshot_identifier` (`string`) optional
Final snapshot identifier e.g.: some-db-final-snapshot-2019-06-26-06-05 **Default value:** `""`
`host_name` (`string`) optional
The DB host name created in Route53 **Default value:** `"db"`
`iam_database_authentication_enabled` (`bool`) optional
Specifies whether or mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled **Default value:** `false`
`iops` (`number`) optional
The amount of provisioned IOPS. Setting this implies a storage_type of 'io1'. Default is 0 if rds storage type is not 'io1' **Default value:** `0`
`kms_alias_name_ssm` (`string`) optional
KMS alias name for SSM **Default value:** `"alias/aws/ssm"`
`kms_key_arn` (`string`) optional
The ARN of the existing KMS key to encrypt storage **Default value:** `""`
`license_model` (`string`) optional
License model for this DB. Optional, but required for some DB Engines. Valid values: license-included | bring-your-own-license | general-public-license **Default value:** `""`
`maintenance_window` (`string`) optional
The window to perform maintenance in. Syntax: 'ddd:hh24:mi-ddd:hh24:mi' UTC **Default value:** `"Mon:03:00-Mon:04:00"`
`major_engine_version` (`string`) optional
Database MAJOR engine version, depends on engine type **Default value:** `""`
`max_allocated_storage` (`number`) optional
The upper limit to which RDS can automatically scale the storage in GBs **Default value:** `0`
`monitoring_interval` (`string`) optional
The interval, in seconds, between points when Enhanced Monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. Valid Values are 0, 1, 5, 10, 15, 30, 60. **Default value:** `"0"`
`monitoring_role_arn` (`string`) optional
The ARN for the IAM role that permits RDS to send enhanced monitoring metrics to CloudWatch Logs **Default value:** `null`
`multi_az` (`bool`) optional
Set to true if multi AZ deployment must be supported **Default value:** `false`
`option_group_name` (`string`) optional
Name of the DB option group to associate **Default value:** `""`
`parameter_group_name` (`string`) optional
Name of the DB parameter group to associate **Default value:** `""`
`performance_insights_enabled` (`bool`) optional
Specifies whether Performance Insights are enabled. **Default value:** `false`
`performance_insights_kms_key_id` (`string`) optional
The ARN for the KMS key to encrypt Performance Insights data. Once KMS key is set, it can never be changed. **Default value:** `null`
`performance_insights_retention_period` (`number`) optional
The amount of time in days to retain Performance Insights data. Either 7 (7 days) or 731 (2 years). **Default value:** `7`
`publicly_accessible` (`bool`) optional
Determines if database can be publicly available (NOT recommended) **Default value:** `false`
`replicate_source_db` (`any`) optional
If the rds db instance is a replica, supply the source database identifier here **Default value:** `null`
`security_group_ids` (`list(string)`) optional
The IDs of the security groups from which to allow `ingress` traffic to the DB instance **Default value:** `[ ]`
`skip_final_snapshot` (`bool`) optional
If true (default), no snapshot will be made before deleting DB **Default value:** `true`
`snapshot_identifier` (`string`) optional
Snapshot identifier e.g: rds:production-2019-06-26-06-05. If specified, the module create cluster from the snapshot **Default value:** `null`
`ssm_enabled` (`bool`) optional
If `true` create SSM keys for the database user and password. **Default value:** `false`
`ssm_key_format` (`string`) optional
SSM path format. The values will will be used in the following order: `var.ssm_key_prefix`, `var.name`, `var.ssm_key_*` **Default value:** `"/%v/%v/%v"`
`ssm_key_hostname` (`string`) optional
The SSM key to save the hostname. See `var.ssm_path_format`. **Default value:** `"admin/db_hostname"`
`ssm_key_password` (`string`) optional
The SSM key to save the password. See `var.ssm_path_format`. **Default value:** `"admin/db_password"`
`ssm_key_port` (`string`) optional
The SSM key to save the port. See `var.ssm_path_format`. **Default value:** `"admin/db_port"`
`ssm_key_prefix` (`string`) optional
SSM path prefix. Omit the leading forward slash `/`. **Default value:** `"rds"`
`ssm_key_user` (`string`) optional
The SSM key to save the user. See `var.ssm_path_format`. **Default value:** `"admin/db_user"`
`storage_encrypted` (`bool`) optional
(Optional) Specifies whether the DB instance is encrypted. The default is false if not specified **Default value:** `true`
`storage_throughput` (`number`) optional
The storage throughput value for the DB instance. Can only be set when `storage_type` is `gp3`. Cannot be specified if the `allocated_storage` value is below a per-engine threshold. **Default value:** `null`
`storage_type` (`string`) optional
One of 'standard' (magnetic), 'gp2' (general purpose SSD), or 'io1' (provisioned IOPS SSD) **Default value:** `"standard"`
`timezone` (`string`) optional
Time zone of the DB instance. timezone is currently only supported by Microsoft SQL Server. The timezone can only be set on creation. See [MSSQL User Guide](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SQLServer.html#SQLServer.Concepts.General.TimeZone) for more information. **Default value:** `null`
`use_dns_delegated` (`bool`) optional
Use the dns-delegated dns_zone_id **Default value:** `false`
`use_eks_security_group` (`bool`) optional
Use the eks default security group **Default value:** `false`
`use_private_subnets` (`bool`) optional
Use private subnets **Default value:** `true`
`vpc_component_name` (`string`) optional
VPC component name **Default value:** `"vpc"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`exports`
Map of exports for use in deployment configuration templates
`kms_key_alias`
The KMS key alias
`psql_helper`
A helper output to use with psql for connecting to this RDS instance.
`rds_address`
Address of the instance
`rds_arn`
ARN of the instance
`rds_database_ssm_key_prefix`
SSM prefix
`rds_endpoint`
DNS Endpoint of the instance
`rds_hostname`
DNS host name of the instance
`rds_id`
ID of the instance
`rds_name`
RDS DB name
`rds_option_group_id`
ID of the Option Group
`rds_parameter_group_id`
ID of the Parameter Group
`rds_port`
RDS DB port
`rds_resource_id`
The RDS Resource ID of this instance.
`rds_security_group_id`
ID of the Security Group
`rds_subnet_group_id`
ID of the created Subnet Group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.3` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_rds` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `rds_client_sg` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `rds_instance` | 1.1.2 | [`cloudposse/rds/aws`](https://registry.terraform.io/modules/cloudposse/rds/aws/1.1.2) | n/a `rds_monitoring_role` | 0.22.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.22.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.rds_database_hostname`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.rds_database_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.rds_database_port`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.rds_database_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.database_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.database_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## redshift This component provisions an AWS Redshift cluster and seeds relevant database information (hostnames, username, password, etc.) into AWS SSM Parameter Store. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: redshift: vars: enabled: true name: redshift database_name: redshift publicly_accessible: false node_type: dc2.large number_of_nodes: 1 cluster_type: single-node ssm_enabled: true log_exports: - userlog - connectionlog - useractivitylog admin_user: redshift custom_sg_enabled: true custom_sg_rules: - type: ingress key: postgres description: Allow inbound traffic to the redshift cluster from_port: 5439 to_port: 5439 protocol: tcp cidr_blocks: - 10.0.0.0/8 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`admin_password` (`string`) optional
Password for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `null`
`admin_user` (`string`) optional
Username for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `null`
`allow_version_upgrade` (`bool`) optional
Whether or not to enable major version upgrades which are applied during the maintenance window to the Amazon Redshift engine that is running on the cluster **Default value:** `false`
`cluster_type` (`string`) optional
The cluster type to use. Either `single-node` or `multi-node` **Default value:** `"single-node"`
`custom_sg_allow_all_egress` (`bool`) optional
Whether to allow all egress traffic or not **Default value:** `true`
`custom_sg_enabled` (`bool`) optional
Whether to use custom security group or not **Default value:** `false`
`custom_sg_rules` optional
An array of custom security groups to create and assign to the cluster. **Type:** ```hcl list(object({ key = string type = string from_port = number to_port = number protocol = string cidr_blocks = list(string) description = string })) ``` **Default value:** `[ ]`
`database_name` (`string`) optional
The name of the first database to be created when the cluster is created **Default value:** `null`
`engine_version` (`string`) optional
The version of the Amazon Redshift engine to use. See https://docs.aws.amazon.com/redshift/latest/mgmt/cluster-versions.html **Default value:** `"1.0"`
`kms_alias_name_ssm` (`string`) optional
KMS alias name for SSM **Default value:** `"alias/aws/ssm"`
`node_type` (`string`) optional
The node type to be provisioned for the cluster. See https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html#working-with-clusters-overview **Default value:** `"dc2.large"`
`number_of_nodes` (`number`) optional
The number of compute nodes in the cluster. This parameter is required when the ClusterType parameter is specified as multi-node **Default value:** `1`
`port` (`number`) optional
The port number on which the cluster accepts incoming connections **Default value:** `5439`
`publicly_accessible` (`bool`) optional
If true, the cluster can be accessed from a public network **Default value:** `false`
`security_group_ids` (`list(string)`) optional
An array of security group IDs to associate with the endpoint. **Default value:** `null`
`ssm_enabled` (`bool`) optional
If `true` create SSM keys for the database user and password. **Default value:** `false`
`ssm_key_format` (`string`) optional
SSM path format. The values will will be used in the following order: `var.ssm_key_prefix`, `var.name`, `var.ssm_key_*` **Default value:** `"/%v/%v/%v"`
`ssm_key_hostname` (`string`) optional
The SSM key to save the hostname. See `var.ssm_path_format`. **Default value:** `"admin/db_hostname"`
`ssm_key_password` (`string`) optional
The SSM key to save the password. See `var.ssm_path_format`. **Default value:** `"admin/db_password"`
`ssm_key_port` (`string`) optional
The SSM key to save the port. See `var.ssm_path_format`. **Default value:** `"admin/db_port"`
`ssm_key_prefix` (`string`) optional
SSM path prefix. Omit the leading forward slash `/`. **Default value:** `"redshift"`
`ssm_key_user` (`string`) optional
The SSM key to save the user. See `var.ssm_path_format`. **Default value:** `"admin/db_user"`
`use_private_subnets` (`bool`) optional
Whether to use private or public subnets for the Redshift cluster **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amazon Resource Name (ARN) of cluster
`cluster_identifier`
The Cluster Identifier
`cluster_security_groups`
The security groups associated with the cluster
`database_name`
The name of the default database in the Cluster
`dns_name`
The DNS name of the cluster
`endpoint`
The connection endpoint
`id`
The Redshift Cluster ID
`port`
The Port the cluster responds on
`redshift_database_ssm_key_prefix`
SSM prefix
`vpc_security_group_ids`
The VPC security group IDs associated with the cluster
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.17, < 6.0.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.17, < 6.0.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `redshift_cluster` | 1.3.1 | [`cloudposse/redshift-cluster/aws`](https://registry.terraform.io/modules/cloudposse/redshift-cluster/aws/1.3.1) | n/a `redshift_sg` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.redshift_database_hostname`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.redshift_database_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.redshift_database_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.redshift_database_port`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.redshift_database_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.admin_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.admin_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: --- ## redshift-serverless This component is responsible for provisioning Redshift Serverless clusters. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: ```yaml components: terraform: redshift-serverless: settings: spacelift: workspace_enabled: true vars: enabled: true name: redshift-serverless admin_user: admin database_name: dev ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`admin_password` (`string`) optional
Password for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `null`
`admin_user` (`string`) optional
Username for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `null`
`base_capacity` (`number`) optional
The base data warehouse capacity (4 minimum) of the workgroup in Redshift Processing Units (RPUs). **Default value:** `4`
`config_parameter` optional
A list of Redshift config parameters to apply to the workgroup. **Type:** ```hcl list(object({ parameter_key = string parameter_value = any })) ``` **Default value:** `[ ]`
`custom_sg_allow_all_egress` (`bool`) optional
Whether to allow all egress traffic or not **Default value:** `true`
`custom_sg_enabled` (`bool`) optional
Whether to use custom security group or not **Default value:** `false`
`custom_sg_rules` optional
Custom security group rules **Type:** ```hcl list(object({ key = string type = string from_port = number to_port = number protocol = string cidr_blocks = list(string) description = string })) ``` **Default value:** `[ ]`
`database_name` (`string`) optional
The name of the first database to be created when the cluster is created **Default value:** `null`
`default_iam_role_arn` (`string`) optional
The Amazon Resource Name (ARN) of the IAM role to set as a default in the namespace **Default value:** `null`
`endpoint_name` (`string`) optional
Endpoint name for the redshift endpoint, if null, is set to $stage-$name **Default value:** `null`
`enhanced_vpc_routing` (`bool`) optional
The value that specifies whether to turn on enhanced virtual private cloud (VPC) routing, which forces Amazon Redshift Serverless to route traffic through your VPC instead of over the internet. **Default value:** `true`
`iam_roles` (`list(string)`) optional
A list of IAM roles to associate with the namespace. **Default value:** `[ ]`
`import_profile_name` (`string`) optional
AWS Profile name to use when importing a resource **Default value:** `null`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`kms_alias_name_ssm` (`string`) optional
KMS alias name for SSM **Default value:** `"alias/aws/ssm"`
`kms_key_id` (`string`) optional
The ARN of the Amazon Web Services Key Management Service key used to encrypt your data. **Default value:** `null`
`log_exports` (`set(string)`) optional
The types of logs the namespace can export. Available export types are `userlog`, `connectionlog`, and `useractivitylog`. **Default value:** `[ ]`
`publicly_accessible` (`bool`) optional
If true, the cluster can be accessed from a public network **Default value:** `false`
`security_group_ids` (`list(string)`) optional
An array of security group IDs to associate with the endpoint. **Default value:** `null`
`ssm_path_prefix` (`string`) optional
SSM path prefix (without leading or trailing slash) **Default value:** `"redshift"`
`use_private_subnets` (`bool`) optional
Whether to use private or public subnets for the Redshift cluster **Default value:** `true`
`vpc_security_group_ids` (`list(string)`) optional
An array of security group IDs to associate with the workgroup. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`endpoint_address`
The DNS address of the VPC endpoint.
`endpoint_arn`
Amazon Resource Name (ARN) of the Redshift Serverless Endpoint Access.
`endpoint_id`
The Redshift Endpoint Access Name.
`endpoint_name`
Endpoint Name.
`endpoint_port`
The port that Amazon Redshift Serverless listens on.
`endpoint_subnet_ids`
Subnets used in redshift serverless endpoint.
`endpoint_vpc_endpoint`
The VPC endpoint or the Redshift Serverless workgroup. See VPC Endpoint below.
`namespace_arn`
Amazon Resource Name (ARN) of the Redshift Serverless Namespace.
`namespace_id`
The Redshift Namespace Name.
`namespace_namespace_id`
The Redshift Namespace ID.
`workgroup_arn`
Amazon Resource Name (ARN) of the Redshift Serverless Workgroup.
`workgroup_endpoint`
The Redshift Serverless Endpoint.
`workgroup_id`
The Redshift Workgroup Name.
`workgroup_workgroup_id`
The Redshift Workgroup ID.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `redshift_sg` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_redshiftserverless_endpoint_access.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshiftserverless_endpoint_access) (resource) - [`aws_redshiftserverless_namespace.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshiftserverless_namespace) (resource) - [`aws_redshiftserverless_workgroup.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshiftserverless_workgroup) (resource) - [`aws_ssm_parameter.admin_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.admin_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.endpoint`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.admin_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.admin_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: --- ## route53-resolver-dns-firewall This component is responsible for provisioning [Route 53 Resolver DNS Firewall](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/resolver-dns-firewall.html) resources, including Route 53 Resolver DNS Firewall, domain lists, firewall rule groups, firewall rules, and logging configuration. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml # stacks/catalog/route53-resolver-dns-firewall/defaults.yaml components: terraform: route53-resolver-dns-firewall/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true firewall_fail_open: "ENABLED" query_log_enabled: true logs_bucket_component_name: "route53-resolver-dns-firewall-logs-bucket" domains_config: allowed-domains: # Concat the lists of domains passed in the `domains` field and loaded from the file `domains_file` # The file is in the `components/terraform/route53-resolver-dns-firewall/config` folder domains_file: "config/allowed_domains.txt" domains: [] blocked-domains: # Concat the lists of domains passed in the `domains` field and loaded from the file `domains_file` # The file is in the `components/terraform/route53-resolver-dns-firewall/config` folder domains_file: "config/blocked_domains.txt" domains: [] rule_groups_config: blocked-and-allowed-domains: # 'priority' must be between 100 and 9900 exclusive priority: 101 rules: allowed-domains: firewall_domain_list_name: "allowed-domains" # 'priority' must be between 100 and 9900 exclusive priority: 101 action: "ALLOW" blocked-domains: firewall_domain_list_name: "blocked-domains" # 'priority' must be between 100 and 9900 exclusive priority: 200 action: "BLOCK" block_response: "NXDOMAIN" ``` ```yaml # stacks/mixins/stage/dev.yaml import: - catalog/route53-resolver-dns-firewall/defaults components: terraform: route53-resolver-dns-firewall/example: metadata: component: route53-resolver-dns-firewall inherits: - route53-resolver-dns-firewall/defaults vars: name: route53-dns-firewall-example vpc_component_name: vpc ``` Execute the following command to provision the `route53-resolver-dns-firewall/example` component using Atmos: ```shell atmos terraform apply route53-resolver-dns-firewall/example -s ``` ## Variables ### Required Variables
`domains_config` required
Map of Route 53 Resolver DNS Firewall domain configurations **Type:** ```hcl map(object({ domains = optional(list(string)) domains_file = optional(string) })) ```
`region` (`string`) required
AWS Region
`rule_groups_config` required
Rule groups and rules configuration **Type:** ```hcl map(object({ priority = number mutation_protection = optional(string) # https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_rule rules = map(object({ action = string priority = number block_override_dns_type = optional(string) block_override_domain = optional(string) block_override_ttl = optional(number) block_response = optional(string) firewall_domain_list_name = string })) })) ```
`vpc_component_name` (`string`) required
The name of a VPC component where the Network Firewall is provisioned
### Optional Variables
`firewall_fail_open` (`string`) optional
Determines how Route 53 Resolver handles queries during failures, for example when all traffic that is sent to DNS Firewall fails to receive a reply. By default, fail open is disabled, which means the failure mode is closed. This approach favors security over availability. DNS Firewall blocks queries that it is unable to evaluate properly. If you enable this option, the failure mode is open. This approach favors availability over security. In this case, DNS Firewall allows queries to proceed if it is unable to properly evaluate them. Valid values: ENABLED, DISABLED. **Default value:** `"ENABLED"`
`logs_bucket_component_name` (`string`) optional
Flow logs bucket component name **Default value:** `null`
`query_log_config_name` (`string`) optional
Route 53 Resolver query log config name. If omitted, the name will be generated by concatenating the ID from the context with the VPC ID **Default value:** `null`
`query_log_enabled` (`bool`) optional
Flag to enable/disable Route 53 Resolver query logging **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`domains`
Route 53 Resolver DNS Firewall domain configurations
`query_log_config`
Route 53 Resolver query logging configuration
`rule_group_associations`
Route 53 Resolver DNS Firewall rule group associations
`rule_groups`
Route 53 Resolver DNS Firewall rule groups
`rules`
Route 53 Resolver DNS Firewall rules
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `logs_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `route53_resolver_dns_firewall` | 0.3.0 | [`cloudposse/route53-resolver-dns-firewall/aws`](https://registry.terraform.io/modules/cloudposse/route53-resolver-dns-firewall/aws/0.3.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## runs-on Component: `runs-on` This component provisions RunsOn for GitHub Actions self-hosted runners. After deploying this component, install the RunsOn GitHub App in your organization to enable runner registration. See the RunsOn documentation for GitHub App installation and configuration details. Compatibility: Requires RunsOn CloudFormation template version 2.8.2 or newer due to output changes. ## Usage Stack Level: Regional (`runs-on/defaults.yaml`) ```yaml components: terraform: runs-on/defaults: metadata: component: runs-on type: abstract vars: name: runs-on enabled: true capabilities: ["CAPABILITY_IAM"] on_failure: "ROLLBACK" timeout_in_minutes: 30 # template_url: https://runs-on.s3.eu-west-1.amazonaws.com/cloudformation/template.yaml # See latest version and changelog at https://runs-on.com/changelog/ template_url: https://runs-on.s3.eu-west-1.amazonaws.com/cloudformation/template-v2.8.3.yaml parameters: AppCPU: 256 AppMemory: 512 EmailAddress: developer@cloudposse.com # Environments let you run multiple Stacks in one organization and segregate resources. # If you specify an environment, then all the jobs must also specify which environment they are running in. # To keep things simple, we use the default environment ("production") and leave the `env` label unset in the workflow. EncryptEbs: true # With the default value of SSHAllowed: true, the runners that are placed in a public subnet # will allow ingress on port 22. This is highly abused (scanners running constantly looking for vulnerable SSH servers) # and should not be allowed. If you need access to the runners, use Session Manager (SSM). SSHAllowed: false LicenseKey: Private: false # always | true | false - Always will default place in private subnet, true will place in private subnet if tag `private=true` present on workflow, false will place in public subnet RunnerLargeDiskSize: 120 # Disk size in GB for disk=large runners Ec2LogRetentionInDays: 30 VpcFlowLogRetentionInDays: 14 ``` ### Embedded networking (Runs On managed VPC) When no VPC details are set, the component will create a new VPC and subnets via the CloudFormation template. Set the `VpcCidrBlock` parameter to the CIDR block of the VPC that will be created. (`runs-on.yaml`) ```yaml import: - orgs/acme/core/auto/_defaults - mixins/region/us-east-1 - catalog/runs-on/defaults components: terraform: runs-on: metadata: inherits: - runs-on/defaults component: runs-on vars: networking_stack: embedded parameters: VpcCidrBlock: 10.100.0.0/16 ``` ### External networking (Use existing VPC) Use an existing VPC by setting `vpc_id`, `subnet_ids`, and `security_group_id`. (`_defaults.yaml`) ```yaml terraform: hooks: store-outputs: name: auto/ssm ``` (`runs-on.yaml`) ```yaml import: - orgs/acme/core/auto/_defaults - mixins/region/us-east-1 - catalog/vpc/defaults - catalog/runs-on/defaults components: terraform: runs-on: metadata: inherits: - runs-on/defaults component: runs-on vars: networking_stack: external # There are other ways to get the vpc_id, subnet_ids, and security_group_id. # Hardcode # Use Atmos KV Store # Use atmos !terraform.output yaml function vpc_id: !store auto/ssm vpc vpc_id subnet_ids: !store auto/ssm vpc private_subnet_ids security_group_id: !store auto/ssm vpc default_security_group_id ```
(DEPRECATED) Configuring with Transit Gateway It's important to note that the embedded networking will require some customization to work with Transit Gateway. The following configuration assumes you are using the Cloud Posse Components for Transit Gateway ([tgw/hub](https://docs.cloudposse.com/components/library/aws/tgw/hub/) & [tgw/spoke](https://docs.cloudposse.com/components/library/aws/tgw/spoke/)). The outputs of this component contain the same outputs as the `vpc` component. This is because the runs-on cloudformation stack creates a VPC and subnets. First we need to update the TGW/Hub - this stores information about the VPCs that are allowed to be used by TGW Spokes. Assuming your TGW/Hub lives in the `core-network` account and your Runs-On is deployed to `core-auto` (`tgw-hub.yaml`) ```yaml vars: connections: - account: tenant: core stage: auto vpc_component_names: - vpc - runs-on ``` ```yaml components: terraform: tgw/hub/defaults: metadata: type: abstract component: tgw/hub vars: enabled: true name: tgw-hub tags: Team: sre Service: tgw-hub tgw/hub: metadata: inherits: - tgw/hub/defaults component: tgw/hub vars: connections: - account: tenant: core stage: network - account: tenant: core stage: auto vpc_component_names: - vpc - runs-on - account: tenant: plat stage: sandbox - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod ``` We then need to create a spoke that refers to the VPC created by Runs-On. (`tgw-spoke.yaml`) ```yaml tgw/spoke/runs-on: metadata: component: tgw/spoke inherits: - tgw/spoke-defaults vars: own_vpc_component_name: runs-on attributes: - "runs-on" connections: - account: tenant: core stage: network - account: tenant: core stage: auto vpc_component_names: - vpc - runs-on - account: tenant: plat stage: sandbox - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod ``` Finally we need to update the spokes of the TGW/Spokes to allow Runs-On traffic to the other accounts. Typically this includes `core-auto`, `core-network`, and your platform accounts. (`tgw-spoke.yaml`) ```yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: connections: # ... vpc_component_names: - vpc - runs-on # ... ```
## Variables ### Required Variables
`region` (`string`) required
AWS Region
`template_url` (`string`) required
Amazon S3 bucket URL location of a file containing the CloudFormation template body. Maximum file size: 460,800 bytes
### Optional Variables
`capabilities` (`list(string)`) optional
A list of capabilities. Valid values: CAPABILITY_IAM, CAPABILITY_NAMED_IAM, CAPABILITY_AUTO_EXPAND **Default value:** `[ ]`
`networking_stack` (`string`) optional
Let RunsOn manage your networking stack (`embedded`), or use a vpc under your control (`external`). Null will default to whatever the template used as default. If you select `external`, you will need to provide the VPC ID, the subnet IDs, and optionally the security group ID, and make sure your whole networking setup is compatible with RunsOn (see https://runs-on.com/networking/embedded-vs-external/ for more details). To get started quickly, we recommend using the 'embedded' option. **Default value:** `"embedded"`
`on_failure` (`string`) optional
Action to be taken if stack creation fails. This must be one of: `DO_NOTHING`, `ROLLBACK`, or `DELETE` **Default value:** `"ROLLBACK"`
`parameters` (`map(string)`) optional
Key-value map of input parameters for the Stack Set template. (_e.g._ map("BusinessUnit","ABC") **Default value:** `{ }`
`policy_body` (`string`) optional
Structure containing the stack policy body **Default value:** `""`
`security_group_id` (`string`) optional
Security group ID. If not set, a new security group will be created. **Default value:** `null`
`security_group_rules` optional
Security group rules. These are either added to the security passed in, or added to the security group created when var.security_group_id is not set. Types include `ingress` and `egress`. **Type:** ```hcl list(object({ type = string from_port = number to_port = number protocol = string cidr_blocks = list(string) })) ``` **Default value:** `null`
`subnet_ids` (`list(string)`) optional
Subnet IDs **Default value:** `null`
`timeout_in_minutes` (`number`) optional
The amount of time that can pass before the stack status becomes `CREATE_FAILED` **Default value:** `30`
`vpc_id` (`string`) optional
VPC ID **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
ID of the CloudFormation Stack
`name`
Name of the CloudFormation Stack
`nat_gateway_ids`
NAT Gateway IDs
`nat_instance_ids`
NAT Instance IDs
`outputs`
Outputs of the CloudFormation Stack
`private_route_table_ids`
Private subnet route table IDs
`private_subnet_ids`
Private subnet IDs
`public_subnet_ids`
Public subnet IDs
`security_group_id`
Security group ID
`vpc_cidr`
CIDR of the VPC created by RunsOn CloudFormation Stack
`vpc_id`
ID of the VPC created by RunsOn CloudFormation Stack
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudformation_stack` | 0.7.1 | [`cloudposse/cloudformation-stack/aws`](https://registry.terraform.io/modules/cloudposse/cloudformation-stack/aws/0.7.1) | n/a `iam_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | Typically when runs-on is installed, and we're using the embedded networking stack, we need a security group. This is a batties included optional feature. `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_security_group_rule.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_nat_gateways.ngws`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/nat_gateways) (data source) - [`aws_subnets.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) (data source) - [`aws_subnets.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) (data source) - [`aws_vpc.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## s3-bucket This component is responsible for provisioning S3 buckets. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/s3/defaults.yaml` file (base component for all S3 buckets with default settings): ```yaml components: terraform: s3-bucket-defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true account_map_tenant_name: core # Suggested configuration for all buckets user_enabled: false acl: "private" grants: null force_destroy: false versioning_enabled: false allow_encrypted_uploads_only: true block_public_acls: true block_public_policy: true ignore_public_acls: true restrict_public_buckets: true allow_ssl_requests_only: true lifecycle_configuration_rules: - id: default enabled: true abort_incomplete_multipart_upload_days: 90 filter_and: prefix: "" tags: {} transition: - storage_class: GLACIER days: 60 noncurrent_version_transition: - storage_class: GLACIER days: 60 noncurrent_version_expiration: days: 90 expiration: days: 120 ``` ```yaml import: - catalog/s3/defaults components: terraform: template-bucket: metadata: component: s3-bucket inherits: - s3-bucket-defaults vars: enabled: true name: template logging_bucket_name_rendering_enabled: true logging: bucket_name: s3-access-logs prefix: logs/ ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`acl` (`string`) optional
The [canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to apply. We recommend `private` to avoid exposing sensitive information. Conflicts with `grants`. **Default value:** `"private"`
`allow_encrypted_uploads_only` (`bool`) optional
Set to `true` to prevent uploads of unencrypted objects to S3 bucket **Default value:** `false`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `false`
`allowed_bucket_actions` (`list(string)`) optional
List of actions the user is permitted to perform on the S3 bucket **Default value:** ```hcl [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] ```
`block_public_acls` (`bool`) optional
Set to `false` to disable the blocking of new public access lists on the bucket **Default value:** `true`
`block_public_policy` (`bool`) optional
Set to `false` to disable the blocking of new public policies on the bucket **Default value:** `true`
`bucket_key_enabled` (`bool`) optional
Set this to true to use Amazon S3 Bucket Keys for SSE-KMS, which reduce the cost of AWS KMS requests. For more information, see: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html **Default value:** `false`
`bucket_name` (`string`) optional
Bucket name. If provided, the bucket will be created with this name instead of generating the name from the context **Default value:** `""`
`cors_configuration` optional
Specifies the allowed headers, methods, origins and exposed headers when using CORS on this bucket **Type:** ```hcl list(object({ allowed_headers = list(string) allowed_methods = list(string) allowed_origins = list(string) expose_headers = list(string) max_age_seconds = number })) ``` **Default value:** `null`
`custom_policy_account_names` (`list(string)`) optional
List of accounts names to assign as principals for the s3 bucket custom policy **Default value:** `[ ]`
`custom_policy_actions` (`list(string)`) optional
List of S3 Actions for the custom policy **Default value:** `[ ]`
`custom_policy_enabled` (`bool`) optional
Whether to enable or disable the custom policy. If enabled, the default policy will be ignored **Default value:** `false`
`force_destroy` (`bool`) optional
When `true`, permits a non-empty S3 bucket to be deleted by first deleting all objects in the bucket. THESE OBJECTS ARE NOT RECOVERABLE even if they were versioned and stored in Glacier. **Default value:** `false`
`grants` optional
A list of policy grants for the bucket, taking a list of permissions. Conflicts with `acl`. Set `acl` to `null` to use this. **Type:** ```hcl list(object({ id = string type = string permissions = list(string) uri = string })) ``` **Default value:** `[ ]`
`iam_policy_statements` (`any`) optional
Map of IAM policy statements to use in the bucket policy. **Default value:** `{ }`
`ignore_public_acls` (`bool`) optional
Set to `false` to disable the ignoring of public access lists on the bucket **Default value:** `true`
`kms_master_key_arn` (`string`) optional
The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms` **Default value:** `""`
`lifecycle_configuration_rules` optional
A list of lifecycle V2 rules **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = any expiration = any transition = list(any) noncurrent_version_expiration = any noncurrent_version_transition = list(any) })) ``` **Default value:** `[ ]`
`logging` optional
Bucket access logging configuration. **Type:** ```hcl object({ bucket_name = string prefix = string }) ``` **Default value:** `null`
`logging_bucket_name_rendering_enabled` (`bool`) optional
Whether to render the logging bucket name, prepending context **Default value:** `false`
`logging_bucket_name_rendering_template` (`string`) optional
The template for the template used to render Bucket Name for the Logging bucket. Default is appropriate when using `tenant` and default label order with `null-label`. Use `"%s-%s-%s-%%s"` when not using `tenant`. **Default value:** `"%s-%s-%s-%s-%s"`
`logging_bucket_prefix_rendering_template` (`string`) optional
The template for the template used to render Bucket Prefix for the Logging bucket, uses the format `var.logging.prefix`/`var.name` **Default value:** `"%s/%s/"`
`object_lock_configuration` optional
A configuration for S3 object locking. With S3 Object Lock, you can store objects using a `write once, read many` (WORM) model. Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. **Type:** ```hcl object({ mode = string # Valid values are GOVERNANCE and COMPLIANCE. days = number years = number }) ``` **Default value:** `null`
`privileged_principal_actions` (`list(string)`) optional
List of actions to permit `privileged_principal_arns` to perform on bucket and bucket prefixes (see `privileged_principal_arns`) **Default value:** `[ ]`
`privileged_principal_arns` (`list(map(list(string)))`) optional
List of maps. Each map has one key, an IAM Principal ARN, whose associated value is a list of S3 path prefixes to grant `privileged_principal_actions` permissions for that principal, in addition to the bucket itself, which is automatically included. Prefixes should not begin with '/'. **Default value:** `[ ]`
`restrict_public_buckets` (`bool`) optional
Set to `false` to disable the restricting of making the bucket public **Default value:** `true`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. **Default value:** `"ObjectWriter"`
`s3_replica_bucket_arn` (`string`) optional
A single S3 bucket ARN to use for all replication rules. Note: The destination bucket can be specified in the replication rule itself (which allows for multiple destinations), in which case it will take precedence over this variable. **Default value:** `""`
`s3_replication_enabled` (`bool`) optional
Set this to true and specify `s3_replication_rules` to enable replication. `versioning_enabled` must also be `true`. **Default value:** `false`
`s3_replication_rules` (`list(any)`) optional
Specifies the replication rules for S3 bucket replication if enabled. You must also set s3_replication_enabled to true. **Default value:** `null`
`s3_replication_source_roles` (`list(string)`) optional
Cross-account IAM Role ARNs that will be allowed to perform S3 replication to this bucket (for replication within the same AWS account, it's not necessary to adjust the bucket policy). **Default value:** `[ ]`
`source_policy_documents` (`list(string)`) optional
List of IAM policy documents that are merged together into the exported document. Statements defined in source_policy_documents or source_json must have unique SIDs. Statement having SIDs that match policy SIDs generated by this module will override them. **Default value:** `[ ]`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms` **Default value:** `"AES256"`
`transfer_acceleration_enabled` (`bool`) optional
Set this to true to enable S3 Transfer Acceleration for the bucket. **Default value:** `false`
`user_enabled` (`bool`) optional
Set to `true` to create an IAM user with permission to access the bucket **Default value:** `false`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
`website_inputs` optional
Specifies the static website hosting configuration object. **Type:** ```hcl list(object({ index_document = string error_document = string redirect_all_requests_to = string routing_rules = string })) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
Bucket ARN
`bucket_domain_name`
Bucket domain name
`bucket_id`
Bucket ID
`bucket_region`
Bucket region
`bucket_regional_domain_name`
Bucket region-specific domain name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `template`, version: `>= 2.2.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `template`, version: `>= 2.2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `bucket_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `s3_bucket` | 4.10.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.10.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.custom_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`template_file.bucket_policy`](https://registry.terraform.io/providers/cloudposse/template/latest/docs/data-sources/file) (data source) --- ## security-hub This component is responsible for configuring Security Hub within an AWS Organization. Amazon Security Hub enables users to centrally manage and monitor the security and compliance of their AWS accounts and resources. It aggregates, organizes, and prioritizes security findings from various AWS services, third-party tools, and integrated partner solutions. Here are the key features and capabilities of Amazon Security Hub: - Centralized security management: Security Hub provides a centralized dashboard where users can view and manage security findings from multiple AWS accounts and regions. This allows for a unified view of the security posture across the entire AWS environment. - Automated security checks: Security Hub automatically performs continuous security checks on AWS resources, configurations, and security best practices. It leverages industry standards and compliance frameworks, such as AWS CIS Foundations Benchmark, to identify potential security issues. - Integrated partner solutions: Security Hub integrates with a wide range of AWS native services, as well as third-party security products and solutions. This integration enables the ingestion and analysis of security findings from diverse sources, offering a comprehensive security view. - Security standards and compliance: Security Hub provides compliance checks against industry standards and regulatory frameworks, such as PCI DSS, HIPAA, and GDPR. It identifies non-compliant resources and provides guidance on remediation actions to ensure adherence to security best practices. - Prioritized security findings: Security Hub analyzes and prioritizes security findings based on severity, enabling users to focus on the most critical issues. It assigns severity levels and generates a consolidated view of security alerts, allowing for efficient threat response and remediation. - Custom insights and event aggregation: Security Hub supports custom insights, allowing users to create their own rules and filters to focus on specific security criteria or requirements. It also provides event aggregation and correlation capabilities to identify related security findings and potential attack patterns. - Integration with other AWS services: Security Hub seamlessly integrates with other AWS services, such as AWS CloudTrail, Amazon GuardDuty, AWS Config, and AWS IAM Access Analyzer. This integration allows for enhanced visibility, automated remediation, and streamlined security operations. - Alert notifications and automation: Security Hub supports alert notifications through Amazon SNS, enabling users to receive real-time notifications of security findings. It also facilitates automation and response through integration with AWS Lambda, allowing for automated remediation actions. By utilizing Amazon Security Hub, organizations can improve their security posture, gain insights into security risks, and effectively manage security compliance across their AWS accounts and resources. ## Usage **Stack Level**: Regional ## Deployment Overview This component is complex in that it must be deployed multiple times with different variables set to configure the AWS Organization successfully. It is further complicated by the fact that you must deploy each of the component instances described below to every region that existed before March 2019 and to any regions that have been opted-in as described in the [AWS Documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions). In the examples below, we assume that the AWS Organization Management account is `root` and the AWS Organization Delegated Administrator account is `security`, both in the `core` tenant. ### Deploy to Delegated Administrator Account First, the component is deployed to the [Delegated Administrator](https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_organizations.html) account in each region to configure the Security Hub instance to which each account will send its findings. ```yaml # core-ue1-security components: terraform: security-hub/delegated-administrator/ue1: metadata: component: security-hub vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 ``` ```bash atmos terraform apply security-hub/delegated-administrator/ue1 -s core-ue1-security atmos terraform apply security-hub/delegated-administrator/ue2 -s core-ue2-security atmos terraform apply security-hub/delegated-administrator/uw1 -s core-uw1-security # ... other regions ``` ### Deploy to Organization Management (root) Account Next, the component is deployed to the AWS Organization Management (a/k/a `root`) Account in order to set the AWS Organization Designated Administrator account. Note that `SuperAdmin` permissions must be used as we are deploying to the AWS Organization Management account. Since we are using the `SuperAdmin` user, it will already have access to the state bucket, so we set the `role_arn` of the backend config to null and set `var.privileged` to `true`. ```yaml # core-ue1-root components: terraform: security-hub/root/ue1: metadata: component: security-hub backend: s3: role_arn: null vars: enabled: true delegated_administrator_account_name: core-security environment: ue1 region: us-east-1 privileged: true ``` ```bash atmos terraform apply security-hub/root/ue1 -s core-ue1-root atmos terraform apply security-hub/root/ue2 -s core-ue2-root atmos terraform apply security-hub/root/uw1 -s core-uw1-root # ... other regions ``` ### Deploy Organization Settings in Delegated Administrator Account Finally, the component is deployed to the Delegated Administrator Account again in order to create the organization-wide Security Hub configuration for the AWS Organization, but with `var.admin_delegated` set to `true` this time to indicate that the delegation from the Organization Management account has already been performed. ```yaml # core-ue1-security components: terraform: security-hub/org-settings/ue1: metadata: component: security-hub vars: enabled: true delegated_administrator_account_name: core-security environment: use1 region: us-east-1 admin_delegated: true ``` ```bash atmos terraform apply security-hub/org-settings/ue1 -s core-ue1-security atmos terraform apply security-hub/org-settings/ue2 -s core-ue2-security atmos terraform apply security-hub/org-settings/uw1 -s core-uw1-security # ... other regions ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_tenant` (`string`) optional
The tenant where the `account_map` component required by remote-state is deployed **Default value:** `"core"`
`admin_delegated` (`bool`) optional
A flag to indicate if the AWS Organization-wide settings should be created. This can only be done after the Security Hub Administrator account has already been delegated from the AWS Org Management account (usually 'root'). See the Deployment section of the README for more information. **Default value:** `false`
`auto_enable_organization_members` (`bool`) optional
Flag to toggle auto-enablement of Security Hub for new member accounts in the organization. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_organization_configuration#auto_enable **Default value:** `true`
`cloudwatch_event_rule_pattern_detail_type` (`string`) optional
The detail-type pattern used to match events that will be sent to SNS. For more information, see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html https://docs.aws.amazon.com/eventbridge/latest/userguide/event-types.html **Default value:** `"Security Hub Findings - Imported"`
`create_sns_topic` (`bool`) optional
Flag to indicate whether an SNS topic should be created for notifications. If you want to send findings to a new SNS topic, set this to true and provide a valid configuration for subscribers. **Default value:** `false`
`default_standards_enabled` (`bool`) optional
Flag to indicate whether default standards should be enabled **Default value:** `true`
`delegated_administrator_account_name` (`string`) optional
The name of the account that is the AWS Organization Delegated Administrator account **Default value:** `"core-security"`
`enabled_standards` (`set(string)`) optional
A list of standards to enable in the account. For example: - standards/aws-foundational-security-best-practices/v/1.0.0 - ruleset/cis-aws-foundations-benchmark/v/1.2.0 - standards/pci-dss/v/3.2.1 - standards/cis-aws-foundations-benchmark/v/1.4.0 **Default value:** `[ ]`
`finding_aggregation_region` (`string`) optional
If finding aggregation is enabled, the region that collects findings **Default value:** `null`
`finding_aggregator_enabled` (`bool`) optional
Flag to indicate whether a finding aggregator should be created If you want to aggregate findings from one region, set this to `true`. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_finding_aggregator **Default value:** `false`
`finding_aggregator_linking_mode` (`string`) optional
Linking mode to use for the finding aggregator. The possible values are: - `ALL_REGIONS` - Aggregate from all regions - `ALL_REGIONS_EXCEPT_SPECIFIED` - Aggregate from all regions except those specified in `var.finding_aggregator_regions` - `SPECIFIED_REGIONS` - Aggregate from regions specified in `var.finding_aggregator_regions` **Default value:** `"ALL_REGIONS"`
`finding_aggregator_regions` (`any`) optional
A list of regions to aggregate findings from. This is only used if `finding_aggregator_enabled` is `true`. **Default value:** `null`
`findings_notification_arn` (`string`) optional
The ARN for an SNS topic to send findings notifications to. This is only used if create_sns_topic is false. If you want to send findings to an existing SNS topic, set this to the ARN of the existing topic and set create_sns_topic to false. **Default value:** `null`
`global_environment` (`string`) optional
Global environment name **Default value:** `"gbl"`
`import_profile_name` (`string`) optional
AWS Profile name to use when importing a resource **Default value:** `null`
`import_role_arn` (`string`) optional
IAM Role ARN to use when importing a resource **Default value:** `null`
`organization_management_account_name` (`string`) optional
The name of the AWS Organization management account **Default value:** `null`
`privileged` (`bool`) optional
true if the default provider already has access to the backend **Default value:** `false`
`root_account_stage` (`string`) optional
The stage name for the Organization root (management) account. This is used to lookup account IDs from account names using the `account-map` component. **Default value:** `"root"`
`subscribers` optional
A map of subscription configurations for SNS topics For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription#argument-reference protocol: The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see link) (email is an option but is unsupported in terraform, see link). endpoint: The endpoint to send data to, the contents will vary with the protocol. (see link for more information) endpoint_auto_confirms: Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty. Default is false. raw_message_delivery: Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property). Default is false. **Type:** ```hcl map(object({ protocol = string endpoint = string endpoint_auto_confirms = bool raw_message_delivery = bool })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`delegated_administrator_account_id`
The AWS Account ID of the AWS Organization delegated administrator account
`sns_topic_name`
The name of the SNS topic created by the component
`sns_topic_subscriptions`
The SNS topic subscriptions created by the component
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Providers - `aws`, version: `>= 5.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `security_hub` | 0.12.2 | [`cloudposse/security-hub/aws`](https://registry.terraform.io/modules/cloudposse/security-hub/aws/0.12.2) | If we are running in the AWS Org designated administrator account, enable Security Hub and optionally enable standards and finding aggregation `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_securityhub_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_account) (resource) - [`aws_securityhub_organization_admin_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_organization_admin_account) (resource) - [`aws_securityhub_organization_configuration.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_organization_configuration) (resource) - [`awsutils_security_hub_organization_settings.this`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/resources/security_hub_organization_settings) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## ses This component provisions Amazon Simple Email Service (SES) to act as an SMTP gateway. The credentials used for sending email can be retrieved from SSM. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: ses: settings: spacelift: workspace_enabled: true vars: enabled: true name: ses # {environment}.{stage}.{tenant}.acme.org domain_template: "%s[2]s.%[3]s.%[1]s.acme.org" # use this when `account-map` is deployed in a separate `tenant` tags: Team: sre Service: ses ``` ## Variables ### Required Variables
`domain_template` (`string`) required
The `format()` string to use to generate the base domain name for sending and receiving email with Amazon SES, `format(var.domain_template, var.tenant, var.environment, var.stage)
`region` (`string`) required
AWS Region
### Optional Variables
`dns_delegated_environment_name` (`string`) optional
`dns-delegated` component environment name **Default value:** `null`
`ses_verify_dkim` (`bool`) optional
If provided the module will create Route53 DNS records used for DKIM verification. **Default value:** `true`
`ses_verify_domain` (`bool`) optional
If provided the module will create Route53 DNS records used for domain verification. **Default value:** `true`
`ssm_prefix` (`string`) optional
The prefix to use for the SSM parameters **Default value:** `"/ses"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`domain`
The SES domain name
`smtp_password`
The SMTP password. This will be written to the state file in plain text.
`smtp_user`
Access key ID of the IAM user with permission to send emails from SES domain
`user_arn`
The ARN the IAM user with permission to send emails from SES domain
`user_name`
Normalized name of the IAM user with permission to send emails from SES domain
`user_unique_id`
The unique ID of the IAM user with permission to send emails from SES domain
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `awsutils`, version: `>= 0.11.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_gbl_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `kms_key_ses` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `ses` | 0.25.1 | [`cloudposse/ses/aws`](https://registry.terraform.io/modules/cloudposse/ses/aws/0.25.1) | n/a `ssm_parameter_store` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_ses`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## sftp This component is responsible for provisioning SFTP Endpoints. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: sftp: settings: spacelift: workspace_enabled: true vars: enabled: true ``` ## Variables ### Required Variables
`hosted_zone_suffix` (`string`) required
The hosted zone name suffix. The stage name will be prefixed to this suffix.
`region` (`string`) required
VPN Endpoints are region-specific. This identifies the region. AWS Region
`s3_bucket_context` (`any`) required
The s3 bucket context map of inputs. The same null label inputs can be provided. Provide the `name` to find the s3 bucket using a data source.
### Optional Variables
`address_allocation_ids` (`list(string)`) optional
A list of address allocation IDs that are required to attach an Elastic IP address to your SFTP server's endpoint. This property can only be used when endpoint_type is set to VPC. **Default value:** `[ ]`
`domain` (`string`) optional
Where your files are stored. S3 or EFS **Default value:** `"S3"`
`domain_name` (`string`) optional
Domain to use when connecting to the SFTP endpoint **Default value:** `""`
`eip_enabled` (`bool`) optional
Whether to provision and attach an Elastic IP to be used as the SFTP endpoint. An EIP will be provisioned per subnet. **Default value:** `false`
`force_destroy` (`bool`) optional
Forces the AWS Transfer Server to be destroyed **Default value:** `false`
`restricted_home` (`bool`) optional
Restricts SFTP users so they only have access to their home directories. **Default value:** `true`
`security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group. **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "from_port": 22, "protocol": "tcp", "to_port": 22, "type": "ingress" } ] ```
`security_policy_name` (`string`) optional
Specifies the name of the security policy that is attached to the server. Possible values are TransferSecurityPolicy-2018-11, TransferSecurityPolicy-2020-06, and TransferSecurityPolicy-FIPS-2020-06. Default value is: TransferSecurityPolicy-2018-11. **Default value:** `"TransferSecurityPolicy-2018-11"`
`sftp_users` (`any`) optional
List of SFTP usernames and public keys **Default value:** `{ }`
`vpc_endpoint_id` (`string`) optional
The ID of the VPC endpoint. This property can only be used when endpoint_type is set to VPC_ENDPOINT **Default value:** `null`
`vpc_security_group_ids` (`list(string)`) optional
A list of security groups IDs that are available to attach to your server's endpoint. If no security groups are specified, the VPC's default security groups are automatically assigned to your endpoint. This property can only be used when endpoint_type is set to VPC. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`sftp`
The SFTP module outputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` - `awsutils`, version: `>= 0.11.0, < 6.0.0` - `local`, version: `>= 2.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `s3_context` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `sftp` | 2.3.1 | [`cloudposse/transfer-sftp/aws`](https://registry.terraform.io/modules/cloudposse/transfer-sftp/aws/2.3.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## site-to-site-vpn This component provisions a [Site-To-Site VPN](https://aws.amazon.com/vpn/site-to-site-vpn/) with a target AWS VPC on one side of the tunnel. The other (customer) side can be any VPN gateway endpoint, e.g. a hardware device, other cloud VPN, etc. AWS Site-to-Site VPN is a fully-managed service that creates a secure connection between your data center or branch office and your AWS resources using IP Security (IPSec) tunnels. When using Site-to-Site VPN, you can connect to both your Amazon Virtual Private Clouds (VPC) and AWS Transit Gateway, and two tunnels per connection are used for increased redundancy. The component provisions the following resources: - AWS Virtual Private Gateway (a representation of the AWS side of the tunnel) - AWS Customer Gateway (a representation of the other (remote) side of the tunnel). It requires: - The gateway's Border Gateway Protocol (BGP) Autonomous System Number (ASN) - `/32` IP of the VPN endpoint - AWS Site-To-Site VPN connection. It creates two VPN tunnels for redundancy and requires: - The IP CIDR ranges on each side of the tunnel - Pre-shared Keys for each tunnel (can be auto-generated if not provided and saved into SSM Parameter Store) - (Optional) IP CIDR ranges to be used inside each VPN tunnel - Route table entries to direct the appropriate traffic from the local VPC to the other side of the tunnel ## Usage Stack Level: Regional Example configuration: ```yaml components: terraform: site-to-site-vpn: metadata: component: site-to-site-vpn vars: enabled: true name: "site-to-site-vpn" vpc_component_name: vpc customer_gateway_bgp_asn: 65000 customer_gateway_ip_address: 20.200.30.0 vpn_gateway_amazon_side_asn: 64512 vpn_connection_static_routes_only: true vpn_connection_tunnel1_inside_cidr: 169.254.20.0/30 vpn_connection_tunnel2_inside_cidr: 169.254.21.0/30 vpn_connection_local_ipv4_network_cidr: 10.100.128.0/24 vpn_connection_remote_ipv4_network_cidr: 10.10.80.0/24 vpn_connection_static_routes_destinations: - 10.100.128.0/24 vpn_connection_tunnel1_startup_action: add vpn_connection_tunnel2_startup_action: add transit_gateway_enabled: false vpn_connection_tunnel1_cloudwatch_log_enabled: false vpn_connection_tunnel2_cloudwatch_log_enabled: false preshared_key_enabled: true ssm_enabled: true ssm_path_prefix: "/site-to-site-vpn" ``` Provisioning: ```sh atmos terraform plan site-to-site-vpn -s atmos terraform apply site-to-site-vpn -s ``` Post-tunnel creation requirements: Once the site-to-site VPN resources are deployed, send the VPN configuration from the AWS side to the administrator of the remote side of the VPN connection. To do this: 1. Determine the infrastructure that will be used for the remote side, specifically vendor, platform, software version, and IKE version. 2. Log into the target AWS account and open the VPC console. 3. Navigate to `Virtual Private Network` > `Site-to-Site VPN Connections`. 4. Select the VPN connection that was created via this component. 5. Click `Download Configuration` (top right). 6. Enter the information you obtained and click `Download`. 7. Send the configuration file to the administrator of the remote side of the tunnel. Amazon side Autonomous System Number (ASN): The variable `vpn_gateway_amazon_side_asn` (Amazon side ASN) is not strictly required when creating an AWS VPN Gateway. If you do not specify it during creation, AWS will automatically assign a default ASN (7224 for the Amazon side). Specifying the Amazon side ASN can be important if you integrate the VPN with an on-premises network that uses BGP and you want to avoid ASN conflicts or require a specific ASN for routing policies. If your use case involves BGP peering and you need a specific ASN for the Amazon side, explicitly set `vpn_gateway_amazon_side_asn`. Otherwise, it can be omitted (set to `null`) and AWS will handle it automatically. ## Variables ### Required Variables
`customer_gateway_bgp_asn` (`number`) required
The Customer Gateway's Border Gateway Protocol (BGP) Autonomous System Number (ASN)
`region` (`string`) required
AWS Region
### Optional Variables
`customer_gateway_ip_address` (`string`) optional
The IPv4 address for the Customer Gateway device's outside interface. Set to `null` to not create the Customer Gateway **Default value:** `null`
`existing_transit_gateway_id` (`string`) optional
Existing Transit Gateway ID. If provided, the module will not create a Virtual Private Gateway but instead will use the transit_gateway. For setting up transit gateway we can use the cloudposse/transit-gateway/aws module and pass the output transit_gateway_id to this variable **Default value:** `""`
`preshared_key_enabled` (`bool`) optional
Flag to enable adding the preshared keys to the VPN connection **Default value:** `true`
`ssm_enabled` (`bool`) optional
Flag to enable saving the `tunnel1_preshared_key` and `tunnel2_preshared_key` in the SSM Parameter Store **Default value:** `false`
`ssm_path_prefix` (`string`) optional
SSM Key path prefix for the associated SSM parameters **Default value:** `""`
`transit_gateway_enabled` (`bool`) optional
Set to true to enable VPN connection to transit gateway and then pass in the existing_transit_gateway_id **Default value:** `false`
`transit_gateway_route_table_id` (`string`) optional
The ID of the route table for the transit gateway that you want to associate + propagate the VPN connection's TGW attachment **Default value:** `null`
`transit_gateway_routes` optional
A map of transit gateway routes to create on the given TGW route table (via `transit_gateway_route_table_id`) for the created VPN Attachment. Use the key in the map to describe the route **Type:** ```hcl map(object({ blackhole = optional(bool, false) destination_cidr_block = string })) ``` **Default value:** `{ }`
`vpc_component_name` (`string`) optional
Atmos VPC component name **Default value:** `"vpc"`
`vpn_connection_local_ipv4_network_cidr` (`string`) optional
The IPv4 CIDR on the Customer Gateway (on-premises) side of the VPN connection **Default value:** `"0.0.0.0/0"`
`vpn_connection_log_retention_in_days` (`number`) optional
Specifies the number of days you want to retain log events **Default value:** `30`
`vpn_connection_remote_ipv4_network_cidr` (`string`) optional
The IPv4 CIDR on the AWS side of the VPN connection **Default value:** `"0.0.0.0/0"`
`vpn_connection_static_routes_destinations` (`list(string)`) optional
List of CIDR blocks to be used as destination for static routes. Routes to destinations will be propagated to the VPC route tables **Default value:** `[ ]`
`vpn_connection_static_routes_only` (`bool`) optional
If set to `true`, the VPN connection will use static routes exclusively. Static routes must be used for devices that don't support BGP **Default value:** `false`
`vpn_connection_tunnel1_cloudwatch_log_enabled` (`bool`) optional
Enable or disable VPN tunnel logging feature for the tunnel **Default value:** `false`
`vpn_connection_tunnel1_cloudwatch_log_output_format` (`string`) optional
Set log format for the tunnel. Default format is json. Possible values are `json` and `text` **Default value:** `"json"`
`vpn_connection_tunnel1_dpd_timeout_action` (`string`) optional
The action to take after DPD timeout occurs for the first VPN tunnel. Specify restart to restart the IKE initiation. Specify `clear` to end the IKE session. Valid values are `clear` | `none` | `restart` **Default value:** `"clear"`
`vpn_connection_tunnel1_ike_versions` (`list(string)`) optional
The IKE versions that are permitted for the first VPN tunnel. Valid values are ikev1 | ikev2 **Default value:** `[ ]`
`vpn_connection_tunnel1_inside_cidr` (`string`) optional
The CIDR block of the inside IP addresses for the first VPN tunnel **Default value:** `null`
`vpn_connection_tunnel1_phase1_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase1_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase1_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 2 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the first VPN tunnel for phase 2 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the first VPN tunnel for phase 2 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel1_preshared_key` (`string`) optional
The preshared key of the first VPN tunnel. The preshared key must be between 8 and 64 characters in length and cannot start with zero. Allowed characters are alphanumeric characters, periods(.) and underscores(_) **Default value:** `""`
`vpn_connection_tunnel1_startup_action` (`string`) optional
The action to take when the establishing the tunnel for the first VPN connection. By default, your customer gateway device must initiate the IKE negotiation and bring up the tunnel. Specify `start` for AWS to initiate the IKE negotiation. Valid values are `add` | `start` **Default value:** `"add"`
`vpn_connection_tunnel2_cloudwatch_log_enabled` (`bool`) optional
Enable or disable VPN tunnel logging feature for the tunnel **Default value:** `false`
`vpn_connection_tunnel2_cloudwatch_log_output_format` (`string`) optional
Set log format for the tunnel. Default format is json. Possible values are `json` and `text` **Default value:** `"json"`
`vpn_connection_tunnel2_dpd_timeout_action` (`string`) optional
The action to take after DPD timeout occurs for the second VPN tunnel. Specify restart to restart the IKE initiation. Specify clear to end the IKE session. Valid values are `clear` | `none` | `restart` **Default value:** `"clear"`
`vpn_connection_tunnel2_ike_versions` (`list(string)`) optional
The IKE versions that are permitted for the second VPN tunnel. Valid values are ikev1 | ikev2 **Default value:** `[ ]`
`vpn_connection_tunnel2_inside_cidr` (`string`) optional
The CIDR block of the inside IP addresses for the second VPN tunnel **Default value:** `null`
`vpn_connection_tunnel2_phase1_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the second VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase1_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the second VPN tunnel for phase 1 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase1_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the second VPN tunnel for phase 1 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the second VPN tunnel for phase 2 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the second VPN tunnel for phase 2 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the second VPN tunnel for phase 2 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel2_preshared_key` (`string`) optional
The preshared key of the second VPN tunnel. The preshared key must be between 8 and 64 characters in length and cannot start with zero. Allowed characters are alphanumeric characters, periods(.) and underscores(_) **Default value:** `""`
`vpn_connection_tunnel2_startup_action` (`string`) optional
The action to take when the establishing the tunnel for the second VPN connection. By default, your customer gateway device must initiate the IKE negotiation and bring up the tunnel. Specify `start` for AWS to initiate the IKE negotiation. Valid values are `add` | `start` **Default value:** `"add"`
`vpn_gateway_amazon_side_asn` (`number`) optional
The Autonomous System Number (ASN) for the Amazon side of the VPN Gateway. If you don't specify an ASN, the Virtual Private Gateway is created with the default ASN **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`customer_gateway_id`
Customer Gateway ID
`vpn_connection_customer_gateway_configuration`
The configuration information for the VPN connection's Customer Gateway (in the native XML format)
`vpn_connection_id`
VPN Connection ID
`vpn_connection_tunnel1_address`
The public IP address of the first VPN tunnel
`vpn_connection_tunnel1_cgw_inside_address`
The RFC 6890 link-local address of the first VPN tunnel (Customer Gateway side)
`vpn_connection_tunnel1_vgw_inside_address`
The RFC 6890 link-local address of the first VPN tunnel (Virtual Private Gateway side)
`vpn_connection_tunnel2_address`
The public IP address of the second VPN tunnel
`vpn_connection_tunnel2_cgw_inside_address`
The RFC 6890 link-local address of the second VPN tunnel (Customer Gateway side)
`vpn_connection_tunnel2_vgw_inside_address`
The RFC 6890 link-local address of the second VPN tunnel (Virtual Private Gateway side)
`vpn_gateway_id`
Virtual Private Gateway ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.2` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `vpn_connection` | 1.8.1 | [`cloudposse/vpn-connection/aws`](https://registry.terraform.io/modules/cloudposse/vpn-connection/aws/1.8.1) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.tunnel1_preshared_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.tunnel2_preshared_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.tunnel1_preshared_key`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_password.tunnel2_preshared_key`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: --- ## snowflake-account This component sets up the requirements for all other Snowflake components, including creating the Terraform service user. Before running this component, follow the manual, Click-Ops steps below to create a Snowflake subscription. ## Deployment Steps 1. Open the AWS Console for the given stack. 2. Go to AWS Marketplace Subscriptions. 3. Click "Manage Subscriptions", click "Discover products", type "Snowflake" in the search bar. 4. Select "Snowflake Data Cloud" 5. Click "Continue to Subscribe" 6. Fill out the information steps using the following as an example. Note, the provided email cannot use labels such as `mdev+sbx01@example.com`. ``` First Name: John Last Name: Smith Email: aws@example.com Company: Example Country: United States ``` 7. Select "Standard" and the current region. In this example, we chose "US East (Ohio)" which is the same as `us-east-1`. 8. Continue and wait for Sign Up to complete. Note the Snowflake account ID; you can find this in the newly accessible Snowflake console in the top right of the window. 9. Check for the Account Activation email. Note, this may be collected in a Slack notifications channel for easy access. 10. Follow the given link to create the Admin user with username `admin` and a strong password. Be sure to save that password somewhere secure. 11. Upload that password to AWS Parameter Store under `/snowflake/$ACCOUNT/users/admin/password`, where `ACCOUNT` is the value given during the subscription process. This password will only be used to create a private key, and all other authentication will be done with said key. Below is an example of how to do that with a [chamber](https://github.com/segmentio/chamber) command: ``` AWS_PROFILE=$NAMESPACE-$TENANT-gbl-sbx01-admin chamber write /snowflake/$ACCOUNT/users/admin/ admin $PASSWORD ``` 11. Finally, use atmos to deploy this component: ``` atmos terraform deploy snowflake/account --stack $TENANT-use2-sbx01 ``` ## Migrate `chanzuckerberg/snowflake` to `snowflakedb/snowflake` provider 5/25/2022 the provider has been transferred from the Chan Zuckerberg Initiative (CZI) GitHub organization to snowflakedb org. To upgrade from CZI, please run the following command: ```shell terraform state replace-provider chanzuckerberg/snowflake snowflakedb/snowflake ``` ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component: ```yaml components: terraform: snowflake-account: settings: spacelift: workspace_enabled: false vars: enabled: true snowflake_account: "AB12345" snowflake_account_region: "us-east-2" snowflake_user_email_format: "aws.dev+%s@example.com" tags: Team: data Service: snowflake ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`snowflake_account` (`string`) required
The Snowflake account given with the AWS Marketplace Subscription.
`snowflake_account_region` (`string`) required
AWS Region with the Snowflake subscription
### Optional Variables
`default_warehouse_size` (`string`) optional
The size for the default Snowflake Warehouse **Default value:** `"xsmall"`
`global_environment_name` (`string`) optional
Global environment name **Default value:** `"gbl"`
`privileged` (`bool`) optional
True if the default provider already has access to the backend **Default value:** `false`
`required_tags` (`list(string)`) optional
List of required tag names **Default value:** `[ ]`
`root_account_stage_name` (`string`) optional
The stage name for the AWS Organization root (master) account **Default value:** `"root"`
`service_user_id` (`string`) optional
The identifier for the service user created to manage infrastructure. **Default value:** `"terraform"`
`snowflake_admin_username` (`string`) optional
Snowflake admin username created with the initial account subscription. **Default value:** `"admin"`
`snowflake_role_description` (`string`) optional
Comment to attach to the Snowflake Role. **Default value:** `"Terraform service user role."`
`snowflake_username_format` (`string`) optional
Snowflake username format **Default value:** `"%s-%s"`
`ssm_path_snowflake_user_format` (`string`) optional
SSM parameter path format for a Snowflake user. For example, /snowflake/\{\{ account \}\}/users/\{\{ username \}\}/ **Default value:** `"/%s/%s/%s/%s/%s"`
`terraform_user_first_name` (`string`) optional
Snowflake Terraform first name given with User creation **Default value:** `"Terrafrom"`
`terraform_user_last_name` (`string`) optional
Snowflake Terraform last name given with User creation **Default value:** `"User"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`snowflake_account`
The Snowflake account ID.
`snowflake_region`
The AWS Region with the Snowflake account.
`snowflake_terraform_role`
The name of the role given to the Terraform service user.
`ssm_path_terraform_user_name`
The path to the SSM parameter for the Terraform user name.
`ssm_path_terraform_user_private_key`
The path to the SSM parameter for the Terraform user private key.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0, < 6.0.0` - `random`, version: `>= 2.3` - `snowflake`, version: `>= 0.25` - `tls`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` - `random`, version: `>= 2.3` - `snowflake`, version: `>= 0.25` - `tls`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `introspection` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | introspection module will contain the additional tags `snowflake_account` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `snowflake_role` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | The identifier must start with an alphabetic character and cannot contain spaces or special characters unless the entire identifier string is enclosed in double quotes (e.g. "My object"). Identifiers enclosed in double quotes are also case-sensitive. `snowflake_warehouse` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | Identifier for the virtual warehouse; must be unique for your account. In addition, the identifier must start with an alphabetic character and cannot contain spaces or special characters unless the entire identifier string is enclosed in double quotes (e.g. "My object" ). `ssm_parameters` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a ## Resources The following resources are used by this module: - [`random_password.terraform_user_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`snowflake_role.terraform`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/role) (resource) - [`snowflake_role_grants.grant_custom_roles`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/role_grants) (resource) - [`snowflake_role_grants.grant_system_roles`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/role_grants) (resource) - [`snowflake_user.terraform`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/user) (resource) - [`snowflake_warehouse.default`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/warehouse) (resource) - [`tls_private_key.terraform_user_key`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.snowflake_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## snowflake-database All data in Snowflake is stored in database tables, logically structured as collections of columns and rows. This component will create and control a Snowflake database, schema, and set of tables. ## Migrate `chanzuckerberg/snowflake` to `snowflakedb/snowflake` provider 5/25/2022 the provider has been transferred from the Chan Zuckerberg Initiative (CZI) GitHub organization to snowflakedb org. To upgrade from CZI, please run the following command: ```shell terraform state replace-provider chanzuckerberg/snowflake snowflakedb/snowflake ``` ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component: ```yaml components: terraform: snowflake-database: vars: enabled: true tags: Team: data Service: snowflake tables: example: comment: "An example table" columns: - name: "data" type: "text" - name: "DATE" type: "TIMESTAMP_NTZ(9)" - name: "extra" type: "VARIANT" comment: "extra data" primary_key: name: "pk" keys: - "data" views: select-example: comment: "An example view" statement: | select * from "example"; ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`data_retention_time_in_days` (`string`) optional
Time in days to retain data in Snowflake databases, schemas, and tables by default. **Default value:** `1`
`database_comment` (`string`) optional
The comment to give to the provisioned database. **Default value:** `"A database created for managing programmatically created Snowflake schemas and tables."`
`database_grants` (`list(string)`) optional
A list of Grants to give to the database created with component. **Default value:** ```hcl [ "MODIFY", "MONITOR", "USAGE" ] ```
`required_tags` (`list(string)`) optional
List of required tag names **Default value:** `[ ]`
`schema_grants` (`list(string)`) optional
A list of Grants to give to the schema created with component. **Default value:** ```hcl [ "MODIFY", "MONITOR", "USAGE", "CREATE TABLE", "CREATE VIEW" ] ```
`table_grants` (`list(string)`) optional
A list of Grants to give to the tables created with component. **Default value:** ```hcl [ "SELECT", "INSERT", "UPDATE", "DELETE", "TRUNCATE", "REFERENCES" ] ```
`tables` (`map(any)`) optional
A map of tables to create for Snowflake. A schema and database will be assigned for this group of tables. **Default value:** `{ }`
`view_grants` (`list(string)`) optional
A list of Grants to give to the views created with component. **Default value:** ```hcl [ "SELECT", "REFERENCES" ] ```
`views` (`map(any)`) optional
A map of views to create for Snowflake. The same schema and database will be assigned as for tables. **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0, < 6.0.0` - `snowflake`, version: `>= 0.25` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` - `snowflake`, version: `>= 0.25` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `introspection` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | introspection module will contain the additional tags `snowflake_account` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `snowflake_database` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `snowflake_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | Create a standard label to define resource name for Snowflake best practice. `snowflake_schema` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `snowflake_sequence` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a ## Resources The following resources are used by this module: - [`snowflake_database.this`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/database) (resource) - [`snowflake_database_grant.grant`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/database_grant) (resource) - [`snowflake_schema.this`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/schema) (resource) - [`snowflake_schema_grant.grant`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/schema_grant) (resource) - [`snowflake_sequence.this`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/sequence) (resource) - [`snowflake_table.tables`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/table) (resource) - [`snowflake_table_grant.grant`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/table_grant) (resource) - [`snowflake_view.view`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/view) (resource) - [`snowflake_view_grant.grant`](https://registry.terraform.io/providers/snowflakedb/snowflake/latest/docs/resources/view_grant) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.snowflake_private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.snowflake_username`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## sns-topic This component is responsible for provisioning an SNS topic. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/catalog/sns-topic/defaults.yaml` file (base component for all SNS topics with default settings): ```yaml components: terraform: sns-topic/defaults: metadata: type: abstract settings: spacelift: workspace_enabled: true vars: enabled: true tags: Team: sre Service: sns-topic subscribers: {} allowed_aws_services_for_sns_published: [] kms_master_key_id: alias/aws/sns encryption_enabled: true sqs_queue_kms_master_key_id: alias/aws/sqs sqs_queue_kms_data_key_reuse_period_seconds: 300 allowed_iam_arns_for_sns_publish: [] sns_topic_policy_json: "" sqs_dlq_enabled: false sqs_dlq_max_message_size: 262144 sqs_dlq_message_retention_seconds: 1209600 delivery_policy: null fifo_topic: false fifo_queue_enabled: false content_based_deduplication: false redrive_policy_max_receiver_count: 5 redrive_policy: null ``` ```yaml import: - catalog/sns-topic/defaults components: terraform: sns-topic-example: metadata: component: sns-topic inherits: - sns-topic/defaults vars: enabled: true name: sns-topic-example sqs_dlq_enabled: false subscribers: opsgenie: protocol: "https" endpoint: "https://api.example.com/v1/" endpoint_auto_confirms: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`allowed_aws_services_for_sns_published` (`list(string)`) optional
AWS services that will have permission to publish to SNS topic. Used when no external JSON policy is used **Default value:** `[ ]`
`allowed_iam_arns_for_sns_publish` (`list(string)`) optional
IAM role/user ARNs that will have permission to publish to SNS topic. Used when no external json policy is used. **Default value:** `[ ]`
`content_based_deduplication` (`bool`) optional
Enable content-based deduplication for FIFO topics **Default value:** `false`
`delivery_policy` (`string`) optional
The SNS delivery policy as JSON. **Default value:** `null`
`encryption_enabled` (`bool`) optional
Whether or not to use encryption for SNS Topic. If set to `true` and no custom value for KMS key (kms_master_key_id) is provided, it uses the default `alias/aws/sns` KMS key. **Default value:** `true`
`fifo_queue_enabled` (`bool`) optional
Whether or not to create a FIFO (first-in-first-out) queue **Default value:** `false`
`fifo_topic` (`bool`) optional
Whether or not to create a FIFO (first-in-first-out) topic **Default value:** `false`
`kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SNS or a custom CMK. **Default value:** `"alias/aws/sns"`
`redrive_policy` (`string`) optional
The SNS redrive policy as JSON. This overrides `var.redrive_policy_max_receiver_count` and the `deadLetterTargetArn` (supplied by `var.fifo_queue = true`) passed in by the module. **Default value:** `null`
`redrive_policy_max_receiver_count` (`number`) optional
The number of times a message is delivered to the source queue before being moved to the dead-letter queue. When the ReceiveCount for a message exceeds the maxReceiveCount for a queue, Amazon SQS moves the message to the dead-letter-queue. **Default value:** `5`
`sns_topic_policy_json` (`string`) optional
The fully-formed AWS policy as JSON **Default value:** `""`
`sqs_dlq_enabled` (`bool`) optional
Enable delivery of failed notifications to SQS and monitor messages in queue. **Default value:** `false`
`sqs_dlq_max_message_size` (`number`) optional
The limit of how many bytes a message can contain before Amazon SQS rejects it. An integer from 1024 bytes (1 KiB) up to 262144 bytes (256 KiB). The default for this attribute is 262144 (256 KiB). **Default value:** `262144`
`sqs_dlq_message_retention_seconds` (`number`) optional
The number of seconds Amazon SQS retains a message. Integer representing seconds, from 60 (1 minute) to 1209600 (14 days). **Default value:** `1209600`
`sqs_queue_kms_data_key_reuse_period_seconds` (`number`) optional
The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again **Default value:** `300`
`sqs_queue_kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SQS Queue or a custom CMK **Default value:** `"alias/aws/sqs"`
`subscribers` optional
Required configuration for subscribes to SNS topic. **Type:** ```hcl map(object({ protocol = string # The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see below) (email is an option but is unsupported, see below). endpoint = string # The endpoint to send data to, the contents will vary with the protocol. (see below for more information) endpoint_auto_confirms = optional(bool) # Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty (default is false) raw_message_delivery = optional(bool) # Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property) (default is false) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dead_letter_queue_arn`
The ARN of the dead letter queue.
`dead_letter_queue_id`
The ID for the created dead letter queue. Same as the URL.
`dead_letter_queue_name`
The name for the created dead letter queue.
`dead_letter_queue_url`
The URL for the created dead letter SQS queue.
`sns_topic_arn`
SNS topic ARN.
`sns_topic_id`
SNS topic ID.
`sns_topic_name`
SNS topic name.
`sns_topic_owner`
SNS topic owner.
`sns_topic_subscriptions`
SNS topic subscription.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `sns_topic` | 0.21.0 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.21.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## spa-s3-cloudfront This component provisions infrastructure to serve a Single Page Application (SPA) via Amazon S3 and Amazon CloudFront. - S3 bucket to host SPA assets - CloudFront distribution for global CDN delivery - ACM certificate issued in `us-east-1` (required by CloudFront) NOTE: The component does not use the ACM created by `dns-delegated`, because the ACM region has to be `us-east-1`. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: An import for all instantiations of the `spa-s3-cloudfront` component can be created at `stacks/spa/spa-defaults.yaml`: ```yaml components: terraform: spa-s3-cloudfront: vars: # lookup GitHub Runner IAM role via remote state github_runners_deployment_principal_arn_enabled: true github_runners_component_name: github-runners github_runners_tenant_name: core github_runners_environment_name: ue2 github_runners_stage_name: auto origin_force_destroy: false origin_versioning_enabled: true origin_block_public_acls: true origin_block_public_policy: true origin_ignore_public_acls: true origin_restrict_public_buckets: true origin_encryption_enabled: true cloudfront_index_document: index.html cloudfront_ipv6_enabled: false cloudfront_compress: true cloudfront_default_root_object: index.html cloudfront_viewer_protocol_policy: redirect-to-https ``` An import for all instantiations for a specific SPA can be created at `stacks/spa/example-spa.yaml`: ```yaml components: terraform: example-spa: component: spa-s3-cloudfront vars: name: example-spa site_subdomain: example-spa cloudfront_allowed_methods: - GET - HEAD cloudfront_cached_methods: - GET - HEAD cloudfront_custom_error_response: - error_caching_min_ttl: 1 error_code: 403 response_code: 200 response_page_path: /index.html cloudfront_default_ttl: 60 cloudfront_min_ttl: 60 cloudfront_max_ttl: 60 ``` Finally, the `spa-s3-cloudfront` component can be instantiated in a stack config: ```yaml import: - spa/example-spa components: terraform: example-spa: component: spa-s3-cloudfront settings: spacelift: workspace_enabled: true vars: {} ``` ### Failover Origins Failover origins are supported via `var.failover_s3_origin_name` and `var.failover_s3_origin_region`. ### Preview Environments SPA Preview environments (i.e. `subdomain.example.com` mapping to a `/subdomain` path in the S3 bucket) powered by Lambda@Edge are supported via `var.preview_environment_enabled`. See the both the variable description and inline documentation for an extensive explanation for how these preview environments work. ### Customizing Lambda@Edge This component supports customizing Lambda@Edge functions for the CloudFront distribution. All Lambda@Edge function configuration is deep merged before being passed to the `cloudposse/cloudfront-s3-cdn/aws//modules/lambda@edge` module. You can add additional functions and overwrite existing functions as such: ```yaml import: - catalog/spa-s3-cloudfront/defaults components: terraform: refarch-docs-site-spa: metadata: component: spa-s3-cloudfront inherits: - spa-s3-cloudfront-defaults vars: enabled: true lambda_edge_functions: viewer_request: # overwrite existing function source: null # this overwrites the 404 viewer request source with deep merging source_zip: "./dist/lambda_edge_paywall_viewer_request.zip" runtime: "nodejs16.x" handler: "index.handler" event_type: "viewer-request" include_body: false viewer_response: # new function source_zip: "./dist/lambda_edge_paywall_viewer_response.zip" runtime: "nodejs16.x" handler: "index.handler" event_type: "viewer-response" include_body: false ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region.
### Optional Variables
`block_origin_public_access_enabled` (`bool`) optional
When set to 'true' the s3 origin bucket will have public access block enabled. **Default value:** `true`
`cloudfront_access_log_bucket_name` (`string`) optional
When `cloudfront_access_log_create_bucket` is `false`, this is the name of the existing S3 Bucket where CloudFront Access Logs are to be delivered and is required. IGNORED when `cloudfront_access_log_create_bucket` is `true`. **Default value:** `""`
`cloudfront_access_log_bucket_name_rendering_enabled` (`bool`) optional
If set to `true`, then the CloudFront origin access logs bucket name will be rendered by calling `format("%v-%v-%v-%v", var.namespace, var.environment, var.stage, var.cloudfront_access_log_bucket_name)`. Otherwise, the value for `cloudfront_access_log_bucket_name` will need to be the globally unique name of the access logs bucket. For example, if this component produces an origin bucket named `eg-ue1-devplatform-example` and `cloudfront_access_log_bucket_name` is set to `example-cloudfront-access-logs`, then the bucket name will be rendered to be `eg-ue1-devplatform-example-cloudfront-access-logs`. **Default value:** `false`
`cloudfront_access_log_create_bucket` (`bool`) optional
When `true` and `cloudfront_access_logging_enabled` is also true, this module will create a new, separate S3 bucket to receive CloudFront Access Logs. **Default value:** `true`
`cloudfront_access_log_prefix` (`string`) optional
Prefix to use for CloudFront Access Log object keys. Defaults to no prefix. **Default value:** `""`
`cloudfront_access_log_prefix_rendering_enabled` (`bool`) optional
Whether or not to dynamically render $\{module.this.id\} at the end of `var.cloudfront_access_log_prefix`. **Default value:** `false`
`cloudfront_allowed_methods` (`list(string)`) optional
List of allowed methods (e.g. GET, PUT, POST, DELETE, HEAD) for AWS CloudFront. **Default value:** ```hcl [ "DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT" ] ```
`cloudfront_aws_shield_protection_enabled` (`bool`) optional
Enable or disable AWS Shield Advanced protection for the CloudFront distribution. If set to 'true', a subscription to AWS Shield Advanced must exist in this account. **Default value:** `false`
`cloudfront_aws_waf_component_name` (`string`) optional
The name of the component used when deploying WAF ACL **Default value:** `"waf"`
`cloudfront_aws_waf_environment` (`string`) optional
The environment where the WAF ACL for CloudFront distribution exists. **Default value:** `null`
`cloudfront_aws_waf_protection_enabled` (`bool`) optional
Enable or disable AWS WAF for the CloudFront distribution. This assumes that the `aws-waf-acl-default-cloudfront` component has been deployed to the regional stack corresponding to `var.waf_acl_environment`. **Default value:** `true`
`cloudfront_cached_methods` (`list(string)`) optional
List of cached methods (e.g. GET, PUT, POST, DELETE, HEAD). **Default value:** ```hcl [ "GET", "HEAD" ] ```
`cloudfront_compress` (`bool`) optional
Compress content for web requests that include Accept-Encoding: gzip in the request header. **Default value:** `false`
`cloudfront_custom_error_response` optional
List of one or more custom error response element maps. **Type:** ```hcl list(object({ error_caching_min_ttl = optional(string, "10") error_code = string response_code = string response_page_path = string })) ``` **Default value:** `[ ]`
`cloudfront_default_root_object` (`string`) optional
Object that CloudFront return when requests the root URL. **Default value:** `"index.html"`
`cloudfront_default_ttl` (`number`) optional
Default amount of time (in seconds) that an object is in a CloudFront cache. **Default value:** `60`
`cloudfront_index_document` (`string`) optional
Amazon S3 returns this index document when requests are made to the root domain or any of the subfolders. **Default value:** `"index.html"`
`cloudfront_ipv6_enabled` (`bool`) optional
Set to true to enable an AAAA DNS record to be set as well as the A record. **Default value:** `true`
`cloudfront_lambda_function_association` optional
A config block that configures the CloudFront distribution with lambda@edge functions for specific events. **Type:** ```hcl list(object({ event_type = string include_body = bool lambda_arn = string })) ``` **Default value:** `[ ]`
`cloudfront_max_ttl` (`number`) optional
Maximum amount of time (in seconds) that an object is in a CloudFront cache. **Default value:** `31536000`
`cloudfront_min_ttl` (`number`) optional
Minimum amount of time that you want objects to stay in CloudFront caches. **Default value:** `0`
`cloudfront_viewer_protocol_policy` (`string`) optional
Limit the protocol users can use to access content. One of `allow-all`, `https-only`, or `redirect-to-https`. **Default value:** `"redirect-to-https"`
`comment` (`string`) optional
Any comments you want to include about the distribution. **Default value:** `"Managed by Terraform"`
`custom_origins` optional
A list of additional custom website [origins](https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#origin-arguments) for this distribution. **Type:** ```hcl list(object({ domain_name = string origin_id = string origin_path = string custom_headers = list(object({ name = string value = string })) custom_origin_config = object({ http_port = number https_port = number origin_protocol_policy = string origin_ssl_protocols = list(string) origin_keepalive_timeout = number origin_read_timeout = number }) })) ``` **Default value:** `[ ]`
`dns_delegated_environment_name` (`string`) optional
The environment where `dns-delegated` component is deployed to **Default value:** `"gbl"`
`external_aliases` (`list(string)`) optional
List of FQDN's - Used to set the Alternate Domain Names (CNAMEs) setting on CloudFront. No new Route53 records will be created for these. Setting `process_domain_validation_options` to true may cause the component to fail if an external_alias DNS zone is not controlled by Terraform. Setting `preview_environment_enabled` to `true` will cause this variable to be ignored. **Default value:** `[ ]`
`failover_criteria_status_codes` (`list(string)`) optional
List of HTTP Status Codes to use as the origin group failover criteria. **Default value:** ```hcl [ 403, 404, 500, 502 ] ```
`failover_s3_origin_environment` (`string`) optional
The [fixed name](https://github.com/cloudposse/terraform-aws-utils/blob/399951e552483a4f4c1dc7fbe2675c443f3dbd83/main.tf#L10) of the AWS Region where the failover S3 origin exists. Setting this variable will enable use of a failover S3 origin, but it is required for the failover S3 origin to exist beforehand. This variable is used in conjunction with `var.failover_s3_origin_format` to build out the name of the Failover S3 origin in the specified region. For example, if this component creates an origin of name `eg-ue1-devplatform-example` and this variable is set to `uw1`, then it is expected that a bucket with the name `eg-uw1-devplatform-example-failover` exists in `us-west-1`. **Default value:** `null`
`failover_s3_origin_format` (`string`) optional
If `var.failover_s3_origin_environment` is supplied, this is the format to use for the failover S3 origin bucket name when building the name via `format([format], var.namespace, var.failover_s3_origin_environment, var.stage, var.name)` and then looking it up via the `aws_s3_bucket` Data Source. For example, if this component creates an origin of name `eg-ue1-devplatform-example` and `var.failover_s3_origin_environment` is set to `uw1`, then it is expected that a bucket with the name `eg-uw1-devplatform-example-failover` exists in `us-west-1`. **Default value:** `"%v-%v-%v-%v-failover"`
`forward_cookies` (`string`) optional
Specifies whether you want CloudFront to forward all or no cookies to the origin. Can be 'all' or 'none' **Default value:** `"none"`
`forward_header_values` (`list(string)`) optional
A list of whitelisted header values to forward to the origin (incompatible with `cache_policy_id`) **Default value:** ```hcl [ "Access-Control-Request-Headers", "Access-Control-Request-Method", "Origin" ] ```
`github_actions_allowed_repos` (`list(string)`) optional
A list of the GitHub repositories that are allowed to assume this role from GitHub Actions. For example, ["cloudposse/infra-live"]. Can contain "*" as wildcard. If org part of repo name is omitted, "cloudposse" will be assumed. **Default value:** `[ ]`
`github_actions_iam_role_attributes` (`list(string)`) optional
Additional attributes to add to the role name **Default value:** `[ ]`
`github_actions_iam_role_enabled` (`bool`) optional
Flag to toggle creation of an IAM Role that GitHub Actions can assume to access AWS resources **Default value:** `false`
`github_runners_component_name` (`string`) optional
The name of the component that deploys GitHub Runners, used in remote-state lookup **Default value:** `"github-runners"`
`github_runners_deployment_principal_arn_enabled` (`bool`) optional
A flag that is used to decide whether or not to include the GitHub Runner's IAM role in origin_deployment_principal_arns list **Default value:** `true`
`github_runners_environment_name` (`string`) optional
The name of the environment where the CloudTrail bucket is provisioned **Default value:** `"ue2"`
`github_runners_stage_name` (`string`) optional
The stage name where the CloudTrail bucket is provisioned **Default value:** `"auto"`
`github_runners_tenant_name` (`string`) optional
The tenant name where the GitHub Runners are provisioned **Default value:** `null`
`http_version` (`string`) optional
The maximum HTTP version to support on the distribution. Allowed values are http1.1, http2, http2and3 and http3 **Default value:** `"http2"`
`lambda_edge_allowed_ssm_parameters` (`list(string)`) optional
The Lambda@Edge functions will be allowed to access the list of AWS SSM parameter with these ARNs **Default value:** `[ ]`
`lambda_edge_destruction_delay` (`string`) optional
The delay, in [Golang ParseDuration](https://pkg.go.dev/time#ParseDuration) format, to wait before destroying the Lambda@Edge functions. This delay is meant to circumvent Lambda@Edge functions not being immediately deletable following their dissociation from a CloudFront distribution, since they are replicated to CloudFront Edge servers around the world. If set to `null`, no delay will be introduced. By default, the delay is 20 minutes. This is because it takes about 3 minutes to destroy a CloudFront distribution, and around 15 minutes until the Lambda@Edge function is available for deletion, in most cases. For more information, see: https://github.com/hashicorp/terraform-provider-aws/issues/1721. **Default value:** `"20m"`
`lambda_edge_functions` optional
Lambda@Edge functions to create. The key of this map is the name of the Lambda@Edge function. This map will be deep merged with each enabled default function. Use deep merge to change or overwrite specific values passed by those function objects. **Type:** ```hcl map(object({ source = optional(list(object({ filename = string content = string }))) source_dir = optional(string) source_zip = optional(string) runtime = string handler = string event_type = string include_body = bool })) ``` **Default value:** `{ }`
`lambda_edge_handler` (`string`) optional
The default Lambda@Edge handler for all functions. This value is deep merged in `module.lambda_edge_functions` with `var.lambda_edge_functions` and can be overwritten for any individual function. **Default value:** `"index.handler"`
`lambda_edge_runtime` (`string`) optional
The default Lambda@Edge runtime for all functions. This value is deep merged in `module.lambda_edge_functions` with `var.lambda_edge_functions` and can be overwritten for any individual function. **Default value:** `"nodejs16.x"`
`ordered_cache` optional
An ordered list of [cache behaviors](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#cache-behavior-arguments) resource for this distribution. List in order of precedence (first match wins). This is in addition to the default cache policy. Set `target_origin_id` to `""` to specify the S3 bucket origin created by this module. Set `cache_policy_id` to `""` to use `cache_policy_name` for creating a new policy. At least one of the two must be set. Set `origin_request_policy_id` to `""` to use `origin_request_policy_name` for creating a new policy. At least one of the two must be set. **Type:** ```hcl list(object({ target_origin_id = string path_pattern = string allowed_methods = list(string) cached_methods = list(string) compress = bool trusted_signers = list(string) trusted_key_groups = list(string) cache_policy_name = optional(string) cache_policy_id = optional(string) origin_request_policy_name = optional(string) origin_request_policy_id = optional(string) viewer_protocol_policy = string min_ttl = number default_ttl = number max_ttl = number response_headers_policy_id = string forward_query_string = bool forward_header_values = list(string) forward_cookies = string forward_cookies_whitelisted_names = list(string) lambda_function_association = list(object({ event_type = string include_body = bool lambda_arn = string })) function_association = list(object({ event_type = string function_arn = string })) origin_request_policy = optional(object({ cookie_behavior = optional(string, "none") header_behavior = optional(string, "none") query_string_behavior = optional(string, "none") cookies = optional(list(string), []) headers = optional(list(string), []) query_strings = optional(list(string), []) }), {}) })) ``` **Default value:** `[ ]`
`origin_allow_ssl_requests_only` (`bool`) optional
Set to `true` in order to have the origin bucket require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `true`
`origin_bucket` (`string`) optional
Name of an existing S3 bucket to use as the origin. If this is not provided, this component will create a new s3 bucket using `var.name` and other context related inputs **Default value:** `null`
`origin_deployment_actions` (`list(string)`) optional
List of actions to permit `origin_deployment_principal_arns` to perform on bucket and bucket prefixes (see `origin_deployment_principal_arns`) **Default value:** ```hcl [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] ```
`origin_deployment_principal_arns` (`list(string)`) optional
List of role ARNs to grant deployment permissions to the origin Bucket. **Default value:** `[ ]`
`origin_encryption_enabled` (`bool`) optional
When set to 'true' the origin Bucket will have aes256 encryption enabled by default. **Default value:** `true`
`origin_force_destroy` (`bool`) optional
A boolean string that indicates all objects should be deleted from the origin Bucket so that the Bucket can be destroyed without error. These objects are not recoverable. **Default value:** `false`
`origin_s3_access_log_bucket_name` (`string`) optional
Name of the existing S3 bucket where S3 Access Logs for the origin Bucket will be delivered. Default is not to enable S3 Access Logging for the origin Bucket. **Default value:** `""`
`origin_s3_access_log_bucket_name_rendering_enabled` (`bool`) optional
If set to `true`, then the S3 origin access logs bucket name will be rendered by calling `format("%v-%v-%v-%v", var.namespace, var.environment, var.stage, var.origin_s3_access_log_bucket_name)`. Otherwise, the value for `origin_s3_access_log_bucket_name` will need to be the globally unique name of the access logs bucket. For example, if this component produces an origin bucket named `eg-ue1-devplatform-example` and `origin_s3_access_log_bucket_name` is set to `example-s3-access-logs`, then the bucket name will be rendered to be `eg-ue1-devplatform-example-s3-access-logs`. **Default value:** `false`
`origin_s3_access_log_prefix` (`string`) optional
Prefix to use for S3 Access Log object keys. Defaults to `logs/${module.this.id}` **Default value:** `""`
`origin_s3_access_logging_enabled` (`bool`) optional
Set `true` to deliver S3 Access Logs to the `origin_s3_access_log_bucket_name` bucket. Defaults to `false` if `origin_s3_access_log_bucket_name` is empty (the default), `true` otherwise. Must be set explicitly if the access log bucket is being created at the same time as this module is being invoked. **Default value:** `null`
`origin_versioning_enabled` (`bool`) optional
Enable or disable versioning for the origin Bucket. Versioning is a means of keeping multiple variants of an object in the same bucket. **Default value:** `false`
`parent_zone_name` (`string`) optional
Parent domain name of site to publish. Defaults to format(parent_zone_name_pattern, stage, environment). **Default value:** `""`
`preview_environment_enabled` (`bool`) optional
Enable or disable SPA Preview Environments via Lambda@Edge, i.e. mapping `subdomain.example.com` to the `/subdomain` path in the origin S3 bucket. This variable implicitly affects the following variables: * `s3_website_enabled` * `s3_website_password_enabled` * `block_origin_public_access_enabled` * `origin_allow_ssl_requests_only` * `forward_header_values` * `cloudfront_default_ttl` * `cloudfront_min_ttl` * `cloudfront_max_ttl` * `cloudfront_lambda_function_association` **Default value:** `false`
`process_domain_validation_options` (`bool`) optional
Flag to enable/disable processing of the record to add to the DNS zone to complete certificate validation **Default value:** `true`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control on the origin bucket. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. **Default value:** `"ObjectWriter"`
`s3_origins` optional
A list of S3 [origins](https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#origin-arguments) (in addition to the one created by this component) for this distribution. S3 buckets configured as websites are `custom_origins`, not `s3_origins`. Specifying `s3_origin_config.origin_access_identity` as `null` or `""` will have it translated to the `origin_access_identity` used by the origin created by this component. **Type:** ```hcl list(object({ domain_name = string origin_id = string origin_path = string s3_origin_config = object({ origin_access_identity = string }) })) ``` **Default value:** `[ ]`
`s3_website_enabled` (`bool`) optional
Set to true to enable the created S3 bucket to serve as a website independently of CloudFront, and to use that website as the origin. Setting `preview_environment_enabled` will implicitly set this to `true`. **Default value:** `false`
`s3_website_password_enabled` (`bool`) optional
If set to true, and `s3_website_enabled` is also true, a password will be required in the `Referrer` field of the HTTP request in order to access the website, and CloudFront will be configured to pass this password in its requests. This will make it much harder for people to bypass CloudFront and access the S3 website directly via its website endpoint. **Default value:** `false`
`site_fqdn` (`string`) optional
Fully qualified domain name of site to publish. Overrides site_subdomain and parent_zone_name. **Default value:** `""`
`site_subdomain` (`string`) optional
Subdomain to plug into site_name_pattern to make site FQDN. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudfront_distribution_alias`
Cloudfront Distribution Alias Record.
`cloudfront_distribution_domain_name`
Cloudfront Distribution Domain Name.
`cloudfront_distribution_identity_arn`
CloudFront Distribution Origin Access Identity IAM ARN.
`failover_s3_bucket_name`
Failover Origin bucket name, if enabled.
`github_actions_iam_role_arn`
ARN of IAM role for GitHub Actions
`github_actions_iam_role_name`
Name of IAM role for GitHub Actions
`origin_s3_bucket_arn`
Origin bucket ARN.
`origin_s3_bucket_name`
Origin bucket name.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `acm_request_certificate` | 0.18.0 | [`cloudposse/acm-request-certificate/aws`](https://registry.terraform.io/modules/cloudposse/acm-request-certificate/aws/0.18.0) | Create an ACM and explicitly set it to us-east-1 (requirement of CloudFront) `dns_delegated` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `gha_assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `gha_role_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `github_runners` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `lambda_edge` | 1.0.1 | [`cloudposse/cloudfront-s3-cdn/aws//modules/lambda@edge`](https://registry.terraform.io/modules/cloudposse/cloudfront-s3-cdn/aws/modules/lambda@edge/1.0.1) | n/a `lambda_edge_functions` | 1.0.2 | [`cloudposse/config/yaml//modules/deepmerge`](https://registry.terraform.io/modules/cloudposse/config/yaml/modules/deepmerge/1.0.2) | n/a `spa_web` | 1.0.1 | [`cloudposse/cloudfront-s3-cdn/aws`](https://registry.terraform.io/modules/cloudposse/cloudfront-s3-cdn/aws/1.0.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a `waf` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudfront_cache_policy.created_cache_policies`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_cache_policy) (resource) - [`aws_cloudfront_origin_request_policy.created_origin_request_policies`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_origin_request_policy) (resource) - [`aws_iam_policy.additional_lambda_edge_permission`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.github_actions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.additional_lambda_edge_permission`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_shield_protection.shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.additional_lambda_edge_permission`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.github_actions_iam_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_s3_bucket.failover_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## spacelift These components are responsible for setting up Spacelift and include three components: `spacelift/admin-stack`, `spacelift/spaces`, and `spacelift/worker-pool`. Spacelift is a specialized, Terraform-compatible continuous integration and deployment (CI/CD) platform for infrastructure-as-code. It's designed and implemented by long-time DevOps practitioners based on previous experience with large-scale installations - dozens of teams, hundreds of engineers and tens of thousands of cloud resources. ## Stack Configuration Spacelift exists outside of the AWS ecosystem, so we define these components as unique to our standard stack organization. Spacelift Spaces are required before tenant-specific stacks are created in Spacelift, and the root administrator stack, referred to as `root-gbl-spacelift-admin-stack`, also does not belong to a specific tenant. Therefore, we define both outside of the standard `core` or `plat` stacks directories. That root administrator stack is responsible for creating the tenant-specific administrator stacks, `core-gbl-spacelift-admin-stack` and `plat-gbl-spacelift-admin-stack`. Our solution is to define a spacelift-specific configuration file per Spacelift Space. Typically our Spaces would be `root`, `core`, and `plat`, so we add three files: ```diff + stacks/orgs/NAMESPACE/spacelift.yaml + stacks/orgs/NAMESPACE/core/spacelift.yaml + stacks/orgs/NAMESPACE/plat/spacelift.yaml ``` ### Global Configuration In order to apply common Spacelift configuration to all stacks, we need to set a few global Spacelift settings. The `pr-comment-triggered` label will be required to trigger stacks with GitHub comments but is not required otherwise. More on triggering Spacelift stacks to follow. Add the following to `stacks/orgs/NAMESPACE/_defaults.yaml`: ```yaml settings: spacelift: workspace_enabled: true # enable spacelift by default before_apply: - spacelift-configure-paths before_init: - spacelift-configure-paths - spacelift-write-vars - spacelift-tf-workspace before_plan: - spacelift-configure-paths labels: - pr-comment-triggered ``` Furthermore, specify additional tenant-specific Space configuration for both `core` and `plat` tenants. For example, for `core` add the following to `stacks/orgs/NAMESPACE/core/_defaults.yaml`: ```yaml terraform: settings: spacelift: space_name: core ``` And for `plat` add the following to `stacks/orgs/NAMESPACE/plat/_defaults.yaml`: ```yaml terraform: settings: spacelift: space_name: plat ``` ### Spacelift `root` Space The `root` Space in Spacelift is responsible for deploying the root administrator stack, `admin-stack`, and the Spaces component, `spaces`. This Spaces component also includes Spacelift policies. Since the root administrator stack is unique to tenants, we modify the stack context to create a unique stack slug, `root-gbl-spacelift`. `stacks/orgs/NAMESPACE/spacelift.yaml`: ```yaml import: - mixins/region/global-region - orgs/NAMESPACE/_defaults - catalog/terraform/spacelift/admin-stack - catalog/terraform/spacelift/spaces # These intentionally overwrite the default values vars: tenant: root environment: gbl stage: spacelift components: terraform: # This admin stack creates other "admin" stacks admin-stack: metadata: component: spacelift/admin-stack inherits: - admin-stack/default settings: spacelift: root_administrative: true labels: - root-admin - admin vars: enabled: true root_admin_stack: true # This stack will be created in the root space and will create all the other admin stacks as children. context_filters: # context_filters determine which child stacks to manage with this admin stack administrative: true # This stack is managing all the other admin stacks root_administrative: false # We don't want this stack to also find itself in the config and add itself a second time labels: - admin # attachments only on the root stack root_stack_policy_attachments: - TRIGGER Global administrator # this creates policies for the children (admin) stacks child_policy_attachments: - TRIGGER Global administrator ``` #### Deployment > [!TIP] > > The following steps assume that you've already authenticated with Spacelift locally. First deploy Spaces and policies with the `spaces` component: ```bash atmos terraform apply spaces -s root-gbl-spacelift ``` In the Spacelift UI, you should see each Space and each policy. Next, deploy the `root` `admin-stack` with the following: ```bash atmos terraform apply admin-stack -s root-gbl-spacelift ``` Now in the Spacelift UI, you should see the administrator stacks created. Typically these should look similar to the following: ```diff + root-gbl-spacelift-admin-stack + root-gbl-spacelift-spaces + core-gbl-spacelift-admin-stack + plat-gbl-spacelift-admin-stack + core-ue1-auto-spacelift-worker-pool ``` > [!TIP] > > The `spacelift/worker-pool` component is deployed to a specific tenant, stage, and region but is still deployed by the > root administrator stack. Verify the administrator stack by checking the `managed-by:` label. Finally, deploy the Spacelift Worker Pool (change the stack-slug to match your configuration): ```bash atmos terraform apply spacelift/worker-pool -s core-ue1-auto ``` ### Spacelift Tenant-Specific Spaces A tenant-specific Space in Spacelift, such as `core` or `plat`, includes the administrator stack for that specific Space and _all_ components in the given tenant. This administrator stack uses `var.context_filters` to select all components in the given tenant and create Spacelift stacks for each. Similar to the root administrator stack, we again create a unique stack slug for each tenant. For example `core-gbl-spacelift` or `plat-gbl-spacelift`. For example, configure a `core` administrator stack with `stacks/orgs/NAMESPACE/core/spacelift.yaml`. ```yaml import: - mixins/region/global-region - orgs/NAMESPACE/core/_defaults - catalog/terraform/spacelift/admin-stack vars: tenant: core environment: gbl stage: spacelift components: terraform: admin-stack: metadata: component: spacelift/admin-stack inherits: - admin-stack/default settings: spacelift: labels: # Additional labels for this stack - admin-stack-name:core vars: enabled: true context_filters: tenants: ["core"] labels: # Additional labels added to all children - admin-stack-name:core # will be used to automatically create the `managed-by:stack-name` label child_policy_attachments: - TRIGGER Dependencies ``` Deploy the `core` `admin-stack` with the following: ```bash atmos terraform apply admin-stack -s core-gbl-spacelift ``` Create the same for the `plat` tenant in `stacks/orgs/NAMESPACE/plat/spacelift.yaml`, update the tenant and configuration as necessary, and deploy with the following: ```bash atmos terraform apply admin-stack -s plat-gbl-spacelift ``` Now all stacks for all components should be created in the Spacelift UI. ## Triggering Spacelift Runs Cloud Posse recommends two options to trigger Spacelift stacks. ### Triggering with Policy Attachments Historically, all stacks were triggered with three `GIT_PUSH` policies: 1. [GIT_PUSH Global Administrator](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.administrative.rego) triggers admin stacks 2. [GIT_PUSH Proposed Run](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.proposed-run.rego) triggers Proposed runs (typically Terraform Plan) for all non-admin stacks on Pull Requests 3. [GIT_PUSH Tracked Run](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.tracked-run.rego) triggers Tracked runs (typically Terraform Apply) for all non-admin stacks on merges into `main` Attach these policies to stacks and Spacelift will trigger them on the respective git push. ### Triggering with GitHub Comments (Preferred) Atmos support for `atmos describe affected` made it possible to greatly improve Spacelift's triggering workflow. Now we can add a GitHub Action to collect all affected components for a given Pull Request and add a GitHub comment to the given PR with a formatted list of the affected stacks. Then Spacelift can watch for a GitHub comment event and then trigger stacks based on that comment. In order to set up GitHub Comment triggers, first add the following `GIT_PUSH Plan Affected` policy to the `spaces` component. For example, `stacks/catalog/spacelift/spaces.yaml` ```yaml components: terraform: spaces: metadata: component: spacelift/spaces settings: spacelift: administrative: true space_name: root vars: spaces: root: policies: --- # This policy will automatically assign itself to stacks and is used to trigger stacks directly from the `cloudposse/github-action-atmos-affected-trigger-spacelift` GitHub action # This is only used if said GitHub action is set to trigger on "comments" "GIT_PUSH Plan Affected": type: GIT_PUSH labels: - autoattach:pr-comment-triggered body: | package spacelift # This policy runs whenever a comment is added to a pull request. It looks for the comment body to contain either: # /spacelift preview input.stack.id # /spacelift deploy input.stack.id # # If the comment matches those patterns it will queue a tracked run (deploy) or a proposed run (preview). In the case of # a proposed run, it will also cancel all of the other pending runs for the same branch. # # This is being used on conjunction with the GitHub actions `atmos-trigger-spacelift-feature-branch.yaml` and # `atmos-trigger-spacelift-main-branch.yaml` in .github/workflows to automatically trigger a preview or deploy run based # on the `atmos describe affected` output. track { commented contains(input.pull_request.comment, concat(" ", ["/spacelift", "deploy", input.stack.id])) } propose { commented contains(input.pull_request.comment, concat(" ", ["/spacelift", "preview", input.stack.id])) } # Ignore if the event is not a comment ignore { not commented } # Ignore if the PR has a `spacelift-no-trigger` label ignore { input.pull_request.labels[_] = "spacelift-no-trigger" } # Ignore if the PR is a draft and deesnt have a `spacelift-trigger` label ignore { input.pull_request.draft not has_spacelift_trigger_label } has_spacelift_trigger_label { input.pull_request.labels[_] == "spacelift-trigger" } commented { input.pull_request.action == "commented" } cancel[run.id] { run := input.in_progress[_] run.type == "PROPOSED" run.state == "QUEUED" run.branch == input.pull_request.head.branch } # This is a random sample of 10% of the runs sample { millis := round(input.request.timestamp_ns / 1e6) millis % 100 <= 10 } ``` This policy will automatically attach itself to _all_ components that have the `pr-comment-triggered` label, already defined in `stacks/orgs/NAMESPACE/_defaults.yaml` under `settings.spacelift.labels`. Next, create two new GitHub Action workflows: ```diff + .github/workflows/atmos-trigger-spacelift-feature-branch.yaml + .github/workflows/atmos-trigger-spacelift-main-branch.yaml ``` The feature branch workflow will create a comment event in Spacelift to run a Proposed run for a given stack. Whereas the main branch workflow will create a comment event in Spacelift to run a Deploy run for those same stacks. #### Feature Branch ```yaml name: "Plan Affected Spacelift Stacks" on: pull_request: types: - opened - synchronize - reopened branches: - main jobs: context: runs-on: ["self-hosted"] steps: - name: Atmos Affected Stacks Trigger Spacelift uses: cloudposse/github-action-atmos-affected-trigger-spacelift@v1 with: atmos-config-path: ./rootfs/usr/local/etc/atmos github-token: ${{ secrets.GITHUB_TOKEN }} ``` This will add a GitHub comment such as: ``` /spacelift preview plat-ue1-sandbox-foobar ``` #### Main Branch ```yaml name: "Deploy Affected Spacelift Stacks" on: pull_request: types: [closed] branches: - main jobs: run: if: github.event.pull_request.merged == true runs-on: ["self-hosted"] steps: - name: Atmos Affected Stacks Trigger Spacelift uses: cloudposse/github-action-atmos-affected-trigger-spacelift@v1 with: atmos-config-path: ./rootfs/usr/local/etc/atmos deploy: true github-token: ${{ secrets.GITHUB_TOKEN }} head-ref: ${{ github.sha }}~1 ``` This will add a GitHub comment such as: ``` /spacelift deploy plat-ue1-sandbox-foobar ``` --- ## admin-stack This component is responsible for creating an administrative [stack](https://docs.spacelift.io/concepts/stack/) and its corresponding child stacks in the Spacelift organization. The component uses a series of `context_filters` to select atmos component instances to manage as child stacks. ## Usage **Stack Level**: Global The following are example snippets of how to use this component. For more on Spacelift admin stack usage, see the [Spacelift README](https://docs.cloudposse.com/components/library/aws/spacelift/) First define the default configuration for any admin stack: ```yaml # stacks/catalog/spacelift/admin-stack.yaml components: terraform: admin-stack/default: metadata: type: abstract component: spacelift/admin-stack settings: spacelift: administrative: true autodeploy: true before_apply: - spacelift-configure-paths before_init: - spacelift-configure-paths - spacelift-write-vars - spacelift-tf-workspace before_plan: - spacelift-configure-paths drift_detection_enabled: true drift_detection_reconcile: true drift_detection_schedule: - 0 4 * * * manage_state: false policies: {} vars: # Organization specific configuration branch: main repository: infrastructure worker_pool_name: "acme-core-ue1-auto-spacelift-worker-pool" runner_image: 111111111111.dkr.ecr.us-east-1.amazonaws.com/infrastructure:latest spacelift_spaces_stage_name: "root" # These values need to be manually updated as external configuration changes # This should match the version set in the Dockerfile and be updated when the version changes. terraform_version: "1.3.6" # Common configuration administrative: true # Whether this stack can manage other stacks component_root: components/terraform ``` Then define the root-admin stack: ```yaml # stacks/orgs/acme/spacelift.yaml import: - mixins/region/global-region - orgs/acme/_defaults - catalog/terraform/spacelift/admin-stack - catalog/terraform/spacelift/spaces # These intentionally overwrite the default values vars: tenant: root environment: gbl stage: spacelift components: terraform: # This admin stack creates other "admin" stacks admin-stack: metadata: component: spacelift/admin-stack inherits: - admin-stack/default settings: spacelift: root_administrative: true labels: - root-admin - admin vars: enabled: true root_admin_stack: true # This stack will be created in the root space and will create all the other admin stacks as children. context_filters: # context_filters determine which child stacks to manage with this admin stack administrative: true # This stack is managing all the other admin stacks root_administrative: false # We don't want this stack to also find itself in the config and add itself a second time labels: - admin # attachments only on the root stack root_stack_policy_attachments: - TRIGGER Global administrator # this creates policies for the children (admin) stacks child_policy_attachments: - TRIGGER Global administrator ``` Finally, define any tenant-specific stacks: ```yaml # stacks/orgs/acme/core/spacelift.yaml import: - mixins/region/global-region - orgs/acme/core/_defaults - catalog/terraform/spacelift/admin-stack vars: tenant: core environment: gbl stage: spacelift components: terraform: admin-stack: metadata: component: spacelift/admin-stack inherits: - admin-stack/default settings: spacelift: labels: # Additional labels for this stack - admin-stack-name:core vars: enabled: true context_filters: tenants: ["core"] labels: # Additional labels added to all children - admin-stack-name:core # will be used to automatically create the `managed-by:stack-name` label child_policy_attachments: - TRIGGER Dependencies ``` ## Variables ### Required Variables
`component_root` (`string`) required
The path, relative to the root of the repository, where the component can be found
`context_filters` required
Context filters to select atmos stacks matching specific criteria to create as children. **Type:** ```hcl object({ namespaces = optional(list(string), []) environments = optional(list(string), []) tenants = optional(list(string), []) stages = optional(list(string), []) tags = optional(map(string), {}) administrative = optional(bool) root_administrative = optional(bool) }) ```
`repository` (`string`) required
The name of your infrastructure repo
### Optional Variables
`admin_stack_label` (`string`) optional
Label to use to identify the admin stack when creating the child stacks **Default value:** `"admin-stack-name"`
`allow_public_workers` (`bool`) optional
Whether to allow public workers to be used for this stack **Default value:** `false`
`autodeploy` (`bool`) optional
Controls the Spacelift 'autodeploy' option for a stack **Default value:** `false`
`autoretry` (`bool`) optional
Controls the Spacelift 'autoretry' option for a stack **Default value:** `false`
`aws_role_arn` (`string`) optional
ARN of the AWS IAM role to assume and put its temporary credentials in the runtime environment **Default value:** `null`
`aws_role_enabled` (`bool`) optional
Flag to enable/disable Spacelift to use AWS STS to assume the supplied IAM role and put its temporary credentials in the runtime environment **Default value:** `false`
`aws_role_external_id` (`string`) optional
Custom external ID (works only for private workers). See https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html for more details **Default value:** `null`
`aws_role_generate_credentials_in_worker` (`bool`) optional
Flag to enable/disable generating AWS credentials in the private worker after assuming the supplied IAM role **Default value:** `true`
`azure_devops` (`map(any)`) optional
Azure DevOps VCS settings **Default value:** `null`
`bitbucket_cloud` (`map(any)`) optional
Bitbucket Cloud VCS settings **Default value:** `null`
`bitbucket_datacenter` (`map(any)`) optional
Bitbucket Datacenter VCS settings **Default value:** `null`
`branch` (`string`) optional
Specify which branch to use within your infrastructure repo **Default value:** `"main"`
`child_policy_attachments` (`set(string)`) optional
List of policy attachments to attach to the child stacks created by this module **Default value:** `[ ]`
`cloudformation` (`map(any)`) optional
CloudFormation-specific configuration. Presence means this Stack is a CloudFormation Stack. **Default value:** `null`
`commit_sha` (`string`) optional
The commit SHA for which to trigger a run. Requires `var.spacelift_run_enabled` to be set to `true` **Default value:** `null`
`component_env` (`any`) optional
Map of component ENV variables **Default value:** `{ }`
`component_vars` (`any`) optional
All Terraform values to be applied to the stack via a mounted file **Default value:** `{ }`
`context_attachments` (`list(string)`) optional
A list of context IDs to attach to this stack **Default value:** `[ ]`
`description` (`string`) optional
Specify description of stack **Default value:** `null`
`drift_detection_enabled` (`bool`) optional
Flag to enable/disable drift detection on the infrastructure stacks **Default value:** `false`
`drift_detection_reconcile` (`bool`) optional
Flag to enable/disable infrastructure stacks drift automatic reconciliation. If drift is detected and `reconcile` is turned on, Spacelift will create a tracked run to correct the drift **Default value:** `false`
`drift_detection_schedule` (`list(string)`) optional
List of cron expressions to schedule drift detection for the infrastructure stacks **Default value:** ```hcl [ "0 4 * * *" ] ```
`drift_detection_timezone` (`string`) optional
Timezone in which the schedule is expressed. Defaults to UTC. **Default value:** `null`
`excluded_context_filters` optional
Context filters to exclude from stacks matching specific criteria of `var.context_filters`. **Type:** ```hcl object({ namespaces = optional(list(string), []) environments = optional(list(string), []) tenants = optional(list(string), []) stages = optional(list(string), []) tags = optional(map(string), {}) }) ``` **Default value:** `{ }`
`github_enterprise` (`map(any)`) optional
GitHub Enterprise (self-hosted) VCS settings **Default value:** `null`
`gitlab` (`map(any)`) optional
GitLab VCS settings **Default value:** `null`
`labels` (`list(string)`) optional
A list of labels for the stack **Default value:** `[ ]`
`local_preview_enabled` (`bool`) optional
Indicates whether local preview runs can be triggered on this Stack **Default value:** `false`
`manage_state` (`bool`) optional
Flag to enable/disable manage_state setting in stack **Default value:** `false`
`protect_from_deletion` (`bool`) optional
Flag to enable/disable deletion protection. **Default value:** `false`
`pulumi` (`map(any)`) optional
Pulumi-specific configuration. Presence means this Stack is a Pulumi Stack. **Default value:** `null`
`root_admin_stack` (`bool`) optional
Flag to indicate if this stack is the root admin stack. In this case, the stack will be created in the root space and will create all the other admin stacks as children. **Default value:** `false`
`root_stack_policy_attachments` (`set(string)`) optional
List of policy attachments to attach to the root admin stack **Default value:** `[ ]`
`runner_image` (`string`) optional
The full image name and tag of the Docker image to use in Spacelift **Default value:** `null`
`showcase` (`map(any)`) optional
Showcase settings **Default value:** `null`
`space_id` (`string`) optional
Place the stack in the specified space_id **Default value:** `"root"`
`spacelift_run_enabled` (`bool`) optional
Enable/disable creation of the `spacelift_run` resource **Default value:** `false`
`spacelift_spaces_component_name` (`string`) optional
The component name of the spacelift spaces component **Default value:** `"spacelift/spaces"`
`spacelift_spaces_environment_name` (`string`) optional
The environment name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_stage_name` (`string`) optional
The stage name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_tenant_name` (`string`) optional
The tenant name of the spacelift spaces component **Default value:** `null`
`spacelift_stack_dependency_enabled` (`bool`) optional
If enabled, the `spacelift_stack_dependency` Spacelift resource will be used to create dependencies between stacks instead of using the `depends-on` labels. The `depends-on` labels will be removed from the stacks and the trigger policies for dependencies will be detached **Default value:** `false`
`stack_destructor_enabled` (`bool`) optional
Flag to enable/disable the stack destructor to destroy the resources of the stack before deleting the stack itself **Default value:** `false`
`stack_name` (`string`) optional
The name of the Spacelift stack **Default value:** `null`
`terraform_smart_sanitization` (`bool`) optional
Whether or not to enable [Smart Sanitization](https://docs.spacelift.io/vendors/terraform/resource-sanitization) which will only sanitize values marked as sensitive. **Default value:** `false`
`terraform_version` (`string`) optional
Specify the version of Terraform to use for the stack **Default value:** `null`
`terraform_version_map` (`map(string)`) optional
A map to determine which Terraform patch version to use for each minor version **Default value:** `{ }`
`terraform_workflow_tool` (`string`) optional
Defines the tool that will be used to execute the workflow. This can be one of OPEN_TOFU, TERRAFORM_FOSS or CUSTOM. Defaults to TERRAFORM_FOSS. **Default value:** `"TERRAFORM_FOSS"`
`terraform_workspace` (`string`) optional
Specify the Terraform workspace to use for the stack **Default value:** `null`
`webhook_enabled` (`bool`) optional
Flag to enable/disable the webhook endpoint to which Spacelift sends the POST requests about run state changes **Default value:** `false`
`webhook_endpoint` (`string`) optional
Webhook endpoint to which Spacelift sends the POST requests about run state changes **Default value:** `null`
`webhook_secret` (`string`) optional
Webhook secret used to sign each POST request so you're able to verify that the requests come from Spacelift **Default value:** `null`
`worker_pool_name` (`string`) optional
The atmos stack name of the worker pool. Example: `acme-core-ue2-auto-spacelift-default-worker-pool` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`child_stacks`
All children stacks managed by this component
`root_stack`
The root stack, if enabled and created by this component
`root_stack_id`
The stack id
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 4.0, < 6.0.0` - `null`, version: `>= 3.0` - `spacelift`, version: `>= 0.1.31` - `utils`, version: `>= 1.14.0` ### Providers - `null`, version: `>= 3.0` - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `all_admin_stacks_config` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stacks-from-atmos-config`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-stacks-from-atmos-config/1.7.2) | This gets the atmos stack config for all of the administrative stacks `child_stack` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stack`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-stack/1.7.2) | n/a `child_stacks_config` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stacks-from-atmos-config`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-stacks-from-atmos-config/1.7.2) | Get all of the stack configurations from the atmos config that matched the context_filters and create a stack for each one. `root_admin_stack` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stack`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-stack/1.7.2) | n/a `root_admin_stack_config` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stacks-from-atmos-config`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-stacks-from-atmos-config/1.7.2) | The root admin stack is a special stack that is used to manage all of the other admin stacks in the the Spacelift organization. This stack is denoted by setting the root_administrative property to true in the atmos config. Only one such stack is allowed in the Spacelift organization. `spaces` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`null_resource.child_stack_parent_precondition`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`null_resource.public_workers_precondition`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`null_resource.spaces_precondition`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`null_resource.workers_precondition`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`spacelift_policy_attachment.root`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`spacelift_policies.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/data-sources/policies) (data source) - [`spacelift_stacks.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/data-sources/stacks) (data source) - [`spacelift_worker_pools.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/data-sources/worker_pools) (data source) --- ## idp-group-mappings This component is responsible for creating and managing the IdP group mappings within the Spacelift organization. It ensures that Identity Provider (IdP) groups are correctly mapped to specific roles across designated Spacelift spaces, enabling precise access control and role-based permissions. ## Usage **Stack Level**: Global Here's an example snippet for how to use this component. ```yaml # stacks/catalog/spacelift/idp-group-mappings.yaml components: terraform: idp-group-mappings: metadata: component: spacelift/idp-group-mappings settings: spacelift: enabled: true vars: spacelift_spaces_tenant_name: root spacelift_spaces_environment_name: gbl spacelift_spaces_stage_name: spacelift spacelift_spaces_component_name: spaces # These must match the group names from the IdP provider idp-group-mappings: spacelift-admin: spacelift_role_name: "ADMIN" spaces: - dev - staging - prod spacelift-writer: spacelift_role_name: "WRITE" spaces: - dev - staging - prod spacelift-reader: spacelift_role_name: "READ" spaces: - dev - staging - prod ``` ## Variables ### Required Variables
### Optional Variables
`idp_group_mappings` optional
Map of IDP group mappings with role names and associated spaces. The key is the IDP group name. **Type:** ```hcl map(object({ spacelift_role_name = string spaces = list(string) })) ``` **Default value:** `{ }`
`spacelift_spaces_component_name` (`string`) optional
The component name of the spacelift spaces component **Default value:** `"spacelift/spaces"`
`spacelift_spaces_environment_name` (`string`) optional
The environment name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_stage_name` (`string`) optional
The stage name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_tenant_name` (`string`) optional
The tenant name of the spacelift spaces component **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
The ID of the component
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `spacelift`, version: `>= 0.1.31` ### Providers - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `spaces` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`spacelift_idp_group_mapping.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/idp_group_mapping) (resource) ## Data Sources The following data sources are used by this module: --- ## spaces This component is responsible for creating and managing the [spaces](https://docs.spacelift.io/concepts/spaces/) in the Spacelift organization. ## Usage **Stack Level**: Global The following are example snippets of how to use this component: ```yaml # stacks/catalog/spacelift/spaces.yaml components: terraform: spaces: metadata: component: spacelift/spaces settings: spacelift: administrative: true space_name: root vars: spaces: # root is a special space that is the parent of all other spaces and cannot be deleted or renamed. Only the # policies block is actually consumed by the component to create policies for the root space. root: parent_space_id: root description: The root space inherit_entities: true policies: GIT_PUSH Global Administrator: type: GIT_PUSH body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/git_push.administrative.rego TRIGGER Global Administrator: type: TRIGGER body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/trigger.administrative.rego GIT_PUSH Proposed Run: type: GIT_PUSH body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/git_push.proposed-run.rego GIT_PUSH Tracked Run: type: GIT_PUSH body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/git_push.tracked-run.rego PLAN Default: type: PLAN body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/plan.default.rego TRIGGER Dependencies: type: TRIGGER body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/trigger.dependencies.rego PLAN Warn On Resource Changes Except Image ID: type: PLAN body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/plan.warn-on-resource-changes-except-image-id.rego core: parent_space_id: root description: The space for the core tenant inherit_entities: true labels: - core plat: parent_space_id: root description: The space for platform tenant inherit_entities: true labels: - plat ``` ## Variables ### Required Variables
`spaces` required
A map of all Spaces to create in Spacelift **Type:** ```hcl map(object({ parent_space_id = string, description = optional(string), inherit_entities = optional(bool, false), labels = optional(set(string), []), policies = optional(map(object({ body = optional(string), body_url = optional(string), body_url_version = optional(string, "master"), body_file_path = optional(string), type = optional(string), labels = optional(set(string), []), })), {}), })) ```
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`policies`
The policies created by this component
`spaces`
The spaces created by this component
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 4.0, < 6.0.0` - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `policy` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-policy`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-policy/1.7.2) | n/a `space` | 1.7.2 | [`cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-space`](https://registry.terraform.io/modules/cloudposse/cloud-infrastructure-automation/spacelift/modules/spacelift-space/1.7.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## worker-pool-asg This component provisions Spacelift worker pools on AWS using an Auto Scaling Group. By default, workers are granted pull access to the configured ECR and permission to assume the `spacelift` team role in the identity account (ensure the `spacelift` team in the identity account allows this via `trusted_role_arns`). Workers also get these AWS managed IAM policies: - AmazonSSMManagedInstanceCore - AutoScalingReadOnlyAccess - AWSXRayDaemonWriteAccess - CloudWatchAgentServerPolicy With SSM agent installed, workers can be accessed via SSM Session Manager. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml # stacks/catalog/spacelift/worker-pool.yaml components: terraform: spacelift/worker-pool: settings: spacelift: administrative: true space_name: root vars: enabled: true spacelift_api_endpoint: https://.app.spacelift.io spacelift_spaces_tenant_name: "acme" spacelift_spaces_environment_name: "gbl" spacelift_spaces_stage_name: "root" account_map_tenant_name: core ecr_environment_name: ue1 ecr_repo_name: infrastructure ecr_stage_name: artifacts ecr_tenant_name: core # Set a low scaling threshold to ensure new workers are launched as soon as the current one(s) are busy cpu_utilization_high_threshold_percent: 10 cpu_utilization_low_threshold_percent: 5 default_cooldown: 300 desired_capacity: null health_check_grace_period: 300 health_check_type: EC2 infracost_enabled: true instance_type: t3.small max_size: 3 min_size: 1 name: spacelift-worker-pool scale_down_cooldown_seconds: 2700 spacelift_agents_per_node: 1 wait_for_capacity_timeout: 5m block_device_mappings: - device_name: "/dev/xvda" no_device: null virtual_name: null ebs: delete_on_termination: null encrypted: false iops: null kms_key_id: null snapshot_id: null volume_size: 100 volume_type: "gp2" ``` To connect to a worker via SSM Session Manager, use: ```bash aws ssm start-session --target ``` ### Impacts on billing While scaling the workload for Spacelift, keep in mind that each agent connection counts against your quota of self-hosted workers. The number of EC2 instances you have running is not going to affect your Spacelift bill. For example, if you had 3 EC2 instances in your Spacelift worker pool, and you configured `spacelift_agents_per_node` to be `3`, you would see your Spacelift bill report 9 agents being run. Take care while configuring the worker pool for your Spacelift infrastructure. ## Configuration ### Docker Image on ECR Build and tag a Docker image for this repository and push to ECR. Ensure the account where this component is deployed has read-only access to the ECR repository. ### API Key Prior to deployment, the API key must exist in SSM. The key must have admin permissions. To generate the key, please follow [these instructions](https://docs.spacelift.io/integrations/api.html#spacelift-api-key-token). Once generated, write the API key ID and secret to the SSM key store at the following locations within the same AWS account and region where the Spacelift worker pool will reside. | Key | SSM Path | Type | | ------- | ----------------------- | -------------- | | API ID | `/spacelift/key_id` | `SecureString` | | API Key | `/spacelift/key_secret` | `SecureString` | Hint: The API key ID is displayed as an upper-case, 16-character alphanumeric value next to the key name in the API key list. Save the keys using `chamber` using the correct profile for where the Spacelift worker pool is provisioned: ``` AWS_PROFILE=acme-gbl-auto-admin chamber write spacelift key_id 1234567890123456 AWS_PROFILE=acme-gbl-auto-admin chamber write spacelift key_secret abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz ``` ### IAM configuration After provisioning the component, you must give the created instance role permission to assume the Spacelift worker role. This is done by adding `iam_role_arn` from the output to the `trusted_role_arns` list for the `spacelift` role in `aws-teams`. ## Variables ### Required Variables
`cpu_utilization_high_threshold_percent` (`number`) required
CPU utilization high threshold
`cpu_utilization_low_threshold_percent` (`number`) required
CPU utilization low threshold
`ecr_repo_name` (`string`) required
ECR repository name
`max_size` (`number`) required
The maximum size of the autoscale group
`min_size` (`number`) required
The minimum size of the autoscale group
`region` (`string`) required
AWS Region
`spacelift_api_endpoint` (`string`) required
The Spacelift API endpoint URL (e.g. https://example.app.spacelift.io)
`wait_for_capacity_timeout` (`string`) required
A maximum duration that Terraform should wait for ASG instances to be healthy before timing out. (See also Waiting for Capacity below.) Setting this to '0' causes Terraform to skip all Capacity Waiting behavior
### Optional Variables
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`architecture` (`list(string)`) optional
OS architecture of the EC2 instance AMI **Default value:** ```hcl [ "x86_64" ] ```
`aws_config_file` (`string`) optional
The AWS_CONFIG_FILE used by the worker. Can be overridden by `/.spacelift/config.yml`. **Default value:** `"/etc/aws-config/aws-config-spacelift"`
`aws_profile` (`string`) optional
The AWS_PROFILE used by the worker. If not specified, `"${var.namespace}-identity"` will be used. Can be overridden by `/.spacelift/config.yml`. **Default value:** `null`
`block_device_mappings` optional
Specify volumes to attach to the instance besides the volumes specified by the AMI **Type:** ```hcl list(object({ device_name = string no_device = bool virtual_name = string ebs = object({ delete_on_termination = bool encrypted = bool iops = number kms_key_id = string snapshot_id = string volume_size = number volume_type = string }) })) ``` **Default value:** `[ ]`
`custom_spacelift_ami` (`bool`) optional
Custom spacelift AMI **Default value:** `false`
`default_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes before another scaling activity can start **Default value:** `300`
`desired_capacity` (`number`) optional
The number of Amazon EC2 instances that should be running in the group, if not set will use `min_size` as value **Default value:** `null`
`ebs_optimized` (`bool`) optional
If true, the launched EC2 instance will be EBS-optimized **Default value:** `false`
`ecr_environment_name` (`string`) optional
The name of the environment where `ecr` is provisioned **Default value:** `""`
`ecr_region` (`string`) optional
AWS region that contains the ECR infrastructure repo **Default value:** `""`
`ecr_stage_name` (`string`) optional
The name of the stage where `ecr` is provisioned **Default value:** `"artifacts"`
`ecr_tenant_name` (`string`) optional
The name of the tenant where `ecr` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`github_netrc_enabled` (`bool`) optional
Whether to create a GitHub .netrc file so Spacelift can clone private GitHub repositories. **Default value:** `false`
`github_netrc_ssm_path_token` (`string`) optional
If `github_netrc` is enabled, this is the SSM path to retrieve the GitHub token. **Default value:** `"/github/token"`
`github_netrc_ssm_path_user` (`string`) optional
If `github_netrc` is enabled, this is the SSM path to retrieve the GitHub user **Default value:** `"/github/user"`
`health_check_grace_period` (`number`) optional
Time (in seconds) after instance comes into service before checking health **Default value:** `300`
`health_check_type` (`string`) optional
Controls how health checking is done. Valid values are `EC2` or `ELB` **Default value:** `"EC2"`
`iam_attributes` (`list(string)`) optional
Additional attributes to add to the IDs of the IAM role and policy **Default value:** `[ ]`
`infracost_api_token_ssm_path` (`string`) optional
This is the SSM path to retrieve and set the INFRACOST_API_TOKEN environment variable **Default value:** `"/infracost/token"`
`infracost_cli_args` (`string`) optional
These are the CLI args passed to infracost **Default value:** `""`
`infracost_enabled` (`bool`) optional
Whether to enable infracost for Spacelift stacks **Default value:** `false`
`infracost_warn_on_failure` (`bool`) optional
A failure executing Infracost, or a non-zero exit code being returned from the command will cause runs to fail. If this is true, this will only warn instead of failing the stack. **Default value:** `true`
`instance_lifetime` (`number`) optional
Number of seconds after which the instance will be terminated. The default is set to 14 days. **Default value:** `1209600`
`instance_refresh` optional
The instance refresh definition. If this block is configured, an Instance Refresh will be started when the Auto Scaling Group is updated **Type:** ```hcl object({ strategy = string preferences = object({ instance_warmup = optional(number, null) min_healthy_percentage = optional(number, null) skip_matching = optional(bool, null) auto_rollback = optional(bool, null) }) triggers = optional(list(string), []) }) ``` **Default value:** `null`
`instance_type` (`string`) optional
EC2 instance type to use for workers **Default value:** `"r5n.large"`
`launch_template_version` (`string`) optional
Launch template version to use for workers. Note that instance refresh settings are IGNORED unless template version is empty **Default value:** `"$Latest"`
`mixed_instances_policy` optional
Policy to use a mixed group of on-demand/spot of different types. Launch template is automatically generated. https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#mixed_instances_policy-1 **Type:** ```hcl object({ instances_distribution = object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string }) override = list(object({ instance_type = string weighted_capacity = number })) }) ``` **Default value:** `null`
`scale_down_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`space_name` (`string`) optional
The name of the Space to create the worker pool in **Default value:** `"root"`
`spacelift_agents_per_node` (`number`) optional
Number of Spacelift agents to run on one worker node. NOTE: This affects billable units. Spacelift charges per agent. **Default value:** `1`
`spacelift_ami_id` (`string`) optional
AMI ID of Spacelift worker pool image **Default value:** `null`
`spacelift_aws_account_id` (`string`) optional
AWS Account ID owned by Spacelift **Default value:** `"643313122712"`
`spacelift_domain_name` (`string`) optional
Top-level domain name to use for pulling the launcher binary **Default value:** `"spacelift.io"`
`spacelift_runner_image` (`string`) optional
URL of ECR image to use for Spacelift **Default value:** `""`
`spacelift_spaces_component_name` (`string`) optional
The name of the spacelift spaces component **Default value:** `"spacelift/spaces"`
`spacelift_spaces_environment_name` (`string`) optional
The environment name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_stage_name` (`string`) optional
The stage name of the spacelift spaces component **Default value:** `null`
`spacelift_spaces_tenant_name` (`string`) optional
The tenant name of the spacelift spaces component **Default value:** `null`
`termination_policies` (`list(string)`) optional
A list of policies to decide how the instances in the auto scale group should be terminated. The allowed values are `OldestInstance`, `NewestInstance`, `OldestLaunchConfiguration`, `ClosestToNextInstanceHour`, `Default` **Default value:** ```hcl [ "OldestLaunchConfiguration" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`autoscaling_group_arn`
The ARN for this AutoScaling Group
`autoscaling_group_default_cooldown`
Time between a scaling activity and the succeeding scaling activity
`autoscaling_group_health_check_grace_period`
Time after instance comes into service before checking health
`autoscaling_group_health_check_type`
`EC2` or `ELB`. Controls how health checking is done
`autoscaling_group_id`
The autoscaling group id
`autoscaling_group_max_size`
The maximum size of the autoscale group
`autoscaling_group_min_size`
The minimum size of the autoscale group
`autoscaling_group_name`
The autoscaling group name
`iam_role_arn`
Spacelift IAM Role ARN
`iam_role_id`
Spacelift IAM Role ID
`iam_role_name`
Spacelift IAM Role name
`launch_template_arn`
The ARN of the launch template
`launch_template_id`
The ID of the launch template
`security_group_arn`
Spacelift Security Group ARN
`security_group_id`
Spacelift Security Group ID
`security_group_name`
Spacelift Security Group Name
`worker_pool_id`
Spacelift worker pool ID
`worker_pool_name`
Spacelift worker pool name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `cloudinit`, version: `>= 2.2.0` - `spacelift`, version: `>= 0.1.2` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `cloudinit`, version: `>= 2.2.0` - `spacelift`, version: `>= 0.1.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `autoscale_group` | 0.42.0 | [`cloudposse/ec2-autoscale-group/aws`](https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.42.0) | n/a `ecr` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `spaces` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`spacelift_worker_pool.primary`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/worker_pool) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.spacelift`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_iam_policy_document.assume_role_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.spacelift_key_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.spacelift_key_secret`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`cloudinit_config.config`](https://registry.terraform.io/providers/hashicorp/cloudinit/latest/docs/data-sources/config) (data source) --- ## sqs-queue This component is responsible for creating an SQS queue. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: sqs-queue/defaults: vars: enabled: true # org defaults sqs-queue: metadata: component: sqs-queue inherits: - sqs-queue/defaults vars: name: sqs visibility_timeout_seconds: 30 message_retention_seconds: 86400 # 1 day delay_seconds: 0 max_message_size_bytes: 262144 receive_wait_time_seconds: 0 fifo_queue: false content_based_deduplication: false dlq_enabled: true dlq_name_suffix: "dead-letter" # default is dlq dlq_max_receive_count: 1 dlq_kms_data_key_reuse_period_seconds: 86400 # 1 day kms_data_key_reuse_period_seconds: 86400 # 1 day # kms_master_key_id: "alias/aws/sqs" # Use KMS # default null sqs_managed_sse_enabled: true # SSE vs KMS (Priority goes to KMS) iam_policy_limit_to_current_account: true # default true iam_policy: - version: 2012-10-17 policy_id: Allow-S3-Event-Notifications statements: - sid: Allow-S3-Event-Notifications effect: Allow principals: - type: Service identifiers: ["s3.amazonaws.com"] actions: - SQS:SendMessage resources: [] # auto includes this queue's ARN conditions: ## this is included when `iam_policy_limit_to_current_account` is true #- test: StringEquals # variable: aws:SourceAccount # value: "1234567890" - test: ArnLike variable: aws:SourceArn values: - "arn:aws:s3:::*" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`content_based_deduplication` (`bool`) optional
Enables content-based deduplication for FIFO queues. For more information, see the [related documentation](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html#FIFO-queues-exactly-once-processing) **Default value:** `false`
`create_dlq_redrive_allow_policy` (`bool`) optional
Determines whether to create a redrive allow policy for the dead letter queue. **Default value:** `true`
`deduplication_scope` (`string`) optional
Specifies whether message deduplication occurs at the message group or queue level **Default value:** `null`
`delay_seconds` (`number`) optional
The time in seconds that the delivery of all messages in the queue will be delayed. An integer from 0 to 900 (15 minutes). The default for this attribute is 0 seconds. **Default value:** `0`
`dlq_content_based_deduplication` (`bool`) optional
Enables content-based deduplication for FIFO queues **Default value:** `null`
`dlq_deduplication_scope` (`string`) optional
Specifies whether message deduplication occurs at the message group or queue level **Default value:** `null`
`dlq_delay_seconds` (`number`) optional
The time in seconds that the delivery of all messages in the queue will be delayed. An integer from 0 to 900 (15 minutes) **Default value:** `null`
`dlq_enabled` (`bool`) optional
Boolean designating whether the Dead Letter Queue should be created by this component. **Default value:** `false`
`dlq_kms_data_key_reuse_period_seconds` (`number`) optional
The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. An integer representing seconds, between 60 seconds (1 minute) and 86,400 seconds (24 hours) **Default value:** `null`
`dlq_kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK **Default value:** `null`
`dlq_max_receive_count` (`number`) optional
The number of times a message can be unsuccessfully dequeued before being moved to the Dead Letter Queue. **Default value:** `5`
`dlq_message_retention_seconds` (`number`) optional
The number of seconds Amazon SQS retains a message. Integer representing seconds, from 60 (1 minute) to 1209600 (14 days) **Default value:** `null`
`dlq_name_suffix` (`string`) optional
The suffix of the Dead Letter Queue. **Default value:** `"dlq"`
`dlq_receive_wait_time_seconds` (`number`) optional
The time for which a ReceiveMessage call will wait for a message to arrive (long polling) before returning. An integer from 0 to 20 (seconds) **Default value:** `null`
`dlq_redrive_allow_policy` (`any`) optional
The JSON policy to set up the Dead Letter Queue redrive permission, see AWS docs. **Default value:** `{ }`
`dlq_sqs_managed_sse_enabled` (`bool`) optional
Boolean to enable server-side encryption (SSE) of message content with SQS-owned encryption keys **Default value:** `true`
`dlq_tags` (`map(string)`) optional
A mapping of additional tags to assign to the dead letter queue **Default value:** `{ }`
`dlq_visibility_timeout_seconds` (`number`) optional
The visibility timeout for the queue. An integer from 0 to 43200 (12 hours) **Default value:** `null`
`fifo_queue` (`bool`) optional
Boolean designating a FIFO queue. If not set, it defaults to false making it standard. **Default value:** `false`
`fifo_throughput_limit` (`string`) optional
Specifies whether the FIFO queue throughput quota applies to the entire queue or per message group. Valid values are perQueue and perMessageGroupId. This can be specified if fifo_queue is true. **Default value:** `null`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`iam_policy_limit_to_current_account` (`bool`) optional
Boolean designating whether the IAM policy should be limited to the current account. **Default value:** `true`
`kms_data_key_reuse_period_seconds` (`number`) optional
The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again. An integer representing seconds, between 60 seconds (1 minute) and 86,400 seconds (24 hours). The default is 300 (5 minutes). **Default value:** `300`
`kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SQS or a custom CMK. For more information, see Key Terms. **Default value:** `null`
`max_message_size` (`number`) optional
The limit of how many bytes a message can contain before Amazon SQS rejects it. An integer from 1024 bytes (1 KiB) up to 262144 bytes (256 KiB). The default for this attribute is 262144 (256 KiB). **Default value:** `262144`
`message_retention_seconds` (`number`) optional
The number of seconds Amazon SQS retains a message. Integer representing seconds, from 60 (1 minute) to 1209600 (14 days). The default for this attribute is 345600 (4 days). **Default value:** `345600`
`receive_wait_time_seconds` (`number`) optional
The time for which a ReceiveMessage call will wait for a message to arrive (long polling) before returning. An integer from 0 to 20 (seconds). The default for this attribute is 0, meaning that the call will return immediately. **Default value:** `0`
`sqs_managed_sse_enabled` (`bool`) optional
Boolean to enable server-side encryption (SSE) of message content with SQS-owned encryption keys **Default value:** `true`
`visibility_timeout_seconds` (`number`) optional
The visibility timeout for the queue. An integer from 0 to 43200 (12 hours). The default for this attribute is 30. For more information about visibility timeout, see AWS docs. **Default value:** `30`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`sqs_queue`
The SQS queue.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `queue_policy` | 2.0.2 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.2) | n/a `sqs` | 4.3.1 | [`terraform-aws-modules/sqs/aws`](https://registry.terraform.io/modules/terraform-aws-modules/sqs/aws/4.3.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_sqs_queue_policy.sqs_queue_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) --- ## ssm-parameters This component is responsible for provisioning Parameter Store resources against AWS SSM. It supports normal parameter store resources that can be configured directly in YAML OR pulling secret values from a local Sops file. ## Usage **Stack Level**: Regional Here are some example snippets for how to use this component: `stacks/dev/us-east-1.yaml` file: ```yaml components: terraform: ssm-parameters: vars: sops_source_file: ../../config/secrets/dev.yaml sops_source_key: ssm_params params: /DEV/TESTING: value: This is a test of the emergency broadcast system. description: This is a test. overwrite: true type: String ``` ## Variables ### Required Variables
`params` required
A map of parameter values to write to SSM Parameter Store **Type:** ```hcl map(object({ value = string description = string overwrite = optional(bool, false) tier = optional(string, "Standard") type = string ignore_value_changes = optional(bool, false) })) ```
`region` (`string`) required
AWS Region
### Optional Variables
`kms_arn` (`string`) optional
The ARN of a KMS key used to encrypt and decrypt SecretString values **Default value:** `""`
`sops_source_file` (`string`) optional
The relative path to the SOPS file which is consumed as the source for creating parameter resources. **Default value:** `""`
`sops_source_key` (`string`) optional
The SOPS key to pull from the source file. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`created_params`
The keys of created SSM parameter store resources.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0, < 6.0.0` - `sops`, version: `>= 0.5, < 1.0` ### Providers - `aws`, version: `>= 4.0, < 6.0.0` - `sops`, version: `>= 0.5, < 1.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.destination`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.destination_ignored`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`sops_file.source`](https://registry.terraform.io/providers/carlpett/sops/latest/docs/data-sources/file) (data source) --- ## sso-saml-provider This component reads sso credentials from SSM Parameter store and provides them as outputs ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: sso-saml-provider: settings: spacelift: workspace_enabled: true vars: enabled: true ssm_path_prefix: "/sso/saml/google" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`ssm_path_prefix` (`string`) required
Top level SSM path prefix (without leading or trailing slash)
### Optional Variables
`emailAttr` (`string`) optional
Email attribute **Default value:** `null`
`groupsAttr` (`string`) optional
Group attribute **Default value:** `null`
`usernameAttr` (`string`) optional
User name attribute **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ca`
Raw signing certificate
`emailAttr`
Email attribute
`groupsAttr`
Groups attribute
`issuer`
Identity Provider Single Sign-On Issuer URL
`url`
Identity Provider Single Sign-On URL
`usernameAttr`
User name attribute
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `store_read` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## strongdm This component provisions [strongDM](https://www.strongdm.com/) gateway, relay and roles ## Usage **Stack Level**: Regional Use this in the catalog or use these variables to overwrite the catalog values. ```yaml components: terraform: strong-dm: vars: enabled: true ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`ssm_account` (`string`) required
Account (stage) housing SSM parameters
`ssm_region` (`string`) required
AWS Region housing SSM parameters
### Optional Variables
`create_roles` (`bool`) optional
Set `true` to create roles (should only be set in one account) **Default value:** `false`
`dns_zone` (`string`) optional
DNS zone (e.g. example.com) into which to install the web host. **Default value:** `null`
`gateway_count` (`number`) optional
Number of gateways to provision **Default value:** `2`
`install_gateway` (`bool`) optional
Set `true` to install a pair of gateways **Default value:** `false`
`install_relay` (`bool`) optional
Set `true` to install a pair of relays **Default value:** `true`
`kms_alias_name` (`string`) optional
AWS KMS alias used for encryption/decryption default is alias used in SSM **Default value:** `"alias/aws/ssm"`
`kubernetes_namespace` (`string`) optional
The Kubernetes namespace to install the release into. Defaults to `default`. **Default value:** `null`
`register_nodes` (`bool`) optional
Set `true` to register nodes as SSH targets **Default value:** `true`
`relay_count` (`number`) optional
Number of relays to provision **Default value:** `2`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional tags for appending to tags_as_list_of_maps. Not added to `tags`. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {} } ```
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for default, which is `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
The letter case of label keys (`tag` names) (i.e. `name`, `namespace`, `environment`, `stage`, `attributes`) to use in `tags`. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The naming order of the id output and Name tag. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 5 elements, but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
The letter case of output label values (also used in `tags` and `id`). Possible values: `lower`, `title`, `upper` and `none` (no transformation). Default value: `lower`. **Required:** No **Default value:** `null`
`name` (`string`) optional
Solution name, e.g. 'app' or 'jenkins' **Required:** No **Default value:** `null`
`namespace` (`string`) optional
Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit','XYZ')` **Required:** No **Default value:** `{ }`
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.0, < 6.0.0` - `helm`, version: `>= 2.2.0` - `sdm`, version: `>= 1.0.19` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` - `aws`, version: `>= 3.0, < 6.0.0` - `helm`, version: `>= 2.2.0` - `sdm`, version: `>= 1.0.19` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `iam_roles_network` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.gateway_tokens`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.relay_tokens`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.ssh_admin_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`helm_release.cleanup`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`helm_release.gateway`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`helm_release.node`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`helm_release.relay`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`sdm_node.gateway`](https://registry.terraform.io/providers/strongdm/sdm/latest/docs/resources/node) (resource) - [`sdm_node.relay`](https://registry.terraform.io/providers/strongdm/sdm/latest/docs/resources/node) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.api_access_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.api_secret_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.ssh_admin_token`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## tfstate-backend This component is responsible for provisioning an S3 Bucket and DynamoDB table that follow security best practices for usage as a Terraform backend. It also creates IAM roles for access to the Terraform backend. Once the initial S3 backend is configured, this component can create additional backends, allowing you to segregate them and control access to each backend separately. This may be desirable because any secret or sensitive information (such as generated passwords) that Terraform has access to gets stored in the Terraform state backend S3 bucket, so you may wish to restrict who can read the production Terraform state backend S3 bucket. However, perhaps counter-intuitively, all Terraform users require read access to the most sensitive accounts, such as `root` and `audit`, in order to read security configuration information, so careful planning is required when architecting backend splits. ## Prerequisites :::tip Part of cold start, so it has to initially be run with `SuperAdmin`, multiple times: to create the S3 bucket and then to move the state into it. Follow the guide **[here](https://docs.cloudposse.com/layers/accounts/tutorials/manual-configuration/#provision-tfstate-backend-component)** to get started. ::: - This component assumes you are using the `aws-teams` and `aws-team-roles` components. - Before the `account` and `account-map` components are deployed for the first time, you'll want to run this component with `access_roles_enabled` set to `false` to prevent errors due to missing IAM Role ARNs. This will enable only enough access to the Terraform state for you to finish provisioning accounts and roles. After those components have been deployed, you will want to run this component again with `access_roles_enabled` set to `true` to provide the complete access as configured in the stacks. ### Access Control For each backend, this module will create an IAM role with read/write access and, optionally, an IAM role with read-only access. You can configure who is allowed to assume these roles. - While read/write access is required for `terraform apply`, the created role only grants read/write access to the Terraform state, it does not grant permission to create/modify/destroy AWS resources. - Similarly, while the read-only role prohibits making changes to the Terraform state, it does not prevent anyone from making changes to AWS resources using a different role. - Many Cloud Posse components store information about resources they create in the Terraform state via their outputs, and many other components read this information from the Terraform state backend via the CloudPosse `remote-state` module and use it as part of their configuration. For example, the `account-map` component exists solely for the purpose of organizing information about the created AWS accounts and storing it in its Terraform state, making it available via `remote-state`. This means that you if you are going to restrict access to some backends, you need to carefully orchestrate what is stored there and ensure that you are not storing information a component needs in a backend it will not have access to. Typically, information in the most sensitive accounts, such as `root`, `audit`, and `security`, is nevertheless needed by every account, for example to know where to send audit logs, so it is not obvious and can be counter-intuitive which accounts need access to which backends. Plan carefully. - Atmos provides separate configuration for Terraform state access via the `backend` and `remote_state_backend` settings. Always configure the `backend` setting with a role that has read/write access (and override that setting to be `null` for components deployed by SuperAdmin). If a read-only role is available (only helpful if you have more than one backend), use that role in `remote_state_backend.s3.role_arn`. Otherwise, use the read/write role in `remote_state_backend.s3.role_arn`, to ensure that all components can read the Terraform state, even if `backend.s3.role_arn` is set to `null`, as it is with a few critical components meant to be deployed by SuperAdmin. - Note that the "read-only" in the "read-only role" refers solely to the S3 bucket that stores the backend data. That role still has read/write access to the DynamoDB table, which is desirable so that users restricted to the read-only role can still perform drift detection by running `terraform plan`. The DynamoDB table only stores checksums and mutual-exclusion lock information, so it is not considered sensitive. The worst a malicious user could do would be to corrupt the table and cause a denial-of-service (DoS) for Terraform, but such DoS would only affect making changes to the infrastructure, it would not affect the operation of the existing infrastructure, so it is an ineffective and therefore unlikely vector of attack. (Also note that the entire DynamoDB table is optional and can be deleted entirely; Terraform will repopulate it as new activity takes place.) - For convenience, the component automatically grants access to the backend to the user deploying it. This is helpful because it allows that user, presumably SuperAdmin, to deploy the normal components that expect the user does not have direct access to Terraform state, without requiring custom configuration. However, you may want to explicitly grant SuperAdmin access to the backend in the `allowed_principal_arns` configuration, to ensure that SuperAdmin can always access the backend, even if the component is later updated by the `root-admin` role. ### Quotas When allowing access to both SAML and AWS SSO users, the trust policy for the IAM roles created by this component can exceed the default 2048 character limit. If you encounter this error, you can increase the limit by requesting a quota increase [here](https://us-east-1.console.aws.amazon.com/servicequotas/home/services/iam/quotas/L-C07B4B0D). Note that this is the IAM limit on "The maximum number of characters in an IAM role trust policy" and it must be configured in the `us-east-1` region, regardless of what region you are deploying to. Normally 3072 characters is sufficient, and is recommended so that you still have room to expand the trust policy in the future while perhaps considering how to reduce its size. ## Usage **Stack Level**: Regional (because DynamoDB is region-specific), but deploy only in a single region and only in the `root` account **Deployment**: Must be deployed by SuperAdmin using `atmos` CLI This component configures the shared Terraform backend, and as such is the first component that must be deployed, since all other components depend on it. In fact, this component even depends on itself, so special deployment procedures are needed for the initial deployment (documented in the "Cold Start" procedures). Here's an example snippet for how to use this component. ```yaml terraform: tfstate-backend: backend: s3: role_arn: null settings: spacelift: workspace_enabled: false vars: enable_server_side_encryption: true enabled: true force_destroy: false name: tfstate prevent_unencrypted_uploads: true access_roles: default: &tfstate-access-template write_enabled: true allowed_roles: core-identity: ["devops", "developers", "managers", "spacelift"] core-root: ["admin"] denied_roles: {} allowed_permission_sets: core-identity: ["AdministratorAccess"] denied_permission_sets: {} allowed_principal_arns: [] denied_principal_arns: [] ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`access_roles` optional
Map of access roles to create (key is role name, use "default" for same as component). See iam-assume-role-policy module for details. **Type:** ```hcl map(object({ write_enabled = bool allowed_roles = map(list(string)) denied_roles = map(list(string)) allowed_principal_arns = list(string) denied_principal_arns = list(string) allowed_permission_sets = map(list(string)) denied_permission_sets = map(list(string)) })) ``` **Default value:** `{ }`
`access_roles_enabled` (`bool`) optional
Enable access roles to be assumed. Set `false` for cold start (before account-map has been created), because the role to ARN mapping has not yet been created. Note that the current caller and any `allowed_principal_arns` will always be allowed to assume the role. **Default value:** `true`
`dynamodb_enabled` (`bool`) optional
Whether to create the DynamoDB table. **Default value:** `true`
`enable_point_in_time_recovery` (`bool`) optional
Enable DynamoDB point-in-time recovery **Default value:** `true`
`force_destroy` (`bool`) optional
A boolean that indicates the terraform state S3 bucket can be destroyed even if it contains objects. These objects are not recoverable. **Default value:** `false`
`prevent_unencrypted_uploads` (`bool`) optional
Prevent uploads of unencrypted objects to S3 **Default value:** `true`
`s3_state_lock_enabled` (`bool`) optional
Whether to use S3 for state lock. If true, the DynamoDB table will not be created. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`tfstate_backend_access_role_arns`
IAM Role ARNs for accessing the Terraform State Backend
`tfstate_backend_dynamodb_table_arn`
Terraform state DynamoDB table ARN
`tfstate_backend_dynamodb_table_id`
Terraform state DynamoDB table ID
`tfstate_backend_dynamodb_table_name`
Terraform state DynamoDB table name
`tfstate_backend_s3_bucket_arn`
Terraform state S3 bucket ARN
`tfstate_backend_s3_bucket_domain_name`
Terraform state S3 bucket domain name
`tfstate_backend_s3_bucket_id`
Terraform state S3 bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` - `awsutils`, version: `>= 0.16.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `assume_role` | latest | [`../account-map/modules/team-assume-role-policy`](https://registry.terraform.io/modules/../account-map/modules/team-assume-role-policy/) | n/a `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `tfstate_backend` | 1.7.0 | [`cloudposse/tfstate-backend/aws`](https://registry.terraform.io/modules/cloudposse/tfstate-backend/aws/1.7.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) ## Data Sources The following data sources are used by this module: - [`aws_arn.cold_start_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/arn) (data source) - [`aws_iam_policy_document.cold_start_assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.tfstate`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`awsutils_caller_identity.current`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/data-sources/caller_identity) (data source) --- ## tgw AWS Transit Gateway connects your Amazon Virtual Private Clouds (VPCs) and on-premises networks through a central hub. This connection simplifies your network and puts an end to complex peering relationships. Transit Gateway acts as a highly scalable cloud router—each new connection is made only once. For more on Transit Gateway, see [the AWS documentation](https://aws.amazon.com/transit-gateway/). ## Requirements In order to connect accounts with Transit Gateway, we deploy Transit Gateway to a central account, typically `core-network`, and then deploy Transit Gateway attachments for each connected account. Each connected accounts needs a Transit Gateway attachment for the given account's VPC, either by VPC attachment or by Peering Connection attachment. Furthermore, each private subnet in each connected VPC needs to explicitly list the CIDRs for all allowed connections. ## Solution First we deploy the Transit Gateway Hub, `tgw/hub`, to a central network account. The component prepares the Transit Gateway network with the following steps: 1. Provision Transit Gateway in the network account 2. Collect VPC and EKS component output from every account connected to Transit Gateway 3. Share the Transit Gateway with the Organization using Resource Access Manager (RAM) By using the `tgw/hub` component to collect Terraform output from connected accounts, only this single component requires access to the Terraform state of all connected accounts. Next we deploy `tgw/spoke` to the network account and then to every connected account. This spoke component connects the given account to the central hub and any listed connection with the following steps: 1. Create a Transit Gateway VPC attachment in the spoke account. This connects the account's VPC to the shared Transit Gateway from the hub account. 2. Define all allowed routes for private subnets. Each private subnet in an account's VPC has it's own route table. This route table needs to explicitly list any allowed connection to another account's VPC CIDR. 3. (Optional) Create an EKS Cluster Security Group rule to allow traffic to the cluster in the given account. ## Implementation 1. Deploy `tgw/hub` to the network account. List every allowed connection: ```yaml # stacks/catalog/tgw/hub components: terraform: tgw/hub/defaults: metadata: type: abstract component: tgw/hub vars: enabled: true name: tgw-hub tags: Team: sre Service: tgw-hub tgw/hub: metadata: inherits: - tgw/hub/defaults component: tgw/hub vars: # These are all connections available for spokes in this region # Defaults environment to this region connections: - account: tenant: core stage: network - account: tenant: core stage: auto eks_component_names: - eks/cluster - account: tenant: plat stage: sandbox eks_component_names: [] # No clusters deployed for sandbox - account: tenant: plat stage: dev eks_component_names: - eks/cluster - account: tenant: plat stage: staging eks_component_names: - eks/cluster - account: tenant: plat stage: prod eks_component_names: - eks/cluster ``` 2. Deploy `tgw/spoke` to network. List every account connected to network (all accounts): ```yaml # stacks/catalog/tgw/spoke components: terraform: tgw/spoke-defaults: metadata: type: abstract component: tgw/spoke vars: enabled: true name: tgw-spoke tgw_hub_tenant_name: core tgw_hub_stage_name: network # default, added for visibility tags: Team: sre Service: tgw-spoke ``` ```yaml # stacks/orgs/acme/core/network/us-east-1/network.yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: # This is what THIS spoke is allowed to connect to connections: - account: tenant: core stage: network - account: tenant: core stage: auto - account: tenant: plat stage: sandbox - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod ``` 3. Finally, deploy `tgw/spoke` for each connected account and list the allowed connections: ```yaml # stacks/orgs/acme/plat/dev/us-east-1/network.yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: connections: # Always list self - account: tenant: plat stage: dev - account: tenant: core stage: network - account: tenant: core stage: auto ``` ### Alternate Regions In order to connect any account to the network, the given account needs: 1. Access to the shared Transit Gateway hub 2. An attachment for the given Transit Gateway hub 3. Routes to and from each private subnet However, sharing the Transit Gateway hub via RAM is only supported in the same region as the primary hub. Therefore, we must instead deploy a new hub in the alternate region and create a [Transit Gateway Peering Connection](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-peering.html) between the two Transit Gateway hubs. Furthermore, since this Transit Gateway hub for the alternate region is now peered, we must create a Peering Transit Gateway attachment, opposed to a VPC Transit Gateway Attachment. #### Cross Region Deployment 1. Deploy `tgw/hub` and `tgw/spoke` into the primary region as described in [Implementation](#implementation) 2. Deploy `tgw/hub` and `tgw/cross-region-hub` into the new region in the network account. See the following configuration: ```yaml # stacks/catalog/tgw/cross-region-hub import: - catalog/tgw/hub components: terraform: # Cross region TGW requires additional hub in the alternate region tgw/hub: vars: # These are all connections available for spokes in this region # Defaults environment to this region connections: # Hub for this region is always required - account: tenant: core stage: network # VPN source - account: tenant: core stage: network environment: use1 # Github Runners - account: tenant: core stage: auto environment: use1 eks_component_names: - eks/cluster # All stacks where a spoke will be deployed - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod # This alternate hub needs to be connected to the primary region's hub tgw/cross-region-hub-connector: vars: enabled: true primary_tgw_hub_region: us-east-1 ``` 3. Deploy a `tgw/spoke` for network in the new region. For example: ```yaml # stacks/orgs/acme/core/network/us-west-2/network.yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: peered_region: true # Required for alternate region spokes connections: # This stack, always included - account: tenant: core stage: network # VPN - account: tenant: core environment: use1 stage: network # Automation runners - account: tenant: core environment: use1 stage: auto eks_component_names: - eks/cluster # All other connections - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod ``` 4. Deploy the `tgw/spoke` components for all connected accounts. For example: ```yaml # stacks/orgs/acme/plat/dev/us-west-2/network.yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: peered_region: true # Required for alternate region spokes connections: # This stack, always included - account: tenant: plat stage: dev # TGW Hub, always included - account: tenant: core stage: network # VPN - account: tenant: core environment: use1 stage: network # Automation runners - account: tenant: core environment: use1 stage: auto eks_component_names: - eks/cluster ``` 5. Update any existing `tgw/spoke` connections to allow the new account and region. For example: ```yaml # stacks/orgs/acme/core/auto/us-east-1/network.yaml tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: connections: - account: tenant: core stage: network - account: tenant: core stage: corp - account: tenant: core stage: auto - account: tenant: plat stage: sandbox - account: tenant: plat stage: dev - account: tenant: plat stage: staging - account: tenant: plat stage: prod # Alternate regions <-------- These are added for alternate region - account: tenant: core stage: network environment: usw2 - account: tenant: plat stage: dev environment: usw2 - account: tenant: plat stage: staging environment: usw2 - account: tenant: plat stage: prod environment: usw2 ``` ## Destruction When destroying Transit Gateway components, order of operations matters. Always destroy any removed `tgw/spoke` components before removing a connection from the `tgw/hub` component. The `tgw/hub` component creates map of VPC resources that each `tgw/spoke` component references. If the required reference is removed before the `tgw/spoke` is destroyed, Terraform will fail to destroy the given `tgw/spoke` component. :::info Pro Tip! [Atmos Workflows](https://atmos.tools/core-concepts/workflows/) make applying and destroying Transit Gateway much easier! For example, to destroy components in the correct order, use a workflow similar to the following: ```yaml # stacks/workflows/network.yaml workflows: destroy/tgw: description: Destroy the Transit Gateway "hub" and "spokes" for connecting VPCs. steps: - command: echo 'Destroying platform spokes for Transit Gateway' type: shell name: plat-spokes - command: terraform destroy tgw/spoke -s plat-use1-sandbox --auto-approve - command: terraform destroy tgw/spoke -s plat-use1-dev --auto-approve - command: terraform destroy tgw/spoke -s plat-use1-staging --auto-approve - command: terraform destroy tgw/spoke -s plat-use1-prod --auto-approve - command: echo 'Destroying core spokes for Transit Gateway' type: shell name: core-spokes - command: terraform destroy tgw/spoke -s core-use1-auto --auto-approve - command: terraform destroy tgw/spoke -s core-use1-network --auto-approve - command: echo 'Destroying Transit Gateway Hub' type: shell name: hub - command: terraform destroy tgw/hub -s core-use1-network --auto-approve ``` ::: # FAQ ## `tgw/spoke` Fails to Recreate VPC Attachment with `DuplicateTransitGatewayAttachment` Error ```bash ╷ │ Error: creating EC2 Transit Gateway VPC Attachment: DuplicateTransitGatewayAttachment: tgw-0xxxxxxxxxxxxxxxx has non-deleted Transit Gateway Attachments with same VPC ID. │ status code: 400, request id: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee │ │ with module.tgw_spoke_vpc_attachment.module.standard_vpc_attachment.aws_ec2_transit_gateway_vpc_attachment.default["core-use2-network"], │ on .terraform/modules/tgw_spoke_vpc_attachment.standard_vpc_attachment/main.tf line 43, in resource "aws_ec2_transit_gateway_vpc_attachment" "default": │ 43: resource "aws_ec2_transit_gateway_vpc_attachment" "default" { │ ╵ Releasing state lock. This may take a few moments... exit status 1 ``` This is caused by Terraform attempting to create the replacement VPC attachment before the original is completely destroyed. Retry the apply. Now you should see only "create" actions. # Changelog ## Upgrading to `v1.276.0` Components PR [#804](https://github.com/cloudposse/terraform-aws-components/pull/804) ### Affected Components - `tgw/hub` - `tgw/spoke` - `tgw/cross-region-hub-connector` ### Summary This change to the Transit Gateway components, [PR #804](https://github.com/cloudposse/terraform-aws-components/pull/804), added support for cross-region connections. As part of that change, we've added `environment` to the component identifier used in the Terraform Output created by `tgw/hub`. Because of that map key change, all resources in Terraform now have a new resource identifier and therefore must be recreated with Terraform or removed from state and imported into the new resource ID. Recreating the resources is the easiest solution but means that Transit Gateway connectivity will be lost while the changes apply, which typically takes an hour. Alternatively, removing the resources from state and importing back into the new resource ID is much more complex operationally but means no lost Transit Gateway connectivity. Since we use Transit Gateway for VPN and GitHub Automation runner access, a temporarily lost connection is not a significant concern, so we choose to accept lost connectivity and recreate all `tgw/spoke` resources. ### Steps 1. Notify your team of a temporary VPN and Automation outage for accessing private networks 2. Deploy all `tgw/hub` components. There should be a hub component in each region of your network account connected to Transit Gateway 3. Deploy all `tgw/spoke` components. There should be a spoke component in every account and every region connected to Transit Gateway #### Tips Use workflows to deploy `tgw` across many accounts with a single command: ```bash atmos workflow deploy/tgw -f network ``` ```yaml # stacks/workflows/network.yaml workflows: deploy/tgw: description: Provision the Transit Gateway "hub" and "spokes" for connecting VPCs. steps: - command: terraform deploy tgw/hub -s core-use1-network name: hub - command: terraform deploy tgw/spoke -s core-use1-network - command: echo 'Creating core spokes for Transit Gateway' type: shell name: core-spokes - command: terraform deploy tgw/spoke -s core-use1-corp - command: terraform deploy tgw/spoke -s core-use1-auto - command: terraform deploy tgw/spoke -s plat-use1-sandbox - command: echo 'Creating platform spokes for Transit Gateway' type: shell name: plat-spokes - command: terraform deploy tgw/spoke -s plat-use1-dev - command: terraform deploy tgw/spoke -s plat-use1-staging - command: terraform deploy tgw/spoke -s plat-use1-prod ``` --- ## attachment This component creates a Transit Gateway VPC Attachment and optionally creates an association with a Transit Gateway Route Table. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: tgw/attachment: vars: enabled: true transit_gateway_id: !terraform.output tgw/hub core-use1-network transit_gateway_id transit_gateway_route_table_id: !terraform.output tgw/hub core-use1-network transit_gateway_route_table_id create_transit_gateway_route_table_association: false ``` #### Transit Gateway Route Table Association In the primary account, the account that has the Transit Gateway and the Transit Gateway Route Table, we need to create an association with the Transit Gateway Route Table. This is necessary for attachments to connect to the Transit Gateway Route Table. For example, if you have a Transit Gateway Route Table in the _core-network_ account, you will need to create an association for each VPCs connected to that Transit Gateway Route Table. The intention is to have all configuration for a given account in the same stack as that account. For example, since the Transit Gateway Route Table is in the _core-network_ account, we would create all necessary associations in the _core-network_ account. ```yaml # core-network stack components: terraform: tgw/attachment: vars: enabled: true transit_gateway_id: !terraform.output tgw/hub core-usw2-network transit_gateway_id transit_gateway_route_table_id: !terraform.output tgw/hub core-usw2-network transit_gateway_route_table_id # Add an association for this account itself create_transit_gateway_route_table_association: true # Include association for each of the connected accounts, if necessary additional_associations: - attachment_id: !terraform.output tgw/attachment plat-usw2-dev transit_gateway_attachment_id route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id - attachment_id: !terraform.output tgw/attachment plat-usw2-prod transit_gateway_attachment_id route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id ``` In connected accounts, an account that does _not_ have a Transit Gateway and Transit Gateway Route Table, you do not need to create any associations. ```yaml # plat-dev stack components: terraform: tgw/attachment: vars: enabled: true transit_gateway_id: !terraform.output tgw/hub core-usw2-network transit_gateway_id transit_gateway_route_table_id: !terraform.output tgw/hub core-usw2-network transit_gateway_route_table_id # Do not create an association in this account since there is no Transit Gateway Route Table in this account. create_transit_gateway_route_table_association: false ``` Plus the same for all other connected accounts. ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`transit_gateway_id` (`string`) required
ID of the Transit Gateway to attach to
`transit_gateway_route_table_id` (`string`) required
ID of the Transit Gateway Route Table
### Optional Variables
`private_subnet_ids` (`list(string)`) optional
IDs of the private subnets to attach to. Required if `vpc_id` is defined. **Default value:** `[ ]`
`vpc_component_name` (`string`) optional
The name of the vpc component **Default value:** `"vpc"`
`vpc_id` (`string`) optional
ID of the VPC to attach to **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`transit_gateway_vpc_attachment_id`
ID of the Transit Gateway VPC Attachment
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `standard_vpc_attachment` | 0.13.0 | [`cloudposse/transit-gateway/aws`](https://registry.terraform.io/modules/cloudposse/transit-gateway/aws/0.13.0) | Create a TGW attachment from this account's VPC to the TGW Hub `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## hub This component is responsible for provisioning an [AWS Transit Gateway](https://aws.amazon.com/transit-gateway) `hub` that acts as a centralized gateway for connecting VPCs from other `spoke` accounts. ## Usage **Stack Level**: Regional ## Basic Usage with `tgw/spoke` Here's an example snippet for how to configure and use this component: ```yaml components: terraform: tgw/hub/defaults: metadata: type: abstract component: tgw/hub vars: enabled: true name: tgw-hub expose_eks_sg: false tags: Team: sre Service: tgw-hub tgw/hub: metadata: inherits: - tgw/hub/defaults component: tgw/hub vars: connections: - account: tenant: core stage: network vpc_component_names: - vpc-dev - account: tenant: core stage: artifacts - account: tenant: core stage: auto eks_component_names: - eks/cluster - account: tenant: plat stage: dev vpc_component_names: - vpc - vpc/data/1 eks_component_names: - eks/cluster - account: tenant: plat stage: staging vpc_component_names: - vpc - vpc/data/1 eks_component_names: - eks/cluster - account: tenant: plat stage: prod vpc_component_names: - vpc - vpc/data/1 eks_component_names: - eks/cluster ``` To provision the Transit Gateway and all related resources, run the following commands: ```sh atmos terraform plan tgw/hub -s --network atmos terraform apply tgw/hub -s --network ``` ## Alternate Usage with `tgw/attachment`, `tgw/routes`, and `vpc/routes` ### Components Overview - **`tgw/hub`**: Creates the Transit Gateway in the network account - **`tgw/attachment`**: Creates and manages Transit Gateway VPC attachments in connected accounts - **`tgw/hub-connection`**: Creates the Transit Gateway peering connection between two `tgw/hub` deployments - **`tgw/routes`**: Manages Transit Gateway route tables in the network account - **`vpc-routes`** (`vpc/routes/private`): Configures VPC route tables in connected accounts to route traffic through the Transit Gateway (Note: This component lives outside the `tgw/` directory since it's not specific to Transit Gateway) ### Architecture The Transit Gateway components work together in the following way: 1. Transit Gateway is created in the network account (`tgw/hub`) 2. VPCs in other accounts attach to the Transit Gateway (`tgw/attachment`) 3. Route tables in connected VPCs direct traffic across accounts (`vpc-routes`) 4. Transit Gateway route tables control routing between attachments (`tgw/routes`) ```mermaid graph TD subgraph core-use1-network TGW[Transit Gateway] TGW_RT[TGW Route Tables] end subgraph plat-use1-dev VPC1[VPC] VPC1_RT[VPC Route Tables] ATT1[TGW Attachment] end subgraph core-use1-auto VPC2[VPC] VPC2_RT[VPC Route Tables] ATT2[TGW Attachment] end ATT1 <--> TGW ATT2 <--> TGW TGW <--> TGW_RT VPC1_RT <--> VPC1 VPC2_RT <--> VPC2 VPC1 <--> ATT1 VPC2 <--> ATT2 ``` ### Deployment Steps #### 1. Deploy Transit Gateway Hub First, create the Transit Gateway in the network account. :::tip Leave `var.connections` empty. With this refactor, the `tgw/hub` component is only responsible for creating the Transit Gateway and its route tables. We do not need to fetch and store outputs for the connected components anymore. ::: ```yaml components: terraform: tgw/hub: vars: connections: [] ``` #### 2. Deploy VPC Attachments Important: Deploy attachments in connected accounts first, before deploying attachments in the network account. ##### Connected Account Attachments ```yaml components: terraform: tgw/attachment: vars: transit_gateway_id: !terraform.output tgw/hub core-use1-network transit_gateway_id transit_gateway_route_table_id: !terraform.output tgw/hub core-use1-network transit_gateway_route_table_id create_transit_gateway_route_table_association: false ``` ##### Network Account Attachment ```yaml components: terraform: tgw/attachment: vars: transit_gateway_id: !terraform.output tgw/hub core-use1-network transit_gateway_id transit_gateway_route_table_id: !terraform.output tgw/hub core-use1-network transit_gateway_route_table_id # Route table associations are required so that route tables can propagate their routes to other route tables. # Set the following to true in the same account where the Transit Gateway and its route tables are deployed create_transit_gateway_route_table_association: true # Associate connected accounts with the Transit Gateway route table additional_associations: - attachment_id: !terraform.output tgw/attachment core-use1-auto transit_gateway_vpc_attachment_id route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id - attachment_id: !terraform.output tgw/attachment plat-use1-dev transit_gateway_vpc_attachment_id route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id ``` #### 3. Configure VPC Routes Configure routes in all connected VPCs. ```yaml components: terraform: vpc/routes/private: metadata: component: vpc-routes vars: route_table_ids: !terraform.output vpc private_route_table_ids routes: # Route to network account - destination: cidr_block: !terraform.output vpc core-use1-network vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub core-use1-network transit_gateway_id # Route to core-auto account, if necessary - destination: cidr_block: !terraform.output vpc core-use1-auto vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub core-use1-network transit_gateway_id ``` Configure routes in the Network Account VPCs. ```yaml components: terraform: vpc/routes/private: vars: route_table_ids: !terraform.output vpc private_route_table_ids routes: # Routes to connected accounts - destination: cidr_block: !terraform.output vpc core-use1-auto vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub transit_gateway_id - destination: cidr_block: !terraform.output vpc plat-use1-dev vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub transit_gateway_id ``` ### 4. Deploy Transit Gateway Route Table Routes Deploy the `tgw/routes` component in the network account to create route tables and routes. ```yaml components: terraform: tgw/routes: vars: transit_gateway_route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id # Use propagated routes to route through VPC attachments propagated_routes: # Route to this account - attachment_id: !terraform.output tgw/attachment core-use1-network transit_gateway_attachment_id # Route to any connected account - attachment_id: !terraform.output tgw/attachment core-use1-auto transit_gateway_attachment_id - attachment_id: !terraform.output tgw/attachment plat-use1-dev transit_gateway_attachment_id ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`allow_external_principals` (`bool`) optional
Set true to allow the TGW to be RAM shared with external principals specified in ram_principals **Default value:** `false`
`connections` optional
A list of objects to define each TGW connections. By default, each connection will look for only the default `vpc` component. **Type:** ```hcl list(object({ account = object({ stage = string environment = optional(string, "") tenant = optional(string, "") }) vpc_component_names = optional(list(string), ["vpc"]) eks_component_names = optional(list(string), []) })) ``` **Default value:** `[ ]`
`expose_eks_sg` (`bool`) optional
Set true to allow EKS clusters to accept traffic from source accounts **Default value:** `true`
`ram_principals` (`list(string)`) optional
A list of AWS account IDs to share the TGW with outside the organization **Default value:** `[ ]`
`ram_resource_share_enabled` (`bool`) optional
Enable Resource Access Managment Share **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`eks`
Accounts with EKS and EKSs information
`tgw_config`
Transit Gateway config
`transit_gateway_arn`
Transit Gateway ARN
`transit_gateway_id`
Transit Gateway ID
`transit_gateway_route_table_id`
Transit Gateway route table ID
`vpcs`
Accounts with VPC and VPCs information
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `eks` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `tgw_hub` | 0.13.0 | [`cloudposse/transit-gateway/aws`](https://registry.terraform.io/modules/cloudposse/transit-gateway/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a --- ## hub-connector This component is responsible for provisioning an [AWS Transit Gateway Peering Connection](https://aws.amazon.com/transit-gateway) to connect TGWs from different accounts and(or) regions. Transit Gateway does not support sharing the Transit Gateway hub across regions. You must deploy a Transit Gateway hub for each region and connect the alternate hub to the primary hub. ## Usage **Stack Level**: Regional This component is deployed to each alternate region with `tgw/hub`. For example if your primary region is `us-east-1` and your alternate region is `us-west-2`, deploy another `tgw/hub` in `us-west-2` and peer the two with `tgw/cross-region-hub-connector` with the following stack config, imported into `us-west-2`: ```yaml import: - catalog/tgw/hub components: terraform: # Cross region TGW requires additional hub in the alternate region tgw/hub: vars: # These are all connections available for spokes in this region # Defaults environment to this region connections: # Hub for this region is always required - account: tenant: core stage: network # VPN source - account: tenant: core stage: network environment: use1 # Github Runners - account: tenant: core stage: auto environment: use1 eks_component_names: - eks/cluster # All stacks where a spoke will be deployed - account: tenant: plat stage: dev eks_component_names: [] # Add clusters here once deployed # This alternate hub needs to be connected to the primary region's hub tgw/cross-region-hub-connector: vars: enabled: true primary_tgw_hub_region: us-east-1 ``` ## Variables ### Required Variables
`primary_tgw_hub_region` (`string`) required
The name of the AWS region where the primary Transit Gateway hub is deployed. This value is used with `var.env_naming_convention` to determine the primary Transit Gateway hub's environment name.
`region` (`string`) required
AWS Region
### Optional Variables
`account_map_component_name` (`string`) optional
The name of the account-map component **Default value:** `"account-map"`
`account_map_environment_name` (`string`) optional
The name of the environment where `account_map` is provisioned **Default value:** `"gbl"`
`account_map_stage_name` (`string`) optional
The name of the stage where `account_map` is provisioned **Default value:** `"root"`
`account_map_tenant_name` (`string`) optional
The name of the tenant where `account_map` is provisioned **Default value:** `"core"`
`env_naming_convention` (`string`) optional
The cloudposse/utils naming convention used to translate environment name to AWS region name. Options are `to_short` and `to_fixed` **Default value:** `"to_short"`
`primary_tgw_hub_stage` (`string`) optional
The name of the stage where the primary Transit Gateway hub is deployed. Defaults to `module.this.stage` **Default value:** `""`
`primary_tgw_hub_tenant` (`string`) optional
The name of the tenant where the primary Transit Gateway hub is deployed. Only used if tenants are deployed and defaults to `module.this.tenant` **Default value:** `""`
`tgw_hub_primary_region_component_name` (`string`) optional
The component name of the tgw hub in the primary region **Default value:** `"tgw/hub"`
`tgw_hub_this_region_component_name` (`string`) optional
The component name of the tgw hub in this region **Default value:** `"tgw/hub"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_ec2_transit_gateway_peering_attachment_id`
Transit Gateway Peering Attachment ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` - `utils`, version: `>= 1.10.0` ### Providers - `aws`, version: `>= 4.1, < 6.0.0` - `aws`, version: `>= 4.1, < 6.0.0` - `utils`, version: `>= 1.10.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `account_map` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `tgw_hub_primary_region` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `tgw_hub_this_region` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | Used to translate region to environment ## Resources The following resources are used by this module: - [`aws_ec2_transit_gateway_peering_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_peering_attachment) (resource) - [`aws_ec2_transit_gateway_peering_attachment_accepter.primary_region`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_peering_attachment_accepter) (resource) - [`aws_ec2_transit_gateway_route_table_association.primary_region`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_association) (resource) - [`aws_ec2_transit_gateway_route_table_association.this_region`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_association) (resource) ## Data Sources The following data sources are used by this module: - [`utils_component_config.primary_tgw_hub`](https://registry.terraform.io/providers/cloudposse/utils/latest/docs/data-sources/component_config) (data source) --- ## routes Manages AWS Transit Gateway (TGW) route tables, including static routes and route propagation from VPC attachments. Enables controlled routing between VPCs connected to a TGW by configuring TGW route table associations, propagations, and explicit routes as needed. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: tgw/routes: vars: transit_gateway_route_table_id: "tgw-rtb-0123456789abcdef0" # Static routes for specific CIDR blocks static_routes: - cidr_block: "10.100.0.0/16" attachment_id: "tgw-attach-0123456789abcdef0" # Route propagation from VPC attachments propagated_routes: - attachment_id: "tgw-attach-0123456789abcdef1" ``` The same configuration using terraform outputs: ```yaml components: terraform: tgw/routes: vars: transit_gateway_route_table_id: !terraform.output tgw/hub transit_gateway_route_table_id # Static routes for specific CIDR blocks static_routes: - cidr_block: !terraform.output vpc edge-vpc vpc_cidr attachment_id: !terraform.output tgw/attachment edge-vpc transit_gateway_attachment_id # Route propagation from VPC attachments propagated_routes: - attachment_id: !terraform.output tgw/attachment app-vpc transit_gateway_attachment_id ``` ### Multiple Environment Example For environments with multiple routing requirements, here's an example using physical IDs: ```yaml components: terraform: tgw/routes/nonprod: metadata: component: tgw/routes vars: transit_gateway_route_table_id: "tgw-rtb-0123456789abcdef1" # Static routes for specific destinations static_routes: - cidr_block: "10.20.0.0/16" attachment_id: "tgw-attach-0123456789abcdef2" - cidr_block: "10.30.0.0/16" attachment_id: "tgw-attach-0123456789abcdef3" # Enable route propagation from specific VPCs propagated_routes: - attachment_id: "tgw-attach-0123456789abcdef4" - attachment_id: "tgw-attach-0123456789abcdef5" ``` The same configuration using terraform outputs: ```yaml components: terraform: tgw/routes/nonprod: metadata: component: tgw/routes vars: transit_gateway_route_table_id: !terraform.output tgw/hub transit-use1-nonprod transit_gateway_route_table_id # Static routes for specific destinations static_routes: - cidr_block: !terraform.output vpc dev-use1-edge vpc_cidr attachment_id: !terraform.output tgw/attachment dev-use1-edge transit_gateway_attachment_id - cidr_block: !terraform.output vpc staging-use1-edge vpc_cidr attachment_id: !terraform.output tgw/attachment staging-use1-edge transit_gateway_attachment_id # Enable route propagation from specific VPCs propagated_routes: - attachment_id: !terraform.output tgw/attachment dev-use1-network transit_gateway_attachment_id - attachment_id: !terraform.output tgw/attachment staging-use1-network transit_gateway_attachment_id ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`transit_gateway_route_tables` optional
List of Transit Gateway route table configurations **Type:** ```hcl list(object({ transit_gateway_route_table_id = string routes = list(object({ attachment_id = string propagated = optional(bool, true) associated = optional(bool, true) })) static_routes = optional(list(object({ cidr_block = string attachment_id = string blackhole = optional(bool, false) })), []) })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`mock`
Mock output example for the Cloud Posse Terraform component template
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Providers - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ec2_transit_gateway_route.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route) (resource) - [`aws_ec2_transit_gateway_route_table_association.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_association) (resource) - [`aws_ec2_transit_gateway_route_table_propagation.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_propagation) (resource) ## Data Sources The following data sources are used by this module: --- ## spoke This component is responsible for provisioning [AWS Transit Gateway](https://aws.amazon.com/transit-gateway) attachments to connect VPCs in a `spoke` account to different accounts through a central `hub`. ## Usage **Stack Level**: Regional Here's an example snippet for how to configure and use this component: stacks/catalog/tgw/spoke.yaml ```yaml components: terraform: tgw/spoke-defaults: metadata: type: abstract component: tgw/spoke vars: enabled: true name: tgw-spoke tags: Team: sre Service: tgw-spoke expose_eks_sg: false tgw_hub_tenant_name: core tgw_hub_stage_name: network tgw/spoke: metadata: inherits: - tgw/spoke-defaults vars: # This is what THIS spoke is allowed to connect to. # since this is deployed to each plat account (dev->prod), # we allow connections to network and auto. connections: - account: tenant: core stage: network # Set this value if the vpc component has a different name in this account vpc_component_names: - vpc-dev - account: tenant: core stage: auto ``` stacks/ue2/dev.yaml ```yaml import: - catalog/tgw/spoke components: terraform: tgw/spoke: vars: # use when there is not an EKS cluster in the stack expose_eks_sg: false # override default connections connections: - account: tenant: core stage: network vpc_component_names: - vpc-dev - account: tenant: core stage: auto - account: tenant: plat stage: dev eks_component_names: - eks/cluster - account: tenant: plat stage: qa eks_component_names: - eks/cluster ``` To provision the attachments for a spoke account: ```sh atmos terraform plan tgw/spoke -s -- atmos terraform apply tgw/spoke -s -- ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`connections` optional
A list of objects to define each TGW connections. By default, each connection will look for only the default `vpc` component. **Type:** ```hcl list(object({ account = object({ stage = string environment = optional(string, "") tenant = optional(string, "") }) vpc_component_names = optional(list(string), ["vpc"]) eks_component_names = optional(list(string), []) })) ``` **Default value:** `[ ]`
`cross_region_hub_connector_components` (`map(object({ component = string, environment = string }))`) optional
A map of cross-region hub connector components that provide this spoke with the appropriate Transit Gateway attachments IDs. - The key should be the environment that the remote VPC is located in. - The component is the name of the component in the remote region (e.g. `tgw/cross-region-hub-connector`) - The environment is the region that the cross-region-hub-connector is deployed in. e.g. the following would configure a component called `tgw/cross-region-hub-connector/use1` that is deployed in the If use2 is the primary region, the following would be its configuration: use1: component: "tgw/cross-region-hub-connector" environment: "use1" (the remote region) and in the alternate region, the following would be its configuration: use2: component: "tgw/cross-region-hub-connector" environment: "use1" (our own region) **Default value:** `{ }`
`default_route_enabled` (`bool`) optional
Enable default routing via transit gateway, requires also nat gateway and instance to be disabled in vpc component. Default is disabled. **Default value:** `false`
`default_route_outgoing_account_name` (`string`) optional
The account name which is used for outgoing traffic, when using the transit gateway as default route. **Default value:** `null`
`expose_eks_sg` (`bool`) optional
Set true to allow EKS clusters to accept traffic from source accounts **Default value:** `true`
`own_eks_component_names` (`list(string)`) optional
The name of the eks components in the owning account. **Default value:** `[ ]`
`own_vpc_component_name` (`string`) optional
The name of the vpc component in the owning account. Defaults to "vpc" **Default value:** `"vpc"`
`peered_region` (`bool`) optional
Set `true` if this region is not the primary region **Default value:** `false`
`static_routes` optional
A list of static routes to add to the transit gateway, pointing at this VPC as a destination. **Type:** ```hcl set(object({ blackhole = bool destination_cidr_block = string })) ``` **Default value:** `[ ]`
`static_tgw_routes` (`list(string)`) optional
A list of static routes to add to the local routing table with the transit gateway as a destination. **Default value:** `[ ]`
`tgw_hub_component_name` (`string`) optional
The name of the transit-gateway component **Default value:** `"tgw/hub"`
`tgw_hub_stage_name` (`string`) optional
The name of the stage where `tgw/hub` is provisioned **Default value:** `"network"`
`tgw_hub_tenant_name` (`string`) optional
The name of the tenant where `tgw/hub` is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Providers - `aws`, version: `>= 4.1, < 6.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cross_region_hub_connector` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `iam_roles` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `tgw_hub` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `tgw_hub_role` | latest | [`../../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../../account-map/modules/iam-roles/) | n/a `tgw_hub_routes` | 0.13.0 | [`cloudposse/transit-gateway/aws`](https://registry.terraform.io/modules/cloudposse/transit-gateway/aws/0.13.0) | n/a `tgw_spoke_vpc_attachment` | latest | [`./modules/standard_vpc_attachment`](https://registry.terraform.io/modules/./modules/standard_vpc_attachment/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_route.back_route`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.default_route`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) ## Data Sources The following data sources are used by this module: --- ## vpc This component is responsible for provisioning a VPC and corresponding Subnets. Additionally, VPC Flow Logs can optionally be enabled for auditing purposes. See the existing VPC configuration documentation for the provisioned subnets. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml # catalog/vpc/defaults or catalog/vpc components: terraform: vpc/defaults: metadata: type: abstract component: vpc settings: spacelift: workspace_enabled: true vars: enabled: true name: vpc availability_zones: - "a" - "b" - "c" nat_gateway_enabled: true nat_instance_enabled: false max_subnet_count: 3 vpc_flow_logs_enabled: true vpc_flow_logs_bucket_environment_name: vpc_flow_logs_bucket_stage_name: audit vpc_flow_logs_traffic_type: "ALL" subnet_type_tag_key: "example.net/subnet/type" assign_generated_ipv6_cidr_block: true ``` ```yaml import: - catalog/vpc components: terraform: vpc: metadata: component: vpc inherits: - vpc/defaults vars: ipv4_primary_cidr_block: "10.111.0.0/18" ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`subnet_type_tag_key` (`string`) required
Key for subnet type tag to provide information about the type of subnets, e.g. `cpco/subnet/type=private` or `cpcp/subnet/type=public`
### Optional Variables
`assign_generated_ipv6_cidr_block` (`bool`) optional
When `true`, assign AWS generated IPv6 CIDR block to the VPC. Conflicts with `ipv6_ipam_pool_id`. **Default value:** `false`
`availability_zone_ids` (`list(string)`) optional
List of Availability Zones IDs where subnets will be created. Overrides `availability_zones`. Can be the full name, e.g. `use1-az1`, or just the part after the AZ ID region code, e.g. `-az1`, to allow reusable values across regions. Consider contention for resources and spot pricing in each AZ when selecting. Useful in some regions when using only some AZs and you want to use the same ones across multiple accounts. **Default value:** `[ ]`
`availability_zones` (`list(string)`) optional
List of Availability Zones (AZs) where subnets will be created. Ignored when `availability_zone_ids` is set. Can be the full name, e.g. `us-east-1a`, or just the part after the region, e.g. `a` to allow reusable values across regions. The order of zones in the list ***must be stable*** or else Terraform will continually make changes. If no AZs are specified, then `max_subnet_count` AZs will be selected in alphabetical order. If `max_subnet_count > 0` and `length(var.availability_zones) > max_subnet_count`, the list will be truncated. We recommend setting `availability_zones` and `max_subnet_count` explicitly as constant (not computed) values for predictability, consistency, and stability. **Default value:** `[ ]`
`gateway_vpc_endpoints` (`set(string)`) optional
A list of Gateway VPC Endpoints to provision into the VPC. Only valid values are "dynamodb" and "s3". **Default value:** `[ ]`
`interface_vpc_endpoints` (`set(string)`) optional
A list of Interface VPC Endpoints to provision into the VPC. **Default value:** `[ ]`
`ipv4_additional_cidr_block_associations` optional
IPv4 CIDR blocks to assign to the VPC. `ipv4_cidr_block` can be set explicitly, or set to `null` with the CIDR block derived from `ipv4_ipam_pool_id` using `ipv4_netmask_length`. Map keys must be known at `plan` time, and are only used to track changes. **Type:** ```hcl map(object({ ipv4_cidr_block = string ipv4_ipam_pool_id = string ipv4_netmask_length = number })) ``` **Default value:** `{ }`
`ipv4_cidr_block_association_timeouts` optional
Timeouts (in `go` duration format) for creating and destroying IPv4 CIDR block associations **Type:** ```hcl object({ create = string delete = string }) ``` **Default value:** `null`
`ipv4_cidrs` optional
Lists of CIDRs to assign to subnets. Order of CIDRs in the lists must not change over time. Lists may contain more CIDRs than needed. **Type:** ```hcl list(object({ private = list(string) public = list(string) })) ``` **Default value:** `[ ]`
`ipv4_primary_cidr_block` (`string`) optional
The primary IPv4 CIDR block for the VPC. Either `ipv4_primary_cidr_block` or `ipv4_primary_cidr_block_association` must be set, but not both. **Default value:** `null`
`ipv4_primary_cidr_block_association` optional
Configuration of the VPC's primary IPv4 CIDR block via IPAM. Conflicts with `ipv4_primary_cidr_block`. One of `ipv4_primary_cidr_block` or `ipv4_primary_cidr_block_association` must be set. Additional CIDR blocks can be set via `ipv4_additional_cidr_block_associations`. **Type:** ```hcl object({ ipv4_ipam_pool_id = string ipv4_netmask_length = number }) ``` **Default value:** `null`
`map_public_ip_on_launch` (`bool`) optional
Instances launched into a public subnet should be assigned a public IP address **Default value:** `true`
`max_nats` (`number`) optional
Upper limit on number of NAT Gateways/Instances to create. Set to 1 or 2 for cost savings at the expense of availability. Default creates a NAT Gateway in each public subnet. **Default value:** `null`
`max_subnet_count` (`number`) optional
Sets the maximum amount of subnets to deploy. 0 will deploy a subnet for every provided availability zone (in `region_availability_zones` variable) within the region **Default value:** `0`
`nat_eip_aws_shield_protection_enabled` (`bool`) optional
Enable or disable AWS Shield Advanced protection for NAT EIPs. If set to 'true', a subscription to AWS Shield Advanced must exist in this account. **Default value:** `false`
`nat_gateway_enabled` (`bool`) optional
Flag to enable/disable NAT gateways **Default value:** `true`
`nat_instance_ami_id` (`list(string)`) optional
A list optionally containing the ID of the AMI to use for the NAT instance. If the list is empty (the default), the latest official AWS NAT instance AMI will be used. NOTE: The Official NAT instance AMI is being phased out and does not support NAT64. Use of a NAT gateway is recommended instead. **Default value:** `[ ]`
`nat_instance_enabled` (`bool`) optional
Flag to enable/disable NAT instances **Default value:** `false`
`nat_instance_type` (`string`) optional
NAT Instance type **Default value:** `"t3.micro"`
`public_subnets_enabled` (`bool`) optional
If false, do not create public subnets. Since NAT gateways and instances must be created in public subnets, these will also not be created when `false`. **Default value:** `true`
`subnets_per_az_count` (`number`) optional
The number of subnet of each type (public or private) to provision per Availability Zone. **Default value:** `1`
`subnets_per_az_names` (`list(string)`) optional
The subnet names of each type (public or private) to provision per Availability Zone. This variable is optional. If a list of names is provided, the list items will be used as keys in the outputs `named_private_subnets_map`, `named_public_subnets_map`, `named_private_route_table_ids_map` and `named_public_route_table_ids_map` **Default value:** ```hcl [ "common" ] ```
`vpc_flow_logs_bucket_component_name` (`string`) optional
The name of the VPC flow logs bucket component **Default value:** `"vpc-flow-logs-bucket"`
`vpc_flow_logs_bucket_environment_name` (`string`) optional
The name of the environment where the VPC Flow Logs bucket is provisioned **Default value:** `""`
`vpc_flow_logs_bucket_stage_name` (`string`) optional
The stage (account) name where the VPC Flow Logs bucket is provisioned **Default value:** `""`
`vpc_flow_logs_bucket_tenant_name` (`string`) optional
The name of the tenant where the VPC Flow Logs bucket is provisioned. If the `tenant` label is not used, leave this as `null`. **Default value:** `null`
`vpc_flow_logs_destination_options_file_format` (`string`) optional
VPC Flow Logs file format **Default value:** `"parquet"`
`vpc_flow_logs_destination_options_hive_compatible_partitions` (`bool`) optional
Flag to enable/disable VPC Flow Logs hive compatible partitions **Default value:** `false`
`vpc_flow_logs_destination_options_per_hour_partition` (`bool`) optional
Flag to enable/disable VPC Flow Logs per hour partition **Default value:** `false`
`vpc_flow_logs_enabled` (`bool`) optional
Enable or disable the VPC Flow Logs **Default value:** `true`
`vpc_flow_logs_log_destination_type` (`string`) optional
The type of the logging destination. Valid values: `cloud-watch-logs`, `s3` **Default value:** `"s3"`
`vpc_flow_logs_traffic_type` (`string`) optional
The type of traffic to capture. Valid values: `ACCEPT`, `REJECT`, `ALL` **Default value:** `"ALL"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`availability_zones`
List of Availability Zones where subnets were created
`az_private_subnets_map`
Map of AZ names to list of private subnet IDs in the AZs
`az_public_subnets_map`
Map of AZ names to list of public subnet IDs in the AZs
`flow_log_destination`
Destination bucket for VPC flow logs
`flow_log_id`
ID of the VPC flow log
`gateway_vpc_endpoints`
Map of Gateway VPC Endpoints in this VPC, keyed by service (e.g. "s3").
`igw_id`
The ID of the Internet Gateway
`interface_vpc_endpoints`
Map of Interface VPC Endpoints in this VPC.
`max_subnet_count`
Maximum allowed number of subnets before all subnet CIDRs need to be recomputed
`named_route_tables`
Map of route table IDs, keyed by subnets_per_az_names. If subnets_per_az_names is not set, items are grouped by key 'common'
`named_subnets`
Map of subnets IDs, keyed by subnets_per_az_names. If subnets_per_az_names is not set, items are grouped by key 'common'
`nat_eip_protections`
List of AWS Shield Advanced Protections for NAT Elastic IPs.
`nat_gateway_ids`
NAT Gateway IDs
`nat_gateway_public_ips`
NAT Gateway public IPs
`nat_instance_ids`
NAT Instance IDs
`private_route_table_ids`
Private subnet route table IDs
`private_subnet_cidrs`
Private subnet CIDRs
`private_subnet_ids`
Private subnet IDs
`public_route_table_ids`
Public subnet route table IDs
`public_subnet_cidrs`
Public subnet CIDRs
`public_subnet_ids`
Public subnet IDs
`route_tables`
Route tables info map
`subnets`
Subnets info map
`vpc`
VPC info map
`vpc_cidr`
VPC CIDR
`vpc_default_network_acl_id`
The ID of the network ACL created by default on VPC creation
`vpc_default_security_group_id`
The ID of the security group created by default on VPC creation
`vpc_endpoint_dynamodb_id`
ID of the DynamoDB gateway endpoint
`vpc_endpoint_dynamodb_prefix_list_id`
Prefix list ID for DynamoDB gateway endpoint
`vpc_endpoint_interface_security_group_id`
Security group ID for interface VPC endpoints
`vpc_endpoint_s3_id`
ID of the S3 gateway endpoint
`vpc_endpoint_s3_prefix_list_id`
Prefix list ID for S3 gateway endpoint
`vpc_id`
VPC ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Providers - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `endpoint_security_groups` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | We could create a security group per endpoint, but until we are ready to customize them by service, it is just a waste of resources. We use a single security group for all endpoints. Security groups can be updated without recreating the endpoint or interrupting service, so this is an easy change to make later. `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `subnets` | 2.4.2 | [`cloudposse/dynamic-subnets/aws`](https://registry.terraform.io/modules/cloudposse/dynamic-subnets/aws/2.4.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a `vpc` | 3.0.0 | [`cloudposse/vpc/aws`](https://registry.terraform.io/modules/cloudposse/vpc/aws/3.0.0) | n/a `vpc_endpoints` | 3.0.0 | [`cloudposse/vpc/aws//modules/vpc-endpoints`](https://registry.terraform.io/modules/cloudposse/vpc/aws/modules/vpc-endpoints/3.0.0) | n/a `vpc_flow_logs_bucket` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a ## Resources The following resources are used by this module: - [`aws_flow_log.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) (resource) - [`aws_shield_protection.nat_eip_shield_protection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/shield_protection) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_eip.eip`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eip) (data source) --- ## vpc-flow-logs-bucket This component provisions an encrypted S3 bucket configured to receive VPC Flow Logs. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. **IMPORTANT**: This component expects the `aws_flow_log` resource to be created externally. Typically that is accomplished through [the `vpc` component](https://github.com/cloudposse-terraform-components/aws-vpc-flow-logs-bucket/tree/main/vpc-flow-logs-bucket/../vpc/). ```yaml components: terraform: vpc-flow-logs-bucket: vars: name: "vpc-flow-logs" noncurrent_version_expiration_days: 180 noncurrent_version_transition_days: 30 standard_transition_days: 60 glacier_transition_days: 180 expiration_days: 365 ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`lifecycle_prefix` (`string`) optional
Prefix filter. Used to manage object lifecycle events **Default value:** `""`
`lifecycle_rule_enabled` (`bool`) optional
Enable lifecycle events on this bucket **Default value:** `true`
`lifecycle_tags` (`map(string)`) optional
Tags filter. Used to manage object lifecycle events **Default value:** `{ }`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Specifies when noncurrent object versions transitions **Default value:** `30`
`standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`traffic_type` (`string`) optional
The type of traffic to capture. Valid values: `ACCEPT`, `REJECT`, `ALL` **Default value:** `"ALL"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`vpc_flow_logs_bucket_arn`
VPC Flow Logs bucket ARN
`vpc_flow_logs_bucket_id`
VPC Flow Logs bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `flow_logs_s3_bucket` | 1.3.1 | [`cloudposse/vpc-flow-logs-s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/vpc-flow-logs-s3-bucket/aws/1.3.1) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## vpc-peering This component is responsible for creating a peering connection between two VPCs existing in different AWS accounts. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. Default VPC peering settings for all accounts: ```yaml # stacks/catalog/vpc-peering/defaults.yaml components: terraform: vpc-peering/defaults: settings: spacelift: workspace_enabled: true metadata: component: vpc-peering type: abstract vars: enabled: true requester_allow_remote_vpc_dns_resolution: true accepter_allow_remote_vpc_dns_resolution: true ``` Use case: Peering v1 accounts to v2 ```yaml # stacks/catalogs/vpc-peering/ue1-prod.yaml import: - catalog/vpc-peering/defaults components: terraform: vpc-peering-use1: metadata: component: vpc-peering inherits: - vpc-peering/defaults vars: accepter_region: us-east-1 accepter_vpc_id: vpc-xyz accepter_aws_assume_role_arn: arn:aws:iam:::role/acme-vpc-peering ``` Use case: Peering v2 accounts to v2 ```yaml vpc-peering/-vpc0: metadata: component: vpc-peering inherits: - vpc-peering/defaults vars: requester_vpc_component_name: vpc accepter_region: us-east-1 accepter_stage_name: accepter_vpc: tags: # Fill in with your own information Name: acme---- ``` ## Legacy Account Configuration The `vpc-peering` component peers the `dev`, `prod`, `sandbox` and `staging` VPCs to a VPC in the legacy account. The `dev`, `prod`, `sandbox` and `staging` VPCs are the requesters of the VPC peering connection, while the legacy VPC is the accepter of the peering connection. To provision VPC peering and all related resources with Terraform, we need the following information from the legacy account: - Legacy account ID - Legacy VPC ID - Legacy AWS region - Legacy IAM role (the role must be created in the legacy account with permissions to create VPC peering and routes). The name of the role could be `acme-vpc-peering` and the ARN of the role should look like `arn:aws:iam:::role/acme-vpc-peering` ### Legacy Account IAM Role In the legacy account, create IAM role `acme-vpc-peering` with the following policy: NOTE: Replace `` with the ID of the legacy account. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["ec2:CreateRoute", "ec2:DeleteRoute"], "Resource": "arn:aws:ec2:*::route-table/*" }, { "Effect": "Allow", "Action": [ "ec2:DescribeVpcPeeringConnections", "ec2:DescribeVpcs", "ec2:ModifyVpcPeeringConnectionOptions", "ec2:DescribeSubnets", "ec2:DescribeVpcAttribute", "ec2:DescribeRouteTables" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AcceptVpcPeeringConnection", "ec2:DeleteVpcPeeringConnection", "ec2:CreateVpcPeeringConnection", "ec2:RejectVpcPeeringConnection" ], "Resource": [ "arn:aws:ec2:*::vpc-peering-connection/*", "arn:aws:ec2:*::vpc/*" ] }, { "Effect": "Allow", "Action": ["ec2:DeleteTags", "ec2:CreateTags"], "Resource": "arn:aws:ec2:*::vpc-peering-connection/*" } ] } ``` Add the following trust policy to the IAM role: NOTE: Replace `` with the ID of the `identity` account in the new infrastructure. ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": ["arn:aws:iam:::root"] }, "Action": ["sts:AssumeRole", "sts:TagSession"], "Condition": {} } ] } ``` The trust policy allows the `identity` account to assume the role (and provision all the resources in the legacy account). ## Provisioning Provision the VPC peering connections in the `dev`, `prod`, `sandbox` and `staging` accounts by executing the following commands: ```sh atmos terraform plan vpc-peering -s ue1-sandbox atmos terraform apply vpc-peering -s ue1-sandbox atmos terraform plan vpc-peering -s ue1-dev atmos terraform apply vpc-peering -s ue1-dev atmos terraform plan vpc-peering -s ue1-staging atmos terraform apply vpc-peering -s ue1-staging atmos terraform plan vpc-peering -s ue1-prod atmos terraform apply vpc-peering -s ue1-prod ``` ## Variables ### Required Variables
`accepter_region` (`string`) required
Accepter AWS region
`accepter_vpc` (`any`) required
Accepter VPC map of id, cidr_block, or default arguments for the data source
`region` (`string`) required
AWS Region
### Optional Variables
`accepter_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow accepter VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the requester VPC **Default value:** `true`
`accepter_aws_assume_role_arn` (`string`) optional
Accepter AWS assume role ARN **Default value:** `null`
`accepter_stage_name` (`string`) optional
Accepter stage name if in v1 **Default value:** `null`
`add_attribute_tag` (`bool`) optional
If `true` will add additional attribute tag to the requester and accceptor resources **Default value:** `true`
`auto_accept` (`bool`) optional
Automatically accept peering request **Default value:** `true`
`requester_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow requester VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the accepter VPC **Default value:** `true`
`requester_role_arn` (`string`) optional
Requester AWS assume role ARN, if not provided it will be assumed to be the current terraform role. **Default value:** `null`
`requester_vpc_component_name` (`string`) optional
Requester vpc component name **Default value:** `"vpc"`
`requester_vpc_id` (`string`) optional
Requester VPC ID, if not provided, it will be looked up by component using variable `requester_vpc_component_name` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`vpc_peering`
VPC peering outputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0, < 6.0.0` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `requester_vpc` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc_peering` | 1.0.0 | [`cloudposse/vpc-peering-multi-account/aws`](https://registry.terraform.io/modules/cloudposse/vpc-peering-multi-account/aws/1.0.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_vpc.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## vpc-routes This component provisions routes in AWS VPC route tables. It is commonly used to configure routing between VPCs via Transit Gateways and to manage multiple route tables with similar route configurations on a regional basis. ## Usage **Stack Level**: Regional Here's a simple example using physical IDs: ```yaml components: terraform: vpc/routes/private: metadata: component: vpc-routes vars: route_table_ids: ["rtb-0123456789abcdef0", "rtb-0123456789abcdef1"] routes: - destination: cidr_block: "10.100.0.0/16" # Target VPC CIDR target: type: transit_gateway_id value: "tgw-0123456789abcdef0" ``` The same configuration using terraform outputs: ```yaml components: terraform: vpc/routes/private: metadata: component: vpc-routes vars: route_table_ids: !terraform.output vpc private_route_table_ids routes: - destination: cidr_block: !terraform.output vpc target-vpc vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub transit_gateway_id ``` ### Multiple Routes Example Example using physical IDs: ```yaml components: terraform: vpc/routes/private: metadata: component: vpc-routes vars: route_table_ids: ["rtb-0123456789abcdef0"] routes: # Route to network account - destination: cidr_block: "10.0.0.0/16" target: type: transit_gateway_id value: "tgw-0123456789abcdef0" # Route to transit account - destination: cidr_block: "10.1.0.0/16" target: type: transit_gateway_id value: "tgw-0123456789abcdef0" ``` The same configuration using terraform outputs: ```yaml components: terraform: vpc/routes/private: metadata: component: vpc-routes vars: route_table_ids: !terraform.output vpc private_route_table_ids routes: # Route to network account - destination: cidr_block: !terraform.output vpc network-vpc vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub transit_gateway_id # Route to transit account - destination: cidr_block: !terraform.output vpc transit-vpc vpc_cidr target: type: transit_gateway_id value: !terraform.output tgw/hub transit_gateway_id ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
### Optional Variables
`route_table_ids` (`list(string)`) optional
List of route table IDs **Default value:** `[ ]`
`routes` optional
A list of route objects to add to route tables. Each route object has a destination and a target. **Type:** ```hcl list(object({ destination = object({ cidr_block = string }) target = object({ type = string value = optional(string, "") }) })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_routes`
VPC routes
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.1, < 6.0.0` ### Providers - `aws`, version: `>= 4.1, < 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) ## Data Sources The following data sources are used by this module: --- ## waf This component is responsible for provisioning an AWS Web Application Firewall (WAF) with an associated managed rule group. ## Usage **Stack Level**: Regional Here's an example snippet for how to use this component. ```yaml components: terraform: waf: vars: enabled: true name: waf acl_name: default default_action: allow description: Default web ACL visibility_config: cloudwatch_metrics_enabled: false metric_name: "default" sampled_requests_enabled: false managed_rule_group_statement_rules: - name: "OWASP-10" # Rules are processed in order based on the value of priority, lowest number first priority: 1 statement: name: AWSManagedRulesCommonRuleSet vendor_name: AWS visibility_config: # Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: false metric_name: "OWASP-10" sampled_requests_enabled: false ``` ## Variables ### Required Variables
`acl_name` (`string`) required
Friendly name of the ACL. The ACL ARN will be stored in SSM under \{ssm_path_prefix\}/\{acl_name\}/arn
`region` (`string`) required
AWS Region
`visibility_config` required
Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl object({ cloudwatch_metrics_enabled = bool metric_name = string sampled_requests_enabled = bool }) ```
### Optional Variables
`alb_names` (`list(string)`) optional
list of ALB names to associate with the web ACL. **Default value:** `[ ]`
`alb_tags` (`list(map(string))`) optional
list of tags to match one or more ALBs to associate with the web ACL. **Default value:** `[ ]`
`association_resource_arns` (`list(string)`) optional
A list of ARNs of the resources to associate with the web ACL. This must be an ARN of an Application Load Balancer, Amazon API Gateway stage, or AWS AppSync. Do not use this variable to associate a Cloudfront Distribution. Instead, you should use the `web_acl_id` property on the `cloudfront_distribution` resource. For more details, refer to https://docs.aws.amazon.com/waf/latest/APIReference/API_AssociateWebACL.html **Default value:** `[ ]`
`association_resource_component_selectors` optional
A list of Atmos component selectors to get from the remote state and associate their ARNs with the web ACL. The components must be Application Load Balancers, Amazon API Gateway stages, or AWS AppSync. component: Atmos component name component_arn_output: The component output that defines the component ARN Set `tenant`, `environment` and `stage` if the components are in different OUs, regions or accounts. Do not use this variable to select a Cloudfront Distribution component. Instead, you should use the `web_acl_id` property on the `cloudfront_distribution` resource. For more details, refer to https://docs.aws.amazon.com/waf/latest/APIReference/API_AssociateWebACL.html **Type:** ```hcl list(object({ component = string namespace = optional(string, null) tenant = optional(string, null) environment = optional(string, null) stage = optional(string, null) component_arn_output = string })) ``` **Default value:** `[ ]`
`byte_match_statement_rules` optional
A rule statement that defines a string match search for AWS WAF to apply to web requests. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: positional_constraint: Area within the portion of a web request that you want AWS WAF to search for search_string. Valid values include the following: EXACTLY, STARTS_WITH, ENDS_WITH, CONTAINS, CONTAINS_WORD. search_string String value that you want AWS WAF to search for. AWS WAF searches only in the part of web requests that you designate for inspection in field_to_match. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`custom_response_body` optional
Defines custom response bodies that can be referenced by custom_response actions. The map keys are used as the `key` attribute which is a unique key identifying the custom response body. content: Payload of the custom response. The response body can be plain text, HTML or JSON and cannot exceed 4KB in size. content_type: Content Type of Response Body. Valid values are `TEXT_PLAIN`, `TEXT_HTML`, or `APPLICATION_JSON`. **Type:** ```hcl map(object({ content = string content_type = string })) ``` **Default value:** `{ }`
`default_action` (`string`) optional
Specifies that AWS WAF should allow requests by default. Possible values: `allow`, `block`. **Default value:** `"block"`
`default_block_response` (`string`) optional
A HTTP response code that is sent when default action is used. Only takes effect if default_action is set to `block`. **Default value:** `null`
`description` (`string`) optional
A friendly description of the WebACL. **Default value:** `"Managed by Terraform"`
`geo_allowlist_statement_rules` optional
A rule statement used to identify a list of allowed countries which should not be blocked by the WAF. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: country_codes: A list of two-character country codes. forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`geo_match_statement_rules` optional
A rule statement used to identify web requests based on country of origin. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: country_codes: A list of two-character country codes. forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`ip_set_reference_statement_rules` optional
A rule statement used to detect web requests coming from particular IP addresses or address ranges. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The ARN of the IP Set that this statement references. ip_set: Defines a new IP Set description: A friendly description of the IP Set addresses: Contains an array of strings that specifies zero or more IP addresses or blocks of IP addresses. All addresses must be specified using Classless Inter-Domain Routing (CIDR) notation. ip_address_version: Specify `IPV4` or `IPV6` ip_set_forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. position: The position in the header to search for the IP address. Possible values include: `FIRST`, `LAST`, or `ANY`. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`log_destination_component_selectors` optional
A list of Atmos component selectors to get from the remote state and associate their names/ARNs with the WAF logs. The components must be Amazon Kinesis Data Firehose, CloudWatch Log Group, or S3 bucket. component: Atmos component name component_output: The component output that defines the component name or ARN Set `tenant`, `environment` and `stage` if the components are in different OUs, regions or accounts. Note: data firehose, log group, or bucket name must be prefixed with `aws-waf-logs-`, e.g. `aws-waf-logs-example-firehose`, `aws-waf-logs-example-log-group`, or `aws-waf-logs-example-bucket`. **Type:** ```hcl list(object({ component = string namespace = optional(string, null) tenant = optional(string, null) environment = optional(string, null) stage = optional(string, null) component_output = string })) ``` **Default value:** `[ ]`
`log_destination_configs` (`list(string)`) optional
A list of resource names/ARNs to associate Amazon Kinesis Data Firehose, Cloudwatch Log log group, or S3 bucket with the WAF logs. Note: data firehose, log group, or bucket name must be prefixed with `aws-waf-logs-`, e.g. `aws-waf-logs-example-firehose`, `aws-waf-logs-example-log-group`, or `aws-waf-logs-example-bucket`. **Default value:** `[ ]`
`logging_filter` optional
A configuration block that specifies which web requests are kept in the logs and which are dropped. You can filter on the rule action and on the web request labels that were applied by matching rules during web ACL evaluation. **Type:** ```hcl object({ default_behavior = string filter = list(object({ behavior = string requirement = string condition = list(object({ action_condition = optional(object({ action = string }), null) label_name_condition = optional(object({ label_name = string }), null) })) })) }) ``` **Default value:** `null`
`managed_rule_group_statement_rules` optional
A rule statement used to run the rules that are defined in a managed rule group. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. override_action: The override action to apply to the rules in a rule group. Possible values: `count`, `none` captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: name: The name of the managed rule group. vendor_name: The name of the managed rule group vendor. version: The version of the managed rule group. You can set `Version_1.0` or `Version_1.1` etc. If you want to use the default version, do not set anything. scope_down_not_statement_enabled: Whether to wrap the scope_down_statement inside of a not_statement. Refer to https://docs.aws.amazon.com/waf/latest/developerguide/waf-bot-control-example-scope-down-your-bot.html scope_down_statement: Nested statement that narrows the scope of the rate-based statement to matching web requests. rule_action_override: Action settings to use in the place of the rule actions that are configured inside the rule group. You specify one override for each rule whose action you want to change. managed_rule_group_configs: Additional information that's used by a managed rule group. Only one rule attribute is allowed in each config. Refer to https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-list.html for more details. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number override_action = optional(string) captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = object({ name = string vendor_name = string version = optional(string) scope_down_not_statement_enabled = optional(bool, false) scope_down_statement = optional(object({ byte_match_statement = object({ positional_constraint = string search_string = string field_to_match = object({ all_query_arguments = optional(bool) body = optional(bool) method = optional(bool) query_string = optional(bool) single_header = optional(object({ name = string })) single_query_argument = optional(object({ name = string })) uri_path = optional(bool) }) text_transformation = list(object({ priority = number type = string })) }) }), null) rule_action_override = optional(map(object({ action = string custom_request_handling = optional(object({ insert_header = object({ name = string value = string }) }), null) custom_response = optional(object({ response_code = string response_header = optional(object({ name = string value = string }), null) }), null) })), null) managed_rule_group_configs = optional(list(object({ aws_managed_rules_anti_ddos_rule_set = optional(object({ sensitivity_to_block = optional(string) client_side_action_config = optional(object({ challenge = object({ usage_of_action = string sensitivity = optional(string) exempt_uri_regular_expression = optional(list(object({ regex_string = string }))) }) })) })) aws_managed_rules_bot_control_rule_set = optional(object({ inspection_level = string enable_machine_learning = optional(bool, true) }), null) aws_managed_rules_atp_rule_set = optional(object({ enable_regex_in_path = optional(bool) login_path = string request_inspection = optional(object({ payload_type = string password_field = object({ identifier = string }) username_field = object({ identifier = string }) }), null) response_inspection = optional(object({ body_contains = optional(object({ success_strings = list(string) failure_strings = list(string) }), null) header = optional(object({ name = string success_values = list(string) failure_values = list(string) }), null) json = optional(object({ identifier = string success_strings = list(string) failure_strings = list(string) }), null) status_code = optional(object({ success_codes = list(string) failure_codes = list(string) }), null) }), null) }), null) aws_managed_rules_acfp_rule_set = optional(object({ creation_path = string enable_regex_in_path = optional(bool) registration_page_path = string request_inspection = optional(object({ payload_type = string password_field = optional(object({ identifier = string }), null) username_field = optional(object({ identifier = string }), null) email_field = optional(object({ identifier = string }), null) address_fields = optional(object({ identifiers = list(string) }), null) phone_number_fields = optional(object({ identifiers = list(string) }), null) }), null) response_inspection = optional(object({ body_contains = optional(object({ success_strings = list(string) failure_strings = list(string) }), null) header = optional(object({ name = string success_values = list(string) failure_values = list(string) }), null) json = optional(object({ identifier = string success_values = list(string) failure_values = list(string) }), null) status_code = optional(object({ success_codes = list(string) failure_codes = list(string) }), null) }), null) })) })), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`rate_based_statement_rules` optional
A rate-based rule tracks the rate of requests for each originating IP address, and triggers the rule action when the rate exceeds a limit that you specify on the number of requests in any 5-minute time span. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: aggregate_key_type: Setting that indicates how to aggregate the request counts. Possible values include: `FORWARDED_IP` or `IP` limit: The limit on requests per 5-minute period for a single originating IP address. evaluation_window_sec: The amount of time, in seconds, that AWS WAF should include in its request counts, looking back from the current time. Valid values are 60, 120, 300, and 600. Defaults to 300 (5 minutes). forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. byte_match_statement: field_to_match: Part of a web request that you want AWS WAF to inspect. positional_constraint: Area within the portion of a web request that you want AWS WAF to search for search_string. Valid values include the following: `EXACTLY`, `STARTS_WITH`, `ENDS_WITH`, `CONTAINS`, `CONTAINS_WORD`. search_string: String value that you want AWS WAF to search for. AWS WAF searches only in the part of web requests that you designate for inspection in `field_to_match`. The maximum length of the value is 50 bytes. text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = object({ limit = number aggregate_key_type = string evaluation_window_sec = optional(number) forwarded_ip_config = optional(object({ fallback_behavior = string header_name = string }), null) scope_down_statement = optional(object({ byte_match_statement = object({ positional_constraint = string search_string = string field_to_match = object({ all_query_arguments = optional(bool) body = optional(bool) method = optional(bool) query_string = optional(bool) single_header = optional(object({ name = string })) single_query_argument = optional(object({ name = string })) uri_path = optional(bool) }) text_transformation = list(object({ priority = number type = string })) }) }), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`redacted_fields` optional
The parts of the request that you want to keep out of the logs. You can only specify one of the following: `method`, `query_string`, `single_header`, or `uri_path` method: Whether to enable redaction of the HTTP method. The method indicates the type of operation that the request is asking the origin to perform. uri_path: Whether to enable redaction of the URI path. This is the part of a web request that identifies a resource. query_string: Whether to enable redaction of the query string. This is the part of a URL that appears after a `?` character, if any. single_header: The list of names of the query headers to redact. **Type:** ```hcl map(object({ method = optional(bool, false) uri_path = optional(bool, false) query_string = optional(bool, false) single_header = optional(list(string), null) })) ``` **Default value:** `{ }`
`regex_match_statement_rules` optional
A rule statement used to search web request components for a match against a single regular expression. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: regex_string: String representing the regular expression. Minimum of 1 and maximum of 512 characters. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl.html#field_to_match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. At least one required. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`regex_pattern_set_reference_statement_rules` optional
A rule statement used to search web request components for matches with regular expressions. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The Amazon Resource Name (ARN) of the Regex Pattern Set that this statement references. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`rule_group_reference_statement_rules` optional
A rule statement used to run the rules that are defined in an WAFv2 Rule Group. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. override_action: The override action to apply to the rules in a rule group. Possible values: `count`, `none` captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The ARN of the `aws_wafv2_rule_group` resource. rule_action_override: Action settings to use in the place of the rule actions that are configured inside the rule group. You specify one override for each rule whose action you want to change. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number override_action = optional(string) captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = object({ arn = string rule_action_override = optional(map(object({ action = string custom_request_handling = optional(object({ insert_header = object({ name = string value = string }) }), null) custom_response = optional(object({ response_code = string response_header = optional(object({ name = string value = string }), null) }), null) })), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`scope` (`string`) optional
Specifies whether this is for an AWS CloudFront distribution or for a regional application. Possible values are `CLOUDFRONT` or `REGIONAL`. To work with CloudFront, you must also specify the region us-east-1 (N. Virginia) on the AWS provider. **Default value:** `"REGIONAL"`
`size_constraint_statement_rules` optional
A rule statement that uses a comparison operator to compare a number of bytes against the size of a request component. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: comparison_operator: The operator to use to compare the request part to the size setting. Possible values: `EQ`, `NE`, `LE`, `LT`, `GE`, or `GT`. size: The size, in bytes, to compare to the request part, after any transformations. Valid values are integers between `0` and `21474836480`, inclusive. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`sqli_match_statement_rules` optional
An SQL injection match condition identifies the part of web requests, such as the URI or the query string, that you want AWS WAF to inspect. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. rule_label: A List of labels to apply to web requests that match the rule match statement captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. statement: field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`ssm_path_prefix` (`string`) optional
SSM path prefix (with leading but not trailing slash) under which to store all WAF info **Default value:** `"/waf"`
`token_domains` (`list(string)`) optional
Specifies the domains that AWS WAF should accept in a web request token. This enables the use of tokens across multiple protected websites. When AWS WAF provides a token, it uses the domain of the AWS resource that the web ACL is protecting. If you don't specify a list of token domains, AWS WAF accepts tokens only for the domain of the protected resource. With a token domain list, AWS WAF accepts the resource's host domain plus all domains in the token domain list, including their prefixed subdomains. **Default value:** `null`
`xss_match_statement_rules` optional
A rule statement that defines a cross-site scripting (XSS) match search for AWS WAF to apply to web requests. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the WAF WebACL.
`id`
The ID of the WAF WebACL.
`logging_config_id`
The ARN of the WAFv2 Web ACL logging configuration.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 6.2.0` ### Providers - `aws`, version: `>= 6.2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `association_resource_components` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `aws_waf` | 1.15.0 | [`cloudposse/waf/aws`](https://registry.terraform.io/modules/cloudposse/waf/aws/1.15.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `log_destination_components` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/remote-state/1.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.acl_arn`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`aws_alb.alb`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/alb) (data source) - [`aws_lbs.alb_by_tags`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lbs) (data source) --- ## zscaler This component is responsible for provisioning ZScaler Private Access Connector instances on Amazon Linux 2 AMIs. Prior to provisioning this component, it is required that a SecureString SSM Parameter containing the ZScaler App Connector Provisioning Key is populated in each account corresponding to the regional stack the component is deployed to, with the name of the SSM Parameter matching the value of `var.zscaler_key`. This parameter should be populated using `chamber`, which is included in the geodesic image: ``` chamber write zscaler key ``` Where `` is the ZScaler App Connector Provisioning Key. For more information on how to generate this key, see: [ZScaler documentation on Configuring App Connectors](https://help.zscaler.com/zpa/configuring-connectors). ## Usage **Stack Level**: Regional The typical stack configuration for this component is as follows: ```yaml components: terraform: zscaler: vars: zscaler_count: 2 ``` Preferably, regional stack configurations can be kept _DRY_ by importing `catalog/zscaler` via the `imports` list at the top of the configuration. ``` import: ... - catalog/zscaler ``` ## Variables ### Required Variables
`region` (`string`) required
AWS region
### Optional Variables
`ami_owner` (`string`) optional
The owner of the AMI used for the ZScaler EC2 instances. **Default value:** `"amazon"`
`ami_regex` (`string`) optional
The regex used to match the latest AMI to be used for the ZScaler EC2 instances. **Default value:** `"^amzn2-ami-hvm.*"`
`aws_ssm_enabled` (`bool`) optional
Set true to install the AWS SSM agent on each EC2 instances. **Default value:** `true`
`instance_type` (`string`) optional
The instance family to use for the ZScaler EC2 instances. **Default value:** `"m5n.large"`
`secrets_store_type` (`string`) optional
Secret store type for Zscaler provisioning keys. Valid values: `SSM`, `ASM` (but `ASM` not currently supported) **Default value:** `"SSM"`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully completed with `aws_security_group_rule` resource. To get more info see [security_group_rule](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule). **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "from_port": 0, "protocol": "-1", "to_port": 65535, "type": "egress" } ] ```
`zscaler_count` (`number`) optional
The number of Zscaler instances. **Default value:** `1`
`zscaler_key` (`string`) optional
SSM key (without leading `/`) for the Zscaler provisioning key secret. **Default value:** `"zscaler/key"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional tags for appending to tags_as_list_of_maps. Not added to `tags`. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {} } ```
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for default, which is `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
The letter case of label keys (`tag` names) (i.e. `name`, `namespace`, `environment`, `stage`, `attributes`) to use in `tags`. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The naming order of the id output and Name tag. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 5 elements, but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
The letter case of output label values (also used in `tags` and `id`). Possible values: `lower`, `title`, `upper` and `none` (no transformation). Default value: `lower`. **Required:** No **Default value:** `null`
`name` (`string`) optional
Solution name, e.g. 'app' or 'jenkins' **Required:** No **Default value:** `null`
`namespace` (`string`) optional
Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit','XYZ')` **Required:** No **Default value:** `{ }`
## Outputs
`instance_id`
Instance ID
`private_ip`
Private IP of the instance
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.0, < 6.0.0` - `null`, version: `>= 3.0` - `random`, version: `>= 3.0` - `template`, version: `>= 2.2` - `utils`, version: `>= 1.10.0` ### Providers - `aws`, version: `>= 3.0, < 6.0.0` - `template`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ec2_zscaler` | 2.0.0 | [`cloudposse/ec2-instance/aws`](https://registry.terraform.io/modules/cloudposse/ec2-instance/aws/2.0.0) | n/a `iam_roles` | latest | [`../account-map/modules/iam-roles`](https://registry.terraform.io/modules/../account-map/modules/iam-roles/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_role_policy_attachment.ssm_core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.amazon_linux_2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_ssm_parameter.zscaler_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`template_file.userdata`](https://registry.terraform.io/providers/cloudposse/template/latest/docs/data-sources/file) (data source) --- ## Terraform Components(Library) import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a library of reusable Terraform "root module" components. --- ## GitHub Actions import Intro from '@site/src/components/Intro' import DocCardList from '@theme/DocCardList' In this library you'll find all the GitHub Actions we've implemented to solve common CI/CD challenges. --- ## GitHub Actions(Actions) import Intro from '@site/src/components/Intro' import DocCardList from '@theme/DocCardList' In this library you'll find all the GitHub Actions we've implemented to solve common CI/CD challenges. --- ## atmos-affected-stacks # GitHub Action: `atmos-affected-stacks` A GitHub Action to get a list of affected atmos stacks for a pull request ## Introduction This is a GitHub Action to get a list of affected atmos stacks for a pull request. It optionally installs `atmos` and `jq` and runs `atmos describe affected` to get the list of affected stacks. It provides the raw list of affected stacks as an output as well as a matrix that can be used further in GitHub action jobs. ## Usage ### Config :::important **Please note!** This GitHub Action only works with `atmos >= 1.99.0`. If you are using `atmos >= 1.80.0, < 1.99.0` please use `v5` version of this action. If you are using `atmos >= 1.63.0, < 1.80.0` please use `v3` or `v4` version of this action. If you are using `atmos < 1.63.0` please use `v2` version of this action. ::: The action expects the atmos configuration file `atmos.yaml` to be present in the repository. The action supports AWS and Azure to store Terraform plan files. You can read more about plan storage in the [cloudposse/github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage?tab=readme-ov-file#aws-default) documentation. Depends of cloud provider the following fields should be set in the `atmos.yaml`: #### AWS The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops # Set `apply` empty if you don't want to assume IAM role before terraform apply apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` #### Azure The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: plan-repository-type: azureblob blob-account-name: tfplans blob-container-name: plans metadata-repository-type: cosmos cosmos-container-name: terraform-plan-storage cosmos-database-name: terraform-plan-storage cosmos-endpoint: "https://my-cosmo-account.documents.azure.com:443/" # We remove the `role` section as it is AWS specific matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` ### Stack level configuration :::important Wherever it is possible to specify `integration.github.gitops` on stack level it is required to define default values in `atmos.yaml` ::: It is possible to override integration settings on a stack level by defining `settings.integrations`. ```yaml components: terraform: foobar: settings: integrations: github: gitops: artifact-storage: bucket: cptest-plat-ue2-auto-gitops table: cptest-plat-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-plat-ue2-auto-gitops-gha role: # Set `plan` empty if you don't want to assume IAM role before terraform plan plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops ``` ### Support OpenTofu This action supports [OpenTofu](https://opentofu.org/). :::important **Please note!** OpenTofu supported by Atmos `>= 1.73.0`. For details [read](https://atmos.tools/core-concepts/projects/configuration/opentofu/) ::: To enable OpenTofu add the following settings to `atmos.yaml` * Set the `opentofu-version` in the `atmos.yaml` to the desired version * Set `components.terraform.command` to `tofu` #### Example ```yaml components: terraform: command: tofu ... integrations: github: gitops: opentofu-version: 1.7.3 ... ``` ### Workflow example ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: atmos-affected: runs-on: ubuntu-latest steps: - id: affected uses: cloudposse/github-action-atmos-affected-stacks@v6 with: atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.99.0 nested-matrices-count: 1 outputs: matrix: ${{ steps.affected.outputs.matrix }} has-affected-stacks: ${{ steps.affected.outputs.has-affected-stacks }} # This job is an example how to use the affected stacks with the matrix strategy atmos-plan: needs: ["atmos-affected"] if: ${{ needs.atmos-affected.outputs.has-affected-stacks == 'true' }} name: Plan ${{ matrix.stack_slug }} runs-on: ubuntu-latest strategy: max-parallel: 10 fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(needs.atmos-affected.outputs.matrix) }} ## Avoid running the same stack in parallel mode (from different workflows) concurrency: group: ${{ matrix.stack_slug }} cancel-in-progress: false steps: - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v4 with: component: ${{ matrix.component }} stack: ${{ matrix.stack }} atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.99.0 ``` ### Migrating from `v5` to `v6` The notable changes in `v6` are: - `v6` works only with `atmos >= 1.99.0` - `v6` allow to skip internal checkout with `skip-checkout` input The only required migration step is updating atmos version to `>= 1.80.0` ### Migrating from `v4` to `v5` The notable changes in `v5` are: - `v5` works only with `atmos >= 1.80.0` - `v5` supports atmos templating The only required migration step is updating atmos version to `>= 1.80.0` ### Migrating from `v3` to `v4` The notable changes in `v4` are: - `v4` perform aws authentication assuming `integrations.github.gitops.role.plan` IAM role No special migration steps required ### Migrating from `v2` to `v3` The notable changes in `v3` are: - `v3` works only with `atmos >= 1.63.0` - `v3` drops `install-terraform` input because terraform is not required for affected stacks call - `v3` drops `atmos-gitops-config-path` input and the `./.github/config/atmos-gitops.yaml` config file. Now you have to use GitHub Actions environment variables to specify the location of the `atmos.yaml`. The following configuration fields now moved to GitHub action inputs with the same names | name | |-------------------------| | `atmos-version` | | `atmos-config-path` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | |--------------------------|-------------------------------------------------| | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | For example, to migrate from `v2` to `v3`, you should have something similar to the following in your `atmos.yaml`: `./.github/config/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` `.github/workflows/main.yaml` ```yaml - id: affected uses: cloudposse/github-action-atmos-affected-stacks@v3 with: atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.63.0 ``` This corresponds to the `v2` configuration (deprecated) below. The `v2` configuration file `./.github/config/atmos-gitops.yaml` looked like this: ```yaml atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` And the `v2` GitHub Action Workflow looked like this. `.github/workflows/main.yaml` ```yaml - id: affected uses: cloudposse/github-action-atmos-affected-stacks@v2 with: atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` ### Migrating from `v1` to `v2` `v2` moves most of the `inputs` to the Atmos GitOps config path `./.github/config/atmos-gitops.yaml`. Simply create this file, transfer your settings to it, then remove the corresponding arguments from your invocations of the `cloudposse/github-action-atmos-affected-stacks` action. | name | |--------------------------| | `atmos-version` | | `atmos-config-path` | | `terraform-state-bucket` | | `terraform-state-table` | | `terraform-state-role` | | `terraform-plan-role` | | `terraform-apply-role` | | `terraform-version` | | `aws-region` | | `enable-infracost` | If you want the same behavior in `v2` as in `v1` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v1` inputs. ```yaml - name: Determine Affected Stacks uses: cloudposse/github-action-atmos-affected-stacks@v2 id: affected with: atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml nested-matrices-count: 1 ``` Which would produce the same behavior as in `v1`, doing this: ```yaml - name: Determine Affected Stacks uses: cloudposse/github-action-atmos-affected-stacks@v1 id: affected with: atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-config-path | The path to the atmos.yaml file | N/A | true | | atmos-include-dependents | Whether to include dependents of affected stacks in the output | false | false | | atmos-include-settings | Include the `settings` section for each affected component | false | false | | atmos-include-spacelift-admin-stacks | Whether to include the Spacelift admin stacks of affected stacks in the output | false | false | | atmos-pro-base-url | The base URL of Atmos Pro | https://atmos-pro.com | false | | atmos-pro-token | The API token to allow Atmos Pro to upload affected stacks | | false | | atmos-pro-upload | Whether to upload affected stacks directly to Atmos Pro | false | false | | atmos-stack | The stack to operate on | | false | | atmos-version | The version of atmos to install | >= 1.99.0 | false | | base-ref | The base ref to checkout. If not provided, the head default branch is used. | N/A | false | | default-branch | The default branch to use for the base ref. | $\{\{ github.event.repository.default\_branch \}\} | false | | head-ref | The head ref to checkout. If not provided, the head default branch is used. | $\{\{ github.sha \}\} | false | | install-atmos | Whether to install atmos | true | false | | install-jq | Whether to install jq | false | false | | jq-force | Whether to force the installation of jq | true | false | | jq-version | The version of jq to install if install-jq is true | 1.7 | false | | nested-matrices-count | Number of nested matrices that should be returned as the output (from 1 to 3) | 2 | false | | process-functions | Whether to process atmos functions | true | false | | process-templates | Whether to process atmos templates | true | false | | skip-atmos-functions | Skip all Atmos functions such as terraform.output | false | false | | skip-checkout | Disable actions/checkout for head-ref and base-ref. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | ## Outputs | Name | Description | |------|-------------| | affected | The affected stacks | | has-affected-stacks | Whether there are affected stacks | | matrix | The affected stacks as matrix structure suitable for extending matrix size workaround (see README) | --- ## atmos-affected-trigger-spacelift # GitHub Action: `atmos-affected-trigger-spacelift` GitHub Action for Triggering Affected Spacelift Stacks ## Introduction This repo contains a GitHub Action that determines the affected [Atmos](https://atmos.tools) stacks for a PR, then creates a comment on the PR which Spacelift can use to trigger the corresponding stacks via a push policy. Optionally, you can use the `spacectl` trigger method, which uses the `spacectl` CLI to trigger the corresponding spacelift stacks directly rather than via comment/push policy. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Atmos Affected Stacks Trigger Spacelift (via comment) uses: cloudposse/github-action-atmos-affected-trigger-spacelift@main id: example with: atmos-config-path: ./rootfs/usr/local/etc/atmos github-token: ${{ secrets.GITHUB_TOKEN }} - name: Atmos Affected Stacks Trigger Spacelift (direct) uses: cloudposse/github-action-atmos-affected-trigger-spacelift@main id: example with: atmos-config-path: ./rootfs/usr/local/etc/atmos github-token: ${{ secrets.GITHUB_TOKEN }} trigger-method: spacectl spacelift-endpoint: https://unicorn.app.spacelift.io spacelift-api-key-id: ${{ secrets.SPACELIFT_API_KEY_ID }} spacelift-api-key-secret: ${{ secrets.SPACELIFT_API_KEY_SECRET }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-config-path | A path to the folder where atmos.yaml is located | . | false | | atmos-version | The version of atmos to install if install-atmos is true | latest | false | | default-branch | The default branch to use for the base ref. | $\{\{ github.event.repository.default\_branch \}\} | false | | deploy | A flag to indicate if a deployment should be triggered. If false, a preview will be triggered. | false | false | | github-token | A GitHub token for running the spacelift-io/setup-spacectl action | N/A | true | | head-ref | The head ref to checkout. If not provided, the head default branch is used. | N/A | false | | install-atmos | Whether to install atmos | true | false | | install-jq | Whether to install jq | false | false | | install-spacectl | Whether to install spacectl | true | false | | jq-force | Whether to force the installation of jq | true | false | | jq-version | The version of jq to install if install-jq is true | 1.6 | false | | skip-atmos-functions | Skip all Atmos functions such as terraform.output in `atmos describe affected` | false | false | | skip-checkout | Disable actions/checkout for head-ref and base-ref. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | | spacectl-version | The version of spacectl to install if install-spacectl is true | latest | false | | spacelift-api-key-id | The SPACELIFT\_API\_KEY\_ID | N/A | false | | spacelift-api-key-secret | The SPACELIFT\_API\_KEY\_SECRET | N/A | false | | spacelift-endpoint | The Spacelift endpoint. For example, https://unicorn.app.spacelift.io | N/A | false | | trigger-method | The method to use to trigger the Spacelift stack. Valid values are `comment` and `spacectl` | comment | false | ## Outputs | Name | Description | |------|-------------| | affected | The affected stacks | | has-affected-stacks | Whether there are affected stacks | --- ## atmos-component-updater # GitHub Action: `atmos-component-updater` This is GitHub Action that can be used as a workflow for automatic updates via Pull Requests in your infrastructure repository according to versions in components sources. ## Introduction This is GitHub Action that can be used as a workflow for automatic updates via Pull Requests in your infrastructure repository according to versions in components sources. ### Key Features: - **Selective Component Processing:** Configure the action to `exclude` or `include` specific components using wildcards, ensuring that only relevant updates are processed. - **PR Management:** Limit the number of PRs opened at a time, making it easier to manage large-scale updates without overwhelming the system. Automatically close old component-update PRs, so they don't pile up. - **Material Changes Focus:** Automatically open pull requests only for components with significant changes, skipping minor updates to `component.yaml` files to reduce unnecessary PRs and maintain a streamlined system. - **Informative PRs:** Link PRs to release notes for new components, providing easy access to relevant information, and use consistent naming for easy tracking. - **Scheduled Updates:** Run the action on a cron schedule tailored to your organization's needs, ensuring regular and efficient updates. ## Usage ### Prerequisites This GitHub Action once used in workflow needs permissions to create/update branches and open/close pull requests so the access token needs to be passed. It can be done in two ways: - create a dedicated Personal Access Token (PAT) - use [`GITHUB_TOKEN`](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#about-the-github_token-secret) If you would like to use `GITHUB_TOKEN` make sure to set permissions in the workflow as follow: ```yaml permissions: contents: write issues: write pull-requests: write ``` Also, make sure that you set to `Allow GitHub Actions to create and approve pull requests` on both organization and repository levels: - `https://github.com/organizations/YOUR-ORG/settings/actions` - `https://github.com/YOUR-ORG/YOUR-REPO/settings/actions` ### Workflow example ```yaml name: "atmos-components" on: workflow_dispatch: {} schedule: - cron: '0 8 * * 1' # Execute every week on Monday at 08:00 permissions: contents: write pull-requests: write jobs: update: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 with: fetch-depth: 0 - name: Update Atmos Components uses: cloudposse/github-action-atmos-component-updater@v2 with: github-access-token: ${{ secrets.GITHUB_TOKEN }} max-number-of-prs: 5 include: | aws-* eks/* bastion exclude: aws-sso,aws-saml ``` ### Using a Custom Atmos CLI Config Path (`atmos.yaml`) If your [`atmos.yaml` file](https://atmos.tools/cli/configuration) is not located in the root of the infrastructure repository, you can specify the path to it using [`ATMOS_CLI_CONFIG_PATH` env variable](https://atmos.tools/cli/configuration/#environment-variables). ```yaml # ... - name: Update Atmos Components uses: cloudposse/github-action-atmos-component-updater@v2 env: # Directory containing the `atmos.yaml` file ATMOS_CLI_CONFIG_PATH: ${{ github.workspace }}/rootfs/usr/local/etc/atmos/ with: github-access-token: ${{ secrets.GITHUB_TOKEN }} max-number-of-prs: 5 ``` ### Customize Pull Request labels, title and body ```yaml # ... - name: Update Atmos Components uses: cloudposse/github-action-atmos-component-updater@v2 with: github-access-token: ${{ secrets.GITHUB_TOKEN }} max-number-of-prs: 5 pr-title: 'Update Atmos Component \`{{ component_name }}\` to {{ new_version }}' pr-body: | ## what Component \`{{ component_name }}\` was updated [{{ old_version }}]({{ old_version_link }}) → [{{ old_version }}]({{ old_version_link }}). ## references - [{{ source_name }}]({{ source_link }}) pr-labels: | component-update automated atmos ``` **IMPORTANT:** The backtick symbols must be escaped in the GitHub Action parameters. This is because GitHub evaluates whatever is in the backticks and it will render as an empty string. #### For `title` template these placeholders are available: - `component_name` - `source_name` - `old_version` - `new_version` #### For `body` template these placeholders are available: - `component_name` - `source_name` - `source_link` - `old_version` - `new_version` - `old_version_link` - `new_version_link` - `old_component_release_link` - `new_component_release_link` ## FAQ ### The action cannot find any components You may see that the action returns zero components: ```console [06-03-2024 17:53:47] INFO Found 0 components [] ``` This is a common error when the workflow has not checked out the repository before calling this action. Add the following before calling this action. ```yaml - name: Checkout uses: actions/checkout@v4 ``` ### The action cannot find the `component.yaml` file You may see the action fail to find the `component.yaml` file for a given component as such: ```console FileNotFoundError: [Errno 2] No such file or directory: 'components/terraform/account-map/component.yaml' ``` This is likely related to a missing or invalid `atmos.yaml` configuration file. Set `ATMOS_CLI_CONFIG_PATH` to the path to your Atmos configuration file. ```yaml env: ATMOS_CLI_CONFIG_PATH: ${{ github.workspace }}/rootfs/usr/local/etc/atmos/ ``` ### The action does not have permission to create Pull Requests Your action may fail with the following message: ```console github.GithubException.GithubException: 403 {"message": "GitHub Actions is not permitted to create or approve pull requests.", "documentation_url": "https://docs.github.com/rest/pulls/pulls#create-a-pull-request"} ``` In order to create Pull Requests in your repository, we need to set the permissions for the workflow: ```yaml permissions: contents: write issues: write pull-requests: write ``` _And_ you need to allow GitHub Actions to create and approve pulls requests in both the GitHub Organization and Repository: 1. `https://github.com/organizations/YOUR-ORG/settings/actions` > `Allow GitHub Actions to create and approve pull requests` 2. `https://github.com/YOUR-ORG/YOUR-REPO/settings/actions` > `Allow GitHub Actions to create and approve pull requests` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-version | Atmos version to use for vendoring. Default 'latest' | latest | false | | dry-run | Skip creation of remote branches and pull requests. Only print list of affected componented into file that is defined in 'outputs.affected-components-file' | false | false | | exclude | Comma or new line separated list of component names to exclude. For example: 'vpc,eks/\*,rds'. By default no components are excluded. Default '' | | false | | github-access-token | GitHub Token used to perform git and GitHub operations | $\{\{ github.token \}\} | false | | include | Comma or new line separated list of component names to include. For example: 'vpc,eks/\*,rds'. By default all components are included. Default '\*' | \* | false | | infra-repo-dir | Path to the infra repository. Default '/github/workspace/' | /github/workspace/ | false | | infra-terraform-dirs | Comma or new line separated list of terraform directories in infra repo. For example 'components/terraform,components/terraform-old. Default 'components/terraform' | components/terraform | false | | log-level | Log level for this action. Default 'INFO' | INFO | false | | max-number-of-prs | Number of PRs to create. Maximum is 10. | 10 | false | | pr-body-template | A string representing a Jinja2 formatted template to be used as the content of a Pull Request (PR) body. If not set template from `src/templates/pr\_body.j2.md` will be used | | false | | pr-labels | Comma or new line separated list of labels that will added on PR creation. Default: `component-update` | component-update | false | | pr-title-template | A string representing a Jinja2 formatted template to be used as the content of a Pull Request (PR) title. If not, set template from `src/templates/pr\_title.j2.md` will be used | | false | | vendoring-enabled | Do not perform 'atmos vendor component-name' on components that wasn't vendored | true | false | ## Outputs | Name | Description | |------|-------------| | affected | The affected components | | has-affected-stacks | Whether there are affected components | --- ## atmos-get-setting # GitHub Action: `atmos-get-setting` GitHub Action to retrieve a setting from [atmos](https://github.com/cloudposse/atmos) configuration. ## Introduction GitHub Action to retrieve settings from [atmos](https://github.com/cloudposse/atmos) configuration. There are two ways to use this action. The first is to retrieve a single setting and to get its value returned via the `value` output. The second is to retrieve multiple settings as an object returned via the `settings` output. ## Usage ``` # Example stacks/dev.yaml components: terraform: foo: settings: roleArn: arn:aws:iam::000000000000:role/MyRole secretsArn: arn:aws:secretsmanager:us-east-1:000000000000:secret:MySecret-PlMes3 vars: foo: bar ``` ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: # The following example will return a single setting value of # `arn:aws:secretsmanager:us-east-1:000000000000:secret:MySecret-PlMes3` in the `value` output: - name: Get Atmos Single Setting for Secret ARN uses: cloudposse/github-action-atmos-get-setting@main id: example with: component: foo stack: core-ue1-dev settings-path: settings.secrets-arn # The following example will return an object with the following structure in the `settings` output: # {"secretsArn":"arn:aws:secretsmanager:us-east-1:000000000000:secret:MySecret-PlMes3", "roleArn":"arn:aws:iam::000000000000:role/MyRole"} - name: Get Atmos Multiple Settings uses: cloudposse/github-action-atmos-get-setting@main id: example with: settings: | - component: foo stack: core-ue1-dev settingsPath: settings.secrets-arn outputPath: secretsArn - component: foo stack: core-ue1-dev settingsPath: settings.secrets-arn outputPath: roleArn ``` ## Migrating from `v0` to `v1` Starting from `v1` the action is no longer restricted to retrieving the component config from only the `settings` section. If you want the same behavior in `v1` as in`v0`, you should add the `settings.` prefix to the value of the `settings-path` variable. For example, in `v1` you would provide `settings.secrets-arn` as the value to the `settings-path` ```yaml - name: Get Atmos Setting for Secret ARN uses: cloudposse/github-action-atmos-get-setting@v1 id: example with: component: foo stack: core-ue1-dev settings-path: settings.secrets-arn ``` Which would provide the same output as passing only `secrets-arn` in `v0` ```yaml - name: Get Atmos Setting for Secret ARN uses: cloudposse/github-action-atmos-get-setting@v0 id: example with: component: foo stack: core-ue1-dev settings-path: secrets-arn ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | component | The atmos component extract the settings for. | N/A | false | | process-functions | Enable/disable processing of Terraform functions in Atmos stacks manifests. | true | false | | process-templates | Enable/disable processing of Go templates in Atmos stacks manifests. | true | false | | settings | The settings to extract. | N/A | false | | settings-path | The settings path using JSONPath expressions. | N/A | false | | stack | The atmos stack extract the settings for. | N/A | false | ## Outputs | Name | Description | |------|-------------| | settings | The settings values when multiple settings are returned. | | value | The value of the setting when a single setting is returned. | --- ## atmos-terraform-apply # GitHub Action: `atmos-terraform-apply` This Github Action is used to run Terraform apply for a single, Atmos-supported component with a saved planfile in S3 and DynamoDB. ## Introduction This Github Action is used to run Terraform apply for a single, Atmos-supported component with a saved planfile in S3 and DynamoDB. Before running this action, first create and store a planfile with the companion action, [github-action-atmos-terraform-plan](https://github.com/cloudposse/github-action-atmos-terraform-plan). For more, see [Atmos GitHub Action Integrations](https://atmos.tools/integrations/github-actions/atmos-terraform-apply) ## Usage ### Prerequisites This GitHub Action requires AWS access for two different purposes. This action will attempt to first pull a Terraform planfile from a S3 Bucket with metadata in a DynamoDB table with one role. Then the action will run `terraform apply` against that component with another role. We recommend configuring [OpenID Connect with AWS](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services) to allow GitHub to assume roles in AWS and then deploying both a Terraform Apply role and a Terraform State role. For Cloud Posse documentation on setting up GitHub OIDC, see our [`github-oidc-provider` component](https://docs.cloudposse.com/components/library/aws/github-oidc-provider/). In order to retrieve Terraform Plan Files (not to be confused with Terraform State files, e.g. `tfstate`), we configure an S3 Bucket to store plan files and a DynamoDB table to track plan metadata. Both need to be deployed before running this action. For more on setting up those components, see the [`gitops` component](https://docs.cloudposse.com/components/library/aws/gitops/). This action will then use the [github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage) action to update these resources. ### Config :::important **Please note!** This GitHub Action only works with `atmos >= 1.186.0`. If you are using `atmos >= 1.158.0, < 1.186.0` please use `v4` version of this action. If you are using `atmos >= 1.99.0, < 1.158.0` please use `v3` version of this action. If you are using `atmos >= 1.63.0, < 1.99.0` please use `v2` version of this action. If you are using `atmos < 1.63.0` please use `v1` version of this action. ::: The action expects the atmos configuration file `atmos.yaml` to be present in the repository. The action supports AWS and Azure to store Terraform plan files. You can read more about plan storage in the [cloudposse/github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage?tab=readme-ov-file#aws-default) documentation. Depends of cloud provider the following fields should be set in the `atmos.yaml`: #### AWS The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops # Set `apply` empty if you don't want to assume IAM role before terraform apply apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` #### Azure The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: plan-repository-type: azureblob blob-account-name: tfplans blob-container-name: plans metadata-repository-type: cosmos cosmos-container-name: terraform-plan-storage cosmos-database-name: terraform-plan-storage cosmos-endpoint: "https://my-cosmo-account.documents.azure.com:443/" # We remove the `role` section as it is AWS specific matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` ### Stack level configuration :::important Wherever it is possible to specify `integration.github.gitops` on stack level it is required to define default values in `atmos.yaml` ::: It is possible to override integration settings on a stack level by defining `settings.integrations`. ```yaml components: terraform: foobar: settings: integrations: github: gitops: artifact-storage: bucket: cptest-plat-ue2-auto-gitops table: cptest-plat-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-plat-ue2-auto-gitops-gha role: # Set `plan` empty if you don't want to assume IAM role before terraform plan plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops ``` ### Support OpenTofu This action supports [OpenTofu](https://opentofu.org/). :::important **Please note!** OpenTofu supported by Atmos `>= 1.73.0`. For details [read](https://atmos.tools/core-concepts/projects/configuration/opentofu/) ::: To enable OpenTofu add the following settings to `atmos.yaml` * Set the `opentofu-version` in the `atmos.yaml` to the desired version * Set `components.terraform.command` to `tofu` #### Example ```yaml components: terraform: command: tofu ... integrations: github: gitops: opentofu-version: 1.7.3 ... ``` ### Workflow example In this example, the action is triggered when certain events occur, such as a manual workflow dispatch or the opening, synchronization, or reopening of a pull request, specifically on the main branch. It specifies specific permissions related to assuming roles in AWS. Within the "apply" job, the "component" and "stack" are hardcoded (`foobar` and `plat-ue2-sandbox`). In practice, these are usually derived from another action. :::tip ::: We recommend combining this action with the [`affected-stacks`](https://atmos.tools/integrations/github-actions/affected-stacks) GitHub Action inside a matrix to plan all affected stacks in parallel. ```yaml name: "atmos-terraform-apply" on: workflow_dispatch: pull_request: types: - closed branches: - main # These permissions are required for GitHub to assume roles in AWS permissions: id-token: write contents: read jobs: apply: runs-on: ubuntu-latest steps: - name: Terraform Apply uses: cloudposse/github-action-atmos-terraform-apply@v5 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-config-path: ./rootfs/usr/local/etc/atmos/ ``` ### Migrating from `v4` to `v5` The notable changes in `v5` are: - `v5` works only with `atmos >= 1.186.0` - `v5` uses `atmos terraform plan-diff` to ensure changes to be applied are consistent with the stored (approved by the user) planfile. ### Migrating from `v3` to `v4` The notable changes in `v4` are: - `v4` works only with `atmos >= 1.158.0` - `v4` supports atnos `templates` and `functions` ### Migrating from `v2` to `v3` The notable changes in `v3` are: - `v3` works only with `atmos >= 1.99.0` - `v3` support azure plan and metadata storage - `v3` supports stack level integration gitops settings - `v3` allow to skip internal checkout with `skip-checkout` input The only required migration step is updating atmos version to `>= 1.99.0` ### Migrating from `v1` to `v2` The notable changes in `v2` are: - `v2` works only with `atmos >= 1.63.0` - `v2` drops `install-terraform` input because terraform is not required for affected stacks call - `v2` drops `atmos-gitops-config-path` input and the `./.github/config/atmos-gitops.yaml` config file. Now you have to use GitHub Actions environment variables to specify the location of the `atmos.yaml`. The following configuration fields now moved to GitHub action inputs with the same names | name | |-------------------------| | `atmos-version` | | `atmos-config-path` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | |--------------------------|-------------------------------------------------| | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | For example, to migrate from `v1` to `v2`, you should have something similar to the following in your `atmos.yaml`: `./.github/config/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` `.github/workflows/main.yaml` ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-apply@v2 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.63.0 ``` This corresponds to the `v1` configuration (deprecated) below. The `v1` configuration file `./.github/config/atmos-gitops.yaml` looked like this: ```yaml atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` And the `v1` GitHub Action Workflow looked like this. `.github/workflows/main.yaml` ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-apply@v1 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` ### Migrating from `v0` to `v1` 1. `v1` drops the `component-path` variable and instead fetches if directly from the [`atmos.yaml` file](https://atmos.tools/cli/configuration/) automatically. Simply remove the `component-path` argument from your invocations of the `cloudposse/github-action-atmos-terraform-apply` action. 2. `v1` moves most of the `inputs` to the Atmos GitOps config path `./.github/config/atmos-gitops.yaml`. Simply create this file, transfer your settings to it, then remove the corresponding arguments from your invocations of the `cloudposse/github-action-atmos-terraform-apply` action. | name | |--------------------------| | `atmos-version` | | `atmos-config-path` | | `terraform-state-bucket` | | `terraform-state-table` | | `terraform-state-role` | | `terraform-plan-role` | | `terraform-apply-role` | | `terraform-version` | | `aws-region` | | `enable-infracost` | If you want the same behavior in `v1` as in `v0` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v0` inputs. ```yaml - name: Terraform apply uses: cloudposse/github-action-atmos-terraform-apply@v1 with: atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml component: "foobar" stack: "plat-ue2-sandbox" ``` Which would produce the same behavior as in `v0`, doing this: ```yaml - name: Terraform apply uses: cloudposse/github-action-atmos-terraform-apply@v0 with: component: "foobar" stack: "plat-ue2-sandbox" component-path: "components/terraform/s3-bucket" terraform-apply-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops" terraform-state-bucket: "acme-core-ue2-auto-gitops" terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha" terraform-state-table: "acme-core-ue2-auto-gitops" aws-region: "us-east-2" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-config-path | The path to the atmos.yaml file | N/A | true | | atmos-version | The version of atmos to install | >= 1.186.0 | false | | branding-logo-image | Branding logo image url | https://cloudposse.com/logo-300x69.svg | false | | branding-logo-url | Branding logo url | https://cloudposse.com/ | false | | component | The name of the component to apply. | N/A | true | | debug | Enable action debug mode. Default: 'false' | false | false | | infracost-api-key | Infracost API key | N/A | false | | plan-storage | Enable plan storage. Default: 'true'. Set to 'false' to disable plan storage. | true | false | | sha | Commit SHA to apply. Default: github.sha | $\{\{ github.event.pull\_request.head.sha \}\} | true | | skip-checkout | Disable actions/checkout. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | | stack | The stack name for the given component. | N/A | true | | token | Used to pull node distributions for Atmos from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | $\{\{ github.server\_url == 'https://github.com' && github.token \|\| '' \}\} | false | ## Outputs | Name | Description | |------|-------------| | status | Apply Status. Either 'succeeded' or 'failed' | --- ## atmos-terraform-drift-detection # GitHub Action: `atmos-terraform-drift-detection` This Github Action is used to detect drift ## Introduction This Github Action is used to detect drift. It will create or update github issue once drift is detect. It is expected to run this action in a workflow with a scheduled run. There is another companion action [github-action-atmos-terraform-drift-remediation](https://github.com/cloudposse/github-action-atmos-terraform-drift-remediation). ## Usage ### Workflow example ```yaml name: 👽 Atmos Terraform Drift Detection on: schedule: - cron: "0 * * * *" permissions: id-token: write contents: write issues: write jobs: select-components: runs-on: ubuntu-latest name: Select Components outputs: matrix: ${{ steps.components.outputs.matrix }} steps: - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v0 with: jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")' debug: ${{ env.DEBUG_ENABLED }} plan-atmos-components: needs: - select-components runs-on: ubuntu-latest if: ${{ needs.select-components.outputs.matrix != '{"include":[]}' }} strategy: fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(needs.select-components.outputs.matrix) }} name: ${{ matrix.stack_slug }} env: GITHUB_TOKEN: "${{ github.token }}" steps: - name: Plan Atmos Component id: atmos-plan uses: cloudposse/github-action-atmos-terraform-plan@v0 with: component: ${{ matrix.component }} stack: ${{ matrix.stack }} component-path: ${{ matrix.component_path }} drift-detection-mode-enabled: "true" terraform-plan-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops" terraform-state-bucket: "acme-core-ue2-auto-gitops" terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha" terraform-state-table: "acme-core-ue2-auto-gitops" aws-region: "us-east-2" drift-detection: needs: - plan-atmos-components runs-on: ubuntu-latest steps: - name: Drift Detection uses: cloudposse/github-action-atmos-terraform-drift-detection@v0 with: max-opened-issues: '3' ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | assignee-teams | Comma-separated list of teams to assign issues to. You have to pass github token with `read:org` scope. This is used only when issue is getting created. | | false | | assignee-users | Comma-separated list of users to assign issues to. This is used only when issue is getting created. | | false | | labels | Comma-separated list of additional labels to assign issues to. | | false | | max-opened-issues | Number of open drift detection issues. Use `-1` to open unlimited number of issues. Default: 10 | 10 | false | | process-all | Process all issues or only the ones that relates to affected stacks. Default: false | false | false | | token | Used to pull node distributions for Atmos from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | $\{\{ github.server\_url == 'https://github.com' && github.token \|\| '' \}\} | false | --- ## atmos-terraform-drift-remediation # GitHub Action: `atmos-terraform-drift-remediation` This Github Action is used to remediate drift ## Introduction This action is used for drift remediation. There is another companion action [github-action-atmos-terraform-drift-detection](https://github.com/cloudposse/github-action-atmos-terraform-drift-detection). ## Usage ### Config :::important **Please note!** This GitHub Action only works with `atmos >= 1.158.0`. If you are using `atmos >= 1.99.0, < 1.158.0` please use `v3` version of this action. If you are using `atmos >= 1.63.0, < 1.99.0` please use `v2` version of this action. If you are using `atmos < 1.63.0` please use `v1` version of this action. ::: The action expects the atmos configuration file `atmos.yaml` to be present in the repository. The action supports AWS and Azure to store Terraform plan files. You can read more about plan storage in the [cloudposse/github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage?tab=readme-ov-file#aws-default) documentation. Depends of cloud provider the following fields should be set in the `atmos.yaml`: #### AWS The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops # Set `apply` empty if you don't want to assume IAM role before terraform apply apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` #### Azure The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: plan-repository-type: azureblob blob-account-name: tfplans blob-container-name: plans metadata-repository-type: cosmos cosmos-container-name: terraform-plan-storage cosmos-database-name: terraform-plan-storage cosmos-endpoint: "https://my-cosmo-account.documents.azure.com:443/" # We remove the `role` section as it is AWS specific matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` ### Stack level configuration :::important Wherever it is possible to specify `integration.github.gitops` on stack level it is required to define default values in `atmos.yaml` ::: It is possible to override integration settings on a stack level by defining `settings.integrations`. ```yaml components: terraform: foobar: settings: integrations: github: gitops: artifact-storage: bucket: cptest-plat-ue2-auto-gitops table: cptest-plat-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-plat-ue2-auto-gitops-gha role: # Set `plan` empty if you don't want to assume IAM role before terraform plan plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops ``` ### Support OpenTofu This action supports [OpenTofu](https://opentofu.org/). :::important **Please note!** OpenTofu supported by Atmos `>= 1.73.0`. For details [read](https://atmos.tools/core-concepts/projects/configuration/opentofu/) ::: To enable OpenTofu add the following settings to `atmos.yaml` * Set the `opentofu-version` in the `atmos.yaml` to the desired version * Set `components.terraform.command` to `tofu` #### Example ```yaml components: terraform: command: tofu ... integrations: github: gitops: opentofu-version: 1.7.3 ... ``` ### Workflow example In this example drift will be remediated when user sets label `apply` to an issue. ```yaml name: 👽 Atmos Terraform Drift Remediation run-name: 👽 Atmos Terraform Drift Remediation on: issues: types: - labeled - closed permissions: id-token: write contents: read jobs: remediate-drift: runs-on: ubuntu-latest name: Remediate Drift if: | github.event.action == 'labeled' && contains(join(github.event.issue.labels.*.name, ','), 'apply') steps: - name: Remediate Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1 with: issue-number: ${{ github.event.issue.number }} action: remediate atmos-config-path: ./rootfs/usr/local/etc/atmos/ discard-drift: runs-on: ubuntu-latest name: Discard Drift if: | github.event.action == 'closed' && !contains(join(github.event.issue.labels.*.name, ','), 'remediated') steps: - name: Discard Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1 with: issue-number: ${{ github.event.issue.number }} action: discard atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` ### Migrating from `v3` to `v4` The notable changes in `v4` are: - `v4` works only with `atmos >= 1.158.0` - `v4` supports atnos `templates` and `functions` ### Migrating from `v2` to `v3` The notable changes in `v3` are: - `v3` works only with `atmos >= 1.99.0` - `v3` use `cloudposse/github-action-atmos-terraform-apply@v3` - `v3` supports stack level integration gitops settings - `v3` allow to skip internal checkout with `skip-checkout` input The only required migration step is updating atmos version to `>= 1.99.0` ### Migrating from `v1` to `v2` The notable changes in `v2` are: - `v2` works only with `atmos >= 1.63.0` - `v2` drops `install-terraform` input because terraform is not required for affected stacks call - `v2` drops `atmos-gitops-config-path` input and the `./.github/config/atmos-gitops.yaml` config file. Now you have to use GitHub Actions environment variables to specify the location of the `atmos.yaml`. The following configuration fields now moved to GitHub action inputs with the same names | name | |-------------------------| | `atmos-version` | | `atmos-config-path` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | |--------------------------|-------------------------------------------------| | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | For example, to migrate from `v1` to `v2`, you should have something similar to the following in your `atmos.yaml`: `./.github/config/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` `.github/workflows/main.yaml` ```yaml - name: Remediate Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v2 with: issue-number: ${{ github.event.issue.number }} action: remediate atmos-config-path: ./rootfs/usr/local/etc/atmos/ ``` This corresponds to the `v1` configuration (deprecated) below. The `v1` configuration file `./.github/config/atmos-gitops.yaml` looked like this: ```yaml atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` And the `v1` GitHub Action Workflow looked like this. `.github/workflows/main.yaml` ```yaml - name: Remediate Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1 with: issue-number: ${{ github.event.issue.number }} action: remediate atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` ### Migrating from `v0` to `v1` 1. `v2` drops the `component-path` variable and instead fetches if directly from the [`atmos.yaml` file](https://atmos.tools/cli/configuration/) automatically. Simply remove the `component-path` argument from your invocations of the `cloudposse/github-action-atmos-terraform-plan` action. 2. `v2` moves most of the `inputs` to the Atmos GitOps config path `./.github/config/atmos-gitops.yaml`. Simply create this file, transfer your settings to it, then remove the corresponding arguments from your invocations of the `cloudposse/github-action-atmos-terraform-plan` action. | name | |--------------------------| | `atmos-version` | | `atmos-config-path` | | `terraform-state-bucket` | | `terraform-state-table` | | `terraform-state-role` | | `terraform-plan-role` | | `terraform-apply-role` | | `terraform-version` | | `aws-region` | | `enable-infracost` | If you want the same behavior in `v1` as in`v0` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v0` inputs. ```yaml - name: Remediate Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v1 with: issue-number: ${{ github.event.issue.number }} action: remediate atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` Which would produce the same behavior as in `v0`, doing this: ```yaml - name: Remediate Drift uses: cloudposse/github-action-atmos-terraform-drift-remediation@v0 with: issue-number: ${{ github.event.issue.number }} action: remediate atmos-config-path: "${{ github.workspace }}/rootfs/usr/local/etc/atmos/" terraform-plan-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops" terraform-state-bucket: "acme-core-ue2-auto-gitops" terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha" terraform-state-table: "acme-core-ue2-auto-gitops" aws-region: "us-east-2" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | action | Drift remediation action. One of ['remediate', 'discard'] | remediate | false | | atmos-config-path | The path to the atmos.yaml file | N/A | true | | atmos-version | The version of atmos to install | >= 1.158.0 | false | | debug | Enable action debug mode. Default: 'false' | false | false | | issue-number | Issue Number | N/A | true | | skip-checkout | Disable actions/checkout. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | | token | Used to pull node distributions for Atmos from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | $\{\{ github.server\_url == 'https://github.com' && github.token \|\| '' \}\} | false | --- ## atmos-terraform-plan # GitHub Action: `atmos-terraform-plan` This Github Action is used to run Terraform plan for a single, Atmos-supported component and save the given planfile to S3 and DynamoDB. ## Introduction This Github Action is used to run Terraform plan for a single, Atmos-supported component and save the given planfile to S3 and DynamoDB. After running this action, apply Terraform with the companion action, [github-action-atmos-terraform-apply](https://github.com/cloudposse/github-action-atmos-terraform-apply) ## Usage ### Prerequisites This GitHub Action requires AWS access for two different purposes. This action will attempt to first run `terraform plan` against a given component and then will use another role to save that given Terraform Plan to an S3 Bucket with metadata in a DynamoDB table. We recommend configuring [OpenID Connect with AWS](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services) to allow GitHub to assume roles in AWS and then deploying both a Terraform Plan role and a Terraform State role. For Cloud Posse documentation on setting up GitHub OIDC, see our [`github-oidc-provider` component](https://docs.cloudposse.com/components/library/aws/github-oidc-provider/). In order to store Terraform State, we configure an S3 Bucket to store plan files and a DynamoDB table to track plan metadata. Both will need to be deployed before running this action. For more on setting up those components, see the `gitops` component (__documentation pending__). This action will then use the [github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage) action to update these resources. ### Config :::important **Please note!** This GitHub Action only works with `atmos >= v1.158.0` If you are using `atmos >= 1.99.0, < 1.158.0` please use `v4` version of this action. If you are using `atmos >= 1.63.0, < 1.99.0` please use `v2` or `v3` version of this action. If you are using `atmos < 1.63.0` please use `v1` version of this action. ::: The action expects the atmos configuration file `atmos.yaml` to be present in the repository. The action supports AWS and Azure to store Terraform plan files. You can read more about plan storage in the [cloudposse/github-action-terraform-plan-storage](https://github.com/cloudposse/github-action-terraform-plan-storage?tab=readme-ov-file#aws-default) documentation. Depending on the cloud provider, the following fields should be set in the `atmos.yaml`: #### AWS The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: plan-repository-type: s3 metadata-repository-type: dynamo region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: # Set `plan` empty if you don't want to assume IAM role before terraform plan plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` #### Azure The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: plan-repository-type: azureblob metadata-repository-type: cosmos blob-account-name: tfplans blob-container-name: plans cosmos-container-name: terraform-plan-storage cosmos-database-name: terraform-plan-storage cosmos-endpoint: "https://my-cosmo-account.documents.azure.com:443/" # We remove the `role` section as it is AWS specific matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` ### Stack level configuration :::important Wherever it is possible to specify `integration.github.gitops` on stack level it is required to define default values in `atmos.yaml` ::: It is possible to override integration settings on a stack level by defining `settings.integrations`. ```yaml components: terraform: foobar: settings: integrations: github: gitops: artifact-storage: bucket: cptest-plat-ue2-auto-gitops table: cptest-plat-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-plat-ue2-auto-gitops-gha role: # Set `plan` empty if you don't want to assume IAM role before terraform plan plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-plat-gbl-identity-gitops ``` ### Support OpenTofu This action supports [OpenTofu](https://opentofu.org/). :::important **Please note!** OpenTofu supported by Atmos `>= 1.73.0`. For details [read](https://atmos.tools/core-concepts/projects/configuration/opentofu/) ::: To enable OpenTofu add the following settings to `atmos.yaml` * Set the `opentofu-version` in the `atmos.yaml` to the desired version * Set `components.terraform.command` to `tofu` #### Example ```yaml components: terraform: command: tofu ... integrations: github: gitops: opentofu-version: 1.7.3 ... ``` ### Workflow example ```yaml name: "atmos-terraform-plan" on: workflow_dispatch: {} pull_request: types: - opened - synchronize - reopened branches: - main # These permissions are required for GitHub to assume roles in AWS permissions: id-token: write contents: read jobs: plan: runs-on: ubuntu-latest steps: - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v2 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.158.0 ``` ### Migrating from `v4` to `v5` The notable changes in `v5` are: - `v5` works only with `atmos >= 1.158.0` - `v5` supports atnos `templates` and `functions` ### Migrating from `v3` to `v4` The notable changes in `v4` are: - `v4` works only with `atmos >= 1.99.0` - `v4` support azure plan and metadata storage - `v4` supports stack level integration gitops settings - `v4` allow to skip internal checkout with `skip-checkout` input - `v4` support creating summary comments to PR The only required migration step is updating atmos version to `>= 1.99.0` ### Migrating from `v2` to `v3` The notable changes in `v3` are: - `v3` use `actions/upload-artifact@v4` to share artifacts so it is not compatible with `cloudposse/github-action-atmos-terraform-drift-detection` `< v2.0.0` - `v3` support .terraform caching to performance improvment No special migration steps required ### Migrating from `v1` to `v2` The notable changes in `v2` are: - `v2` works only with `atmos >= 1.63.0` - `v2` drops `install-terraform` input because terraform is not required for affected stacks call - `v2` drops `atmos-gitops-config-path` input and the `./.github/config/atmos-gitops.yaml` config file. Now you have to use GitHub Actions environment variables to specify the location of the `atmos.yaml`. The following configuration fields now moved to GitHub action inputs with the same names | name | |-------------------------| | `atmos-version` | | `atmos-config-path` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | |--------------------------|-------------------------------------------------| | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | For example, to migrate from `v1` to `v2`, you should have something similar to the following in your `atmos.yaml`: `./.github/config/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` `.github/workflows/main.yaml` ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v2 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-config-path: ./rootfs/usr/local/etc/atmos/ atmos-version: 1.63.0 ``` This corresponds to the `v1` configuration (deprecated) below. The `v1` configuration file `./.github/config/atmos-gitops.yaml` looked like this: ```yaml atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` And the `v1` GitHub Action Workflow looked like this. `.github/workflows/main.yaml` ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v1 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` ### Migrating from `v1` to `v2` 1. `v2` drops the `component-path` variable and instead fetches if directly from the [`atmos.yaml` file](https://atmos.tools/cli/configuration/) automatically. Simply remove the `component-path` argument from your invocations of the `cloudposse/github-action-atmos-terraform-plan` action. 2. `v2` moves most of the `inputs` to the Atmos GitOps config path `./.github/config/atmos-gitops.yaml`. Simply create this file, transfer your settings to it, then remove the corresponding arguments from your invocations of the `cloudposse/github-action-atmos-terraform-plan` action. | name | |--------------------------| | `atmos-version` | | `atmos-config-path` | | `terraform-state-bucket` | | `terraform-state-table` | | `terraform-state-role` | | `terraform-plan-role` | | `terraform-apply-role` | | `terraform-version` | | `aws-region` | | `enable-infracost` | If you want the same behavior in `v2` as in `v1` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v1` inputs. ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v1 with: component: "foobar" stack: "plat-ue2-sandbox" atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml ``` Which would produce the same behavior as in `v1`, doing this: ```yaml - name: Plan Atmos Component uses: cloudposse/github-action-atmos-terraform-plan@v1 with: component: "foobar" stack: "plat-ue2-sandbox" component-path: "components/terraform/s3-bucket" terraform-plan-role: "arn:aws:iam::111111111111:role/acme-core-gbl-identity-gitops" terraform-state-bucket: "acme-core-ue2-auto-gitops" terraform-state-role: "arn:aws:iam::999999999999:role/acme-core-ue2-auto-gitops-gha" terraform-state-table: "acme-core-ue2-auto-gitops" aws-region: "us-east-2" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-config-path | The path to the atmos.yaml file | N/A | true | | atmos-pro-base-url | The base URL of Atmos Pro | https://atmos-pro.com | false | | atmos-pro-upload-status | If set atmos will upload the plan result to the pro API | false | false | | atmos-version | The version of atmos to install | >= 1.158.0 | false | | branding-logo-image | Branding logo image url | https://cloudposse.com/logo-300x69.svg | false | | branding-logo-url | Branding logo url | https://cloudposse.com/ | false | | component | The name of the component to plan. | N/A | true | | debug | Enable action debug mode. Default: 'false' | false | false | | drift-detection-mode-enabled | Indicate whether this action is used in drift detection workflow. | false | true | | infracost-api-key | Infracost API key | N/A | false | | metadata-retention-days | Infracost API key | 1 | false | | plan-storage | Enable plan storage. Default: 'true'. Set to 'false' to disable plan storage. | true | false | | pr-comment | Set to 'true' to create a PR comment with the summary of the plan | false | false | | sha | Commit SHA to plan. Default: github.sha | $\{\{ github.event.pull\_request.head.sha \}\} | true | | skip-checkout | Disable actions/checkout. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | | stack | The stack name for the given component. | N/A | true | | token | Used to pull node distributions for Atmos from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | $\{\{ github.server\_url == 'https://github.com' && github.token \|\| '' \}\} | false | ## Outputs | Name | Description | |------|-------------| | plan\_file | Path to the terraform plan file | | plan\_json | Path to the terraform plan in JSON format | | summary | Summary | --- ## atmos-terraform-select-components # GitHub Action: `atmos-terraform-select-components` GitHub Action that outputs list of Atmos components by jq query ## Introduction GitHub Action that outputs list of Atmos components by jq query. For example following query will fetch components that have in settings set `github.actions_enabled: true`: ``` .value.settings.github.actions_enabled // false ``` Output of this action is a list of basic component information. For example: ```json [ { "stack": "plat-ue2-sandbox", "component": "test-component-01", "stack_slug": "plat-ue2-sandbox-test-component-01", "component_path": "components/terraform/s3-bucket" } ] ``` ## Usage ### Config The action expects the atmos configuration file `atmos.yaml` to be present in the repository. The config should have the following structure: ```yaml integrations: github: gitops: opentofu-version: 1.7.3 terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` :::important **Please note!** This GitHub Action only works with `atmos >= 1.63.0`. If you are using `atmos < 1.63.0` please use `v1` version of this action. ::: ### Support OpenTofu This action supports [OpenTofu](https://opentofu.org/). :::important **Please note!** OpenTofu supported by Atmos `>= 1.73.0`. For details [read](https://atmos.tools/core-concepts/projects/configuration/opentofu/) ::: To enable OpenTofu add the following settings to `atmos.yaml` * Set the `opentofu-version` in the `atmos.yaml` to the desired version * Set `components.terraform.command` to `tofu` #### Example ```yaml components: terraform: command: tofu ... integrations: github: gitops: opentofu-version: 1.7.3 ... ``` ### GitHub Actions Workflow Example In following GitHub workflow example first job will filter components that have settings `github.actions_enabled: true` and then in following job `stack_slug` will be printed to stdout. ```yaml jobs: selected-components: runs-on: ubuntu-latest name: Select Components outputs: matrix: ${{ steps.components.outputs.matrix }} steps: - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v2 with: atmos-config-path: "${{ github.workspace }}/rootfs/usr/local/etc/atmos/" jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")' print-stack-slug: runs-on: ubuntu-latest needs: - selected-components if: ${{ needs.selected-components.outputs.matrix != '{"include":[]}' }} strategy: matrix: ${{ fromJson(needs.selected-components.outputs.matrix) }} name: ${{ matrix.stack_slug }} steps: - name: echo run: echo "${{ matrix.stack_slug }}" ``` ### Migrating from `v1` to `v2` The notable changes in `v2` are: - `v2` works only with `atmos >= 1.63.0` - `v2` drops `install-terraform` input because terraform is not required for affected stacks call - `v2` drops `atmos-gitops-config-path` input and the `./.github/config/atmos-gitops.yaml` config file. Now you have to use GitHub Actions environment variables to specify the location of the `atmos.yaml`. The following configuration fields now moved to GitHub action inputs with the same names | name | |-------------------------| | `atmos-version` | | `atmos-config-path` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | |--------------------------|-------------------------------------------------| | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | | `process-functions` | `integrations.github.gitops.matrix.process-functions` | For example, to migrate from `v1` to `v2`, you should have something similar to the following in your `atmos.yaml`: `./.github/config/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.5.2 infracost-enabled: false artifact-storage: region: us-east-2 bucket: cptest-core-ue2-auto-gitops table: cptest-core-ue2-auto-gitops-plan-storage role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha role: plan: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops apply: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` `.github/workflows/main.yaml` ```yaml - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v2 with: atmos-config-path: ./rootfs/usr/local/etc/atmos/ jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")' ``` This corresponds to the `v1` configuration (deprecated) below. The `v1` configuration file `./.github/config/atmos-gitops.yaml` looked like this: ```yaml atmos-version: 1.45.3 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: cptest-core-ue2-auto-gitops terraform-state-table: cptest-core-ue2-auto-gitops terraform-state-role: arn:aws:iam::xxxxxxxxxxxx:role/cptest-core-ue2-auto-gitops-gha terraform-plan-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::yyyyyyyyyyyy:role/cptest-core-gbl-identity-gitops terraform-version: 1.5.2 aws-region: us-east-2 enable-infracost: false sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` And the `v1` GitHub Action Workflow looked like this. `.github/workflows/main.yaml` ```yaml - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v1 with: atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")' ``` ### Migrating from `v0` to `v1` 1. `v1` replaces the `jq-query` input parameter with a new parameter called `selected-filter` to simplify the query for end-users. Now you need to specify only the part used inside of the `select(...)` function of the `jq-query`. 2.`v1` moves most of the `inputs` to the Atmos GitOps config path `./.github/config/atmos-gitops.yaml`. Simply create this file, transfer your settings to it, then remove the corresponding arguments from your invocations of the `cloudposse/github-action-atmos-terraform-select-components` action. | name | |--------------------------| | `atmos-version` | | `atmos-config-path` | If you want the same behavior in `v2` as in `v1` you should create config `./.github/config/atmos-gitops.yaml` with the same variables as in `v0` inputs. ```yaml - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v1 with: atmos-gitops-config-path: ./.github/config/atmos-gitops.yaml select-filter: '.settings.github.actions_enabled // false' ``` Which would produce the same behavior as in `v2`, doing this: ```yaml - name: Selected Components id: components uses: cloudposse/github-action-atmos-terraform-select-components@v0 with: atmos-config-path: "${{ github.workspace }}/rootfs/usr/local/etc/atmos/" jq-query: 'to_entries[] | .key as $parent | .value.components.terraform | to_entries[] | select(.value.settings.github.actions_enabled // false) | [$parent, .key] | join(",")' ``` Please note that the `atmos-gitops-config-path` is not the same file as the `atmos-config-path`. ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-config-path | The path to the atmos.yaml file | N/A | true | | atmos-version | The version of atmos to install | >= 1.63.0 | false | | debug | Enable action debug mode. Default: 'false' | false | false | | head-ref | The head ref to checkout. If not provided, the head default branch is used. | $\{\{ github.sha \}\} | false | | jq-version | The version of jq to install if install-jq is true | 1.7 | false | | nested-matrices-count | Number of nested matrices that should be returned as the output (from 1 to 3) | 2 | false | | process-functions | Whether to process atmos functions | true | false | | select-filter | jq query that will be used to select atmos components | . | false | | skip-checkout | Disable actions/checkout for head-ref. Useful for when the checkout happens in a previous step and file are modified outside of git through other actions | false | false | ## Outputs | Name | Description | |------|-------------| | has-selected-components | Whether there are selected components | | matrix | The selected components as matrix structure suitable for extending matrix size workaround (see README) | | selected-components | Selected GitOps components | --- ## auto-format # GitHub Action: `auto-format` Github Action Auto-Format runs several repository "hygiene" tasks for repositories: - The `readme` target will rebuild `README.md` from `README.yaml`. - The `github_format` target adds all of Cloud Posse's standard repository housekeeping files (including GitHub Actions workflows) to the repository's `.github` folder. - The `terraform_format` target ensures consistent formatting across all Terraform files in the repository. ## Usage If you haven't already, follow the steps in the [quickstart](#quickstart) section. To choose which pieces of functionality will be executed, modify the `script-names:` input to the `cloudposse/github-action-auto-format` step to be a comma-separated list of one or more targets (e.g., `script-names: readme,terraform_format,github_format`). This is an exhaustive list of all valid `script-name`s: - `readme` - `github_format` - `terraform_format` If you're using the `auto-format.yml` workflow file distributed within this repository, then the Auto-format GitHub Action will trigger on pull request events, once a day at 7am UTC, and upon manual triggering via the `workflow_dispatch` mechanism. ## Quick Start Here's how to get started... 1. Copy `.github/workflows/auto-format.yml` to the corresponding folder in your target repo. 2. Generate a Personal Access Token (PAT) that with the `workflow` permission *using a GitHub account that has `write` permissions in the target repo* by following the directions [here](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) and selecting the `workflow` tick box on the token creation screen. 3. Add this token as a GitHub secret in your target repository and set the `workflow-token` input of the `github-action-auto-format` step to the name of your secret. 4. Set the `bot-name` input of the `github-action-auto-format` step to the GitHub username of the user who generated the token in step 2. *This user must have `write` permissions in the target repo.` 5. By default, the Auto-Format GitHub Action will execute all of its scripts when run. If you'd like to use a subset of the full functionality, modify the `script-names` input of the `github-action-auto-format` step as described in the [usage](#usage) section. 6. (Optional) You may want to change when the scheduled cron trigger is executed. If you'd like a guide, here's a useful resource for help in crafting cron strings - https://crontab.guru/ 7. (Optional) CloudPosse recommends pinning to specific versions of actions for ease of long-term maintenance. If you care to edit the pin in `auto-format.yml` from `main` to a specific version, feel free to consult https://github.com/cloudposse/github-action-auto-format/releases for a list of available versions. ## Examples Here's a real world example: - [`github-action-auto-format`](https://github.com/cloudposse/github-action-auto-format/.github/workflows/auto-format.yml) - Cloud Posse's self-testing Auto-Format GitHub Action ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | actions-files-checkout-path | The path on the github-runner where the auto-format action scripts are checked out at runtime | github-action-auto-format | false | | bot-email | Email address associated with the GitHub user for writing new commits | N/A | false | | bot-name | GitHub username for writing new commits | cloudpossebot | false | | format-task | Name of formatting task to execute. (Options include: readme, github, terraform, and context.) | N/A | true | | workflow-token | GitHub Token for use in `github\_format.sh` and PR creation steps. This token must be granted `workflows` permissions. | N/A | true | --- ## auto-release # GitHub Action: `auto-release` This is an opinionated composite Github Action that implements a workflow based on the popular `release-drafter` action to automatically draft releases with release notes that are derived from PR descriptions as they are merged into the default branch. Under default settings, `auto-release` will also cut a new release from the default branch after every merge into it. However, releases are not cut for merges of pull requests with a `no-release` label attached. In that case, the release notes are left as a draft and a release with all unreleased changes will be made the next time a pull request without the `no-release` label is merged into the default branch. ## Usage Copy the `.github/workflows/auto-release.yml` and `.github/configs/release-drafter.yml` files from this repository into the corresponding folders of the repository to which you'd like to add Auto-release functionality. This will trigger the `auto-release` functionality every time merges are made into the default branch. ## Quick Start Here's how to get started... 1. Copy the `.github/workflows/auto-release.yml` github action workflow from this repository into the corresponding folder of the target repo 2. Copy the `.github/configs/release-drafter.yml` auto-release config file from this repository into the corresponding folder of the target repo 3. Customize the config file as desired, per the [config documentation](https://github.com/release-drafter/release-drafter#configuration) ## Examples Here's a real world example: - [`github-action-auto-release`](https://github.com/cloudposse/github-action-auto-release/.github/workflows/auto-release.yml) - The self-testing Cloud Posse Auto-format GitHub Action ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | config-name | If your workflow requires multiple release-drafter configs it is helpful to override the config-name.The config should still be located inside `.github` as that's where we are looking for config files. | configs/draft-release.yml | false | | latest | A string indicating whether the release being created or updated should be marked as latest. | | false | | prerelease | Boolean indicating whether this release should be a prerelease | | false | | publish | Whether to publish a new release immediately | false | false | | summary-enabled | Enable github action summary. | true | false | | token | Standard GitHub token (e.g., secrets.GITHUB\_TOKEN) | $\{\{ github.token \}\} | false | ## Outputs | Name | Description | |------|-------------| | body | The body of the drafted release. | | exists | Tag exists so skip new release issue | | html\_url | The URL users can navigate to in order to view the release | | id | The ID of therelease that was created or updated. | | major\_version | The next major version number. For example, if the last tag or release was v1.2.3, the value would be v2.0.0. | | minor\_version | The next minor version number. For example, if the last tag or release was v1.2.3, the value would be v1.3.0. | | name | The name of the release | | patch\_version | The next patch version number. For example, if the last tag or release was v1.2.3, the value would be v1.2.4. | | resolved\_version | The next resolved version number, based on GitHub labels. | | tag\_name | The name of the tag associated with the release. | | upload\_url | The URL for uploading assets to the release, which could be used by GitHub Actions for additional uses, for example the @actions/upload-release-asset GitHub Action. | --- ## aws-region-reduction-map # GitHub Action: `aws-region-reduction-map` Converts AWS region names from full names to abbreviations ## Introduction Converts AWS region names from full names to either "fixed" (always 3 characters) or "short" (usually 4 or 5 characters) abbreviations, following the same map as https://github.com/cloudposse/terraform-aws-utils Short abbreviations are generally the same as official AWS availability zone IDs. Generally, AWS region names have 3 parts and the "fixed" abbreviation is the first character of each part. Exceptions (due to collisions): - Africa and China use second letter of first part. - ap-south-1 is shortened to as0 to avoid conflict with ap-southeast-1 - cn-north-1 is shortened to nn0 to avoid conflict with cn-northwest-1 You should be able to list all regions with this command: ```shell aws ec2 describe-regions --all-regions --query "Regions[].{Name:RegionName}" --output text ``` but actually it leaves out GovCloud and China See https://github.com/jsonmaur/aws-regions for more complete list | long | fixed | short | |------------------|-------|---------| | `ap-east-1` | `ae1` | `ape1` | | `ap-northeast-1` | `an1` | `apne1` | | `ap-northeast-2` | `an2` | `apne2` | | `ap-northeast-3` | `an3` | `apne3` | | `ap-south-1` | `as0` | `aps1` | | `ap-southeast-1` | `as1` | `apse1` | | `ap-southeast-2` | `as2` | `apse2` | | `ca-central-1` | `cc1` | `cac1` | | `eu-central-1` | `ec1` | `euc1` | | `eu-north-1` | `en1` | `eun1` | | `eu-south-1` | `es1` | `eus1` | | `eu-west-1` | `ew1` | `euw1` | | `eu-west-2` | `ew2` | `euw2` | | `eu-west-3` | `ew3` | `euw3` | | `af-south-1` | `fs1` | `afs1` | | `us-gov-east-1` | `ge1` | `usge1` | | `us-gov-west-1` | `gw1` | `usgw1` | | `me-south-1` | `ms1` | `mes1` | | `cn-north-1` | `nn0` | `cnn1` | | `cn-northwest-1` | `nn1` | `cnnw1` | | `sa-east-1` | `se1` | `sae1` | | `us-east-1` | `ue1` | `use1` | | `us-east-2` | `ue2` | `use2` | | `us-west-1` | `uw1` | `usw1` | | `us-west-2` | `uw2` | `usw2` | ## Usage ### Convert AWS region (ex.: `us-west-2`) to fixed abbreviation - will be `uw2`. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-aws-region-reduction-map@main id: aws_map with: region: 'us-west-2' ## Format can be skipped - default format would be `fixed` if region is long format: 'fixed' outputs: result: ${{ steps.aws_map.outputs.result }} ``` ### Convert AWS region (ex.: `us-west-2`) to short abbreviation - will be `usw2`. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-aws-region-reduction-map@main id: aws_map with: region: 'us-west-2' format: 'short' outputs: result: ${{ steps.aws_map.outputs.result }} ``` ### Convert short AWS region (ex.: `usw2`) to long abbreviation - will be `us-west-2`. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-aws-region-reduction-map@main id: aws_map with: region: 'usw2' format: 'long' outputs: result: ${{ steps.aws_map.outputs.result }} ``` ### Convert fixed AWS region (ex.: `uw2`) to long abbreviation - will be `us-west-2`. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-aws-region-reduction-map@main id: aws_map with: region: 'uw2' format: 'long' outputs: result: ${{ steps.aws_map.outputs.result }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | format | Format convert to. Valid values ('long', 'short', 'fixed'). If empty short and fixed inputs are converted to long, long inputs are converted to fixed. | N/A | false | | region | Input region code | N/A | true | ## Outputs | Name | Description | |------|-------------| | result | Converted AWS region | --- ## datadog-notify # GitHub Action: `datadog-notify` Create Datadog Notify Event ## Introduction This repository contains the action for sending an event to datadog. ## Usage Minimal Usage: ```yaml - name: Notify Datadog uses: cloudposse/github-action-datadog-notify@main with: api_key: ## ${{ env.DATADOG_API_KEY }} ## ${{secrets.DATADOG_API_KEY}} title: "GitHub Action: ${{ github.event_name }}" text: "GitHub Action: ${{ github.event_name }}" tags: "source:github,repo:${{ github.repository }},event:${{ github.event_name }}" alert_type: "info" ``` Below is a snippet that will send an event to datadog when a pull request is sync'd. Below uses the `dkershner6/aws-ssm-getparameters-action` to get the datadog api key from ssm. ```yaml name: Datadog Notify on: workflow_dispatch: pull_request: branches: - 'main' permissions: contents: read pull-requests: write id-token: write jobs: datadog-notify: runs-on: ["self-hosted"] steps: - uses: actions/checkout@v3 - name: Configure AWS credentials id: aws-credentials uses: aws-actions/configure-aws-credentials@v1 with: role-to-assume: ${{ secrets.AWS_ROLE_TO_ASSUME }} role-session-name: "gha-datadog-notify" aws-region: "us-east-1" - uses: dkershner6/aws-ssm-getparameters-action@v1 with: parameterPairs: "/datadog/datadog_api_key = DATADOG_API_KEY" - name: Notify Datadog uses: cloudposse/github-action-datadog-notify@main with: api_key: ${{ env.DATADOG_API_KEY }} title: "GitHub Action: ${{ github.event_name }}" text: "GitHub Action: ${{ github.event_name }}" tags: "source:github,repo:${{ github.repository }},event:${{ github.event_name }}" alert_type: "info" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | alert\_type | Type of the event, one of: error,warning,info,success,user\_update,recommendation,snapshot | info | true | | api\_key | Datadog API Key | N/A | true | | append\_hostname\_tag | Should we append the hostname as a tag to the event, set this to the key of the tag | | false | | tags | Space separated list of Tags for the event | N/A | true | | text | Description of the event | N/A | true | | title | Title of the event | N/A | true | ## Outputs | Name | Description | |------|-------------| --- ## deploy-argocd # GitHub Action: `deploy-argocd` Deploy on Kubernetes with ArgoCD ## Introduction Deploy on Kubernetes with Helm/HelmFile and ArgoCD. ## Usage Deploy environment ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened] jobs: deploy: runs-on: ubuntu-latest environment: name: preview url: ${{ steps.deploy.outputs.webapp-url }} steps: - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1.7.0 with: aws-region: us-west-2 role-to-assume: arn:aws:iam::111111111111:role/preview role-session-name: deploy - name: Deploy uses: cloudposse/github-action-deploy-argocd@main id: deploy with: cluster: https://github.com/cloudposse/argocd-deploy-non-prod-test/blob/main/plat/ue2-sandbox/apps toolchain: helmfile environment: preview namespace: preview application: test-app github-pat: ${{ secrets.GITHUB_AUTH_PAT }} repository: ${{ github.repository }} ref: ${{ github.event.pull_request.head.ref }} image: nginx image-tag: latest operation: deploy debug: false synchronously: true ``` Destroy environment ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [closed] jobs: destroy: runs-on: ubuntu-latest steps: - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1.7.0 with: aws-region: us-west-2 role-to-assume: arn:aws:iam::111111111111:role/preview role-session-name: destroy - name: Destroy uses: cloudposse/github-action-deploy-helmfile@main id: destroy with: cluster: https://github.com/cloudposse/argocd-deploy-non-prod-test/blob/main/plat/ue2-sandbox/apps toolchain: helmfile environment: preview namespace: preview application: test-app github-pat: ${{ secrets.GITHUB_AUTH_PAT }} repository: ${{ github.repository }} ref: ${{ github.event.pull_request.head.ref }} image: "" image-tag: "" operation: destroy debug: false ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | application | Application name | N/A | true | | aws-region | AWS region | us-east-1 | false | | check-retry-count | Check retry count (for synchronously mode) | 5 | false | | check-retry-interval | Check retry interval (in seconds) (for synchronously mode) | 10 | false | | cluster | Cluster name | N/A | true | | commit-retry-count | Commit retry count | 4 | false | | commit-retry-interval | Commit retry interval (in seconds) | 10 | false | | commit-status-github-token | Github token to access the app repository. Defaults to github-pat if not set. | N/A | false | | commit-timeout | Commit timeout (in seconds) | 60 | false | | debug | Debug mode | false | false | | environment | Helmfile environment | preview | false | | github-pat | Github PAT to access argocd configuration repository | N/A | true | | gitref-sha | Git SHA (Depricated. Use `ref` instead) | | false | | helm-args | Additional helm arguments | | false | | helm-dependency-build | Run helm dependency build, only for helm toolchain, `true` or `false` | false | false | | helm-version | Helm version | v3.10.2 | false | | helmfile-args | Additional helmfile arguments | | false | | helmfile-version | Helmfile version | v0.148.1 | false | | image | Docker image | N/A | true | | image-tag | Docker image tag | N/A | true | | namespace | Kubernetes namespace | N/A | true | | operation | Operation with helmfiles. (valid options - `deploy`, `destroy`) | deploy | true | | path | The path where lives the helmfile or helm chart. | N/A | true | | ref | Git ref | N/A | true | | release\_label\_name | The name of the label used to describe the helm release | release | false | | repository | Application GitHub repository full name | N/A | true | | ssm-path | SSM path to read environment secrets | N/A | true | | synchronously | Wait until ArgoCD successfully apply the changes | false | false | | toolchain | Toolchain ('helm', 'helmfile') | helmfile | false | | values\_file | Helm values file, this can be a single file or a comma separated list of files | | false | ## Outputs | Name | Description | |------|-------------| | sha | Git commit SHA into argocd repo | | webapp-url | Web Application url | --- ## deploy-ecspresso # GitHub Action: `deploy-ecspresso` Deploy on ECS with [Escpresso](https://github.com/kayac/ecspresso) ## Introduction This is template repository to create composite GitHub Actions. Feel free to use it as reference and starting point. ## Usage ```yaml name: Pull Request on: push: branches: [ 'main' ] jobs: context: runs-on: ubuntu-latest steps: - name: Example action uses: cloudposse/example-github-action-deploy-ecspresso@main id: example with: image: 1111111111111.dkr.ecr.us-east-2.amazonaws.com/cloudposse/example-app-on-ecs image-tag: latest region: us-east-2 operation: deploy cluster: acme-plat-ue2-sandbox application: acme-plat-ue2-sandbox-example-app-on-ecs taskdef-path: taskdef.json outputs: result: ${{ steps.example.outputs.webapp-url }} ``` ## S3 Mirroring S3 Mirroring is a pattern of uploading the deployed task definition to an S3 Bucket. This is so that the task definition can be updated with the latest image tag, and the terraform does not reset it back to a previous tag set in the infrastructure repository. ## Partial Task Definition A "Partial Task Definition" is an authoring pattern where the application repository maintains only the parts of the ECS task definition that the app team owns or changes frequently (for example, container image/tag, environment variables, command/args, CPU/memory for a container). The more static, infrastructure-owned parts (for example, IAM roles, volumes, EFS mounts, log configuration, task-level networking) are provided by a template maintained in the infrastructure repository. During deployment, this action merges the infrastructure-provided template (optionally fetched from S3) with the local partial task definition from the application repo to produce a complete `task-definition.json`, which is then deployed by `ecspresso`. ![S3 Mirroring](https://github.com/cloudposse/github-action-deploy-ecspresso/tree/main/docs/github-action-deploy-ecspresso_s3-mirroring.drawio.png) ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | application | Application name | N/A | true | | cluster | Cluster name | N/A | true | | debug | Debug mode | false | false | | ecspresso-version | Ecspresso version | v2.1.0 | false | | image | Docker image | N/A | true | | image-tag | Docker image tag | N/A | true | | mirror\_to\_s3\_bucket | Mirror task definition to s3 bucket | N/A | false | | operation | Operation (valid options - `deploy`, `destroy`) | deploy | true | | region | AWS Region | N/A | true | | taskdef-path | Task definition path | N/A | true | | timeout | Ecspresso timeout | 5m | false | | use\_partial\_taskdefinition | NOTE: Experimental. Load templated task definition from S3 bucket, which is created by the `ecs-service` component. This is useful when you want to manage the task definition in the infrastructure repository and the application repository. The infrastructure repository manages things like Volumes and EFS mounts, and the Application repository manages the application code and environment variables. | N/A | false | ## Outputs | Name | Description | |------|-------------| | webapp-url | Web Application url | --- ## deploy-helmfile # GitHub Action: `deploy-helmfile` Deploy on Kubernetes with HelmFile ## Introduction Deploy on Kubernetes with HelmFile. ## Usage Deploy environment ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened] jobs: deploy: runs-on: ubuntu-latest environment: name: preview url: ${{ steps.deploy.outputs.webapp-url }} steps: - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1.7.0 with: aws-region: us-west-2 role-to-assume: arn:aws:iam::111111111111:role/preview role-session-name: deploy - name: Deploy uses: cloudposse/github-action-deploy-helmfile@main id: deploy with: aws-region: us-west-2 cluster: preview-eks environment: preview namespace: preview image: nginx image-tag: latest operation: deploy debug: false ``` Destroy environment ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [closed] jobs: destroy: runs-on: ubuntu-latest steps: - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1.7.0 with: aws-region: us-west-2 role-to-assume: arn:aws:iam::111111111111:role/preview role-session-name: destroy - name: Destroy uses: cloudposse/github-action-deploy-helmfile@main id: destroy with: aws-region: us-west-2 cluster: preview-eks environment: preview namespace: preview image: "" image-tag: "" operation: destroy debug: false ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | aws-region | AWS region | us-east-1 | false | | chamber\_version | Kubectl version | 2.11.1 | false | | cluster | Cluster name | N/A | true | | debug | Debug mode | false | false | | environment | Helmfile environment | preview | false | | gitref-sha | Git SHA | | false | | helm\_version | Helm version | 3.11.1 | false | | helmfile | Helmfile name | helmfile.yaml | false | | helmfile-path | The path where lives the helmfile. | deploy | false | | helmfile\_version | Helmfile version | 0.143.5 | false | | image | Docker image | N/A | true | | image-tag | Docker image tag | N/A | true | | kubectl\_version | Kubectl version | 1.26.3 | false | | namespace | Kubernetes namespace | N/A | true | | operation | Operation with helmfiles. (valid options - `deploy`, `destroy`) | deploy | true | | release\_label\_name | The name of the label used to describe the helm release | release | false | | url-resource-type | The type of the resource to get the URL from | ingress | false | | values\_yaml | YAML string with extra values to use in a helmfile deploy | N/A | false | ## Outputs | Name | Description | |------|-------------| | webapp-url | Web Application url | --- ## deploy-spacelift # GitHub Action: `deploy-spacelift` Opinionated way to deploy Docker image app with Spacelift ## Introduction Set Docker image uri into SSM parameter store and trigger Spacelift stack that should handle deployment. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: deploy: runs-on: ubuntu-latest environment: name: production url: ${{ steps.deploy.outputs.webapp-url }} steps: - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1.7.0 with: aws-region: us-west-2 role-to-assume: arn:aws:iam::123456789012:role/AllowWriteSSM - name: Deploy uses: cloudposse/github-action-deploy-spacelift@main id: deploy with: stack: ecs-service-production region: us-west-2 ssm-path: /ecs-service/image image: nginx image-tag: latest operation: deploy debug: false github_token: ${{ secrets.GITHUB_TOKEN }} organization: acme api_key_id: ${{ secrets.SPACELIFT_API_KEY_ID }} api_key_secret: ${{ secrets.SPACELIFT_API_KEY_SECRET }} outputs: url: ${{ steps.deploy.outputs.webapp-url }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | api\_key\_id | Spacelift API Key ID | N/A | true | | api\_key\_secret | Spacelift API Key Secret | N/A | true | | debug | Debug mode | false | false | | github\_token | GitHub Token | N/A | true | | image | Docker image | N/A | true | | image-tag | Docker image tag | N/A | true | | namespace | Namespace | N/A | false | | operation | Operation (valid options - `deploy`, `destroy`) | deploy | true | | organization | Spacelift organization name | N/A | true | | region | AWS Region | N/A | true | | ssm-path | SSM path for Docker image | N/A | false | | stack | Spacelift stack name | N/A | true | | webapp-output-name | Spacelist stack output field contains webapp host name | full\_domain | false | ## Outputs | Name | Description | |------|-------------| | webapp-url | Web Application url | --- ## docker-build-push # GitHub Action: `docker-build-push` Build Docker image and push it ## Introduction Build Docker image and push it. ## Usage ```yaml name: Push into main branch on: push: branches: [ master ] jobs: context: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@main with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" login: "${{ secrets.DOCKERHUB_USERNAME }}" password: "${{ secrets.DOCKERHUB_PASSWORD }}" platforms: linux/amd64,linux/arm64 outputs: image: ${{ steps.build.outputs.image }} tag: ${{ steps.build.outputs.tag }} ``` :::tip If omitted, `cache-from` and `cache-to` will default to `gha`. In an AWS environment, we recommend using [ECR as a remote cache](https://aws.amazon.com/blogs/containers/announcing-remote-cache-support-in-amazon-ecr-for-buildkit-clients/). ::: ```diff - name: Build id: build uses: cloudposse/github-action-docker-build-push@main with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" + cache-from: "type=registry,ref=registry.hub.docker.com/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:cache" + cache-to: "mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=registry.hub.docker.com/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:cache" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | allow | List of extra privileged entitlement (e.g., network.host,security.insecure) | N/A | false | | binfmt-image | Binfmt image | public.ecr.aws/eks-distro-build-tooling/binfmt-misc:qemu-v7.0.0 | false | | build-args | List of build-time variables | N/A | false | | build-contexts | List of additional build contexts (e.g., name=path) | N/A | false | | buildkitd-flags | BuildKit daemon flags | --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host | false | | cache-from | List of external cache sources for buildx (e.g., user/app:cache, type=local,src=path/to/dir) | type=gha | false | | cache-to | List of cache export destinations for buildx (e.g., user/app:cache, type=local,dest=path/to/dir) | type=gha,mode=max | false | | debug | Enable debug mode | false | false | | docker-metadata-pr-head-sha | Set to `true` to tag images with the PR HEAD SHA instead of the merge commit SHA within pull requests. | false | false | | driver-opts | List of additional driver-specific options. (eg. image=moby/buildkit:master) | image=public.ecr.aws/vend/moby/buildkit:buildx-stable-1 | false | | file | Dockerfile name | Dockerfile | false | | image\_name | Image name (excluding registry). Defaults to \{\{$organization/$repository\}\}. | | false | | inspect | Set to `true` will pull and inspect the image and output it to the step summary. | false | false | | login | Docker login | | false | | network | Set the networking mode for the RUN instructions during build | N/A | false | | no-cache | Send the --no-cache flag to the docker build process | false | false | | organization | Organization | N/A | true | | password | Docker password | | false | | platforms | List of target platforms for build (e.g. linux/amd64,linux/arm64,linux/riscv64,linux/ppc64le,linux/s390x,etc) | linux/amd64 | false | | provenance | Generate provenance attestation for the build | N/A | false | | registry | Docker registry | N/A | true | | repository | Repository | N/A | true | | secret-files | List of secret files to expose to the build (e.g., key=filename, MY\_SECRET=./secret.txt) | N/A | false | | secrets | List of secrets to expose to the build (e.g., key=string, GIT\_AUTH\_TOKEN=mytoken) | N/A | false | | ssh | List of SSH agent socket or keys to expose to the build | N/A | false | | tags | List of tags (supports https://github.com/docker/metadata-action#tags-input) | N/A | false | | target | Sets the target stage to build | | false | | workdir | Working directory | ./ | false | ## Outputs | Name | Description | |------|-------------| | image | Docker image name | | metadata | Docker image metadata | | tag | Docker image tag | --- ## docker-compose-test-run # GitHub Action: `docker-compose-test-run` Up docker compose and run tests in specific container ## Introduction Run tests in enviroment defined with Docker Compose ## Usage ```yaml name: Push into Main on: push: branches: [ master ] jobs: test: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Tests uses: cloudposse/github-action-docker-compose-test-run@main with: file: test/docker-compose.yml service: app command: test/unit-tests.sh ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | command | Command to run tests | N/A | true | | docker-compose-version | Docker compose version | 1.29.2 | false | | entrypoint | Entrypoint | /bin/sh | false | | file | Docker compose file | N/A | true | | login | Docker login | | false | | password | Docker password | | false | | registry | Docker registry | N/A | true | | service | Service run tests inside | N/A | true | | workdir | Working directory | ./ | false | ## Outputs | Name | Description | |------|-------------| --- ## docker-image-exists # GitHub Action: `docker-image-exists` Check if docker image exists by pulling it ## Usage ```yaml name: Push into main branch on: push: branches: [ master ] jobs: context: runs-on: ubuntu-latest continue-on-error: true steps: - name: Check image id: image_exists uses: cloudposse/github-action-docker-image-exists@main with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" login: "${{ secrets.DOCKERHUB_USERNAME }}" password: "${{ secrets.DOCKERHUB_PASSWORD }}" tag: latest outputs: result: ${{ steps.image_exists.conclusion }} image: ${{ steps.image_exists.outputs.image }} tag: ${{ steps.image_exists.outputs.tag }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | image\_name | Image name (excluding registry). Defaults to \{\{$organization/$repository\}\}. | | false | | login | Docker login | | false | | organization | Organization | N/A | true | | password | Docker password | | false | | registry | Docker registry | N/A | true | | repository | Repository | N/A | true | | tag | Tag | N/A | true | ## Outputs | Name | Description | |------|-------------| | image | Docker image name | | tag | Docker image tag | --- ## docker-promote # GitHub Action: `docker-promote` Promote docker image ## Introduction Promote Docker image to specific tags provided explicitly or implicitly with [Docker Metadata action](https://github.com/marketplace/actions/docker-metadata-action) ## Usage ### Promote a docker image to specific tag ```yaml name: Release on: release: types: [published] permissions: id-token: write contents: write jobs: promote: runs-on: ubuntu-latest steps: - name: Docker image promote uses: cloudposse/github-action-docker-promote@main id: promote with: registry: registry.hub.docker.com organization: ${{ github.event.repository.owner.login }} repository: ${{ github.event.repository.name }} login: ${{ secrets.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_PASSWORD }} from: sha-${{ github.sha }} to: ${{ github.event.release.tag_name }} use_metadata: false outputs: image: ${{ steps.promote.outputs.image }} tag: ${{ steps.promote.outputs.tag }} ``` ### Promote a docker image to tags detected from metadata Promote action use [Docker Metadata action](https://github.com/marketplace/actions/docker-metadata-action) under the hood and can detect `to` tags based on Git reference and GitHub events. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 with: fetch-depth: 2 - name: Get previous commit id: prev-commit run: echo "sha=$(git rev-parse --verify HEAD^1)" >> $GITHUB_OUTPUT - name: Docker image promote uses: cloudposse/github-action-docker-promote@main id: promote with: registry: registry.hub.docker.com organization: ${{ github.event.repository.owner.login }} repository: ${{ github.event.repository.name }} login: ${{ secrets.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_PASSWORD }} from: sha-${{ steps.prev-commit.outputs.sha }} use_metadata: true outputs: image: ${{ steps.promote.outputs.image }} tag: ${{ steps.promote.outputs.tag }} ``` ### Promote a docker image with `from` fetched from metadata If you skip `from` tag then it would be populated as SHA of the current commit in long format. ```yaml name: Release on: release: types: [published] permissions: id-token: write contents: write jobs: promote: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Docker image promote uses: cloudposse/github-action-docker-promote@main id: promote with: registry: registry.hub.docker.com organization: ${{ github.event.repository.owner.login }} repository: ${{ github.event.repository.name }} login: ${{ secrets.DOCKERHUB_USERNAME }} password: ${{ secrets.DOCKERHUB_PASSWORD }} ## `from` is long SHA to: ${{ github.event.release.tag_name }} use_metadata: true outputs: image: ${{ steps.promote.outputs.image }} tag: ${{ steps.promote.outputs.tag }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | from | Source tag | N/A | false | | image\_name | Image name (excluding registry). Defaults to \{\{$organization/$repository\}\}. | | false | | login | Docker login | | false | | organization | Organization | N/A | true | | password | Docker password | | false | | promote-retry-max-attempts | Promote retry max attempts | 3 | false | | promote-retry-timeout-seconds | Promote retry timeout seconds | 3000 | false | | promote-retry-wait-seconds | Promote retry wait seconds | 30 | false | | registry | Docker registry | N/A | true | | repository | Repository | N/A | true | | to | Target tags | N/A | false | | use\_metadata | Extract target tags from Git reference and GitHub events | true | false | ## Outputs | Name | Description | |------|-------------| | image | Docker image name | | tag | Docker image tag | --- ## interface-environment # GitHub Action: `interface-environment` Get Environments settings from private settings action provider ## Introduction Get Environments settings from private settings action provider. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Example action uses: cloudposse/example-github-action-composite@main id: environment with: implementation_repository: cloudposse/actions-private implementation_path: environments implementation_ref: main implementation_github_pat: ${{ secrets.GITHUB_REPO_ACCESS_TOKEN }} environment: dev namespace: dev outputs: name: "${{ steps.environment.outputs.name }}" region: "${{ steps.environment.outputs.region }}" role: "${{ steps.environment.outputs.role }}" cluster: "${{ steps.environment.outputs.cluster }}" namespace: "${{ steps.environment.outputs.namespace }}" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | application | Application name | N/A | false | | attributes | Comma separated attributes | N/A | false | | environment | Environment name | N/A | true | | implementation\_file | Repository filename with Environment action implementation | action.yaml | true | | implementation\_github\_pat | GitHub PAT allow fetch environment action implementation | N/A | true | | implementation\_path | Repository path with Environment action implementation | | true | | implementation\_ref | Ref of environment action implementation | main | true | | implementation\_repository | Repository with Environment action implementation | N/A | true | | namespace | Namespace name | N/A | true | | repository | Repository name | N/A | false | ## Outputs | Name | Description | |------|-------------| | cluster | Environments that need to be destroyed | | name | Environment name | | namespace | Namespace | | region | JSON formatted \{label\}: \{environment\} map | | role | Environments that need to be deployed | | s3-bucket | S3 Bucket for ECS taskdef mirroring | | ssm-path | Path to ssm secrets | --- ## jq # GitHub Action: `jq` Process a input with a jq script and output result as step output ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: example: outputs: result: "${{ steps.current.outputs.output }}" steps: - uses: cloudposse/github-action-jq@main id: current with: compact: true input: '["test", "test2", "test3"]' script: |- map(select(. == "test")) ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | compact | Compact instead of pretty-printed output | false | false | | input | JSON file or JSON formatted string | N/A | true | | raw-output | Output raw strings, not JSON texts | false | false | | remove-trailing-newline | Remove trailing newline | true | false | | script | JQ query string | N/A | true | ## Outputs | Name | Description | |------|-------------| | output | Output from the jq command | --- ## kubernetes-environment # GitHub Action: `kubernetes-environment` This repository wraps the environment information action, allowing it to be used as a replacement in support of various string functions and namespace standardization. ## Introduction We often find when deploying with various environments that we need to standardize the namespace names. This repository wraps the environment information action, allowing it to be used as a replacement in support of various string functions and namespace standardization. With this action, you can use pipe functions to standardize the namespace names, for example, to lowercase, or to replace a dash with an underscore you can use ` | kebabcase` or `| toLower` ## Usage To use this action, you'll want to create a workflow and argocd repository. This action is intended to replace cloudposse/github-action-yaml-config-query by wrapping it with helper actions. With this action your `config` input can have several helper functions. * `reformat` this replaces the namespace with a flavor of your choice. this is a key added to an environments configuration. See snipped below for example * `branch-name` will use the branch name as the namespace * `pr-number` will use the PR number as the namespace * `| functions`: you can now perform simple string operations on keys in your environment configuration. This can help prevent dns invalid characters from becoming your namespace based on the branch name. * `| kebabcase` will convert the string to kebabcase (alternatively you can use `| toKebab` or `| kebab`) * `| lowercase` will convert the string to lowercase (alternatively you can use `| toLower` or `| lower`) * `| uppercase` will convert the string to uppercase (alternatively you can use `| toUpper` or `| upper`) Though this is perhaps less helpful as it is not valid in kubernetes nor dns. ```yaml - name: Environment info # We recommend pinning this action to a specific release or version range to ensure stability uses: cloudposse/github-action-kubernetes-environment@main id: result with: environment: ${{ inputs.environment }} namespace: ${{ inputs.namespace }} application: ${{ inputs.application }} config: | preview: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-dev/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: ${{ inputs.namespace }} reformat: branch-name # reformats namespace to be branch name as kebabcase, alternatively use `pr-number` here for `pr-123` as your namespace ssm-path: platform/dev-cluster qa1: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: QA1/MY-APP | kebabcase # output namespace will become qa1-my-app ssm-path: platform/staging-cluster ``` * To get custom key value pairs you can query the selected environment with a follow up step: ```yaml - name: Environment info uses: cloudposse/github-action-yaml-config-query@v1.0.0 id: environment-info with: query: . config: ${{ steps.result.outputs.environment-config }} ```
Full Workflow example: ```yaml name: 'Environments - ArgoCD' description: 'Get information about environment' inputs: environment: description: "Environment name" required: true application: description: "The application name" required: false namespace: description: "Namespace name" required: true outputs: name: description: "Environment name" value: ${{ inputs.environment }} region: description: "Default AWS Region" value: us-east-2 role: description: "Environments that need to be deployed" value: ${{ steps.result.outputs.role }} cluster: description: "Environments that need to be destroyed" value: ${{ steps.result.outputs.cluster }} namespace: description: "Namespace" value: ${{ steps.result.outputs.namespace }} ssm-path: description: "Path to ssm secrets" value: ${{ steps.result.outputs.ssm-path }} runs: using: "composite" steps: - name: Environment info # We recommend pinning this action to a specific release or version range to ensure stability uses: cloudposse/github-action-kubernetes-environment@main id: result with: environment: ${{ inputs.environment }} namespace: ${{ inputs.namespace }} application: ${{ inputs.application }} config: | preview: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-dev/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: ${{ inputs.namespace }} ssm-path: platform/dev-cluster reformat: branch-name qa1: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: qa1 ssm-path: platform/staging-cluster qa2: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: qa2 ssm-path: platform/staging-cluster qa3: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: qa3 ssm-path: platform/staging-cluster qa4: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: qa4 ssm-path: platform/staging-cluster production: cluster: https://github.com/athoteldev/argocd-deploy-prod/blob/main/plat/ue2-prod/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: production ssm-path: platform/prod-cluster staging: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-staging/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: staging ssm-path: platform/staging-cluster sandbox: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-sandbox/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: sandbox ssm-path: platform/sandbox-cluster dev: cluster: https://github.com/cloudposse/argocd-repo/blob/main/plat/ue2-dev/apps cluster-role: arn:aws:iam::123456789012:role/my-gha-cluster-role namespace: dev ssm-path: platform/dev-cluster ```
## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | application | The application name | N/A | false | | config | configuration | N/A | true | | environment | Environment | N/A | true | | namespace | Kubernetes namespace | N/A | true | | namespace-deny-list | Kubernetes namespace deny list, generated names cannot contain this comma separated list. | kube-system,kube-public,default | false | | namespace-prefix | Kubernetes namespace prefix | | false | | namespace-suffix | Kubernetes namespace suffix | | false | ## Outputs | Name | Description | |------|-------------| | cluster | Environments that need to be destroyed | | environment-config | Environment configuration | | name | Environment name | | namespace | Namespace | | role | Environments that need to be deployed | | ssm-path | Path to ssm secrets | --- ## major-release-tagger # GitHub Action: `major-release-tagger` GitHub Action that automatically generates or updates `v` tags every time a new release is published. ## Introduction This GitHub Action automatically generates or updates `v` tags every time a new release is published, making it effortless to keep track of your project's major versions. Imagine your Git repository has the following tags: ``` 1.0.0 1.1.0 2.0.0 2.0.1 2.1.0 3.0.0 ``` By simply incorporating Major Release Tagger, your repo will be enriched with the corresponding v-tags: ``` 1.0.0 1.1.0 v1 2.0.0 2.0.1 v2 2.1.0 3.0.0 v3 ``` When you create a new release tagged `3.1.0`, the `v3` tag will automatically point to it: ``` 1.0.0 1.1.0 v1 2.0.0 2.0.1 v2 2.1.0 3.0.0 3.1.0 v3 ``` Stay organized and efficient with Major Release Tagger - the ultimate GitHub Action to streamline your versioning process. ## Usage ```yaml name: Major Release Tagger on: release: types: - published jobs: publish: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-major-release-tagger@v1 ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | dry-run | Run action without pushing changes to upstream | false | false | | git-user-email | Git user email that will be used for git config | actions-bot@users.noreply.github.com | false | | git-user-name | Git user name that will be used for git config | actions-bot | false | | log-level | Log level for this action. Available options: ['off', 'error', 'warn', 'info', 'debug']. Default 'info' | info | false | | token | Standard GitHub token (e.g., secrets.GITHUB\_TOKEN) | $\{\{ github.token \}\} | false | ## Outputs | Name | Description | |------|-------------| | response | Response in json format for example: \{"succeeded":true,"reason":"MAPPED\_TAGS","message":"Successfully created/update v-tags.","data":\{"v1": \{"state":"updated", "oldSHA": "d9b3a3034766ac20294fd1c36cacc017ae4a3898", "newSHA":"e5c6309b473934cfe3e556013781b8757c1e0422"\}, "v2": \{"state":"created", "oldSHA": "bbf9f924752c61dcef084757bcf4440e23f2e16b", "newSHA":"5ae37ee514b73cf8146fe389ad839469e7f3a6d2"\}\}\} | --- ## matrix-extended # GitHub Action: `matrix-extended` GitHub Action that when used together with reusable workflows makes it easier to workaround the limit of 256 jobs in a matrix. ## Introduction GitHub Actions matrix have [limit to 256 items](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#using-a-matrix-strategy) There is workaround to extend the limit with [reusable workflows](https://github.com/orgs/community/discussions/38704) This GitHub Action outputs a JSON structure for up to 3 levels deep of nested matrixes. In theory run 256 ^ 3 (i.e., 16 777 216) jobs per workflow run! | Matrix max nested level | Total jobs count limit | |-------------------------|--------------------------| | 1 | 256 | | 2 | 65 536 | | 3 | 16 777 216 | If `nested-matrices-count` input is `1`, the output `matrix` would be JSON formatted string with the following structure ```yaml { "include": [matrix items] } ``` If `nested-matrices-count` input is `2` output `matrix` whould be a JSON formatted string with the following structure ```yaml { "include": [{ "name": "group name", "items": { "include": [matrix items] } ## serialized as string }] } ``` If `nested-matrices-count` input is `3` output `matrix` would be a JSON formatted string with the following structure ```yaml { "include": [{ "name": "group name", "items": [{ "name": "chunk 256 range name", "include": [ "items": { "include": [matrix items] ## serialized as string } ] }] ## serialized as string } ## serialized as string }] } ``` :::warning Make sure you [restrict the concurrency](https://docs.github.com/en/actions/using-jobs/using-concurrency) of your jobs to avoid DDOS'ing the GitHub Actions API, which might cause restrictions to be applied to your account. | Matrix max nested level | First Matrix Concurrency | Second Matrix Concurrency | Third Matrix Concurrency | |-------------------------|--------------------------|---------------------------|--------------------------| | 1 | x | - | - | | 2 | 1 | x | - | | 3 | 1 | 1 | x | ::: ## Usage The action have 3 modes depends of how many nested levels you want. The settings affect to reusable workflows count and usage pattern. ## 1 Level of nested matrices `.github/workflows/matrices-1.yml` ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: matrix-builder: runs-on: self-hosted name: Affected stacks outputs: matrix: ${{ steps.extend.outputs.matrix }} steps: - id: setup-matrix uses: druzsan/setup-matrix@v1 with: matrix: | os: ubuntu-latest windows-latest macos-latest, python-version: 3.8 3.9 3.10 arch: arm64 amd64 - uses: cloudposse/github-action-matrix-extended@main id: extend with: matrix: ${{ steps.setup-matrix.outputs.matrix }} sort-by: '[.python-version, .os, .arch] | join("-")' group-by: '.arch' nested-matrices-count: '1' operation: if: ${{ needs.matrix-builder.outputs.matrix != '{"include":[]}' }} needs: - matrix-builder strategy: max-parallel: 10 fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(needs.matrix-builder.outputs.matrix) }} name: Do (${{ matrix.arch }}) runs-on: self-hosted steps: - shell: bash run: | echo "Do real work - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ matrix.python-version }}" ``` ## 2 Level of nested matrices `.github/workflows/matrices-1.yml` ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: matrix-builder: runs-on: self-hosted name: Affected stacks outputs: matrix: ${{ steps.extend.outputs.matrix }} steps: - id: setup-matrix uses: druzsan/setup-matrix@v1 with: matrix: | os: ubuntu-latest windows-latest macos-latest, python-version: 3.8 3.9 3.10 arch: arm64 amd64 - uses: cloudposse/github-action-matrix-extended@main id: extend with: sort-by: '[.python-version, .os, .arch] | join("-")' group-by: '.arch' nested-matrices-count: '1' matrix: ${{ steps.setup-matrix.outputs.matrix }} operation: if: ${{ needs.matrix-builder.outputs.matrix != '{"include":[]}' }} uses: ./.github/workflows/matrices-2.yml needs: - matrix-builder strategy: max-parallel: 1 # This is important to avoid ddos GHA API fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(needs.matrix-builder.outputs.matrix) }} name: Group (${{ matrix.name }}) with: items: ${{ matrix.items }} ``` `.github/workflows/matrices-2.yml` ```yaml name: Reusable workflow for 2 level of nested matrices on: workflow_call: inputs: items: description: "Items" required: true type: string jobs: operation: if: ${{ inputs.items != '{"include":[]}' }} strategy: max-parallel: 10 fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(inputs.items) }} name: Do (${{ matrix.arch }}) runs-on: self-hosted steps: - shell: bash run: | echo "Do real work - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ matrix.python-version }}" ``` ## 3 Level of nested matrices `.github/workflows/matrices-1.yml` ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: matrix-builder: runs-on: self-hosted name: Affected stacks outputs: matrix: ${{ steps.extend.outputs.matrix }} steps: - id: setup-matrix uses: druzsan/setup-matrix@v1 with: matrix: | os: ubuntu-latest windows-latest macos-latest, python-version: 3.8 3.9 3.10 arch: arm64 amd64 - uses: cloudposse/github-action-matrix-extended@main id: query with: sort-by: '[.python-version, .os, .arch] | join("-")' group-by: '.arch' nested-matrices-count: '1' matrix: ${{ steps.setup-matrix.outputs.matrix }} operation: if: ${{ needs.matrix-builder.outputs.matrix != '{"include":[]}' }} uses: ./.github/workflows/matrices-2.yml needs: - matrix-builder strategy: max-parallel: 1 # This is important to avoid ddos GHA API fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(needs.matrix-builder.outputs.matrix) }} name: Group (${{ matrix.name }}) with: items: ${{ matrix.items }} ``` `.github/workflows/matrices-2.yml` ```yaml name: Reusable workflow for 2 level of nested matrices on: workflow_call: inputs: items: description: "Items" required: true type: string jobs: operation: if: ${{ inputs.items != '{"include":[]}' }} uses: ./.github/workflows/matrices-3.yml strategy: max-parallel: 1 # This is important to avoid ddos GHA API fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(inputs.items) }} name: Group (${{ matrix.name }}) with: items: ${{ matrix.items }} ``` `.github/workflows/matrices-3.yml` ```yaml name: Reusable workflow for 3 level of nested matrices on: workflow_call: inputs: items: description: "Items" required: true type: string jobs: operation: if: ${{ inputs.items != '{"include":[]}' }} strategy: max-parallel: 10 fail-fast: false # Don't fail fast to avoid locking TF State matrix: ${{ fromJson(inputs.items) }} name: Do (${{ matrix.arch }}) runs-on: self-hosted steps: - shell: bash run: | echo "Do real work - ${{ matrix.os }} - ${{ matrix.arch }} - ${{ matrix.python-version }}" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | group-by | Group by query | empty | false | | matrix | Matrix inputs (JSON array or object which includes property passed as string or file path) | N/A | true | | nested-matrices-count | Number of nested matrices that should be returned as the output (from 1 to 3) | 1 | false | | sort-by | Sort by query | empty | false | ## Outputs | Name | Description | |------|-------------| | matrix | A matrix suitable for extending matrix size workaround (see README) | --- ## matrix-outputs-read # GitHub Action: `matrix-outputs-read` [Workaround implementation](https://github.com/community/community/discussions/17245#discussioncomment-3814009) - Read matrix jobs outputs ## Introduction GitHub actions have an [Jobs need a way to reference all outputs of matrix jobs](https://github.com/community/community/discussions/17245) issue. If there is a job that runs multiple times with `strategy.matrix` only the latest iteration's output availiable for reference in other jobs. There is a [workaround](https://github.com/community/community/discussions/17245#discussioncomment-3814009) to address the limitation. We implement the workaround with two GitHub Actions: * [Matrix Outputs Write](https://github.com/cloudposse/github-action-matrix-outputs-write) * [Matrix Outputs Read](https://github.com/cloudposse/github-action-matrix-outputs-read) ## v1 - What's new :::important cloudposse/github-action-matrix-outputs-read@v1+ is not currently supported on GHES yet. If you are on GHES, you must use [v0](https://github.com/cloudposse/github-action-matrix-outputs-read/releases/tag/0.1.2). ::: The release of `cloudposse/github-action-matrix-outputs-write@v1` and `cloudposse/github-action-matrix-outputs-read@v1` are major changes to the backend architecture of Artifacts. They have numerous performance and behavioral improvements. For more information, see the [`@actions/artifact`](https://github.com/actions/toolkit/tree/main/packages/artifact) documentation. ### Breaking Changes 1. On self hosted runners, additional [firewall rules](https://github.com/actions/toolkit/tree/main/packages/artifact#breaking-changes) may be required. 2. `cloudposse/github-action-matrix-outputs-read@v1` can not be read outputs writen by `cloudposse/github-action-matrix-outputs-write@v0`. ## Usage Example how you can use workaround to reference matrix job outputs. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: runs-on: ubuntu-latest strategy: matrix: platform: ["i386", "arm64v8"] steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@1.9.0 with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" build-args: |- PLATFORM=${{ matrix.platform }} ## Write for matrix outputs workaround - uses: cloudposse/github-action-matrix-outputs-write@v1 id: out with: matrix-step-name: ${{ github.job }} matrix-key: ${{ matrix.platform }} outputs: |- image: ${{ steps.build.outputs.image }}:${{ steps.build.outputs.tag }} ## Read matrix outputs read: runs-on: ubuntu-latest needs: [build] steps: - uses: cloudposse/github-action-matrix-outputs-read@v1 id: read with: matrix-step-name: build outputs: result: "${{ steps.read.outputs.result }}" ## This how you can reference matrix output assert: runs-on: ubuntu-latest needs: [read] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.i386 }} - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:arm64v8 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.arm64v8 }} ``` ### Reusable workflow example Reusable workflow that support matrix outputs `./.github/workflow/build-reusabled.yaml` ```yaml name: Build - Reusable workflow on: workflow_call: inputs: registry: required: true type: string organization: required: true type: string repository: required: true type: string platform: required: true type: string matrix-step-name: required: false type: string matrix-key: required: false type: string outputs: image: description: "Image" value: ${{ jobs.write.outputs.image }} jobs: build: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@1.9.0 with: registry: ${{ inputs.registry }} organization: ${{ inputs.organization }} repository: ${{ inputs.repository }} build-args: |- PLATFORM=${{ inputs.platform }} outputs: image: ${{ needs.build.outputs.image }}:${{ needs.build.outputs.tag }} write: runs-on: ubuntu-latest needs: [build] steps: ## Write for matrix outputs workaround - uses: cloudposse/github-action-matrix-outputs-write@v1 id: out with: matrix-step-name: ${{ inputs.matrix-step-name }} matrix-key: ${{ inputs.matrix-key }} outputs: |- image: ${{ needs.build.outputs.image }} outputs: image: ${{ fromJson(steps.out.outputs.result).image }} ``` Then you can use the workflow with matrix ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: usage: ./.github/workflow/build-reusabled.yaml strategy: matrix: platform: ["i386", "arm64v8"] with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" platform: ${{ matrix.platform }} matrix-step-name: ${{ github.job }} matrix-key: ${{ matrix.platform }} ## Read matrix outputs read: runs-on: ubuntu-latest needs: [build] steps: - uses: cloudposse/github-action-matrix-outputs-read@v1 id: read with: matrix-step-name: build outputs: result: "${{ steps.read.outputs.result }}" ## This how you can reference matrix output assert: runs-on: ubuntu-latest needs: [read] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.i386 }} - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:arm64v8 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.arm64v8 }} ``` or as a simple job ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: usage: ./.github/workflow/build-reusabled.yaml with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" platform: "i386" ## This how you can reference single job output assert: runs-on: ubuntu-latest needs: [build] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ needs.build.outputs.image }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | matrix-step-name | Matrix step name | N/A | true | ## Outputs | Name | Description | |------|-------------| | result | Outputs result | --- ## matrix-outputs-write # GitHub Action: `matrix-outputs-write` [Workaround implementation](https://github.com/community/community/discussions/17245#discussioncomment-3814009) - Write matrix jobs outputs ## Introduction GitHub actions have an [Jobs need a way to reference all outputs of matrix jobs](https://github.com/community/community/discussions/17245) issue. If there is a job that runs multiple times with `strategy.matrix` only the latest iteration's output availiable for reference in other jobs. There is a [workaround](https://github.com/community/community/discussions/17245#discussioncomment-3814009) to address the limitation. We implement the workaround with two GitHub Actions: * [Matrix Outputs Write](https://github.com/cloudposse/github-action-matrix-outputs-write) * [Matrix Outputs Read](https://github.com/cloudposse/github-action-matrix-outputs-read) ## v1 - What's new :::important cloudposse/github-action-matrix-outputs-write@v1+ is not currently supported on GHES yet. If you are on GHES, you must use [v0](https://github.com/cloudposse/github-action-matrix-outputs-write/releases/tag/0.5.0). ::: The release of `cloudposse/github-action-matrix-outputs-write@v1` and `cloudposse/github-action-matrix-outputs-read@v1` are major changes to the backend architecture of Artifacts. They have numerous performance and behavioral improvements. For more information, see the [`@actions/artifact`](https://github.com/actions/toolkit/tree/main/packages/artifact) documentation. ### Breaking Changes 1. On self hosted runners, additional [firewall rules](https://github.com/actions/toolkit/tree/main/packages/artifact#breaking-changes) may be required. 2. Outputs writen with `cloudposse/github-action-matrix-outputs-write@v1` can not be read by `cloudposse/github-action-matrix-outputs-read@v0`and below versions. ## Usage Example how you can use workaround to reference matrix job outputs. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: runs-on: ubuntu-latest strategy: matrix: platform: ["i386", "arm64v8"] steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@1.9.0 with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" build-args: |- PLATFORM=${{ matrix.platform }} ## Write for matrix outputs workaround - uses: cloudposse/github-action-matrix-outputs-write@v1 id: out with: matrix-step-name: ${{ github.job }} matrix-key: ${{ matrix.platform }} outputs: |- image: ${{ steps.build.outputs.image }}:${{ steps.build.outputs.tag }} ## Multiline string tags: ${{ toJson(steps.build.outputs.image) }} ## Read matrix outputs read: runs-on: ubuntu-latest needs: [build] steps: - uses: cloudposse/github-action-matrix-outputs-read@v1 id: read with: matrix-step-name: build outputs: result: "${{ steps.read.outputs.result }}" ## This how you can reference matrix output assert: runs-on: ubuntu-latest needs: [read] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.i386 }} - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:arm64v8 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.arm64v8 }} ``` ### Reusable workflow example Reusable workflow that support matrix outputs `./.github/workflow/build-reusabled.yaml` ```yaml name: Build - Reusable workflow on: workflow_call: inputs: registry: required: true type: string organization: required: true type: string repository: required: true type: string platform: required: true type: string matrix-step-name: required: false type: string matrix-key: required: false type: string outputs: image: description: "Image" value: ${{ jobs.write.outputs.image }} jobs: build: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@1.9.0 with: registry: ${{ inputs.registry }} organization: ${{ inputs.organization }} repository: ${{ inputs.repository }} build-args: |- PLATFORM=${{ inputs.platform }} outputs: image: ${{ needs.build.outputs.image }}:${{ needs.build.outputs.tag }} write: runs-on: ubuntu-latest needs: [build] steps: ## Write for matrix outputs workaround - uses: cloudposse/github-action-matrix-outputs-write@v1 id: out with: matrix-step-name: ${{ inputs.matrix-step-name }} matrix-key: ${{ inputs.matrix-key }} outputs: |- image: ${{ needs.build.outputs.image }} outputs: image: ${{ fromJson(steps.out.outputs.result).image }} image_alternative: ${{ steps.out.outputs.image }} ``` Then you can use the workflow with matrix ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: usage: ./.github/workflow/build-reusabled.yaml strategy: matrix: platform: ["i386", "arm64v8"] with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" platform: ${{ matrix.platform }} matrix-step-name: ${{ github.job }} matrix-key: ${{ matrix.platform }} ## Read matrix outputs read: runs-on: ubuntu-latest needs: [build] steps: - uses: cloudposse/github-action-matrix-outputs-read@v1 id: read with: matrix-step-name: build outputs: result: "${{ steps.read.outputs.result }}" ## This how you can reference matrix output assert: runs-on: ubuntu-latest needs: [read] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.i386 }} - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:arm64v8 ## This how you can reference matrix output actual: ${{ fromJson(needs.read.outputs.result).image.arm64v8 }} ``` or as a simple job ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: build: usage: ./.github/workflow/build-reusabled.yaml with: registry: registry.hub.docker.com organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" platform: "i386" ## This how you can reference single job output assert: runs-on: ubuntu-latest needs: [build] steps: - uses: nick-fields/assert-action@v1 with: expected: ${{ registry.hub.docker.com }}/${{ github.event.repository.owner.login }}/${{ github.event.repository.name }}:i386 ## This how you can reference matrix output actual: ${{ needs.build.outputs.image }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | matrix-key | Matrix key | N/A | false | | matrix-step-name | Matrix step name | N/A | false | | outputs | YAML structured map of outputs | N/A | false | ## Outputs | Name | Description | |------|-------------| | result | Outputs result (Deprecated!!!) | --- ## monorepo-random-controller # GitHub Action: `monorepo-random-controller` Monorepo random controller used for demo ## Introduction Monorepo pattern for CI/CD use this action as controller to detect list of applications, applications with changes. The GitHub action detects as applications directories from specified path and use random to separate them into changed and unchanged lists. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: monorepo: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v3 - name: Monorepo controller id: controller uses: cloudposse/github-action-monorepo-random-controller@0.1.1 with: dir: ./applications/ outputs: applications: ${{ steps.controller.outputs.apps }} changes: ${{ steps.controller.outputs.changes }} no-changes: ${{ steps.controller.outputs.no-changes }} ci: runs-on: ubuntu-latest needs: [monorepo] if: ${{ needs.monorepo.outputs.applications != '[]' }} strategy: matrix: application: ${{ fromJson(needs.monorepo.outputs.applications) }} steps: - name: Checkout uses: actions/checkout@v3 - name: Build id: build uses: cloudposse/github-action-docker-build-push@1.9.0 with: registry: registry.hub.docker.com organization: ${{ github.event.repository.owner.login }} repository: ${{ github.event.repository.name }}/${{ matrix.application }} workdir: ./applications/${{ matrix.application }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | dir | Applications dir | N/A | true | ## Outputs | Name | Description | |------|-------------| | apps | Applications list | | changes | Applications that have changes | | no-changes | Applications that have no changes | --- ## preview-environment-controller # GitHub Action: `preview-environment-controller` Action to manage to deploy and purge preview environments depends on PR labels ## Introduction Testing Pull Request changes usually lead to having it deployed on a preview environment. The environment can be ephemeral or pre-provisioned. In the last case, there is a countable number of preview environments. This GitHub Action follows a pattern when the developer set PR label to specify a preview environment to deploy. `github-action-preview-environment-controller` allow to define map of `environment => label`. Depending on current PR labels the action outputs a list of deploy and destroy environments. So it performs a `controller` role and does not limit deployment methods or tools. ## Usage Use `github-action-preview-environment-controller` in Pull Request triggered pipeline, and use it's outputs to determinate what environments should be deployed and what cleaned up. ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Preview deployments controller uses: cloudposse/github-action-preview-environment-controller@main id: controller with: labels: ${{ toJSON(github.event.pull_request.labels.*.name) }} open: ${{ github.event.pull_request.state == 'open' }} env-label: | preview: deploy qa1: deploy/qa1 qa2: deploy/qa2 outputs: labels_env: ${{ steps.controller.outputs.labels_env }} deploy_envs: ${{ steps.controller.outputs.deploy_envs }} destroy_envs: ${{ steps.controller.outputs.destroy_envs }} deploy: runs-on: ubuntu-latest if: ${{ needs.context.outputs.deploy_envs != '[]' }} strategy: matrix: env: ${{ fromJson(needs.context.outputs.deploy_envs) }} environment: name: ${{ matrix.env }} needs: [ context ] steps: - name: Deploy uses: example/deploy@main id: deploy with: environment: ${{ matrix.env }} operation: deploy destroy: runs-on: ubuntu-latest if: ${{ needs.context.outputs.destroy_envs != '[]' }} strategy: matrix: env: ${{ fromJson(needs.context.outputs.destroy_envs) }} needs: [ context ] steps: - name: Destroy uses: example/deploy@main id: deploy with: environment: ${{ matrix.env }} operation: destroy ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | env-label | YAML formatted \{environment\}: \{label\} map | preview: deploy | true | | labels | Existing PR labels | [] | true | | open | Is PR open? | true | true | ## Outputs | Name | Description | |------|-------------| | deploy\_envs | Environments that need to be deployed | | destroy\_envs | Environments that need to be destroyed | | labels\_env | JSON formatted \{label\}: \{environment\} map | --- ## preview-labels-cleanup # GitHub Action: `preview-labels-cleanup` Remove labels used to control deployments with [github-action-preview-environment-controller](https://github.com/cloudposse/github-action-preview-environment-controller) ## Introduction On close a pull request we need to cleanup all labels that specify a preview environments where the PR was deployed. This GitHub action integrates with [github-action-preview-environment-controller](https://github.com/cloudposse/github-action-preview-environment-controller) action ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Preview deployments controller uses: cloudposse/github-action-preview-environment-controller@v0.7.0 id: controller with: labels: ${{ toJSON(github.event.pull_request.labels.*.name) }} open: ${{ github.event.pull_request.state == 'open' }} env-label: | preview: deploy outputs: labels_env: ${{ steps.controller.outputs.labels_env }} destroy: runs-on: ubuntu-latest if: ${{ github.event.pull_request.state != 'open' }} needs: [ context ] steps: - name: Cleanup label uses: cloudposse/github-action-preview-labels-cleanup with: labels_env: ${{ needs.context.outputs.labels_env }} env: preview ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | env | Environment | N/A | true | | labels\_env | JSON formatted \{label\}: \{environment\} map | \{\} | true | | pr\_number | The number of the pull request, which will default to extracting from the workflow event if not specified. | N/A | false | ## Outputs | Name | Description | |------|-------------| --- ## release-branch-manager # GitHub Action: `release-branch-manager` GitHub Action for Managing Release Branches ## Introduction This GitHub Action adopts a streamlined approach to managing release branches, drawing on a trunk-based branching strategy. In this model, the `DEFAULT_BRANCH` consistently represents the most recent release, while release branches are exclusively created for previous major releases, if applicable. This structure simplifies the process for contributors when submitting Pull Requests for bug fixes or backporting modifications to older releases, as it enables them to target a specific major release. **How it works:** upon publishing a new major release `N`, a corresponding branch for the previous release `N-1` will be automatically generated. Imagine you have tags like this in your repo: ``` 0.1.0 0.2.0 1.0.0 1.1.0 1.2.0 1.2.1 2.0.0 2.1.0 2.2.0 3.0.0 3.1.0 main ``` Upon the first release published event, the "release branch manager" will generate new branches named `release/vN-1`, where N corresponds to the latest tag of each major release. In this case, several new branches will be created: ``` 0.1.0 0.2.0 release/v0 1.0.0 1.1.0 1.2.0 1.2.1 release/v1 2.0.0 2.1.0 2.2.0 release/v2 3.0.0 3.1.0 main ``` Note that `3.1.0` is latest tag and release branch manager wouldn't create release branch because latest major release is maintained in `main` branch. If you wish to make changes to `2.2.0`, you must create a pull request for the `release/v2` branch and generate a corresponding release/tag with a major version of `2`, for example, `2.3.0`. This action requires GitHub releases to follow the [SemVer versioning](https://semver.org/) scheme. ## Usage Example of workflow that that will create major release tags. To use it, just add this workflow to your `.github/workflows` directory. ```yaml name: Manager Release Branch on: release: types: - published jobs: publish: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-release-branch-manager@v1 ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | dry-run | Run action without pushing changes to upstream | false | false | | git-user-email | Git user email that will be used for git config | actions-bot@users.noreply.github.com | false | | git-user-name | Git user name that will be used for git config | actions-bot | false | | log-level | Log level for this action. Available options: ['off', 'error', 'warn', 'info', 'debug']. Default 'info' | info | false | | minimal-version | Minimal 'major' version that release branch creation should start from | 0 | false | | token | GitHub Token used to perform git and GitHub operations | $\{\{ github.token \}\} | false | ## Outputs | Name | Description | |------|-------------| | response | Response in json format for example: \{"succeeded":true,"reason":"CREATED\_BRANCHES","message":"Successfully created release branches","data":\{"release/v3":"3.1.0","release/v2":"2.0.0","release/v1":"1.1.0"\}\} | --- ## release-label-validator # GitHub Action: `release-label-validator` This GitHub Action validates that the major label is only assigned to Pull Requests targeting the default branch, enhancing the management of significant changes. ## Introduction This is a GitHub Action to validate that only Pull Requests targeting the default branch can have the `major` label set. This is useful in combination with the [`release-drafter`](https://github.com/release-drafter/release-drafter) and the Cloud Posse [`release-branch-manager`](https://github.com/cloudposse/github-action-release-branch-manager) GitHub Actions, to ensure the `major` label can only be assigned to Pull Requests created against the default branch, ensuring that significant changes are clearly identified and properly managed. ## Usage ```yaml name: validate-release-labels on: pull_request: types: - labeled - unlabeled - opened - synchronize - reopened jobs: validate: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-release-label-validator@v1 ``` --- ## run-ecspresso # GitHub Action: `run-ecspresso` Run ECS task with [Escpresso](https://github.com/kayac/ecspresso) ## Introduction This is template repository to create composite GitHub Actions. Feel free to use it as reference and starting point. ## Usage ```yaml name: Pull Request on: push: branches: [ 'main' ] jobs: context: runs-on: ubuntu-latest steps: - name: Example action uses: cloudposse/example-github-action-run-ecspresso@main id: example with: image: 1111111111111.dkr.ecr.us-east-2.amazonaws.com/cloudposse/example-app-on-ecs image-tag: latest region: us-east-2 operation: deploy cluster: acme-plat-ue2-sandbox application: acme-plat-ue2-sandbox-example-app-on-ecs taskdef-path: taskdef.json overrides: |- { "containerOverrides":[ { "name": "app", "command": ["/db-migrate.sh"] } ] } outputs: result: ${{ steps.example.outputs.webapp-url }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | application | Application name | N/A | true | | cluster | Cluster name | N/A | true | | debug | Debug mode | false | false | | ecspresso-version | Ecspresso version | v2.1.0 | false | | image | Docker image | N/A | true | | image-tag | Docker image tag | N/A | true | | mirror\_to\_s3\_bucket | Mirror task definition to s3 bucket | N/A | false | | overrides | A list of container overrides in JSON format that specify the name of a container in the specified task definition and the overrides it should receive. | \{\} | false | | region | AWS Region | N/A | true | | taskdef-path | Task definition path | N/A | true | | timeout | Ecspresso timeout | 5m | false | | use\_partial\_taskdefinition | NOTE: Experimental. Load templated task definition from S3 bucket, which is created by the `ecs-service` component. This is useful when you want to manage the task definition in the infrastructure repository and the application repository. The infrastructure repository manages things like Volumes and EFS mounts, and the Application repository manages the application code and environment variables. | N/A | false | ## Outputs | Name | Description | |------|-------------| | webapp-url | Web Application url | --- ## secret-outputs # GitHub Action: `secret-outputs` This GitHub Action implement [workaround](https://nitratine.net/blog/post/how-to-pass-secrets-between-runners-in-github-actions/) for the problem [`Combining job outputs with masking leads to empty output`](https://github.com/actions/runner/issues/1498). The problem was described in [`GitHub Action documentation`](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idoutputs) - `Outputs containing secrets are redacted on the runner and not sent to GitHub Actions`. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Step with the secret output id: iam run: | echo "role=arn:aws:iam::${{ secrets.AWS_ACCOUNT_ID }}:role/admin" >> $GITHUB_OUTPUT - uses: cloudposse/github-action-secret-outputs@main id: role with: ## PASSWORD is a gpg passphrase stored in Github Secrets. secret: ${{ secrets.PASSWORD }} op: encode in: ${{ steps.iam.outputs.role }} outputs: role: ${{ steps.role.outputs.out }} usage: runs-on: ubuntu-latest needs: [context] steps: - uses: cloudposse/github-action-secret-outputs@main id: role with: ## PASSWORD is a gpg passphrase stored in Github Secrets. secret: ${{ secrets.PASSWORD }} op: decode in: ${{ needs.context.outputs.role }} - name: Configure AWS Credentials uses: aws-actions/configure-aws-credentials@v1 with: role-to-assume: ${{ steps.role.outputs.out }} aws-region: us-east-2 ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | in | Input data | N/A | true | | op | Operation to perform (encode or decode) | encode | true | | secret | Secret to encrypt/decrypt data | N/A | true | ## Outputs | Name | Description | |------|-------------| | out | Result of encryption/decryption | --- ## seek-deployment # GitHub Action: `seek-deployment` Get GitHub deployment object by ref and environment name ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Seek deployment uses: cloudposse/github-action-seek-deployment@main id: deployment with: github-token: ${{ secrets.GITHUB_TOKEN }} environment: dev ref: ${{ github.event.pull_request.head.ref }} status: success outputs: id: "${{ steps.deployment.outputs.id }}" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | environment | Environment name | N/A | true | | github-token | The GitHub token | $\{\{ github.token \}\} | true | | ref | Branch or commit SHA | N/A | true | | status | Deployment status | N/A | false | ## Outputs | Name | Description | |------|-------------| | id | Top Deployment ID | | ids | All Matching Deployment IDs | --- ## setup-atmos # GitHub Action: `setup-atmos` Install atmos for use in GitHub Actions ## Introduction This repo contains a GitHub Action to setup [atmos](https://github.com/cloudposse/atmos) for use in GitHub Actions. It installs the specified version of atmos and adds it to the `PATH` so it can be used in subsequent steps. In addition, it optionally installs a wrapper script that will capture the `stdout`, `stderr`, and `exitcode` of the `atmos` command and make them available to subsequent steps via outputs of the same name. ## Usage ```yaml steps: - uses: hashicorp/setup-terraform@v2 - name: Setup atmos uses: cloudposse/github-action-setup-atmos@v2 ```` To install a specific version of atmos, set the `version` input: ```yaml steps: - uses: hashicorp/setup-terraform@v2 - name: Setup atmos uses: cloudposse/github-action-setup-atmos@v2 with: version: 0.15.0 ```` The wrapper script installation can be skipped by setting the `install-wrapper` input to `false`: ```yaml steps: - uses: hashicorp/setup-terraform@v2 - name: Setup atmos uses: cloudposse/github-action-setup-atmos@v2 with: install-wrapper: false ```` Subsequent steps of the GitHub action can use the wrapper scipt to capture the `stdout`, `stderr`, and `exitcode` if the wrapper script was installed: ```yaml steps: - uses: hashicorp/setup-terraform@v2 - name: Setup atmos uses: cloudposse/github-action-setup-atmos@v2 with: install-wrapper: true - name: Run atmos id: atmos run: atmos terraform plan - run: echo ${{ steps.atmos.outputs.stdout }} - run: echo ${{ steps.atmos.outputs.stderr }} - run: echo ${{ steps.atmos.outputs.exitcode }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | atmos-version | Version Spec of the version to use. Examples: 1.x, 10.15.1, >=10.15.0. | latest | false | | install-wrapper | Flag to indicate if the wrapper script will be installed to wrap subsequent calls of the `atmos` binary and expose its STDOUT, STDERR, and exit code as outputs named `stdout`, `stderr`, and `exitcode` respectively. Defaults to `true`. | true | false | | token | Used to pull atmos distributions from Cloud Posse's GitHub repository. Since there's a default, this is typically not supplied by the user. When running this action on github.com, the default value is sufficient. When running on GHES, you can pass a personal access token for github.com if you are experiencing rate limiting. | $\{\{ github.server\_url == 'https://github.com' && github.token \|\| '' \}\} | false | ## Outputs | Name | Description | |------|-------------| | atmos-version | The installed atmos version. | --- ## spacelift-stack-deploy # GitHub Action: `spacelift-stack-deploy` Trigger Spacelist stack synchronously ## Introduction [Spacelift](https://spacelift.io) is a sophisticated, continuous integration and deployment (CI/CD) platform for infrastructure-as-code. The GitHub action triggers Spacelift stack run to provistion infrastructure. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: infrastructure: runs-on: ubuntu-latest steps: - uses: cloudposse/github-action-spacelift-stack-deploy@main id: spacelift with: stack: eks-cluster github_token: ${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }} organization: acme api_key_id: ${{ secrets.SPACELIFT_API_KEY_ID }} api_key_secret: ${{ secrets.SPACELIFT_API_KEY_SECRET }} outputs: outputs: ${{ steps.spacelift.outputs.outputs }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | api\_key\_id | API Key ID | N/A | true | | api\_key\_secret | API Key Secret | N/A | true | | autodeploy | If true, automatically deploy the stack without manual confirmation | false | false | | github\_token | GitHub Token (Required to install Spacelift CLI) | N/A | true | | organization | Organization name | N/A | true | | stack | Stack name | N/A | true | ## Outputs | Name | Description | |------|-------------| | outputs | Stack outputs | --- ## sync-docker-repos # GitHub Action: `sync-docker-repos` GitHub Action to sync two docker repositories. ## Introduction GitHub Action to sync two docker repositories ## Usage Below is an example workflow that uses the `github-action-sync-docker-repos` action to sync a Docker Hub repository with an AWS ECR repository. ```yaml jobs: example: runs-on: ubuntu-latest steps: - name: Configure AWS credentials id: login-aws uses: aws-actions/configure-aws-credentials@v2 with: aws-region: us-east-1 aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} - name: Login to Amazon ECR Private id: login-ecr uses: aws-actions/amazon-ecr-login@v1 with: mask-password: "true" - name: sync uses: cloudposse/github-action-sync-docker-repos@main with: src: busybox dest: 111111111111.dkr.ecr.us-east-1.amazonaws.com dest-credentials: "${{ steps.login-ecr.outputs.docker_username_111111111111_dkr_ecr_us_east_1_amazonaws_com }}:${{ steps.login-ecr.outputs.docker_password_111111111111_dkr_ecr_us_east_1_amazonaws_com }}" ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | dest | The destination repository to sync to. | N/A | true | | dest-credentials | The destination repository credentials. | N/A | false | | override-arch | Override the architecture of the src image. | N/A | false | | override-multi-arch | If one of the images in src refers to a list of images, instead of copying just the image which matches thecurrent OS and architecture, attempt to copy all of the images in the list, and the list itself. | true | false | | override-os | Override the operating system of the src image. | N/A | false | | src | The source repository to sync from. | N/A | true | | src-credentials | The source repository credentials. | N/A | false | --- ## terraform-auto-context # GitHub Action: `terraform-auto-context` This is a Github Action that will automatically update the `context.tf` file in the calling repo against the most recent version published by Cloud Posse. If a new version is detected, a Pull Request will be opened to update it. If the repo version is found to be out of date, a pull request is opened to update it. ## Usage Copy this repository's `.github/workflows/auto-context.yml` file into the `.github/workflows` folder of the repository to which you'd like to add Terraform Auto-context functionality. This will cause Auto-context functionality to execute daily at the time specified by the `cron` option (all times are UTC). If you'd like to modify the schedule of the Auto-context action, you can follow the standard [cron](https://en.wikipedia.org/wiki/Cron) syntax, as detailed below: ``` schedule: # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed # Update README.md nightly at 4am UTC - cron: '0 4 * * *' ``` The default username and email address attached to the commits generated by the Auto-context action are `cloudpossebot` and `11232728+cloudpossebot@users.noreply.github.com`. If you would like to change these defaults, please set the `bot-name` and `bot-email` inputs in the workflow file: ``` with: bot-name: [name] bot-email: [email] ``` ## Quick Start Here's how to get started... 1. Copy this repository's `.github/workflows/auto-context.yml` file into the `.github/workflows` folder of the repository to which you'd like to add Terraform Auto-context functionality. 2. (Optional) Update the `main` pin inside `auto-context.yml` to a fixed version. Consult https://github.com/cloudposse/github-action-auto-context/releases for a list of available versions. ## Examples Here are some real world examples: - [`github-action-auto-context`](https://github.com/cloudposse/github-action-auto-context/.github/workflows/auto-context.yml) - Cloud Posse's self-testing Auto-context GitHub Action ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | bot-email | Email to write commits under | N/A | false | | bot-name | Username to write commits under | cloudpossebot | false | | branch-name | Name of branch to commit updated context.tf to | auto-update/context.tf | false | | token | Token for authenticating to GitHub API server. No special permissions needed. | N/A | true | --- ## terraform-plan-storage # GitHub Action: `terraform-plan-storage` A GitHub Action to securely store Terraform plan files in a cloud storage (S3 or Azure Blob Storage) with metadata storage in cloud document database (DynamoDB or CosmosDB). ## Introduction A Github Action to securely store Terraform plan files in a cloud storage (S3 or Azure Blob Storage) with metadata storage in cloud document database (DynamoDB or CosmosDB). This is useful in CI/CD pipelines where you want to store the plan files when a feature branch is opened and applied when merged. ## Usage ## AWS (default) Standard usage for this action is with AWS. In AWS, we store Terraform plan files in a S3 Bucket and store metadata in DynamoDB. Specify the DynamoDB table name and S3 bucket name with `tableName` and `bucketName` respectively. The filepath in S3 and the attributes in DynamoDB will use the given `component` and `stack` values to update or create a unique target for each Terraform plan file. The plan file itself is pulled from or writen to a local file path. Set this with `planPath`. Finally, choose whether to store the plan file or retrieve an existing plan file. To create or update a plan file, set `action` to `storePlan`. To pull an existing plan file, set `action` to `getPlan`. ```yaml - name: Store Plan uses: cloudposse/github-action-terraform-plan-storage@v1 id: store-plan with: action: storePlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 tableName: acme-terraform-plan-metadata bucketName: acme-terraform-plans - name: Get Plan uses: cloudposse/github-action-terraform-plan-storage@v1 id: get-plan with: action: getPlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 tableName: acme-terraform-plan-metadata bucketName: acme-terraform-plans ``` ## Azure This action also supports Azure. In Azure, we store Terraform plan files with Blob Storage and store metadata in Cosmos DB. To use the Azure implementation rather than the default AWS implementation, specify `planRepositoryType` as `azureblob` and `metadataRepositoryType` as `cosmos`. Then pass the Blob Account and Container names with `blobAccountName` and `blobContainerName` and the Cosmos Container name, Database name, and Endpoint with `cosmosContainerName`, `cosmosDatabaseName`, and `cosmosEndpoint`. Again set the `component`, `stack`, `planPath`, and `action` in the same manner as AWS above. ```yaml - name: Store Plan uses: cloudposse/github-action-terraform-plan-storage@v1 id: store-plan with: action: storePlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 planRepositoryType: azureblob blobAccountName: tfplans blobContainerName: plans metadataRepositoryType: cosmos cosmosContainerName: terraform-plan-storage cosmosDatabaseName: terraform-plan-storage cosmosEndpoint: "https://my-cosmo-account.documents.azure.com:443/" - name: Get Plan uses: cloudposse/github-action-terraform-plan-storage@v1 id: get-plan with: action: getPlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 planRepositoryType: azureblob blobAccountName: tfplans blobContainerName: plans metadataRepositoryType: cosmos cosmosContainerName: terraform-plan-storage cosmosDatabaseName: terraform-plan-storage cosmosEndpoint: "https://my-cosmo-account.documents.azure.com:443/" ``` ## Google Cloud This action supports Google Cloud Platform (GCP). In GCP, we store Terraform plan files in Google Cloud Storage and metadata in Firestore. To use the GCP implementation, specify `planRepositoryType` as `gcs` and `metadataRepositoryType` as `firestore`, then provide the following GCP-specific settings: `googleProjectId` to specify the project for both GCS bucket and Firestore, `bucketName` for GCS storage, and `googleFirestoreDatabaseName`/`googleFirestoreCollectionName` for Firestore metadata. The `component`, `stack`, `planPath`, and `action` parameters work the same way as in AWS and Azure examples. ```yaml - name: Store Plan uses: cloudposse/github-action-terraform-plan-storage@v2 id: store-plan with: action: storePlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 planRepositoryType: gcs metadataRepositoryType: firestore bucketName: my-terraform-plans gcpProjectId: my-gcp-project gcpFirestoreDatabaseName: terraform-plan-metadata gcpFirestoreCollectionName: terraform-plan-storage - name: Get Plan uses: cloudposse/github-action-terraform-plan-storage@v2 id: get-plan with: action: getPlan planPath: my-plan.tfplan component: mycomponent stack: core-mycomponent-use1 planRepositoryType: gcs metadataRepositoryType: firestore bucketName: my-terraform-plans gcpProjectId: my-gcp-project gcpFirestoreDatabaseName: terraform-plan-metadata gcpFirestoreCollectionName: terraform-plan-storage ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | action | which action to perform. Valid values are: 'storePlan', 'getPlan', 'taintPlan' | storePlan | true | | blobAccountName | the name of the Azure Blob Storage account to store the plan file | N/A | false | | blobContainerName | the name of the Azure Blob Storage container to store the plan file | N/A | false | | bucketName | the name of the S3 or GCS bucket to store the plan file | terraform-plan-storage | false | | commitSHA | Commit SHA to use for fetching plan | | false | | component | the name of the component corresponding to the plan file | N/A | false | | cosmosConnectionString | the connection string to the CosmosDB account to store the metadata | N/A | false | | cosmosContainerName | the name of the CosmosDB container to store the metadata | N/A | false | | cosmosDatabaseName | the name of the CosmosDB database to store the metadata | N/A | false | | cosmosEndpoint | the endpoint of the CosmosDB account to store the metadata | N/A | false | | failOnMissingPlan | Fail if plan is missing | true | false | | gcpFirestoreCollectionName | the name of the Firestore collection to store the metadata | terraform-plan-storage | false | | gcpFirestoreDatabaseName | the name of the Firestore database to store the metadata | (default) | false | | gcpProjectId | the Google Cloud project ID for GCP services (GCS, Firestore) | N/A | false | | metadataRepositoryType | the type of repository where the plan file is stored. Valid values are: 'dynamo', 'cosmodb', 'firestore' | dynamo | false | | planPath | path to the Terraform plan file. Required for 'storePlan' and 'getPlan' actions | N/A | false | | planRepositoryType | the type of repository where the metadata is stored. Valid values are: 's3', 'azureblob', 'gcs' | s3 | false | | stack | the name of the stack corresponding to the plan file | N/A | false | | tableName | the name of the dynamodb table to store metadata | terraform-plan-storage | false | ## Outputs | Name | Description | |------|-------------| --- ## terratest # GitHub Action: `terratest` A GitHub Action to run Terratest tests and post the results as a build artifact. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Run Terratest uses: cloudposse/github-action-terratest@main with: sourceDir: test/src ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | sourceDir | The directory containing the source code to test | . | true | --- ## validate-codeowners # GitHub Action: `validate-codeowners` This is a Github Action to validate the `CODEOWNERS` file by running a series of checks against the `CODEOWNERS` file to ensure that it's valid and well-linted. Ensuring your repository's `CODEOWNERS` file is valid can be critical to the development process if, for instance, your project uses [branch protection](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches) conditions that rely on definitions in `CODEOWNERS`. ## Usage Copy the `.github/workflows/validate-codeowners.yml` file from this repository into the `.github/workflows` folder of the repository to which you'd like to add Validate `CODEOWNERS` functionality, and ensure that you are using an appropriate token in the workflow file. This will cause the validation functionality to execute whenever any event occurs on any pull request. ## Quick Start Here's how to get started... 1. Copy the `.github/workflows/validate-codeowners.yml` file from this repository into the `.github/workflows` folder of the repository to which you'd like to add Validate CODEOWNERS functionality. 2. Replace `${{ secrets.CODEOWNERS_VALIDATOR_TOKEN_PUBLIC }}` with the name of a token whose permissions are in line with your target repo's requirements, according to the instructions [here](https://github.com/mszostok/codeowners-validator/blob/main/docs/gh-token.md). 3. (Optional) Update the `main` pin inside `validate-codeowners.yml` to a fixed version. Consult https://github.com/cloudposse/github-action-validate-codeowners/releases for a list of available versions. ## Examples Here's a real world example: - [`github-action-validate-codeowners`](https://github.com/cloudposse/github-action-validate-codeowners/.github/workflows/validate-codeowners.yml) - Cloud Posse's self-testing Validate CODEOWNERS GitHub Action ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | token | GitHub token (see: https://github.com/mszostok/codeowners-validator/blob/main/docs/gh-token.md) | N/A | false | --- ## wait-commit-status # GitHub Action: `wait-commit-status` Wait for commit status ## Introduction Checks GitHub API for a given commit and look the commit status. ## Usage ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: context: runs-on: ubuntu-latest steps: - name: Wait commit status uses: cloudposse/github-action-wait-commit-status@main with: repository: ${{ github.repository }} sha: ${{ github.sha }} status: continuous-delivery/example-app lookup: "success" token: ${{ github.token }} check-timeout: 120 check-retry-count: 5 check-retry-interval: 20 ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | check-retry-count | Check retry count | 5 | false | | check-retry-interval | Check retry interval (in seconds) | 10 | false | | expected\_state | Commit status state wait for. Valid values 'success', 'error', 'failure', 'pending' | success | false | | repository | Repository | N/A | true | | sha | Commit SHA | N/A | true | | status | Commit status name | N/A | true | | token | Github authentication token | $\{\{ github.token \}\} | false | --- ## yaml-config-query # GitHub Action: `yaml-config-query` Define YAML document, filter it with JSON query and get result as outputs ## Introduction Utility action allow to declare YAML structured document as an input and get it's part as the action outputs referenced using JQ. This action is useful in simplifing complext GitHub action workflows in different ways. For examples follow [usage](#usage) section. ## Migration `v0` to `v1` There is an issue [The query contains `true` or `false` fails with an error](https://github.com/alexxander/jq-tools/issues/4). A workaround is to use a quote around `"true" and `"false" in a query. To migrate from `v0` to `v1`, quote in your queries all `true`/`false` and Github actions substitutions resovled to the values. ### Example * `query: .true` replace with `query: ."true"` * `query: .${{ inputs.from == '' }}` replace with `query: ."${{ inputs.from == '' }}"` ## Usage ### Define constants ```yaml name: Pull Request on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] jobs: demo: runs-on: ubuntu-latest steps: - name: Context id: context uses: cloudposse/github-action-yaml-config-query@main with: config: | image: acme/example tag: sha-${{ github.sha }} - run: | docker run ${{ steps.context.outputs.image }}:${{ steps.context.outputs.tag }} ``` ### Implement if/else ```yaml name: Promote on: workflow_call: inputs: from: required: false type: string jobs: demo: runs-on: ubuntu-latest steps: - name: Context id: from uses: cloudposse/github-action-yaml-config-query@main with: query: ."${{ inputs.from == '' }}" config: |- true: tag: ${{ github.sha }} false: tag: ${{ inputs.from }} - run: | docker tag acme/example:${{ steps.context.outputs.tag }} ``` ### Implement switch ```yaml name: Build on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened] push: branches: [ main ] release: types: [published] jobs: context: runs-on: ubuntu-latest steps: - name: Context id: controller uses: cloudposse/github-action-yaml-config-query@main with: query: .${{ github.event_name }} config: |- pull_request: build: true promote: false test: true deploy: ["preview"] push: build: true promote: false test: true deploy: ["dev"] release: build: false promote: true test: false deploy: ["staging", "production"] outputs: build: ${{ steps.controlle.outputs.build }} promote: ${{ steps.controlle.outputs.promote }} test: ${{ steps.controlle.outputs.test }} deploy: ${{ steps.controlle.outputs.deploy }} build: needs: [context] if: ${{ needs.context.outputs.build }} uses: ./.github/workflows/reusable-build.yaml test: needs: [context, test] if: ${{ needs.context.outputs.test }} uses: ./.github/workflows/reusable-test.yaml promote: needs: [context] if: ${{ needs.context.outputs.promote }} uses: ./.github/workflows/reusable-promote.yaml deploy: needs: [context] if: ${{ needs.context.outputs.deploy != '[]' }} strategy: matrix: environment: ${{ fromJson(needs.context.outputs.deploy) }} uses: ./.github/workflows/reusable-deploy.yaml with: environment: ${{ matrix.environment }} ``` ## Inputs | Name | Description | Default | Required | |------|-------------|---------|----------| | config | YAML config | N/A | true | | query | JQ Query | . | true | --- ## GitHub Actions(Library) import Intro from '@site/src/components/Intro' import DocCardList from '@theme/DocCardList' In this library you'll find all the GitHub Actions we've implemented to solve common CI/CD challenges. --- ## Reference Architecture Overview import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import DocCardList from '@theme/DocCardList'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; Cloud Posse's reference architecture offers a unique, open-source approach to infrastructure, fostering a collaborative ecosystem with the industry's largest library of Terraform modules and a comprehensive framework using [Atmos](https://atmos.tools). Whether you're an experienced developer or a Terraform newbie, our tools and community are here to help you succeed without sacrificing customization. ```mermaid graph LR A(Build Your Foundation) --> B(Set Up Your Platform) B --> C(Deploy Your Apps) C --> D(Monitor Everything) D --> E(Upgrade &\n Maintain) E --> E ``` Before you jump into the Cloud Posse reference architecture, let’s review what makes it tick. Cloud Posse has helped companies—from scrappy startups to massive enterprises—win big with Terraform. But here’s the key: we do things differently. Everything is based on open source that your team owns and operates. We’re not another enterprise platform with hidden lock-in. If your team depends on our work, we offer [support](/support) to help you move faster and build with confidence. Our goal? To create a collaborative ecosystem where everyone, regardless of the company, can work together on infrastructure, so we stop reinventing the wheel. How do we make this magic happen? First, we built the industry's largest library of Terraform modules for AWS, Datadog, and GitHub Actions. Then, we crafted reusable components to give you rock-solid ways to set up your infrastructure. Finally, we wrapped it all up in a neat framework using [Atmos](https://atmos.tools) that ties everything together, fully automated with GitHub Actions. ## Documentation Structure This documentation site breaks down SweetOps into the following sections to help you get up and running: ### Learn This is section where you go to [**learn our framework**](/learn) for AWS and follow the structured guide to get it set up. Each section of our "Learn" journey is designed to help you get up and running with SweetOps and the Cloud Posse Reference Architecture. - **Build your foundation** (Organization, OUs Accounts, Network & VPCs, IAM & Single Sign-On) - **Set up your platform** (Kubernetes, ECS, EKS, etc.) - **Deploy your apps** (CI/CD, GitHub Actions, GitOps.) - **Monitor everything** (Datadog, Prometheus, Grafana, etc. Also, monitoring for Security & Compliance) - **Upgrade & Maintain** (Day-2 operations including upgrades, backups, disaster recovery, etc.) Inside of each of these sections, you'll find: - [**Design Decisions**](/tags/design-decision): Up-to-date context on the decisions for implementing well-built infrastructure - [**Setup**](/tags/): show how to step up a specific layer or resource in easy-to-follow steps. - [**How-To Guides & Tutorials**](/tags/tutorial): show how to solve specific problems with SweetOps via a series of easy-to-follow steps. Then we include some of our [Best Practices](/best-practices) and [Architectural Design Decisions](/resources/adrs) to help you understand the reasoning behind our choices. ### Reference The [reference documentation](/reference) is where you find the the underlying building blocks such as Terraform modules & components, GitHub Actions, and more. This is where you go when you need to look up the documentation for how a particular component or module works, or when you're trying to build something, this is the first place to look to see if we already support it. It's the nuts and bolts of the SweetOps framework for AWS. ### Community The [**Community**](/community) section is for those who want to engage with the SweetOps community and get support. - Join our Slack community - Attend our weekly office hours - Contribute back to the project ## Who is this documentation for? This documentation is written for DevOps or platform engineering teams that want an opinionated way to build software platforms in the cloud. If the below sounds like you, then SweetOps is what you’re looking for: 1. **You’re on AWS** (the majority of our modules are for AWS) 2. **You’re using Terraform** as your primary IaC tool (and not Cloud Formation) 3. **Your platform needs to be secure** and potentially requires passing compliance audits (PCI, SOC2, HIPAA, HITRUST, FedRAMP, etc.) 4. You don’t want to reinvent the wheel With SweetOps you can implement the following complex architectural patterns with ease: 1. An AWS multi-account Landing Zone built on strong, well-established principles including Separation of Concerns and Principle of Least Privilege (POLP). 2. Multi-region, globally available application environments with disaster recovery capabilities. 3. Foundational AWS-focused security practices that make complex compliance audits a breeze. 4. Microservice architectures that are ready for massive scale running on Docker and Kubernetes. 5. Reusable service catalogs and components to promote reuse across an organization and accelerate adoption ## What are the alternatives? The reference architecture is comparable to various other solutions that bundle ready-to-go Terraform "templates" and offer subscription plans for access to their modules. How does it differentiate from these solutions? 1. **It’s based 100% on Open Source**: SweetOps [is on GitHub](https://github.com/cloudposse) and is free to use with no strings attached under Apache 2.0. 2. **It’s comprehensive**: SweetOps is not only about Terraform. It provides a framework with conventions for building cloud-native platforms that are security-focused, Kubernetes or ECS-based, with comprehensive monitoring and incident management, and driven by continuous delivery. 3. **It’s community-focused**: SweetOps has [over 9500 users in Slack](https://sweetops.com/slack/), and well-attended weekly [office hours](https://cloudposse.com/office-hours/). Now that you know what the reference architecture is about, you're ready to get started by with your first project. Next Step --- ## Choose Your Path import Intro from "@site/src/components/Intro"; import ActionCard from "@site/src/components/ActionCard"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; import PrimaryCTA from "@site/src/components/PrimaryCTA"; import SecondaryCTA from "@site/src/components/SecondaryCTA"; import Steps from "@site/src/components/Steps"; There are two easy ways to get started. You can dive right in with our Quickstart documentation if you're a hands-on learner. Alternatively, if you prefer Cloud Posse to do it for you, try our Jumpstart. Pick what works best for you! Plus, we provide [multiple support options](/support) if you get stuck. All of the documentation on this site corresponds to our [AWS Reference Architecture](https://cloudposse.com/services). We give away all the tools for free—[Terraform modules](https://github.com/cloudposse), [components](https://github.com/cloudposse-terraform-components), [Atmos](https://atmos.tools), and TONS more. But the system that ties it all together? That’s what we sell. Our reference architecture funds the Open Source that powers your business. Our Quickstart provides an end-to-end configuration of our AWS reference architecture [customized to your needs](/quickstart/kickoff/#-review-design-decisions) and implemented by you at your own pace. You can start today, by following along with our [Quickstart documentation](/quickstart) to get a sense of what's involved. To get started, roll up your sleeves and follow the steps below to start building out your infrastructure! - [Buy our "Quickstart"](https://cloudposse.com/pricing) to receive all the configurations. - We’ll send you a form so you can share your [Design Decisions](/quickstart/kickoff/#-review-design-decisions) with us. - Then schedule a [kick off call](/quickstart/kickoff/) to review them with you. - Receive tailored configurations in 2-3 business days after the kick off call. Buy Quickstart Read the Quickstart Docs Every investment in Cloud Posse—whether through QuickStart, JumpStart, or Support—helps us keep building, maintaining, and improving the open source ecosystem for everyone. When you invest in Cloud Posse, you’re not just helping your team—you’re strengthening the ecosystem your business depends on. ### Just need a little help? If you just need a little help getting started, we offer multiple [support options](/support) to help you get unstuck. Our [Jumpstart accelerator](https://cloudposse.com/services) provides an end-to-end implementation by Cloud Posse of this reference architecture in your AWS organization, customized to your needs, with a guaranteed outcome, fixed price, predictable timeline, and a money-back guarantee. If you need assistance, our Jumpstart service provides an end-to-end implementation in your AWS organization. We swiftly implement the reference architecture, tailored to your design decisions, with a guaranteed outcome, fixed price, timeline, and a hassle-free money-back guarantee. Get Price Learn More --- ## Action Items import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import Admonition from '@theme/Admonition'; import Note from '@site/src/components/Note'; Cloud Posse will need a few subscriptions set up from you in order to deploy your infrastructure. Some of these may not apply to all engagements, but please start setting up the relevant subscriptions now. ## Getting Started Before we can get started, here's the minimum information we need from you. ### 1Password Cloud Posse will use 1Password to share secrets your team. You do not need to use 1Password internally, but Cloud Posse will need to use 1Password to transfer secrets. You can either create your own 1Password Vault and add Cloud Posse as members or request that Cloud Posse create a temporary vault (free for you). However, if Cloud Posse creates that vault for you, only three users can be added at a time. **We cannot create AWS accounts until we have access to 1Password.** ### Slack We should already be using Slack for a shared general channel between Cloud Posse and your team. However, we will need an additional channel for AWS notifications and to access AWS account setup emails. We'll also use this channel for AWS budget alerts. - [ ] Create a new Slack channel for AWS notifications, for example `#aws-notifications` - [ ] Invite Cloud Posse - [ ] [Set Up AWS Email Notifications](/layers/accounts/tutorials/how-to-set-up-aws-email-notifications) with your chosen email address for each account. If you are using plus-addressing, you will only need to connect the primary email address. - [ ] [Create a Slack Webhook for that same channel](https://api.slack.com/messaging/webhooks). This is required to enable Budget alerts in Slack. Please share the Webhook URL and the final name of the Slack channel with Cloud Posse. ### Create New AWS Root Account (a.k.a. "Payer Account") We will be launching a new AWS Organization from a single root account. Cloud Posse will be terraforming your entire organization, creating 12-plus accounts, and doing everything from the ground up. We're responsible for configuring SSO, fine-grained IAM roles, and more. We'll need a net-new Organization, so we cannot jeopardize any of your current operations. Please create a new AWS root account and add the root credentials to 1Password. Cloud Posse will take it from there. ### Share GitHub Repository for Infrastructure as Code Please create a new repository in your GitHub organization and grant the Cloud Posse team access. We will need GitHub access to create your Infrastructure as Code repository. ### AWS IAM Identity Center (AWS SSO) In order connect your chosen IdP to AWS IAM Identity Center (AWS SSO), we will to configure your provider and create a metadata file. Please follow the relevant linked guide and follow the steps for the Identity Provider. All steps in AWS will be handled by Cloud Posse. Please also provision a single test user in your IdP for Cloud Posse to use for testing and add those user credentials to 1Password. - [Setup AWS Identity Center (SSO)](/layers/identity/aws-sso/) - GSuite does not automatically sync Users and Groups with AWS Identity Center without additional configuration! If using GSuite as an IdP, considering deploying the [ssosync tool](https://github.com/awslabs/ssosync). - The official AWS documentation for setting up JumpCloud with AWS IAM Identity Center is not accurate. Instead, please refer to the [JumpCloud official documentation](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center) ### AWS SAML (Optional) If deploying AWS SAML as an alternative to AWS SSO, we will need a separate configuration and metadata file. Again, please refer to the relevant linked guide. Please see the following guide and follow the steps to export metadata for your Identity Provider integration. All steps in AWS will be handled by Cloud Posse. - [Setup AWS SAML](/layers/identity/optional/aws-saml/) ## GitHub Self-Hosted Runners ### Self-Hosted Github Runners on EKS If you are deploying the Actions Runner Controller solution for Self-Hosted Github Runners, please generate the required secrets following the [GitHub Action Runner Controller setup docs](/layers/github-actions/eks-github-actions-controller#requirements). Feel free to store these secrets in 1Password if you do not have AWS access yet. Cloud Posse can complete the setup from there. ### Self-Hosted Github Runners with Philips Labs (ECS) If you have chosen ECS as a platform, we recommend deploying Philips Labs GitHub Action Runners. Please read through the [Philips Labs GitHub Action Runners Setup Requirements](/layers/github-actions/philips-labs-github-runners#requirements). In particular, we will need a new GitHub App including a Private Key, an App ID, and an App Installation ID. Please store these secrets in 1Password. ## Atmos Component Updater Requirements Cloud Posse will deploy a GitHub Action that will automatically suggest pull requests in your new repository. To do so, we need to create and install a GitHub App and allow GitHub Actions to create and approve pull requests within your GitHub Organization. For more on the Atmos Component Updater, see [atmos.tools](https://atmos.tools/integrations/github-actions/component-updater). ### Create and install a GitHub App for Atmos 1. Create a new GitHub App 2. Name this new app whatever you prefer. For example, `Atmos Component Updater`. 3. List a Homepage URL of your choosing. This is required by GitHub, but you can use any URL. For example use our documentation page: `https://atmos.tools/integrations/github-actions/component-updater/` 4. (Optional) Add an icon for your new app (example provided below) 5. Assign only the following Repository permissions: ```diff + Contents: Read and write + Pull Requests: Read and write + Metadata: Read-only ``` 6. Generate a new private key [following the GitHub documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps#generating-private-keys). 7. Share both the App ID and the new private key with Cloud Posse in 1Password
Feel free to download and use our Atmos icon with your GitHub App! ![App Icon](/assets/refarch/github-app-icon.png)
### Allow GitHub Actions to create and approve pull requests 1. Go to `https://github.com/organizations/YOUR_ORG/settings/actions` 2. Check "Allow GitHub Actions to create and approve pull requests" ### Create `atmos` GitHub Environment If you grant Cloud Posse `admin` in your new infrastructure repository, we will do this for you. We recommend creating a new GitHub environment for Atmos. With environments, the Atmos Component Updater workflow will be required to follow any branch protection rules before running or accessing the environment's secrets. Plus, GitHub natively organizes these Deployments separately in the GitHub UI. 1. Open "Settings" for your repository 1. Navigate to "Environments" 1. Select "New environment" 1. Name the new environment, "atmos". 1. In the drop-down next to "Deployment branches and tags", select "Protected branches only" 1. In "Environment secrets", create the two required secrets for App ID and App Private Key created above and in 1Password. We will pull these secrets from GitHub Actions with `secrets.ATMOS_APP_ID` and `secrets.ATMOS_PRIVATE_KEY` respectively.
### Requirements for Purchasing Domains If we plan to use the `core-dns` account to register domains, we will need to add a credit card directly to that individual account. When the account is ready, please add a credit card to the `core-dns` account following the [AWS documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/manage-cc.html#Add-cc). ## Additional Integrations Confirm if you plan to deploy any of the following integrations. If so, we will need access to these services. If you haven't already signed up for these services, please soon. ### Datadog Access Sign up for Datadog following the [How to Sign Up for Datadog?](/layers/monitoring/datadog/tutorials/how-to-sign-up-for-datadog) documentation. Cloud Posse will need "admin" access for Datadog as well to complete the Datadog setup. ## Release Engineering If your engagement with Cloud Posse includes Release Engineering, we will also need some more things. ### Sign up GitHub Enterprise (Optional) GitHub Enterprise is required to support native approval gates on deployments to environments. Startups can score a discount for the first 20 users. Reach out to GitHub for details. ### Configure GitHub Settings If we are deploying release engineering as part of the engagement, we will need a few additional items from your team. - [ ] [Enable GitHub Actions for your GitHub Organization](https://docs.github.com/en/organizations/managing-organization-settings/disabling-or-limiting-github-actions-for-your-organization). - [ ] [Allow access via fine-grained personal access tokens for your GitHub Organization](https://docs.github.com/en/organizations/managing-programmatic-access-to-your-organization/setting-a-personal-access-token-policy-for-your-organization#restricting-access-by-fine-grained-personal-access-tokens). - [ ] Create an empty `example-app` private repository in your Organization. We'll deploy an example for release engineering here. ### PATs for ECS with `ecspresso` - Create one fine-grained PAT with the following permission. Please see [Creating a fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token) This PAT needs read access to your `infrastructure` repository: ```diff Repository + Contents: Read-only + Metadata: Read-only ``` - Save the new fine-grained PAT as a GitHub environment secret in the new `example-app` private repository in your Organization. ### PATs for EKS with ArgoCD ArgoCD requires a number of PATs. Please see [How to set up Authorization for ArgoCD with GitHub PATs](/layers/software-delivery/eks-argocd/tutorials/pats) --- ## FAQ import Intro from '@site/src/components/Intro'; These are some of the most frequently asked questions by customers during our Kick Off calls. ### What is the difference between a Service Discovery Domain and a Vanity Domain? This is an extremely common question. Please see [What is the difference between a Vanity and a Service Domain?](/layers/network/faq/#what-is-the-difference-between-a-vanity-and-a-service-domain) ### Do we have to use 1Password? Yes, for Cloud Posse engagements we only uses 1Password to share secrets. You do not need to use 1Password internally, but Cloud Posse will need to use 1Password to transfer secrets to your team. You can either create your own 1Password Vault and add Cloud Posse as members or request that Cloud Posse create a temporary vault (free for you). ### Do we have to create a new Organization? Yes! We need this single root account to start a new AWS Organization. Cloud Posse will be terraforming your entire organization, creating 12-plus accounts, and doing everything from the ground up. We're responsible for configuring SSO, fine-grained IAM roles, and more. We'll need a net-new Organization, so we cannot jeopardize any of your current operations. Once created, we will invite your team to join the new Organization. ### How many email addresses do we need to create? Only one email with `+` addressing is required. This email will be used to create your AWS accounts. For example, `aws+%s@acme.com`. ### What is plus email addressing? Plus email addressing, also known as plus addressing or subaddressing, is a feature offered by some email providers that allows users to create multiple variations of their email address by adding a "+" sign and a unique identifier after their username and before the "@" symbol. For example, if the email address is "john.doe@example.com", a user can create variations such as "john.doe+newsletter@example.com" or "john.doe+work@example.com". Emails sent to these variations will still be delivered to the original email address, but the unique identifier can be used to filter or organize incoming emails. ### How can we track progress? We send status updates on Fridays via Slack! Or feel free to reach out anytime for an update. ### Why are the initial Pull Requests so large? The reason that these PRs are so large is because we are generating content for your entire infrastructure repository. A complete infrastructure set up requires dozens of components, each with Terraform modules, configuration, account setup, and documentation. We've organized these full infrastructure configurations into "layers", which generally reflect the topics of the handoff calls. Specifically, these layers are typically: baseline, accounts, identity, network, spacelift, eks, monitoring, and data, as well as a few miscellaneous additions for smaller addons. In order to deploy any given layer, we must create all content for that given layer. For example, eks adds 200+ files. These are all required to be able to deploy EKS, so we cannot make this PR smaller. However, as the foundation is built out, these PRs will naturally become small, as additional layers have fewer requirements. Regarding your team's internal review, we do not intend for your team to be required to review these massive PRs. Cloud Posse internally reviews these PRs extensively to ensure that the final product works as intended. Once we're confident that we've deployed a given layer entirely, then we schedule the handoff calls. A handoff call is intended to explain a given topic and provide the opportunity for your team to review and provide feedback on any given layer, as well as answer other questions. ### How can we customize our architecture? Customizations are out of scope typically, but we can assess each on a case-by-case basis. You will learn your environment and be confident to make customizations on your own. Often we can deploy an example of the customization, but it's up to you to implement the full deployment ### What if we need more help? Cloud Posse offers multiple levels of support designed to fit your needs and budget. For more information, see [Support](/support). For everything else, we provide fanatical support via our Professional Services, exclusive to reference architecture customers. We can help with anything from architecture reviews, security audits, custom development, migrations, and more. Please [book a call](https://cloudposse.com/meet) to discuss your needs. --- ## Watch All Handoffs import Slider, { Slide } from '@site/src/components/Slider' import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import PrimaryCTA from '@site/src/components/PrimaryCTA' import ReactPlayer from 'react-player' import Intro from '@site/src/components/Intro' We've organized everything into "layers" that represent the different concerns of our infrastructure. Watch these short videos to get an overview of each layer, the problems we faced, and how we solved them. https://github.com/facebook/docusaurus/issues/6201 ## Placeholder {#hidden}
AI generated voice
Learn about the essential tools Cloud Posse uses to manage infrastructure as code. This guide covers the Geodesic Toolbox Container for standardizing development environments, the Atmos framework for implementing conventions and workflows, Terraform for managing cloud infrastructure, and GitHub Actions for CI/CD automation. Get Started
AI generated voice
Review how Cloud Posse designs and manages AWS Account architectures using Atmos and Terraform, aligning with the AWS Well-Architected Framework. Get Started
AI generated voice
Learn how Cloud Posse sets up fine-grained access control for an entire organization using IAM roles, AWS SAML, and AWS IAM Identity Center (SSO). We addresses the challenges we encountered of using various login methods and tools and introduce our solution involving Teams and Team Roles to manage access across multiple AWS accounts. Get Started
AI generated voice
Understand Cloud Posse’s approach to designing robust and scalable Network and DNS architectures on AWS, with a focus on symmetry, account-level isolation, security, and reusability. We cover essential topics such as account isolation, connecting multiple accounts together using Transit Gateways, deploying AWS Client VPN for remote network access by developers, and differentiating between DNS service discovery and branded vanity domains used by customers. Get Started
AI generated voice
Get Started
AI generated voice
Get Started
AI generated voice
Get Started
AI generated voice
Get Started
AI generated voice
Get Started
--- ## Get a Jumpstart with Cloud Posse import Intro from '@site/src/components/Intro'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import PillBox from '@site/src/components/PillBox'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; Done For You Our Jumpstart accelerator provides an end-to-end implementation by Cloud Posse of this reference architecture in your AWS organization, customized to your needs, with a guaranteed outcome, fixed price, predictable timeline, and a money-back guarantee. This documentation will guide you through the end-to-end configuration of our reference architecture for your AWS organization. While Cloud Posse will implement everything for you, we strongly encourage that you follow along by reading this documentation. Also, don't forget we offer [multiple support options](/support) if you get stuck. ## Start your Engagement with Cloud Posse All of the documentation refers to the prebuilt configurations that we'll implement for you based on our discussions. We'll assume from here on out that you've already started an enagement with Cloud Posse. If you haven't, please [request a quote](https://cloudposse.com/meet) to get started. ## Schedule your Kickoff Call with Cloud Posse This is an opportunity to review your design decisions with Cloud Posse, and ask any questions before you get started. Review Agenda ## Tackle all the Action Items First, we'll need to collect some information from you before we can get started. Please review the action items and complete them as soon as possible. Get Started ## Watch the Overview Videos We've organized everything into "layers" that represent the different concerns of our infrastructure. Watch these short videos to get an overview of each layer, the problems we faced, and how we solved them. Get Started --- ## Kick Off with Cloud Posse import Link from "@docusaurus/Link"; import KeyPoints from "@site/src/components/KeyPoints"; import Steps from "@site/src/components/Steps"; import Step from "@site/src/components/Step"; import StepNumber from "@site/src/components/StepNumber"; import Intro from "@site/src/components/Intro"; import ActionCard from "@site/src/components/ActionCard"; import PrimaryCTA from "@site/src/components/PrimaryCTA"; import TaskList from "@site/src/components/TaskList"; import Admonition from "@theme/Admonition"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; The kickoff process for Jumpstart engagements with Cloud Posse ensures a smooth start with clear communication for a successful project delivery. During the call, we will confirm contract requirements and set project expectations. We also cover how Cloud Posse will deploy your infrastructure in layers using our reference architecture and introduce various [support options available](/support), including Slack, customer workshops, and office hours. - **Kickoff Process:** Establish roles, confirm requirements, and set project expectations for a smooth start - **Implementation Phase:** Understand how we go about provisioning infrastructure in layers using the reference architecture, including comprehensive handoffs with documentation - **Support and Communication:** Review the multiple [support options](/support), how we'll use Slack, office hours, and detailed documentation to ensure successful engagement ## Preparing for the Kickoff Meeting This document outlines what to expect from your first call with Cloud Posse. In order to make the most of this meeting, please read through this document and come prepared with questions. In particular, please review the following: 1. Identify stakeholders and establish ownership of the engagement within your Organization. 2. Read through the [Design Decisions](#review-design-decisions) and prepare questions and decisions. 3. Review the list of [Actions Items](#action-items) following this call. ## Kickoff Meeting Agenda ### Introductions Here we will review who is on the call, what their roles are, and identify our technical point of contact at Cloud Posse. We will also review the working timezones of the teams. ### Project Overview Cloud Posse will begin deploying your infrastructure [starting with the foundation](/layers/project) based your team's design decisions. The Reference Architecture is a collection of best practices for building a secure, scalable, and highly available infrastructure on AWS. The Reference Architecture is a living document that is constantly evolving as we learn from our customers and the community. We will deploy your infrastructure in _layers_. These layers are designed to manage collections of deliverables and will be a mix of generated content from a private reference, vendored Terraform from open-source libraries, and any customization for your Organization. Because we are delivering an entire infrastructure repository, these initial PRs will be massive; a complete infrastructure setup requires dozens of components, each with Terraform modules, configuration, account setup, and documentation. You are absolutely welcome to follow along, but we do not intend for your team to be required to review these massive PRs. Cloud Posse internally reviews these PRs extensively to ensure that the final product works as intended. Once we're confident that we've deployed a given layer entirely, we then schedule the [Hand-Off Calls](#handoff-calls). A handoff call is intended to explain a given topic and provide the opportunity for your team to review and provide feedback on any given layer, as well as answer other questions. Before each Hand-Off Call, review the fundamentals of every layer by watching the videos. While we can use the handoff calls, teams can feel overwhelmed if they haven't watched any of the handoff videos beforehand. These calls can be a lecture on the material for any given layer, a demo from Cloud Posse, or an opportunity to practice with hands-on labs. If you come prepared for Hand-Off calls, we can skip the lecture and spend more time answering questions or working through hands-on labs. ### Shared Customer Workshop > **When:** Thursdays, 7:00-7:30A PT/ 9:00-9:30A CT/ 10:00-10:30A ET > **Where:** Zoom > **Who:** [Essential Support Customers Only](/support/essential) > **When:** Wednesdays, 2:30-3:00P PT/ 4:30-5:00P CT/ 5:30-6:00P ET > **Where:** Zoom > **Who:** [Essential Support Customers Only](/support/essential) This is a great opportunity to get your questions answered and to get help with your project. ### Community Office Hours (FREE) > **When:** Wednesdays, 11:30a-12:30p PT/ 1:30p-2:30p CT/ 2:30p-3:30p ET > **Where:** Zoom > **Who:** Anyone This is a good way to keep up with the latest developments and trends in the DevOps community. Office Hours are less focused on technical questions that [Customer Workshops](/support/essential), but you can ask anything you like. Sign up at [cloudposse.com/office-hours](https://cloudposse.com/office-hours/) ### SweetOps Slack If you are looking for a community of like-minded DevOps practitioners, please join the [SweetOps Slack](https://slack.sweetops.com/). ### Review Design Decisions - [ ] [Decide on Terraform Version](/layers/project/design-decisions/decide-on-terraform-version) - [ ] [Decide on Namespace Abbreviation](/layers/project/design-decisions/decide-on-namespace-abbreviation) - [ ] [Decide on Infrastructure Repository Name](/layers/project/design-decisions/decide-on-infrastructure-repository-name) - [ ] [Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts) - [ ] [Decide on IdP](/layers/identity/design-decisions/decide-on-idp) - [ ] [Decide on IdP Integration Method](/layers/identity/design-decisions/decide-on-idp-integration) - [ ] [Decide on Primary AWS Region and Secondary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) - [ ] [Decide on CIDR Allocation Strategy](/layers/network/design-decisions/decide-on-cidr-allocation) - [ ] [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain) - [ ] [Decide on Vanity Domain](/layers/network/design-decisions/decide-on-vanity-branded-domain) - [ ] [Decide on Release Engineering Strategy](/layers/software-delivery/design-decisions/decide-on-release-engineering-strategy) These are the design decisions you can customize as part of the Jumpstart package. [All other decisions are pre-made](/tags/design-decision/) for you, but you're welcome to review them. If you'd like to make additional changes, [let us know—we're happy to provide a quote](https://cloudposse.com/meet). ## Review Handoff Calls Generally, expect to schedule the following Handoff calls. These are subject to change and should be adaptable to fit your individual engagement. - [Kick Off](/jumpstart/kickoff) - [Introduction to Toolset](/layers/project) - [Identity and Authentication](/layers/identity) - [Component Development](/learn/component-development) - [Account Management](/layers/accounts/) - [Network and DNS](/layers/network/) - [Automate Terraform](/layers/atmos-pro) - [ECS](/layers/ecs) or [EKS](/layers/eks) - [Monitoring](/layers/monitoring) - [Release Engineering](/layers/software-delivery) - Final Call (Sign-off) ## How to Succeed Cloud Posse has noticed several patterns that lead to successful projects. ### Come Prepared Review six pagers and documentation before Hand-Off calls. This will help you to know what questions need to be asked. Coming unprepared will lead to a lot of questions and back-and-forth. This will slow down material resulting in less time for new material. ### Take Initiative The most successful customers take initiative to make customizations to their Reference Architecture. This is a great way to make the Reference Architecture your own. It also helps to build a deeper understanding of the Reference Architecture and how it works. ### Cameras On We recommend that all participants have their cameras on. This helps to build trust and rapport. It also helps to keep everyone engaged and focused. This also lets us gauge how everyone is understanding the material. If you are having trouble understanding something, please ask questions. ### Ask Questions We encourage you to ask questions. We want to make sure that everyone understands the material. We also want to make sure that we are providing the right level of detail. Our meetings are intended to be interactive and encourage conversation. Please feel free to interject at any time if you have a question or a comment to add to the discussion. ## Get Support The Jumpstart accelerator does not include support. To get help, we offer multiple à la carte [support options](/support) to fit your needs and budget. ### Slack If you need help scheduling meetings, please post in your team's Cloud Posse channel (e.g., `#acme-general`). This keeps discussions open and prevents duplicated or siloed information in direct messages (DMs). In general, please avoid DMs, as they make it harder to escalate requests or follow up. Feel free to @ a team member if you need assistance—we're here to help! You can also reach out to our community with our [SweetOps Slack community](#sweetops-slack). ### Community Office Hours [Community Office Hours](https://cloudposse.com/office-hours) are great opportunities to ask general questions and get help. If you need more technical help, please consider one of our excellent [support options](/support). ### Documentation You can always find how-to guides, design decisions, and other helpful pages at [docs.cloudposse.com](/) --- ## Onboarding import TaskList from '@site/src/components/TaskList'; After ensuring you’ve satisfied all the prerequisites, we recommend doing the following - [ ] Join the Shared “Slack connect” channels with our team. These are usually named something like `#cloudposse-` on the customer’s side. - [ ] Ensure you’ve been added to any calls with the Cloud Posse team. Depending on the type of engagement, you may have weekly cadance calls (Enterprise only). Cloud Posse PMs can help add members to the calendar event. - [ ] Ensure you’ve been added to the Cloud Posse Linear Project for your company’s engagement with our team. Cloud Posse PMs can help add members. **Only applicable for Enterprise engagements**. - [ ] Learn more about Cloud Posse because as a [DevOps Accelerator](https://cloudposse.com) we are very different from typical professional services companies. - [ ] Review the different ways you can get [Support](/support) to fit your needs and budget. --- ## How to Provision Shared Slack Channels import Note from '@site/src/components/Note' import Intro from '@site/src/components/Intro' import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' ## Problem Collaborating effectively between teams requires rapid back-and-forth communication. Email threads grow unwieldy and adding members to threads so they get all the context is difficult. Email as a medium is in general more insecure (e.g. more easily susceptible to phishing attacks). Email forums are for long-form communication, not the sort we’re accustomed to when needing to get quick answers. ## Solution Slack supports sharing slack channels between slack teams (organizations) using what they call _Slack Connect_. Each organization gets to manage its side of the slack channel, including what to call the slack channel and enforce policies. ## Slack Connect Documentation The official Slack Connect documentation: [https://slack.com/help/articles/115004151203-A-guide-to-Slack-Connect](https://slack.com/help/articles/115004151203-A-guide-to-Slack-Connect) ## Step by Step Instructions 1. Create a new Slack Channel as you would normally 2. Give it a name, usually of the partner, customer or vendor, followed by `-general` since there may be future channels shared down the road. After clicking the “share outside ...” checkbox, the title of the dialg changes to “Slack Connect” which what Slack calls shared channels. all members can rename the channel on their end without affecting the local name of the channel. This means you should name the channel following your organization’s conventions. 3. Proceed by clicking Next. Then you’re prompted to share the channel. There are (2) ways to do this: either by clicking the “copy share link” option or by entering in the email address of _anyone_ from the other organization. The other organization’s admins will then be prompted to accept the invitation. Then you’ll see a notice like this from the Slackbot. 4. After the invitation is accepted by the other organization, the inviting organization may need to re-confirm the connect request. This may appear as a notice from the `Slackbot` to the slack admins. 5. Once the connection is established, any members that exist in the channel can chat publicly or via direct message (DMs). ## Managing Slack Connections You can accept/decline slack connections from the "Slack Connect" sidebar menu. --- ## Offboarding Cloud Posse ### Problem - Your company is ready to take over all operations and needs to restrict Cloud Posse’s access to mission-critical environments for regulatory compliance (e.g. for HIPAA compliance). - Your engagement with Cloud Posse is coming to an end and you need to shut down entirely Cloud Posse’s access to all environments or you are pausing the engagement - Cloud Posse has access to multiple systems and you may want to restrict access accordingly. ## Solution ### AWS #### **Option 1:** Restrict Federated IAM Access to Some Accounts #### **Option 2:** Restrict Federated IAM Access to a Single Account #### **Option 3:** Disable Federated IAM Access to All Accounts :::info After disabling all Federated IAM access, you have the option to issue Cloud Posse team members SSO access via your own IdP. ::: #### **Option 4**: Issue IAM User Accounts :::caution We strongly discourage this approach as it’s generally an anti-pattern to bypass SSO and introduces new requirements for offboarding team members. ::: ### Customer managed IdP For things like Okta, Workspaces, or Azure AD: - Remove Cloud Posse team members from your IdP. - Remove any test accounts that were used for evaluating teams/groups. ### GitHub Typically customers provision a “Cloud Posse” team within their GitHub org. #### Offboarding Github Access - Option 1: Revoke All Revoking this team’s access from repositories should be sufficient to remove all of our access. Also, ensure that any repositories do not have Cloud Posse usernames directly added as external contributors. This happens if repositories were created by our team in your organization. - Option 2: Downgrade Access Changing our team’s access to read-only will enable us to still participate in Code Reviews. #### Offboarding GitHub Ownership The `CODEOWNERS` file should be checked to make sure that no Cloud Posse usernames or groups are listed. This file is typically located in the root of the repository. ### Spacelift Depending on how Spacelift was configured, make sure the `LOGIN` policy does not include any Cloud Posse users. Go to `https://.app.spacelift.io/policies` Then remove our team’s access or any hardcoded usernames. Also, make sure to sign out any logged in sessions, by going to `https://.app.spacelift.io/settings/sessions` ### Slack :::tip Leave Channels Open We recommend keeping open channels of communication between our teams. That way we are able to help you out in a pinch. ::: All customer channels are managed via Slack Connect. Some channels may be owned by your team, others by our team. If you desire to close the connection, ask your Slack administrator to remove our organization from the slack connection. See: - [Removing Orgs from Slack Connect](https://slack.com/help/articles/360026489273-Remove-organizations-from-a-Slack-Connect-channel-) - [Removing external members from channels](https://slack.com/help/articles/5682545991443-Slack-Connect--Manage-external-people-#remove-external-people) ### Datadog Offboard any `@cloudposse.com` email addresses. ### Customer Jira & Confluence Some customers have added our team directly to their Atlassian products. Make sure to offboard any `@cloudposse.com` email addresses. ### 1password 1Password vaults may be shared between our teams. Sometimes customers add Cloud Posse to their vaults, other times customers were added to vaults controlled by Cloud Posse. At the end of an engagement, we recommend to stop sharing vaults. #### Customer Managed Vaults If your company controls the vault, simply remove Cloud Posse's team access from the vault. We recommend rotating all credentials both as an exercise and as an extra precaution. #### Cloud Posse Managed Vaults When vaults are controlled by Cloud Posse, we require the customer to take over ownership by creating their own vault, and manually copying over the secrets. - Create a new vault for your team - Recreate all the credentials in the new vault. **We recommend rotating credentials.** - Share the new vault with your team - Request Cloud Posse destroy it's vault --- ## Tutorials import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; These tutorials apply specifically to the Jumpstart engagements and provide guides typically related to onboarding and setting up systems that we will need to conduct this engagement. --- ## Deploy CloudTrail and Account Budgets import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Now that all the accounts have been deployed, we need to finalize the setup of the accounts. This includes deploying CloudTrail and optionally account budgets. These foundational components will be necessary to move forward with the rest of the deployment. | Steps | Actions | | -------------------------- | ----------------------------------- | | Deploy baseline components | `atmos workflow deploy -f quickstart/foundation/baseline` | | Deploy account budgets | Create Slack Webhook and `atmos workflow deploy -f quickstart/foundation/accounts` | ## Deploy CloudTrail Deploy CloudTrail and the the CloudTrail bucket: ## (Optional) Deploy Account Budgets Budgets are an optional feature that can be enabled with [the `account-settings` component](/components/library/aws/account-settings/) for the Organization as a whole or for individual accounts. Budgets *do not restrict spending* but provide visibility into spending and can be used to set alerts when spending exceeds a certain threshold. We recommend using a dedicated Slack channel for these alerts, which we will set up with a webhook. 1. [Create a Slack Webhook](https://api.slack.com/messaging/webhooks). Take note of the Webhook URL and the final name of the Slack channel. The Slack channel is case-sensitive and needs to match the name of the channel exactly as the name appears in owning Slack server (not the name if changed as a shared channel). 2. Update the `account-settings` component with the Slack Webhook URL and the Slack channel name. ```yaml # stacks/catalog/account-settings.yaml components: terraform: account-settings: vars: budgets_enabled: true budgets_notifications_enabled: true budgets_slack_webhook_url: https://url.slack.com/abcd/1234 budgets_slack_username: AWS Budgets budgets_slack_channel: aws-budgets-notifications ``` 3. **Enable Cost Explorer** (if not already enabled). When attempting to create budgets, you may encounter the following error: ```console Error: creating Budget (xxx-core-gbl-artifacts-1000-total-monthly): operation error Budgets: CreateBudget, https response error StatusCode: 400, RequestID: xxx, AccessDeniedException: Account xxx is a linked account. To enable budgets for your account, ask the payer account to enable budgets first. ``` This error occurs when AWS Cost Explorer has not been activated for the organization. To resolve this, navigate to the AWS Cost Explorer console in your organization's management account and verify that Cost Explorer is enabled. Cost Explorer must be enabled at the organization level before individual member accounts can create budgets. 4. **To enable budgets for the entire organization**, update `account-settings` in the same account as the Organization root account, typically `core-root`. This budget will include the total spending of all accounts in the Organization. ```yaml # stacks/orgs/acme/core/root/global-region/baseline.yaml import: - catalog/account-settings components: terraform: account-settings: vars: # Budgets in `root` apply to the Organization as a whole budgets: - name: Total AWS Organization Cost per Month budget_type: COST limit_amount: 10000 limit_unit: USD time_unit: MONTHLY notification: - comparison_operator: GREATER_THAN notification_type: FORECASTED threshold_type: PERCENTAGE threshold: 80 subscribers: - slack - comparison_operator: GREATER_THAN notification_type: FORECASTED threshold_type: PERCENTAGE threshold: 100 subscribers: - slack - comparison_operator: GREATER_THAN notification_type: ACTUAL threshold_type: PERCENTAGE threshold: 100 subscribers: - slack ``` 5. **To enable budgets for individual accounts**, update `account-settings` in the account you want to enable budgets for or as the default setting for all `account-settings` components to apply to every account. This budget will include the spending of the given account only. ```yaml # stacks/catalog/account-settings.yaml components: terraform: account-settings: vars: ... budgets: - name: 1000-total-monthly budget_type: COST limit_amount: "1000" limit_unit: USD time_unit: MONTHLY - name: s3-3GB-limit-monthly budget_type: USAGE limit_amount: "3" limit_unit: GB time_unit: MONTHLY ``` 6. Finally, reapply `account-settings` in any changed account to apply the new settings --- ## Account Management import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from "react-player"; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import TaskList from '@site/src/components/TaskList'; import CategoryList from '@site/src/components/CategoryList'; This chapter presents how Cloud Posse designs and manages AWS Account architectures. We will explain how Cloud Posse provisions and manages AWS Accounts using Atmos and Terraform, the reasoning behind our decisions, and how this architecture will better align your organization with the [AWS Well-Architected Framework](https://docs.aws.amazon.com/pdfs/wellarchitected/latest/userguide/wellarchitected-ug.pdf). - Why to leverage multiple AWS accounts within an AWS Organization - How we organize accounts into organizational units (OUs) to manage access and apply Service Control Policies (SCPs) to provide guard rails - The set of components we use to provision, configure, and manage AWS accounts, including account-level settings, service control policies, and Terraform state backends, using native Terraform with Atmos
AI generated voice
## The Problem The [AWS Well-Architected Framework](https://docs.aws.amazon.com/pdfs/wellarchitected/latest/userguide/wellarchitected-ug.pdf) defines AWS architectural best practices and presents a set of foundational questions to enable you to understand how a specific architecture aligns with cloud best practices. The AWS Well-Architected Framework provides several foundational recommendations, one of which is to distribute workloads across multiple AWS accounts. However, the framework does not prescribe how this should be achieved. AWS offers resources such as Control Tower or Account Factory for provisioning accounts, but these resources have some limitations. The primary issue is that they cannot be managed sufficiently with Terraform, which means manual effort is required to use them. ## Our Solution Cloud Posse has developed a set of components to provision, configure, and manage AWS Accounts and Organizations. ### Using an Organization Leveraging multiple AWS accounts within an AWS Organization is the only way to satisfy these requirements. Guard rails can be created to restrict what can happen in an account and by whom. We then further organize the flat account structure into organizational units. Organizational units (OUs) can then leverage things like Service Control Policies to restrict what can happen inside the accounts.
`core` (OU)
Responsible for management accounts, such as the organizational root account or a network hub. These accounts are singletons and will never need to be duplicated.
`plat` (OU)
Responsible for platform accounts, such as sandbox, dev, staging, and prod. These accounts are dynamic and can be specific to the needs of your Organizations.
### Account Boundaries Constructs like VPCs only provide network-level isolation, but not IAM-level isolation. And within a single AWS account, there’s no practical way to manage IAM-level boundaries between multiple stages like dev/staging/prod. For example, to provision most Terraform modules, “administrative” level access is required because provisioning any IAM roles requires admin privileges. That would mean that a developer needs to be an “admin” in order to iterate on a module. Multiple AWS accounts should be used to provide a higher degree of isolation by segmenting/isolating workloads. There is no additional cost for operating multiple AWS accounts. It does add additional overhead to manage as a standard set of components will to manage the account. AWS Support only applies to one account, so it may need to be purchased for each account unless the organization upgrades to Enterprise Support. Multiple AWS accounts are all managed underneath an AWS Organization and organized into multiple organizational units (OUs). Service Control Policies can restrict what runs in an account and place boundaries around an account that even account-level administrators cannot bypass. ### Account Architecture By convention, we prefix each account name with its organizational unit (OU) to distinguish it from other accounts of the same type. For example, if we have an OU called `plat` (short for platform) and an account called "production" (or `prod` for short), we would name the account `plat-prod`. In practice, there might be multiple production accounts, such as in a `data` OU, a `network` OU, and a `plat` OU. By prefixing each account with its OU, it is sufficiently disambiguated and follows a consistent convention.
core-root
The "root" (parent, billing) account creates all child accounts. The root account has special capabilities not found in any other account An administrator in the root account by default has the OrganizationAccountAccessRole to all other accounts (admin access) Organizational CloudTrails can only be provisioned in this account. It’s the only account that can have member accounts associated with it Service Control Policies can only be set in this account It’s the only account that can manage the AWS Organization
core-audit
The "audit" account is where all logs end up
core-security
The "security" account is where to run automated security scanning software that might operate in a read-only fashion against the audit account.
core-identity
The "identity" account is where to add users and delegate access to the other accounts and is where users log in
core-network
The “network” account is where the transit gateway is managed and all inter-account routing
core-dns
The “dns” account is the owner for all zones (may have a legal role with Route53Registrar.*{" "} permissions). Cannot touch zones or anything else. Includes billing.
core-auto
The “automation” account is where any gitops automation will live. Some automation (like Spacelift) has “god” mode in this account. The auto account will typically have transit gateway access to all other accounts, therefore we want to limit what is deployed in the automation account to only those services which need it.
core-artifacts
This “artifacts” account is where we recommend centralizing and storing artifacts (e.g. ECR, assets, etc) for CI/CD
plat-prod
The "production" is the account where you run your most mission-critical applications
plat-staging
The “staging” account is where QA and integration tests will run for public consumption. This is production for QA engineers and partners doing integration tests. It must be stable for third-parties to test. It runs a kubernetes cluster.
plat-dev
The "dev" account is where to run automated tests, load tests infrastructure code. This is where the entire engineering organization operates daily. It needs to be stable for developers. This environment is Production for developers to develop code.
plat-sandbox
The "sandbox" account is where you let your developers have fun and break things. Developers get admin. This is where changes happen first. It will be used by developers who need the bleeding edge. Only DevOps work here or developers trying to get net-new applications added to tools like slice.
### Terraform State We need someplace to store the terraform state. Multiple options exist (e.g. Vault, Terraform Enterprise, GitLab, Spacelift), but the only one we’ll focus on right now is using S3. The terraform state may contain secrets, which is unavoidable for certain kinds of resources (e.g. master credentials for RDS clusters). For this reason, it is advisable for companies with security and compliance requirements to segment their state backends to make it easier to control with IAM who has access to what. While on the other hand adding multiple state backends is good from a security perspective, on the other it unnecessarily complicates the architecture for companies that do not need the added layer of security. We will use a single S3 bucket, as it is the least complicated to maintain. Anyone who should be able to run terraform locally will need read/write access to this state bucket. ### Components Cloud Posse manages this process with the following components.
account
This component is responsible for provisioning the full account hierarchy along with Organizational Units (OUs). It includes the ability to associate Service Control Policies (SCPs) to the Organization, each Organizational Unit and account.
account-settings
This component is responsible for provisioning account level settings: IAM password policy, AWS Account Alias, EBS encryption, and Service Quotas. We can also leverage this component to enable account or organization level budgets.
account-map
Transforms account metadata to a safe place for all designated roles to able to access. IAM roles should not able to read `account`. Once `account-map` is provisioned, other components can utilized `remote-state` to pull account metadata such as Account ID mapping or IAM Roles to assume for a given account.
account-quotas
This component is responsible for requesting service quota increases. We recommend making requests here rather than in `account-settings` because `account-settings` is a restricted component that can only be applied by SuperAdmin.
tfstate-backend
Provisions the Terraform state backends. This component already follows all standard best practices around private ACLs, encryption, versioning, locking, etc.
cloudtrail
This component is responsible for provisioning cloudtrail auditing in an individual account. It's expected to be used alongside the cloudtrail-bucket component as it utilizes that bucket via remote state.
cloudtrail-bucket
This component is responsible for provisioning a bucket for storing cloudtrail logs for auditing purposes
## Design Decisions [Review Design Decisions](/layers/project/design-decisions) and record your decisions now. You will need the results of these decisions going forward. Next, we'll prepare the organization to provision the Terraform State backend, followed by account provisioning. If you're curious about the though that went into this process, please review the design decisions documentation. Next Step Review Design Decisions ## References - [Decide on AWS Organization Strategy](/layers/accounts/design-decisions/decide-on-aws-organization-strategy/) - [Decide on AWS Account Flavors and Organizational Units](/layers/accounts/design-decisions/decide-on-aws-account-flavors-and-organizational-units/) - [Decide on AWS Support](/layers/accounts/design-decisions/decide-on-aws-support/) - [Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts/) - [Structure of Terraform S3 State Backend Bucket](/layers/accounts/tutorials/terraform-s3-state/) --- ## Deploying AWS Accounts import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Note from '@site/src/components/Note'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import TaskList from '@site/src/components/TaskList'; This step-by-step process outlines how to deploy AWS accounts using `atmos` workflows and ClickOps steps. It covers necessary preparations, such as configuring the AWS Organization, increasing account quotas, and verifying account configurations. The guide details the deployment processes and post-deployment configurations, including setting up account settings, enabling AWS RAM for Organizations, and performing certain manual configurations via ClickOps. | Steps | Actions | | ------------------------- | ---------------------------------------------------- | | Deploy AWS Organization | `atmos workflow deploy/organization -f quickstart/foundation/accounts` | | Prepare accounts creation | Click Ops | | Deploy accounts | `atmos workflow deploy/accounts -f quickstart/foundation/accounts` | | Deploy accounts settings | `atmos workflow deploy/account-settings -f quickstart/foundation/accounts` | | Finalize account setup | Click Ops | ## Prepare Account Deployment Review the "account" configuration in the stack catalog. **This is the hardest part to change/fix once the accounts are provisioned**. If you aren't confident about the email configuration, account names, or anything else, now is the time to make changes or ask for help. You should double-check the following: - Check that `stacks/catalog/account.yaml` has the values you expect, especially account email format - Run `atmos describe component account -s core-gbl-root` to inspect the final component configuration (e.g. _after_ all the mixins have been imported) - Plan the run with `atmos terraform plan account -s core-gbl-root` ## Deploy the AWS Organization ## Confirm the Root Account is configured as an Organization The previous step will create the AWS Organization and configure the `core-root` account as the "root" account. Take the time now to verify that the root account is configured as an AWS Organization and that [AWS RAM for Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_enable-ram.html) is enabled, which is required for connecting the Organization. **Check Organization Status:** ```bash # Check if AWS Organization exists and get its details aws organizations describe-organization # Or specifically check if RAM for Organizations is enabled aws organizations describe-organization --query 'Organization.FeatureSet' ``` The `FeatureSet` should return `ALL` if RAM for Organizations is enabled, or `CONSOLIDATED_BILLING` if it's not enabled. ## Confirm Root Account Name After deploying the AWS Organization, verify the **account name** of the root (a.k.a. management/payer) account and ensure it's set in the `account-map` catalog. This is the human-readable **account name**, not the account alias. While we default to `root`, it may have been set to your company name or another custom label when the account was originally created.
Why this is needed? The root account has two different names: - **AWS Account Name**: The name you typed when originally creating the AWS account (often "root" but can be different) - **Account Alias**: The deterministic name set in the account configuration (e.g., "core-root")
The account-map component needs to know the actual AWS account name to properly map accounts. **To find your root account name:** ```bash # Make sure you're using a user with access to your root account, such as SuperAdmin aws organizations list-accounts \ --query "Accounts[?Id=='$(aws organizations describe-organization --query 'Organization.MasterAccountId' --output text)'].Name" \ --output text ``` **To configure the account-map:** Update `stacks/catalog/account-map.yaml` to set the correct root account name: ```yaml components: terraform: account-map: vars: root_account_aws_name: "your-actual-root-account-name" # The name from the AWS Organizations output root_account_account_name: "core-root" # This should always be "core-root" ``` If you encounter an error such as the following in subsequent deployments, it's usually because the `root_account_aws_name` is not correctly set in the `account-map` configuration. ```console The given key does not identify an element in this collection value ```
## Raise Account Limits If you haven't already completed the Account Quota increase, now is the time to do so. To deploy all accounts, we need to request an increase of the Account Quota from AWS support, which requires an AWS Organization to be created first. From the `root` account (not `SuperAdmin`), increase the [account quota to 20+](https://us-east-1.console.aws.amazon.com/servicequotas/home/services/organizations/quotas) for the Cloud Posse reference architecture, or more depending on your business use-case. **Alternative: Use AWS CLI** You can also use the AWS CLI to request the quota increase: ```bash aws service-quotas request-service-quota-increase \ --service-code organizations \ --quota-code L-E619E033 \ --desired-value 20 ``` Where `L-E619E033` is the quota code for "Default maximum number of accounts" under "AWS Organizations" in "us-east-1". This quota increase can also be requested through our [account-quotas](/components/library/aws/account-quotas/) component, but it's generally faster to handle this manually or through the API since it's a one time request. ## Deploy Accounts Again review the "account" configuration in `stacks/catalog/account.yaml`. In particular, check the email address and account names. In the next step, we will create and configure all accounts in the AWS Organization using the configuration in that stack file. Once confident, begin the accounts deployment: With the addition of support for dynamic Terraform roles, our `baseline` cold start refarch layer now depends on/requires that we have `aws-teams` and `aws-team-roles` stacks configured. This is because `account-map` uses those stacks to determine which IAM role to assume when performing Terraform in the account, and almost every other component uses `account-map` (indirectly) to chose the role to assume. However, these components do _not_ need to be deployed yet. These deployments will create all AWS member accounts and store relevant account metadata as "mappings" in the Terraform outputs of the `account-map` component. Rather than querying this `account` component each time we need an Account ID or role, we provision a static component `account-map`. Always run `atmos terraform apply account-map -s core-gbl-root` after provisioning accounts. ## Deploy Accounts Settings Once you've created the accounts, you'll need to provision the baseline configuration within the accounts themselves. Run the following: The workflows will kick off several sequential Terraform runs to provision all the AWS member account settings for member accounts in the Organization. ## ClickOps to Complete Account Setup For each new account, you will need to perform some manual configurations via ClickOps. These configurations include setting up the root account credentials, enabling MFA, and unsubscribing the account's email address from all marketing emails. 1. Reset the root user password: 1. Perform a password reset by attempting to log in to the AWS console as a "root user" using that account's email address 1. Click the "Forgot password?" link 1. You will receive a password reset link via email, which should be forwarded to the shared Slack channel for automated messages. Click the link 1. Enter a new password Use 1Password to create a password 26-38 characters long, including at least 3 of each class of character: lower case, uppercase, digit, and symbol 1. Save the email address and generated password as web login credentials in 1Password 1. Finally, record the account number in a separate field of the 1Password item, and save it. This is optional but recommended. 1. Configure MFA: 1. Log in to the AWS console using the new password 1. Choose "My Security Credentials" from the account dropdown menu 1. Set up Multi-Factor Authentication (MFA) to use a Virtual MFA device 1. Save the MFA TOTP key in 1Password with 1Password's "One-Time Password" field 1. Enter the generated MFA codes in AWS to verify the MFA device 1. Save the Virtual MFA ARN in the same 1Password entry. We will need this to set up the MFA device for `SuperAdmin` in Leapp 1. Enable any necessary optional regions 1. Optional, but highly recommended - unsubscribe the account's email address from all marketing emails 1. Go to [AWS Marketing Preferences](https://pages.awscloud.com/communication-preferences.html) 1. Click "Unsubscribe from Email" 1. Enter the account's email address 1. Check "Unsubscribe from all AWS marketing emails" For more details, review [the detailed "AWS Cold Start" documentation](/layers/accounts/tutorials/manual-configuration/#configure-root-account-credentials-for-each-account).
--- ## Decide on AWS Account Flavors and Organizational Units import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When setting up your AWS infrastructure, you need to decide how to organize your workloads across multiple AWS accounts to ensure optimal isolation and management. This involves deciding the appropriate account structure and organizational units (OUs) that align with your operational needs and security requirements. ## Context and Problem Statement The [AWS Well-Architected Framework](https://docs.aws.amazon.com/wellarchitected/latest/userguide/wellarchitected-ug.pdf) recommends splitting workloads across multiple AWS accounts. When moving to an infrastructure-as-code (IaC) model of infrastructure provisioning, many of the same best practices that apply to regular software development should apply to IaC. Part of that is not making changes to a production environment that hasn't been tested in a staging environment. If the production and staging environments are in the same account, then there are insufficient assurances/guarantees/protections in place to prevent breaking production. Constructs like VPCs only provide network-level isolation, but not IAM-level isolation. And within a single AWS account, there’s no practical way to manage IAM-level boundaries between multiple stages like dev/staging/prod. For example, to provision most terraform modules, “administrative” level access is required because provisioning any IAM roles requires admin privileges. That would mean that a developer needs to be an “admin” in order to iterate on a module. Leveraging multiple AWS accounts within an AWS Organization is the _only way_ to satisfy these requirements. Guardrails can be be in place to restrict what can happen in an account and by whom. We must decide how to organize the flat account structure into organizational units. Organizational units can then leverage things like Service Control Policies to restrict what can happen inside the accounts. Multiple AWS accounts should be used to provide a higher degree of isolation by segmenting/isolating workloads. There is no additional cost for operating multiple AWS accounts. It does add additional overhead to manage as a standard set of components will to manage the account. AWS support only applies to one account, so it may need to be purchased for each account unless the organization upgrades to Enterprise Support. Multiple AWS accounts are all managed underneath an AWS Organization and organized into multiple organizational units (OUs). Service Control Policies can restrict what runs in an account and place boundaries around an account that even account-level administrators cannot bypass. ## Considered Options ### AWS Well-Architected Account Designations Here are some common account designations. Not all are required. :::tip This is our recommended approach. ::: :::note It is advised to keep the names of accounts as short as possible because of resources with low max character limits [AWS Resources Limitations](/resources/legacy/aws-feature-requests-and-limitations) :::
core-root
The "root" (parent, billing) account creates all child accounts. The root account has special capabilities not found in any other account An administrator in the root account by default has the{" "} OrganizationAccountAccessRole to all other accounts (admin access) Organizational CloudTrails can only be provisioned in this account It’s the only account that can have member accounts associated with it Service Control Policies can only be set in this account It’s the only account that can manage the AWS Organization
plat-prod
The "production" is the account where you run your most mission-critical applications
plat-staging
The “staging” account is where QA and integration tests will run for public consumption. This is production for QA engineers and partners doing integration tests. It must be stable for third-parties to test. It runs a kubernetes cluster.
plat-sandbox
The "sandbox" account is where you let your developers have fun and break things. Developers get admin. This is where changes happen first. It will be used by developers who need the bleeding edge. Only DevOps work here or developers trying to get net-new applications added to tools like slice.
plat-dev
The "dev" account is where to run automated tests, load tests infrastructure code. This is where the entire engineering organization operates daily. It needs to be stable for developers. This environment is Production for developers to develop code.
plat-uat, qa, etc
Additional or alternative platform accounts
core-audit
The "audit" account is where all logs end up
core-corp
The "corp" account is where you run the shared platform services for the company. Google calls it “corp”
core-security
The "security" account is where to run automated security scanning software that might operate in a read-only fashion against the audit account.
core-identity
The "identity" account is where to add users and delegate access to the other accounts and is where users log in
core-network
The “network” account is where the transit gateway is managed and all inter-account routing
core-dns
The “dns” account is the owner for all zones (may have a legal role with{" "} Route53Registrar.* permissions). Cannot touch zones or anything else. Includes billing. Example use-case: Legal team needs to manage DNS and it’s easier to give them access to an account specific to DNS rather than multiple set of resources.
core-automation
The “automation” account is where any gitops automation will live. Some automation (like Spacelift) has “god” mode in this account. The network account will typically have transit gateway access to all other accounts, therefore we want to limit what is deployed in the automation account to only those services which need it.
core-artifacts
This “artifacts” account is where we recommend centralizing and storing artifacts (e.g. ECR, assets, etc) for CI/CD
core-public
For public S3 buckets, public ECRs, public AMIs, anything public. This will be the only account that doesn’t have a SCP that blocks public s3 buckets. Use-cases All s3 buckets are private by default using a SCP in every account except for the public account
data-prod, data-staging, data-dev
The "data" account is where the quants live =) Runs systems like Airflow, Jupyterhub, Batch processing, Redshift
$tenant
The “$tenant” account is a symbolic account representing dedicated account environment. It’s architecture will likely resemble prod. This relates to{" "} this link
### Multi-Account (Production, Staging, Dev) :::caution Not recommended because there’s not enough isolation. ::: - Strict, enforceable boundaries between multiple environments (aka stages) at the IAM layer - Ability to create a release process whereby we stage changes in one account before applying them to the next account - Ability to grant developers administrative access to sandbox account (dev) so that they can develop/iterate on IAM policies. These policies then are committed as code and submitted as part of a Pull Request, where they get code reviewed. - API limits are scoped to an account. A bug in staging can't take out production. ### Single-account Strategy (Production=Staging=Dev) - NOT RECOMMENDED - Editing live IAM permissions in the mono account is the equivalent "cowboy coding" in production; we don't do this with our software, so we should not do this with our infrastructure - No strict separation between stages; copying and pasting infrastructure could accidentally lead to catastrophic outcomes - Very difficult to write/manage complex IAM policies (especially without a staging organization!) - No way to grant someone IAM permissions to create/manage policies while also restricting access to other production resources using IAM policies. This makes it very slow/tedious for developers to work on AWS and puts all the burden to develop IAM policies on a select few individuals, which often leads to a bottleneck - VPCs only provide network-level isolation. We need IAM level isolation. - AWS API limits are at the account level. A bug in staging/dev can directly DoS production services. ## Related Components - [account](/components/library/aws/account/) ## References Here are some great videos for context - Re:invent (2016) [https://www.youtube.com/watch?v=pqq39mZKQXU](https://www.youtube.com/watch?v=pqq39mZKQXU) - Re:invent (2017) [https://www.youtube.com/watch?v=71fD8Oenwxc](https://www.youtube.com/watch?v=71fD8Oenwxc) --- ## Decide on AWS Organization Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When establishing your AWS infrastructure strategy, you need to decide whether to create a new AWS Organization or reuse an existing one. This decision involves evaluating the limitations and capabilities of AWS Control Tower, the special roles within the root account, and the ability to manage organizational configurations. Cloud Posse recommends starting with a new organization to ensure a clear separation between historical and next-gen infrastructure, leveraging transit gateways or VPC peering for integration. :::tip Cloud Posse recommends starting with a **Net-New Organization** ::: ## Problem - Only one AWS Control Tower can exist in an organization. - AWS Control Tower only recently became manageable with Terraform, and full support is not available. Depending on the Scope of Work, Cloud Posse is usually responsible for provisioning accounts with terraform which requires all the same access as Control Tower. - Member accounts can only be provisioned from the top-level root “organization” account - The “root” account has a special `OrganizationalAccessRole` that can be used to administrate member accounts - With only one root organization, a business has no way to model/test/experiment with organizational-level configuration, which is a massive liability for onboarding new staff engineers responsible for training and to manage an organization ## Solution Here are some considerations for how we can work around the problems. ### Use Net-New Organization (Recommended) This process involves [How to Register Pristine AWS Root Account](/layers/accounts/tutorials/how-to-register-pristine-aws-root-account) which will serve as a net-new AWS organization (e.g. top-level payer account). Use transit gateway or VPC peering between heritage accounts and new accounts. This process ensures that there’s a clear delineation between the historical infrastructure and next-gen. We’ll treat the historical infrastructure as _tainted_, as in we do not know what was done with IaC; we’ll treat the next-gen accounts as pristine, hermetic environments where all changes are performed using IaC. :::info Companies with an AWS Enterprise Agreement (EA) can arrange for consolidated invoicing by speaking with their AWS Account Representative. ::: :::caution Reserved Instances and AWS Savings Plans cannot be shared across organizations. ::: ### Use Existing Organization You will need to grant Cloud Posse Administrative permissions in the root account in order to perform terraform automation of organizational infrastructure (E.g. Accounts, SCPs, Cloud Trails, etc). :::danger Cloud Posse does not prefer to work with **Existing Organizations** due to the liability. Cloud Posse does not know your organization as well as you do, and in order for us to manage an organization with Terraform we need to be Organizational Admins. ::: :::caution Cloud Posse recommends using the **Model Organization** pattern if you wish to use an **Existing Organization**. ::: ### Use Model Organization This pattern assumes we’ll provision a Net-New Organization that will be used for model purposes. It will not be used in a real production setting, but instead will be used as part of the engagement to enable Cloud Posse to set up all scaffolding. This pattern builds on the **Net-New Organization** but anticipates the customer re-implementing the entire process in their own existing organization. This process takes longer, but ensures your team gets the maximum onboarding experience and validates the documentation end-to-end by running your team through the process. :::caution Cloud Posse cannot estimate the time it will take your team to follow the documentation and implement it in your existing organization. ::: :::caution Reserved Instances and AWS Savings Plans cannot be shared across organizations. ::: :::danger We do not recommend this pattern if Release Engineering is in scope with your engagement with Cloud Posse. ::: --- ## Decide on AWS Support import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When setting up AWS Support, you need to decide which accounts require paid support plans. If you don’t have an AWS Enterprise Agreement, it’s recommended to enable Business-level support in the root account to expedite requests and manage organizational limits effectively. AWS Support is always enabled on a per-account basis. With an AWS Enterprise Agreement, AWS support is already included from a billing perspective for all accounts, but it still needs to be enabled on a per-account basis. :::caution Cross-account support is not provided by AWS. AWS Support will not address support questions that affect one account, from another account’s support subscription. See [https://aws.amazon.com/premiumsupport/faqs/#Cross-account_support](https://aws.amazon.com/premiumsupport/faqs/#Cross-account_support) ::: If no Enterprise Agreement, then at a minimum we recommend enabling Business-level support in the root account, which should cost roughly $100/mo (since nothing else runs in the root account). This enables us to expedite requests so that organizational limits may be raised (e.g. member accounts). Without paid support, requests may take several days and are more likely to be denied. For the latest pricing, go to [https://aws.amazon.com/premiumsupport/plans/](https://aws.amazon.com/premiumsupport/plans/) ## Sample Pricing --- ## Decide on Email Address Format for AWS Accounts import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import Note from "@site/src/components/Note"; When creating AWS accounts, you need to decide on the email address format. Each AWS account requires a unique email address that cannot be reused across multiple accounts. The chosen format should align with your organization’s email management strategy and ensure proper delivery and handling of AWS notifications. Every AWS account needs a unique email address. Email address cannot be reused across multiple AWS accounts. we are referring AWS accounts that contain resources, not individual user accounts ### Use Plus Addressing We'll use `+` addressing for each account (e.g. `ops+prod@ourcompany.com`) :::info Office 365 has finally added support for [https://docs.microsoft.com/en-us/exchange/recipients-in-exchange-online/plus-addressing-in-exchange-online](https://docs.microsoft.com/en-us/exchange/recipients-in-exchange-online/plus-addressing-in-exchange-online). ::: ### Use Slack Email Gateway - Create email group/alias for AWS accounts e.g. `ops@ourcompany.com` - Ideally set up to forward to a shared slack channel like `#aws-notifications` Follow [this guide to set up slack forwarding](/layers/accounts/tutorials/how-to-set-up-aws-email-notifications/). ### Use Mailgun Mailgun supports plus addressing and complex forwarding rules. It’s free for 5,000 emails. ### Use Google Group - Recommended Google Groups are probably the most common solution we see. It [works very well with plus addressing](https://support.google.com/a/users/answer/9308648?hl=en). ### Use AWS SES with Lambda Forwarder (catch-22) Provisioning AWS SES is nice, but we need an email address even for the root account, so it doesn’t solve the cold-start problem. [https://github.com/cloudposse/terraform-aws-ses-lambda-forwarder](https://github.com/cloudposse/terraform-aws-ses-lambda-forwarder) --- ## Decide on MFA Solution for AWS Root Accounts import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When setting up MFA for AWS root accounts, you need to decide on the most suitable solution to ensure security and manageability. The two most common options are TOTP (Time-Based One-Time Password) and U2F (Universal 2nd Factor). Cloud Posse recommends using 1Password for Teams or 1Password for Business to securely share TOTP tokens among stakeholders, ensuring both accessibility and protection. We need an MFA solution for protecting the master AWS accounts. The two most common options are TOTP and U2F (universal authenticator devices). :::tip Cloud Posse recommends **1Password for Teams** or **1Password for Business** ::: ### 1Password for Teams, 1Password for Business (TOTP) - Recommended TOTP tokens can be stored in a shared authenticator app like 1Password. This allows sharing of the secret amongst designated stakeholders. Additionally, using MFA with 1Password (like Duo) protects access to 1Password. For this reason, Cloud Posse recommends **1Password for Teams** or [1Password for Business.](https://1password.com/teams/pricing/) ### Yubikey (U2F) This is by far the most secure option but comes with a significant liability, if you do not add at least two or more physical devices. This physical device can be lost, broken, or damaged. Distributed team environments where the key can not be easily passed around adds to the difficulty of maintaining continuity when team members are out-of-the-office. Getting locked out of an AWS root account is not fun. For these reasons, we do not recommend it from a practical security perspective. ### Slack Bots One option is to hook up a Slack bot to a restricted channel. Using ChatOps, admins can request a token. The nice part about this is there's a clear audit trail of who is logging in. Also, we recommend a buddy system where each time a code is requested, a "Buddy" confirms this request to ensure it was merited. For this to be more secure, MFA must be enabled on Slack. ### Authy :::danger Does not support shared TOTP credentials ::: Authy is the original cloud-based authenticator solution. The downside is it doesn't support shared TOTP secrets, so a shared login must instead be used. This is not recommended. ### LastPass :::danger Does not support shared TOTP credentials ::: LastPass is **not** an option. It does not support shared TOTP secrets. Do not confuse this with the ability to authenticate with **LastPass** using TOTP/Authenticator apps. That's not what we need here. ## Related Tasks - [REFARCH-62 - Setup Root Account Root Credentials MFA](/layers/accounts/prepare-aws-organization/#set-up-mfa-on-root-account) --- ## Decide on Terraform State Backend Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When organizing your Terraform state, you need to decide on the backend architecture. Using S3, you can either opt for a single bucket, which simplifies management but grants broad access, or multiple buckets, which enhance security by segmenting access but add complexity. Consider your company’s security and compliance needs when making this decision. ## Context and Problem Statement We need someplace to store the terraform state. Multiple options exist (e.g. Vault, Terraform Enterprise, GitLab, Spacelift), but the only one we’ll focus on right now is using S3. The terraform state may contain secrets, which is unavoidable for certain kinds of resources (e.g. master credentials for RDS clusters). For this reason, it is advisable for companies with security and compliance requirements to segment their state backends to make it easier to control with IAM who has access to what. While on the other hand adding multiple state backends is good from a security perspective, on the other it unnecessarily complicates the architecture for companies that do not need the added layer of security. ## Considered Options We’ll use the [https://github.com/cloudposse/terraform-aws-tfstate-backend](https://github.com/cloudposse/terraform-aws-tfstate-backend) module to provision the state backends. This module already follows all standard best practices around private ACLs, encryption, versioning, locking, etc. Now we need to consider the options for how many buckets to manage. This decision is reversible but very tedious to change down the road. Therefore, we recommend doing what suits the long-term objectives of your company. Anyone who should be able to run `terraform` locally will need read/write access to a state bucket. ### Option 1: Single Bucket (Recommended for Companies without Compliance Requirements ) :::tip Our Recommendation is to use Option 1 because it’s the least complicated to maintain. Additionally, if you have a small team, there won’t be a distinction between those who have or do not have access to the bucket. ::: #### Pros - Single bucket to manage and protect #### Cons - Anyone doing terraform will need access to all state and can modify that state ### Option 2: Hierarchical State Buckets In this model, there will be one primary bucket that manages the state of all the other state buckets. But based on the number of segments you need, there will be multiple buckets that maintain the state for all the resources therein. As part of this decision, we’ll need to decide on what those segments are (e.g. `admin`, `restricted`, `unrestricted`, `security`; or one state bucket per account) for your use-case. #### Pros - It’s easier to secure who can access a state bucket when there are more buckets #### Cons - With more buckets, it’s more to oversee - Remote state lookups need to be aware of which bucket, account and IAM role is required to access the bucket ## References - Links to any research, ADRs or related Jiras --- ## Design Decisions import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; These are some of the design decisions you should be aware of when provisioning a new AWS organization. --- ## FAQ(Accounts) import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; Frequently asked questions about managing AWS accounts with Cloud Posse's reference architecture. ### Why not use Control Tower? AWS Control Tower cannot be managed with Terraform. Depending on the Scope of Work, Cloud Posse is usually responsible for provisioning accounts with terraform which requires all the same access as Control Tower. ### Why are there so many accounts? Leveraging multiple AWS accounts within an AWS Organization is the only way to satisfy IAM level isolation. Each account has a very specific purpose, that all associated resources are isolated in that given account. ### How we can set budgets? Create budgets with the `account-settings` component. For more, see [the `account-settings` component documentation](/components/library/aws/account-settings/) :::info Budgets created for the `root` account apply to the AWS Organization as a whole ::: ### How do you add or remove Service Control Policies? Service Control Policies are managed with the `account` component variable, `service_control_policies_config_paths`. For more, see [the `account` component documentation](/components/library/aws/account/) :::caution This component manages the state of all AWS accounts, so apply with extreme caution! ::: ### How can you create an Account? [Follow the documentation for creating and setting up AWS Accounts](/layers/accounts/tutorials/how-to-create-and-setup-aws-accounts/) ### How do you delete an Account? [Follow the documentation for deleting AWS Accounts](/layers/accounts/tutorials/how-to-delete-aws-accounts/) ### How can you create a Tenant? [Follow the documentation for creating a new Organizational Unit](/layers/accounts/tutorials/how-to-add-a-new-organizational-unit/) ### How do I use mixins and imports with Atmos As infrastructure grows, we end up with hundreds or thousands of settings for components and stack configurations. If we copy and paste these settings everywhere, it’s error-prone and not DRY. What we really want to do is to define a sane set of defaults and override those defaults when we need them to change. We accomplish this with Mixins. Mixins are imported into all stacks and each follow a set of rules. We use the `mixins/region` and `mixins/account` configurations to define common **variables** for all stacks. For example, `mixins/region/us-east-1.yaml` will define the variable `region: us-east-1`. **Note.** Do not import components into the account or region mixins. These are imported multiple times to define common variables, so any component imports would be duplicated and cause an Atmos error such as this: ``` Executing command: /usr/bin/atmos terraform deploy account-settings -s core-gbl-artifacts Found duplicate config for the component 'account-settings' for the stack 'core-gbl-artifacts' in the files: orgs/cch/core/artifacts/global-region/baseline, orgs/cch/core/artifacts/global-region/monitoring, orgs/cch/core/artifacts/global-region/identity. Check that all context variables in the stack name pattern '{tenant}-{environment}-{stage}' are correctly defined in the files and not duplicated. Check that all imports are valid. exit status 1 ``` ### Why does `account-map` fail with "The given key does not identify an element in this collection value"? This is a common error you may encounter when reading from `account-map`: ```bash Acquiring state lock. This may take a few moments... module.iam_roles.module.account_map.data.utils_component_config.config[0]: Reading... module.iam_roles.module.account_map.data.utils_component_config.config[0]: Read complete after 0s [id=xyx] module.iam_roles.module.account_map.data.terraform_remote_state.data_source[0]: Reading... module.iam_roles.module.account_map.data.terraform_remote_state.data_source[0]: Read complete after 5s module.iam_roles.data.awsutils_caller_identity.current[0]: Reading... module.iam_roles.data.awsutils_caller_identity.current[0]: Read complete after 1s [id=123] Planning failed. Terraform encountered an error while generating this plan. ╷ │ Error: Invalid index │ │ on ../account-map/modules/iam-roles/main.tf line 46, in locals: │ 46: is_root_user = local.current_identity_account == local.account_map.full_account_map[local.root_account_name] │ ├──────────────── │ │ local.account_map.full_account_map is map of string with 12 elements │ │ local.root_account_name is "core-root" │ │ The given key does not identify an element in this collection value. ╵ Releasing state lock. This may take a few moments...``` ``` This error typically occurs when the `root_account_aws_name` is not correctly configured in the `account-map` component. The root account has two different names: - **AWS Account Name**: The name you typed when originally creating the AWS account (often "root" but can be different) - **Account Alias**: The deterministic name set in the account configuration (e.g., "core-root") To fix this: 1. Find your root account name using AWS Organizations: ```bash aws organizations list-accounts \ --query "Accounts[?Id=='$(aws organizations describe-organization --query 'Organization.MasterAccountId' --output text)'].Name" \ --output text ``` 2. Update `stacks/catalog/account-map.yaml`: ```yaml components: terraform: account-map: vars: root_account_aws_name: "your-actual-root-account-name" # The name from step 1 root_account_account_name: "core-root" # This should always be "core-root" ``` 3. Reapply the account-map: ```bash atmos terraform apply account-map -s core-gbl-root ``` For more details, see [the account deployment documentation](/layers/accounts/deploy-accounts/#confirm-root-account-name). --- ## Initializing the Terraform State S3 Backend import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Follow these steps to configure and initialize the Terraform state backend using Atmos, ensuring proper setup of the infrastructure components and state management. | Steps | Actions | | ------------------------- | ----------------------------------------- | | Configure Terraform state | `atmos workflow init/tfstate -f quickstart/foundation/baseline` | ## Setting up the Terraform State Backend This is where we configure and run Atmos. Atmos is a workflow automation tool that we will use to call Terraform which will provision all the accounts and resources you need to create and manage infrastructure. The Atmos configuration can be found in the `atmos.yaml`. If you're unfamiliar with atmos, you can read more about it [here](https://atmos.tools). If you look at `components/terraform/`, you'll see a bunch of directories. These contain Terraform "root modules" that are provisioned with Atmos. At first they'll only have their vendor files, such as `components/terraform/tfstate-backend/component.yaml`. ## Vendor the Terraform State Backend component Vendor the Terraform State Backend component by running the following command. The steps here require the `tfstate-backend` and `account-map` components. :::tip What is Vendoring? Vendoring downloads the upstream component files from a central repository at a specified version. In this case, we are pulling the baseline components, which include all account components, the Terraform State component, and other necessary files for setting up the account foundation. This step only downloads the files to your local project - it does not deploy or make any changes to your infrastructure. [Read more about vendoring with Atmos](https://atmos.tools/core-concepts/vendor/) ::: ## Initialize the Terraform State Backend Run the following command to initialize the Terraform State Backend. This workflow has two steps: - Create the backend using a local Terraform state - Once the backend bucket exists, migrate the state file into the newly created S3 bucket ## Migrate all workspaces to S3 When prompted, type `yes` to migrate all workspaces to S3. ```shell Initializing the backend... Do you want to migrate all workspaces to "s3"? ``` :::info Granting SuperAdmin Access to Terraform State The IAM User for SuperAdmin will be granted access to Terraform State by principal ARN. This ARN is passed to the `tfstate-backend` stack catalog under `allowed_principal_arns`. Verify that this ARN is correct now. You may need to update the root account ID. ::: ## References - Review the [Structure of Terraform S3 State Backend Bucket](/layers/accounts/tutorials/terraform-s3-state/) --- ## Preparing Your AWS Organization import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; The Cold Start involves more manual steps than other layers. Read through the following steps and see [the detailed documentation](/layers/accounts/tutorials/manual-configuration/) for edge cases. In short, the steps are... | Steps | Actions | | ------------------------- | ----------------------------------- | | Install requirements | | | Build Geodesic | `make all` | | Vendor components | `atmos workflow vendor -f quickstart/foundation/baseline` | | Configure root SuperAdmin | Click Ops | :::tip Cold Start The set up process for the "baseline" or "account" layer is commonly referred to as the Cold Start. ::: ## Prerequisites Follow the [prerequisites steps in the How-to Get Started guide](/layers/project/#0-prerequisites). Start your Geodesic shell before continuing. ## Before Running Terraform (ClickOps) First, you'll need to perform some ClickOps to ensure things are set up before we use Terraform to manage AWS accounts. From the root account: 1. ### Get Business Class Support Enable [business support](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_policies_support.html) in the `root` account (in order to expedite requests to raise the AWS member account limits) 1. ### Set up MFA on Root Account Set up up the Virtual MFA device on the root account, following [the instructions in the AWS documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable_virtual.html#enable-virt-mfa-for-root) for enabling a virtual MFA device for an AWS account root user. Save the MFA TOTP key in 1Password by using 1Password's TOTP field and built-in screen scanner to scan the QR code presented on the web page. 1. ### Create the `SuperAdmin` IAM User [Create a `SuperAdmin` IAM User](/layers/accounts/tutorials/how-to-create-superadmin-user/). Do not enable "console login", do set up MFA, and then create a single Access key. Create an API Credential for the `SuperAdmin` credentials in 1Password and store the Access Key ID, Secret Access Key, Assigned MFA device ARN, and TOTP key. This is the user we will use until AWS Team Roles (`aws-team-roles` component) are provisioned. 1. ### Enable IAM Access for Billing For billing users, you need to enable IAM access. As the root user [open up the account settings for AWS Billing](https://us-east-1.console.aws.amazon.com/billing/home?region=us-east-1#/Account), then scroll to the section "IAM user and role access to Billing information" and enable it. 1. ### Enable Regions (Optional) The 17 original AWS regions are enabled by default. If you are using a region that is not enabled by default (such as Middle East/Bahrain), you need to take extra steps. For details, see [the detailed documentation](/layers/accounts/tutorials/manual-configuration/#optional-enable-regions) 1. ### Prepare for Account Quota Increase In order to deploy all accounts, you need to request an increase of the Account Quota from AWS support. This requires an AWS Organization to be created first, which we will create with Terraform in the [Deploy Accounts guide](/layers/accounts/deploy-accounts/#-prepare-account-deployment). This request can take a few days to process, so it's important to get it started early so that it doesn't become a blocker. At this time we don't need to request the increase, but we should be prepared to do so as soon as the AWS Organization is created. For more details, see [the detailed documentation](/layers/accounts/tutorials/manual-configuration/#provision-account-settings-component). --- ## Implement AWS Cold Start import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; These documents explain the complete cold-start process required to initialize a net-new AWS Organization using the SweetOps process by Cloud Posse. The cold-start process is the first step in setting up a new AWS Organization and is the most involved part of the process. It requires a lot of manual steps and remediations to bootstrap all the tooling and configurations before we can start to do anything meaningful. Please take the time to read through the entire process before starting, and reach out if you have any questions! ## Problem AWS provides poor documentation for all the steps required for setting up a new AWS Organization. Once the Organization is set up, there’s _a lot_ of manual steps and remediations required to bootstrap all the tooling and configurations before we can start to do anything meaningful. ## Solution The purpose of the cold-start process is to provision the initial set of AWS accounts and IAM roles so that users can log in to AWS with existing credentials and build the rest of the infrastructure. Many of the same steps need to be repeated in order to later add a new AWS account or organizational unit, so these documents also implicitly provide guidance for that task. ## Layout - [Detailed Cold Start Implementation](/layers/accounts/tutorials/manual-configuration) - [How to Set Up AWS Email Notifications](/layers/accounts/tutorials/how-to-set-up-aws-email-notifications) - [How to Register Pristine AWS Root Account](/layers/accounts/tutorials/how-to-register-pristine-aws-root-account) - [How to Create SuperAdmin user](/layers/accounts/tutorials/how-to-create-superadmin-user) - [Deprecated Cold Start Components](/layers/accounts/tutorials/deprecated-cold-start-components) ## Conventions There are a few things that are required but that change from project to project. To avoid excessive use of generic placeholders in the text, we use the following values as placeholders, and you should change them as needed: - `acme` is the placeholder for the project's _namespace_. Each project needs to have a project namespace, preferably 3 or 4 letters, that is unique in the Cloud Posse architecture. It is part of every identifier and is what makes it possible to generate globally unique names for things like S3 buckets. - `uw2` is the placeholder for the chosen abbreviation style of the default AWS Region. If you are using the “tenant” feature, you need to replace `uw2` with `-` where `` is the tenant abbreviation for the tenant holding the root AWS organization and `` is the abbreviated default AWS region. - "1Password" is the placeholder for the application or system used to store and share secrets like AWS credentials. ## Prerequisites This document assumes as a baseline that DevOps and developer personnel who need AWS access have a working computing environment that: - Docker installed - Has a host file system that containers can access (when properly configured) - Can install and run GUI applications (E.g. Leapp) - Has `bash` v3 or newer installed - Has `git` installed - Has a GNU Make-compatible version of `make` installed (e.g. `brew install make`). NOTE: make version 4.4+ has been known to have issues with our build-harness. If the command times out, please try make 4.3 or earlier. Typically our recommendation is Docker Desktop running on a local, dedicated (to the user) macOS or Windows computer. If the user cannot install new applications due to administrative restrictions, then all of the above components, plus [Leapp](https://leapp.cloud/), should be pre-installed for them. :::caution Apple M1 Support Geodesic works on the M1 running as `amd64` (not `arm64`). Docker auto-detects this by default, but otherwise it’s possible to pass `--platform linux/amd64` to `docker` to force the platform. Geodesic bundles [dozens (hundreds?) of executable binaries](https://github.com/cloudposse/packages), of which most do not yet have M1 `arm64` versions available to install. In order to support M1, we would need to build (compile and link) Python3 for M1. ::: --- ## Old Cold Start Components (Deprecated) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; This page serves as a tombstone for deprecated components included in the cold start. Here you will find documentation on how you could set those components up. | **Deprecated Component** | **Replaced By:** | | ------------------------ | ---------------- | | `sso` | `aws-saml` | | `iam-primary-roles` | `aws-teams` | | `iam-delegated-roles` | `aws-team-roles` | ### Deprecated Steps for Deploying IAM Roles With the addition of `aws-teams`, `aws-team-roles`, and `aws-saml`, the `iam-primary-roles`, `iam-deprecated-roles`, and `sso` components are deprecated. Use the respective components as a replacement. #### Provision `sso` component :::caution Deprecated with the addition of `aws-teams`. Use `aws-saml` instead ::: GSuite integration requires collaboration between a GSuite administrator and a DevOps engineer with `root` account access. You should have previously installed the IdP metadata files and configured the `sso` component in `gbl-identity.yaml` . In the Geodesic shell, execute the following commands: ``` atmos terraform apply sso --stack gbl-identity ``` In the output, make note of the SAML provider ARNs, as you will need them when provisioning user access in GSuite or Okta. :::info Security Note The following guidance provides clarity on the security of the metadata files. [https://security.stackexchange.com/questions/65743/saml-2-0-idp-metadata-security](https://security.stackexchange.com/questions/65743/saml-2-0-idp-metadata-security) **TL;DR** they are safe to commit to VCS. ::: Learn more about the [sso](/components/library/aws/identity-center/) component. #### Provision `iam-primary-roles` component :::caution Deprecated with the addition of `aws-teams`. Use `aws-teams` instead ::: This component creates "primary" IAM roles in the `identity` account. "Primary" roles are the ones that users are allowed to log into via SSO. All other roles, which we term "delegated" roles, must be _assumed_ from a "primary" role, and will be generated in the next step through the [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) component. `sso` project must be completed so the identity providers exist. If identity providers are added or deleted, Refer to the `full_account_map` you saved from `account-map` and copy the account ID (number) for the `identity` account to the `gbl-identity.yaml` stack as the value for `components.terraform.iam-primary-roles.vars.primary_account_id`. _Be sure to enclose the account ID in quotation marks,_ because AWS treats it as an opaque string, but YAML will do conversions on numbers. Before roles are created, they cannot be referenced as principals within IAM principals, and so other roles cannot be given permission to assume them. In general, we do not need roles in delegated accounts to be able to assume other roles in delegated accounts, so this is not a problem. However, we want to allow this is in `identity`, so we have a flag we can change before and after the roles are created. This component requires `root-admin` role to run. Use `assume-role` (or `assume-role-saml` for SAML auth) to assume the role. In the Geodesic shell, execute the following commands to provision the IAM roles in `identity`: ``` atmos terraform apply iam-primary-roles --stack gbl-identity -var=assume_role_restricted=false ``` This provisions the primary roles. Now that they have been created, we can restrict which roles can assume them. In the Geodesic shell, execute the following to put the restrictions in place: ``` atmos terraform apply iam-primary-roles --stack gbl-identity ``` Learn more about the [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-primary-roles) component. #### Provision `iam-delegated-roles` component :::caution Deprecated with the addition of `aws-teams`. Use `aws-team-roles` instead ::: The [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) component creates the delegated IAM roles in the member accounts allowing the `identity` account to assume roles into the member accounts. **NOTE:** `sso` must be completed so the identity providers exist. `iam-primary-roles` must be completed to generate the primary roles in the `identity` account. In the Geodesic shell, execute the following commands for each account except for `identity` and `root`: ``` atmos terraform apply iam-delegated-roles --stack gbl-${acct} ``` `root` account is a bit special. As we said regarding primary roles in the `identity` account, before roles are created, other roles cannot be given permission to assume them. We want to allow roles in `root` to assume other roles in `root` for a few reasons: - We want people to voluntarily reduce their privileges when they are not using them, so we want admins to be able to assume less powerful roles. - We want people to be able to use Terraform, which we generally configure to assume a role in the `root` account in order to access the Terraform state. Again as with `identity`, we have a flag we can change before and after the roles are created, although it has a different name, because here the choice is between `restricted` and `no` assume role capability whereas in `identity` the choice is between `restricted` and `unrestricted` assume role capability. In the Geodesic shell, execute the following commands to provision the delegated IAM roles in `root`: ``` atmos terraform apply iam-delegated-roles --stack gbl-root -var=same_account_assume_role_enabled=false ``` Then execute the following commands to enable assume role within the `root` account: ``` atmos terraform apply iam-delegated-roles --stack gbl-root ``` Note in the output the ARN of the `...root-terraform` role, you will need it later. Learn more about the [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) component. ## Deactivate SuperAdmin user :::info SuperAdmin is no longer deactivated with the addition of the `aws-teams` components. SuperAdmin is required to make changes to root components, since the `...root-terraform` role no longer has permission to make changes to the the `root` account ::: After all the cold-start components have been provisioned, we have SSO and all IAM roles configured, and users will be able to login to AWS accounts using GSuite by assuming the IAM roles. We can deactivate `SuperAdmin` user now. Login to AWS IAM console using the root credentials, select the `SuperAdmin` user, go to the "Security credentials" tab, and click "Make inactive" for any keys listed under "Access keys". This will prevent them from being used until you need them again, and at the same time will enable you to use them without having to reconfigure Leapp. Alternately (and somewhat more securely), you can delete the keys, and when you need to use them again create new ones and add them to Leapp as you did with the first keys. --- ## Add a new Organizational Unit import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; Learn how to add a new Organizational Unit (OU) to an existing AWS Organization set up to Cloud Posse standards. The process involves updating the `account` catalog with the new OU details and reapplying the `account` component using the `SuperAdmin` user, ensuring all planned changes by Terraform are carefully reviewed before applying. ## Problem We want to create a new Organizational Unit with an existing AWS Organization set up to Cloud Posse standards ## Solution Update the `account` catalog Add the new OU to the `account` catalog and reapply the component. :::info The `account` component must be applied with the SuperAdmin user, which is typically found in 1Password. For more on SuperAdmin, see[How to Create SuperAdmin user](/layers/accounts/tutorials/how-to-create-superadmin-user) ::: For example to add a new Organizational Unit called `example` with one account called `foo`, add the following to `stacks/catalog/account.yaml`: ``` components: terraform: account: vars: organizational_units: - name: example accounts: - name: example-foo tenant: example stage: foo tags: eks: false ``` Then reapply the `account` component: :::caution The `account` component is potentially dangerous! Double-check all changes planned by Terraform ::: ``` assume-role SuperAdmin atmos terraform plan account -s core-gbl-root assume-role SuperAdmin atmos terraform apply account -s core-gbl-root ``` --- ## How to add or mirror a new region import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; **DRAFT** ## Problem ## Solution The current primary region is `us-west-2` and the new desired region is `us-east-2` ### Additional region 1. Create root stacks of the new region e.g. `ue2*.yaml` 2. Update VPC CIDR documentation 3. Create minimal components in the yaml such as `vpc`, `transit-gateway`, and perhaps `compliance` (or others) if applicable 4. Deploy minimal components 5. Optionally deploy `dns-delegated` if a new Hosted Zone is required per region 6. This is no longer used going forward as we can use a single Hosted Zone for `.example.com` and create multi domain records within it such as `postgres-example.ue2` without having to create a `ue2..example.com` HZ. 7. Optionally deploy `transit-gateway-cross-region` component to peer both regions ``` TBD ``` 7. Optionally deploy new github runners (if applicable) 1. Retrieve the new github runner IAM role arn 1. Update `iam-primary-roles` to include the new IAM role and deploy it to update `identity-cicd` role 1. Optionally deploy new `spacelift-worker-pool` (if applicable) 1. Set a worker pool id map in the `spacelift` component 1. Set a `worker_pool_name` global variable in the new region 1. Update `iam-primary-roles` to include the new IAM role and deploy it to update `identity-ops` role ### If new region needs to be a mirror of the primary region 1. Same steps as above, except instead of minimal components, we want to copy and paste all of the primary region into the new desired region. We will not reprovision anything from `gbl*`. 2. Mirror the SSM parameters by exporting them from the primary region and importing them into the new region ``` stage=sandbox CURRENT_REGION=us-west-2 NEW_REGION=us-east-2 # get services services=$(AWS_PROFILE=$NAMESPACE-gbl-$stage-admin AWS_REGION=$CURRENT_REGION aws ssm describe-parameters --query 'Parameters[].Name' | grep / | cut -d'/' -f2 | sort | uniq | tr '\n' ' ') # export AWS_PROFILE=$NAMESPACE-gbl-$stage-admin AWS_REGION=$CURRENT_REGION chamber export -o chamber-$stage.json $services # import for service in $services; do AWS_PROFILE=$NAMESPACE-gbl-$stage-admin AWS_REGION=$NEW_REGION chamber import $service chamber-$stage.json; done ``` 3. Ensure all hostnames use the correct regional endpoints (either by Hosted Zone or by record) 4. Optionally, it’s not recommended, but if the tfstate bucket needs to be migrated 5. Make sure everything in Spacelift is confirmed/discarded/failed so nothing is left in an unconfirmed state. 6. Schedule a date with the customer so no applies go through 7. Set desired count on the spacelift worker pool to 0 with a max and min count of 0 8. Manually copy from old tfstate bucket to new tfstate bucket 9. PR to change all the `backend.tf.json` files over to the new bucket and set new bucket in the global vars 10. Check locally to see that new bucket is used and stacks show no changes 11. Merge PR 12. revert spacelift worker pool 13. Ensure everything is working in Spacelift ### If an old region needs to be destroyed The following can be destroyed in Spacelift using a run task with `terraform destroy -auto-approve` The following should be destroyed locally with `atmos` --- ## How to Adopt/Import Legacy AWS Accounts for Management with Atmos import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Sometimes provisioning a new account and migrating resources isn’t an option. Adopting a legacy account into your new organization allows you to manage new resources using Atmos and Terraform while leaving old ones unmanaged. This guide provides step-by-step instructions for importing legacy AWS accounts into the Infrastructure Monorepo, updating account configurations, and integrating with Atmos and Spacelift for automation. ## Problem Legacy AWS accounts may be owned by an organization (the company, not the AWS organization) but not part of the AWS Organization provisioned by the Infrastructure Monorepo via Atmos. This process is meant to “adopt” or “import” the accounts in question for use with the Infrastructure Monorepo, so that their infrastructure may be automated via Atmos and Spacelift. ## Solution :::caution When you remove a member account from an Organization, the member account’s access to AWS services that are integrated with the Organization are lost. In some cases, resources in the member account might be deleted. For example, when an account leaves the Organization, the AWS CloudFormation stacks created using StackSets are removed from the management of StackSets. You can choose to either delete or retain the resources managed by the stack. For a list of AWS services that can be integrated with Organizations, see [AWS services that you can use with AWS Organizations](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_integrate_services_list.html). ([source](https://aws.amazon.com/premiumsupport/knowledge-center/organizations-move-accounts/)) ::: :::tip Legacy AWS Accounts must be invited to the AWS Organization managed by the Infrastructure Monorepo, and imported into the `account` component. From there, the Geodesic image and the Atmos stacks must be updated to reflect the presence of the new accounts. ::: ### Invite the AWS Account(s) Into the Organization Managed by the Infrastructure Monorepo Before the AWS account(s) can be managed by Terraform + Atmos in the Infrastructure Monorepo, they need to invited to the AWS Organization managed by the Infrastructure Monorepo. From the management account (usually named `root` or `mgmt-root`), navigate to `AWS Organizations` → `AWS Accounts` → `Add an AWS Account` → `Invite an existing AWS account`. Enter the ID of the AWS account that is to be invited, then `Send Invitation`. This step does **not** have to be done (and in terms of best-practices **should not** be done) via the root user. It can be done via an assumed role provisioned by `iam-delegated-roles` that has the `organizations:DescribeOrganization` and `organizations:InviteAccountToOrganization` permissions — i.e. the `admin` delegated role. The email address of the management account must be verified, if not already done so. Otherwise, the invitation cannot be sent. An email will be sent to the email address(es) associated with the AWS account(s) in question. The link leads to the AWS console where the invitation can be accepted or denied. Once again, this step does **not** have to be done (and in terms of best-practices **should not** be done) via the root user. However, since this is a legacy account, it does not have the roles deployed by `iam-delegated-roles`. Thus, ensure that the assumed role used to accept the invitation has the `organizations:ListHandshakesForAccount`, `organizations:AcceptHandshake` and `iam:CreateServiceLinkedRole` permissions, for example via the built-in `AdministratorAccess` policy. ### Add the Account(s) to the `account` Component Configuration :::info The OU that you are adding the legacy AWS accounts to needs to be adjusted to match your business use case. Some organizations may choose to employ a “legacy” OU. Others, as in the example below, have an OU for each “tenant” within the organization, and legacy AWS accounts will live alongside current AWS accounts within a single OU. ::: Your existing `account` component configuration should look something like this: ```yaml components: terraform: account: backend: s3: workspace_key_prefix: account role_arn: null vars: enabled: true account_email_format: aws+%s@acme.com account_iam_user_access_to_billing: DENY organization_enabled: true aws_service_access_principals: - cloudtrail.amazonaws.com - ram.amazonaws.com - sso.amazonaws.com enabled_policy_types: - SERVICE_CONTROL_POLICY - TAG_POLICY organization_config: root_account: name: mgmt-root stage: root tenant: mgmt tags: eks: false accounts: [] organization: service_control_policies: [] organizational_units: - name: mgmt accounts: - name: mgmt-artifacts stage: artifacts tenant: mgmt tags: eks: false - name: mgmt-audit stage: audit tenant: mgmt tags: eks: false - name: mgmt-automation stage: automation tenant: mgmt tags: eks: true - name: mgmt-corp stage: corp tenant: mgmt tags: eks: true - name: mgmt-dns stage: dns tenant: mgmt tags: eks: false - name: mgmt-identity stage: identity tenant: mgmt tags: eks: false - name: mgmt-network stage: network tenant: mgmt tags: eks: false - name: mgmt-sandbox stage: sandbox tenant: mgmt tags: eks: true - name: mgmt-security stage: security tenant: mgmt tags: eks: false service_control_policies: - DenyLeavingOrganization - name: core accounts: - name: core-dev stage: dev tenant: core tags: eks: true - name: core-staging stage: staging tenant: core tags: eks: true - name: core-prod stage: prod tenant: core tags: eks: true service_control_policies: - DenyLeavingOrganization service_control_policies_config_paths: - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/cloudwatch-logs-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/deny-all-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/iam-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/kms-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/organization-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/route53-policies.yaml" - "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.9.1/catalog/s3-policies.yaml ``` Entries corresponding to the legacy AWS accounts need to be added before the accounts can be imported: ``` ... - name: core-legacydev stage: dev tenant: core tags: eks: true - name: core-legacystaging stage: staging tenant: core tags: eks: true - name: core-legacyprod stage: prod tenant: core tags: eks: true ... ``` Once entries are created, a Terraform plan can be run against the `account` component when assuming the `admin` delegated role in `mgmt-root`: :::caution If the Terraform plan attempts to destroy the newly-added legacy account, do not apply it! See section on working around destructive Terraform plans for newly-added legacy accounts. ::: ``` √ : [eg-mgmt-gbl-root-admin] (HOST) infrastructure ⨠ atmos terraform plan account -s mgmt-gbl-root ``` ``` Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: + create Terraform will perform the following actions: # aws_organizations_account.organizational_units_accounts["core-legacydev"] will be created + resource "aws_organizations_account" "organizational_units_accounts" { + arn = (known after apply) + email = "aws+core-legacydev@acme.com" + iam_user_access_to_billing = "DENY" + id = (known after apply) + joined_method = (known after apply) + joined_timestamp = (known after apply) + name = "core-legacydev" + parent_id = "ou-[REDACTED]" + status = (known after apply) + tags = { + "Environment" = "gbl" + "Name" = "core-legacydev" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } + tags_all = { + "Environment" = "gbl" + "Name" = "core-legacydev" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } } # aws_organizations_account.organizational_units_accounts["core-legacyprod"] will be created + resource "aws_organizations_account" "organizational_units_accounts" { + arn = (known after apply) + email = "aws+core-legacyprod@acme.com" + iam_user_access_to_billing = "DENY" + id = (known after apply) + joined_method = (known after apply) + joined_timestamp = (known after apply) + name = "core-legacyprod" + parent_id = "ou-[REDACTED]" + status = (known after apply) + tags = { + "Environment" = "gbl" + "Name" = "core-legacyprod" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } + tags_all = { + "Environment" = "gbl" + "Name" = "core-legacyprod" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } } # aws_organizations_account.organizational_units_accounts["core-legacystaging"] will be created + resource "aws_organizations_account" "organizational_units_accounts" { + arn = (known after apply) + email = "aws+core-legacystaging@acme.com" + iam_user_access_to_billing = "DENY" + id = (known after apply) + joined_method = (known after apply) + joined_timestamp = (known after apply) + name = "core-legacystaging" + parent_id = "ou-[REDACTED]" + status = (known after apply) + tags = { + "Environment" = "gbl" + "Name" = "core-legacystaging" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } + tags_all = { + "Environment" = "gbl" + "Name" = "core-legacystaging" + "Namespace" = "eg" + "Stage" = "root" + "Tenant" = "mgmt" + "eks" = "true" } } Plan: 3 to add, 0 to change, 0 to destroy. Changes to Outputs: ... ``` ##### Workaround for Destructive Terraform Plans for Legacy Accounts Terraform will sometimes try to destroy a legacy account because the `iam_user_access_to_billing` attribute is not modifiable via Terraform (or the AWS CLI, for that matter): [https://github.com/hashicorp/terraform-provider-aws/issues/12959](https://github.com/hashicorp/terraform-provider-aws/issues/12959) [https://github.com/hashicorp/terraform-provider-aws/issues/12585](https://github.com/hashicorp/terraform-provider-aws/issues/12585) [https://github.com/aws/aws-cli/issues/6252](https://github.com/aws/aws-cli/issues/6252) The workaround for this is to edit the `iam_user_access_to_billing` attribute manually in the Terraform state: 1. Download the Terraform state locally in Geodesic via `assume-role aws s3 cp s3://eg-mgmt-ue1-root-admin-tfstate/account/mgmt-root/terraform.tfstate ./terraform.tfstate` 2. Find the `iam_user_access_to_billing` attribute for the account in question and change it to either `ENABLE` or `DENY`, depending on whether or not IAM access to billing is enabled for the account in question. Ensure that the value inputted into the state manually reflects the actual state of the AWS account in question. Overwrite the Terraform state via `assume-role aws s3 cp --sse AES256 ./terraform.tfstate s3://eg-mgmt-ue1-root-admin-tfstate/account/mgmt-root/terraform.tfstate`. 1. Run the same Terraform plan and copy the expected MD5 digest for the Terraform state. 2. In DynamoDB, find the MD5 digest item for the Terraform state for the workspace in question (e.g. `mgmt-root` in `eg-mgmt-ue1-root`) and overwrite the current digest value with the expected digest value. ### Import Legacy AWS Account(s) Into the `account` Component Workspace and Update Legacy Account Attributes Using the resources address from the Terraform plan, the AWS account(s) can now be imported: ``` $ cd components/terraform/account $ terraform import 'aws_organizations_account.organizational_units_accounts["core-legacydev"]' 123456789024 $ terraform import 'aws_organizations_account.organizational_units_accounts["core-legacystaging"]' 123456789025 $ terraform import 'aws_organizations_account.organizational_units_accounts["core-legacyprod"]' 123456789026 ``` Once the legacy accounts have been imported to the `account` component workspace, their attributes can be updated by running a Terraform apply. ``` $ cd ../../../ # go back to the root of the infrastructure monoorepo $ atmos terraform plan account -s mgmt-gbl-root ... # verify that there are no destructive changes, and that all of the intended attributes are correct, especially the email address associated with each account $ atmos terraform apply account -s mgmt-gbl-root ``` ### Update Geodesic Image The Geodesic container image contains an `aws-accounts` script that is responsible for creating the AWS CLI profile used within the image both by users and Spacelift. It needs to be updated to reflect the newly-added AWS accounts. The existing `rootfs/usr/local/bin/aws-accounts` script should have a section with an associative array representing the account names and their IDs, and a list representing the order of profiles corresponding to each AWS account: ``` ... declare -A accounts # root account intentionally omitted accounts=( [mgmt-artifacts]="123456789012" [mgmt-audit]="123456789013" [mgmt-automation]="123456789014" [mgmt-corp]="123456789015" [mgmt-dns]="123456789016" [mgmt-identity]="123456789017" [mgmt-network]="123456789018" [mgmt-sandbox]="123456789019" [mgmt-security]="123456789020" [core-dev]="123456789021" [core-prod]="123456789022" [core-staging]="123456789023" ) # When choosing a profile, the users will be presented with a # list of profiles in this order readonly profile_order=( mgmt-artifacts mgmt-audit mgmt-automation mgmt-corp mgmt-dns mgmt-identity mgmt-network mgmt-sandbox mgmt-security mgmt-root core-dev core-prod core-staging ) ... ``` Both the associative array and the list need to be updated to reflect the newly added legacy AWS accounts: ``` ... declare -A accounts # root account intentionally omitted accounts=( ... [core-legacydev]="123456789024" [core-legacyprod]="123456789025" [core-legacystaging]="123456789026" ) # When choosing a profile, the users will be presented with a # list of profiles in this order readonly profile_order=( ... core-legacydev core-legacyprod core-legacystaging ) ... ``` The script then needs to be re-run to update both the local AWS CLI config and the versioned CI/CD AWS CLI config: ``` $ aws-accounts config-saml >> ~/.aws/config $ aws-accounts config-cicd > rootfs/etc/aws-config/aws-config-cicd # (and commit this change upstream) ``` ### Add and Deploy Atmos Stacks for Newly-added Legacy AWS Accounts Once the Geodesic image has been updated, global and regional stacks for the newly-added legacy accounts need to be added. These stacks should also contain core components for the newly-added accounts. ``` ... core-gbl-legacydev.yaml core-gbl-legacyprod.yaml core-gbl-legacystaging.yaml core-uw2-legacydev.yaml core-uw2-legacyprod.yaml core-uw2-legacystaging.yaml ... ``` The global stacks should contain the `iam-delegated-roles` and `dns-delegated` components: ``` import: - core-gbl-globals - catalog/iam-delegated-roles - catalog/dns-delegated vars: stage: legacydev terraform: vars: {} components: terraform: dns-delegated: vars: zone_config: - subdomain: legacydev zone_name: core.acme-infra.net ``` The regional stacks should contain `dns-delegated` and `vpc` components: ``` import: - core-uw2-globals - catalog/dns-delegated - catalog/vpc vars: stage: legacydev terraform: vars: {} helmfile: vars: {} components: terraform: dns-delegated: vars: zone_config: - subdomain: uw2.legacydev zone_name: core.acme-infra.net vpc: vars: cidr_block: 172.2.0.0/18 # change this to the VPC CIDR block in the legacy account ``` The `vpc` component will need to have the existing VPC in the legacy account imported to the component’s workspace: ``` $ atmos terraform plan vpc -s core-uw2-legacydev ... # copy resource ID of VPC $ cd components/terraform/vpc $ terraform import ... # import VPC and subnets ``` Once the stacks are added, deploy the following components: :::info If AWS SSO is being used via the `aws-sso` component, the configuration of the aforementioned component needs to be updated in order to configure permission sets for the newly added accounts. Then, the component needs to be redeployed. ::: - `account-map` (in `mgmt-gbl-root`) - `iam-primary-roles` (in `mgmt-gbl-identity`) - `iam-delegated-roles` (in each new global stack) - `dns-delegated` (in each new global and regional stack) - `vpc` (in each new regional stack) - `compliance` (if being used) ### Validate Access :::info Existing AWS Accounts invited to an AWS organization lack the OrganizationAccountAccessRole IAM role created automatically when creating the account with the AWS Organizations service. The role grants admin permissions to the member account to delegated IAM users in the master account. Thus the [https://github.com/cloudposse/terraform-aws-organization-access-role](https://github.com/cloudposse/terraform-aws-organization-access-role) module should be leveraged as part of the `account-settings` component in order to deploy this IAM role. ::: Finally, validate access to the newly-added legacy accounts via the delegated IAM roles by signing into the AWS console and also running `assume-role eg-core-gbl-legacydev-admin aws sts get-caller-identity` (for each newly added account) within Geodesic. ### References - [https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_invites.html](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_invites.html) - [account](/components/library/aws/account/) - [Implement AWS Cold Start](/layers/accounts/tutorials/cold-start/) --- ## How to Create and Setup AWS Accounts import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Note from '@site/src/components/Note'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; This guide covers the process of creating and setting up new AWS accounts. It includes detailed instructions for updating account catalogs, configuring stack settings, deploying necessary components, and managing AWS account profiles. ## Problem Setting up new AWS accounts can be complex and error-prone without proper guidance and tooling. ## Solution This guide provides a step-by-step approach to create and configure new AWS accounts using Cloud Posse's reference architecture components and conventions. ### Update Account Catalog First, add the new account to your account catalog configuration. This defines the account structure within your AWS Organization. Navigate to your account catalog file (typically `stacks/catalog/account.yaml`) and add the new account under the appropriate Organizational Unit (OU). **Example for a new account named `foobar`:** ```yaml organizational_units: - name: core accounts: # ... existing accounts ... - name: foobar stage: foobar tags: eks: false ``` Choose the appropriate OU based on your organization's account strategy. Common OUs include `core` for core infrastructure and `plat` for platform accounts. ### Create Stack Configuration Files Create the corresponding stack configuration files for your new account in the `stacks/orgs///` folder (or follow the convention your organization uses). See the following example where the namespace is `acme`, the tenant is `core`, and the stage is `foobar`. **Example Directory Structure:** For a new account `foobar` in the `core` tenant under namespace `acme`: ``` stacks/orgs/acme/core/foobar/ ├── _defaults.yaml ├── global-region/ │ ├── baseline.yaml # Account settings │ └── identity.yaml # AWS Team Roles └── us-east-1/ ├── baseline.yaml # Regional baseline └── app.yaml # Regional application platform ``` Start with a basic global-region stack for account settings and create placeholder regional stacks for future expansion. You can add more regional stacks and components as your account requirements grow. ### Define Stage Mixin By convention, we treat every account as an operating stage. Stages are configured as mixins, so that each stack operating in that stage can import that mixin to have a consistent stage name. **For example:** `stacks/mixins/stage/foobar.yaml` Create a mixin file for your stage to keep configuration DRY. This file contains only variables. ```yaml # This file is for vars only; do not import components here # For more information, see "Mixins and Imports with Atmos" in the baseline documentation vars: stage: foobar ``` ### Configure Account Defaults **For example:** `stacks/orgs/acme/core/foobar/_defaults.yaml` This file is necessary for keeping configuration DRY and establishing common settings to be imported in all subsequent files. ```yaml import: - orgs/acme/core/_defaults - mixins/stage/foobar ``` ### Configure Account Settings **For example:** `stacks/orgs/acme/core/foobar/global-region/baseline.yaml` This file configures account-level settings and policies. ```yaml import: - orgs/acme/core/foobar/_defaults - mixins/region/global-region - catalog/account-settings components: terraform: account-settings: vars: # Allow creating public S3 buckets in this account # Public buckets are used, for example, to deploy documentation websites into preview environments and serve them via Lambda@Edge block_public_acls: false ignore_public_acls: false block_public_policy: false restrict_public_buckets: false ``` ### Configure AWS Team Roles **For example:** `stacks/orgs/acme/core/foobar/global-region/identity.yaml` This file configures cross-account IAM roles and team access permissions. ```yaml import: - orgs/acme/core/foobar/_defaults - mixins/region/global-region - catalog/aws-team-roles components: terraform: aws-team-roles: vars: roles: reader: trusted_teams: - devops - developers - managers poweruser: trusted_teams: - devops - developers - managers terraform: trusted_teams: - devops - developers - managers - gitops ``` ### Regional Stack **For example:** `stacks/orgs/acme/core/foobar/us-east-1/baseline.yaml` Create regional stacks based on your specific needs. Start with a basic placeholder that you can expand later. ```yaml import: - orgs/acme/core/foobar/_defaults - mixins/region/us-east-1 components: terraform: {} ``` ### Submit and Merge Configuration PR Create a pull request with your configuration changes. This is a critical step because once the account is created, it's difficult to reverse the process. Always review configuration changes carefully before merging, as account creation is irreversible and affects your AWS Organization structure. Once the PR is reviewed, approved, and merged, proceed to the next step. ### Deploy Account Infrastructure Deploy the necessary components to create and configure your new account. Use `plan` and `apply` commands without `-auto-approve` for safety. **Prerequisites:** - Ensure you have `SuperAdmin`, `managers`, or appropriate elevated permissions **Deploy Components:** ```bash # Create the new account atmos terraform apply account --stack core-gbl-root # Update account map to include the new account atmos terraform apply account-map --stack core-gbl-root # Configure account settings for the new account atmos terraform apply account-settings --stack core-gbl-foobar # Deploy AWS Team Roles to enable cross-account access atmos terraform apply aws-team-roles --stack core-gbl-foobar ``` The order of deployment is important. The `account` component creates the account, `account-map` updates the account mapping required to map accounts to IAM roles, `account-settings` configures the account, and `aws-team-roles` enables cross-account role assumption. ### Complete Account Setup via ClickOps After deploying the account infrastructure, you need to perform some manual configurations to finalize the account setup. **Recommended Steps:** - Reset the root user password and set up MFA - Enable any necessary optional AWS regions - Unsubscribe the account's email address from marketing emails Save the root credentials, MFA TOTP key, and account number in 1Password. Use a highly restricted vault, and only share access on a strict need-to-know basis. For detailed step-by-step instructions, see [Complete Account Setup via ClickOps](/layers/accounts/deploy-accounts/#step-number-clickops-to-complete-account-setup). ### Update AWS Configuration After deploying the infrastructure components, update your AWS configuration files to include the new account. **Generate AWS Configuration Files:** **Commit and Push Changes!** ### Verify Account Setup Verify that your new account has been properly configured and is accessible. **Check Account Creation:** - Verify the account appears in your AWS Organization console - Confirm the account has the correct email address and tags - Ensure the account is in the correct Organizational Unit **Test Cross-Account Access:** - Verify you can assume roles in the new account from your identity account - Test Terraform operations in the new account - Confirm AWS Team Roles are properly configured If you encounter issues, check the Terraform outputs from each component deployment for error messages and configuration details. ## References - [How to Delete AWS Accounts](/layers/accounts/tutorials/how-to-delete-aws-accounts) (in case a mistake was made) - [AWS Teams and Team Roles](/layers/identity/centralized-terraform-access) - [Account Management](/layers/accounts) - [Identity and Access Management](/layers/identity) --- ## How to Create `SuperAdmin` user import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; The `SuperAdmin` user is recommended for performing certain high-risk administrative tasks that modify IAM permissions affecting users on SSO or SAML. We don't recommend making these types of changes with your own SSO or SAML identity because a misconfiguration could lead to account lockout. This guide outlines the steps to create a secure `SuperAdmin` user in AWS, including setting permissions, enabling MFA, and storing credentials safely in 1Password. [Follow the prerequisites steps in the How-to Get Started guide](/layers/project/#0-prerequisites) ### Create the SuperAdmin User First, create the SuperAdmin IAM user in the AWS web console. 1. Login to the AWS `root` account using the root credentials. 1. In the IAM console, select "Users" on the sidebar. 1. Click "Add users" button 1. Enter "SuperAdmin" for "User name" and check "Programmatic access" and leave "AWS Management Console access" unchecked. Click "Next: Permissions" 1. Under "Set permissions", select "Attach existing policies directly". A list should appear, from which you should check "AdministratorAccess". Click "Next: Tags" 1. Skip the tags, Click "Next: Review" 1. Review and click "Create user" 1. The Success page should show you the "Access key ID" and hidden "Secret access key" which can be revealed by clicking "Show". Copy these to your secure credentials storage as you will need them shortly 1. Click "Close" to return to the IAM console. Select "Users" on the sidebar if it is not already selected. You should see a list of users. Click the user name "SuperAdmin" (which should be a hyperlink) to take you to the Users -> SuperAdmin "Summary" page 1. On the "Users -> SuperAdmin" "Summary" page, click on the "Security credentials" tab 1. In the "Sign-in credentials" section, find: "Assigned MFA device: Not assigned | Manage" and click "Manage" 1. Choose "Virtual MFA device" and click "Continue" 1. Press the "Show secret key" button 1. Copy the key into 1Password as a AWS Credential using the "MFA" field 1. Use the MFA codes from 1Password to complete the MFA setup process (you will input 2 consecutive codes) 1. You should be taken back to the "Security Credentials" tab, but now the "Assigned MFA device" field should have an ARN like `arn:aws:iam:::mfa/SuperAdmin` 1. Copy the ARN and keep it with the Access Key in 1Password 1. Now we need to create an Access Key for CLI access. Click on the "Create Access Key" under "Access Keys" 1. Select "Command Line Interface" and click the "I understand..." checkbox then click 'Next' 1. Enter a description if you like, such as 'SuperAdmin CLI Access' and click 'Create' ### Store SuperAdmin Credentials in 1Password The `SuperAdmin` credentials should be properly stored in 1Password. Relative to other potential 1Password item types, the most appropriate 1Password item type for these credentials is `login`. Since these are programmatic credentials and not an actual login with an endpoint from which the website favicon can be retrieved. Additionally, the password field should be kept empty. For convenience in retrieving the TOTP code when using Leapp, save `com.leapp.app` as a website URL. 1. Set the username to `SuperAdmin` 1. Create fields for the `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and the TOTP (known as One Time Password field type in 1password) via the AWS virtual MFA device's secret 1. Add a note in the following format: ``` This account's Access Key should be made inactive when not needed. CURRENT STATUS: ACTIVE Use this account for API/command line access to administrative functions that IAM roles cannot do, such as provision IAM roles. This account should not be allowed to log in to the AWS console, and therefore does not have a password. Root account ID: [AWS ACCOUNT ID] User ARN arn:aws:iam::[AWS ACCOUNT ID]:user/SuperAdmin MFA Device ARN arn:aws:iam::[AWS ACCOUNT ID]:mfa/SuperAdmin ``` The resulting entry in 1password should appear as follows: 1. Hit save once you are done. Once the SuperAdmin credentials need to be disabled, do not forget to update the notes in this item ## References [REFARCH-73 - Provision SuperAdmin User for Root Level IAM Management](/layers/accounts/prepare-aws-organization/#create-the-superadmin-iam-user) --- ## How to Delete AWS Accounts import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Note from '@site/src/components/Note'; import Admonition from '@theme/Admonition'; Learn the step-by-step instructions for deleting AWS accounts that are no longer needed or were provisioned by accident. We cover renaming email addresses to avoid future conflicts, using cloud-nuke to delete all resources, and manually closing the account through the AWS Console. It also includes recommendations for renaming or repurposing accounts to avoid the overhead of deletion. ## Problem Sometimes accounts are provisioned that are no longer needed or were provisioned by accident. AWS provides no easy programmatic way to destroy accounts. The following ClickOps method is required to destroy the account. Email addresses associated with AWS accounts are globally unique and cannot be reused even after the account deletion. If you ever intend to use the email address again with AWS on another account, we strongly recommend that you first rename the email address on record before proceeding to delete the account. ## Solution We recommend renaming or repurposing accounts rather than deleting them due to the overhead and complexity of the deletion process. ### Delete All Account Resources Closing an account might not automatically terminate all active resources. You may continue to incur charges for active resources even after closing the account. To avoid tedious manual steps, leverage [cloud-nuke](https://github.com/gruntwork-io/cloud-nuke) to delete all resources. **Install cloud-nuke:** ```bash brew install cloud-nuke ``` **Dry run to see what will be deleted:** ```bash cloud-nuke aws --dry-run ``` **Delete all resources (WARNING: This will delete ALL resources in the account):** ```bash cloud-nuke aws ``` **Export required AWS config for running cloud-nuke:** Save the following in `.envrc`: ```bash # or wherever the configuration file is export AWS_CONFIG_FILE=rootfs/etc/aws-config/aws-config-teams # This is necessary for cloud-nuke export AWS_SDK_LOAD_CONFIG=true AWS_PROFILE=core-gbl-root-admin cloud-nuke aws --dry-run ``` Instead of using the AWS profile, you can also use the SuperAdmin user credentials directly. This is often simpler for one-off operations like account deletion. **Create a shell script to automate cloud-nuke across accounts:** Create `nuke-echo.sh`: ```bash #!/usr/bin/env bash cat rootfs/etc/aws-config/aws-config-teams | grep '\[profile' | cut -d' ' -f2 | cut -d']' -f1 | grep admin | while read profile; do echo AWS_PROFILE=$profile cloud-nuke aws $@; done ``` **Run with specific regions and exclusions:** ```bash ./nuke-echo.sh --region us-east-2 --region us-west-2 --region eu-west-3 --exclude-resource-type s3 --exclude-resource-type guardduty --exclude-resource-type transit-gateway ``` Delete resources in the following order for best results: - Security accounts - Audit accounts - Platform accounts (dev, staging, qa, prod, perf, sandbox) - Corp accounts - Auto accounts - Network accounts (due to transit gateway) Consider skipping: - DNS accounts - Identity accounts - Root accounts Skip the following resources until the very end: - `iam` - due to IAM roles used to initiate cloud-nuke - `s3` - due to the time it takes to delete S3 objects - `guardduty` - controlled by security account across all accounts - `asg` - can fail to destroy EKS ASGs - `transit-gateway` - controlled by network account ### Handle Manual Deletions Some resources are not covered by cloud-nuke and require manual deletion: **GuardDuty:** - Navigate to the security account in AWS console - Go to GuardDuty settings, disassociate all members, then suspend and disable - Go to root account and remove security from being the GuardDuty delegate **Per-region resources:** - AWS Backup - MWAA (Managed Workflows for Apache Airflow) - MSK (Managed Streaming for Apache Kafka) - ElastiCache - EFS (Elastic File System) - EC2 Client VPN - Transit Gateway attachments in network account - Any other resources that use EC2 Network Interfaces (ENI) ### Prepare Account for Deletion **Remove Organization Restrictions:** - In the root account, remove the `DenyLeavingOrganization` SCP from the account to be deleted **Update Account Configuration:** - Comment out the account line in `stacks/catalog/account.yaml` - Apply changes to the account component: ```bash atmos terraform apply account -s core-gbl-root ``` **Access the Account:** - Look up the account's root user email address and password in 1Password - If no password exists, request a password reset - Open the email (accessible via `#aws-notifications` Slack channel) and click the reset link - Set a new password and save it in 1Password **Rename Email Address (Recommended):** - Change the email address to something disposable by appending "deleted" and the date - Use a "plus address" if not already using one - Example: `aws+prod@cpco.co` → `aws+prod-deleted-2024-01-01@cpco.co` - Navigate to Account Settings > Edit > Edit Email in the AWS console - Validate the email address from the account dashboard **Complete Account Setup:** - Set account contact information (if not inherited from Organizations) - Accept the AWS Customer Agreement (if not inherited from Organizations) ### Delete the Account You can only close up to 10% of accounts per month. A valid payment method is no longer required to close an account, but the API does not support this feature yet. Each account must be manually closed. **Close Account in AWS Console:** - Open the AWS Console and go to the root account for the Organization - Navigate to Organizations > AWS accounts > select account > Close - Check all boxes and enter Account ID > Close account **Clean Up Infrastructure Configuration:** - Remove the account from Terraform state for the account component - Remove the account from `stacks/catalog/account.yaml` - Follow standard process for opening a pull request and updating the codebase - Apply changes to the account component: ```bash atmos terraform apply account -s core-gbl-root ``` **Update AWS Configuration:** - Regenerate AWS configuration files to remove the deleted account: ```bash aws-config teams > rootfs/etc/aws-config/aws-config-teams aws-config switch-roles > rootfs/etc/aws-config/aws-extend-switch-roles ``` - Commit and push the updated configuration files ## References - [AWS Account Closure Guide](https://aws.amazon.com/premiumsupport/knowledge-center/close-aws-account/) - [Terminate Resources Before Account Closure](https://aws.amazon.com/premiumsupport/knowledge-center/terminate-resources-account-closure/) - [AWS Organizations Account Management](https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_remove.html) - [Automating AWS Account Deletion](https://onecloudplease.com/blog/automating-aws-account-deletion#deleting-an-aws-account) (including for comedic value) 🤣 - [How to Create and Setup AWS Accounts](/layers/accounts/tutorials/how-to-create-and-setup-aws-accounts) - [cloud-nuke GitHub Repository](https://github.com/gruntwork-io/cloud-nuke) --- ## How to manage Account Settings import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; Manage and update AWS account settings and budgets by modifying and reapplying the account-settings component. ## Problem We want to update Account Settings for a given AWS Account ## Solution Update the `account-settings` component Account Settings are managed by the `account-settings` component and deployed for each account. Update the `account-settings` catalog and reapply the component. For example to add password requirements, add the following to `stacks/catalog/account-settings.yaml`: ``` components: terraform: account-settings: backend: s3: role_arn: null vars: enabled: true minimum_password_length: 20 maximum_password_age: 120 ``` Then reapply the `account-settings` component for the given account. `example` tenant and `foo` stage are used in this example ``` atmos terraform apply account-settings -s example-gbl-foo ``` ### How to set Budgets Budgets are also managed with the `account-settings` component. In order to create budgets, enable budgets in the `account-settings` component :::info Budgets were added to the `account-settings` component in early 2022. Make sure the component contains `budgets.tf`. If not, pull the latest from [the upstream modules](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/account-settings). ::: ``` components: terraform: account-settings: vars: enabled: true budgets_enabled: true budgets_notifications_enabled: true budgets_slack_webhook_url: https://url.slack.com/abcd/1234 budgets_slack_username: AWS Budgets budgets_slack_channel: aws-budgets-notifications budgets: - name: 1000-total-monthly budget_type: COST limit_amount: "1000" limit_unit: USD time_unit: MONTHLY - name: s3-3GB-limit-monthly budget_type: USAGE limit_amount: "3" limit_unit: GB time_unit: MONTHLY ``` Then reapply the `account-settings` component for all accounts. This example only applies to one account. Repeat this step for all accounts ``` atmos terraform apply account-settings -s example-gbl-foo ``` --- ## How to Register Pristine AWS Root Account import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Step-by-step instructions for setting up a new AWS root account, including prerequisites such as deciding on an email address format, provisioning a shared 1Password vault, and gathering necessary contact and billing information. [REFARCH-60 - Register Pristine AWS Root Account](/layers/accounts/tutorials/how-to-register-pristine-aws-root-account/) ### Prerequisites 1. [REFARCH-51 - Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts/) 2. [REFARCH-31 - Provision 1Password with Shared Vault](/layers/project/design-decisions/decide-on-1password-strategy/) 3. [REFARCH-471 - Decide on AWS Organization Strategy](/layers/accounts/design-decisions/decide-on-aws-organization-strategy/) 4. Company primary contact information 5. Company credit card and billing information 6. Company business mobile phone number you have access to use for SMS 7. Email address that supports [plus addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) (e.g. aws+root@example.com) ### Instructions :::info See the official AWS Documentation for the most up-to-date instructions. [https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) ::: 1. Navigate to this link to create an AWS account: [https://portal.aws.amazon.com/billing/signup#/start](https://portal.aws.amazon.com/billing/signup#/start) 2. Specify the email address defined in the design decision and append `+root` before the `@`sign (e.g. `ops+root@ourcompany.com`) 3. In AWS, every AWS account needs a unique email address. We use `+` addressing for each account for disambiguation. Since this is the `root` account, we’re appending `+root` 4. `+` addressing is not universally supported. E.g. Gsuite supports it but Microsoft Exchange does not. 5. Generate a strong password and add it to the appropriate 1Password vault. Make sure it’s the vault you’ve shared with Cloud Posse. 6. AWS account name will be `root` 7. Click `Continue (step 1 of 5)` 8. Select “How do you plan to use AWS?” radio button: `Business - for your work, school or organization` 9. Add the primary contact’s full name 10. Enter your company’s name as it appears on legal documentation 11. Enter the primary contact’s business phone number 12. Enter the company’s legal address 13. Click the link provided to read the terms `AWS Customer Agreement` and check the box 14. Click `Continue (step 2 of 5)` 15. Enter billing information 16. Click `Verify and Continue (step 3 of 5)` 17. Select “How should we send you the verification code?” radio button: `Text message (SMS)` 18. Enter a business mobile phone number that you have access to use. Ideally, this is a number that can forward text messages to your team (e.g. Google Voice or Twillio). 19. Complete the Security check 20. Click `Send SMS (step 4 of 5)` 21. Enter the verification code that was sent as an SMS message to the mobile phone number provided in step 16 22. Click `Continue (step 4 of 5)` 23. Select `Business support - From $100/month` 24. We recommend this support plan so we can use it to expedite account limit increases for the organization. This will be useful throughout the engagement. 25. Click “Complete sign up” 26. Click on the button `Go to the AWS Management Console` 27. Select the radio button `Root user` 28. Enter the Root user email address and click `Next` 29. This is the same address we set up in step 2 (`ops+root@ourcompany.com`) 30. Complete the Security check and click `Submit` 31. Enter the password that was stored in 1Password for this account in step 3 and click `Sign in` :::tip Congratulations! You are now able to proceed with the rest of the cold start process. ::: ### Related articles - [Decide on AWS Organization Strategy](/layers/accounts/design-decisions/decide-on-aws-organization-strategy) - [AWS Documentation: How do I create and activate a new AWS account?](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) | Related issues | | | -------------- | --- | --- ## Set Up AWS Email Notifications import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Admonition from '@theme/Admonition'; Learn how to setup AWS account email notifications by routing emails from AWS accounts to a dedicated Slack channel. It covers using plus addressing for efficient email management and provides detailed steps for creating the Slack channel, generating its email address, and configuring email forwarding. :::caution Make sure to use the email address corresponding to the [Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts) ADR. ::: There should exist an `[organization]-aws-notifications` Slack channel dedicated for emails addressed to the dedicated AWS address, e.g. `aws@[domain]`. Cloud Posse recommends using `+` addressing, such that all AWS accounts will have their account email set to be `aws+[account name]@[domain]`. Once emails addressed to `aws@[domain]` are set up to be routed to this Slack channel's email address, all emails addressed to any of the AWS accounts' emails will appear in this channel, thanks to the use of `+` addressing. If the use of `+` addressing is not possible, a dedicated email address such as `aws.[account name]@[domain]` can be set up for each AWS account, and a routing rule for each of these addresses to the AWS Notifications Slack channel's email address can be created. The following is an example of how to set up this channel and configure email routing to the dedicated Slack channel's email address: 1. ## Create the shared Slack channel (under Slack Connect): (see also: [How to Provision Shared Slack Channels](/jumpstart/tutorials/how-to-provision-shared-slack-channels) ) 2. ## Generate the Slack channel's email address (at the top of the newly-created Slack channel): 3. ## Set up email forwarding (example: G Suite / Google Workspace) in **`Google Workspace -> Settings for Gmail -> Routing -> Recipient address map`:** 4. ## Manage Incoming Emails Depending on your current Slack Workspace permissions, you may need to [manage incoming emails for your Slack workspace or organization](https://slack.com/help/articles/360053335433-Manage-incoming-emails-for-your-workspace-or-organization) and allow incoming email --- ## Manual Cold Start Implementation import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; This guide covers the manual cold-start process for setting up a new AWS Organization using Cloud Posse’s SweetOps process. It involves numerous manual steps and configurations to establish the necessary tooling and environments before you can use Terraform effectively. These steps are part of laying the foundation for your environment and are necessary prerequisites for using Terraform. :::tip Automated Solution as an Alternative (Recommended) This document includes many advanced and alternate configurations for the Cold Start implementation, including detailed explanations and manual setup steps. We also do not regularly test this, and it's likely out of date. Alternatively, we have a simplified [Automated Cold Start Implementation](/layers/accounts/deploy-accounts/) that includes the minimum requirements for provisioning a Cold Start. This automated setup document makes use of Atmos Workflows to alleviate Click Ops woes but does not account for edge cases. ::: ## Manual Preparation Steps - This first set of steps can be done in any order. The remaining steps need to be done in the order listed. - Decide on a "namespace" for your organization in consultation with Cloud Posse. [Decide on Namespace Abbreviation](/layers/project/design-decisions/decide-on-namespace-abbreviation) - Decide on a default AWS Region to deploy the initial set of resources. Optionally, decide on a failover Region for backup resources. Decide on the [Region abbreviation scheme](/resources/adrs/adopted/use-aws-region-codes/) you want to use: `short` or `fixed`. Cloud Posse slightly favors `fixed` to keep IDs and labels shorter. [Decide on Primary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) - Decide on whether or not you are going to use Spot Instances. Decide on your desired default quota for Spot Instance and On-Demand Instance vCPUs. Also, decide on default quotas for any special instance types such as GPU-enabled instances. - One of the main drivers for this decision is going to be the EKS Node Pool Architecture. Specifically, whether or not to use Spotinst Ocean for managing the EKS Node Pool. - Create 2 Slack Channels and share them with Cloud Posse (this requires a paid Slack account): one for collaboration and discussion and another one to receive automated messages. - The collaboration Slack channel should be named something like `[organization]-general`. - The AWS notifications Slack channel should be named something like `[organization]-aws-notifications`. See [How to Set Up AWS Email Notifications](/layers/accounts/tutorials/how-to-set-up-aws-email-notifications) for more information. - Choose, provision, and share with Cloud Posse a mechanism for securely storing and sharing secrets, including MFA TOTP keys. Cloud Posse uses [1Password](https://1password.com/) and can provide a 1Password `vault` if needed. In this document, we will refer to this storage as 1Password, but you can substitute any other system you want. [Decide on 1Password Strategy](/layers/project/design-decisions/decide-on-1password-strategy) - Identify a credit card for AWS to bill charges to and provide the details to whomever is going to set up the initial AWS account. - Install [Leapp](https://www.leapp.cloud/) to manage AWS credentials for command-line use (this is a free desktop app installed on user's computers). - If using GSuite or Okta SSO integration, collect IdP metadata files, e.g. `GoogleIDPMetadata-acme.com.xml`. After the `git` repository is created and populated, configure `sso.vars.saml_providers` with the names of the files, and commit the files into `git` under `components/terraform/sso` - Set up GitHub (requires a paid account, steps to be completed in this order) - Create a GitHub organization for the project if you do not already have one. - Create a `CloudPosse` team and invite designated Cloud Posse personnel to it. - Create (or have Cloud Posse create) an "infrastructure" Git repository that will host configuration and tools for the company. - Decide on your account, organization names, and organization structure. [Decide on AWS Account Flavors and Organizational Units](/layers/accounts/design-decisions/decide-on-aws-account-flavors-and-organizational-units) [Decide on AWS Organization Strategy](/layers/accounts/design-decisions/decide-on-aws-organization-strategy) - Decide on and set up email addresses for all accounts. We recommend using a single email address/account with "+" addressing for each AWS account, and forwarding all emails to a single shared Slack channel. **Ensure that the email addresses are set up and forwarded to a shared Slack channel that Cloud Posse has access to before continuing.** [Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts) - [Sign up with AWS](/layers/accounts/tutorials/how-to-register-pristine-aws-root-account), create the `root` account (you may want to call it `main`, `master`, `management`, or something else, but we will refer to it in this document as `root`) using the email address you decided on. This is often performed by multiple people, so coordinate with all involved. If you already have a billing relationship with AWS, then this may be a different process than if you do not. The link shows up how to get started if you have no relationship to AWS, and you will need to provide valid billing details including a credit card, so sometimes this is best done by someone in the finance department. - **Getting started with "root credentials".** Each person involved in creating and setting up the `root` account should work toward ensuring the account is configured as described in this subsection. The configuration must be done with "root" credentials, which are the credentials created during the initial signup process or, if already signed up when the account is created. **The email address, password, and other information should be stored in 1Password.** If the password is unavailable when you need it (for example, if you have an existing relationship with AWS and thus simply created a child account, which has an assigned password you are not told), start the login process on the AWS console using the "Root user email address" (the email address you assigned to the account) and use the "Forgot password?" link to generate an email which will be forwarded to the Slack channel. Click the link in that email to be allowed to set a new password and log in. Save the email address, password, and account number in 1Password in a new "login" credential. - As soon as possible, share the AWS account number with your Cloud Posse contact, so Cloud Posse can assist in expediting the requests. - Set up billing, including providing a credit card to charge. - Activate [IAM User and Role Access to Billing Information](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/control-access-billing.html#ControllingAccessWebsite-Activate) using the root account root user, navigate to account settings, scroll down, edit IAM User and Role Access, and enable it. This is important for non-root access to billing in the future. - Set your support plan to "Business" level. This is necessary to expedite the quota increase requests we will soon make, which otherwise could hold up the cold start process for days. - **Ensure the account does not belong to an organization** so we can make it the manager of the new organization we are going to create. Depending on how the account was created, it may or may not belong to an organization. If you just created it via "Create an AWS account" on the AWS website, then it will not belong to an organization. If you had a pre-existing relationship with AWS and created the new account from the AWS web console while logged in to an existing account, then the account probably _does_ belong to an organization. If it does belong to an organization, contact the organization manager (someone in your company responsible for your AWS account management, probably the person who actually created the account) and have it removed. - If you are planning to use an AWS Region that is not enabled by default (such as the Middle East/Bahrain), you need to take the following extra steps. Most companies only use the default regions and can skip these steps. - Navigate to the [Billing home page](https://console.aws.amazon.com/billing/home?#/account). In the "**AWS Regions**" section, enable the regions you want to enable. - Go to the IAM [account settings page](https://console.aws.amazon.com/iam/home?#/account_settings) and edit the **STS Global endpoint** to create session tokens valid in all AWS regions. As of this writing, once you enable the regions in the prior step, the AWS web console will prompt you to make this change and you can just follow the prompt. - Secure the root account. - Verify the securitiy of the `root` account root user password (meets security standards and has not been shared over an insecure channel like Slack, email, or SMS) and that it is properly stored in 1Password. If in any doubt, reset the "root user" password (change it if you know it, use the "Forgot password?" link to reset it if you do not.) Use 1Password or [Random.org](https://www.random.org/passwords) to create a password 26-38 characters long, including at least 3 of each class of character: lower case, uppercase, digit, and symbol. You may need to manually combine or add to the generated password to ensure 3 symbols and digits are present. Save the email address and generated password as web login credentials in 1Password, and save the account number as a separate field in the same credential. - Set up the root user to use a Virtual MFA device, following [the instructions in the AWS documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_enable_virtual.html#enable-virt-mfa-for-root) for enabling a virtual MFA device for an AWS account root user. Save the MFA TOTP key in 1Password by using 1Password's TOTP field and built-in screen scanner to scan the QR code presented on the web page. - Create a `SuperAdmin` [IAM user](https://console.aws.amazon.com/iamv2/home?#/users) in the `root` account (needed to provision initial AWS resources, including member accounts, SSO and IAM roles). Do _not_ enable "console login", do set up MFA, and then create a single Access key. Create a Secure Note for the SuperAdmin credentials in 1Password and store the Access Key ID, Secret Access Key, Assigned MFA device ARN, and TOTP key in it. See [create SuperAdmin user](/layers/accounts/tutorials/how-to-create-superadmin-user) for more details. - Configure Leapp to use the `SuperAdmin` Access key with the profile `acme-SuperAdmin`, where `acme` is the "namespace" you have chosen for your organization, and using the AWS Region you have chosen as default. Refer to the [Leapp documentation for setting up an AWS IAM user](https://docs.leapp.cloud/latest/configuring-session/configure-aws-iam-user/) for details. - Generate and check into `git` and GitHub an initial configuration. Include SSO IdP metadata files. Update reference configuration (including Dockerfile). For more information, see [How to Create an Infrastructure repository](/layers/project/create-repository/) We are almost done with the manual steps. Unfortunately, we will have to take a few more manual steps later, after we have some other things provisioned. ## Remaining steps Here is a summary of the remaining steps, which will be explained in greater detail below. Most of these steps use `atmos` which uses `terraform` and although they are executed manually, are not considered entirely manual steps because of the support and automation those tools provide. - Build and run Geodesic shell - Provision Terraform state backend (S3 bucket to store state and DynamoDB table for state locking). - Provision AWS organization - Perform some manual tasks to finish setting up the organization - Provision AWS organization units and accounts - If necessary, enable optional AWS regions. - Update the Account Map. - Provision account settings (set the default mode for EBS volumes to be encrypted, provision account aliases and password policies). - Provision [IAM SAML Identity Provider](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml.html). The SAML identity provider is used to establish trust between your company IdP (usually Google Workspace or Okta) and AWS, allowing you to login to AWS using your existing credentials, also known as Single Sign-On or SSO. If using Okta, also provision Okta API users with permissions to list IAM roles in the account (needed for Okta AWS integration). - Provision primary IAM roles in the `identity` account. These roles have trust policies to allow the SAML provider to assume them. - Provision delegated IAM roles in all other accounts. These roles have trust policies to allow the primary IAM roles in the `identity` account to assume them, effectively allowing you to first login to the `identity` account with SSO and then assume the delegated roles to access the other accounts (`prd`, `dev`, `stg`, etc.). - Deactivate the SuperAdmin user. To add a new account later, follow all the same steps, except you can skip steps that the new account does not affect, like provisioning the `tfstate-backend` component or enabling resource sharing. ### Build and run the Geodesic shell Prerequisites for your host computer: - Docker installed - `make` installed, preferably GNU Make - `git` installed - Infrastructure Git repo cloned If all goes well, you should be able to build and run the Infrastructure Docker image from your host by executing `make all` from the command line in the root directory of your Git repo clone. If you have issues at this step, contact Cloud Posse or look for help in the Cloud Posse [Geodesic](https://github.com/cloudposse/geodesic/) or [Reference Architecture](https://github.com/cloudposse/learns) repos. At this point (after `make all` concludes successfully) you should be running a `bash` shell inside the Infrastructure Docker container (which we will also call the "Geodesic shell") and your prompt should look something like this: ``` ⧉ Infrastructure ✗ . [none] / ⨠ ``` From here forward, any command-line commands are meant to be run from within the Geodesic shell. For the most part you will be using **Cloud Posse's** `atmos` command, which is quite powerful, but has a non-obvious limitation: it is sensitive to the current working directory and **must be run from the root of the repository**. ## Use SuperAdmin credentials After starting Geodesic, use Leapp to activate the SuperAdmin credentials. Then configure Geodesic to use the SuperAdmin credentials with ``` export AWS_PROFILE=acme-SuperAdmin ``` where `acme` is the namespace you have chosen for your project and `acme-SuperAdmin` matches the profile name you configured in Leapp for the SuperAdmin user. (Geodesic may complain about not being able to find a profile matching the role or something like that; this can be safely ignored.) At this point the Geodesic prompt should look something like ``` ⧉ Infrastructure √ . [SuperAdmin] / ⨠ ``` Note that the "✗" has changed to "√" to indicate that valid AWS credentials are present, and `[none]` has changed to `[SuperAdmin]` to indicate what IAM role your credentials are associated with. The `/` indicates the current directory is `/`, which is correct for running `atmos`. ### Provision `tfstate-backend` component All Terraform components require a state backend. In this section, we explain how to provision and use the S3 backend (note that other backends could be used, e.g. `remote`, `vault` etc.). The S3 backend normally consists of an AWS S3 bucket and a DynamoDB table provisioned in the `root` account. The `tfstate-backend` component initializes the backend state, so we have a cart-before-horse scenario to resolve. To overcome this, initialize the component in the following order: - Be sure you are running with `SuperAdmin` credentials as explained above in [Use SuperAdmin credentials](#Use SuperAdmin credentials) - In the Geodesic shell, run `cd /localhost/` - In the Geodesic shell, run `cd "$REPO_ROOT"` where `$REPO_ROOT` is the path on your host computer to the repository. Note that Geodesic should have set up mounts and symbolic links such that the host path should resolve and reference your host computer's directory if you set it up as instructed above. (Only the part of the host file system under `$HOME` is accessible. If you have checked out your repository in a different part of the file system, you will need to move it, at least temporarily.) Run `ls` to confirm you are seeing the `Dockerfile` and `Makefile` you expect. The Geodesic prompt should include `(HOST)` to indicate the current directory is on the host filesystem, not the container's file system. - Run `atmos terraform apply tfstate-backend -var=access_roles_enabled=false --stack core-uw2-root --auto-generate-backend-file=false` - Verify the plan matches the expected output before approving the plan, or type “no” to the `Enter a value:` prompt if something looks wrong. - Verify the S3 bucket and the DynamoDB table exist within the `root` account - Note that at this stage the Terraform state will be stored locally - Run `atmos terraform apply tfstate-backend -var=access_roles_enabled=false --stack core-uw2-root --init-run-reconfigure=false` - This will generate the S3 backend files `backend.tf.json` for the `tfstate-backend` component - It will prompt you to migrate all workspaces to S3. Type `yes` and it will push the previously created local state to S3. We have now provisioned Terraform state backend and migrated the `tfstate-backend` component's state to the newly created S3 backend. We will come back later to provision the restricted IAM roles that allow access to the backend, but for now, SuperAdmin has the needed access. We'll now provision all other components using the S3 backend. ### Provision AWS Organization using the `account` component :::caution Verify and confirm all account settings now. Changing account settings after provisioning accounts can be highly difficult. In particular, be sure to double and triple check the provided email address. You must have access to this email address to access, update, or delete the account. ::: Your AWS Organization is managed by the `account` component, along with accounts and organizational units. However, because the AWS defaults for an Organization and its accounts are not exactly what we want, and there is no way to change them via Terraform, we have to first provision the AWS Organization, then take some steps on the AWS console, and then we can provision the rest. ### Use AWS Console to create and set up the Organization Unfortunately, there are some tasks that need to be done via the console. Log into the AWS Console with the root (not SuperAdmin) credentials you have saved in 1Password. #### Request an increase in the maximum number of accounts allowed :::caution Make sure your support plan for the _root_ account was upgraded to the "Business" level (or Higher). This is necessary to expedite the quota increase requests, which could take several days on a basic support plan. Without it, AWS support will claim that since we’re not currently utilizing any of the resources, so they do not want to approve the requests. AWS support is not aware of your other organization. If AWS still gives you problems, please escalate to your AWS TAM. See [AWS](/resources/glossary/aws). ::: 1. From the region list, select "US East (N. Virginia) us-east-1". 2. From the account dropdown menu, select "My Service Quotas". 3. From the Sidebar, select "AWS Services". 4. Type "org" in the search field under "AWS services" 5. Click on "AWS Organizations" in the "Service" list 6. Click on "Default maximum number of accounts", which should take you to a new view 7. Click on "Request quota increase" on the right side of the view, which should pop us a request form 8. At the bottom of the form, under "Change quota value", enter the number you decided on in the previous step (probably "20") and click "Request" #### (Optional) Create templates to request other quota increases New accounts start with a low limit on the number of instances you can create. However, as you add accounts, and use more instances, the numbers automatically adjust up. So you may or may not want to create a template to generate automatic quota increase requests, depending on how many instances per account you expect to want to provision right away. Create a [Quota request template](https://docs.aws.amazon.com/servicequotas/latest/userguide/organization-templates.html) for the organization. From the Sidebar, click "Quota request template" Add each EC2 quota increase request you want to make: 1. Click "Add Quota" on the right side of the view 2. Under "Region", select your default region (repeat with the backup region if you are using one) 3. Under "Service", type "EC2" and select "Amazon Elastic Compute Cloud (Amazon EC2)" 4. Under "Quota", find the quota you want to increase. The likely candidates are: 5. type "stand" and select "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) Instances" 6. type "stand" and select "All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Request" 7. type "g i" and select "Running On-Demand G Instances" 8. type "all g" and select "All G Spot Instance Requests" 9. Under "Desired quota value" enter your desired default quota 10. Click "Add" After you have added all the templates, click "Enable" on the Quota request template screen to enable the templates. #### Enable resource sharing with AWS Organization [AWS Resource Access Manager (RAM)](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) lets you share your resources with any AWS account or through AWS Organizations. If you have multiple AWS accounts, you can create resources centrally and use AWS RAM to share those resources with other accounts. Resource sharing through AWS Organization will be used to share the Transit Gateway deployed in the `network` account with other accounts to connect their VPCs to the shared Transit Gateway. This is a one-time manual step in the AWS Resource Access Manager console. When you share resources within your organization, AWS RAM does not send invitations to principals. Principals in your organization get access to shared resources without exchanging invitations. To enable resource sharing with AWS Organization via AWS Management Console - Open the Settings page of AWS Resource Access Manager console at [https://console.aws.amazon.com/ram/home#Settings](https://console.aws.amazon.com/ram/home#Settings) - Choose "Enable sharing with AWS Organizations" To enable resource sharing with AWS Organization via AWS CLI ``` √ . [acme-SuperAdmin] (HOST) infra ⨠ aws ram enable-sharing-with-aws-organization { "returnValue": true } ``` For more information, see: - [https://docs.aws.amazon.com/ram/latest/userguide/what-is.html](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) - [https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html](https://docs.aws.amazon.com/ram/latest/userguide/getting-started-sharing.html) - [https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-ram.html](https://docs.aws.amazon.com/organizations/latest/userguide/services-that-can-integrate-ram.html) ### Import the organization into Terraform using the `account` component After we are done with the above ClickOps and the Service Quota Increase for maximum number of accounts has been granted, we can then do the rest via Terraform. In the Geodesic shell, as SuperAdmin, execute the following command to get the AWS Organization ID that will be used to import the organization: ``` aws organizations describe-organization ``` From the output, identify the _organization-id_: ``` { "Organization": { "Id": "o-7qcakq6zxw", "Arn": "arn:aws:organizations:: ... ``` Using the example above, the _organization-id_ is o-7qcakq6zxw. In the Geodesic shell, as SuperAdmin, execute the following command to import the AWS Organization, changing the stack name `core-gbl-root` if needed, to reflect the stack where the organization management account is defined, and changing the last argument to reflect the _organization-id_ from the output of the previous command. ``` atmos terraform import account --stack core-gbl-root 'aws_organizations_organization.this[0]' 'o-7qcakq6zxw' ``` ### Provision AWS OUs and Accounts using the `account` component AWS accounts and organizational units are generated dynamically by the `terraform/account` component using the configuration in the `gbl-root` stack. :::info Special note In the rare case where you will need to be enabling non-default AWS Regions, temporarily comment out the `DenyRootAccountAccess` service control policy setting in `gbl-root.yaml`. You will restore it later, after enabling the optional Regions. See related: [Decide on Opting Into Non-default Regions](/layers/network/design-decisions/decide-on-opting-into-non-default-regions) ::: :::caution You must wait until your quota increase request has been granted. If you try to create the accounts before the quota increase is granted, you can expect to see failures like `ACCOUNT_NUMBER_LIMIT_EXCEEDED`. ::: In the Geodesic shell, execute the following commands to provision AWS Organizational Units and AWS accounts: ``` atmos terraform apply account --stack gbl-root ``` Review the Terraform plan, _**ensure that no new organization will be created**_ (look for `aws_organizations_organization.this[0]`), type "yes" to approve and apply. This creates the AWS organizational units and AWS accounts. ### Configure root account credentials for each account Note: unless you need to enable non-default AWS regions (see next step), this step can be done later or in parallel with other steps, for example while waiting for Terraform to create resources. **For** _**each**_ **new account:** 1. Perform a password reset by attempting to [log in to the AWS console](https://signin.aws.amazon.com/signin) as a "root user", using that account's email address, and then clicking the "Forgot password?" link. You will receive a password reset link via email, which should be forwarded to the shared Slack channel for automated messages. Click the link and enter a new password. (Use 1Password or [Random.org](https://www.random.org/passwords) to create a password 26-38 characters long, including at least 3 of each class of character: lower case, uppercase, digit, and symbol. You may need to manually combine or add to the generated password to ensure 3 symbols and digits are present.) Save the email address and generated password as web login credentials in 1Password. While you are at it, save the account number in a separate field. 2. Log in using the new password, choose "My Security Credentials" from the account dropdown menu and set up Multi-Factor Authentication (MFA) to use a Virtual MFA device. Save the MFA TOTP key in 1Password by using 1Password's TOTP field and built-in screen scanner. Also, save the Virtual MFA ARN (sometimes shown as "serial number"). 3. While logged in, enable optional regions as described in the next step, if needed. 4. (Optional, but highly recommended): [Unsubscribe](https://pages.awscloud.com/communication-preferences.html) the account's email address from all marketing emails. ### (Optional) Enable regions Most AWS regions are enabled by default. If you are using a region that is not enabled by default (such as Middle East/Bahrain), you need to take extra steps. 1. While logged in using root credentials (see the previous step), in the account dropdown menu, select "My Account" to get to the [Billing home page](https://console.aws.amazon.com/billing/home?#/account). 2. In the "AWS Regions" section, enable the regions you want to enable. 3. Go to the IAM [account settings page](https://console.aws.amazon.com/iam/home?#/account_settings) and edit the STS Global endpoint to create session tokens valid in all AWS regions. You will need to wait a few minutes for the regions to be enabled before you can proceed to the next step. Until they are enabled, you may get what look like AWS authentication or permissions errors. After enabling the regions in all accounts, re-enable the `DenyRootAccountAccess` service control policy setting in `gbl-root.yaml` and rerun ``` atmos terraform apply account --stack gbl-root ``` ### Update the `account-map` The `account-map` component collects and publishes information in the form of terraform outputs about the accounts that is used by other Terraform projects. It needs to be updated whenever accounts are added or removed. In the Geodesic shell, execute the following commands: ``` atmos terraform deploy account-map --stack gbl-root ``` One of the outputs of `account-map` is `full_account_map`. It lists the name and corresponding account number for each account. Save that somewhere, you will need it later. Learn more about the [account-map](/components/library/aws/account-map/) component. ### Provision `account-settings` component The `account-settings` component sets the default mode for EBS volumes to be encrypted, and provisions account aliases and password policies. In the Geodesic shell, execute the following command for each account, replacing `${account}` with the name of the account, e.g. `root` or `prod`: ``` atmos terraform apply account-settings --stack gbl-${account} ``` The `apply` gives you a chance to review the actions that will be taken before you `apply` them; type “yes” to actually make the changes or anything else to stop. If you do not want to review each change, you can use `deploy` instead, which skips the review stage. If you want to get everything done with a one-liner, you can use this: ``` for stack in ./stacks/gbl*yaml; do \ atmos terraform deploy account-settings --stack $(basename $stack .yaml); \ done ``` Learn more about the [account-settings](/components/library/aws/account-settings/) component. ### Provision `aws-saml` component GSuite integration requires collaboration between a GSuite administrator and a DevOps engineer with `root` account access. You should have previously installed the IdP metadata files and configured the component in `gbl-identity.yaml` . In the Geodesic shell, execute the following commands: ``` atmos terraform apply aws-saml --stack core-gbl-identity ``` In the output, make note of the SAML provider ARNs, as you will need them when provisioning user access in GSuite or Okta. :::info Security Note The following guidance provides clarity on the security of the metadata files. [https://security.stackexchange.com/questions/65743/saml-2-0-idp-metadata-security](https://security.stackexchange.com/questions/65743/saml-2-0-idp-metadata-security) **TL;DR** they are safe to commit to VCS. ::: ### Provision `aws-teams` component This component is responsible for provisioning all primary user and system roles into the centralized identity account. This is expected to be used alongside the [aws-team-roles](/components/library/aws/aws-team-roles/) component to provide fine-grained role delegation across the account hierarchy. This component was previously name `iam-primary-roles`, but was renamed as part of refactoring to better convey how it should be used. **NOTE:** [aws-saml](/components/library/aws/aws-saml/) project must be completed so the identity providers exist. If identity providers are added or deleted. This component requires `SuperAdmin` role to run. Use `assume-role` (or `assume-role-saml` for SAML auth) to assume the role. In the Geodesic shell, execute the following commands to provision the IAM roles in `identity`: ``` atmos terraform apply aws-teams --stack gbl-identity ``` This provisions the primary roles. If you get an error similar to the following, then either populate the arn for auto, or add an empty list to stacks/orgs/(namespace)/(tenant)/identity/global-region.yaml as shown further below ``` ata.aws_iam_policy_document.support_access_aggregated[0]: Reading... data.aws_iam_policy_document.support_access_aggregated[0]: Read complete after 0s [id=224330982] ╷ │ Error: Invalid function argument │ │ on ../account-map/modules/team-assume-role-policy/main.tf line 4, in locals: │ 4: allowed_principals = sort(distinct(concat(var.allowed_principal_arns, module.allowed_role_map.principals, module.allowed_role_map.permission_set_arn_like))) │ ├──────────────── │ │ while calling concat(seqs...) │ │ var.allowed_principal_arns is null │ │ Invalid value for "seqs" parameter: argument must not be null. ╵ ╷ │ Error: Invalid function argument │ │ on ../account-map/modules/team-assume-role-policy/main.tf line 24, in data "aws_arn" "allowed": │ 24: count = local.enabled ? length(var.allowed_principal_arns) : 0 │ ├──────────────── │ │ while calling length(value) │ │ var.allowed_principal_arns is null │ │ Invalid value for "value" parameter: argument must not be null. ╵ Releasing state lock. This may take a few moments... exit status 1 ``` If the account-id for auto (or whichever account runs spacelift workers) is known, edit stacks/orgs/(namespace)/(tenant)/identity/global-region.yaml and add the arn: ``` import: - orgs/acme/gov/iam/_defaults - mixins/region/global-region #... components: terraform: aws-teams: vars: teams_config: spacelift: trusted_role_arns: - "arn:aws:iam::(auto_account-id):role/(namespace)-(tenant)-(primary_region)-(auto_account_name)-spacelift-worker-pool-admin" ``` If the auto account id is not known, create an empty list instead: ``` import: - orgs/acme/gov/iam/_defaults - mixins/region/global-region #... components: terraform: aws-teams: vars: teams_config: spacelift: trusted_role_arns: [] ``` Save the file and re-run the atmos command for aws-teams for the global identity stack. ### Update `tfstate-backend` component If you provisioned the `tfstate-backend` component with `--var=access_roles_enabled=false`, now the component needs to be redeployed with the `access_roles` enable. Reapply the component without the extra settings: ``` atmos terraform apply tfstate-backend --stack core-uw2-root ``` Make note of the outputs, specifically the remote_state_backend S3 rw and ro `role_arn`s. These will be needed in a couple of steps. ### Provision `aws-team-roles` component The `aws-team-roles` component creates the delegated IAM roles in the member accounts allowing the `identity` account to assume roles into the member accounts. **NOTE:** [aws-saml](/components/library/aws/aws-saml/) must be completed so the identity providers exist. [aws-teams](/components/library/aws/aws-teams/) must be completed to generate the primary roles in the `identity` account. In the Geodesic shell, execute the following commands for each account except for `identity`: ``` atmos terraform apply aws-team-roles --stack ${tenant}-gbl-${stage} ``` ## Update the repo contents Our initial `git` check-in was missing some information. Now it is time to update it. - Update `stacks/globals.yaml`, setting the `backend` and `remote_state_backend` S3 `role_arn` to the read/write and read-only Terraform roles from `tfstate-backend` output. - (Obsolete, now handled automatically by `aws-config`) ~~Update~~ `rootfs/usr/local/bin/aws-accounts` ~~with the list of accounts from~~ `full_account_map` ~~above. Note that you not only need to update account names and numbers, you also need to update~~ `profile_order`, `region` (~~the default region abbreviation),~~ `role_prefix`, ~~and possibly other items.~~ - ~~After making those updates run~~ `aws-accounts gen-saml > aws-config-saml` ~~and, if needed~~ `aws-accounts gen-sso > aws-config-sso` ~~to create AWS~~ `config` ~~files and check them in under~~ `rootfs/etc/aws-config/` - Change directory to `rootfs/etc/aws-config` and update the config files: - `aws-config saml > aws-config-saml` - `aws-config spacelift > aws-config-spacelift` - `aws-config switch-roles > aws-switch-roles` - If still needed (may be phased out when `helmfile` is phased out), the `account_number` setting for each account, currently set in each region's regional stack file. - If still needed, any hard-coded account numbers or IAM ARNs in the stacks, such as for compliance If you have added IdP metadata files, add/check those into `git` under the appropriate component. Add/check into `git` newly created or updated config files under `rootfs/etc/aws-config` and newly created shell files under `components/terraform/account-map/account-info` and `components/terraform/aws-team-roles/iam-role-info`. --- ## Structure of Terraform S3 State Backend Bucket import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; This guide explains the structure of a Terraform S3 state backend bucket, including the use of workspaces, key prefixes, and buckets. It details how the `backend.tf.json` file is used to configure the S3 backend for storing Terraform state, and how DynamoDB is used for state locking and consistency checking. The document provides examples and best practices for managing and accessing the Terraform state backend. Understand the anatomy of a Terraform S3 state backend bucket and how workspaces, key prefixes and buckets are used. From HashiCorp > Stores the state as a given key in a given bucket on [Amazon S3](https://aws.amazon.com/s3/). This backend also > supports state locking and consistency checking via [Dynamo DB](https://aws.amazon.com/dynamodb/), which can be > enabled by setting the `dynamodb_table` field to an existing DynamoDB table name. A single DynamoDB table can be used > to lock multiple remote state files. Terraform generates key names that include the values of the `bucket` and `key` > variables. ### The `backend.tf.json` File This file is programmatically generated by [Atmos](/resources/legacy/fundamentals/atmos) using all the capabilities of [Stacks](/resources/legacy/fundamentals/stacks) to deep merge. Every component defines a `backend.tf.json`, which is what distinguishes it as a root module (as opposed to a terraform child module). The backend tells terraform where to access the last known deployed state of infrastructure for the given component. Since the backend is stored in S3, it’s easily accessed by in a distributed manner by anyone running terraform. :::info An identical `backend.tf.json` file is used by all environments (stacks). Environments are selected using the `terraform workspace` command, which happens automatically when using `atmos` together with the `--stack`argument. ::: For reference, this is the anatomy of the backend configuration: (note this is just a JSON representation of HCL) ``` { "terraform": { "backend": { "s3": { "acl": "bucket-owner-full-control", "bucket": "acme-ue2-root-tfstate", "dynamodb_table": "acme-ue2-root-tfstate-lock", "encrypt": true, "key": "terraform.tfstate", "profile": "acme-gbl-root-terraform", "region": "us-east-2", "workspace_key_prefix": "vpc" } } } } ``` :::note Either `profile` or `role_arn` can be used here ::: ### S3 Backend The S3 bucket is created in the cold start using the [tfstate-backend](/components/library/aws/tfstate-backend/) component provisioned in the root account. The state format is `s3://{bucket_name}/{component}/{stack}/terraform.tfstate` - The `bucket name` format is `{namespace}-{optional tenant}-{environment}-{stage}-tfstate` - We deploy this bucket in the `root` account so here are some example bucket names `acme-ue2-root-tfstate` (without tenant) `acme-mgmt-ue2-root-tfstate` (with `tenant: mgmt`) - The `component` name provided is used as the terraform state’s `workspace_key_prefix` in each component’s `backend.tf.json`. Therefore, this will be the first s3 key after the bucket name. - The `stack` is where the component is provisioned and the name of the workspace created - Finally, the `terraform.tfstate` is the `key` provided in each component’s `backend.tf.json` The terraform commands run by `atmos` for the backend `s3://acme-ue2-root-tfstate/vpc/ue2-prod/terraform.tfstate` ``` atmos terraform deploy vpc --stack ue2-prod | atmos will create the input variables from the YAML and run the following commands | -- terraform init | -- terraform workspace ue2-prod | -- terraform plan | -- terraform apply ``` To better visualize what’s going on, we recommend running the commands below to explore your own state bucket. Make sure to use the correct `profile` for your organization (`acme-gbl-root-admin` is just a placeholder). Find the bucket. It should contain `tfstate` in its name. In the example below, we can see the [vpc](/components/library/aws/vpc/) component is deployed to `use2-auto`, `use2-corp`, `use2-dev`, `use2-qa`, `use2-sbx01`, `use2-staging`. As you can see, the `workspace` is constructed as the `{environment}-{stage}`. This setting is defined in the `atmos.yaml` config with the `stacks.name_pattern` setting (see [Atmos](/resources/legacy/fundamentals/atmos) for all settings). ``` $ aws --profile acme-gbl-root-admin \ s3 ls --recursive ... 2021-11-01 19:53:48 120926 vpc/use2-auto/terraform.tfstate # workspace key prefix: vpc, workspace name is `use2-auto` 2021-11-01 19:49:12 123604 vpc/use2-corp/terraform.tfstate 2021-11-01 19:50:18 123486 vpc/use2-dev/terraform.tfstate 2021-11-01 19:48:39 123354 vpc/use2-qa/terraform.tfstate 2021-11-01 19:49:46 123735 vpc/use2-sbx01/terraform.tfstate 2021-11-01 19:50:50 124014 vpc/use2-staging/terraform.tfstate ``` See where all the VPC components contain state ``` aws --profile acme-gbl-root-admin \ s3 ls s3://{bucket_name}/vpc/ ``` :::note If a component is mistakenly deployed somewhere and destroyed, a leftover `terraform.tfstate` file will be present on your local filesystem with a small file size so while this is a good way to search for backends, it's not the best way to determine where a component is deployed. Also, the S3 bucket has versioning enabled, ensuring we can always (manually) revert to a previous state if need be. ::: ### DynamoDB Locking Find the table. It should contain `tfstate-lock` in its name. ``` aws --profile acme-gbl-root-admin \ dynamodb list-tables ``` Get a LockID ``` aws --profile acme-gbl-root-admin \ dynamodb get-item \ --table-name {table_name} \ --key '{"LockID": {"S": "{bucket_name}/{component}/{stack}/terraform.tfstate-md5"}}' ``` ### References - [https://www.terraform.io/docs/language/settings/backends/s3.html](https://www.terraform.io/docs/language/settings/backends/s3.html) backend configuration documentation. --- ## Atmos Pro import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; Atmos Pro is our advanced GitOps solution for Terraform and OpenTofu automation, designed to provide enterprise-grade workflow orchestration in GitHub Actions. It extends the capabilities of standard GitHub Actions with powerful features for managing infrastructure deployments, ensuring compliance, and maintaining infrastructure as code at scale. :::tip Atmos Pro is coming soon! Our team is actively developing Atmos Pro and its exciting new capabilities. In the meantime, explore the documentation below for a preview of what's ahead. We'll provide full details and launch timing as we approach the official release date. ::: ## The Problem While GitHub Actions provides a solid foundation for CI/CD, it lacks the specialized tooling needed for complex infrastructure deployments. Teams face several challenges: - **Limited Deployment Control**: GitHub Actions doesn't provide built-in mechanisms for ordered deployments or dependency management between infrastructure components - **Poor Visibility**: Understanding the impact of changes across your infrastructure requires manual investigation - **Insufficient Guardrails**: Basic GitHub Actions workflows don't enforce deployment policies or prevent dangerous operations - **Drift Management**: Keeping infrastructure in sync with code requires additional tooling and manual processes - **Multi-Cloud Complexity**: Managing deployments across different cloud providers adds another layer of complexity Traditional Terraform Automation and Collaboration Software (TACOS) solutions like Terraform Cloud, Spacelift, or Env0 attempt to solve these problems but often come with significant costs and vendor lock-in. ## Our Solution Atmos Pro enhances GitHub Actions with enterprise-grade features specifically designed for infrastructure deployment: * **Ordered Deployments**: Ensure infrastructure components are deployed in the correct sequence based on their dependencies * **Dependency Visualization**: Automatically generate and maintain dependency graphs for your infrastructure * **Drift Detection**: Continuously monitor and report on infrastructure drift with automated remediation options * **Enhanced Guardrails**: Implement policy controls and approval gates to prevent dangerous operations * **Native GitOps**: Leverage Git as the single source of truth with full audit trails and change history * **Beautiful Job Summaries**: Clear, actionable insights into deployment status and changes ## How It Works Atmos Pro integrates seamlessly with your GitOps workflow, providing automated infrastructure planning and deployment through two main processes: ### Pull Request Workflow When a developer creates a pull request, Atmos Pro automatically triggers the planning process: 1. **Developer Makes a Change**: Infrastructure code is modified in a feature branch 2. **Code Is Pushed**: Changes are pushed to the feature branch 3. **GitHub Actions Trigger**: Atmos affected stacks are identified 4. **Atmos Uploads**: Affected stacks information is uploaded 5. **Atmos Pro Dispatches**: Plan workflows are triggered for affected components 6. **Status Updates**: Atmos Pro maintains a status comment showing the progress of plans ### Merge Workflow When a pull request is merged, Atmos Pro automatically handles the deployment: 1. **Pull Request Is Merged**: Changes are merged into the main branch 2. **GitHub Actions Trigger**: Atmos affected stacks are identified 3. **Atmos Uploads**: Affected stacks information is uploaded 4. **Atmos Pro Dispatches**: Apply workflows are triggered for affected components 5. **Status Updates**: Atmos Pro maintains a status comment showing the progress of deployments ## References [Setup Documentation](/layers/atmos-pro/setup): Learn how to set up and configure Atmos Pro for your infrastructure, including prerequisites and initial configuration steps. [atmos-pro.com](https://www.atmos-pro.com): Explore the full documentation for Atmos Pro features and capabilities, including detailed guides and API references. [atmos.tools](https://www.atmos.tools): Learn about the core Atmos CLI and its features, which form the foundation for Atmos Pro's enhanced capabilities. --- ## Setup Atmos Pro import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CodeBlock from '@theme/CodeBlock'; import PartialAtmosPro from '@site/examples/snippets/.github/workflows/atmos-pro.yaml'; import PartialAtmosProTerraformPlan from '@site/examples/snippets/.github/workflows/atmos-pro-terraform-plan.yaml'; import PartialAtmosProTerraformApply from '@site/examples/snippets/.github/workflows/atmos-pro-terraform-apply.yaml'; Setting up Atmos Pro is straightforward: install the GitHub App, grant repository permissions, set up the workflows, and deploy the AWS infrastructure. This guide provides an overview of each step, with detailed instructions available in the linked pages. ## Setup Process ### Sign up for Atmos Pro The first step is to sign up for Atmos Pro. The sign up process includes creating a workspace in the Atmos Pro web console and installing the Atmos Pro GitHub App into your infrastructure repository. This will set up the connection between your repository and Atmos Pro. - Sign up for Atmos Pro - Create or join a workspace - Install the Atmos Pro GitHub App - Import your repositories For step-by-step instructions, see the [official Atmos Pro installation guide](https://atmos-pro.com/docs/install). ### Grant Repository Permissions The second step is to grant repository permissions in Atmos Pro to enable ordered deployments and other features. - Permission: `Affected Stacks Create` - Workflow: `*` - Branch: `*` - Environment: `*` For detailed instructions on repository permissions, see the [official Atmos Pro repository permissions guide](https://atmos-pro.com/docs/ordered-deployments/repository-permissions). ### Set Up Workflows The third step is to configure the workflows in your repository. This includes reviewing the generated workflows, setting up environment variables, and configuring branch protection rules. - Review the 3 GitHub Action workflows - Add the Workspace ID to GitHub repository variables - Merge the workflows into the default branch _The dispatched workflows need to exist in the default branch before they can be triggered!_ The following workflows should be added to your repository: This workflow is triggered by GitHub on pull request events (opened, synchronized, reopened) and when the PR is merged (closed). It uses the `atmos describe affected` command to identify affected components and upload them to Atmos Pro. {PartialAtmosPro} This workflow is dispatched by Atmos Pro to create Terraform plans for affected components. It is a reusable workflow that takes stack and component as inputs. {PartialAtmosProTerraformPlan} This workflow is dispatched by Atmos Pro to apply Terraform changes for affected components. It is a reusable workflow that takes stack and component as inputs. {PartialAtmosProTerraformApply} For additional workflow setup instructions, see the [official Atmos Pro workflow configuration guide](https://atmos-pro.com/docs/ordered-deployments/github-workflow-config). ### Deploy AWS Infrastructure Atmos Pro doesn't run Terraform or Atmos itself. It dispatches GitHub Actions that **you control**. To run Terraform in those GitHub Actions, you need to set up a few things in your cloud environment: - [x] **State Backend** (S3 + DynamoDB) to store Terraform state and enable state locking - [ ] **Plan File Storage** (S3 + DynamoDB) to persist Terraform plan outputs for review and approvals - [ ] **OIDC Integration** with GitHub for workflows to authenticate with your cloud provider If you've been following along with the reference architecture, by now you should already have the Terraform State Backend provisioned. To complete the requirements for Atmos Pro and GitHub Actions, continue with the step-by-step instructions: [Atmos and Terraform deployment](/layers/atmos-pro/tutorials/deploy-with-terraform) 1. Make sure you have admin access to both GitHub and AWS before starting the setup 2. Follow the detailed instructions in each linked guide for specific steps 3. Test the setup with a small change before deploying major infrastructure changes ## Verification After completing all four steps, you can verify the setup by: ### Test GitHub Integration - Create a new pull request with a small stack change - The Atmos Pro GitHub App will automatically comment on the PR - The comment will show the status of affected components - As workflows are dispatched for each component, the comment will automatically update ### Trigger a Plan - In the new pull request, change a value for any component. For example, add a tag to a S3 bucket. - The `atmos-pro.yaml` workflow will discover the newly affected stack and trigger Atmos Pro. - Atmos Pro will run Atmos Terraform Plan for the affected stack. - As the workflow is executed, Atmos Pro will update the comment on the PR with the plan status. ### Merge the PR - Now try merging the PR - Again, the `atmos-pro.yaml` workflow will discover the affected stacks and trigger Atmos Pro. - This time Atmos Pro will determine this is a "merged" event and run Atmos Terraform Apply. - Finally, Atmos Pro will update the comment on the PR will the apply status. --- ## Deploy Infrastructure with CloudFormation import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import Admonition from '@theme/Admonition'; Deploy the required AWS infrastructure for Atmos Pro with just a few clicks using CloudFormation. This approach provides a quick and straightforward way to set up all necessary resources including state backend, plan file storage, and GitHub OIDC integration. - Deploy complete Terraform backend infrastructure in a single CloudFormation stack - Set up S3 buckets for state and plan file storage - Configure DynamoDB tables for state locking and plan file management - Create GitHub OIDC integration for secure authentication - Configure Atmos Pro to use the deployed infrastructure ## Overview Atmos Pro doesn't run Terraform or Atmos itself. It dispatches GitHub Actions that **you control**. To run Terraform in those GitHub Actions, you need to set up a few things in your cloud environment: - **State Backend** (S3 + DynamoDB) to store Terraform state and enable state locking - **Plan File Storage** (S3 + DynamoDB) to persist Terraform plan outputs for review and approvals - **OIDC Integration** with GitHub for workflows to authenticate with your cloud provider To make things easier, we've provided a CloudFormation template that sets up everything for you. ## Deployment Steps ### Authenticate with AWS - Sign in to your AWS account - Ensure you have administrator access - Choose your deployment region (we recommend `us-east-1`) ### Deploy Infrastructure - Click the "Deploy to AWS" button below - Review the CloudFormation template parameters - Click "Create stack" to deploy Your stack name must be unique across all AWS accounts. We use the stack name as part of the S3 bucket and DynamoDB table IDs. [![Launch Stack](https://s3.amazonaws.com/cloudformation-examples/cloudformation-launch-stack.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/new?stackName=my-terraform-backend&templateURL=https://s3.amazonaws.com/cplive-core-ue2-public-cloudformation/aws-cloudformation-terraform-backend.yaml) Or manually deploy the template with the AWS CLI: ```bash aws cloudformation deploy \ --stack-name my-backend \ --template-url https://s3.amazonaws.com/cplive-core-ue2-public-cloudformation/aws-cloudformation-terraform-backend.yaml \ --capabilities CAPABILITY_NAMED_IAM \ --no-fail-on-empty-changeset \ --parameter-overrides GitHubOrg=my-org ``` ### Configure Atmos Pro Once deployed, you will need to add the new role and plan file storage configuration to your Atmos configuration. #### GitHub Integration Configuration ```yaml integrations: github: gitops: opentofu-version: "1.10.0" artifact-storage: region: "us-east-1" # Ensure this matches the region where the template was deployed bucket: "my-backend-tfplan" # Get this value from the PlanBucketName output table: "my-backend-tfplan" # Get this value from the PlanDynamoDBTableName output role: "arn:aws:iam::123456789012:role/my-backend-github-actions" # Get this value from the GitHubActionsRoleARN output role: plan: "arn:aws:iam::123456789012:role/my-backend-github-actions" # Get this value from the GitHubActionsRoleARN output apply: "arn:aws:iam::123456789012:role/my-backend-github-actions" # Get this value from the GitHubActionsRoleARN output ``` #### State Backend Configuration Then use the state backend with Atmos by specifying the S3 bucket and DynamoDB table: ```yaml terraform: backend_type: s3 backend: s3: bucket: my-backend-tfstate # Get this value from the StateBucketName output dynamodb_table: my-backend-tfstate # Get this value from the StateDynamoDBTableName output role_arn: null # Set to null to use the current AWS credentials encrypt: true key: terraform.tfstate acl: bucket-owner-full-control region: us-east-1 # Ensure this matches the region where the template was deployed remote_state_backend: s3: role_arn: null # Set to null to use the current AWS credentials ``` ## CloudFormation Parameters | Parameter | Description | Default | | ----------------------- | ------------------------------------------------------------------------------------------------ | ------- | | `CreateStateBackend` | Set to 'true' to create state backend resources (S3 bucket, DynamoDB table), 'false' to skip | true | | `CreatePlanFileStorage` | Set to 'true' to create plan file storage resources (S3 bucket, DynamoDB table), 'false' to skip | true | | `CreateGitHubAccess` | Set to 'true' to create GitHub access resources (OIDC provider, IAM role), 'false' to skip | true | | `CreateOIDCProvider` | Set to 'true' to create the GitHub OIDC provider, 'false' to skip (if it already exists) | true | | `GitHubOrg` | GitHub organization or username | | | `GitHubRepo` | GitHub repository name. Set to `*` to allow all repositories | \* | ## Review Congratulations! The CloudFormation stack has now deployed: - An IAM role configured with trusted relationships for GitHub Actions - An S3 bucket to store Terraform state files - A DynamoDB table for state locking - An S3 bucket to store Terraform plan files - A DynamoDB table for managing those plan files - GitHub OIDC provider for secure authentication You're now ready to start using Atmos Pro with GitHub Actions. ## Cleanup To destroy the template, run: ```bash aws cloudformation delete-stack --stack-name my-backend ``` This will destroy the stack and all the resources it created. However, if the S3 bucket is not empty, the stack will fail to destroy. To destroy the stack and empty the S3 bucket, run: ```bash aws cloudformation delete-stack --stack-name my-backend --deletion-mode FORCE_DELETE_STACK ``` This will destroy the state files and empty the S3 bucket. This is a destructive action and cannot be undone. --- ## Deploy with Atmos and Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import Admonition from '@theme/Admonition'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import CodeBlock from '@theme/CodeBlock'; Verify and complete the AWS infrastructure setup for Atmos Pro using Atmos and Terraform. This approach checks your existing backend infrastructure and deploys the additional resources needed for plan file storage and GitHub OIDC integration. - Verify existing Terraform backend infrastructure (S3 + DynamoDB) - Deploy new S3 bucket and DynamoDB table for plan file storage - Ensure GitHub OIDC integration is properly configured - Create IAM roles for GitHub Actions authentication ## Overview This deployment method verifies your existing backend infrastructure (which should already be deployed as part of the reference architecture) and deploys the additional resources needed for plan file storage and GitHub OIDC integration. ## Quick Start | Steps | | | :-------- | :------------------------------------- | | 1. Vendor | `atmos workflow vendor -f quickstart/foundation/gitops` | | 2. Deploy | `atmos workflow deploy -f quickstart/foundation/gitops` | Currently, the workflows use the terminology "gitops". In the future, we plan to replace this with "atmos-pro". ## Requirements ### Authentication Prerequisites The GitHub Action workflows expect the `gitops` AWS Team to be properly setup and connected to GitHub OIDC. This component should already be deployed with `aws-teams`/`aws-team-roles` and `github-oidc-provider` respectively. Verify the following to complete the authentication prerequisites. By default in the Reference Architecture, the `trusted_github_repos` input is commented out for `aws-teams`. Now is the time to uncomment those lines. Please see `stacks/catalog/aws-teams.yaml` - The `gitops` Team is defined and deployed by `aws-teams` - The team has trusted relationships with the infrastructure repo via `trusted_github_repos` _Capitalization matters!_ In the reference architecture, these values are initially commented out and will need to be updated with your specific repository information: ```yaml components: terraform: aws-teams: vars: trusted_github_repos: gitops: - "acme/infra:main" ``` - The `aws-team-roles` default catalog allows the `gitops` team to assume the `terraform` role - `github-oidc-provider` is deployed to the account where Atmos Pro infrastructure will be created - The workflows have adequate permission In order to assume GitHub OIDC roles, a workflow needs the following: ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` ## How To Setup ### Vendor Components The `gitops` stacks depends on components that may already exist in your component library (`s3-bucket` and `dynamodb`) and adds new components to manage the GitHub OIDC access. Vendor these components either with the included Atmos Workflows or using [Atmos Vendoring](https://atmos.tools/core-concepts/components/vendoring). ### Deploy Atmos Pro Infrastructure Deploy the Atmos Pro infrastructure components with the following workflow: ## Review Congratulations! The Atmos components have now verified and deployed: - Verified existing Terraform backend infrastructure (S3 bucket and DynamoDB table for state) - Deployed new S3 bucket to store Terraform plan files - Deployed new DynamoDB table for managing plan files - Ensured GitHub OIDC provider is properly configured - Created IAM roles for GitHub Actions authentication You're now ready to start using Atmos Pro with GitHub Actions. --- ## Tutorials(Tutorials) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These tutorials will help you deploy and configure the required AWS infrastructure for Atmos Pro. Choose the deployment method that best fits your needs. --- ## Prepare Container Registry import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Now that the GitHub OIDC Provider has been deployed, we can proceed with setting up the necessary prerequisites for containers. The first prerequisite is deploying Amazon Elastic Container Registry (ECR) repositories that will be used to store container images built by GitHub Actions workflows. | Steps | Actions | | -------------------------- | ----------------------------------- | | Deploy ECR repositories | `atmos workflow deploy/ecr -f quickstart/foundation/baseline` | ## Deploy ECR Repositories Deploy the ECR repositories that will be used by GitHub Actions workflows: We use ECR for two main purposes: 1. Storing the Geodesic base image that provides the development environment and tooling 2. Storing container images built during CI steps of application release workflows --- ## Provision Databases import Intro from '@site/src/components/Intro'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; There are many options for provisioning databases in AWS. The reference architecture provides support for a variety of databases, including Aurora PostgreSQL, Aurora MySQL, DynamoDB, RDS, ElastiCache, DocumentDB, and more. Each database has its own unique features and benefits, so you can choose the one that best fits your needs or deploy multiple databases to support different use cases. ### SQL Database Options The reference architecture supports several SQL database options, including Aurora PostgreSQL, Aurora MySQL, RDS, and more. Aurora PostgreSQL is a fully managed, PostgreSQL-compatible relational database service that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora PostgreSQL is a good choice for applications that require the features and capabilities of PostgreSQL with the scalability, performance, and reliability of a managed service. Get Started Aurora MySQL is a fully managed, MySQL-compatible relational database service that combines the performance and availability of high-end commercial databases with the simplicity and cost-effectiveness of open-source databases. Aurora MySQL is a good choice for applications that require the features and capabilities of MySQL with the scalability, performance, and reliability of a managed service. Get Started Amazon RDS is a managed relational database service that supports multiple database engines, including MySQL, PostgreSQL, MariaDB, Oracle, and SQL Server. RDS is a good choice for applications that require the features and capabilities of a relational database with the scalability, performance, and reliability of a managed service. Get Started Amazon Redshift is a fully managed, petabyte-scale data warehouse service that provides fast query performance using SQL and business intelligence tools. Redshift is a good choice for applications that require high-performance analytics and data warehousing with built-in security, backup, and restore capabilities. Get Started ### NoSQL Database Options The reference architecture also supports several NoSQL database options, including DynamoDB, DocumentDB, Elasticache Redis, and more. SQL and/or NoSQL databases can always be deployed side by side to support different use cases. DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. DynamoDB is a good choice for applications that require low-latency, high-throughput access to data with built-in security, backup, and restore capabilities. Get Started Amazon DocumentDB is a fully managed, MongoDB-compatible document database service that provides fast and scalable performance with built-in security, backup, and restore capabilities. DocumentDB is a good choice for applications that require the features and capabilities of MongoDB with the scalability, performance, and reliability of a managed service. Get Started Amazon ElastiCache is a fully managed in-memory data store service that supports Redis and Memcached. ElastiCache Redis is a good choice for applications that require low-latency, high-throughput access to data with built-in security, backup, and restore capabilities. Get Started --- ## Decide on Amazon Managed Workflows for Apache Airflow (MWAA) Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Requirements for MWAA environments deployed to each active compute environment must be outlined before an MWAA environment is configured and deployed. Customers will likely require integrations with other systems like Redshift or S3. There’s no generic way of handling all these cases, so each will need to be handled separately. Each case may require additional resources like IAM roles to be provisioned, which we cannot anticipate and will rely entirely on information supplied by the customer. ## Context Amazon MWAA environments will be used by applications that use Apache Airflow. ## Considered Options Create a standardized MWAA Environment based on requirements. ### Integrations - What integrations are required with other systems? - e.g. S3 will require IAM roles be provisioned - e.g. RDS will require database users, grants and security groups be opened up - Have those other systems already been deployed? - Should we provide an example? - How will DAGs be managed in S3? #### Standardized Managed Workflows for Apache Airflow (MWAA) Configuration Settings - Number of workers - Min number of workers - Max number of workers - Webserver access mode - Can be one of: `PUBLIC_ONLY`, `PRIVATE_ONLY`. Defaults to `PRIVATE_ONLY`. - If it’s private, how will you intend to access it? e.g. we’ll need something like [Decide on Client VPN Options](/layers/network/design-decisions/decide-on-client-vpn-options) - Environment class - Can be one of: `mw1.small`, `mw1.medium`, `mw1.large` - Airflow version - Supported versions outlined here: [https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html](https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html) - If not specified, the latest available version will be used. The latest available version of Apache Airflow will be used unless a previous minor version must be used to provide compatibility with an application environment. This provides the latest bug fixes and security patches for Apache Airflow, which is especially important if the webserver access mode is set to `PUBLIC_ONLY`. If an older minor version must be used to provide compatibility with an application environment, then the latest available patch version should be used to include all possible bug fixes and security patches. - Use custom `plugins.zip` file? - If so, where are those plugins stored? - What is generating the plugin's artifact? CI/CD for this artifact could be out of scope. - Use custom `requirements.txt` file? - If so, we’ll need the customer to provide this file. - DAG processing logs - From least to most verbose: disabled, `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`. Defaults to `INFO`. - Scheduler logs - From least to most verbose: disabled, `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`. Defaults to `INFO`. - Task logs - From least to most verbose: disabled, `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`. Defaults to `INFO`. - Webserver logs - From least to most verbose: disabled, `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`. Defaults to `INFO`. - Worker logs - From least to most verbose: disabled, `CRITICAL`, `ERROR`, `WARNING`, `INFO`, `DEBUG`. Defaults to `INFO`. ## References - [Amazon Managed Workflows for Apache Airflow (MWAA): Create an Environment](https://docs.aws.amazon.com/mwaa/latest/userguide/create-environment.html) - [Amazon Managed Workflows for Apache Airflow (MWAA): Supported Versions](https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html) --- ## Decide on Amazon OpenSearch Service (Elasticsearch) Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Requirements for the Amazon OpenSearch Service ([formerly known as Elasticsearch](https://aws.amazon.com/blogs/aws/amazon-elasticsearch-service-is-now-amazon-opensearch-service-and-supports-opensearch-10/)) clusters deployed to each active compute environment need to be outlined before an Elasticsearch component is configured and deployed ## Context At a minimum, we need the following for each operating stage (prod, staging, dev, etc) - Instance family for each node - Number of nodes - EBS volume size - Whether or not Kibana is required See [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html) for Amazon’s recommendations. ## Considered Options Create a standardized Elasticsearch (Amazon OpenSearch Service) cluster based on one of these options. We’ll also need to know how these requirements will vary by stage. :::caution This is a reversible decision, however, resizing large OpenSearch clusters can literally take several days. ::: ### Option 1: Use Current Infrastructure Requirements :::info If already opening OpenSearch, we recommend sharing a screenshot of your current setup from the AWS web console for each cluster in every environment. ::: ### Option 2: Use Minimal Elasticsearch (Amazon OpenSearch) Cluster Requirements Because the [Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html) recommends having at least 3 nodes in each Elasticsearch cluster in order to avoid a split-brain scenario, each cluster should contain 3 nodes (if it were to be minimally sized). This, in addition to the requirements outlined in _v1 Infrastructure Requirements_, concludes that each Elasticsearch cluster will have the following requirements: | **Requirement** | **Recommendation** | | | ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------ | | EBS volume size | :::cautionThe volume size is limited by the size of the instance. [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/limits.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/limits.html) ::: | | | Number of nodes | 3 | | | Instance family for each node | Depends on use-case | | | Kibana | Whether or not Kibana is required: not required. :::cautionIf Kibana is required, we’ll need to discuss how to securely access Kibana. We recommend SAML authentication. [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html) ::: | | ## Consequences Provision Amazon OpenSearch Service based on these requirements using the `elasticsearch` component with terraform. - This allows for a standardized Elasticsearch cluster that satisfies the requirements required by the application stack in each active compute environment. - This standard size can be easily adjusted as needed, so this is an easily reversible decision. ## References - [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/sizing-domains.html) - [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/limits.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/limits.html) - [https://docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html) - [https://aws.amazon.com/blogs/aws/amazon-elasticsearch-service-is-now-amazon-opensearch-service-and-supports-opensearch-10/](https://aws.amazon.com/blogs/aws/amazon-elasticsearch-service-is-now-amazon-opensearch-service-and-supports-opensearch-10/) --- ## Decide on Automated Backup Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options ### Option 1 (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - #### Cons - ### Option 2 #### Pros - #### Cons - ### Option 3 #### Pros - #### Cons - ## References - Links to any research, ADRs or related Jiras --- ## Decide on AWS Backup Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem ## Context We need a standardized way to implement backup services for AWS resources (S3, databases, EC2 instances, EFS, etc. etc.) to have the ability to restore data from points in time in the event of data loss or corruption. AWS provides a managed backup service offering called AWS Backup. [https://docs.aws.amazon.com/aws-backup/latest/devguide/whatisbackup.html](https://docs.aws.amazon.com/aws-backup/latest/devguide/whatisbackup.html) We need to determine if we are opting in or opting out using AWS Backup. ## References - [https://www.druva.com/blog/understanding-rpo-and-rto/](https://www.druva.com/blog/understanding-rpo-and-rto/) --- ## Decide on AWS EMR Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We need to document the requirements for the EMR cluster. ## Context If EMR is presently deployed, the best course of action is to replicate the settings you have (share these details if that’s the case). ## Considered Options A list of applications for the cluster. Currently supported options are: Flink, Ganglia, Hadoop, HBase, HCatalog, Hive, Hue, JupyterHub, Livy, Mahout, MXNet, Oozie, Phoenix, Pig, Presto, Spark, Sqoop, TensorFlow, Tez, Zeppelin, and ZooKeeper (as of EMR 5.25.0). For a full list of supported options, review the EMR module. ## References - [https://github.com/cloudposse/terraform-aws-emr-cluster#inputs](https://github.com/cloudposse/terraform-aws-emr-cluster#inputs) --- ## Decide on AWS Managed RabbitMQ Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options ### Option 1 (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - #### Cons - ### Option 2 #### Pros - #### Cons - ### Option 3 #### Pros - #### Cons - ## References - Links to any research, ADRs or related Jiras --- ## Decide on Database Schema Migration Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ### Problem and Context We must decided how and when to run database migrations. The strategy will depend on several factors, including whether or not any automated tools are already in place to handle these migrations and the platform (e.g. ECS or EKS). ## Considerations - During rolling application deployments, there will be a period when 2 versions of your application are live. - Migrations that happen before the rolling update, mean that the previous version of the application will be forced to use the new schema - Migrations that happen after the rolling update, mean that the next version of the application will be forced to use the previous version of the schema - Adjacent releases of the application must be backward compatible with schemas. - **Never delete columns (or rows), only add columns** ### Questions - Should migrations happen before or after the application rollout? - What should happen during application rollbacks? - Does the schema get rolled back or stay ahead? - What software are you using to handle migrations? - What happens if pods or nodes are scaling during a database migration? (e.g. old versions of pods can come up) ## Options **Option 1:** Implement migrations as part of entry point initialization of docker container - If you have 25 containers running, each one will attempt to obtain a lock (if you’re lucky) and perform the migration. Many customers don’t like that each container attempts this and prefer it to happen before or after rollout. - At any given point, different versions of the app will be using different versions of the schema (e.g. rolling updates) - Long migrations will cause health checks to fail and the pods will get restarted aborting the migrations. Extending the timeouts for health checks means slowing down recovery for legitimate failures. **Option 2:** Implement migrations as part of the CD workflow - This is nice because we can control when it happens in the release process - It’s complicated when doing asynchronous deployments with ArgoCD or spacelift. Since deployments are happening outside of the GitHub action workflow, we don’t know when steps are completed. **Option 3:** Implement Manually triggered Workflows (E.g. via GitHub Action workflow dispatches) - You have full control over when it runs, but it’s not automated in relation to your workflows. The actual execution is automated. **Option 4:** Implement Kubernetes Job or ECS Task - Easy to implement - Works well, when it works. When it fails, it’s hard to regulate what happens with the services. - Kubernetes will keep re-attempting the migration if the job exit’s non-zero. If we squash the exit code, then we don’t realize it’s failing, unless there’s other monitoring in place. --- ## Decide on DocumentDB Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Requirements for DocumentDB clusters deployed to each active compute environment need to be outlined before a DocumentDB component is configured and deployed: - Instance family for DB instances - Database Engine - Whether or not to create DB replica instances - Instance family for DB replica instances - Backup retention period ## Considered Options Create a standardized DocumentDB cluster based on current use case: ### v1 Infrastructure Requirements In production, a single `[CHANGE ME]` instance is used. Read-replicas are not enabled. The primary database engine in DocumentDB clusters is [CHANGE ME] The backup retention period for DocumentDB clusters is one day for non-production environments (the minimum retention period), and 35 days for production environments (the maximum retention period). This allows for a point-in-time-restore backup that can be rolled back for the duration of the retention period. ### Standardized DocumentDB Cluster Requirements The [Amazon DocumentDB Service](https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html) recommends deploying at least one additional DocumentDB instance in a different availability zones to ensure High Availability. This instance is automatically designated a read-replica by DocumentDB. During a disaster scenario when the primary instance becomes unavailable, DocumentDB automatically designates one of the other instances as the primary instance. The primary AWS region [uses three availability zones](/layers/network/design-decisions/decide-on-primary-aws-region), therefore it is recommended that DocumentDB is deployed across three availability zones when possible. The [Amazon DocumentDB Service](https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html) recommends that read replicas are of the same instance family as the primary instance: > For consistency, these replica instances should be the same instance family, and should be left to be designated as > replica instances by the DocumentDB service rather than manually designated, in order to simplify management of the > infrastructure. > This, in addition to the requirements outlined in _v1 Infrastructure Requirements_, concludes that each DocumentDB > cluster will have the following requirements: - Instance family for DB instances: `[CHANGE ME]` in non-production environments, `[CHANGE ME]` in production environments - Database Engine: [CHANGE ME] - Whether or not to create DB replica instances: yes, ideally create 3 (one in each of the 3 Availability Zones) - Instance family for DB replica instances: `[CHANGE ME]` in non-production environments, `[CHANGE ME]` in production environments - Backup retention period: 1 day in non-production environments (the minimum retention period), 35 days in production environments (the maximum retention period) ## Decision Outcome Chosen option: "Create a standardized DocumentDB cluster based on current use case", because - This allows for a standardized DocumentDB cluster that satisfies the requirements required by the application stack in each active compute environment. ## Consequences Create a DocumentDB component and tune it to the requirements outlined above. ### References - [https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html](https://docs.aws.amazon.com/documentdb/latest/developerguide/replication.html) - [Primary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) --- ## Decide on DynamoDB Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem **DRAFT** Requirements for DynamoDB tables deployed to each active compute environment need to be outlined before a DynamoDB component is configured and deployed. ## Context We need to at a minimum define the following requirements: - Read/Write capacity Mode (and related settings) - Integrated backup settings: - Point-in-time-recovery (PITR) or on-demand - AWS Backup settings: - Backup frequency - Backup lifecycle - TTL (whether or not to enable) ## Considered Options Create a standardized DynamoDB Table based on the current use case: ### v1 Infrastructure Requirements Currently, DynamoDB tables are used within [CHANGE ME]. DynamoDB tables use `PAY_PER_REQUEST` billing instead of `PROVISIONED` billing because: - The DynamoDB is part of a data pipeline where the table IO operations are not entirely predictable, therefore realized throughput may be significantly below or above the provisioned throughput throughout the day. - The DynamoDB table IO operations are also not entirely consistent, therefore a provisioned capacity table whose realized throughput closely meets its provisioned throughput cover a period of one day may not do so the next day. - If the provisioned capacity tables surpass their provisioned throughput, throttling will occur, unless auto-scaling is implemented for the DynamoDB table. This involves more machinery and is only warranted for DynamoDB tables whose realized throughput is very close to their provisioned throughput, and which need to be able to handle unpredictable spikes from time-to-time. This is not cost-effective for a table whose realized throughput does not meet its provisioned throughput consistently in the first place. - Due to the reasons described above, it is more cost-effective to [CHANGE ME: PAY_PER_REQUEST OR PROVISIONED] Backups (both integrated and via AWS Backup) are configured for DynamoDB as follows: - Integrated Backup type: in production, the DynamoDB tables have integrated point-in-time-recovery (PITR) backups which allow for a to-the-second recovery. The retention period is the maximum PITR retention period, which is 35 days. For non-production environments, PITR is disabled. Integrated on-demand backups are not used, because AWS Backup performs the same function. Enabling integrated PITR backups alongside AWS Backup allows for the "best of both worlds" for DynamoDB backups — that is, the ability to restore to the second for the past 35 days, and to have periodic snapshots for a long period of time. - AWS Backup is disabled for non-production environments. - AWS Backup frequency: the DynamoDB tables are backed up once a month. For simplicity, this is the first day of the month. - AWS Backup lifecycle: the DynamoDB tables backups are transitioned to cold storage after some time and are eventually deleted. The backup is moved to cold storage after [CHANGE ME] days, and is deleted after [CHANGE ME] days. This leaves a short period in the month to restore the table from warm storage, and exactly 3 months in cold storage to recover the table. [TALK ABOUT WHETHER TTL IS GOING TO BE USED] ### Standardized DynamoDB Table Requirements The requirements outlined by _v1 Infrastructure Requirements_ are sufficiently comprehensive to be standardized in the v2 infrastructure: - Read/Write capacity Mode: `[CHANGE ME]` - PointInTimeRecoveryEnabled: `true` for production, `false` for non-production environments - AWS Backup settings (disabled for non-production environments): - BackupCronExpression: `[CHANGE ME]` - DeleteAfterDays: `[CHANGE ME]` - MoveToColdStorageAfterDays: `[CHANGE ME]` - TimeToLiveSpecification: - Enabled:`false` ## Consequences Create a DynamoDB component and tune it to the outlined requirements. ## References - [https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadWriteCapacityMode.html) --- ## Decide on Elasticache Redis Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We need to define the use-cases and outline soft requirements for Elasticache Redis. ## Context Amazon ElastiCache for Redis is Amazon’s fully managed version of Open Source [https://redis.io/](https://redis.io/) in-memory data store that provides sub-millisecond latency and powers some of the largest websites out there. Any applications you have that depend on Redis can work seamlessly with ElastiCache Redis without any code changes. See [https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/nodes-select-size.html](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/nodes-select-size.html) for Amazon’s recommendations on right-sizing clusters. ## Considered Options :::info Ideally, share a screenshot of any existing Elasticache redis requirements and we can provision accordingly. ::: | **Requirement** | **Recommendation** | **Description** | | --------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Cache Engine | Redis | | | Instance family | | | | Encryption in Transit | No | By default, we don’t require this as :::infoNative TLS was not supported prior to open source Redis version 6.0. As a result, not every Redis client library supports TLS.[https://redis.io/topics/encryption](https://redis.io/topics/encryption) ::: | | Encryption at Rest | | | | Security Group Restrictions | 10.0.0.0/0 | | | Automated failover | yes | | | Auto minor upgrade | N/A | Auto Minor Upgrade is only supported for engine type `"redis"` and if the engine version is 6 or higher.(See: [https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/VersionManagement.html](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/VersionManagement.html) ) | | Multi-az enabled | | deployed across 2 AZs (private subnets) | | Number of nodes | | | | Cluster Mode Enabled | | | | AWS Backup requirements | | [https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups.html](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/backups.html) | Additional options - [aws_elasticache_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/elasticache_cluster) - [aws_elasticache_replication_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/elasticache_replication_group) ## Consequences - We’ll provision Elasticache Redis using our `elasticache-redis` component. - Define the catalog entries for the various Redis configurations - Enable AWS backups, as necessary ## References - [https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.html](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/WhatIs.html) - [https://aws.amazon.com/blogs/database/five-workload-characteristics-to-consider-when-right-sizing-amazon-elasticache-redis-clusters/](https://aws.amazon.com/blogs/database/five-workload-characteristics-to-consider-when-right-sizing-amazon-elasticache-redis-clusters/) - [https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/nodes-select-size.html](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/nodes-select-size.html) - --- ## Decide on MSK Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Requirements for MSK clusters deployed to each active compute environment need to be outlined before an MSK component is configured and deployed. ## Context Amazon MSK clusters are going to be used by applications that use Apache Kafka streams. ## Considered Options ### Create a standardized MSK cluster based on requirements. - Number of AZs - Apache Kafka version - Number of broker nodes - Broker instance type - Enhanced monitoring enabled? - Node and JMX Prometheus exporters enabled? - Broker volume size - S3 broker logging enabled? - CloudWatch Logs broker logging enabled? - CloudWatch Logs retention period (if CloudWatch Logs broker logging is enabled) - Kinesis Firehose broker logging enabled? - Authentication - Defaults to Mutual TLS authentication disabled - Private CA ARN can be used for mutual TLS authentication if that’s required. This can be created per account or in a single account and shared across accounts with AWS RAM. - Defaults to IAM (Client SASL IAM) authentication disabled. - Encryption at rest defaults to using amazon-managed “aws/msk” kms key. - MSK properties - Auto create topics ? - `auto.create.topics.enable` (this is explicitly set to false by default) - If auto creating topics is not required but topic creation is required, there is a separate component for it where topics can be explicitly created. - Allow deleting topics ? - `delete.topic.enable` (since kafka 1.0.0, this has defaulted to true but is not in the msk default) [Amazon provides an Excel spreadsheet](https://amazonmsk.s3.amazonaws.com/MSK_Sizing_Pricing.xlsx) to help make these calculations. [msk_sizing_pricing.xlsx](/assets/refarch/msk_sizing_pricing.xlsx) #### Standardized Amazon MSK Cluster Requirements The [Amazon-recommended Apache Kafka version](https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html#2.6.2) (`2.6.2`) will be used in favor of `2.6.0`, because the minor semantic version difference is not expected to cause any compatibility issues, and contains bug fixes and remediations to CVEs (see: [Apache Kafka 2.6.2 release notes](https://downloads.apache.org/kafka/2.6.2/RELEASE_NOTES.html)). As a best practice, CloudWatch Logs broker logging should be enabled in order to have the ability to debug Apache Kafka issues when they arise. (Amazon MSK will log `info` level logs. See: [Apache Kafka log levels](https://httpd.apache.org/docs/2.4/mod/core.html#loglevel).) The retention period for these logs should be long enough for debugging in non-production environments (e.g. 60 days), and even longer in production environments in order to be able to debug issues that may be impacting or have impacted users in the past (e.g. 365 days). [LIST ATTRIBUTES FROM _Create a standardized MSK cluster based on requirements_ AND FILL THEM IN] ## References - [Amazon MSK: Supported Apache Kafka versions](https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html) - [Amazon MSK: Logging](https://docs.aws.amazon.com/msk/latest/developerguide/msk-logging.html) - [Amazon MSK now supports the ability to change the size or family of your Apache Kafka brokers](https://aws.amazon.com/about-aws/whats-new/2021/01/amazon-msk-now-supports-the-ability-to-change-the-size-or-family/) - [https://docs.aws.amazon.com/msk/latest/developerguide/msk-encryption.html](https://docs.aws.amazon.com/msk/latest/developerguide/msk-encryption.html) - [https://docs.aws.amazon.com/msk/latest/developerguide/msk-default-configuration.html](https://docs.aws.amazon.com/msk/latest/developerguide/msk-default-configuration.html) --- ## Decide on RDS Aurora DB Cluster Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Requirements for Amazon Aurora DB clusters deployed to each active compute environment need to be outlined before an Amazon Aurora component is configured and deployed ## Context Amazon RDS provides MySQL and PostgreSQL-compatible relational databases that are built for the cloud with greater performance and availability at 1/10th the cost of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases. RDS Aurora features a distributed, fault-tolerant, self-healing storage system that auto-scales up to 128TB per database instance. It delivers high performance and availability with up to 15 low-latency read replicas, point-in-time recovery, continuous backup to Amazon S3, and replication across three Availability Zones. Amazon Aurora DB clusters (See: [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture)) ### Known Limitations - [Max of 15 Read Replicas](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Replication.html#:~:text=An%20Aurora%20DB%20cluster%20can%20contain%20up%20to%2015%20Aurora%20Replicas.%20The%20Aurora%20Replicas%20can%20be%20distributed%20across%20the%20Availability%20Zones%20that%20a%20DB%20cluster%20spans%20within%20an%20AWS%20Region.) (we had a customer decline RDS Aurora based on this limitation) - ~~Point-in-time recovery (PITR) is not yet supported~~ RDS Aurora now supports PITR. [https://aws.amazon.com/blogs/storage/point-in-time-recovery-and-continuous-backup-for-amazon-rds-with-aws-backup/](https://aws.amazon.com/blogs/storage/point-in-time-recovery-and-continuous-backup-for-amazon-rds-with-aws-backup/) - Cannot be launched on public subnets ## Considered Options Create a standardized Aurora DB cluster based on the current use case: ### Current Infrastructure Requirements ### RDS Aurora Replication RDS aurora replication happens at the filesystem layer versus the conventional database layer. It’s actually a shared filesystem. Hitting the read replicas hard can still impact the masters since they are using the shared filesystem. > Because the cluster volume is shared among all DB instances in your DB cluster, minimal additional work is required to replicate a copy of the data for each Aurora Replica. > [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Replication.html](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Replication.html) ### RDS Serverless v1 vs v2 Using serverless could be more costly that using regular RDS aurora due to not having enough options for CPU. Serverless v1 offers more granular scaling units. Only operating in a single availability zone. Serverless v1 only supports up to v10 of Postgres (v10 will be sunset by Postgres on November 10, 2022). Serverless v2 offers multi-AZ, so that DB subnets can be across multiple availability zones. Supports Postgres 12+. [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.upgrade.html](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.upgrade.html) ### Future Aurora DB Cluster Requirements Because the [RDS Service documentation on Aurora DB clusters](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.html) recommends deploying at least one additional Aurora DB cluster instance to an Availability Zone other than where the primary instance is located in order to ensure High Availability, and since [3 AZs are used by the primary AWS region](/layers/network/design-decisions/decide-on-primary-aws-region), it is recommended that 3 instances are deployed per Aurora DB cluster when High Availability is needed. - Explain how many instances exist in the cluster (or per region, if this is a global cluster) - Explain whether the cluster is global or regional, and reference [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture) - Explain how many secondary DB clusters should exist, if this is a global cluster Lastly, database storage encryption, deletion protection and cloudwatch logs exports should be enabled as a best practice. This, in addition to any of the requirements outlined in _v1 Infrastructure Requirements_, should be captured in the following table. #### **Setting**
Aurora DB cluster Engine
Aurora DB cluster Instance Family
Number of Aurora DB cluster Instances: 1 for all environments except for prod, 3 for prod (or 2 when < 3 AZs are available)
Regional or Global DB Cluster
Security-related settings
Storage Encryption enabled
yes
## Other Considerations - Cost [https://aws.amazon.com/rds/aurora/pricing/](https://aws.amazon.com/rds/aurora/pricing/) ## Consequences Create an Aurora DB Cluster component and tune it to the outlined requirements. ## References - [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture) - [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.html](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.Overview.html) - [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database.html) --- ## Decide on RDS Technology and Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem RDS offers a variety of features and deployment options. A specific RDS deployment option needs to be adopted for the application database solution. Options are not mutually exclusive and multiple types of databases can be deployed depending on your requirements. ## Context Amazon’s RDS Relational Database Service (Amazon RDS) is a fully managed Database-as-a-Service that makes it easy to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while automating time-consuming administration tasks such as hardware provisioning, database setup, patching and backups. ## Considered Options There are several ways in which an RDS Cluster can be deployed. ### **Option 1:** Amazon RDS Instances Amazon RDS Instances are the original version of RDS and provide simple master-slave replication with multiple read replicas and multi-AZ fail-over capabilities. RDS Instances are best suited for one-off databases (e.g. for microservices or dev environments) where performance is likely not an issue and the ability to do point-in-time restores for a database is required. Point in time recovery allows you to create an additional RDS instance (e.g. it does replace your running instance), based on the data as it existed on your instance at any specific point in time by restoring and replaying the journal to a specific point in time. This feature is not supported yet by RDS Aurora. [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html) :::info AWS offers push-button migration to convert existing Amazon RDS MySQL and PostgreSQL RDS instances to RDS Aurora. ::: ### **Option 2:** Amazon Aurora DB Cluster (recommended for most use cases) An RDS Aurora Cluster can be deployed into each VPC. The Aurora DB Cluster must use a DB Subnet Group that spans at least two availability zones. For more information, see: [Creating a DB Cluster (Amazon Aurora)](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Aurora.CreateInstance.html). [https://aws.amazon.com/rds/aurora/faqs/](https://aws.amazon.com/rds/aurora/faqs/) :::caution RDS Aurora does not support Point-in-time Recovery (PITR) like with RDS instances. [https://aws.amazon.com/blogs/storage/point-in-time-recovery-and-continuous-backup-for-amazon-rds-with-aws-backup/](https://aws.amazon.com/blogs/storage/point-in-time-recovery-and-continuous-backup-for-amazon-rds-with-aws-backup/) ::: ### **Option 3:** Amazon Aurora Global Database Amazon RDS Aurora can be deployed as a global database, with an Aurora DB cluster existing in a designated primary AWS region, and up to 5 additional Aurora DB clusters in designated secondary AWS Regions. The Aurora DB clusters in the secondary regions are Aurora replicas, but [write forwarding](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-write-forwarding.html) can be enabled in order to forward write operations to the primary region DB cluster. For more information, see: [Getting Started with Amazon Aurora Global Databases](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-global-database-getting-started.html). :::caution **Major version upgrades must be performed manually (E.g. not with terraform)** ::: > Major version upgrades can contain database changes that are not backward-compatible with previous versions of the database. This functionality can cause your existing applications to stop working correctly. As a result, Amazon Aurora doesn't apply major version upgrades automatically. > [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.PostgreSQL.html#USER_UpgradeDBInstance.PostgreSQL.MajorVersion](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_UpgradeDBInstance.PostgreSQL.html#USER_UpgradeDBInstance.PostgreSQL.MajorVersion)A transit gateway connection will be required between all regions and accounts participating in the Global Database. ### **Option 4:** Amazon Aurora Serverless Amazon Aurora is also offered as Aurora Serverless. This is an on-demand autoscaling configuration that scales automatically horizontally based on usage, and shuts down when it is not in use. The downside to Amazon Aurora Serverless is that there is a warm-up cost that can cause connections to hang for up to 30 seconds. This is potentially mitigated using the [AWS RDS proxy service](https://aws.amazon.com/rds/proxy/). Amazon Aurora Serverless has two releases: v1 and v2. v2 is currently a preview release. [https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-2.limitations.html](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-2.limitations.html) For more information, see: [Using Amazon Aurora Serverless v1](https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless.html). Known issue: [https://github.com/cloudposse/terraform-aws-rds-cluster/issues/81](https://github.com/cloudposse/terraform-aws-rds-cluster/issues/81) :::danger Don't use Aurora Serverless v2 (preview) for production databases. All resources and data will be deleted when the preview ends. ::: ## Other Considerations ### RDS Engine: MySQL or Postgres This decision is determined based on the stack(s) of the applications being onboarded and their supported databases. :::caution `aurora-postgresql` database engine has no minor auto update candidates; therefore, it does not auto update on minor versions. (See [Slack Explanation](https://cloudposse.slack.com/archives/C018WN7NC1W/p1646674264252789)) ::: ### RDS Multi-AZ Lastly, a regular RDS instance can be deployed in a Multi-AZ configuration. A standby instance allows for Multi-AZ redundancy, and [read-replicas](https://aws.amazon.com/rds/features/read-replicas/) can be used to reduce the IO load on the primary RDS instance. This is a more cost-effective option when compared to the Amazon Aurora offerings, but it is also not as scalable. For more information, see: [High availability (Multi-AZ) for Amazon RDS](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.MultiAZ.html). ### Tenancy: Shared or Dedicated Once RDS is deployed, services can either use each database in a shared or dedicated tenancy model. 1. In the shared model, multiple application databases are provisioned on one database instance (or cluster): - This is the most economical option and achieves greater economies of scale. - The downside is that one cannot automatically restore an individual database, making recoveries from human error slower and more manual. 2. In the dedicated model, one application database is provisioned in each database instance (or cluster): - This creates the greatest level of isolation. - Each database has its own automated backup and can be restored as a point-in-time-recovery (PITR) backup ([except for Amazon Aurora](https://aws.amazon.com/blogs/storage/point-in-time-recovery-and-continuous-backup-for-amazon-rds-with-aws-backup/)). - This is the least economical option. ## Consequences A component for an Amazon Aurora RDS cluster will be created and provisioned in each VPC as needed. --- ## Decide on S3 Bucket Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Your organization depends on many S3 buckets, but they all have different purposes, with different requirements. Some are public for serving assets, while others are storing sensitive uploads for customer data. You want to reduce the friction for developers to create new buckets while ensuring some uniformity and standardized policies for how buckets are created. Buckets will be created in multiple stages and seldom if ever shared across stages. Bucket names in AWS S3 are globally unique, so we’ll need to have a convention to name them. ## Considerations :::info We’ll use the [Terraform](/resources/legacy/fundamentals/terraform) to generate bucket names, so a short name for each bucket is all that is required. If you don’t yet know what buckets you will need, then we can provision some dummy buckets as examples. ::: - Short name to describe the bucket (without stage or account) - Define the archetypes/classes of buckets used. The most common types we see are: - Static configurations (e.g. downloaded by mobile clients or EC2 instances) - Static assets (e.g. images, videos, thumbnails, uploads) - Websites, SPAs, Cloudfront origins - Log buckets (e.g. ALB access logs, Cloudtrail Logs, VPC Flow Logs, etc) - Artifact buckets (E.g. for CI/CD, binary executables) - SFTP/upload buckets - Define the lifecycle requirements of objects - Public/private ACLs - Encryption at Rest - Access logs? aggregation of access logs to a centralized location across all accounts? - Cloudfront integration ## Consequences - Catalog configurations will be created for each bucket archetype. - Buckets will be provisioned using the [s3-bucket](/components/library/aws/s3-bucket/) component ## See Also [Decide on Terraform State Backend Architecture](/layers/accounts/design-decisions/decide-on-terraform-state-backend-architecture) --- ## Decide on SFTP Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem **DRAFT** ## Context ## Considered Options - Will SFTP access the existing buckets? or do we need to create a new bucket? - Do you need different users with different levels of access? - Will users need to be to be restricted to different paths of the SFTP file system? - Will the service need to be able to accept multiple logins? - Will the service be public (0.0.0.0/0) or only available to certain CIDRs? - What are the logging requirements? - What are the object lifecycle requirements? - What type of authentication is needed? e.g. Basic certificate-based or SAML? ## References - [https://aws.amazon.com/blogs/storage/using-okta-as-an-identity-provider-with-aws-transfer-for-sftp/](https://aws.amazon.com/blogs/storage/using-okta-as-an-identity-provider-with-aws-transfer-for-sftp/) - [https://github.com/cloudposse/terraform-aws-transfer-sftp](https://github.com/cloudposse/terraform-aws-transfer-sftp) --- ## Decide on the backup AWS region for Aurora Global Cluster import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Considerations - Global Aurora Postgres cluster requires at least two regions in which `aws_rds_cluster` are created - An Aurora global database consists of one primary AWS Region where your data is mastered, and up to five read-only, secondary AWS Regions. Aurora replicates data to the secondary AWS Regions with a typical latency of under a second. You issue write operations directly to the primary DB instance in the primary AWS Region - Related to this: decide on VPC CIDRs for the main and backup regions :::info Consequences of deploying RDS Aurora Global Cluster include: - Provisioning additional VPC and Aurora clusters in the backup region - Setting up peering via the transit gateway because write operations go directly to the primary DB instance in the primary AWS Region ::: ### Related - [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture) --- ## Decide Whether to Use RDS IAM Authentication import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Multiple ways exist to authenticate with the database. Static credentials grow stale and are too easily hardcoded in places making rotation difficult and seldom performed. Generally, short-lived credentials to access only the resources you need to do your job (granting _least privilege_) is preferred. ## Context RDS supports IAM authentication, which means IAM credentials are used to obtain short-lived credentials to access the RDS database. Leveraging RDS IAM Authentication in applications requires application changes to leverage the AWS SDK ([Java Example](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Java.html)). :::caution RDS IAM authentication is not recommended for applications due to a maximum of 200 new connections per second, and therefore only advisable for use with human operators. ::: For applications, the AWS recommended method is using AWS Secrets Manager (as opposed to RDS IAM Authentication) which also has the built-in capability to rotate credentials. Despite these best practices, we primarily provision static credentials randomly generated by terraform using the database provider and then write them to SSM and encrypt with KMS. See [Use SSM over ASM for Infrastructure](/resources/adrs/adopted/use-ssm-over-asm-for-infrastructure) for more context. ## Consequences If we choose to enable RDS IAM Authentication, it’s just a simple feature flag in our rds component. This is an easily reversible decision that can be disabled. ## References - [https://www.ibexlabs.com/iam-database-authentication-for-amazon-rds-in-mysql/](https://www.ibexlabs.com/iam-database-authentication-for-amazon-rds-in-mysql/) - [https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Java.html](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Java.html) - [Use SSM over ASM for Infrastructure](/resources/adrs/adopted/use-ssm-over-asm-for-infrastructure) --- ## Design Decisions(Design-decisions) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for the data layer, including which services you will rely on on their configurations. --- ## (TODO) Decide on RDS Aurora Serverless Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement ## Considered Options ## References --- ## (TODO) Decide on RDS Instance Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem **DRAFT** ## Context [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture) ## Considered Options :::caution RDS Global Databases are only compatible with RDS Aurora. ::: ## References - [https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-right-sizing/tips-for-right-sizing-your-workloads.html](https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-right-sizing/tips-for-right-sizing-your-workloads.html) - [https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-right-sizing/tips-for-right-sizing-your-workloads.html](https://docs.aws.amazon.com/whitepapers/latest/cost-optimization-right-sizing/tips-for-right-sizing-your-workloads.html) --- ## Setup Databases import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Note from '@site/src/components/Note' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Quick Start | Steps | Example | | :------------------------ | :---------------------------------- | | 1. Vendor data components | `atmos workflow vendor -f quickstart/app/data` | | 2. Connect to the VPN | Click Ops | | 3. Deploy clusters | `atmos workflow deploy/all -f quickstart/app/data` | ## Requirements In order to deploy Data layer components, Networking must be fully deployed and functional. See [the network documentation](/layers/network) for details. All deployment steps below assume that the environment has been successfully set up with the following steps. 1. Sign into AWS via Leapp 2. Connect to the VPN 3. Open Geodesic ## Supported databases At the moment we have support for: - [Aurora PostgreSQL](/components/library/aws/aurora-postgres/) - [Aurora PostgreSQL Resources](/components/library/aws/aurora-postgres-resources/) - [Aurora MySQL](/components/library/aws/aurora-mysql/) - [Aurora MySQL Resources](/components/library/aws/aurora-mysql-resources/) - [AWS Backup](/components/library/aws/aws-backup/) - [DocumentDB](/components/library/aws/documentdb/) - [DynamoDB](/components/library/aws/dynamodb/) - [Elasticsearch Cluster](/components/library/aws/elasticsearch/) - [RDS](/components/library/aws/rds/) - [RedShift](/components/library/aws/redshift/) - [ElastiCache Redis](/components/library/aws/elasticache-redis/) ### Vendor Vendor all data components with the following workflow: These run several vendor commands for each included component. You can always run these commands individually to update any single component. For example: We're using `aurora-postgres` for this example. Your database selection may differ. ```bash atmos vendor pull -c aurora-postgres ``` ### Deploy In order to deploy database, deploy both the cluster and the resources component (if applicable). Applying changes to the resources component requires a VPN connection. For example, --- ## How to Enable Cross-Region Backups in AWS-Backup import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; ## Problem AWS Backup is a regional component that can backup a ton of resources. It is often very helpful to save your backups in another region in case of a disaster. ## Solution Create a backup vault and point to it via `destination_vault_arn` variable! Currently, this requires deploying the component into two different regions. The first is a normal aws-backup component. This includes a `plan`, a `vault`, and an `iam` role. The second aws-backup component should be deployed to the cross-region destination. ``` # -.yaml components: terraform: aws-backup: vars: # https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html schedule: cron(0 0 * * ? *) # Daily At 12:00 AM UTC start_window: 60 # Minutes completion_window: 240 # Minutes cold_storage_after: null # Days delete_after: 14 # Days destination_vault_arn: null # Copy to another Region's Vault copy_action_cold_storage_after: null # Copy to another Region's Vault Cold Storage Config (Days) copy_action_delete_after: null # Copy to another Region's Vault Persistence Config (Days) backup_resources: [] selection_tags: - type: "STRINGEQUALS" key: "aws-backup/resource_schedule" value: "dev-daily-14day-backup" ``` ``` # -.yaml components: terraform: aws-backup: vars: plan_enabled: false iam_role_enabled: false ``` :::info This will only create a **vault**! ::: Create the cross-region backup vault first. Grab its ARN, and set it to the value of the `destination_vault_arn`. Apply the component and you now have cross-region backups enabled. --- ## How to Migrate RDS Snapshots import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Note from '@site/src/components/Note' ## Context In this document we will refer to the Legacy Organization as `legacy` and refer to the new Organization as `acme`. At this point we have a populated database in the `legacy-prod` account that we want to migrate to the new organization. This database is encrypted with the Amazon Managed KMS key, `aws/rds`. We have already deployed an empty database to the new account, `acme-prod`. Now we want to migrate all data from the old to the new. ### Additional considerations #### Notes from AWS > You can't share a snapshot that has been encrypted using the default KMS key of the AWS account that shared the > snapshot. To work around the default KMS key issue, perform the following tasks: > > 1. Create a customer managed key and give access to it. > 2. Copy and share the snapshot from the source account. > 3. Copy the shared snapshot in the target account. > > [reference](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ShareSnapshot.html#share-encrypted-snapshot) #### What does this mean 1. We've encrypted the source RDS instance using the AWS managed KMS key for RDS, `aws/rds` 2. We cannot modify the AWS managed KMS key, so we must copy the snapshot using a customer managed KMS key that we create 3. Then we can allow the destination account permission to access our new KMS key, which the destination account will use to access the copied snapshot 4. Restoring a DB instance from a cross account storage encrypted snapshot is not supported, so we must again copy the shared snapshot in the destination account ## Requirements - The developer needs to have `terraform` access in the new Organization for applying Terraform - The developer needs to have `admin` access in target account for the new Organization to read AWS SSM Parameters and KMS keys and create snapshots - The developer needs to have Administrator access in the Legacy Organization source account to create snapshots and KMS keys - We have an populated RDS database cluster in the Legacy Organization source account. This is the source of our data for the migration. - _Any data in this database will be lost!_ We have an empty RDS database cluster in new Organization target account. This is the destination of our data for the migration. We will recreate the DB instance with the new snapshot copy. ## Steps :::info Example Region The remainder of this document assumes both the source and destination regions are `us-west-2`. We will use the environment abbreviation, `usw2`. ::: Connect to _both_ `acme-identity` and `legacy-prod` in Leapp ### Connect to `acme-identity` :::info AWS Team to Team Roles Permission You must have access assume the `acme-plat-gbl-prod-admin` role via your `acme-identity` profile. ::: This is the normal AWS profile we use to connect to Leapp. Follow the steps in [How To Log into AWS](/layers/identity/how-to-log-into-aws). When opening Geodesic, you should see the following with a "green" checkmark: ```console ⧉ acme √ . [acme-identity] (HOST) infrastructure ⨠ ``` ### Connect to `legacy-prod` 1. Open Leapp 2. Create new Integration for the Legacy AWS Organization ```yaml Type: AWS Single Sign-On Alias: Legacy Portal URL: https://.awsapps.com/start/ Region: us-west-2 Auth. method: In-browser # optional ``` 3. Log into the new Integration and accept the pop up windows 4. Find the `legacy-prod` - `AWSAdministratorAccess` Session or whichever Administrator Permission Set you have access to assume. 5. Select the "dots" on the right and click "Change" > "Named Profile" 6. Enter `legacy-prod-admin` 7. Start the session. You should see `legacy-prod-admin` under "Named Profile" 8. Open Geodesic shell 9. Assume the new profile: ```bash assume-role legacy-prod-admin ``` You should now be connected to the Legacy Prod account: ```bash ✗ . [none] (HOST) infrastructure ⨠ assume-role legacy-prod-admin * Found SSH agent config ⧉ acme √ : [legacy-prod-admin] (HOST) infrastructure ⨠ aws sts get-caller-identity { "UserId": "AROXXXXXXXXXXXXXXXXXX:daniel@cloudposse.com", "Account": "111111111111", "Arn": "arn:aws:sts::111111111111:assumed-role/AWSReservedSSO_AWSAdministratorAccess_40xxxxxxxxxxxxxx/daniel@cloudposse.com" } ``` ### Apply Initial Terraform Before we execute the migration script, we need to apply the new `rds` component in the destination or `acme-prod` account. Applying this component will create the customer managed KMS key that we will need to copy the final legacy snapshot into the new account. We should already have the `rds` component configured for `acme-plat-usw2-prod`. This may be configured in the following file: `stacks/orgs/acme/plat/prod/us-west-2/data.yaml`. But please adapt this to your infrastructure file structure. Apply the `rds` component to create the KMS key now, if not already applied: ```bash atmos terraform apply rds -s plat-usw2-prod ``` Once completed, Terraform will return all outputs. Find the output named `kms_key_alias`. For example: ```bash Apply complete! Resources: 0 added, 0 changed, 0 destroyed. Outputs: ... kms_key_alias = "alias/acme-plat-usw2-prod-rds" ... ``` ### Prepare the migration script In order to run a helpful bash tool included with this script, add the following to your local Geodesic infrastructure Dockerfile. To read more about `gum`, please see [charmbracelet/gum](https://github.com/charmbracelet/gum). ``` # Install gum - a CLI tool for making bash scripts "pretty" # https://github.com/charmbracelet/gum RUN mkdir -p /etc/apt/keyrings RUN curl -fsSL https://repo.charm.sh/apt/gpg.key | sudo gpg --dearmor -o /etc/apt/keyrings/charm.gpg RUN echo "deb [signed-by=/etc/apt/keyrings/charm.gpg] https://repo.charm.sh/apt/ * *" | sudo tee /etc/apt/sources.list.d/charm.list RUN apt-get update && apt-get install -y --allow-downgrades \ gum ``` In this guide, we include the `rds-snapshot-migration` script under `rootfs/usr/local/bin/` so that it will be included with your infrastructure Geodesic build. Create a new file called `rootfs/usr/local/bin/rds-snapshot-migration` or choose another path to store this script. Copy and paste the following into that file. :::warning Always Review Before Executing This bash script is a point-in-time copy. Please always review any code you execute, especially against production environments. :::
rds-snapshot-migration The migration script has hard-coded values for our organization. Open the script, add the KMS key alias from the previous step, and verify the rest of the values are what you'd expect. For example: ```bash # Organization specific values legacy_account_id="111111111111" # legacy-prod legacy_rds_instance_id="legacy-prod" legacy_profile="legacy-prod-admin" legacy_region="us-west-2" acme_account_id="222222222222" # acme-prod acme_region="us-west-2" acme_profile="acme-plat-gbl-prod-admin" acme_region="us-west-2" acme_kms_key_alias="alias/acme-plat-usw2-prod-rds" ``` Now copy and paste the following: ```bash #!/bin/bash set -e -o pipefail # Organization specific values legacy_account_id="111111111111" # legacy-prod legacy_rds_instance_id="legacy-prod" legacy_profile="legacy-prod-admin" legacy_region="us-west-2" acme_account_id="222222222222" # acme-prod acme_region="us-west-2" acme_profile="acme-plat-gbl-prod-admin" acme_region="us-west-2" acme_kms_key_alias="alias/acme-plat-usw2-prod-rds" # This path needs to exist. In Geodesic this is created by default to be your infrastructure directory infrastructure_dir_path="$GEODESIC_WORKDIR" # This is the path to the KMS Key policy template. Needs to exist before script runs additional_key_policy="$infrastructure_dir_path/docs/rds-migration/kms_key_policy_addition.json" # This is the path to where we will save the completed KMS key policy. Does not need to exist before script runs updated_key_policy="$infrastructure_dir_path/docs/rds-migration/generated_key_policy.json" function gum_log { gum log --time rfc822 --structured --level info "$1" } function gum_exit { gum log --time rfc822 --structured --level error "Something went wrong..." printf "\n%s\n\n" "$1" exit 1 } function convert_seconds { local total_seconds=$1 local hours=$((total_seconds / 3600)) local minutes=$(( (total_seconds % 3600) / 60)) local seconds=$((total_seconds % 60)) echo "$hours hours, $minutes minutes, $seconds seconds" } function assume_role { profile=$1 region=$2 export AWS_PROFILE=$profile export AWS_DEFAULT_REGION=$region gum spin --spinner dot \ --title "Checking AWS session: $profile" -- \ aws sts get-caller-identity --output json \ || { gum log --time rfc822 --structured --level error \ "Failed to retrieve AWS session name. Please ensure your AWS CLI is configured correctly."; exit 1; } } # The default wait for snapshot may not be enough in every case we've tried wait_for_snapshot() { set +e # continue on errors while true; do gum spin --spinner dot --show-output \ --title "Waiting for database snapshot: $2 ..." -- \ aws rds wait db-snapshot-completed \ --db-instance-identifier "$1" \ --db-snapshot-identifier "$2" if [ $? -eq 0 ]; then set -e # resume failure on errors gum_log "Snapshot completed successfully! Snapshot ID: $2" return 0 else gum_log "Failed to wait for snapshot. Retrying..." fi done } ######################################################################################################### # # Welcome Messages # ######################################################################################################### start_time=$(date +%s) gum_log "Database migration started..." gum style \ --foreground 212 --border-foreground 212 --border double \ --align center --width 50 --margin "1 2" --padding "2 4" \ "Welcome to the RDS Snapshot Migration helper!" gum style \ --foreground 31 \ --margin "1 2" \ "Legacy RDS Instance: ${legacy_rds_instance_id}" \ "..." \ "Legacy Account ID: ${legacy_account_id}" \ "Legacy Account Profile: ${legacy_profile}" \ "Legacy Account Region: ${legacy_region}" \ "..." \ "Destination Account ID: ${acme_account_id}" \ "Destination Account Profile: ${acme_profile}" \ "Destination Account Region: ${acme_region}" printf "Are you ready to start the datastore migration using these values?\n" response=$(gum choose "yes" "no") if [[ "$response" != "yes" ]]; then gum log --time rfc822 --structured --level debug "Exiting..." exit 0 fi ######################################################################################################### # # Begin Migration # ######################################################################################################### assume_role $legacy_profile $legacy_region ######################################################################################################### # # Create or Fetch KMS Key and update the key policy # ######################################################################################################### kms_key_alias="alias/legacy-mockprod-rds-kms-$(date '+%Y%m')" alias_exists=$(gum spin --show-output --spinner dot \ --title "Detecting if KMS key already exists..." -- \ aws kms list-aliases \ --query "Aliases[?AliasName=='$kms_key_alias'] | length(@)" --output text) || gum_log "$alias_exists" if [[ "$alias_exists" -eq "0" ]]; then gum_log "KMS doesn't exist. Creating..." kms_key_id=$(gum spin --show-output --spinner dot --title "Creating KMS key..." -- \ aws kms create-key --query KeyMetadata.KeyId --output text) && gum_log "$kms_key_id" kms_key_alias="legacy-mockprod-rds-kms-$(date '+%Y%m')" gum spin --show-output --spinner dot \ --title "Creating KMS key alias..." -- \ aws kms create-alias \ --target-key-id $kms_key_id \ --alias-name alias/$kms_key_alias else gum_log "KMS exists. Fetching..." kms_key_id=$(gum spin --show-output --spinner dot --title "Retrieving KMS key ID..." -- \ aws kms describe-key --key-id $kms_key_alias --query 'KeyMetadata.KeyId' --output text) || gum_exit "$kms_key_id" fi kms_key_arn=$(gum spin --show-output --spinner dot --title "Retrieving KMS key ARN..." -- \ aws kms describe-key \ --key-id $kms_key_id \ --query 'KeyMetadata.Arn' --output text) gum_log "Key Alias: $kms_key_alias" gum_log "Key ID: $kms_key_id" gum_log "Key ARN: $kms_key_arn" gum_log "Updating KMS key policy..." gum spin --spinner dot --show-output \ --title "Creating KMS key policy..." -- \ sed \ -e "s/THIS_ACCOUNT_ID/${legacy_account_id}/" \ -e "s/ALLOWED_ACCOUNT_ID/${acme_account_id}/" \ $additional_key_policy > $updated_key_policy # Commented out to reduce noise. Uncomment if you need to check the KMS Key Policy # gum_log "Key Policy:" # cat $updated_key_policy | jq gum spin --show-output --spinner dot --show-output \ --title "Updating KMS key policy..." -- \ aws kms put-key-policy \ --key-id $kms_key_id \ --policy file://$updated_key_policy ######################################################################################################### # # Create a RDS snapshot that we can share with the destination # ######################################################################################################### timestamp="$(date '+%Y%m%d%H%M%S')" # Use timestamp to create a useful identifier for the RDS snapshots legacy_rds_snapshot_source_id="$legacy_rds_instance_id-snapshot-$timestamp" legacy_rds_snapshot_share_id="$legacy_rds_instance_id-snapshot-share-$timestamp" gum_log "Creating snapshot of existing RDS instance, encrypted with default KMS key..." gum spin --spinner dot --show-output \ --title "Creating database snapshot: $legacy_rds_snapshot_source_id ..." -- \ aws rds create-db-snapshot \ --db-instance-identifier $legacy_rds_instance_id \ --db-snapshot-identifier $legacy_rds_snapshot_source_id gum_log "Creating the initial snapshot typically takes around 3 minutes" wait_for_snapshot "$legacy_rds_instance_id" "$legacy_rds_snapshot_source_id" gum_log "Copying snapshot to share with new Organization, encrypted with customer managed KMS key..." gum spin --spinner dot --show-output \ --title "Copying database snapshot: $legacy_rds_snapshot_source_id > $legacy_rds_snapshot_share_id ..." -- \ aws rds copy-db-snapshot \ --source-db-snapshot-identifier $legacy_rds_snapshot_source_id \ --target-db-snapshot-identifier $legacy_rds_snapshot_share_id \ --kms-key-id $kms_key_id \ --copy-tags gum_log "Copying and reencrypting snapshots can take more than 20 minutes! Although sometimes it can be much quicker." gum_log "Check the status in the Legacy AWS Console: https://d-926767ca79.awsapps.com/" wait_for_snapshot "$legacy_rds_instance_id" "$legacy_rds_snapshot_share_id" ######################################################################################################### # # Share the snapshot with the destination # ######################################################################################################### # We need the snapshot ARN later. plus check if this snapshot actually exists legacy_rds_snapshot_share_arn=$(gum spin --show-output --spinner dot --title "Retrieving snapshot ARN..." -- \ aws rds describe-db-snapshots \ --db-snapshot-identifier $legacy_rds_snapshot_share_id \ --query 'DBSnapshots[*].DBSnapshotArn' --output text) || gum_exit "$legacy_rds_snapshot_share_arn" gum spin --spinner dot --show-output \ --title "Allowing target account to restore this snapshot..." -- \ aws rds modify-db-snapshot-attribute \ --db-snapshot-identifier $legacy_rds_snapshot_share_id \ --attribute-name "restore" \ --values-to-add $acme_account_id ######################################################################################################### # # Copy the shared snapshot to the destination account so that we can restore RDS from that snapshot # ######################################################################################################### assume_role $acme_profile $acme_region gum_log "Fetching KMS key in destination account..." acme_kms_key_id=$(gum spin --show-output --spinner dot \ --title "Retrieving KMS key ID..." -- \ aws kms list-aliases --query "Aliases[?AliasName=='$acme_kms_key_alias'].TargetKeyId" --output text) || gum_exit "$acme_kms_key_id" gum_log "Copying customer-managed KMS key encrypted snapshot into the destination account..." acme_rds_snapshot_id="${legacy_rds_instance_id}-snapshot-${timestamp}" gum spin --spinner dot --show-output \ --title "Copying database snapshot: $legacy_rds_snapshot_share_arn > $acme_rds_snapshot_id ..." -- \ aws rds copy-db-snapshot \ --source-db-snapshot-identifier $legacy_rds_snapshot_share_arn \ --target-db-snapshot-identifier $acme_rds_snapshot_id \ --source-region $legacy_region \ --kms-key-id $acme_kms_key_id gum_log "Copying and reencrypting snapshots can take more than 20 minutes! Although sometimes it can be much quicker." gum_log "Check the status in the acme AWS Console: https://d-92674d8c2c.awsapps.com/" wait_for_snapshot "$legacy_rds_instance_id" "$acme_rds_snapshot_id" gum_log "Legacy aws/kms Encrypted Snapshot ID: $legacy_rds_snapshot_source_id" gum_log "Legacy Customer-Managed KMS Key Encrypted Snapshot ID: $legacy_rds_snapshot_share_id" gum_log "acme Snapshot ID: $acme_rds_snapshot_id" ######################################################################################################### # # Done! Print some helpful closing messages with the total execution time # ######################################################################################################### end_time=$(date +%s) duration_in_seconds=$((end_time - start_time)) formatted_time=$(convert_seconds $duration_in_seconds) gum_log "The script took: $formatted_time" printf "\n\nNow restore the database in the destination account using the snapshot ID\n\n%s\n\n" $acme_rds_snapshot_id ```
This script refers to a locally stored KMS key policy JSON that we will use to allow cross account access. Create this file locally and set `additional_key_policy` to that JSON file path. The new JSON will be saved to the `updated_key_policy` variable file path. Update this value as well to a path that makes sense for your file structure.
KMS Key Policy The `THIS_ACCOUNT_ID` and `ALLOWED_ACCOUNT_ID` strings will be replaced by `sed` using the `legacy_account_id` and `acme_account_id` values respectively. ```json { "Version": "2012-10-17", "Id": "AllowCrossAccountKMS", "Statement": [ { "Sid": "AllowThisAccountUse", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::THIS_ACCOUNT_ID:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "AllowCrossAccountUse", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::ALLOWED_ACCOUNT_ID:root" }, "Action": [ "kms:CreateGrant", "kms:Decrypt", "kms:DescribeKey", "kms:GenerateDataKey*", "kms:ListGrants", "kms:ReEncryptFrom", "kms:ReEncryptTo", "kms:RevokeGrant" ], "Resource": "*" } ] } ```
Now rebuild Geodesic in order to use the latest image and `rootfs` scripts: ```bash make all ``` ### Execute the migration script Now inside the Geodesic image, we should be authenticated with `acme-identity`. Enter the following command to execute the migration script: ```bash rds-snapshot-migration ``` Your new snapshot name will be output at the end of the script: ```bash Now restore the database in the destination account using the snapshot ID legacy-prod-snapshot-20240517234933 ``` ### Verify the results Your new snapshot should now exist in the target account, `acme-prod`. Optionally, run the following to verify: ```bash AWS_PROFILE=acme-plat-gbl-prod-admin aws rds describe-db-snapshots --db-snapshot-identifier legacy-prod-snapshot-20240517234933 ``` ### Apply Terraform Add the snapshot ID to your `rds` component configuration in the `acme-prod` account. We also need to allow `core-usw2-network` private subnets (where the VPN is deployed) access through the RDS instance's security group in order to locally connect and validate the instance. Therefore to add this unique snapshot identifier for only `acme-plat-usw2-prod`. For example, your `rds` component may be configured in the following file path, but of course adapt this path to your needs: `stacks/orgs/acme/plat/prod/us-west-2/data.yaml` ```yaml component: terraform: rds: vars: # This is the resulting snapshot from the rds-snapshot-migration script snapshot_identifier: legacy-prod-snapshot-20240517234933 # Optionally allow the VPN through the DB's security group allowed_cidr_blocks: - 10.89.16.0/20 # VPN CIDR # The rest of the configuration is irrelevant to this guide # ... ``` And then apply the component, replacing the old DB instance with `-replace`: ```bash atmos terraform apply rds -replace="module.rds_instance.aws_db_instance.default[0]" -s plat-usw2-prod ``` :::tip Replace the Database! A database will only use a snapshot on creation! If you have already created an empty RDS database in the destination account, you must trigger recreation of the database instance. ::: Terraform can take 30 to 40 minutes to apply. ### Setting Database Name and Admin Credentials :::caution Database Name and Admin Credentials When restoring RDS from a snapshot, the instance uses the same database name, admin username, and admin password as the original RDS instance. Make sure to set these values to match, or the next `terraform apply` will destroy and recreate the RDS instance! ::: Add the `database_name` and `database_user` using the same values from the original `legacy-prod` database. Changing either of these values requires database recreation! ```yaml database_name: foobar database_user: admin ``` However, you can leave `database_password` unset. If unset, Terraform will create a new password and store it in AWS SSM. This _does not_ require database recreation. ### Validate the new RDS instance To validate the new RDS instance in `acme-plat-usw2-prod` connect to the VPN, connect to the instance, and list the tables. 1. Connect to the EC2 Client VPN 2. Open Geodesic 3. Get the `psql` command from the output of the `rds` component applied above. Take note of this for later ```bash # As a generic RDS component atmos terraform output rds -s plat-usw2-prod ``` 4. Assume a role that has access to the DB password stored in AWS SSM. For example, we can assume the `admin` role ```bash assume-role acme-plat-gbl-prod-admin ``` 5. Connect to the RDS instance with `psql` using the command we copied from Terraform output above. For example, this would be similar to the following: ```bash PGPASSWORD=$(chamber read app/rds/admin db_password -q) psql --host=acme-plat-usw2-prod-rds.xxxxxxxxxxxx.us-west-2.rds.amazonaws.com --port=5432 --username=admin --dbname=foobar ``` 4. List tables to verify they've been copied over ```bash \dt ``` And that's it! --- ## Tutorials(3) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the associated data layer components. --- ## Deploying the ECS Platform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Quick Start | Steps | Example | | :---------------------- | :-------------------------------------- | | Vendor ECS components | `atmos workflow vendor -f quickstart/platform/ecs` | | Deploy ACM certificates | `atmos workflow deploy/ecs-acm -f quickstart/platform/ecs` | | Connect to the VPN | Click Ops | | Deploy Clusters | `atmos workflow deploy/clusters -f quickstart/platform/ecs` | ## Requirements In order to deploy ECS, Networking must be fully deployed and functional. In particular, the user deploying the cluster must have a working VPN connection to the targeted account. See [the network documentation](/layers/network/) for details. All deployment steps below assume that the environment has been successfully set up with the following steps. 1. Sign into AWS via Leapp 1. Connect to the VPN 1. Open Geodesic # Steps ## Vendor Components Vendor these components with the included Atmos Workflows. ## Deploy ECS ECS provisioning includes deploying certificate requirements, the default ECS cluster, and Echo Server. Echo Server is a basic service used to validate a successful cluster deployed and is an example of an ECS service. Find ECS Service definitions under `catalog/stacks/ecs-services`. To provision each cluster, these components need to be deployed in order. The included Atmos Workflows will carry out this deployment in the proper order, but any of these step can be run outside of a workflow if desired. See the ecs workflow (`stacks/workflows/ecs.yaml`) for each individual deployment step. ## Deploy ACM Certificates First deploy all required ACM certificates for each ECS cluster. These certificates validate the given service domain. You can deploy these certificates before associating the given Route 53 Hosted Zone with the purchased domain in your chosen Domain Registrar, but the certificate will not be ISSUED until the registered domain and Hosted Zone are connected. Run the following to deploy every required ACM certificate for ECS. ## Connect to the VPN In order to complete the following steps, connect to the VPN now. For more on connecting to the VPN, see [`ec2-client-vpn`](https://docs.cloudposse.com/components/library/aws/ec2-client-vpn/#testing). The OVPN configuration for your VPN can be found in the output of the `ecs-client-vpn` component. For example, ```bash atmos terraform output ec2-client-vpn -s core-use1-network ``` ## Deploy All Clusters Run the following to deploy every ECS cluster. This workflow will deploy every required platform cluster. # Related Topics - [ECS Component](/components/library/aws/ecs/) - [ECS Services Component](/components/library/aws/ecs-service/) --- ## Decide on ECS load balancer requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; # Decide on ECS load balancer requirements ## Considerations - Do we need to support internal and external load balancers? - Can we share the load balancer across applications (our default recommendation)? - Do you need to support protocols other than HTTP such as raw TCP or UDP? --- ## Decide on the Application Service Log Destination for ECS import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; ## Context and Problem Statement Should logs be sent to datadog or log only to cloudwatch logs? ## Considered Options ### Option 1. Fluent Bit sidecar is required for ECS Fargate > In addition to your application container, your task definition needs to specify a Fluent Bit sidecar container that’s > responsible for routing logs to Datadog. AWS provides an `aws-for-fluent-bit` Docker image you can use to create the > sidecar container.([source](https://www.datadoghq.com/blog/aws-fargate-monitoring-with-datadog/)) #### Pros - Recommended by Datadog - Works without much configuration - Prior art for this #### Cons - Sidecar required - Fargate cpu/mem requirements would go up per task - More expensive due to fargate pricing is by cpu/memory ### Option 2. Datadog Lambda Forwarder Apps log directly to cloudwatch log groups and the Datadog lambda forwarder would then forwards the logs [https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring) And we have a module for this [https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder](https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder) #### Pros - Apps log to cloudwatch log groups - if logging to a single log group per account, we can use `filter_patterns` to filter the events - if logging to a log group per service, we can give it multiple cloudwatch log group names or deploy a separate lambda per service - Module exposes providing your own lambda in case forking the official Datadog one is needed - Prior art for this #### Cons - Previously recommended by Datadog - Need to log to cloudwatch logs before pushing to Datadog - Maintaining upgrades for the lambdas - Monitoring lambdas ### Option 3. Fluentd with CloudWatch Plugin In the future, if AWS adds a logdriver for `fluentd` without firelens (like with `splunk`), then we can forward to a fluentd aggregator without a sidecar and without logging to cloudwatch. We could create a single service in each ecs cluster to run fluentd and the cloudwatch plugin would be able to push cloudwatch logs to datadog. [https://hub.docker.com/r/fluent/fluentd/](https://hub.docker.com/r/fluent/fluentd/) [https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs#in_cloudwatch_logs](https://github.com/fluent-plugins-nursery/fluent-plugin-cloudwatch-logs#in_cloudwatch_logs) #### Pros - More control over parsing logs - Can output to multiple SIEMs #### Cons - Same cons as option 1 - Maintaining a fluentd cluster - Monitoring a fluentd cluster - We don’t have prior art for this. --- ## Review Design Decisions import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for ECS. These decisions relate to how you will provision your Elastic Container Service clusters on AWS. --- ## ECS Foundational Platform import ReactPlayer from "react-player"; import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Elastic Container Service (ECS) is a fully-managed container orchestration service provided by Amazon Web Services (AWS) that simplifies the process of deploying, managing, and scaling containerized applications. ECS makes it easy to run and manage Docker containers on AWS infrastructure, providing a secure and scalable platform for your applications.
AI generated voice
## The Problem The emergence of Docker necessitated the development of container management solutions, with Kubernetes being one of the most widely adopted options. However, Kubernetes is often considered overly complex for smaller scale operations, akin to using a nuclear reactor to charge a phone. In such scenarios, Elastic Container Service (ECS) is a more practical choice for deploying applications with speed and efficiency. With ECS there is no need to upgrade the underlying platform. Unlike EKS which requires consistent upgrades to stay current, ECS is a managed service that is always up to date. This means that you can focus on your application and not the underlying platform. ## Our Solution We have developed a set of Terraform modules that can be used to deploy ECS clusters and services. ### ECS Cluster Component The [`ecs` component] is used to deploy an ECS cluster and an associated load balancer. #### Application Load Balancer (ALB) Through stack configuration you can determine your domains, subdomains, and the number of instances to deploy. The component also supports the deployment of a bastion host, which can be used to access containers on the ECS Cluster. ```yaml alb_configuration: public: # name of the ALB to be referred to by other configurations internal_enabled: false # sets it to public # resolves to *.public-platform..... route53_record_name: "*.public-platform" private: internal_enabled: true route53_record_name: "*.private-platform" ``` #### Autoscaling The cluster component has the ability to scale with a variety of options. Fargate provides a serverless way of scaling. Spot instances provide a cheaper way to run instances than on demand ec2. you can mix these options to provide a cost effective and scalable solution. ```yaml name: ecs capacity_providers_fargate: true capacity_providers_fargate_spot: true capacity_providers_ec2: default: instance_type: t3.medium max_size: 2 ``` ### ECS Service The [`ecs-service` component] is used to deploy an ECS service. This includes the task and the service definition. By default we also support datadog logging and metrics. This can be disabled by setting `datadog_agent_sidecar_enabled` to false. ```yaml datadog_agent_sidecar_enabled: false ``` --- ## FAQ(Ecs) import Intro from '@site/src/components/Intro'; Frequently asked questions about ECS with Cloud Posse's reference architecture. ## How do I add a new service to an existing ECS cluster? Add a new instance of the `ecs-service` component to your stack configuration. The component will automatically detect the ECS cluster and add the service to it. ## How can I add AWS Policies to my ECS Tasks? Use the `task_policy_arns` to attach policies to individual tasks, this allows those tasks to access AWS resources. ```yaml task_policy_arns: - arn:aws:iam::aws:policy/AmazonS3FullAccess ``` ## How can I inject secrets into my ECS Service? Use the `map_secrets` variable which maps a environment variable key to an SSM param store key. This will inject the value of the SSM param store key into the environment variable. ```yaml map_secrets: SECRET_KEY: /my/secret/key ``` ## How can we create Self Hosted Runners for GitHub with ECS? If we are not deploying EKS for our platform, it doesn't make much sense to configure EKS solely for self-hosted runners. Instead, we deploy the [Philips Labs Action Runners](/layers/github-actions/philips-labs-github-runners) and connect those instances to GitHub. For more on self-hosted GitHub Runners, see [`github-runners`](/layers/github-actions). --- ## Provision Example Services on the ECS Platform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Provision an example service like the Echo server to test your cluster’s functionality. This will ensure you can access the service remotely via the load balancer, both publicly and privately through the VPN. ## Quick Start | Steps | Example | | :----------------- | :----------------------------------------- | | Deploy Echo Server | `atmos workflow deploy/echo-server -f quickstart/platform/ecs` | Once the cluster is up and running, continue with the first ECS service deployment. We deploy Echo Server as an example and to validate a given cluster. This deploys two ECS services: one private and one public. The private deployment should only be accessible by VPN. ## Deploy Echo Server Run the following to deploy `ecs/platform/service/echo-server` and `ecs/platform/service/echo-server-private` to every cluster. ## Verify the Deployment of Public Endpoints In your browser, go to each of the following URLs. Make sure to use your service discovery domain and not the example.
plat-sandbox
Go to `https://echo-server.public-platform.use1.sandbox.plat.acme-svc.com/`
plat-dev
Go to `https://echo-server.public-platform.use1.dev.plat.acme-svc.com/`
plat-staging
Go to `https://echo-server.public-platform.use1.staging.plat.acme-svc.com/`
plat-prod
Go to `https://echo-server.public-platform.use1.prod.plat.acme-svc.com/`
## Verify the Deployment of Private Endpoints Verify these are not publicly accessible. Each of the following should time out if not connected to the VPN.
plat-sandbox
Go to `https://echo-server.private-platform.use1.sandbox.plat.acme-svc.com/`
plat-dev
Go to `https://echo-server.private-platform.use1.dev.plat.acme-svc.com/`
plat-staging
Go to `https://echo-server.private-platform.use1.staging.plat.acme-svc.com/`
plat-prod
Go to `https://echo-server.private-platform.use1.prod.plat.acme-svc.com/`
## Test Private Endpoints using VPN Then connect to the VPN to successfully, and retry the previous step.
--- ## Deploy 1Password SCIM Bridge import Intro from "@site/src/components/Intro"; import Steps from "@site/src/components/Steps"; import Step from "@site/src/components/Step"; import StepNumber from "@site/src/components/StepNumber"; import CollapsibleText from "@site/src/components/CollapsibleText"; The 1Password SCIM Bridge is a service that allows you to automate the management of users and groups in 1Password. This guide will walk you through deploying the SCIM Bridge for ECS environments. ## Implementation The implementation of this is fairly simple. We will generate credentials for the SCIM bridge in 1Password, store them in AWS SSM Parameter Store, deploy the SCIM bridge ECS service, and then finally connect your chosen identity provider. ### Generate Credentials for your SCIM bridge in 1Password The first step is to generate credentials for your SCIM bridge in 1Password. We will pass these credentials to Terraform and the ECS task definition to create the SCIM bridge. 1. Log in to your 1Password account 1. Click Integrations in the sidebar 1. Select "Set up user provisioning" 1. Choose "Custom" 1. You should now see the SCIM bridge credentials. We will need the "scimsession" and "Bearer Token" for the next steps. 1. Save these credentials in a secure location (such as 1Password) for future reference 1. Store only the "scimsession" in AWS SSM Parameter Store. This will allow the ECS task definition to access the credentials securely. Then once the service is running, the server will ask for the bearer token to verify the connection, which we will enter at that time. - Open the AWS Web Console - Navigate to the target account, such as `core-auto`, and target region, such as `us-west-2` - Open "AWS System Manager" > "Parameter Store" - Create a new Secure String parameter using the credentials you generated in the previous step: `/1password/scim/scimsession` There will be additional steps to complete the integration in 1Password, but first we need to deploy the SCIM bridge service. ### Deploy the SCIM bridge ECS Service The next step is to deploy the SCIM bridge ECS service. We will use Terraform to create the necessary resources with our existing `ecs-service` component. Ensure you have the `ecs-service` component and `ecs` cluster before proceeding. If you do not have ECS prerequisites, please see the [ECS layer](/layers/ecs) to create the necessary resources. 1. Create a new stack configuration for the SCIM bridge. The placement of this file will depend on your project structure. For example, you could create a new file such as `stacks/catalog/ecs-services/1password-scim-bridge.yaml` with the following content: ```yaml import: - catalog/terraform/services/defaults components: terraform: 1pass-scim: metadata: component: ecs-service inherits: - ecs-service/defaults vars: enabled: true name: 1pass-scim containers: service: name: op_scim_bridge image: 1password/scim:v2.9.5 cpu: 128 memory: 512 essential: true dependsOn: - containerName: redis condition: START port_mappings: - containerPort: 3002 hostPort: 3002 protocol: tcp map_environment: OP_REDIS_URL: redis://localhost:6379 OP_TLS_DOMAIN: "" OP_CONFIRMATION_INTERVAL: "300" map_secrets: OP_SESSION: "1password/scim/scimsession" log_configuration: logDriver: awslogs options: {} redis: name: redis image: redis:latest cpu: 128 memory: 512 essential: true restart: always port_mappings: - containerPort: 6379 hostPort: 6379 protocol: tcp map_environment: REDIS_ARGS: "--maxmemory 256mb --maxmemory-policy volatile-lru" log_configuration: logDriver: awslogs options: {} ``` 2. Confirm the `map_secrets` value for `OP_SESSION` matches the AWS SSM Parameter Store path you created previously, an confirm they are in the same account and region as this ECS service component. 3. Deploy the ECS service with Atmos: ```bash atmos terraform apply 1pass-scim -s core-usw2-auto ``` ### Validate the Integration After deploying the SCIM bridge ECS service, verify the service is running and accessible. Connect to the VPN (if deployed the ECS service is deployed with a private ALB), navigate to the SCIM bridge URL, and confirm the service is running. For example, go to `https://1pass-scim.platform.usw1.auto.core.acme-svc.com/` ### Connect your Identity Provider Finally, connect your identity provider to the SCIM bridge. The SCIM bridge URL will be the URL you validated in the previous step. Follow the instructions in the 1Password SCIM Bridge documentation to connect your identity provider, using the Bearer Token you generated in the first step. --- ## Setup Vanity Domains on an ALB import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; Learn how to setup ECS Vanity Domains on an existing ALB together with Service Discovery Domains. ## Pre-requisites - [Understand the differences between Vanity Domains and Service Discovery Domains](/resources/legacy/learning-resources/#the-difference-between-vanity-domains-and-service-discovery-domains) - Assumes our standard [Network Architecture](/layers/network/) - Requires `dns-primary` & `dns-delegated` are already deployed. ## Context After setting up your [Network Architecture](/layers/network) you will have 2 hosted zones in each platform account. In `dev` for example, you will have Hosted Zones for `dev-acme.com` and `dev.platform.acme.com`. You should also have an ACM certificate that registers `*.dev-acme.com` and `*.dev.platform.acme.com`. Also ensure you've deployed applications to your ECS cluster and have two ALBs for service discovery, one public, one private. Now we want to set up a vanity subdomain for `echo-server.dev-acme.com` that will point to one of the ALBs used for service discovery. This saves us money by not requiring a new ALB for each vanity domain. ## Implementation The implementation of this is fairly simple. We add additional certificates to our ECS Cluster ALBs, then we add another route to the ALB Listener Rules. ## Setup ACM Certs By default, our `dns-primary` component will create ACM certs for `*.dev-acme.com`. Depending on the level of subdomains you want, you may need to disable this with the variable `request_acm_certificate: false` If a single subdomain is sufficient. e.g. `api.dev-acme.com` then you can leave this enabled. See the troubleshooting section if you run into issues with recreating resources. ## Understand How it Works With a valid ACM cert for your domains we configure the ECS Cluster ALBs to use the certificate. This is done by adding the certificate to the ECS Clusters stack configuration. You can validate your cert is picked up by the ALB by checking the ALB's target group. You should see the certificate listed under the `Certificates` tab. ## Adding the ACM TLS Cert to the ECS Cluster ALBs Here is a snippet of a stack configuration for the ECS Cluster. Note the `additional_certs` variable which declares which additional certs to add to the ALB. ```yaml components: terraform: ecs/cluster: vars: alb_configuration: public: internal_enabled: false # resolves to *.public-platform..... route53_record_name: "*.public-platform" additional_certs: - "dev-acme.com" private: internal_enabled: true route53_record_name: "*.private-platform" additional_certs: - "dev-acme.com" ``` ## Point the Vanity Domain to the ALB Now that our ECS Cluster ALB supports `*.dev-acme.com`, we now need to update our service to point to this new domain as well. We can use the following snippet: ```yaml components: terraform: ecs/platform/service/echo-server: vars: vanity_domain: "dev-acme.com" vanity_alias: - "echo-server.dev-acme.com" ``` ## Troubleshooting The problem with this comes when you need to remove a subdomain or ACM certificate. By running `atmos terraform deploy dns-delegated -s plat--dev` with `request_acm_certificate: false`, you are trying to destroy a single ACM certificate in an account. While this is a small scope deletion, the ACM certificate is in use by the ALB, and the ALB has many different targets. Thus Terraform will stall out. :::warning This is a **destructive** operation and will cause downtime for your applications. ::: You need to: 1. Delete the listeners and targets of the ALB that are using the certificate 2. Delete the ALB 3. Terraform will then successfully delete the ACM certificate. You will notice: 1. The ALB will be recreated 2. Ingresses should reconcile for service discovery domains 3. ALB Targets should be recreated pointing at service discovery domains. Once you recreate the correct ACM certificates and have valid ingresses you should be able to access your applications via the vanity domain. --- ## Tutorials(4) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the associated ECS components. --- ## Deploying the EKS Platform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Note from '@site/src/components/Note' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Quick Start | Steps | Example | | :---------------------------------- | :-------------------------------------------------------- | | 1. Vendor EKS components | `atmos workflow vendor -f quickstart/platform/eks` | | 2. Connect to the VPN | | | 3. Deploy roles for each EKS stack | `atmos workflow deploy/iam-service-linked-roles -f quickstart/platform/eks` | | 4. Deploy cluster into each stack | `atmos workflow deploy/cluster -s plat-use1-dev -f quickstart/platform/eks` | | 5. Deploy resources into each stack | `atmos workflow deploy/resources -s plat-use1-dev -f quickstart/platform/eks` | Repeat steps 4 and 5 for each EKS stack, typically `plat-dev`, `plat-staging`, and `plat-prod` ## Requirements In order to deploy EKS, Networking must be fully deployed and functional. In particular, the user deploying the cluster must have a working VPN connection to the targeted account. See [the network documentation](/layers/network) for details. All deployment steps below assume that the environment has been successfully set up with the following steps. 1. Sign into AWS via Leapp 1. Connect to the VPN 1. Open Geodesic # Steps ## Vendor Components EKS adds many components required to set up a cluster. Generally, all these components are contained in the EKS components and catalog folders, under `components/terraform/eks` and `catalog/stacks/eks` respectively. Vendor these components with the included Atmos Workflows. ## Deploy EKS Cluster EKS provisioning includes many components packaged together into a single import per stack. Leveraging Atmos inheritance, we have defined a baseline set of required components for all EKS deployments and a unique set of additional components for a particular stack's EKS deployment. Find these catalog set definitions under `catalog/stacks/eks/clusters`. To provision a cluster, these components need to be deployed in order. The included Atmos Workflows will carry out this deployment in the proper order, but any of these step can be run outside of a workflow if desired. See the eks workflow (`stacks/workflows/eks.yaml`) for each individual deployment step. ## Deploy IAM Service Linked Roles In order for Karpenter to reserve Spot Instances, the cluster needs to have a Service-Linked Role. Deploy these to all cluster accounts with `iam-service-linked-roles` ## Deploy Initial Platform Dev Cluster First deploy the cluster and AWS EFS. Since Karpenter will be used in the following steps, the initial cluster is deployed without Nodes. Change `use1` to your cluster's environment! ## Deploy Platform Dev Cluster Resources Once the cluster is up and running, continue with the EKS `plat` resources deployment. These need to be deployed in the given order by the include Atmos Workflow. For additional details on each component, see the included `README.md` for the individual component. Run the Atmos Workflow to deploy all required `plat` components. Validate the cluster deployment with `eks/echo-server` and the targeted service domain. The following URL should return a success message for `dev`: https://echo.use1.dev.plat.acme-svc.com/ ## Deploy Staging Once the `dev` cluster is deployed and validated, continue with `staging` and then `prod`. Repeat the same deployment steps in `staging` Validate `staging`: https://echo.use1.staging.plat.acme-svc.com/ ## Deploy Production Then deploy `prod` Validate `prod`: https://echo.use1.prod.plat.acme-svc.com/ # Related Topics - [eks/cluster](/components/library/aws/eks/cluster/) - [Karpenter Documentation](https://karpenter.sh/) --- ## Decide on Default Storage Class for EKS Clusters import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; When provisioning EKS (Kubernetes) clusters, there is no one-size-fits-all recommendation for the default storage class. The right choice depends on your workload’s specific requirements, such as performance, scalability, and cost-efficiency. While only one storage class can be set as the default, storage classes are not mutually exclusive, and the best solution may often involve using a combination of classes to meet various needs. A `StorageClass` in Kubernetes defines the type of storage (e.g., EBS, EFS, etc.) and its parameters (e.g., performance, replication) for dynamically provisioning Persistent Volumes. The default `StorageClass` is automatically used when a `PersistentVolumeClaim` (PVC) is created without specifying a specific storage class, but its configuration varies depending on the cluster setup and cloud provider. Storage classes available on AWS differ from other clouds. ## Default Storage Class Options We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. - **Availability Zone Lock-in:** EBS volumes are restricted to a single Availability Zone, which can impact high availability and disaster recovery strategies. This is a key drawback of using EBS. If you need a Pod to recover across multiple AZs, EFS is a more suitable option, though it comes at a higher cost. - **Performance:** EFS generally offers lower performance when compared to EBS. This can be mitigated by paying for additional bandwidth but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS. - **Cost:** EFS is significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. - **Concurrent Access:** EBS volumes can only be attached to one instance at a time within the same Availability Zone, making them unsuitable for scenarios that require concurrent access from multiple instances. In contrast, EFS allows multiple instances or Pods to access the same file system concurrently, which is useful for distributed applications or workloads requiring shared storage across multiple nodes. ## Amazon EFS **Amazon EFS** provides a scalable, fully managed, elastic file system with NFS compatibility, designed for use with AWS Cloud services and on-premises resources. ### Pros: - **Unlimited Disk Space:** Automatically scales storage capacity as needed without manual intervention. - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios. - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling. - **Availability Zone Failover**: For workloads that require failover across multiple Availability Zones, EFS is a more suitable option. It provides multi-AZ durability, ensuring that Pods can recover and access persistent storage seamlessly across different AZs. ### Cons: - **Lower Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of even modestly demanding applications. - **Higher Cost:** Significantly more expensive than EBS, at least 3x the price per GB and potentially more depending on performance demands, although there may be some savings from not having to reserve size for future growth. - **Higher Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications. - **No Native Backup Support:** EFS lacks a built-in, straightforward backup and recovery solution for EKS. Kubernetes-native tools don’t support EFS backups directly, requiring the use of alternatives like AWS Backup. Recovery, however, can be non-trivial and may involve complex configurations to restore data effectively. ## Amazon EBS **Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads. ### Pros: - **Higher Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications. - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios. - **Native EKS Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management. - **Supports Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning. ### Cons: - **Single-Attach Limitation:** EBS volumes can only be attached to a single node at a time, limiting shared access across multiple Pods or instances. Additional configurations or alternative storage solutions are required for scenarios needing concurrent access. - **Availability Zones:** EBS volumes are confined to a single Availability Zone, limiting high availability and disaster recovery across zones. This limitation can be mitigated by configuring a `StatefulSet` with replicas spread across multiple AZs. However, for workloads using EBS-backed Persistent Volume Claims (PVCs), failover to a different AZ requires manual intervention, including provisioning a new volume in the target zone, as EBS volumes cannot be moved between zones. - **Non-Elastic Storage:** EBS volumes can be manually resized, but this process is not fully automated in EKS. After resizing an EBS volume, additional manual steps are required to expand the associated Persistent Volume (PV) and Persistent Volume Claim (PVC). This introduces operational complexity, especially for workloads with dynamic storage needs, as EBS lacks automatic scaling like EFS. ## Recommendation Use **Amazon EBS** as the primary storage option when: - High performance, low-latency storage is required for workloads confined to a single Availability Zone. - The workload doesn’t require shared access across multiple Pods. - You need cost-effective storage with support for snapshots and backups. - Manual resizing of volumes is acceptable for capacity management, recognizing that failover across AZs requires manual intervention and provisioning. Consider **Amazon EFS** when: - Multiple Pods need concurrent read/write access to shared data across nodes. - Workloads must persist data across multiple Availability Zones for high availability, and the application does not support native replication. - Elastic, automatically scaling storage is necessary to avoid manual provisioning, especially for workloads with unpredictable growth. - You are willing to trade off higher costs and lower performance for multi-AZ durability and easier management of shared storage. :::important EFS should never be used as backend storage for performance-sensitive applications like databases, due to its high latency and poor performance under heavy load. ::: --- ## Decide on EKS Node Pool Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Kubernetes has a concept of Node Pools, which are basically pools of computing resources. Node pools are where the scheduler dispatches workloads based on the taints/tolerations of nodes and pods. ## Types of Node Pools At a minimum recommend one node pool per availability zone to work optimally with the cluster autoscaler. 1 Node Group per AZ is required by the Kubernetes Cluster Autoscaler to effectively scale up nodes in the appropriate AZs. - [https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html#ca-ng-considerations](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html#ca-ng-considerations) :::caution **Important** If you are running a stateful application across multiple Availability Zones that is backed by Amazon EBS volumes and using the Kubernetes [Cluster Autoscaler](https://docs.aws.amazon.com/eks/latest/userguide/cluster-autoscaler.html), you should configure multiple node groups, each scoped to a single Availability Zone. In addition, you should enable the `--balance-similar-node-groups` feature. ::: After that, we should decide the kinds of node pools. Typically a node pool is associated with an instance type. Also, since certain instance types are more expensive than others (e.g. GPU support), creating different pools of resources is suggested. We can create pools with multiple cores, high memory, or GPU. There’s no one-size-fits-all approach with this. The requirements are determined by your applications. If you don’t know the answer right now, we’ll start with a standard node pool and grow from there. This is an easily reversible decision. ## Provisioning of Node Pools We have a few ways that we can provision node pools. 1. Use [https://github.com/cloudposse/terraform-aws-eks-node-group](https://github.com/cloudposse/terraform-aws-eks-node-group) (Fully-managed) 2. Use [https://github.com/cloudposse/terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers) (Self-managed in Auto Scale Groups) 3. Use [https://github.com/cloudposse/terraform-aws-eks-fargate-profile](https://github.com/cloudposse/terraform-aws-eks-fargate-profile) (Fully-managed serverless) 4. Use [https://github.com/cloudposse/terraform-aws-eks-spotinst-ocean-nodepool](https://github.com/cloudposse/terraform-aws-eks-spotinst-ocean-nodepool) ([Spot.io](http://Spot.io) managed node pools - most cost-effective) ## Fargate Node Limitations Currently, EKS Fargate has a lot of limitations [https://docs.aws.amazon.com/eks/latest/userguide/fargate.html](https://docs.aws.amazon.com/eks/latest/userguide/fargate.html) - Classic Load Balancers and Network Load Balancers are not supported on pods running on Fargate. For ingress, we recommend that you use the ALB Ingress Controller on Amazon EKS (minimum version v1.1.4). - Pods must match a Fargate profile at the time that they are scheduled in order to run on Fargate. Pods that do not match a Fargate profile may be stuck as Pending. If a matching Fargate profile exists, you can delete pending pods that you have created to reschedule them onto Fargate. - Daemonsets are not supported on Fargate. If your application requires a daemon, you should reconfigure that daemon to run as a sidecar container in your pods. - Privileged containers are not supported on Fargate. - Pods running on Fargate cannot specify HostPort or HostNetwork in the pod manifest. - GPUs are currently not available on Fargate. - Pods running on Fargate are not assigned public IP addresses, so only private subnets (with no direct route to an Internet Gateway) are supported when you create a Fargate profile. - We recommend using the Vertical Pod Autoscaler with pods running on Fargate to optimize the CPU and memory used for your applications. However, because changing the resource allocation for a pod requires the pod to be restarted, you must set the pod update policy to either Auto or Recreate to ensure correct functionality. - Stateful applications are not recommended for pods running on Fargate. Instead, we recommend that you use AWS solutions such as Amazon S3 or DynamoDB for pod data storage. - Fargate runs each pod in a VM-isolated environment without sharing resources with other pods. However, because Kubernetes is a single-tenant orchestrator, Fargate cannot guarantee pod-level security isolation. You should run sensitive workloads or untrusted workloads that need complete security isolation using separate Amazon EKS clusters. --- ## Decide on email address for cert-manager support emails import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options ### Option 1 (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - #### Cons - ### Option 2 #### Pros - #### Cons - ### Option 3 #### Pros - #### Cons - ## References - Links to any research, ADRs or related Jiras --- ## Decide on Helm Chart Repository Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Helm charts were originally intended to be compiled into packages which are `.tar.gz` archives with a manifest and the templatized kubernetes manifests. This chart repository will house proprietary charts your organization depends on and therefore is on the critical path for both availability and supply-chain attacks. Any adversary who controls your chart repository effectively controls what you can run on your cluster. The chart registry goes down, your company will be unable to deploy applications via helm. The same goes for third-party charts, hosted on external registries. ## Solution Fortunately, charts are very simple to produce and host. They are easily served from a VCS (e.g. git) or S3. There are a few ways to host chart repositories, but our recommendation is just to use VCS. Using `terraform` or `helmfile`, we can just point directly to GitHub and short-circuit the need for managing a chart repository altogether. Terraform natively supports this pattern. :::tip Our recommendation is to use GitHub directly and to avoid using chart repositories unless there are specific requirements for it. ::: ## Considerations If we must, then here are the considerations: 1. **Use VCS** (e.g. GitHub) to pull charts directly and build on-demand. 2. **Use OCI** (docker registry) to push/pull charts (e.g. ECR). Support for terraform will probably drop soon. [https://github.com/hashicorp/terraform-provider-helm/issues/633#issuecomment-1021093381](https://github.com/hashicorp/terraform-provider-helm/issues/633#issuecomment-1021093381) [https://aws.amazon.com/blogs/containers/oci-artifact-support-in-amazon-ecr/](https://aws.amazon.com/blogs/containers/oci-artifact-support-in-amazon-ecr/) 3. **Use GitHub Actions** to build chart artifacts and push them to a static endpoint. [https://github.com/helm/chart-releaser-action](https://github.com/helm/chart-releaser-action) 4. **Use Nexus** - Self-hosted artifactory alternative 5. **Use Artifactory** (SaaS preferred) 6. **Use S3** (public or private chart repositories) [https://github.com/hypnoglow/helm-s3](https://github.com/hypnoglow/helm-s3) 7. **ChartMuseum** → S3 [https://github.com/helm/chartmuseum](https://github.com/helm/chartmuseum) Using Nexus or Artifactory make the most sense if we plan to use them elsewhere to cache artifacts. ## References - [https://helm.sh/docs/topics/chart_repository](https://helm.sh/docs/topics/chart_repository) --- ## Decide on Host OS Flavor for EKS import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We need to pick the AMI which will be used by the EKS cluster nodes. There are a few options and the right one depends on your needs and organization’s requirements for compliance and packaging. It also relates to [Decide on Technical Benchmark Framework](/layers/security-and-compliance/design-decisions/decide-on-technical-benchmark-framework) and [Decide on Strategy for Hardened Base AMIs](/layers/security-and-compliance/design-decisions/decide-on-strategy-for-hardened-base-amis). ## Solution EKS managed & unmanaged node groups support the ability to have custom AMIs. By default, we use Amazon Linux. Where this might be insufficient is if your organization requires a vetted OS or specific tools for audits. Since this is a reversible decision, so we can start with Amazon Linux and change later if needed. ### Option 1: Bottlerocket Amazon’s Bottlerocket OS is a container-native OS built to be immutable and only supports non-root containers. It supports a Kubernetes Operator for automatically updating the cluster, in a way reminiscent of the `update_engine` of the CoreOS (defunct). [https://aws.amazon.com/blogs/containers/amazon-eks-adds-native-support-for-bottlerocket-in-managed-node-groups/](https://aws.amazon.com/blogs/containers/amazon-eks-adds-native-support-for-bottlerocket-in-managed-node-groups/) [https://aws.amazon.com/bottlerocket/](https://aws.amazon.com/bottlerocket/) [https://github.com/bottlerocket-os/bottlerocket-update-operator](https://github.com/bottlerocket-os/bottlerocket-update-operator) ### Option 2: Amazon Linux Our standard recommendation is to use Amazon Linux’s EKS optimized image which is the most battle-tested in the AWS landscape. ### Option 3: DIY If we want to build custom AMIs, then we recommend using Packer with GitHub Action workflows to automatically build and push AMIs. **REVERSIBLE** --- ## Decide on Kubernetes Ingress Controller(s) import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Considerations Kubernetes supports any number of ingress controllers deployed multiple times. The choice of Ingress controller will determine which AWS features we can natively support (e.g. WAF requires an ALB). Our recommendation is to use the `aws-loadbalancer-controller` (aka `aws-alb-ingress-controller` v2) with ACM certificates provisioned by terraform. :::caution TLS terminates at the ALB. It’s then _optionally_ unencrypted if the downstream services support it, such as with self-signed certificates and a TLS sidecar like Envoy or Nginx. Without this, traffic is in clear-text between the ALB and the downstream service or pod. ::: Historically, we’ve recommended `ingress-nginx` (formerly `nginx-ingress`), but prefer to use the AWS load balancer controller due to it’s native support by AWS. --- ## Decide on Secrets Management for EKS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; We need to decide on a secrets management strategy for EKS. We prefer storing secrets externally, like in AWS SSM Parameter Store, to keep clusters more disposable. If we decide on this, we'll need a way to pull these secrets into Kubernetes. ## Problem We aim to design our Kubernetes clusters to be disposable and ephemeral, treating them like cattle rather than pets. This influences how we manage secrets. Ideally, Kubernetes should not be the sole source of truth for secrets, though we still want to leverage Kubernetes’ native `Secret` resource. If the cluster experiences a failure, storing secrets exclusively within Kubernetes risks losing access to them. Additionally, keeping secrets only in Kubernetes limits integration with other services. To address this, several solutions allow secrets to be stored externally (as the source of truth) while still utilizing Kubernetes' `Secret` resources. These solutions, including some open-source tools and recent offerings from Amazon, enhance resilience and interoperability. Any approach must respect IAM permissions and ensure secure secret management for applications running on EKS. We have several options to consider that balance external secret storage with Kubernetes-native functionality. ### Option 1: External Secrets Operator Use [External Secrets Operator](https://external-secrets.io/latest/) with AWS SSM Parameter Store. External Secrets Operator is a Kubernetes operator that manages and stores sensitive information in external secret management systems like AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, HashiCorp Vault, and more. It allows you to use these external secret management systems to securely add secrets in your Kubernetes cluster. Cloud Posse historically recommends using External Secrets Operator with AWS SSM Parameter Store and has existing Terraform modules to support this solution. See the [eks/external-secrets-operator](/components/library/aws/eks/external-secrets-operator/) component. ### Option 2: AWS Secrets Manager secrets with Kubernetes Secrets Store CSI Driver Use [AWS Secrets and Configuration Provider (ASCP) for the Kubernetes Secrets Store CSI Driver](https://docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_csi_driver.html). This option allows you to use AWS Secrets Manager secrets as Kubernetes secrets that can be accessed by Pods as environment variables or files mounted in the pods. The ASCP also works with [Parameter Store parameters](https://docs.aws.amazon.com/systems-manager/latest/userguide/integrating_csi_driver.html) However, Cloud Posse does not have existing Terraform modules for this solution. We would need to build this support. ### Option 3: SOPS Operator Use [SOPS Operator](https://github.com/isindir/sops-secrets-operator) to manage secrets in Kubernetes. SOPS Operator is a Kubernetes operator that builds on the `sops` project by Mozilla to encrypt the sensitive portions of a `Secrets` manifest into a `SopsSecret` resource, and then decrypt and provision `Secrets` in the Kubernetes cluster. 1. **Mozilla SOPS Encryption**: Mozilla SOPS (Secrets OPerationS) is a tool that encrypts Kubernetes secret manifests, allowing them to be stored securely in Git repositories. SOPS supports encryption using a variety of key management services. Most importantly, it supports AWS KMS which enables IAM capabilities for native integration with AWS. 2. **GitOps-Compatible Secret Management**: In a GitOps setup, storing plain-text secrets in Git poses security risks. Using SOPS, we can encrypt sensitive data in Kubernetes secret manifests while keeping the rest of the manifest in clear text. This allows us to store encrypted secrets in Git, track changes with diffs, and maintain security while benefiting from GitOps practices like version control, auditability, and CI/CD pipelines. 3. **AWS KMS Integration**: SOPS uses AWS KMS to encrypt secrets with customer-managed keys (CMKs), ensuring only authorized users—based on IAM policies—can decrypt them. The encrypted secret manifests can be safely committed to Git, with AWS securely managing the keys. Since it's IAM-based, it integrates seamlessly with STS tokens, allowing secrets to be decrypted inside the cluster without hardcoded credentials. 4. **Kubernetes Operator**: The [SOPS Secrets Operator](https://github.com/isindir/sops-secrets-operator) automates the decryption and management of Kubernetes secrets. It monitors a `SopsSecret` resource containing encrypted secrets. When a change is detected, the operator decrypts the secrets using AWS KMS and generates a native Kubernetes `Secret`, making them available to applications in the cluster. AWS KMS uses envelope encryption to manage the encryption keys, ensuring that secrets remain securely encrypted at rest. 5. **Improved Disaster Recovery and Security**: By storing the source of truth for secrets outside of Kubernetes (e.g., in Git), this setup enhances disaster recovery, ensuring secrets remain accessible even if the cluster is compromised or destroyed. While secrets are duplicated across multiple locations, security is maintained by using IAM for encryption and decryption outside Kubernetes, and Kubernetes' native Role-Based Access Control (RBAC) model for managing access within the cluster. This ensures that only authorized entities, both external and internal to Kubernetes, can access the secrets. The SOPS Operator combines the strengths of Mozilla SOPS and AWS KMS, allowing you to: - Encrypt secrets using KMS keys. - Store encrypted secrets in Git repositories. - Automatically decrypt and manage secrets in Kubernetes using the SOPS Operator. This solution is ideal for teams following GitOps principles, offering secure, external management of sensitive information while utilizing Kubernetes' secret management capabilities. However, the redeployment required for secret rotation can be heavy-handed, potentially leading to a period where services are still using outdated or invalid secrets. This could cause services to fail until the new secrets are fully rolled out. ## Recommendation We recommend using the External Secrets Operator with AWS SSM Parameter Store. This is a well-tested solution that we have used in the past. We have existing Terraform modules to support this solution. However, we are in the process of evaluating the AWS Secrets Manager secrets with Kubernetes Secrets Store CSI Driver solution. This is the AWS supported option and may be a better long-term solution. We will build the required Terraform component to support this solution. ## Consequences We will develop the `eks/secrets-store-csi-driver` component using the [Secrets Store CSI Driver](https://secrets-store-csi-driver.sigs.k8s.io/getting-started/installation) --- ## Review Design Decisions(Design-decisions) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for EKS. These decisions relate to how you will provision your Kubernetes clusters on AWS. --- ## EKS Foundational Platform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Amazon EKS is a managed Kubernetes service that allows you to run Kubernetes in AWS cloud and on-premises data centers. AWS handles the availability and scalability of the Kubernetes control plane, which oversees tasks such as scheduling containers, managing application availability, and storing cluster data. While AWS manages control plane upgrades, users are responsible for the worker nodes and the workloads running on them, including operators, controllers, and applications. We use Karpenter for managing node pools and support spot instances to optimize costs. Be aware that you'll need to upgrade the cluster quarterly due to the significant pace of Kubernetes innovation. Although EKS has a steeper learning curve compared to ECS, it offers greater flexibility and control, making it ideal for organizations already utilizing Kubernetes. ## The Problem Although Amazon EKS is a managed service, there is still much that is needed to set up any given cluster. First of all, we must decide how we want to deploy Nodes for the cluster. EC2 instance backed nodes, Amazon Fargate, or Karpenter all provide solutions for the foundation of a cluster. Next we must provide a method to authenticate with the cluster. Amazon IAM roles can grant API access to the EKS service but do not grant control within Kubernetes. Kubernetes system roles are native to the cluster, but we need to be able to scope finer access of users and resources than what is provided natively. Furthermore, we need to connect each cluster to our network and DNS architecture. Clusters must be secure and protected from the public internet, yet developers still need to be able to connect and manage cluster resources. And finally, we need a place to storage application data. ## Our Solution Cloud Posse deploys EKS through a number of components. Each component has a specific responsibility and works in harmony with the rest. We first deploy a nodeless EKS cluster and create an AWS Auth config mapping. This `ConfigMap` connects our existing AWS Teams architecture to the cluster and allows us to assign Kubernetes roles to a given Team Role. Next we use Karpenter to manage nodes on the cluster. Karpenter automatically launches compute resources to handle cluster applications and provides fast and simple compute provisioning for Kubernetes clusters. We then deploy a set of controllers and operators for the cluster. These controllers will automatically connect the cluster to our network and DNS architecture by annotations and manage storage within the cluster. Simply adding the relevant annotation to a given resources triggers the creation and management of Load Balancers in AWS, adds routing to the relevant Route 53 Hosted Zone, provisions certificates, and more. These resources set the foundation for any application platform. From this foundation, your application will be fully secure, scalable, and resilient. ## References - [Decide on EKS Node Pool Architecture](/layers/eks/design-decisions/decide-on-eks-node-pool-architecture) - [Decide on Kubernetes Ingress Controller(s)](/layers/eks/design-decisions/decide-on-kubernetes-ingress-controller-s) - [How to Load Test in AWS](/learn/maintenance/tutorials/how-to-load-test-in-aws) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to Upgrade EKS Cluster Addons](/learn/maintenance/upgrades/how-to-upgrade-eks-cluster-addons) - [How to Upgrade EKS](/learn/maintenance/upgrades/how-to-upgrade-eks) --- ## FAQ(Eks) import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; Frequently asked questions about EKS with Cloud Posse's reference architecture. ## How can I create secrets for an EKS cluster? Consider deploying the [`external-secrets-operator` component](/components/library/aws/eks/external-secrets-operator). This component creates an external SecretStore configured to synchronize secrets from AWS SSM Parameter store as Kubernetes Secrets within the cluster. Per the operator pattern, the `external-secret-operator` pods will watch for any `ExternalSecret` resources which reference the `SecretStore` to pull secrets from. ## How does the `alb-controller-ingress-group` determine the name of the ALB? 1. First the component uses the [null-label](/modules/library/null/label) module to generate our intended name. We do this to meet the character length restrictions on ALB names. [ref](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/eks/alb-controller-ingress-group/main.tf#L75-L83) 1. Then we pass that output to the Kubernetes Ingress resource with an annotation intended to define the ALB's name. [ref](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/eks/alb-controller-ingress-group/main.tf#L98) 1. Now the Ingress is created and `alb-controller` creates an ALB using the annotations on that `Ingress`. This ALB name will have a dynamic character sequence at the end of it, so we cannot know what the name will be ahead of time. 1. Finally, we grab the actual name that is given to the created ALB with the `data.aws_lb` resources. [ref](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/eks/alb-controller-ingress-group/main.tf#L169) 1. Then output that name for future reference. [ref](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/eks/alb-controller-ingress-group/main.tf#L36) ## How can we create Self-Hosted Runners for GitHub with EKS? Self-Hosted Runners are a great way to save cost and add customizations with GitHub Actions. Since we've already implemented EKS for our platform, we can build off that foundation to create another cluster to manage Self-Hosted runners in GitHub. We deploy that new EKS cluster to `core-auto` and install the [Actions Runner Controller (ARC) chart](https://github.com/actions/actions-runner-controller). This controller will launch and scale runners for GitHub automatically. For more on how to set up ARC, see the [GitHub Action Runners setup docs for EKS](/layers/github-actions/eks-github-actions-controller/). ## Common Connectivity Issues and Solutions If you're having trouble connecting to your EKS cluster, follow these comprehensive steps to diagnose and resolve the issue: **1. Test Basic Connectivity** First, test basic connectivity to your cluster endpoint. This helps isolate whether the issue is with basic network connectivity or something more specific: ```bash curl -fsSk --max-time 5 "https://CLUSTER_ENDPOINT/healthz" ``` If these tests fail, it indicates a fundamental connectivity issue that needs to be addressed before proceeding to more specific troubleshooting. **2. Check Node Communication** If worker nodes aren't joining the cluster, follow these detailed steps: - Verify that the addon stack file (e.g., `stacks/catalog/eks/mixins/k8s-1-29.yaml`) is imported into your stack. - Verify cluster add-ons are properly configured for your EKS version. - Check CoreDNS is running - Verify kube-proxy is deployed - Ensure VPC CNI is correctly configured - Confirm the rendered component stack configuration. ```bash atmos describe component eks/cluster -s ``` **3. Verify Network Configuration** - Security Groups: - Control plane security group must allow port 443 inbound from worker nodes - Worker node security group must allow all traffic between nodes - Verify outbound internet access for pulling container images - Subnet Routes: - Verify route tables have paths to all required destinations - Check for conflicting or overlapping CIDR ranges - Ensure NAT Gateway is properly configured for private subnets - Transit Gateway: - Verify TGW attachments are active and associated - Check TGW route tables for correct propagation - Confirm cross-account routing if applicable - Private Subnets Configuration: - Set `cluster_private_subnets_only: true` in your configuration - Ensure private subnets have proper NAT Gateway routing **4. VPN Connectivity** When accessing via AWS Client VPN, verify these configurations: - VPN Routes: - Check route table entries for EKS VPC CIDR - Verify routes are active and not in pending state - Confirm no conflicting routes exist - Subnet Associations: - Ensure VPN endpoint is associated with correct subnets - Verify subnet route tables include VPN CIDR range - Authorization Rules: - Check network ACLs allow VPN CIDR range - Verify security group rules permit VPN traffic - Confirm IAM roles have necessary permissions After making any changes, have clients disconnect and reconnect to receive updated routes. **5. Advanced Diagnostics** - AWS Reachability Analyzer: - Enable cross-account analysis for VPC peering or TGW connections - Test from VPN ENI to cluster endpoint - Test return path from cluster to VPN ENI --- ## EKS as a Foundational Platform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; We first deploy the foundation for the cluster. The `eks/cluster` component deploys the initial EKS resources to AWS, including Auth Config mapping. We do not deploy any nodes with the cluster initially. Then once EKS is available, we connect to the cluster and start deploying resources. First is Karpenter. We deploy the Karpenter chart on a Fargate node and the IAM service role to allow Karpenter to purchase Spot Instances. Karpenter is the only resources that will be deployed to Fargate. Then we deploy Karpenter Node Pools using the CRD created by the initial Karpenter component. These provisioners will automatically launch and scale the cluster to meet our demands. Next we deploy `idp-roles` to manage custom roles for the cluster, and deploy `metrics-server` to provide access to resource metrics. Then we connect the cluster to our network. First we must deploy the `cert-manager` component to provision X.509 certificates on the cluster. Then we deploy the `alb-controller` component to provision and associate ALBs or NLBs based on `Ingress` annotations that route traffic to the cluster. Then we deploy the `alb-controller-ingress-group` to actually create that ALB. Next, we deploy `external-dns` which will look for annotations to make services discoverable, and then create records in our Route 53 Hosted Zones mapping to the cluster. Finally we deploy `echo-server` to validate the complete setup. :::info Connecting to an EKS cluster requires a VPN connection! See [ec2-client-vpn](/components/library/aws/ec2-client-vpn/) for details. ::: Depending on your application requirements we can also deploy a number of operators. The most common is the `efs-controller`, which we use to provide encrypted block storage that is not zone-locked. Other operators are optionally but often include the `external-secrets-operator` to automatically sync secrets from AWS SSM Parameter Store. Monitoring and release engineering are handled separately from the components mentioned here, and we will expand of those implementations in follow up topics. For details, see the [Monitoring](/layers/monitoring/) and [Release Engineering](/layers/software-delivery/fundamentals/) quick start documents. #### Foundation - [`eks/cluster`](/components/library/aws/eks/cluster/): This component is responsible for provisioning an end-to-end EKS Cluster, including IAM role to Kubernetes Auth Config mapping. - [`eks/karpenter`](/components/library/aws/eks/karpenter-controller/): Installs the Karpenter chart on the EKS cluster and prepares the environment for provisioners. - [`eks/karpenter-provisioner`](/components/library/aws/eks/karpenter-node-pool/): Deploys Karpenter Node Pools using CRDs made available by `eks/karpenter` - [`iam-service-linked-roles`](/components/library/aws/iam-service-linked-roles/): Provisions [IAM Service-Linked](https://docs.aws.amazon.com/IAM/latest/UserGuide/using-service-linked-roles.html) roles. These are required for Karpenter to purchase Spot instances. - [`idp-roles`](/components/library/aws/eks/idp-roles): These identity provider roles specify several pre-determined permission levels for cluster users and come with bindings that make them easy to assign to Users and Groups. Use this component to define custom permission within EKS. - [`metrics-server`](/components/library/aws/eks/metrics-server): A Kubernetes addon that provides resource usage metrics used in particular by other addons such Horizontal Pod Autoscaler. For more, see [metrics-server](https://github.com/kubernetes-sigs/metrics-server). - [`reloader`](/components/library/aws/eks/reloader): Installs the [Stakater Reloader](https://github.com/stakater/Reloader) for EKS clusters. `reloader` can watch `ConfigMaps` and `Secrets` for changes and use these to trigger rolling upgrades on pods and their associated `DeploymentConfigs`, `Deployments`, `Daemonsets` `Statefulsets` and `Rollouts`. #### Network - [`cert-manager`](/components/library/aws/eks/cert-manager): A Kubernetes addon that provisions X.509 certificates. - [`alb-controller`](/components/library/aws/eks/alb-controller): A Kubernetes addon that, in the context of AWS, provisions and manages ALBs and NLBs based on `Service` and `Ingress` annotations. This module is also provision a default `IngressClass`. - [`alb-controller-ingress-group`](/components/library/aws/eks/alb-controller-ingress-group): A Kubernetes Service that creates an ALB for a specific `IngressGroup`. An `IngressGroup` is a feature of the `alb-controller` which allows multiple Kubernetes Ingresses to share the same Application Load Balancer. - [`external-dns`](/components/library/aws/eks/external-dns): A Kubernetes addon that configures public DNS servers with information about exposed Kubernetes services to make them discoverable. This component is responsible for adding DNS records to your Route 53 Hosted Zones. - [`echo-server`](/components/library/aws/eks/echo-server): The echo server is a server that sends it back to the client a JSON representation of all the data the server received. We use this component is validate a cluster deployment. #### Storage - [`efs`](/components/library/aws/efs/): Deploys an [EFS](https://aws.amazon.com/efs/) Network File System with KMS encryption-at-rest. EFS is an excellent choice as the default block storage for EKS clusters so that volumes are not zone-locked. - [`eks/efs-controller`](/components/library/aws/eks/storage-class/): Deploys [the Amazon Elastic File System Container Storage Interface (CSI) Driver controller](https://github.com/kubernetes-sigs/aws-efs-csi-driver) to EKS. The Amazon EFS CSI Driver implements the CSI specification for container orchestrators to manage the lifecycle of Amazon EFS file systems. #### Additional Operators - [`external-secrets-operator`](/components/library/aws/eks/external-secrets-operator/): This component (ESO) is used to create an external `SecretStore` configured to synchronize secrets from AWS SSM Parameter store as Kubernetes Secrets within the cluster. --- ## How to Setup Vanity Domains with an ALB on EKS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; Learn how to setup vanity domains on an existing ALB together with Service Discovery Domains. ## Pre-requisites - [Understand the differences between Vanity Domains and Service Discovery Domains](/resources/legacy/learning-resources/#the-difference-between-vanity-domains-and-service-discovery-domains) - Assumes our standard [Network Architecture](/layers/network/) - Requires `dns-primary` & `dns-delegated` are already deployed. ## Context After setting up your [Network Architecture](/layers/network) you will have 2 hosted zones in each platform account. In `dev` for example, you will have Hosted Zones for `dev-acme.com` and `dev.platform.acme.com`. You should also have an ACM certificate that registers `*.dev-acme.com` and `*.dev.platform.acme.com`. We also should've deployed applications to your EKS cluster and have an ALB for service discovery. For example the [`echo-server`](/components/library/aws/eks/echo-server) component. Now we want to set up a vanity subdomain for `dev-acme.com` that will point to the ALB used for service discovery. This saves us money by not requiring a new ALB for each vanity domain. ## Implementation This is fairly simple to implement. All we need to do is set up our Kubernetes ingresses and ensure ACM doesn't have duplicate certs for domains. ## Setup Ingresses Ingresses for your applications can use several different `.spec.rules` to provide access to the application via many different URLs. #### Example ```yaml apiVersion: networking.k8s.io/v1 kind: Ingress metadata: annotations: alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/group.name: alb-controller-ingress-group alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS":443}]' alb.ingress.kubernetes.io/scheme: internet-facing alb.ingress.kubernetes.io/ssl-redirect: "443" alb.ingress.kubernetes.io/target-type: ip external-dns.alpha.kubernetes.io/hostname: my-app-api.dev.plat.acme-svc.com kubernetes.io/ingress.class: alb outputs.platform.cloudposse.com/webapp-url: https://my-app-api.dev.plat.acme-svc.com name: my-app-api namespace: dev spec: rules: # new Vanity Domain - host: api.dev-acme.com http: paths: - backend: service: name: my-app-api port: number: 8081 path: /api/* pathType: ImplementationSpecific # Existing Service discovery domain - host: my-app-api.dev.plat.acme-svc.com http: paths: - backend: service: name: my-app-api port: number: 8081 path: /* pathType: ImplementationSpecific ```
_helpers.tpl ```yaml {{/* Expand the name of the chart. */}} {{- define "this.name" -}} {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} {{- end }} {{/* Create a default fully qualified app name. We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). If release name contains chart name it will be used as a full name. */}} {{- define "this.fullname" -}} {{- if .Values.fullnameOverride }} {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} {{- else }} {{- $name := default .Chart.Name .Values.nameOverride }} {{- if contains $name .Release.Name }} {{- .Release.Name | trunc 63 | trimSuffix "-" }} {{- else }} {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} {{- end }} {{- end }} {{- end }} {{/* Create chart name and version as used by the chart label. */}} {{- define "this.chart" -}} {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} {{- end }} {{/* Common labels helm.sh/chart: {{ include "this.chart" . }} {{- if .Chart.AppVersion }} app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} {{- end }} app.kubernetes.io/managed-by: {{ .Release.Service }} */}} {{- define "this.labels" -}} {{ include "this.selectorLabels" . }} {{- end }} {{/* Selector labels app.kubernetes.io/name: {{ include "this.name" . }} app.kubernetes.io/instance: {{ .Release.Name }} */}} {{- define "this.selectorLabels" -}} app: {{ include "this.fullname" . }} {{- end }} {{/* Create the name of the service account to use */}} {{- define "this.serviceAccountName" -}} {{- if .Values.serviceAccount.create }} {{- default (include "this.fullname" .) .Values.serviceAccount.name }} {{- else }} {{- default "default" .Values.serviceAccount.name }} {{- end }} {{- end }} ```
**Ingress.yaml** ```yaml {{- if or (eq (printf "%v" .Values.ingress.nginx.enabled) "true") (eq (printf "%v" .Values.ingress.alb.enabled) "true") -}} {{- $fullName := include "this.fullname" . -}} {{- $svcName := include "this.name" . -}} {{- $svcPort := .Values.service.port -}} {{- $nginxTlsEnabled := and (eq (printf "%v" .Values.ingress.nginx.enabled) "true") (eq (printf "%v" .Values.tlsEnabled) "true")}} apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: {{ $fullName }} annotations: {{- if eq (printf "%v" .Values.ingress.nginx.enabled) "true" }} kubernetes.io/ingress.class: {{ .Values.ingress.nginx.class }} {{- if (index .Values.ingress.nginx "tls_certificate_cluster_issuer") }} cert-manager.io/cluster-issuer: {{ .Values.ingress.nginx.tls_certificate_cluster_issuer }} {{- end }} {{- else if eq (printf "%v" .Values.ingress.alb.enabled) "true" }} kubernetes.io/ingress.class: {{ .Values.ingress.alb.class }} alb.ingress.kubernetes.io/group.name: {{ .Values.default_alb_ingress_group | default "alb-controller-ingress-group" }} alb.ingress.kubernetes.io/scheme: internet-facing {{- if .Values.ingress.alb.access_logs.enabled }} alb.ingress.kubernetes.io/load-balancer-attributes: access_logs.s3.enabled=true,access_logs.s3.bucket={{.Values.ingress.alb.access_logs.s3_bucket_name}},access_logs.s3.prefix={{.Values.ingress.alb.access_logs.s3_bucket_prefix}} {{- end }} alb.ingress.kubernetes.io/target-type: 'ip' {{- if eq (printf "%v" .Values.ingress.alb.ssl_redirect.enabled) "true" }} alb.ingress.kubernetes.io/ssl-redirect: '{{ .Values.ingress.alb.ssl_redirect.port }}' {{- end }} {{- if eq (printf "%v" .Values.tlsEnabled) "true" }} alb.ingress.kubernetes.io/backend-protocol: HTTP alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS":443}]' {{- else }} alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]' {{- end }} {{- if eq .Values.environment "preview" }} external-dns.alpha.kubernetes.io/hostname: {{ $svcName }}-{{ .Release.Namespace }}.{{ .Values.platform.default_ingress_domain }} outputs.platform.cloudposse.com/webapp-url: "https://{{ $svcName }}-{{ .Release.Namespace }}.{{ .Values.platform.default_ingress_domain }}" {{- else }} external-dns.alpha.kubernetes.io/hostname: {{ $svcName }}.{{ .Values.platform.default_ingress_domain }} outputs.platform.cloudposse.com/webapp-url: "https://{{ $svcName }}.{{ .Values.platform.default_ingress_domain }}" {{- end }} {{- end }} labels: {{- include "this.labels" . | nindent 4 }} spec: {{- if $nginxTlsEnabled }} tls: # < placing a host in the TLS config will indicate a certificate should be created - hosts: - {{ .Values.ingress.hostname }} secretName: {{ $svcName }}-cert # < cert-manager will store the created certificate in this secret. {{- end }} rules: {{- if eq .Values.environment "preview" }} - host: "{{ $svcName }}-{{ .Release.Namespace }}.{{ .Values.platform.default_ingress_domain }}" {{- else }} {{- range .Values.ingress.vanity_domains }} - host: "{{.prefix | default "api" }}.{{ $.Values.platform.default_vanity_domain }}" http: paths: - path: /{{.path | default "*" }} pathType: ImplementationSpecific backend: service: name: {{ $svcName }} port: number: {{ $svcPort }} {{- end }} - host: "{{ $svcName }}.{{ .Values.platform.default_ingress_domain }}" {{- end }} http: paths: - path: /* pathType: ImplementationSpecific backend: service: name: {{ $svcName }} port: number: {{ $svcPort }} {{- end }} ``` **values.yaml** ```yaml --- ingress: vanity_domains: # api.dev-acme.com, path: /* - prefix: "api" # api.dev-acme.com, path: /v2/* - prefix: "api" path: "v2/*" nginx: # ingress.nginx.enabled -- Enable NGiNX ingress enabled: false # annotation values ## kubernetes.io/ingress.class: class: "nginx" ## cert-manager.io/cluster-issuer: tls_certificate_cluster_issuer: "letsencrypt-prod" alb: enabled: true # annotation values ## kubernetes.io/ingress.class: class: "alb" ## alb.ingress.kubernetes.io/load-balancer-name: ### load_balancer_name: "k8s-common" ## alb.ingress.kubernetes.io/group.name: ### group_name: "common" ssl_redirect: enabled: true ## alb.ingress.kubernetes.io/ssl-redirect: port: 443 access_logs: enabled: false ## s3_bucket_name: "acme-ue2-prod-eks-cluster-alb-access-logs" s3_bucket_prefix: "" ```
## Setup ACM Certs By default, our `dns-primary` component and `dns-delegated` component will create ACM certs for each Hosted Zone in the platform account, along with an **additional** cert for `*.dev-acme.com`. Depending on the level of subdomains you want, you may need to disable this with the variable `request_acm_certificate: false` If a single subdomain is sufficient. e.g. `api.dev-acme.com` then you can leave this enabled. The important thing to note is that you **cannot** have duplicate certs in ACM. So if you want to add a new subdomain, you will need to delete the existing cert for `*.dev-acme.com` and create a new one with the new subdomain. This can lead to issues when trying to delete certificates, as they are in use by the ALB. You will need to delete the ALB first, then delete the certificate. See the troubleshooting section if you run into issues with recreating resources.
## How it works: With a single valid ACM cert for your domains, the `alb-controller` is able to register your domain to the ALB. The ALB is able to do this by recognizing the valid certificate in ACM. This is why we need to ensure we have a valid certificate for our domains. You can validate your cert is picked up by the ALB by checking the ALB's target group. You should see the certificate listed under the `Certificates` tab. ## Troubleshooting The problem with this comes when you need to remove a subdomain or ACM certificate. By running `atmos terraform deploy dns-delegated -s plat--dev` with `request_acm_certificate: false`, you are trying to destroy a single ACM certificate in an account. While this is a small scope deletion, the ACM certificate is in use by the ALB, and the ALB has many different targets. Thus Terraform will stall out. You need to: 1. Delete the listeners and targets of the ALB that are using the certificate 2. Delete the ALB 3. Terraform will then successfully delete the ACM certificate. You will notice: 1. The ALB will be recreated 2. Ingresses should reconcile for service discovery domains 3. ALB Targets should be recreated pointing at service discovery domains. Once you recreate the correct ACM certificates and have valid ingresses you should be able to access your applications via the vanity domain. --- ## Tutorials(5) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the associated EKS components. --- ## Build Your Foundation import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; import Step from '@site/src/components/Step'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import ReactPlayer from 'react-player'; To build a reliable infrastructure, we must start with a solid foundation. Our reference architecture is designed with best practices and consistent conventions to ensure it is well-architected from the ground up. As part of this process, you’ll make critical design decisions that will shape your infrastructure. Next, you’ll initialize your infrastructure repository and then begin by provisioning your AWS Organizations, accounts, networks, DNS, and fine-grained IAM roles and policies. Once your foundation is complete, you’ll be ready to build a platform to deliver your applications. ## Set up your project 1. Create a GitHub repository to host your infrastructure toolchain and configurations. 2. Configure repository settings, enable branch protection, and add collaborators. 3. Then import the Cloud Posse reference architecture and prepare the Geodesic toolbox image to get ready to provision your infrastructure.
AI generated voice
Get Started
## Provision New AWS Organization and Accounts 1. Review how Cloud Posse designs and manages AWS Account architectures using Atmos and Terraform, aligning with the AWS Well-Architected Framework. 2. Begin by provisioning the Terraform state backend, which is essential before provisioning and managing infrastructure with Terraform. 3. Then proceed to organize the accounts into Organizational Units (OUs), apply Service Control Policies (SCPs), and configure account-level settings.
AI generated voice
Get Started
## Rollout Identity & Authentication Learn how Cloud Posse sets up fine-grained access control for an entire organization using IAM roles, AWS SAML, and AWS IAM Identity Center (SSO). It addresses the challenges of using various login methods and tools and introduces a solution involving Teams and Team Roles to manage access across multiple AWS accounts. This approach ensures precise control, easy role switching, and compatibility with different identity providers. Additionally, we provide a solution optimized for cross-account Terraform access, programmatic access for GitHub OIDC, and a user-friendly login experience with AWS Identity Center (AWS SSO), using tools like Leapp to facilitate seamless authentication and access management.
AI generated voice
Get Started
## Deploy VPCs & DNS Finally, understand Cloud Posse’s approach to designing robust and scalable Network and DNS architectures on AWS, with a focus on symmetry, account-level isolation, security, and reusability. We cover essential topics such as account isolation, connecting multiple accounts together using Transit Gateways, deploying AWS Client VPN for remote network access by developers, and differentiating between DNS service discovery and branded vanity domains used by customers. The solution includes reusable network building blocks, ensuring consistent deployment of VPCs and subnets, accommodating multi-region global networks, and addressing special network design considerations depending on whether you'll use ECS or EKS.
AI generated voice
Get Started
When you're done with your foundation, our attention will shift to how you [set up your platform](/layers/platform) to deliver your apps. --- ## Decide on Self-Hosted Runner Architecture import Intro from "@site/src/components/Intro"; import Note from "@site/src/components/Note"; Decide on how to operate self-hosted runners that are used to run GitHub Actions workflows. These runners can be set up in various ways and allow us to avoid platform fees while running CI jobs in private infrastructure, enabling access to VPC resources. This approach is ideal for private repositories, providing control over instance size, architecture, and control costs by leveraging spot instances. The right choice depends on your platform, whether you’re using predominantly EKS, ECS, or Lambda. ## Problem When using GitHub Actions, you can opt for both GitHub Cloud-hosted and self-hosted runners, and they can complement each other. In some cases, self-hosted runners are essential—particularly for accessing resources within a VPC, such as databases, Kubernetes API endpoints, or Kafka servers, which is common in GitOps workflows. However, while self-hosted runners are ideal for private infrastructure, they pose risks in public or open-source repositories due to potential exposure of sensitive resources. If your organization maintains open-source projects, this should be a critical consideration, and we recommend using cloud-hosted runners for those tasks. The hosting approach for self-hosted runners should align with your infrastructure. If you use Kubernetes, it's generally best to run your runners on Kubernetes. Conversely, if your infrastructure relies on ECS or Lambdas, you may want to avoid unnecessary Kubernetes dependencies and opt for alternative hosting methods. In Kubernetes-based setups, configuring node pools with Karpenter is key to maintaining stability and ensuring effective auto-scaling with a mix of spot and on-demand instances. However, tuning this setup can be challenging, especially with recent changes to ARC, where the [newer version does not support multiple labels for runner groups](https://github.com/actions/actions-runner-controller/issues/2445), leading to community disagreement over trade-offs. We provide multiple deployment options for self-hosted runners, including EKS, Philips Labs' solution, and Auto Scaling Groups (ASG), tailored to your specific runner management needs. ## Considered Options ### Option 1: EC2 Instances in an Auto Scaling Group (`github-runners`) The first option is to deploy EC2 instances in an Auto Scaling Group. This is the simplest option. We can use the `github-runners` component to deploy the runners. However, this option is not as scalable as the other options. ### Option 2: Actions Runner Controller on EKS (`eks/actions-runner-controller`) The second option is to deploy the Actions Runner Controller on EKS. Since many implementations already have EKS, this option is a good choice to reuse existing infrastructure. We can use the `eks/actions-runner-controller` component to deploy the runners, which is built with the [Actions Runner Controller helm chart](https://github.com/actions/actions-runner-controller). ### Option 3: GitHub Actions Runner on EKS (`eks/github-actions-runner`) Alternatively, we can deploy the GitHub Actions Runner on EKS. This option is similar to the previous one, but it uses the GitHub Actions Runner instead of the Actions Runner Controller. This component deploys self-hosted GitHub Actions Runners and a [Controller](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/quickstart-for-actions-runner-controller#introduction) on an EKS cluster, using "[runner scale sets](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#runner-scale-set)". This solution is supported by GitHub and supersedes the [actions-runner-controller](https://github.com/actions/actions-runner-controller/blob/master/docs/about-arc.md) developed by Summerwind and deployed by Cloud Posse's [actions-runner-controller](https://docs.cloudposse.com/components/library/aws/eks/actions-runner-controller/) component. However, there are some limitations to the official Runner Sets implementation: - #### Limited set of packages The runner image used by Runner Sets contains [no more packages than are necessary](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#about-the-runner-container-image) to run the runner. This is in contrast to the Summerwind implementation, which contains some commonly needed packages like `build-essential`, `curl`, `wget`, `git`, and `jq`, and the GitHub hosted images which contain a robust set of tools. (This is a limitation of the official Runner Sets implementation, not this component per se.) You will need to install any tools you need in your workflows, either as part of your workflow (recommended), by maintaining a [custom runner image](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/about-actions-runner-controller#creating-your-own-runner-image), or by running such steps in a [separate container](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container) that has the tools pre-installed. Many tools have publicly available actions to install them, such as `actions/setup-node` to install NodeJS or `dcarbone/install-jq-action` to install `jq`. You can also install packages using `awalsh128/cache-apt-pkgs-action`, which has the advantage of being able to skip the installation if the package is already installed, so you can more efficiently run the same workflow on GitHub hosted as well as self-hosted runners. There are (as of this writing) open feature requests to add some commonly needed packages to the official Runner Sets runner image. You can upvote these requests [here](https://github.com/actions/actions-runner-controller/discussions/3168) and [here](https://github.com/orgs/community/discussions/80868) to help get them implemented. - #### Docker in Docker (dind) mode only In the current version of this component, only "dind" (Docker in Docker) mode has been tested. Support for "kubernetes" mode is provided, but has not been validated. - #### Limited configuration options Many elements in the Controller chart are not directly configurable by named inputs. To configure them, you can use the `controller.chart_values` input or create a `resources/values-controller.yaml` file in the component to supply values. Almost all the features of the Runner Scale Set chart are configurable by named inputs. The exceptions are: - There is no specific input for specifying an outbound HTTP proxy. - There is no specific input for supplying a [custom certificate authority (CA) certificate](https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/deploying-runner-scale-sets-with-actions-runner-controller#custom-tls-certificates) to use when connecting to GitHub Enterprise Server. You can specify these values by creating a `resources/values-runner.yaml` file in the component and setting values as shown by the default Helm [values.yaml](https://github.com/actions/actions-runner-controller/blob/master/charts/gha-runner-scale-set/values.yaml), and they will be applied to all runners. - #### Component limitations Furthermore, the Cloud Posse component has some additional limitations. In particular: - The controller and all runners and listeners share the Image Pull Secrets. You cannot use different ones for different runners. - All the runners use the same GitHub secret (app or PAT). Using a GitHub app is preferred anyway, and the single GitHub app serves the entire organization. - Only one controller is supported per cluster, though it can have multiple replicas. These limitations could be addressed if there is demand. Contact [Cloud Posse Professional Services](https://cloudposse.com/professional-services/) if you would be interested in sponsoring the development of any of these features. ### Option 4: Philips Labs Runners (`philips-labs-github-runners`) If we are not deploying EKS, it's not worth the additional effort to set up Self-Hosted runners on EKS. Instead, we deploy Self-Hosted runners on EC2 instances. These are managed by an API Gateway and Lambda function that will automatically scale the number of runners based on the number of pending jobs in the queue. The queue is written to by the API Gateway from GitHub Events. For more on this option, see the [Philips Labs GitHub Runner](https://philips-labs.github.io/terraform-aws-github-runner/) documentation. ### Option 5: Managed Runners There are a number of services that offer managed runners. These still have the advantage over GitHub Cloud hosted runners as the can be managed within you private VPCs. One option to consider is [runs-on.com](https://runs-on.com/) which provides a very inexpensive option. ## Recommendation At this time Cloud Posse recommends the Actions Runner Controller on EKS (`eks/actions-runner-controller`) if you are using EKS and the Philips Labs Runners (`philips-labs-github-runners`) if you are not using EKS. --- ## Decide on Self-Hosted Runner Placement import Intro from "@site/src/components/Intro"; Self-hosted runners are custom runners that we use to run GitHub Actions workflows. We can use these runners to access resources in our private networks and reduce costs by using our own infrastructure. We need to decide where to place these runners in your AWS organization. ## Problem We need to decide where to place self-hosted runners in your AWS organization. We support multiple options for deploying self-hosted runners. We can deploy runners with EKS, Philips Labs, or with an ASG. For this decision, we will focus on the placement of the runners themselves. ## Considered Options ### Option 1: Deploy the runners in an `auto` account The first option is to deploy the controller in the `auto` (Automation) account. This account would be dedicated to automation tasks and would have access to all other accounts. We can use this account to deploy the controller and manage the runners in a centralized location. However, compliance is complicated because the `auto` cluster would have access to all environments. ### Option 2: Deploy the runners in each account The second option is to deploy the controller in each account. This option sounds great from a compliance standpoint. Jobs running in each account are scoped to that account, each account has its own controller, and we can manage the runners independently. This might seem like a simplification from a compliance standpoint, but it creates complexity from an implementation standpoint. We would need to carefully consider the following: 1. Scaling runners can inadvertently impact IP space available to production workloads 2. Many accounts do not have a VPC or EKS Cluster (for EKS/ARC solutions). So, we would need to decide how to manage those accounts. 3. We would need to manage the complexity of dynamically selecting the right runner pool when a workflow starts. While this might seem straightforward, it can get tricky in cases like promoting an ECR image from staging to production, where it’s not always clear-cut which runners should be used. ## Recommendation _Option 1: Deploy the runners in an `auto` account_ We will deploy the runners in an `auto` account. This account will be connected to the private network and will have access to all other accounts where necessary. This will simplify the management of the runners and ensure that they are available when needed. ## Consequences We will create an `auto` account and deploy the runners there. --- ## Design Decisions(3) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions of the GitHub Action Layer. These decisions relate to how you will manage self-hosted runners for your GitHub Action workflows. --- ## [EKS] GitHub Action Runner Controller import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; The GitHub Action Runner Controller (ARC) is a Kubernetes operator that automates the management of self-hosted GitHub Actions runners in a Kubernetes cluster, that works very well together with Karpenter for EKS. We recommend using ARC only if your organization uses Kubernetes, otherwise we recommend the [Philips Labs GitHub Runner](/layers/github-actions/philips-labs-github-runners) approach instead. By default, GitHub Actions are run in the cloud on hosted machines, but we can opt to use "Self-Hosted" GitHub Action Runners instead. Historically, we've deployed an Auto Scaling Group that gives each run a dedicated and customized instance. Now that we've deployed EKS, we can save money by utilizing the `actions-runner-controller` to deploy virtual-machines inside of EKS, and run GitHub Actions from these containers. These virtual-machines will be fully customizable, scale automatically, and be cheaper than both GitHub hosted runners and ASG instances. ## Quick Start | Steps | Example | | :---------------------------------------------------- | :------------------------------------------------------------------------------------ | | 1. Generate GitHub Private Key | `ssm_github_secret_path: "/github_runners/controller_github_app_secret"` | | 2. Generate GitHub Webhook Secret Token | `ssm_github_webhook_secret_token_path: "/github_runners/github_webhook_secret_token"` | | 3. Connect to the VPN | | | 4. Deploy cluster and resources into the `auto` stack | `atmos workflow deploy/github-runners -f quickstart/foundation/github` | | 5. Set up Webhook Driven Scaling | Click Ops | ## Requirements In order to deploy Self-Hosted GitHub Runners on EKS, follow the steps outlined in the [EKS setup doc](/layers/eks). Those steps will complete the EKS requirements. - We'll begin by generating the required secrets, which is a manual process. - AWS SSM will be used to store and retrieve secrets. - Then we need to decide on the SSM path for the GitHub secret (Application private key) and GitHub webhook secret. ### GitHub Application Private Key Since the secret is automatically scoped by AWS to the account and region where the secret is stored, we recommend the secret be stored at `/github/acme/github_token`. `stacks/catalog/eks/actions-runner-controller.yaml`: ```yaml ssm_github_secret_path: "/github_runners/controller_github_app_secret" ``` The preferred way to authenticate is by _creating_ and _installing_ a GitHub App. This is the recommended approach as it allows for more much more restricted access than using a personal access token, at least until [fine-grained personal access token permissions](https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/) are generally available. Follow the instructions [here](https://github.com/actions/actions-runner-controller/blob/master/docs/authenticating-to-the-github-api.md) to create and install the GitHub App. At the creation stage, you will be asked to generate a private key. This is the private key that will be used to authenticate the Action Runner Controller. Download the file and store the contents in SSM using the following command, adjusting the profile and file name. The profile should be the `admin` role in the account to which you are deploying the runner controller. The file name should be the name of the private key file you downloaded. ``` AWS_PROFILE=acme-core-use1-auto-admin chamber write github_runners controller_github_app_secret -- "$(cat APP_NAME.DATE.private-key.pem)" ``` You can verify the file was correctly written to SSM by matching the private key fingerprint reported by GitHub with: ``` AWS_PROFILE=acme-core-use1-auto-admin chamber read -q github_runners controller_github_app_secret | openssl rsa -in - -pubout -outform DER | openssl sha256 -binary | openssl base64 ``` At this stage, record the Application ID and the private key fingerprint in your secrets manager (e.g. 1Password). You will need the Application ID to configure the runner controller, and want the fingerprint to verify the private key. Proceed to install the GitHub App in the organization or repository you want to use the runner controller for, and record the Installation ID (the final numeric part of the URL, as explained in the instructions linked above) in your secrets manager. You will need the Installation ID to configure the runner controller. In your stack configuration, set the following variables, making sure to quote the values so they are treated as strings, not numbers. ``` github_app_id: "12345" github_app_installation_id: "12345" ``` ### GitHub Webhook Secret Token If using the Webhook Driven autoscaling (recommended), generate a random string to use as the Secret when creating the webhook in GitHub. Generate the string using 1Password (no special characters, length 45) or by running ```bash dd if=/dev/random bs=1 count=33 2>/dev/null | base64 ``` Store this key in AWS SSM under the same path specified by `ssm_github_webhook_secret_token_path` `stacks/catalog/eks/actions-runner-controller.yaml`: ```yaml ssm_github_webhook_secret_token_path: "/github_runners/github_webhook_secret_token" ``` ## Deploy Automation has an unique set of components from the `plat` clusters and therefore has its own Atmos Workflow. Notably, `auto` includes the `eks/actions-runner-controller` component, which is used to create the `self-hosted` runners for the GitHub Repository or Organization The first three steps before are all included in the following workflow: ### `iam-service-linked-roles` Component At this point we assume that the `iam-service-linked-roles` component is already deployed for `core-auto`. If not, deploy this component now with the following command: ```bash atmos terraform apply iam-service-linked-roles -s core-gbl-auto ``` ### Deploy Automation Cluster and Resources Deploy the cluster with the same commands as `plat` cluster deployments: Validate the `core-auto` deployment using Echo Server. For example: https://echo.use1.auto.core.acme-svc.com/ ### Deploy the Actions Runner Controller Finally, deploy the `actions-runner-controller` component with the following command: ```bash atmos terraform deploy eks/actions-runner-controller -s core-use1-auto ``` ### Using Webhook Driven Autoscaling (Click Ops) To use the Webhook Driven autoscaling, you must also install the GitHub organization-level webhook after deploying the component (specifically, the webhook server). The URL for the webhook is determined by the `webhook.hostname_template` and where it is deployed. Recommended URL is `https://gha-webhook.[environment].[stage].[tenant].[service-discovery-domain]`, which for this organization would be `https://gha-webhook.use1.auto.core.acme-svc.com` As a GitHub organization admin, go to `https://github.com/organizations/acme/settings/hooks`, and then: - Click "Add webhook" and create a new webhook with the following settings: - Payload URL: copy from Terraform output `webhook_payload_url` - Content type: `application/json` - Secret: whatever you configured in the secret above - Which events would you like to trigger this webhook: - Select "Let me select individual events" - Uncheck everything ("Pushes" is likely the only thing already selected) - Check "Workflow jobs" - Ensure that "Active" is checked (should be checked by default) - Click "Add webhook" at the bottom of the settings page After the webhook is created, select "edit" for the webhook and go to the "Recent Deliveries" tab and verify that there is a delivery (of a "ping" event) with a green check mark. If not, verify all the settings and consult the logs of the `actions-runner-controller-github-webhook-server` pod. # Related Topics - [EKS Documentation](/layers/eks/) - [Decision on Self Hosted GitHub Runner Strategy](/layers/software-delivery/design-decisions/decide-on-self-hosted-github-runner-strategy#self-hosted-runners-on-kubernetes) - [Karpenter Documentation](https://karpenter.sh/) --- ## Setup GitHub Actions import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; GitHub Actions (GHA) are one of the cornerstones of your platform, automating everything from Terraform with Atmos to application build, test and deployment, fully integrated into AWS without any hardcoded, static credentials. GitHub Actions offer a convenient way to achieve CI/CD automation directly on GitHub, without additional third-party services (e.g. CircleCI or Jenkins). GitHub doesn't charge extra for self-hosting runners, unlike many other platforms, making them an ideal choice for automation. Using self-hosted runners allows them to reside within your private networks, enabling you to manage resources like databases and Kubernetes clusters in private VPCs without exposing them publicly. We'll show you how to set up self-hosted runners (which are optional but recommended) and configure your IAM architecture to work with GitHub OIDC, so your Actions and workflows can assume AWS roles without relying on static credentials. ## Getting Started --- ## How to use GitHub OIDC with AWS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; This is a detailed guide on how to integrate GitHub OpenID Connect (OIDC) with AWS to facilitate secure and efficient authentication and authorization for GitHub Actions, without the need for permanent (static) AWS credentials, thereby enhancing security and simplifying access management. First we explaini the concept of OIDC, illustrating its use with AWS, and then provide the step-by-step instructions for setting up GitHub as an OIDC provider in AWS. GitHub OIDC (OpenID Connect) for AWS refers to the integration between GitHub as an identity provider using OpenID Connect and AWS services. This integration allows users to use their GitHub credentials to authenticate and access AWS resources securely. By configuring GitHub as an OIDC provider in AWS Identity and Access Management (IAM), organizations can establish a federated identity model. This enables GitHub users to sign in to AWS using their GitHub credentials, streamlining access management and eliminating the need for separate AWS-specific usernames and passwords. The integration also provides a centralized way to manage access permissions and enables Single Sign-On (SSO) capabilities, enhancing security and user experience in the AWS environment. Organizations can configure OIDC settings in AWS, including client IDs, client secrets, and the GitHub OIDC discovery URL, to establish a trust relationship between GitHub and AWS. For the most accurate and up-to-date information, it's recommended to check the official documentation of GitHub and AWS. ## OpenID Connect OIDC is short for OpenID Connect and, like SAML, is a way to federate identities. Federating Identities is a fancy way of saying we trust a 3rd party (the OIDC provider) to handle two tasks: 1. Authentication: verify who the user is 2. Authorization: verify what the user has access to (claims) You can think of this process as similar to arriving at an airport. You present your passport to airport personnel (authentication) so they can identify you along with your boarding pass (authorization claim) indicating you are authorized to pass through security and board a specific flight. ```mermaid --- title: OIDC Authentication Process --- sequenceDiagram participant client as OIDC Client (Relying Party) participant browser as End-User Browser participant provider as OIDC Provider client ->>+ provider: Makes an Authentication Request (AuthN) browser ->>+ provider: Authenticate and Authorize (e.g. username/password) provider ->>+ client: AuthN Response with Token (JWT) client ->>+ client: Validate Token Signature ```
This is similar to how OpenID connect works with DataDog We share this here only if it helps, as the process is conceptually similar for GitHub and GitHub Actions. ![Datadog OIDC Example](/assets/refarch/datadog_oidc_example.png)
## GitHub OIDC and AWS :::tip The primary reason we want to use GitHub OIDC with AWS is so GitHub Actions can assume various AWS Roles without the need to store permanent credentials (e.g. _without_ hardcoding `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`) in GitHub. ::: ```mermaid --- title: GitHub OIDC and AWS --- sequenceDiagram participant aws as AWS API (Relying Party) participant gh as GitHub Action (Relying Party) participant oidc as GitHub OIDC Provider gh ->>+ oidc: Workflow Makes an AuthN Request oidc ->>+ gh: GitHub Returns a Token gh ->>+ aws: API Call aws ->>+ aws: Validate Note right of aws: AWS validates the token was signed by GitHub beforeissuing the temporary credentials aws ->>+ gh: Temporary Credentials Note right of aws: GitHub Action Workflow calls STS AssumeRoleWithWebIdentity,passing the token obtained from the GitHub OIDC providerand receives temporary AWS Credentials in return ``` ## Implementation 1. Install the [GitHub OIDC Provider component](/components/library/aws/github-oidc-provider/) in each account where GitHub Actions need to assume a role. 2. Create the role that GitHub Actions Workflows will assume 3. Configure the GitHub Actions Workflow to assume the role ### Deploy GitHub OIDC Provider Component After deploying the [GitHub OIDC Provider component](/components/library/aws/github-oidc-provider/) into an account, you should see the Identity Provider in IAM in the AWS Web Console. Deploy this component in each account where GitHub Actions need to assume a role. - Import `catalog/github-oidc-provider` in the `gbl` stack for the given account - Deploy the `github-oidc-provider` component: `atmos terraform apply github-oidc-provider -s plat-gbl-dev` ### Option 1: Configure GitHub OIDC Mixin Role and Policy Use the mixin to grant GitHub the ability to assume a role for a specific component. - Add the [GitHub OIDC Mixin](https://github.com/cloudposse/terraform-aws-components/tree/main/mixins/github-actions-iam-role) to any component that needs to generate an IAM Role for GitHub Actions - Implement a custom IAM Policy with least privilege for the role. See [example policies here](https://github.com/cloudposse/terraform-aws-components/tree/main/mixins/github-actions-iam-policy) ### Option 2: Deploy GitHub OIDC Role Component Deploy the [GitHub OIDC Role component](/components/library/aws/github-oidc-role/) to create a generalized role for GitHub to access several resources in AWS. ### Configure GitHub Action Workflows First, give the GitHub Action Workflow the proper permissions ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` Then, use the official [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) action to automatically obtain a token from the GitHub OIDC provider, exchange that token for AWS temporary credentials and set the proper env vars in your GitHub Action Workflow ```yaml - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: aws-region: us-east-2 role-to-assume: arn:aws:iam::111111111111:role/my-github-actions-role role-session-name: my-github-actions-role-session ``` ## FAQ ### Should I use the Mixin or the component to deploy a GitHub OIDC role? Use the mixin when deploying a role tightly coupled with a specific component. For example, use the mixin with `ecr` to grant GitHub access to push and pull ECR images. However, sometimes we need a role with access to many components or resources. In this case, we use the `github-oidc-role` component to define a generalized role for GitHub to assume. For example, we use the component for the `gitops` role. --- ## Philips Labs GitHub Action Runners import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import AtmosWorkflow from "@site/src/components/AtmosWorkflow"; import Steps from "@site/src/components/Steps"; import Step from "@site/src/components/Step"; import StepNumber from "@site/src/components/StepNumber"; import TaskList from "@site/src/components/TaskList"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; If we are not deploying EKS, it's not worth the additional effort to set up Self-Hosted runners on EKS. Instead, we deploy Self-Hosted runners on EC2 instances. These are managed by an API Gateway and Lambda function that will automatically scale the number of runners based on the number of pending jobs in the queue. The queue is written to by the API Gateway from GitHub Events. ## Quick Start | Steps | Actions | | :------------------------------------------------- | :-------------------------------------------------------------------------------------- | | 1. Create GitHub App | ClickOps | | 2. Upload GitHub App ID and Private Key to AWS SSM | Set SSM Param `"/pl-github-runners/id"` and `"/pl-github-runners/key"` (base64 encoded) | | 3. Deploy GitHub OIDC Provider | Deploy GitHub OIDC to every needed account | | 4. Deploy GitHub Runners | `atmos terraform deploy philips-labs-github-runners -s core-use1-auto` | | 5. Update Webhook (if changed or redeployed) | ClickOps | ## Deploy The setup for the Philips Labs GitHub Action Runners requires first creating the GitHub App, then deploying the `philips-labs-github-runner` component, and then finalizing the GitHub App webhook. Cloud Posse typically does not have access to the customer's GitHub Organization settings, so the customer will need to create the initial GitHub App, then hand the setup back to Cloud Posse. Then Cloud Posse can deploy the component and generate the webhook. Finally, the customer will then need to add the webhook to the GitHub App and ensure the App is installed to all relevant GitHub repositories. Follow the guide with the upstream module, [philips-labs/terraform-aws-github-runner](https://github.com/philips-labs/terraform-aws-github-runner#setup-github-app-part-1), or follow the steps below. ### Vendor Components Vendor in the necessary components with the following workflow: ### Create the GitHub App :::info Customer Requirement This step requires access to the GitHub Organization. Customers will need to create this GitHub App in Jumpstart engagements. ::: 1. Create a new GitHub App 1. Choose a name 1. Choose a website (mandatory, not required for the module). 1. Disable the webhook for now (we will configure this later or create an alternative webhook). 1. Add the following permission for your chosen runner scope: #### Repository Permissions - Actions: Read-only (check for queued jobs) - Checks: Read-only (receive events for new builds) - Metadata: Read-only (default/required) - Administration: Read & write (to register runner) #### Repository Permissions - Actions: Read-only (check for queued jobs) - Checks: Read-only (receive events for new builds) - Metadata: Read-only (default/required) #### Organization Permissions - Self-hosted runners: Read & write (to register runner) 1. Generate a Private Key 1. If you are working with Cloud Posse, upload this Private Key and GitHub App ID to 1Password and inform Cloud Posse. Otherwise, continue to the next step. ### Upload AWS SSM Parameters :::tip This step does _not_ require access to the GitHub Organization. Cloud Posse will run this deployment for Jumpstart engagements. ::: Now that the GitHub App has been created, upload the Private Key and GitHub App ID to AWS SSM Parameter Store in `core-use1-auto` (or your chosen region). 1. Upload the PEM file key to the specified ssm path, `/pl-github-runners/key`, in `core-use1-auto` as a base64 encoded string. 2. Upload the GitHub App ID to the specified ssm path, `/pl-github-runners/id`, in `core-use1-auto`. Or run the `upload/pl-secrets` workflow with `atmos` to write the GitHub App information to the `core-use1-auto` SSM account and deploy the component. ### Deploy GitHub OIDC Providers First deploy the GitHub OIDC provider to all accounts where we want to grant GitHub access. The typical list of accounts is included with the `deploy/github-oidc-provider` workflow; run the following with `SuperAdmin`: ### Deploy the Philips Labs GitHub Runners Now that the GitHub App has been created and the SSM parameters have been uploaded, deploy the `philips-labs-github-runners` component. ### Add the Webhook to the GitHub App :::info Customer Requirement This step requires access to the GitHub Organization. Customers will need to finalize the GitHub App in Jumpstart engagements. ::: Now that the component is deployed and the webhook has been created, add that webhook to the GitHub App. Both the webhook URL and secret should now be stored in 1Password. If not, you can retrieve these values from the output of the `philips-labs-github-runners` component in `core-use1-auto` as described in the previous step. 1. Open the GitHub App created in [Create the GitHub App above](/layers/github-actions/philips-labs-github-runners/#create-the-github-app) 1. Enable the webhook. 1. Provide the webhook url, should be part of the output of terraform. 1. Provide the webhook secret (`terraform output -raw `). 1. In the _"Permissions & Events"_ section and then _"Subscribe to Events"_ subsection, check _"Workflow Job"_. 1. Ensure the webhook for the GitHub app is enabled and pointing to the output of the module. The endpoint can be found by running `atmos terraform output philips-labs-github-runners -s core-use1-auto 'webhook'` ## Usage Once you've deployed Self Hosted runners select the appropriate runner set with the `runs-on` configuration in any GitHub Actions workflow. For example, we can use the default runner set as such: ```yaml runs-on: ["self-hosted", "default"] ``` However, it's very likely you will have resource-intensive jobs that the default runner size may not satisfy. We recommend deploying additional runner sets for each tier of workflow resource requirements. For example in our internal GitHub Organization, we have `default`, `medium`, and `large` runners. ### Using the `terraform` Label By default, we configure the Atmos Terraform GitHub Actions to use the `terraform` labeled Self-Hosted runners. ```yaml runs-on: ["self-hosted", "terraform"] ``` However also by default we only have the single runner set. We recommend deploying a second runner set with a larger resource allocation for these specific jobs. Remove the `terraform` label from the default runner set and add the `terraform` label to your new, larger runner set. Since the workflows are all labeled with `terraform` already, they will automatically select the new runner set on their next run. ## FAQ ### I cannot assume the role from GitHub Actions after deploying The following error is very common if the GitHub workflow is missing proper permission. ```bash Error: User: arn:aws:sts::***:assumed-role/acme-core-use1-auto-actions-runner@actions-runner-system/token-file-web-identity is not authorized to perform: sts:TagSession on resource: arn:aws:iam::999999999999:role/acme-plat-use1-dev-gha ``` In order to use a web identity, GitHub Action pipelines must have the following permission. See [GitHub Action documentation for more](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/configuring-openid-connect-in-amazon-web-services#adding-permissions-settings). ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` --- ## Runs On - Self-Hosted Runners import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Admonition from '@theme/Admonition'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import File from '@site/src/components/File'; This guide explains how to set up and manage self-hosted runners using the Runs On component. Self-hosted runners provide more control over the execution environment, allowing you to run workflows on your own infrastructure. [GitHub-OIDC](/layers/github-actions/github-oidc-with-aws/) is not required to deploy Runs On, but it is required for several components to be deployed on or to via SelfHosted Runners. For example, your runners cannot push to ECR without GitHub-OIDC. 1. Vendor the required components 2. Deploy the Runs On component 3. Install the GitHub App 4. Configure your workflows to use the runners ## Vendor Runs On Component Vendor the required components using the included Atmos workflow: ## Deploy Runs On Component Deploy the Runs On component using the included Atmos workflow: ## Install GitHub App After deployment, follow these steps to install the GitHub App: 1. Check the Terraform outputs for `RunsOnEntryPoint` 2. Use the provided URL to install the GitHub App 3. Follow the prompts to complete the installation in your GitHub Organization 4. Ensure you have the necessary permissions in GitHub to install the app ### Configure Workflows Update your GitHub Actions workflow files to use the self-hosted runners: ```yaml jobs: build: runs-on: - "runs-on=${{ github.run_id }}" - "runner=terraform" # Note `terraform` is a runner group name defined by a RunsOn configuration ## If no configuration is present, use # - "runner=2cpu-linux-x64" ## Optional Tags # - "tag=${{ inputs.component }}-${{ inputs.stack }}" steps: - uses: actions/checkout@v3 # Add your build steps here ``` For more information on available runner types and configurations, check the [RunsOn: Runner Types documentation](https://runs-on.com/runners/linux/). ### (Optional) Setup a RunsOn Repo or Organization Configuration In your Repository you can add a file to configure Runs On. This can also extend the configuration for the Organization. Please note this snippet below is an extremely simplified example If you want to see what Cloud Posse uses as a starting point checkout our configuration [here](https://github.com/cloudposse/.github/blob/main/.github/runs-on.yml) Here's a sample configuration. We recommend storing this in a centralized `.github` repository so you can define a shared `runs-on` configuration that you can use across all repositories, without duplicating it in each one. This is especially useful when managing many repositories. ```yaml runners: terraform: image: ubuntu22-full-x64 disk: default spot: price-capacity-optimized retry: when-interrupted private: false ssh: false cpu: [2, 32] ram: [8, 64] tags: - "gha-runner:runs-on/terraform" ``` To use your organization's shared configuration in an individual repository, you need to define a local configuration that uses the `_extends` keyword to inherit from the centralized setup — it won’t be applied automatically. ```yaml # See https://runs-on.com/configuration/repo-config/ _extends: .github terraform: cpu: [4, 32] # example override ``` ## Troubleshooting ### GitHub Action Runner Not Found First determine if the Workflow or the Runner is the issue, sometimes the workflow doesn't kick off because it is on a feature branch and not on the default. If the workflow kicks off but is waiting on a runner, checkout [Runs On Troubleshooting](https://runs-on.com/guides/troubleshoot/) as they have great docs on figuring out why a runner is not available. --- ## Example Workflows import Intro from '@site/src/components/Intro'; import CodeBlock from '@theme/CodeBlock'; import CollapsibleText from '@site/src/components/CollapsibleText'; import PartialAtmosTerraformPlan from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-plan.yaml'; import PartialAtmosTerraformApply from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-apply.yaml'; import PartialAtmosTerraformDispatch from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-dispatch.yaml'; import PartialAtmosTerraformDriftDetection from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-drift-detection.yaml'; import PartialAtmosTerraformDriftRemediation from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-drift-remediation.yaml'; import PartialAtmosTerraformPlanMatrix from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-plan-matrix.yaml'; import PartialAtmosTerraformApplyMatrix from '@site/examples/legacy/snippets/.github/workflows/atmos-terraform-apply-matrix.yaml'; Using GitHub Actions with Atmos and Terraform is fantastic because it gives you full control over the workflow. While we offer some opinionated implementations below, you are free to customize them entirely to suit your needs. The following GitHub Workflows should be used as examples. These are created in a given Infrastructure repository and can be modified however best suites your needs. For example, the labels we've chosen for triggering or skipping workflows are noted here as "Conventions" but can be changed however you would prefer. ### Atmos Terraform Plan :::info Conventions Use the `no-plan` label on a Pull Request to skip this workflow. ::: The Atmos Terraform Plan workflow is triggered for every affected component from the Atmos Describe Affected workflow. This workflow takes a matrix of components and stacks and creates a plan for each, using the [Atmos Terraform Plan composite action](https://github.com/cloudposse/github-action-atmos-terraform-plan). For more on the Atmos Terraform Plan composite action, see [the official atmos.tools documentation](https://atmos.tools/integrations/github-actions/atmos-terraform-plan). If an affected component is disabled with `terraform.settings.github.actions_enabled`, the component will show up as affected but all Terraform steps will be skipped. See [Enabling or disabling components](#enabling-or-disabling-components). {PartialAtmosTerraformPlan} ### Atmos Terraform Apply :::info Conventions 1. Use the `auto-apply` label on Pull Request to apply all plans on merge 1. Use the `no-apply` label on a Pull Request to skip _all workflows_ on merge 1. If a Pull Request has neither label, run drift detection for only the affected components and stacks. ::: The Atmos Terraform Apply workflow runs on merges into main. There are two different workflows that can be triggered based on the given labels. If you attach the Apply label (typically `auto-apply`), this workflow will trigger the [Atmos Terraform Apply composite action](https://github.com/cloudposse/github-action-atmos-terraform-apply) for every affected component in this Pull Request. For more on the Atmos Terraform Apply composite action, see [the official atmos.tools documentation](https://atmos.tools/integrations/github-actions/atmos-terraform-apply). Alternatively, you can choose to merge the Pull Request _without_ labels. If the "apply" label and the "skip" label are not added, this workflow will trigger the [Atmos Drift Detection composite action](https://github.com/cloudposse/github-action-atmos-terraform-drift-detection) for only the affected components in this Pull Request. That action will create a GitHub Issue for every affected component that has unapplied changes. {PartialAtmosTerraformApply} ### Atmos Terraform Drift Detection :::info Max Opened Issues Drift detection is configured to open a set number of Issues at a time. See `max-opened-issues` for the `cloudposse/github-action-atmos-terraform-drift-detection` composite action. ::: The Atmos Terraform Drift Detection workflow runs on a schedule. This workflow will gather _every component in every stack_ and run the [Atmos Drift Detection composite action](https://github.com/cloudposse/github-action-atmos-terraform-drift-detection) for each. For every stack and component included with drift detection, the workflow first triggers an Atmos Terraform Plan. 1. If there are changes, the workflow will then create or update a GitHub Issue for the given component and stack. 2. If there are no changes, the workflow will check if there's an existing Issue. If there's an existing issue, the workflow will then mark that Issue as resolved. {PartialAtmosTerraformDriftDetection} ### Atmos Terraform Drift Remediation :::info Conventions Use the `apply` label to apply the plan for the given stack and component ::: The Atmos Terraform Drift Remediation workflow is triggered from an open Github Issue when the remediation label is added to the Issue. This workflow will run the [Atmos Terraform Drift Remediation composite action](https://github.com/cloudposse/github-action-atmos-terraform-drift-remediation) for the given component and stack in the Issue. This composite action will apply Terraform using the [Atmos Terraform Apply composite action](https://github.com/cloudposse/github-action-atmos-terraform-apply) and close out the Issue if the changes are applied successfully. The `drift` and `remediated` labels are added to Issues by the composite action directly. The `drift` is added to all Issues created by Atmos Terraform Drift Detection. Remediation will only run on Issues that have this label. Whereas the `remediated` label is added to any Issue that has been resolved by Atmos Terraform Drift Remediation. {PartialAtmosTerraformDriftRemediation} ### Atmos Terraform Dispatch The Atmos Terraform Dispatch workflow is optionally included and is not required for any other workflow. This workflow can be triggered by workflow dispatch, will take a single stack and single component as arguments, and will run Atmos Terraform workflows for planning and applying for only the given target. This workflow includes a boolean option for both "Terraform Plan" and "Terraform Apply": 1. If only "Terraform Plan" is selected, the workflow will call the Atmos Terraform Plan Worker (`./.github/workflows/atmos-terraform-plan-matrix.yaml`) workflow to create a new planfile 1. If only "Terraform Apply" is selected, the workflow will call the Atmos Terraform Apply Worker (`./.github/workflows/atmos-terraform-apply-matrix.yaml`) for the given branch. This action will take the latest planfile for the given stack, component, and branch and apply it. 1. If both are selected, the workflow will run both actions. This means it will create a new planfile and then immediately apply it. 1. If neither are selected, the workflow does nothing. {PartialAtmosTerraformDispatch} ### Atmos Terraform Plan Matrix (Reusable) The Atmos Terraform Plan Matrix is reusable workflow that called by another workflows to create a terraform plan. {PartialAtmosTerraformPlanMatrix} ### Atmos Terraform Apply Matrix (Reusable) The Atmos Terraform Apply Matrix is reusable workflow called by another workflow to apply an existing plan file. {PartialAtmosTerraformApplyMatrix} --- ## Frequently Asked Questions ### What are the included labels? By default, Cloud Posse includes a few labels for common use-cases. #### Pull Request Labels - `auto-apply` - If added, the Atmos Terraform Apply workflow will be triggered for all affected components when the Pull Request is merged. - `no-plan` - If added, the Atmos Terraform Plan workflow will be skipped on commits to the Pull Request, and the Atmos Apply workflow will be skipped when the Pull Request is merged. #### Issue Labels - `apply` - Triggers the Atmos Terraform Drift Remediation workflow for a specific component and stack - `discarded` - Issue was closed by the drift detection workflow - `drift` - Indicates that drift was detected by the drift detection workflow - `drift-recovered` - Indicates that an issue is no longer experiencing drift - `error` - Indicates an error occurred during planning for a specific component and stack - `error-recovered` - Indicates that an error state has been resolved - `remediated` - Issue was successfully remediated by the drift remediation action - `removed` - Issue is closed because the component no longer exists (code was deleted) ### Enabling or disabling components Components are included in the Atmos GitHub Action workflows only if they have actions enabled with the `terraform.settings.github.actions_enabled` option. If they do not have this setting or the value is `false`, the component may still appear in the list of affected components, but Terraform will not be run against the given component and stack. :::info Global Defaults Typically Cloud Posse sets the default value to `true` for all components and disables individual components on a case-by-case basis. For example in an `acme` organization, the default value could be set with `stacks/orgs/acme/_defaults.yaml`: ```yaml terraform: # These settings are applied to ALL components by default but can be overwritten settings: github: actions_enabled: true ``` And the `account` component could be disabled with `stacks/catalog/account.yaml`: ```yaml components: terraform: account: settings: github: actions_enabled: false ``` ::: ### I cannot assume the `gitops` role from GitHub Workflows The following error commonly occurs when setting up `gitops` roles and permission: ``` Error: Could not assume role with OIDC: Not authorized to perform sts:AssumeRoleWithWebIdentity ``` To resolve this error, thoroughly read through each of the [Authentication Prerequisites](/layers/gitops/setup#authentication-prerequisites) for GitOps setup. In particular, check the capitalization of `trusted_github_repos` within `aws-teams` and check the `permissions` for the workflow in GitHub. ### How does GitHub OIDC work with AWS? Please see [How to use GitHub OIDC with AWS](/layers/github-actions/github-oidc-with-aws) --- ## Quick Start import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import ReactPlayer from 'react-player'; import CodeBlock from '@theme/CodeBlock'; GitOps is a cloud-native continuous deployment methodology that uses Git as the single source of truth for declarative infrastructure and applications. Changes to infrastructure or applications are made through Git commits, and the actual state is automatically adjusted to match the desired state expressed in the Git repository. This approach provides an audit trail for changes, simplifies rollback, and enhances collaboration and visibility across teams.
AI generated voice
## The Problem Collaboration with Terraform in team environments is more difficult than traditional [release-engineering](/layers/software-delivery) for web applications. Unlike containerized deployments, Terraform deployments are constantly modifying the state of infrastructure and behave a lot more like database migrations, except there are no transactions and therefore no practical way to do automated rollbacks. This means we have to be extra cautious. When teams start collaborating on infrastructure, the rate of change increases, and the likelihood of collisions as well. We need approval gates to control what changes and when, plus have the ability to review changes prior to deployment to make sure nothing catastrophic happens (e.g. database destroyed). Pull Requests are not enough to restrict what changes, since every Pull Request merged, changes the graph and therefore requires all other open Pull Requests to be re-validated. There's also a need to reconcile the desired state of infrastructure in Git with what is deployed; in busy team environments, a change can accidentally not be deployed or at other times, ClickOps can result in a drift between what's running and what's in code. Multiple platforms have emerged that solve this problem, under the general category of "Terraform Automation and Collaboration Software" or TACOS for short. Examples include Terraform Cloud, Spacelift, Env0, Scalr, and that's just a start. TACOs can easily cost tens of thousands of dollars a year and can be cost-prohibitive for certain companies. ## Our Solution We've implemented [GitHub Actions](https://atmos.tools/category/github-actions) designed around our architecture and toolset. These actions run Atmos commands to generate a Terraform planfile, store the planfile in a S3 bucket with metadata in DynamoDB, and generate a plan summary on all pull requests. Then once the pull request is merged, a second workflow runs to pull that same planfile, apply it with Atmos commands, and then generate an apply summary. While this solution does not offer some of the more fine-grained policy controls of TACOs nor provide a centralized UI, it does provide many of the other benefits that the other solutions offer. But the overwhelming benefit is it's much cheaper and fully integrated with Cloud Posse's architecture and design. ### Features * **Implements Native GitOps** with Atmos and Terraform * **GitHub Actions** can be integrated seamlessly anywhere you need to run Terraform * **No hardcoded credentials.** Use GitHub OIDC to assume roles. * **Compatible with GitHub Cloud & Self-hosted Runners** for maximum flexibility. * **Beautiful Job Summaries** don't clutter up pull requests with noisy GitHub comments * **100% Open Source with No Platform Fees** means you can leverage your existing GitHub runners to provision infrastructure Expect these actions to constantly evolve as we build out these workflows. ### Implementation Once the required S3 Bucket, DynamoDB table, and two separate roles to access Terraform planfiles and plan/apply Terraform are deployed, simply add your chosen [workflows](#workflows). Read the [Setup Documentation](/layers/gitops/setup) for details on deploying the requirements. ## Workflows ## References - [Setup Documentation](/layers/gitops/setup) - [Atmos integration documentation](https://atmos.tools/category/integrations/github-actions). - [GitHub OIDC Integration with AWS](/layers/github-actions/github-oidc-with-aws) - [`cloudposse/github-action-atmos-terraform-plan`](https://github.com/cloudposse/github-action-atmos-terraform-plan) - [`cloudposse/github-action-atmos-terraform-apply`](https://github.com/cloudposse/github-action-atmos-terraform-apply) - [`cloudposse/github-action-atmos-terraform-drift-detection`](https://github.com/cloudposse/github-action-atmos-terraform-drift-detection) - [`cloudposse/github-action-atmos-terraform-drift-remediation`](https://github.com/cloudposse/github-action-atmos-terraform-drift-remediation) - [`gitops/s3-bucket`](/components/library/aws/s3-bucket/): Deploy a S3 Bucket using the `s3-bucket` component. This bucket holds Terraform planfiles. - [`gitops/dynamodb`](/components/library/aws/dynamodb/): Deploy a DynamoDB table using the `dynamodb` component. This table is used to hold metadata for Terraform Plans - [`github-oidc-role`](/components/library/aws/github-oidc-role/): Deploys an IAM Role that GitHub is able to assume via GitHub OIDC. This role has access to the bucket and table for planfiles. --- ## Setup GitOps with GitHub Actions import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import Admonition from '@theme/Admonition'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import CodeBlock from '@theme/CodeBlock'; ## Quick Start | Steps | | | :------------------------------ | :----------------------------------------------------- | | 1. Verify Identity requirements | | | 2. Expand GitHub OIDC | `atmos workflow deploy/github-oidc-provider -f quickstart/foundation/gitops` | | 2. Vendor | `atmos workflow vendor -f quickstart/foundation/gitops` | | 3. Deploy | `atmos workflow deploy/gitops -f quickstart/foundation/gitops` | ## Requirements ### Self-Hosted Runners Although not required, we recommend first deploying Self-Hosted GitHub runners if you need to manage any resources inside of a VPC (e.g. RDS Users). For more information on setting up runners, see the [Philips Labs GitHub Runners](/layers/github-actions/philips-labs-github-runners) or the [GitHub Action Runner Controller (EKS)](/layers/github-actions/eks-github-actions-controller) setup documentation. If you do not wish to use Self Hosted runners, simply change the `runs-on` option for all included workflows in `.github/workflows` ### Set Up GitHub Variables The `gitops` stack config depends on the following GitHub variables:
`ATMOS_VERSION`
The version of Atmos to use
`ATMOS_CONFIG_PATH`
The path to the Atmos config file
Please set the following GitHub variables in the repository settings: 1. Open the repository [settings](https://github.com/acme/infra-acme/settings/variables/actions) 2. Set variable `ATMOS_VERSION` to the `1.63.0` value 3. Set variable `ATMOS_CONFIG_PATH` to the `./rootfs/usr/local/etc/atmos/` value
### Authentication Prerequisites The GitHub Action workflows expect both the `gitops` and `planners` AWS Teams to be properly setup and connected to GitHub OIDC. Both of these components should already be deployed with `aws-teams`/`aws-team-roles` and `github-oidc-provider` respectively, but `github-oidc-provider` will likely need to deployed to several additional accounts. Verify the following to complete the authentication prerequisites. By default in the Reference Architecture, the `trusted_github_repos` input is commented out for `aws-teams`. Now is the time to uncomment those lines. Follows the tasks below. Please see `stacks/catalog/aws-teams.yaml` - The `gitops` and `planners` Teams are defined and deployed by `aws-teams`. - Both teams have trusted relationships with the infrastructure repo via `trusted_github_repos`. _Capitalization matters!_ In the reference architecture, these values are initially commented out and will need to be updated with your specific repository information: ```yaml components: terraform: aws-teams: vars: trusted_github_repos: gitops: - "acme/infra:main" planners: - "acme/infra" ``` - The `aws-team-roles` default catalog allows the `gitops` team to assume the `terraform` role, including anywhere `aws-team-roles` is overwritten (`plat-dev` and `plat-sandbox`) - Similarly, the `planners` team can assume the `planner` role in `aws-team-roles` to plan Terraform only. - `tfstate-backend` allows both teams to assume the default access role from the `core-identity` account - `github-oidc-provider` is deployed to every account that GitHub will be able to access. This should be every account except `root`. - The workflows have adequate permission In order to assume GitHub OIDC roles, a workflow needs the following: ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` In order to assume GitHub OIDC roles _and_ manage Github Issues, a workflow needs these permissions: ```yaml permissions: id-token: write # This is required for requesting the JWT contents: write # This is required for actions/checkout and updating Issues issues: write # This is required for creating and updating Issues ```
## How To Setup ### Vendor Components The `gitops` stack config depends on two components that may already exist in your component library (`s3-bucket` and `dynamodb`) and adds one new component (`gitops`) to manage the GitHub OIDC access. Vendor these components either with the included Atmos Workflows or using [Atmos Vendoring](https://atmos.tools/core-concepts/components/vendoring). ### Deploy GitOps Prerequisites Deploy the GitOps prerequisite components, `gitops/s3-bucket`, `gitops/dynamodb`, and `gitops` with the following workflow ### Reapply `aws-teams` Now we need to reapply `aws-teams` to add the trusted GitHub repositories to `gitops` and `planners`. Uncomment or add the `trusted_github_repos` input: ```yaml # stacks/catalog/aws-teams.yaml components: terraform: aws-teams: vars: trusted_github_repos: gitops: - "acme/infra:main" planners: - "acme/infra" ``` Run the following command to apply: `aws-teams` is a sensitive component deployed to the `core-identity` account and therefore needs to be applied with a role or user with access to the account. For example use the `managers` AWS Team or the SuperAdmin user. ```bash atmos terraform apply aws-teams -s core-gbl-identity ``` And that's it! Now you can try creating a new pull request. If properly configured, you should see GitHub Actions kick off `Atmos Terraform Plan`. 1. Enable GitHub Actions support for any component by enabling `settings.github.actions_enabled: true` and let the workflow handle the rest. Keep in mind this setting is likely enabled by default for your organization stack configuration, `stacks/catalog/acme/_defaults.yaml` 1. The roles created by `aws-teams` or `gitops` should already be included in your workflows. Verify these roles match the `env` settings in `.github/workflows/atmos-terraform-*` 1. You do not need to create a GitHub App or complete additional steps to trigger these workflows --- ## AWS Identity Center (SSO) ClickOps import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import Steps from "@site/src/components/Steps"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; import StepNumber from "@site/src/components/StepNumber"; import Step from "@site/src/components/Step"; This guide provides an overview of setting up AWS Identity Center (SSO) with ClickOps, detailing prerequisites and supported external identity providers. It explains how to integrate AWS SSO with providers like Azure AD, JumpCloud, Okta, and Google Workspace, including specific steps for configuring each. ## How it Works AWS Single Sign-On (AWS SSO) is a service that simplifies access management for AWS accounts and applications. It enables users to sign in to AWS once and access multiple AWS accounts and applications without the need to re-enter credentials. To use it with an identity provider (e.g. such as Okta) for AWS SSO, administrators typically need to configure the integration within the AWS Management Console. This involves setting up a new AWS SSO instance, connecting it the IdP, and specifying the users or groups that should have access to AWS resources. AWS SSO provides logging and auditing capabilities, allowing organizations to track user access to AWS resources and monitor security-related events ### SAML-Based Authentication The integration between the IdP and AWS SSO relies on the Security Assertion Markup Language (SAML) for authentication and authorization. SAML enables the exchange of authentication and authorization data between Okta and AWS, allowing users to log in once to Okta and gain access to AWS resources without additional logins. ### User Provisioning AWS SSO can be configured to automatically provision and de-provision user accounts based on changes in the IdP directory. This helps keep user access in sync with changes made in the Okta environment. ### AWS SSO Permission Sets AWS SSO allows administrators to define fine-grained access policies, specifying which AWS accounts and services users from the IdP can access ### Multi-Factor Authentication (MFA) Organizations using Okta for authentication with AWS SSO can enhance security by enforcing multi-factor authentication (MFA) for added identity verification. Once configured, users can experience single sign-on when accessing AWS resources. They log in to their IdP account and seamlessly gain access to AWS without needing to provide credentials again. It's important to note that the specifics of the integration process may be subject to updates or changes, so it's recommended to refer to the official AWS documentation and Okta documentation for the most accurate and up-to-date information based on your current date. ## Prerequisites First, enable the AWS IAM Identity Center (successor to AWS Single Sign-On) service in the `core-root` account. This is the account where the `aws-sso` component will be deployed. 1. Navigate to the `core-root` account in the AWS Web Console 2. Select your primary region 3. Go to AWS IAM Identity Center (successor to AWS Single Sign-On) 4. Enable the service ## Configure your Identity Provider These are the instructions for the most common Identity Providers. Alternatives are available, but the steps may vary depending on the provider. It's important to note that the specifics of the integration process may be subject to updates or changes, so it's recommended to refer to the official AWS documentation and respective IdP documentation for the most accurate and up-to-date information based on your current date. For providers not included in the following section, please [follow the AWS documentation for setting up an IdP integration with AWS](https://docs.aws.amazon.com/singlesignon/latest/userguide/supported-idps.html). This list includes Azure AD, CyberArk, OneLogin, and Ping Identity. Okta is a common business suite that has an active director to manage users and permissions. We can utilize this to login to AWS by leveraging **Applications** that are used to sign in to things from your Okta Account. ### Setup Okta 1. Under the Admin Panel go to **Applications** 2. Click **Browse App Catalog** 3. Search for `AWS IAM Identity Center` and click **Add Integration** 4. Keep the default settings of **App Label** ("AWS IAM Identity Center") and **Application Visibility** 5. Go to **Sign On** and Copy information from the SAML Metadata section, this will be used in AWS SSO. 6. Then go to Provisioning and click **Configure API Integration** ### Setup AWS SSO 1. Sign into AWS SSO under your management account (`core-root`) 2. Go to the AWS IAM Identity Center (successor to AWS Single Sign-On) application 3. Enable IAM Identity Center 4. On the left panel click **Settings** 5. Under Identity Source click edit and add an **External identity provider** 6. Copy the information from Okta into the fields 7. The Okta App will need to be updated with the **Service provider metadata** JumpCloud is a cloud-based directory service that provides secure, frictionless access to AWS resources. It can be used as an identity provider for AWS (Amazon Web Services) through a feature called AWS Single Sign-On (AWS SSO). Follow the JumpCloud official documentation for setting up JumpCloud with AWS IAM Identity Center: [Integrate with AWS IAM Identity Center](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center) :::caution Integrating JumpCloud with AWS IAM Identity Center The official AWS documentation for setting up JumpCloud with AWS IAM Identity Center is not accurate. Instead, please refer to the [JumpCloud official documentation](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center) ::: Microsoft Entra ID (formerly known as Office 365, Microsoft 365, Azure AD) can be used as an identity provider for AWS (Amazon Web Services) through a feature called AWS Single Sign-On (AWS SSO). AWS SSO allows organizations to centralize identity management and provide users with seamless access to AWS resources using their existing Microsoft Entra ID credentials. ### Setup Microsoft Entra ID #### Open Microsoft Entra ID Application Go to [Microsoft Entra's Admin Center](https://entra.microsoft.com/) and search for `Entra ID` Go to [Microsoft Azure](https://portal.azure.com/#view/Microsoft_AAD_IAM/ActiveDirectoryMenuBlade/~/Overview) and search for `Entra ID` #### Add AWS IAM Identity Center as an **Enterprise Application** Click `Add` then `Enterprise Application` Then select `AWS IAM Identity Center (successor to AWS Single Sign-On)` Click `Create`, default options are fine #### Download the Microsoft Entra ID Metadata XML File On the left panel click `Single sign-on`, then download the XML SAML metadata file by pressing the button on step 5 `Set up AWS IAM Identity Center (successor to AWS Single Sign-On)` #### Download the AWS IAM Identity Center Metadata XML File Navigate to the AWS IAM Identity Center (successor to AWS Single Sign-On) application in AWS IAM Identity Center. Go to Setup/Change Identity Source, click `External identity provider` from the available identity sources. Click `Next`. Download the XML SAML Metadata file by clicking `Download metadata file`. #### Upload the Microsoft Entra ID Metadata XML File to AWS Upload the XML SAML metadata file from Microsoft Entra ID to AWS IAM Identity Center. #### **SAVE** Click `Save` to save the changes. ## Automatic Provisioning #### Generate SCIM URL and Secret In AWS IAM Identity Center, you can set up automatic provisioning by generating a URL and secret #### Navigate to the new App Go to Your newly created App for Single Sign On in Microsoft Entra ID, on the left Panel go to `Provisioning`. #### Set the mode to **Automatic** and paste the values from AWS into the **Admin Credentials** Section{" "} #### Verify Connection Click `Test Connection` to verify the connection. For non-explicitly supported Identity Providers, such as GSuite, set up the app integration with a custom external identity provider. The steps may be different for each IdP, but the goal is ultimately the same. :::tip aws-ssosync GSuite does not automatically sync _both_ Users and Groups with AWS Identity Center without additional configuration! If using GSuite as an IdP, considering deploying the [ssosync](https://github.com/awslabs/ssosync) tool. Please see our [aws-ssosync component](/components/library/aws/aws-ssosync/) for details! ::: Open the Identity account in the AWS Console On the Dashboard page of the IAM Identity Center console, select Choose your identity source In the Settings, choose the Identity source tab, select the Actions dropdown in the top right, and then select Change identity source By default, IAM Identity Center uses its own directory as the IdP. To use another IdP, you have to switch to an external identity provider. Select External identity provider from the available identity sources 5. Configure the custom SAML application with the Service provider metadata generated from your IdP. Follow the next steps from your IdP, and then complete this AWS configuration afterwards 6. Open your chosen IdP 7. Create a new SSO application 8. Download the new app's IdP metadata and use this to complete step 5 above 9. Fill in the Service provider details using the data from IAM Identity Center, and then choose Continue. The mapping for the data is as follows: ```console For ACS URL, enter the IAM Identity Center Assertion Consumer Service (ACS) URL. For Entity ID, enter the IAM Identity Center issuer URL. Leave the Start URL field empty. For Name ID format, select EMAIL. ``` 10. If required for the IdP, enable the application for all users 11. Finally, define specific Groups to match the given Group names by the aws-sso component (`stacks/catalog/aws-sso.yaml`). In the default catalog, we define four Groups: `DevOps`, `Developers`, `BillingAdmin`, and `Everyone` If set up properly, Users and Groups added to your IdP will automatically populate and update in AWS. Additional IdP specific setup reference can be found here: - [How to use Google Workspace as an external identity provider for AWS IAM Identity Center](https://aws.amazon.com/blogs/security/how-to-use-g-suite-as-external-identity-provider-aws-sso/) --- ## Centralized access management for Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; This is an overview of our approach to centralized access management and the relationships between the components used to implement it. It explains how SAML, SSO, and custom constructs for AWS teams and team roles interact to provide efficient role delegation and authentication across an AWS Organization. ## AWS Teams and Team Roles Why do we need Teams and Team Roles when we have the same thing in Identity Center? To explain that, let’s first explain what they do and how we’ve integrated them. ### AWS Teams The `aws-teams` component creates AWS Teams in the Identity account. The **Team** is the entry point for a group of users, and the Identity account acts as an "Identity Hub" for centralized role assumption management across all delegated accounts. Think of each team as a group of users, such as `developers` or `devops`. ### AWS Team Roles The `aws-team-roles` component defines common IAM Roles in all accounts, as well as defines the AWS Team(s) able to assume the role in any account. These roles should be identical in all accounts, such that you can always expect `admin` to be `admin`, `poweruser` to be `poweruser`, and `observer` to be `observer`. The only difference between accounts is which AWS Team is able to assume the role. We also have a `terraform` role that is used by Terraform to apply changes across all accounts. ### Example Consider this example. We want to enable the “DevOps” to assume Administrator in multiple accounts, such as `dev` or `prod`. However, “developers” can only assume admin in `dev`. Here’s how that would work: ```mermaid flowchart LR subgraph core_identity["core-identity Teams"] developers["developers"] devops["devops"] end subgraph dev_account_roles["plat-dev Team Roles"] admin_dev["admin"] end subgraph prod_account_roles["plat-prod Team Roles"] observer_prod["observer"] admin_prod["admin"] end developers --> admin_dev developers --> observer_prod devops --> admin_prod devops --> admin_dev ``` In the `core-identity` account, we have a `developers` team and a `devops` team. Then in the `plat-dev` account, we have an `admin` team role. And in the `plat-staging` account we have an `admin` and an `observer` team role. With our design, the `developers` team can assume `admin` only in `plat-dev`, but the `devops` team can assume `admin` in both `plat-dev` and `plat-staging`. ## Using AWS Identity Center for Single Sign On Now how does AWS IAM Identity Center fit in? Let’s see how we can connect AWS Teams and Team roles with Identity Center to complete the picture. ### Connecting a Permission Set to an AWS Team Identity Center has the concept of Permission Sets. Permission Sets determine what a user can or cannot do. We simply connect those Permission Sets to one of our Teams. We define a special Permission Set for the `core-identity` account, called the `Identity{X}TeamAccess` Permission Set. This Permission Set has access to assume the related AWS Team. For example, the `IdentityDeveloperTeamAccess` permission set can assume the `developers` team and the `IdentityDevopsTeamAccess` permission set can assume the `devops` team. From their AWS Team, users can apply Terraform wherever they have access ```mermaid flowchart LR subgraph identity_center_permission_sets["Identity Center Permission Sets"] developer_access["IdentityDeveloperTeamAccess"] devops_access["IdentityDevopsTeamAccess"] end subgraph core_identity_teams["core-identity Teams"] developers["developers"] devops["devops"] end subgraph plat_sandbox_roles["plat-sandbox Team Roles"] terraform_sandbox["terraform"] end subgraph plat_prod_roles["plat-prod Team Roles"] terraform_prod["terraform"] planner_prod["planner"] end developer_access --> developers devops_access --> devops developers --> terraform_sandbox developers --> planner_prod devops --> terraform_sandbox devops --> terraform_prod ``` ### Centrally Manage Users and Groups with an IdP We still need some method to manage users and groups. AWS Teams and Team roles grant groups of users access, but do not define or manage users. AWS IAM Identity Center ties together Teams and Team roles with a predefined set of users and groups. Although you may choose to manage users and groups directly in Identity Center, the vast majority of our customers choose to use a SAML IdP as the source of truth for their users and groups. With a SAML IdP, users and groups are not created in AWS. Instead, we connect a third-party Identity Provider, such as GSuite, JumpCloud or Okta. Then we manage all Users and Groups in that IdP With “automatic provisioning” in AWS, we can synchronize all users and groups from the IdP into AWS. Then we can create Permission Sets in AWS to assign to those groups. We can assign Permission Sets to Groups for any given account. A Permission Set, such as `AdministratorAccess` or `ReadOnlyAccess`, is granted to a given group for some account. This way, we only need to define the Permission Sets once each ```mermaid flowchart LR subgraph external_idp["External Identity Provider"] user1["IdP User: Homer"] user2["IdP User: Lisa"] user3["IdP User: Bart"] group1["IdP Group: DevOps"] user1 --> group1 user2 --> group1 user3 --> group1 end subgraph aws_iam_identity_center["AWS IAM Identity Center"] user1_copy["Identity Center User: Homer"] user2_copy["Identity Center User: Lisa"] user3_copy["Identity Center User: Bart"] group1_copy["Identity Center Group: DevOps"] permissions["Permission Sets"] user1_copy -.-> group1_copy user2_copy -.-> group1_copy user3_copy -.-> group1_copy group1_copy --> permissions end external_idp -->|Automatic Provisioning| aws_iam_identity_center ``` ### Access the AWS Web Console with Identity Center With Identity Center, it’s very easy to log into the AWS web console. Simply open the Single Sign On Portal link, and select the account and role you want to access. For example, if I wanted to connect to `plat-prod` with “Read Only” access, then I would select the `ReadOnlyAccess` Permission Set for `plat-prod`. AWS then will launch a new session with the appropriate permissions in production ### Deploying both AWS IAM Identity Center and AWS SAML AWS IAM Identity Center and AWS SAML are not mutually exclusive solutions. They can be used together to provide a seamless login experience for users. Cloud Posse frequently deploys both in customer environments - AWS SAML to log in ourselves for the duration of the engagement and AWS IAM Identity Center for the customer to use and support. That way the customer can access AWS via their chosen Identity Provider (IdP), and we can use GSuite SAML to access the Identity account team directly. When an engagement is complete, we can easily decommission our access by removing the SAML connection. :::tip AWS IAM Identity Center or AWS SAML? Which do I choose? The vast majority of our customers prefer AWS IAM Identity Center (SSO). The convenience of a web console login is hard to beat. However, some customers prefer SAML for its simplicity and compatibility with existing systems. We support both methods, and you can choose the one that best fits your needs. ::: ```mermaid flowchart TB subgraph "Complete Access Architecture" subgraph "core-root" subgraph sso["Identity Center"] developers_team_access["IdentityDevelopersTeamAccess"] devops_team_access["IdentityDevopsTeamAccess"] end end subgraph "core-identity" subgraph aws_teams["aws-teams"] devops_teams["devops"] developers_teams["developers"] spacelift_teams["spacelift"] end end subgraph "plat-prod" subgraph aws_team_roles_prod["aws-team-roles"] terraform_prod["terraform"] planner_prod["planner"] end end subgraph "plat-sandbox" subgraph aws_team_roles_sandbox["aws-team-roles"] terraform_sandbox["terraform"] end end developers_team["Developer Team"] --> developers_team_access devops_team["DevOps Team"] --> devops_team_access developers_team_access --> developers_teams devops_team_access --> devops_teams g_suite["G Suite"] -->|SAML Federated Identity| devops_teams spacelift["Spacelift"] --> spacelift_teams developers_teams --> planner_prod devops_teams --> terraform_prod spacelift_teams --> terraform_prod developers_teams --> terraform_sandbox devops_teams --> terraform_sandbox spacelift_teams --> terraform_sandbox end ``` ## FAQ ### Why use AWS SAML? SAML offers granular control over identity management with your IdP in AWS. With our `aws-saml` component, you can integrate multiple IdPs, and we explicitly provision every role, policy, and trust relationship needed. The `aws-saml` component allows users to assume any number of roles to access AWS accounts that are associated with their IdP groups. This component creates an Identity Provider (IdP) in the Identity account to allow federated access to an identity role. Follow the Identity Providers documentation for adding a SAML login. With AWS SAML, we create a federated SAML login that connects to the "team" in the identity account, and then users can assume other roles from there. We use the [AWS Extend Switch Roles plugin](https://github.com/tilfinltd/aws-extend-switch-roles) that makes this much easier, but it's not as intuitive as Identity Center. :::info What is a federated login? A federated login means that instead of managing separate credentials for each AWS account, users authenticate through a centralized identity provider (IdP). This allows them to access multiple AWS accounts or services using a single set of credentials, based on trust relationships established between the IdP and AWS. ::: ```mermaid flowchart TB subgraph "Human Access Architecture with AWS SAML" subgraph "core-identity" subgraph aws_teams["aws-teams"] devops_teams["devops"] developers_teams["developers"] billing_teams["billing"] end end subgraph "plat-prod" subgraph aws_team_roles_prod["aws-team-roles"] admin_prod["admin"] billing_prod["billing"] reader_prod["reader"] end end subgraph "plat-sandbox" subgraph aws_team_roles_sandbox["aws-team-roles"] admin_sandbox["admin"] billing_sandbox["billing"] reader_sandbox["reader"] end end g_suite["GSuite\nFederated Identity"] -->|SAML| devops_teams g_suite -->|SAML| developers_teams g_suite -->|SAML| billing_teams developers_teams --> reader_prod devops_teams --> admin_prod billing_teams --> billing_prod developers_teams --> admin_sandbox devops_teams --> admin_sandbox billing_teams --> billing_sandbox end ``` ### How do I use AWS IAM Identity Center (SSO)? AWS IAM Identity Center (SSO) is natively integrated with the AWS Web Console and the AWS CLI. You don't need to do anything special to get started with it once it's been properly configured. Specifically, the `aws-sso` component connects AWS IAM Identity Center (Successor to AWS Single Sign-On) Groups to Permission Sets. Permission Sets grant access to `aws-teams` in the Identity account or (optional) access to an individual account for convenience. Permission Sets other than the Identity Team Access Permission Set are used only for console and CLI access to a single account. These are redundant with AWS Team Roles but are useful for quickly accessing a given account with limited permission. With AWS IAM Identity Center, you can directly access a specific account from the single sign-on page, which will automatically assume the role with the associated `PermissionSet`. This is great for web console usage but not ideal for locally applying Terraform. With Terraform, we need to apply changes across many accounts, where each account has its own role. We don't want to assume a new role each time we change accounts. To solve this, we have a specific Permission Set that can assume an AWS team role in the identity account. When applying Terraform, we assume that "team" role via SSO, and when accessing the web console, we use the Permission Set directly. --- ## Deploy AWS Teams and Team Roles across your AWS Organization import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Note from '@site/src/components/Note'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import TaskList from '@site/src/components/TaskList'; Learn how to provision AWS teams and team roles across an AWS Organization using Atmos workflows, integrate them with AWS SSO, and generate AWS configuration files for role assumption with profiles. | Steps | | | ----------------------------- | ------------------------------------------- | | Install requirements | | | Vendor components | `atmos workflow vendor -f quickstart/foundation/identity` | | [Setup Identity Center](/layers/identity/aws-sso/) | Click Ops | | Add your SAML provider | Click Ops | | Deploy identity components | `atmos workflow deploy/all -f quickstart/foundation/identity` | | Reconfigure Terraform Backend | `atmos workflow deploy/tfstate -f quickstart/foundation/baseline` | | Deploy baseline components | `atmos workflow deploy -f quickstart/foundation/baseline` | The identity components are designed to provision all primary user and system roles into a centralized identity account. These components are expected to be used together to provide fine-grained role delegation across the account hierarchy. :::tip All identity components in this guide required elevated permissions to deploy, and in particular require Administrator access in `core-root` and `core-identity`. Assume the `SuperAdmin` IAM user role before continuing. For more on SuperAdmin, see [How to Create SuperAdmin user](/layers/accounts/tutorials/how-to-create-superadmin-user/). ::: ### Vendor Identity Components Pull the Identity components into your local repository. ### Setup Identity Center Follow the [Setup Identity Center](/layers/identity/aws-sso/) guide to enable AWS IAM Identity Center and connect your IdP. Once you've deployed Identity Center, take note of the AWS access portal URL and region. You can find this in the "Settings Summary" of the AWS Identity Center dashboard. We will need this to sign into AWS. ### Setup AWS SAML The majority of users prefer to solely use AWS Identity Center (`aws-sso`) to manage user integrations. If you are not using AWS SAML, then the `aws-saml` should be disabled. Verify that it's disabled in the stack catalog for `aws-saml` and move on: ```yaml # stacks/catalog/aws-saml/defaults.yaml components: terraform: aws-saml: vars: enabled: false # <---- Disable the component ``` Read more with [Why use AWS SAML?](/layers/identity/centralized-terraform-access/#why-use-aws-saml) and [Setup AWS SAML](/layers/identity/optional/aws-saml/) _If you are using `aws-saml`_, generate the metadata to connect your IdP with the federated identity in AWS. Once you have generated the metadata, store the file with the component's directory (`components/terraform/aws-saml`) in your repository and add the integration to your `aws-saml` stack catalog: ```yaml # stacks/catalog/aws-saml/defaults.yaml components: terraform: aws-saml: vars: enabled: true saml_providers: # Specify a descriptive key. This can be any arbitrary value. # Then set the value to the name of the metadata file. This must exist in the component's directory. # You might add something like this: acme-okta: OktaMetadata-acme.com.xml # For example, at Cloud Posse we use GSuite: cloudposse-gsuite: GoogleIDPMetadata-cloudposse.com.xml ``` ### Review AWS Teams Allowed GitHub Repositories There should be no action required for this step, but this is a very common issue. In the `aws-teams` component, we specify a map of trusted GitHub repositories with `var.trusted_github_repos`. This will connect a given AWS Team to the GitHub OIDC Provider. However, at this point, the GitHub OIDC provider is not provisioned. We need to comment out that variable until the provider is ready. Once ready, we will reapply the `aws-teams` component. ```yaml # stacks/catalog/aws-teams/defaults.yaml components: terraform: aws-teams: vars: # # Uncomment the following to enable GitOps for the GitHub Actions workflow. # This requires the `github-oidc-provider` to be deployed. # # trusted_github_repos: # gitops: # - "cloudposse-sandbox/infra-train:main" # planners: # - "cloudposse-sandbox/infra-train" ``` For more details, see [How to use GitHub OIDC with AWS](/layers/github-actions/github-oidc-with-aws/) and [Setup GitHub Actions](/layers/gitops/setup/). ### Deploy Identity Components Now that we have AWS Identity Center groups available, AWS SAML Metadata ready, and `vars.trusted_github_repos disabled`, deploy these components across all accounts by running the following command. Note that if any AWS Identity Center groups are missing, `aws-sso` will fail. After deploying the identity components, the above workflow will attempt to create local AWS configuration for all new roles and profiles. This will be generated with the `update-aws-config` workflow and stored in the `rootfs` directory locally. Everything included in the `rootfs` directory is passed to the Geodesic image when it is built. This will include the AWS configuration files, included with the `aws-config-teams` file, that are needed to assume roles across the AWS Organization. In particular, we recommend setting a common AWS profile, such as `acme-identity`, as the entry-point for all role assumption. That way all users can configure the same AWS profile name and be allowed to assume roles that they can access. The AWS config script requires local files, which are generated during the deployment of the AWS Team and Team Roles components. If you did not run these steps yourself and they are not committed to git, then you will need to rerun those components to generate these local files. For this reason, we recommend committing these generated files to git so that they are available for future use. If the `aws-config` executable fails, try exiting and rebuilding the Geodesic image! 1. ```bash Executing command: `atmos workflow update-aws-config -f quickstart/foundation/identity` Updating AWS Config files in /workspace/rootfs/etc/aws-config "aws-config": executable file not found in $PATH ``` 2. ```bash exit ``` 2. ```bash make all ``` 3. ```bash √ : [cptrain-SuperAdmin] (HOST) workspace ⨠ atmos workflow update-aws-config -f quickstart/foundation/identity Updating AWS Config files in /workspace/rootfs/etc/aws-config No changes ``` ### Reconfigure Terraform State Backend Apply the changes from the previous step to the Terraform State Backend by running the following commands. This will grant permission for your newly deployed AWS Teams and Team Roles to access the Terraform State Backend. ### Apply Changes to Geodesic To apply these updates to your Geodesic image, exit the image and rerun `make all`. Among the generated configuration files is `aws-config-teams`, which contains AWS authentication settings for all local users. This file is shared across SAML and SSO users, as well as all teams. It references a specific AWS profile for your namespace (e.g., `acme-identity`) and uses that profile as the entry point for role assumption. Users with appropriate permissions can then execute Terraform or assume other roles across the AWS Organization. Moving forward, you'll use your newly configured AWS Team instead of the `SuperAdmin` user. Continue with [Log into AWS](/layers/identity/how-to-log-into-aws/) Anytime that teams or roles are changed, rebuild AWS config --- ## Decide on AWS CLI Login import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import Note from "@site/src/components/Note"; import TaskList from "@site/src/components/TaskList"; Decide on a CLI tool that enables AWS login and credentials via SAML IDP for CLI and web console access. ## Problem Users need some way to login into AWS when using the CLI or applying Terraform changes. We have AWS Identity Center or AWS SAML setup for an AWS organization, but we need a way to login to AWS locally. There are a number of tools that can help with this, but we need to decide on one. ### Option 1: Use the AWS CLI First of all, we could use the AWS CLI to login to AWS. This is the most basic way to login to AWS, but it requires a lot of manual steps and is not very user-friendly. ### Option 2: Use Leapp Alternatively we could use Leapp by Noovolari. This is a tool that allows you to login to AWS using SAML and then automatically generates temporary credentials for you to use in the CLI. Once setup, Leapp makes it very easy to login to AWS and use the CLI, assume roles across your accounts with Role Chaining pre-configured, and even launch directly into the AWS web console. > ![IMPORTANT] > Leapp has been a popular choice for this use case, but with Noovolari announcing the shutdown of their paid service, this could raise concerns about the long-term viability of the project. While the [Leapp](https://github.com/Noovolari/leapp) project will continue to be supported, the discontinuation of the paid option might make it less appealing to future users. Leapp requires several manual steps during the initial setup, which has been a pain point for some users. See [How to Login to AWS (with Leapp)](/layers/identity/how-to-log-into-aws/) for more on the required setup and usage. Leapp requires setup steps outside of our Geodesic containers, which makes it less convenient for users who primarily work in the shell and increases the likelihood of configuration errors. ### Option 3: Use `aws-sso-cli` (AWS SSO Only) The most recent option we've come across is [aws-sso-cli](https://github.com/synfinatic/aws-sso-cli). This is a CLI tool that allows you to login to AWS using SAML and then automatically generates temporary credentials for you to use in the CLI. It is similar to Leapp, and is also open source and free to use. It also has a number of features that make it easier to use, such as the ability to login to multiple AWS accounts and roles at the same time. One potential benefit of `aws-sso-cli` is that it is a CLI tool, which means it could likely be integrated into our Geodesic containers. This would make it easier for users to login to AWS and use the CLI, and would reduce the risk of user configuration errors. However, `aws-sso-cli` is designed specifically for AWS SSO, which means it may not be suitable for users who are using AWS SAML. ### Option 4: Use `saml2aws` (AWS SAML Only) Another option is to use `saml2aws`, which is a CLI tool that allows you to login to AWS using SAML. It is similar to Leapp and `aws-sso-cli`, but is specifically designed for AWS SAML. This means it may not be suitable for users who are using AWS SSO. Most IdPs supported by `aws2saml` with the exception of Okta, depend on screen scraping for SAML logins, which is far from ideal. This approach can lead to issues, especially with services like GSuite that use bot protection, which occasionally disrupts users attempting to log in. Additionally, SAML providers differ in how they handle login processes and multi-factor authentication (MFA), meaning you may need to make specific adjustments to ensure smooth integration with your identity provider. If your organization uses Okta, then `aws2saml` is good option. ### Option 5: Use a browser plugin Another option is to use a browser plugin, such as [aws-extend-switch-roles](https://github.com/tilfinltd/aws-extend-switch-roles), that allows you to login to AWS using SAML. This is a simple and user-friendly way to login to AWS, but it requires you to use a browser and is not suitable for users who are working in the CLI. ### Option 6: Use a custom solution Finally we could build our own custom solution for logging into AWS. This would give us complete control over the process, but would require a lot of development effort and ongoing maintenance. ## Recommendation Cloud Posse continues to recommend Leapp for now, but we are evaluating alternatives. --- ## Decide on Identity Provider (IdP) Integration Method import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Ensure your organization can efficiently and securely manage access to AWS resources. By choosing the appropriate IdP integration method, either AWS Identity Center (SSO) or AWS SAML, you can align your authentication processes with their operational structures, avoiding potential overlaps and inefficiencies. ## Problem After [deciding on which IdP](/layers/identity/design-decisions/decide-on-idp) will be used, companies need to decide on how they will authenticate with AWS using that IdP. Organizations require efficient and streamlined methods to authenticate and manage access to their AWS resources. Without a centralized or user-friendly system, managing access across multiple AWS organizations or accounts becomes cumbersome and prone to errors. Multiple SSO authentication options exist within AWS. Choosing between AWS Identity Center (SSO) and SAML requires organizations to determine which method aligns best with their operational structure and goals. This choice can be daunting without clear guidance. Each authentication method comes with its own set of advantages and limitations. AWS SAML offers centralized access across multiple organizations but may be overkill for entities with a single AWS Organization. On the other hand, AWS Identity Center provides a user-friendly interface for single organizations but may not be as efficient for those managing multiple AWS accounts or organizations. Implementing both methods simultaneously can lead to potential overlaps, confusion, and inefficiencies unless managed correctly. Organizations need clarity on which method to adopt and an understanding of the trade-offs involved to ensure efficient and secure access to AWS resources. The best option depends on your unique organizational structures and user preferences. ## Solution Cloud Posse supports both AWS SAML and AWS Identity Center (AWS SSO) for authenticating with AWS. Choose one or both options. - **[AWS SAML 2.0 based federation](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_saml.html)** is provisioned in the centralized identity account and then permits roles to assume access roles in other accounts across the organization. It works well with multiple IdPs, enabling roles to be programmatically associated with specific providers. Aside from the obvious benefit of using AWS SAML to provide a single authentication page for users, is that the benefit that the AWS SAML approach enables granular control over all the mechanics, giving administrators ultimate control over how it works. Internally, Cloud Posse uses AWS SAML to authenticate with all customers to access many AWS Organizations easily. - **[AWS Identity Center (AWS SSO)](https://aws.amazon.com/iam/identity-center/)**, alternatively, is deployed for a single AWS Organization. With AWS Identity Center, we have a single access page for all accounts in the Organization and can connect directly to a given account. **AWS Identity Center is the recommended choice for customers**, given that most customers manage a single AWS Organization, and the single login page is the most user-friendly option. It's also ideal for business users, requiring no additional software configuration like Leapp to access resources through the AWS web console. However, it is limited to a single IdP, so companies that depend on multiple IdP's should consider other options. Both options can be deployed simultaneously. You can choose to have both or either option deployed. :::info Cloud Posse Access Since AWS Identity Center does not support multiple identity providers, we always deploy AWS SAML for Cloud Posse's access for the duration of our engagement. We do that to control access in our own team and to make it easier for the customer to cut off all of Cloud Posse's access at any time. For more on offboarding, see [Offboarding Cloud Posse](/jumpstart/tutorials/offboarding-cloudposse) ::: ## Consequences ### AWS Identity Center (AWS SSO) In order connect your chosen IdP to AWS SSO, we will to configure your provider and create a metadata file. Please follow the relevant linked guide and follow the steps for the Identity Provider. All steps in AWS will be handled by Cloud Posse. Please also provision a single test user in your IdP for Cloud Posse to use for testing and add those user credentials to 1Password. - [AWS Identity Center (SSO) ClickOps](/layers/identity/aws-sso/) ### AWS SAML If deploying AWS SAML as an alternative to AWS SSO, we will need a separate configuration and metadata file. Again, please refer to the relevant linked guide. - [GSuite](https://aws.amazon.com/blogs/desktop-and-application-streaming/setting-up-g-suite-saml-2-0-federation-with-amazon-appstream-2-0/): Follow Steps 1 through 7. This document refers to Appstream, but the process will be the same for AWS. - [Office 365](/layers/identity/tutorials/how-to-setup-saml-login-to-aws-from-office-365) - [JumpCloud](https://support.jumpcloud.com/support/s/article/getting-started-applications-saml-sso2) - [Okta](https://help.okta.com/en-us/Content/Topics/DeploymentGuides/AWS/aws-configure-identity-provider.htm) --- ## Decide on Identity Provider (IdP) import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import Note from "@site/src/components/Note"; import TaskList from "@site/src/components/TaskList"; Simplify AWS authentication by leveraging existing email providers or Identity Providers (IdPs), ensuring streamlined access management and ease of use for your team. ## Problem Users need a way to authenticate to AWS. ## Solution Verified working IdPs: - GSuite (Google Workspaces) - Office 365 (Microsoft Entra ID) - Okta - JumpCloud - Auth0 Cloud Posse recommends using your existing email provider (e.g. Google, Microsoft, etc) as the IdP, unless you already have a specialized one, such as Okta, Auth0, or JumpCloud. ## Consequences Follow the steps below to integrate your IdP of choice with AWS. Cloud Posse requires this information for your team to sign in to the new AWS Accounts. - [ ] Please create a temporary User in your IdP for the Cloud Posse Team. The Cloud Posse Team will use this account to verify access to several resources. For example `cloudposse@acme.com`. --- ## Design Decisions(4) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions of the Identity Layer. These decisions relate to how you will manage identity and access management (IAM) in your AWS accounts together with your Identity Provider (IdP). --- ## Access Control Architecture import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; This document describes the architecture and design concept of the access control system in the *current* Cloud Posse reference architecture, including the "Dynamic Terraform Roles" feature. For information about earlier versions of the access control architecture and design, See [AWS Access Control Evolution](/layers/identity/docs/aws-access-control-evolution). - A person's access to resources is controlled by IAM policies attached to IAM roles in AWS accounts. - The `aws-team-roles` component is deployed to an account to create its IAM policies and roles, attach policies to the roles, and configure access to the roles themselves. - The Cloud Posse reference architecture supports 3 mechanisms that allow a person to access (assume) a role in an account: - Cloud Posse's `aws-teams` component deployed to a dedicated `identity` account. This is based on recommendations from AWS and Google, and implements a hub-and-spoke pattern where users log into an IAM Role or Identity Center Permission Set in the `identity` account, and then access other accounts by assuming roles in them. We call the Role or Permission Set in the `identity` account a "team", because it acts in much the same way that "groups" do in other access control systems: many people can be members of a team, and the rest of the access control system is configured to grant or deny access to teams rather than individuals. - They can assume a role using by logging directly into it via an external Identity Provider (IdP) - They can assume a role using by logging into the AWS IAM Identity Center (formerly AWS SSO) via an external Identity Provider (not covered here) - For software running on resources in AWS accounts, AWS provides, and Cloud Posse supports, mechanisms for providing an initial role. Once the software is running, it can assume other roles as needed. - Terraform has special support and requirements (explained below). Logging in via an IdP is usually a multistep manual process that would be tedious and annoying to have to do multiple times an hour. Conversely, AWS provides tools that makes changing roles very easy, sometimes even automatic. Because of this, and because automated processes can only access roles (other than their initial roles) by changing roles, a hub-and-spoke pattern was developed where certain roles are defined as "identity" roles, meant to be initial points of access, and other roles, the ones used to interact with resources, are defined to allow access by being assumed from identity roles. This pattern is implemented by both the Cloud Posse reference architecture and AWS IAM Identity Center. As an administrator, you set up people in your Identity Provider (usually either Google or Okta) and configure their access to resources. (The details of how to do that are beyond the scope of this document.) For someone like a junior developer who only needs access to a single "sandbox" account, or an auditor who only needs access to the "audit" account, you can set up their access so that they can log in directly to a role in the account. For someone who needs access to multiple accounts, you can set up their access so that they can log into a role in the "identity" account (which Cloud Posse refers to as a "team"), and then can assume roles in other accounts from there. In order for an Identity Provider to be able to log someone into a role in an AWS account, the account must be configured to trust the Identity Provider to do so. This is done with the `aws-saml` component. This is detailed further below, but the key points are: - The `aws-saml` component must be deployed to every account the Identity Provider wants to log people into. - The `aws-saml` component should always be deployed to the "identity" account. - Typically, the `aws-saml` component is deployed once and only needs updating when certificates expire. It does not have to be updated when users, roles, or policies change. ## Details ### Policies, Roles, and Accounts The most fundamental layers of access control are the AWS account and the Identity and Access Management (IAM) roles and policies. For a person to be able to do anything, they need to be able to assume an IAM role in an AWS account, and that role has to have IAM policies attached to it that grant the permissions needed to do the work. #### Policies and Roles An IAM Policy defines a set of permissions that a user or service can be granted. A policy can be "attached" to a user, group, or role, although for our purposes we will limit our discussion to roles. The role has the permissions defined by all the policies attached to it. An IAM role groups permissions into a single entity that allows many users to share the same permissions by allowing them to "assume" the role. Several people can be allowed to assume the same role, and each person can assume multiple roles. AWS provides a number of "AWS managed" policies that provide common sets of permissions and are usually attached by themselves to IAM roles. For example, the `AdministratorAccess` role provides full access to all AWS services. The `PowerUserAccess` role provides full access to all AWS services except for management of users and groups. The `ReadOnlyAccess` role provides read-only access to all AWS services. The `ViewOnlyAccess` role provides read-only access to most AWS services except where such access might allow the user to see secret or sensitive information. AWS also allows users to define their own policies, called "Customer Managed Policies". #### Accounts An AWS account is a logical grouping of resources. The architecture of IAM Policies makes it difficult to allow access to some resources in an account but not others of the same kind. However, by default, any access granted to resources in one account does not grant access to resources in another account. So we use accounts to group resources that we want to share and separate out resources that we want to keep restricted. Cloud Posse's reference architecture uses a number of accounts to separate out resources in this way. Here is a sample of the accounts we use:
`plat-prod`
Holds the production resources for the company, the ones that provide the company's services to its customers.
`plat-dev`
Holds the development resources for the company, the ones that are used to develop the company's services.
`core-audit`
Holds the logs and other resources that are used to audit the other accounts.
Hopefully you can see how this separation allows us to give developers access to the resources they need, perhaps giving them full administrative access to the `plat-dev` account, while restricting their access to the `plat-prod` account to read-only access and cutting off all access to the `core-audit` account. At the same time, we can give auditors access to the `core-audit` account without giving them access to the `plat-dev` or `plat-prod` accounts. This separation helps to ensure that no one can make changes to critical resources and then cover their tracks by erasing audit trails. This is just one example of how the separation of accounts can be used to provide security. #### Create policies and roles in accounts Cloud Posse provides the `aws-team-roles` component to create policies and roles in accounts (and configure access to them, which will be discussed below). For each account, the `aws-team-roles` component creates a set of roles with specified policies attached. (See the `aws-team-roles` component's documentation for more details.) For ease of administration, auditing, and comprehension, Cloud Posse recommends that you use the same set of roles in each account. For example, you might have `admin`, `developer`, and `reader` roles in each account. The `admin` role would have the `AdministratorAccess` policy attached, the `developer` role would have the `PowerUserAccess` policy attached, and the `reader` role would have the `ReadOnlyAccess` policy attached. This way, you can easily see what permissions a user has in each account by looking at the roles they are allowed to assume. Cloud Posse creates 2 special roles in each account: `terraform` and `planner`. The `terraform` role is used by Terraform to create and manage resources in the account. Any user who needs to make changes to resources via Terraform should be allowed to assume this role. The `planner` role can be used by Terraform to plan changes to resources in the account. Users who need to be able to plan changes to resources via Terraform but who are not allowed to make changes should be allowed to assume this role. ### Identity and Authentication The next layer of access control is the identity and authentication layer. This layer is responsible for determining who is allowed to assume a role in an account and gain the corresponding permissions. #### Identity providers An identity provider (IdP) is a service that provides a way to authenticate a user and determine their identity. The identity provider provides a company with a central place to grant (and limit) access to services for its employees and other individuals. Identity providers are enabled account-by-account via the `aws-saml` component. The most common identity providers Cloud Posse's customers use are Google and Okta. These services provide a way for users to log in and prove their identity to the identity provider. The identity provider then provides a way for other services to verify that the user has been authenticated and is permitted to access the service. In the case of Google and Okta, this is done using [SAML](https://www.cloudflare.com/learning/access-management/what-is-saml/). #### Trust relationships Here is where things can start to become hard to follow. In order for a user to assume a role in an account, AWS needs to know that the user is authorized to do so. Each role in an account has a "trust policy" (also called an "assume role policy") that configures who can assume that role. A role's trust policy can be configured to allow other AWS roles, in the same or other accounts, to assume it. So, once you are in a role, that role can be used to grant you access to other roles. This is called "role chaining", and Cloud Posse's reference architecture leverages this heavily, as will be explained below, but you do not have to use role chaining if you prefer not to. A trust relationship between an IdP (using SAML) and an AWS account is established by creating what AWS calls an "IAM SAML Identity Provider" in the account. (Cloud Posse sometimes calls this as "SAML Connector".) Cloud Posse provides the `aws-saml` component to facilitate managing this via Terraform, but there remain some manual steps since the Identity Provider itself is not configurable via Terraform. (See the `aws-saml` component's documentation for more details.) Once the trust relationship between the IdP and the AWS account is established, each role within the account can use its trust policy to allow the IdP to assume it. This is how users get into their initial roles. #### The chain of trust Let's review two ways to get into a role in an account. Say that both the `plat-dev` and `plat-prod` accounts have the `terraform` and `planner` roles. We want to allow user Sandy to be able to assume the `terraform` role in the `plat-dev` account (in order to be able to make changes by running `terraform apply`) and the `planner` role in the `plat-prod` account (in order to see if changes need to be made, by running `terraform plan`, while not being allowed to actually make changes). ##### Option 1: Login per account One way to do this is for a user named Sandy is: 1. Configure your AWS Organization to use the AWS Identity Center (formerly AWS SSO). 2. Give Sandy access to Permission Sets in the target accounts with the desired access. 3. If you want Sandy to be able to run Terraform in the account, give that Permission Set access to the Terraform state via the `tfstate-backend` component. ##### Option 2: Role chaining Option 1 makes sense when a user only needs to assume one role in one account, or if a user has a unique pattern of access to different accounts. However, if a group of users all need to be able to assume the same set of multiple roles in multiple accounts, it is more convenient to use role chaining via what Cloud Posse calls `aws-teams`. Let's say that Sandy, from the previous example, is one of several developers who all need to be able to assume the `terraform` role in the `plat-dev` account and the `planner` role in the `plat-prod` account. We can create a team called `developers` and give all the developers access to that team. Then we can configure the `terraform` role in the `plat-dev` account and the `planner` role in the `plat-prod` account to allow the `developers` team to assume them. Now, when Sandy logs into the IdP and selects the `developers` team, Sandy will be able to assume the `terraform` role in the `plat-dev` account and the `planner` role in the `plat-prod` account. As an added bonus, for the special cases of using Terraform or accessing a Kubernetes cluster, we can configure the cross-account access to be automatic. This means that Sandy can log in once and access all the accounts and roles that Sandy needs to access without having to log in again or perform some kind of manual role switching. The `developers` team, and Sandy's access to it, is implemented as follows: - Deploy the `aws-saml` component to the `core-identity` account. - Create the `developers` team in `core-identity` using the `aws-teams` component, and set `aws_saml_login_enabled` to `true`. - Configure the `terraform` role in `plat-dev` and the `planner` role in `plat-prod` to allow the `developers` team to assume them by including the `developers` team in the `trusted_teams` list for each role. - Configure Sandy's account in the IdP to allow Sandy to assume the `developers` role in the `core-identity` account. Sandy can then log into the IdP and select the `developers` team to gain access to the `core-identity` account. From there, running any Terraform against any Cloud Posse Terraform component will automatically assume the `terraform` role when operating on the `plat-dev` account or the `planner` role when operating on the `plat-prod` account. Similarly, via the `eks/cluster` component, the `developers` team can be given access to any RBAC role in any EKS clusters in any account, as desired. For example, `developers` can be given access to `cluster-admin` in the `plat-dev` cluster and `view` in the `plat-prod` cluster. They can then use `kubectl` and other Kubernetes tools to access the clusters without having to log in again or perform some kind of manual role switching ( though of course they will need to switch `kubectl` contexts). ## Terraform Support Because of the special role Terraform plays in maintaining the infrastructure, the Cloud Posse reference architecture includes special support for Terraform. To use Terraform, you need 2 kinds of access: 1. Access to the Terraform state backend (S3 and DynamoDB) 2. Access to the AWS account where the infrastructure is deployed When granting access to Terraform to a new user, you must ensure that the user has both kinds of access. ### Access to the Terraform state backend - The `tfstate-backend` component manages the Terraform state backend (S3 and DynamoDB) in the `root` (organization management) account. - It creates a special role in the `root` account for read/write access to the Terraform state bucket and DynamoDB table. - In order to use Terraform to manage the infrastructure, you must be able to assume this role. Assuming the role is handled automatically, but you still need to have permission to assume the role. This permission is granted by `tfstate-backend` (`allowed_roles`). For complicated technical reasons, at present anyone who needs any kind of access to Terraform needs to be able to assume the role that gives them read/write access to the entire Terraform state backend. This is mainly a limitation of Terraform itself. There are some workarounds, but they all have significant trade-offs. Contact Cloud Posse if you need to discuss this further. ### Access to the AWS account where the infrastructure is deployed For users who directly log into an account, they can use Terraform to manage the infrastructure in that account to the extent their IAM permissions allow. However, their role needs to be explicitly granted access to the Terraform state backend (see previous section). For users in teams, they should run Terraform from the team access role in the identity account. This role also needs explicit access to the Terraform state backend, but this is usually enabled in the initial configuration. The teams' access to Terraform is implemented as follows: - The `aws-team-roles` component manages the roles in the AWS accounts. - It creates 2 special roles in each account to allow Teams to run Terraform in that account: `terraform` and `planner`. - To enable a team to run `terraform plan` and `terraform apply` in an account, add the team to the `trusted_teams` list for the `terraform` role in that account. - To enable a team to run `terraform plan` but NOT `terraform apply` in an account, add the team to the `trusted_teams` list for the `planner` role in that account. ### Roles are assumed automatically When you run Terraform, 2 roles are automatically assumed: 1. The role that gives you access to the Terraform state backend (see earlier section) 2. The role that gives you access to the AWS account where the infrastructure is deployed ```mermaid flowchart TD U[User] -->|Assume Terraform State access IAM role| S[(Terraform State)] U -->|Assume terraform or planner IAM role| A[[AWS account's Resources]] ``` As a team member, the role for #2 is `terraform` if you are allowed to access it, otherwise it is `planner`. However, if you are not allowed access to either `terraform` or `planner`, then your current role is checked: - If your current role is a role in the target account, then that role will be used (no role will be assumed) - If your current role is a role in the `root` account, then the target account's OrganizationAccountAccessRole role will be assumed - Otherwise, Terraform will try to assume the `terraform` role in the target account and would be expected to fail --- ## Access Control Evolution import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Overview Unsurprisingly, Cloud Posse has evolved its approach to access control over time. This article documents that evolution in terms of "feature sets" that were added over time. The intent is to help people understand which version of the access control system they are currently using, how it integrates with current and future versions of the Reference Architecture and Components, what features are available but not yet imported into their system, and what are the benefits of upgrading. This article is not intended as an upgrade guide. Upgrade guides are scattered in release notes and individual component READMEs and CHANGELOGs. This article is intended to help you understand what you have and what you can have, and to help you decide when and if you want to upgrade. This article does not cover allowing GitHub actions to access AWS via OIDC. ### Features are Both Code and Configuration The features described here are implemented in both code and configuration. Some upgrades require changes to all of the related components, some only to some of them. Sometimes, only a single component needs to be upgraded. Sometimes, if the components are already feature-capable, then only configuration changes are needed. The components involved are: - `account-map` - `aws-saml` - `aws-sso` - `aws-team-roles` - `aws-teams` - `tfstate-backend` The configuration involved is usually a change to the stacks, but if the input or local variable name begins with `overridable_`, then it must be overridden with a `*_override.tf` file in the same directory as where the variable is defined, and the override accomplished by changing the default or local value. ### Component Versions The feature set descriptions below may reference specific versions of the [terraform-aws-components](https://github.com/cloudposse/terraform-aws-component) and/or Pull Requests (PRs) in that repository. The versions and PRs refer to where the feature was first introduced, as that is often the best place to find upgrade information and more details about the feature. However, there are often refinements and bug fixes in later versions, so at all times it is advisable to use the latest version of the component consistent with the feature set you are using. Pull Request numbers are used because until the pull request is merged, there is no version number associated with it. ## Roadmap These features are being considered for future implementation. If you wish to have them, please contact Cloud Posse to discuss how to get them implemented. ### SSO Permission Sets Automatically Acquire Team Permissions When the SSO Permission Set corresponding to a Team is created, it should automatically be granted the permissions that the Team has in the `identity` account. Currently, the sets have minimal permissions in the `identity` account, although they have the exact same ability to assume roles in other accounts as the Team does. ## Feature Sets (reverse chronological order) ### SSO Permission Sets as Teams - Components [version 1.238.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.238.0) - Components [PR 738](https://github.com/cloudposse/terraform-aws-components/pull/738) - Required component updates: `aws-sso`, `account-map` - Required configuration update complexity: minimal - Optional compatibility settings: - `account-map/modules/roles-to-principals`: - `overridable_team_permission_set_name_pattern`: Set to `Identity%sRoleAccess` if your existing permission sets end with "RoleAccess" rather than "TeamAccess" - `overridable_team_permission_sets_enabled`: Set to `false` to disable this feature - Forensics: You have this version or later if `account-map/modules/roles-to-principals/variables.tf` contains the `overridable_team_permission_sets_enabled` variable. #### Before Previously, for each `aws-team`, there was a configuration in `aws-sso` that enabled automatically creating a corresponding `IdentityTeamAccess` permission set in the `identity` account. This permission set was then granted the ability to assume the corresponding `aws-team` IAM role in the `identity` account. This allowed SSO users to have a workflow like this (in this example, they are accessing the `devops` team): - Log into AWS SSO, selecting the `identity` account and the `IdentityDevopsTeamAccess` permission set. - Assume the `-core-gbl-identity-devops` role in the `identity` account in either of 2 ways: - Use the "chained session" feature of Leapp to assume the role (preferred) - Use the Geodesic `assume-role` command-line function to assume the role #### After With this update, the Team Access permission sets will directly be able to assume all the same roles as the corresponding `aws-teams` can. This eliminates the need to assume the `aws-team` IAM role in the `identity` account via chained sessions or an explicit `assume-role` command. --- ### SAML Login High Availability - Components [version 1.238.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.238.0) - Components [PR 738](https://github.com/cloudposse/terraform-aws-components/pull/738) - Required component updates: `aws-saml` - Required configuration update complexity: none, but `aws-saml` must be reapplied to take advantage of the new features. - Forensics: You have this feature if `aws-saml/main.tf` allows for multiple values of the `SAML:aud` policy variable, and not only `https://signin.aws.amazon.com/saml` #### Before Although the `aws-saml` component is global in scope, the trust policy that was being deployed required you to log in via `us-east-1`. If that region was having an outage, you could not log in. #### After The IdP's trust policy has been updated to allow you to log in via any region. To change login regions, just update the IdP's `ACS URL` to use the [AWS sign-in endpoint](https://docs.aws.amazon.com/general/latest/gr/signin-service.html) for the region you want to log in via. Keep in mind that the IdP may take a few minutes to make the change available to your users. Depending on your priorities, you may want to change your default `ACS URL` to use the AWS sign-in endpoint for either your primary or fail-over region. Setting it to your fail-over region will allow you to log in even if your primary region is having an outage. --- ### SAML Access Auditing - Components [version 1.238.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.238.0) - Components [PR 738](https://github.com/cloudposse/terraform-aws-components/pull/738) - Required component updates: `account-map`, `aws-saml`, `aws-sso` - Required configuration update complexity: none, but the following components must be reapplied to take advantage of the new features: - `aws-saml` - `aws-sso` - `aws-team-roles` - `aws-teams` - `tfstate-backend` - Forensics: You have this feature if every trust policy for user roles (not necessarily EKS IRSA, GitHub Actions roles, or AWS service-linked roles) has the `sts:SetSourceIdentity` permission. Search for relevant trust policies by looking for the `sts:TagSession` permission. #### Before Nearly all of the work done in deploying and maintaining resources is done by "assumed roles". Auditing who the real person is that is doing the work was difficult to impossible, and probably could be circumvented by a malicious actor. This is because the only indication of user's identity was in the IAM session name, which can be changed by the user. #### After To make auditing who is using an assumed role easier and more reliable, AWS introduced the concept of a [source identity](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_control-access_monitor.html) that can be set by the identity provider (IdP) when the user logs in and cannot be changed after it is set. In order for that to be useful, the IdP and all the roles to be assumed must have the `sts:SetSourceIdentity` permission. (See [Permissions required to set source identity](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_control-access_monitor.html#id_credentials_temp_control-access_monitor-perms).) With this update: - `aws-saml` will automatically add this permission to the IdP - `aws-team-roles` will automatically include this permission in the trust policies of all the roles it creates - `aws-teams`'s minimal `team-role-access` policy includes this permission **Important Note:** This update does **NOT** configure the IdP to set the Source Identity. That must be done in the IdP itself, by including the `https://aws.amazon.com/SAML/Attributes/SourceIdentity` attribute in the SAML assertion. See [SourceIdentity SAML attribute](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml_assertions.html#saml_sourceidentity) for more information. However, once that attribute is added to the SAMl assertion, the Source Identity should work automatically from there. --- ### Dynamic Terraform Roles, Normalization of Team Roles, Better Team and Team Role IAM Policy Management - Introduced in Components [version 1.227.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.227.0) - Due to refinements, the minimum recommended version for this feature is [version 1.242.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.242.0) - Components [PR 715](https://github.com/cloudposse/terraform-aws-components/pull/715), further refined in PRs - [730](https://github.com/cloudposse/terraform-aws-components/pull/730) - [740](https://github.com/cloudposse/terraform-aws-components/pull/740) - [748](https://github.com/cloudposse/terraform-aws-components/pull/748) - Required component updates: - `account-map` - `aws-team-roles` - `aws-teams` - `tfstate-backend` - ALL components that want to take advantage of the new features need to be updated. Ideally, the whole component would be updated, but in most cases, it will be sufficient to update `providers.tf` within the component to v1.238.0 or later. - Required configuration update complexity: none if you update to v1.242.0 or later and do not want any of these features - Full implementation configuration update complexity: large, but easy to understand - Forensics: You have this feature if - The `account-map` component includes the `dynamic-roles.tf` file - The `aws-teams` and `aws-team-roles` components include an `additional-policy-map.tf` file - The `tfstate-backend/iam.tf` file has a statement like this (line 38 in v1.227.0), including `local.caller_arn` in `allowed_principal_arns`: ```hcl allowed_principal_arns = concat(each.value.allowed_principal_arns, [local.caller_arn]) ``` :::warning Using profile names instead of IAM roles ARNs is deprecated For a brief period, the Cloud Posse architecture suggested using `AWS_CONFIG_FILE` profiles instead of IAM role ARNs to assume roles in target accounts, due to insufficient support for using IAM role ARNs with AWS SSO. With this feature release, support for using profiles is deprecated, because the Cloud Posse reference architecture has evolved to the point where there is no longer any benefit (and there are some drawbacks) to using profiles. While historical use of profiles will continue to work, the features described here will not work with profiles. If you are using profiles, we recommend you switch to IAM roles and SSO Permission Sets if you are unsatisfied with the support of profiles. ::: #### Summary of new features - A new class of access, "Terraform plan only", can be given to Teams. Team members can preview changes but not make them. - Users in production accounts (not assigned to "teams") can now be allowed to run Terraform in those accounts with whatever permissions the user has in those accounts. This is particularly useful for developers or others who only need access to a small number of accounts. - Most components that previously could only be managed by SuperAdmin can now be managed by any user with the appropriate permissions. - It is now easier to understand the relationships and distinctions between "Teams" and "Team Roles" because there is no longer any overlap in names and there are no more special cases. - Configuration files for the `aws` CLI and the "AWS Extend Switch Roles" browser plugin are now generated from automatically generated Terraform outputs, so they can be updated by running a simple Atmos workflow. - Most custom IAM policies have been removed in favor of using AWS-managed policies. - Obsolete, dedicated outputs from `account-map/modules/iam-roles` in support of `helm` and `cicd` teams have been disabled to discourage future use, but not completely eliminated to minimize the impact on existing code which may reference the outputs without actually using them. #### Background Both before and after this update, all Terraform usage requires the use of 3 or 4 AWS IAM roles: - The "user" role: The initial role of the user when executing the Terraform command (with or without `atmos`) - The "backend" role: The role assumed by Terraform to access the Terraform State backend to read and update the state - The "remote state" role (when needed): the role assumed by Terraform to access the Terraform State backend to read the state of other components - The "Terraform assumed role": The role automatically assumed by Terraform in the target account to make the requested changes Note that the "remote state role" and the "Terraform assumed role" can be configured per user, the other 2 roles cannot. The "user" role is, of necessity, whatever IAM role the user presents credentials for. The "backend" role is the role that Terraform is configured to assume to access the Terraform State backend, which must be accessible before any other configuration can take place. (While it would be possible to use Atmos to configure a backend role based on the user's role, that would make it impossible to use Terraform without Atmos, which is a situation we want to avoid.) ##### The usage of the "user" role Of necessity, the user issuing a Terraform (or Atmos) command can only be doing so with the credentials of a single IAM role. This role is called the "user" role. In a desire to simplify operations, the other roles necessary to execute the Terraform command are assumed by Terraform itself, and the user role is only used to access these other roles. However, the user role is still a full-fledged IAM role, and can be used to access any AWS resources that the user has permissions to access. So for security reasons, it is desirable to limit the direct permissions of the user role and to limit access to the roles under the principle of "least privilege". So to begin with, we create a separate account called the "identity" account which exists for the sole purpose of managing "user" IAM roles, which we now refer to as "aws teams". This means that even if a user gets full Administrator access to the account, they will not be able to access any AWS resources other than IAM roles. In particular, they will not be able to make changes in the Organization `root` account, which is the most sensitive account, unless they are further granted permission to assume a role in the `root` account. In many organizations, it is desirable to restrict even the most privileged users from being able to routinely manage the most sensitive resources, such as creating or changing IAM roles and policies in the `root` or `identity` accounts, so they may grant or restrict the access of the user roles accordingly. ##### The SuperAdmin role Before any of the user (or other) IAM roles can be created by Terraform, a role must be created manually (via "Click Ops") to allow Terraform to create the other roles. This role is called the "SuperAdmin" role. It is created in the `root` account and has full AdministratorAccess permissions in the `root` account. It can also assume the `OrganizationAccountAccessRole` in any other account, giving it full AdministratorAccess to all accounts. Because this role is a technical necessity, some organizations choose to use it as the only way to make changes in the sensitive accounts, while others choose to enable certain teams to also make changes to them. In any case, creation of the IAM roles that make up "user" roles, and all the prerequisites for them, such as creating a Terraform state backend and the AWS accounts themselves, must be done by the SuperAdmin role. ##### Technical limitations of the Terraform state backend and "remote state" **Terraform state backend**: Terraform stores information about how it expects resources are currently deployed in a "state file" in the Terraform state backend. This state file is updated whenever Terraform makes changes to resources. When Terraform is run, it uses the state file and the current configuration to determine what changes need to be made to bring the resources into the desired state. In this situation, this is called the "backend" or "Terraform state". **Remote state**: Terraform can also read the state files of other components to determine how they are configured and use that information to guide the creation of the current component's resources. For example, a component can access the state of the `vpc` component to determine the VPC ID and subnets to use for its resources. This is called "remote state", although it is stored in the same Terraform state "backend" as the current component's state. **Technical limitations**: The Terraform state backend is made up of 2 components: an S3 bucket and a DynamoDB table. The S3 bucket stores the state files, and the DynamoDB table is used to lock the state files to prevent concurrent updates and detect data corruption. Because the Terraform state backend is provisioned when only the `root` account exists, the S3 bucket and DynamoDB table are provisioned in the `root` account. While the S3 bucket supports cross-account access directly, via bucket policies, the DynamoDB table does not. So to allow users in the `identity` account to access the Terraform state backend in the root account, we must create an IAM role in the same account where the DynamoDB table is provisioned (the `root` account), and allow the users to assume that role. This role is called the "backend" role or the "Terraform state read/write role". To read the "remote state" of other components, we can either use the "backend" role or create a separate role for that purpose. This role is called the "remote state" role or the "Terraform state read-only role" (although, to be useful, the "read-only" role still has to have read/write access to the DynamoDB table). Before the user roles are created in the `identity` account and the backend role is created in the `root` account, only the SuperAdmin role (which is also in the `root` account) can access the Terraform state backend. ##### The "Terraform assumed role" In order to make changes to AWS infrastructure in an account, Terraform must assume a role in that account. The "Terraform assumed role" is the role that Terraform assumes in the target account to make the requested changes. #### Before Prior to this update, the design and configuration of these roles and the access to them evolved over years, as new features were added to Terraform and as Atmos was introduced and developed, without a holistic (re-)design process, resulting in a hodgepodge. The following description of the resulting system is presented to describe it, not justify it. ##### Confusion between "team" and "team role" names, inconsistent usage Although components [PR 438](https://github.com/cloudposse/terraform-aws-components/pull/438), version 1.27.0, brought a change from `iam-primary-roles` and `iam-delegated-roles` to `aws-teams` and `aws-team-roles`, the change was not complete. The components were renamed and the implementation changed to make them separate, but the configuration was not updated to follow through on the concept. The teams had the same names as the team-roles (e.g. `admin`, `terraform`) and for security reasons, the `terraform` role in the `root` and `identity` accounts did not behave like they did in other accounts. In the `identity` account, the `terraform` role was a team role without permission to make changes in the `identity` account. In the `root` account, the `terraform` role was the "Terraform state backend" role and again had no permissions to make changes in the `root` account. The "Terraform assumed role" for the `root` and `identity` accounts was the `admin` role. ##### The "Terraform state backend" role was managed by `aws-team-roles`, causing problems for remote state The "Terraform state backend" role was `root-terraform` and was deployed by the `aws-team-roles` component in the `root` account. Because it was managed by `aws-team-roles`, it was not available until after all the team roles had been provisioned. This meant that any components that needed to be deployed before the team roles were provisioned could not use the Terraform state backend role, and had to be provisioned by SuperAdmin and _only_ SuperAdmin. Furthermore, it meant that no Terraform state backend role could be configured for the component, but the role would not be available for SuperAdmin to assume. Although only SuperAdmin could provision certain components, we still wanted any component to be able to read the component's remote state. We had been using the backend role for this purpose, but now we had components with no backend role configured. This led to the creation of the concept of a "remote state" role. This configuration was only used to look up remote state and could be set to use the backend role for components that were deployed after the role was created. Unfortunately, SuperAdmin still could not use this remote state role, so we added an extra flag called `privileged`. When set to `true`, the remote state role would be ignored and the user's role (presumably SuperAdmin) would be used as the remote state role instead. In sum: backend access was controlled by 3 settings: `backend.s3.role_arn`, `remote_state_backend.s3.role_arn`, and `var.privileged`, and still was not as flexible as we would like, because it still meant that only SuperAdmin could manage certain components. ##### Each component could have only one "Terraform assumed role" per account For each component, there was a single mapping of accounts to "Terraform assumed roles". This meant that every user either had full access to make changes or no access at all, because they either were or were not allowed to assume the "Terraform assumed role" for that account. There was no ability to give users partial (e.g. read-only) access, and no ability to give SSO users logged directly into a target account to use their "user" role's permissions to have tailored access to make changes. Furthermore, it meant that if SuperAdmin needed to be able to Terraform the component, no one else could Terraform it, because SuperAdmin could not assume any of the roles that other users could assume. ##### AWS configuration files were manually maintained An `aws-accounts` script was manually maintained and used, among other things, to generate configuration files for the `aws` CLI and the "AWS Extend Switch Roles" browser plugin. Updating this script was a manual process that was error-prone and tedious, and frequently neglected. As a result, new accounts or new roles were not added to these configuration files in a timely manner. ##### Provided IAM policies were static even though based on dynamic AWS managed policies The `aws-teams` and `aws-team-roles` components provided a set of IAM policies that could be assigned. However, in several cases, these policies were run-time concatenations of AWS managed policies, which meant that when AWS updated the policies, the updates did not automatically flow through to the roles. This became a live problem [when AWS updated the Billing IAM actions](https://aws.amazon.com/blogs/aws-cloud-financial-management/changes-to-aws-billing-cost-management-and-account-consoles-permissions/) and the policies needed to be updated. #### After ##### Team Role names are normalized and distinct from Team names Through configuration changes, along with support from `account-map`, the names of the team roles are now normalized so that all accounts use the `terraform` role as the default Terraform assumed role. The `root` and `identity` accounts are no longer treated specially. The Team names are distinct from the Team Role names, allowing `aws-team-roles` to be deployed to the `identity` account. This completes the separation of the concepts of "team" and "team role", with "team" being analogous to a "group" in a traditional access control system, in that it is a single entity that allows access to multiple other entities. ##### The "Terraform state backend" role is managed by `tfstate-backend`, allowing it to be used by all The "Terraform state backend" role is now managed by `tfstate-backend`, meaning it is created as part of the first Terraform step, which enables it to be used by all users and components. This means that SuperAdmin can now assume the role, so all components can be configured with the backend role, and the `remote_state_backend.s3.role_arn` configuration and the `privileged` flag are no longer needed. This removes one of the barriers to allowing users other than SuperAdmin to manage components that are first created before the team roles are created. It also means that, if desired, users in accounts other than the `identity` account can be given access to the Terraform state backend role, allowing them to use Terraform to manage the account associated with their "user" role. ##### Each component can have multiple "Terraform assumed roles" per account Previously, the "Terraform assumed role" for each account (after normalization) was the `terraform` role. The `terraform` role still works as before, and with this upgrade there are 2 new possibilities: 1. There is automated support for a new Team Role called `planner`, the intention being that `planner` would have enough permissions to run `terraform plan` and see what would change if a Terraform `apply` command were run, but not enough permissions to actually make the changes. It can be used for drift detection or to preview changes. 2. If a user does not have permission to assume the `terraform` or `planner` roles, then Terraform will not assume any role, and instead will use the user's role to make changes. This allows SSO users to log directly into a target account and use their "user" role's permissions to have tailored access to make changes. It also allows SuperAdmin to make changes in the `root` account while still allowing authorized Team members to make changes using their Team roles. ##### Updates to AWS configuration files are mostly automatic The `aws-accounts` script has been replaced by the `aws-config` script, which uses files automatically generated by Terraform components `account-map`, `aws-teams`, and `aws-team-roles` to generate configuration files for the `aws` CLI and the "AWS Extend Switch Roles" browser plugin. After updates to the relevant components, the configuration files can be updated by running the following: was manually maintained and used, among other things, to generate configuration files for the `aws` CLI and the "AWS Extend Switch Roles" browser plugin. Updating this script was a manual process that was error-prone and tedious, and frequently neglected. As a result, new accounts or new roles were not added to these configuration files in a timely manner. ##### Custom IAM policies based on AWS-managed Policies have been removed The provided IAM policies are now limited to non-AWS-managed policies, which means that the policies are truly custom and cannot benefit from automatic AWS updates. Policies that can benefit from automatic AWS updates are now configured directly in the Team and Team Role configuration. So instead of the custom, non-updating `billing_read_only` policy: ```yaml billing: role_policy_arns: - "billing_read_only" ``` You would use the AWS-managed `job-function/Billing` policy: ```yaml billing: role_policy_arns: - "arn:aws:iam::aws:policy/AWSBillingReadOnlyAccess" ``` This gives you the benefit of automatic updates to the policy, and you can still customize the policy by adding an additional custom policy if you want to. --- ### Preserve Custom `aws-teams` and `aws-team-roles` IAM Policies on Upgrade - Components [version 1.216.1](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.216.1) - Components [PR 697](https://github.com/cloudposse/terraform-aws-components/pull/697) - Required component updates: `aws-teams`, `aws-team-roles` - Required configuration update complexity: none - Forensics: You have this feature if the `aws-teams` and `aws-team-roles` components have an `additional-policy-map.tf` file #### Before ##### Custom IAM policies were overwritten on upgrade In order to add your own custom IAM policies to `aws-teams` or `aws-team-roles`, you had to modify the component code itself. This meant that when you upgraded the component, your changes would be overwritten. #### After ##### Custom IAM policies are preserved on upgrade Users can now create custom IAM policies and integrate them with the `aws-teams` and `aws-team-roles` components by taking advantage of the Terraform [override feature](https://developer.hashicorp.com/terraform/language/files/override). This allows users to add custom IAM policies to Teams and Team Roles that will not be overwritten by updates to the components, while still getting any new IAM policies added by the component. --- ### Conversion From iam-primary-roles to aws-teams **Special note**: Although this section documents the conversion from `iam-primary-roles` to `aws-teams` as it was first introduced, several advancements (documented above) have been made since then. The recommended approach for converting from `iam-primary-roles` to `aws-teams` is to upgrade to the latest versions of the `account-map`, `aws-teams`, `aws-team-roles`, `aws-saml`, `aws-sso`, and `tfstate-backend` components in one large upgrade rather than incrementally. This will allow you to apply the latest recommended configuration and take advantage of the latest features, including semi-automatic generation of the AWS config files, and run a single migration of your authentication workflows, rather than suffer through multiple migrations with customized intermediate configurations. - Components [version 1.27.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.27.0) - Components [PR 438](https://github.com/cloudposse/terraform-aws-components/pull/438) - Required component updates: replace `iam-primary-roles` and `iam-delegated-roles` with `aws-teams` and `aws-team-roles` - Required configuration update complexity: complex, should be handled by Cloud Posse Professional Services - Forensics: You have this feature if you have `aws-teams` and `aws-team-roles` components. #### Before Before the development of Atmos, it was difficult to create DRY configurations for IAM roles and policies. The reference architecture mitigated this for creating the numerous IAM roles and policies needed to support Terraform and other operations in all the accounts by having a single component, `iam-primary-roles`, do double-duty, both creating the roles used in the `identity` account and creating templates for the roles used in the other accounts. Other accounts deployed the `iam-delegated-roles` component and only had to configure deviations from the templates. This was incredibly confusing to configure properly and was a constant source of errors. It also made it difficult to add new roles to a subset of accounts, or to integrate with AWS SSO permission sets. Additionally, access to the Terraform backend was via a role provisioned by the `iam-delegated-roles` component in the `root` account, which caused 2 problems: - The role was not available until after the `iam-delegated-roles` component was deployed, which meant that components that needed to be deployed before the `iam-delegated-roles` component could not use the Terraform backend role, and had to be provisioned by SuperAdmin and _only_ SuperAdmin. - The role for accessing the Terraform backend was the standard `root-terraform` delegated role with modified, very limited permissions, which meant that to actually run Terraform, you had to use a different role (the `admin` role) which had full permissions. This exception to the rule of which role Terraform would assume to perform updates contributed to confusion and misunderstanding about who could do what, and how. #### After The conceptual framework of access control has been clarified, and with the support of Atmos, simplified while remaining DRY. Also, by the time you get to version 1.238.0, it is well integrated with AWS SSO permission sets. The `aws-teams` component is deployed to the `identity` account to create what we refer to as "Teams", which are analogous to "groups" in a traditional access control system. Users are assigned to one or more Teams via the Identity Provider (usually Okta or Google Workspaces, and possibly via the Identity Provider's "group" mechanism). Each Team has access to a collection of "Team Roles" (standardized AWS IAM Roles deployed by the `aws-team-roles` component) in a collection of AWS accounts (including possibly all accounts). All teams and team roles configure their own permissions and what other principals can assume them. Teams and AWS SSO Permission sets can be configured as principals allowed to assume the roles using symbolic names. Other principals can be configured using ARNs. Inheritance and DRY is provided via Atmos stacks. As of version 1.242.0, the wart of using the `root-terraform` role to access the Terraform backend has been removed, and the `terraform` role is now used consistently in all accounts. The `tfstate-backend` component is now used to create the Terraform backend role, now called `tfstate`, and it is available as soon as the Terraform state backend is created, which is the first step in the Terraform process, and means it is available to SuperAdmin for all subsequent steps, which means that all the components can be configured to use it, which means normal Terraform users can manage nearly all components. --- ## AWS Access Control Overview import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; With the exception of “root accounts” (beyond the scope of this article), all access to AWS resources is controlled by [AWS Identity and Access Management](https://aws.amazon.com/iam/), usually referred to by its initials IAM (pronounced “I am”). People accessing AWS resources in the normal course of duty, in any of the configurations recommended and supported by Cloud Posse, exclusively access AWS resources by “assuming” IAM [Roles](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html). Whether or not you are using the `aws` CLI or other tools that work with AWS, credentials are handled relatively consistently via the AWS SDK (published as a library in several programming languages). Once you [configure the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) correctly, all the other tools should work, too. The `$AWS_CONFIG_FILE` (by default, `$HOME/.aws/config`) contains _profiles_ that specify (among other things) what AWS IAM Role to assume when using that profile and where to get the credentials that allow you to use that profile. These profiles can be categorized into "Primary" and "Derived" profiles. (I don't think AWS has terminology for this, so we invented our own. If they do and someone would inform us about them, then we will use theirs if it is not too confusing.)
Primary Profile
A Primary profile has credentials provided directly. When you log in via Leapp, Leapp provides the credentials for the profile you specify in the Leapp session. `aws-vault` does the same kind of thing. AWS SSO does, too, although a bit differently. The main thing is that a Primary profile does _not_ have a `source_profile` entry.
Derived Profile
A Derived profile _does_ have a `source_profile` entry. It does not get credentials on its own, but instead, it gets credentials from the source profile it references. To use a derived profile, the source profile has to have valid credentials, and it has to be allowed to assume the role specified in the Derived profile. Note that it is _not_ a requirement that the source profile is a Primary profile. The source profile can be another Derived profile; all that matters is that the chain of source profiles end at a role with valid credentials and that at every step in the chain, the source profile is allowed to assume the role specified in the Derived profile that references it.
In the Cloud Posse reference architecture you must use Leapp to log into Primary profiles. Once logged into a Primary profile, any tool can use that profile or any Derived profile simply by specifying which profile to use (usually by setting the AWS_PROFILE environment variable via `export AWS_PROFILE=profile-name` or via a command-line flag, configuration string, or configuration file, such as in the case of `kubectl` where it is specified via the `users.user.exec.env` section of the `$KUBECONFIG` file). The AWS documentation and examples are confusing. In this scenario, it is important that you _do not include_ `mfa_serial` _in the Derived profile configuration_. The use case for having `mfa_serial` in the Derived profile is if you did not use MFA to authenticate to your Primary profile. In that (rare) case, the Derived profile really turns into a sort of Primary profile, because it is responsible for providing a credential (the MFA token) that the source profile does not have. That is when you need `mfa_serial` in the Derived profile and when you should put it in Leapp instead of just the `$AWS_CONFIG_FILE`. If you put `mfa_serial` in your Derived profile, you will repeatedly and needlessly be prompted for an MFA token every time you assume the role, which, because it is a chained role, will be at least once an hour, and that prompt may not be visible because it is coming from the AWS SDK. With your Primary profile, you only get prompted when your session expires, which can be as long as 12 hours. --- ## Restricting Admin Access to Sensitive Accounts import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; The Cloud Posse Reference Architecture provides standard workflows (described elsewhere) for managing resources in a multi-account AWS organization via Terraform. Those workflows cannot apply to certain very sensitive operations due to technical limitations, and by design should require some kind of elevated privilege to perform in sensitive accounts. This document describes options for how to restrict access and perform those operations. In particular, it describes the differing options and configurations available with different versions of the Cloud Posse Reference Architecture. #### What Are the "Sensitive" Accounts? In the Cloud Posse Reference Architecture, some accounts are considered sensitive and organizations should consider tightly controlling access to them, even more so than "production" accounts. This document will focus on the most sensitive accounts, `root` and `identity`, but the same principles can be applied to any account, and organizations should consider treating the `audit` and `security` accounts as sensitive as well. - The `root` account is the first account created and is what AWS terms the "Organization Management Account". It is the only account that can create other accounts, and as such, any privileged user in this account can assume the `OrganizationAccountAccessRole` in any other account, and therefore has full administrative access to all accounts in the organization. - The `identity` account is the account where IAM roles are defined and where users initially log in. This is separated from the `root` account for many reasons, chief among them being that privileged users in this account cannot automatically assume the `OrganizationAccountAccessRole` in other accounts. Other than IAM Roles, very minimal resources are created in this account. #### What Are the Basic Access Controls in Place? The standard Cloud Posse Reference Architecture configuration creates IAM roles Cloud Posse identifies as "teams" in the `identity` account. These roles are assumed by users when they log in via Federated SSO using SAML. Starting with Components [version 1.238.0](https://github.com/cloudposse/terraform-aws-components/releases/tag/1.238.0) Users can also create corresponding AWS SSO (now called AWS IAM Identity Center) Permission Sets that can be assumed by users by logging in via AWS SSO. The "teams" correspond to traditional access control "groups" (although they are implemented as IAM Roles for technical reasons), and are usually configured to provide access to most if not all of the accounts in the organization via what Cloud Posse calls "team roles". The "team roles" are a consistent set of IAM Roles in each account: they have the same permissions in every account and by default allow the same teams to access them in every account. #### What is SuperAdmin? The `SuperAdmin` user is a special IAM User in the `root` account that has full AdministratorAccess permissions in `root`, and can assume the `OrganizationAccountAccessRole` in any other account, giving it full AdministratorAccess to all accounts. It is intended to be used in situations where it is not possible to use teams or team roles (such as initially creating the teams). It should not be used for day-to-day work, and should be tightly controlled. #### Who Should Have Access to the Sensitive Accounts and How? Some organizations choose to use the `SuperAdmin` user as the only way to make changes in the sensitive accounts, while others choose to enable certain teams to also make changes to the sensitive accounts. Considerations include the practicality of controlling access to the `SuperAdmin` user and any privileged team, and the ability to audit changes made by the `SuperAdmin` or privileged teams, especially tracking who the human user is that is using the `SuperAdmin` credentials or assuming the team roles. This document does not attempt to provide or recommend a single solution, but rather to describe the options available and how to implement them. ## Generic Details In the sections below, we will get into details specific to differing versions of the Cloud Posse Reference Architecture. In this section, we will describe the basic concepts and terminology that apply to all versions. #### SuperAdmin The `SuperAdmin` user is a special IAM User in the `root` account that has full AdministratorAccess permissions in `root`, and can assume the `OrganizationAccountAccessRole` in any other account, giving it full AdministratorAccess to all accounts. The `SuperAdmin` user does not have a password and cannot log into the AWS Web console, but it does have MFA enabled. It is created and managed manually by the root user in the root account and is only used via its Access Key. Its access key can be disabled or deleted as desired to prevent its use. For technical reasons, only `SuperAdmin` can deploy the following components: - `tfstate-backend` to create the S3 bucket and DynamoDB table for Terraform state storage and IAM roles that can be assumed to access the state - `account` to create new accounts in the organization - `account-map` to collect and publish critical information about the accounts, used by other components. Although in some instances a team role can be used to update the following components, there is a risk that during the update process, the team or team role will have its permissions removed (before different permissions are later added), causing the update to fail and leaving everyone locked out of the accounts. So it is recommended that `SuperAdmin` be the only user to update these components: - `aws-sso` - `aws-saml` - `aws-teams` - `aws-team-roles` #### The "ops" and "empowered" teams The team and team role names have changed over time, and are fully configurable and customizable anyway, so it is impossible to consistently refer to them by their actual names. In this document we will use the names "ops" and "empowered" to refer to the teams in a generic way, with the understanding that the actual names will be different in different installations. - The teams and team roles are implemented as IAM Roles. - The teams' IAM Roles have limited permissions. They are not intended to be used directly, but rather to provide access to the team roles. - When using Terraform, in most cases Terraform automatically assumes a team role in the account where changes are being made. A user's ability to make changes in an account is determined by whether the team they are logged into in the `identity` account can assume the required team role in the account where the changes are being made. The "ops" team can assume the Terraform role in all accounts except for `identity` and `root` (and possibly other sensitive accounts). This allows that team to administer all other accounts via Terraform and is suitable for day-to-day work. The "empowered" team can assume the Terraform role in all accounts, including `identity` and `root`. This allows that team to administer all accounts via Terraform. This team is intended to be used only for special circumstances. To manage the `identity` and `root` accounts, the most common options are: 1. Do not create an "empowered" team and only use the `SuperAdmin` user to make changes in sensitive accounts 2. Create a separate "empowered" team and limit access to it 3. Empower the "ops" team, so that there is only one team to manage all accounts # TODO: ## Weedy Technical Details of Terraform Configuration - Access role in target account - Access read/write role in root account to access TF state backend - Access read-only role in root account to access TF state backend - `providers.tf` and `account-map` selection of `role_arn` - `backend` and `remote-state` configuration of `role_arn` - The `privileged` input variable ## Configuration Variations - Name of "ops" team - Name of "empowered" team - Name of Access role in target account - Name and ownership of Access read/write role in root account to access TF state backend - Name and ownership of Access read-only role in root account to access TF state backend ## Probably Not Needed Among the "team roles" are the `admin` and `terraform` roles, which have full AdministratorAccess permissions. Access to these roles in the `root` and `identity` accounts should be tightly controlled. Unfortunately, because of the way the Cloud Posse Reference Architecture evolved, there is no single set of instructions or configuration changes to accomplish this. Furthermore, organizations have different priorities for ease of use versus security, so there are multiple options to consider regardless of the base configuration. --- ## Documentation import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are a collection of technical documents recording some of the more low-level aspects of our identity implementation. --- ## Dynamic Terraform Roles import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; This document describes the Dynamic Terraform Roles feature and its benefits, and, for users of the previous architecture, **how to migrate to it**. The [AWS Access Control Architecture](/layers/identity/docs/aws-access-control-architecture) documentation assumes this feature is enabled and describes how to use it. The [AWS Access Control Evolution](/layers/identity/docs/aws-access-control-evolution#dynamic-terraform-roles-normalization-of-team-roles-better-team-and-team-role-iam-policy-management) documentation describes the feature in more detail. :::info More than just Terraform The Dynamic Terraform Roles feature is not just about Terraform. It is part of a larger effort to normalize the roles and permissions in all accounts. As such, it includes not just updates to various components, but it includes a recommended restructuring of "teams" and "team roles". This restructuring makes a clear distinction (via naming conventions) between the IAM roles that represent teams and the IAM roles that team members can assume. The migration portion of this document includes instructions on how to migrate to the new " team" and "team role" names and configuration. ::: This document alone is a lot of information. Combined with the above links and specific migration instructions in the affected components, it could take days to fully read, understand, and implement. Please prepare yourself accordingly. ### Background Cloud Posse's current reference architecture includes a feature we call "Dynamic Terraform Roles". This is in contrast to our previous implementation, where we used a single, static role per account for all Terraform operations. In the old implementation, access to Terraform was all-or-nothing (per account) and based on whether or not the user was allowed to assume the static role. This approach had some difficulties, including problems supporting AWS Identity Center Permission Sets (which are implemented by dynamic roles), and an inability to allow users to run `terraform plan` for drift detection or to validate that their changes at least did not introduce any syntax errors, without also allowing them to run `terraform apply`. It is also did not allow for a "least privilege" approach to access control in the target accounts, because the Terraform role requires full AdminAccess to the account, and there was no way to give users more restricted access and still allow them to use Terraform. The feature (or, more accurately, set of features) we call "Dynamic Terraform Roles" addresses these issues. ## Features and Benefits In this section we describe the features and benefits of the Dynamic Terraform Roles feature at a high level. For a more detailed description, see [AWS Access Control Evolution -> Dynamic Terraform Roles...](/layers/identity/docs/aws-access-control-evolution#dynamic-terraform-roles-normalization-of-team-roles-better-team-and-team-role-iam-policy-management). Freed from the requirement that all users use the same, single, static IAM role for all Terraform operations, we can now implement a more fine-grained access control policy. - **Full access**: This was the only option under the previous architecture. With full access, users are allowed to assume the Terraform role in some or all accounts, and that role can be used for all Terraform operations. As before, users can be limited to only certain accounts, but before, when limited, they would have no Terraform access at all to the accounts to which they were not granted full access. Now there are other options for giving users something less than full access but more than no access. - **Plan-only access**: Users can be granted access to a role that allows only `terraform plan` operations, but does not allow any modifications to the infrastructure. This is particularly useful for automated users like Spacelift or GitHub actions to perform drift detection on critical security settings without giving them the necessary authority to make changes (which would make them a very attractive target for malicious users). This is also useful, for example, for developers who want to see what changes their pull requests would make to infrastructure in production. Typically, DevOps and CI users would be granted full access in most accounts and plan-only access in the most sensitive ones, while developers would get full access only in development and sandbox accounts. - **Individual Per-Account Terraform Users**: Users can be granted access to custom roles in each account, allowing them to modify some infrastructure while not being able to modify other infrastructure, using ordinary IAM policies. For example, developers could be allowed to deploy applications to EKS without being allowed to modify the EKS cluster itself. As of this writing, such individual roles need to be explicitly granted access to the Terraform state via the configuration in the `tfstate-backend` component, but we could add attribute-based access control (ABAC) allowing access via tags on roles and thus eliminate the current requirement to the `tfstate-backend` component to grant access to new roles every time there is a change. Contact [Cloud Posse Professional Services](https://cloudposse.com/professional-services/) if you are interested in this feature. ### Direct Access versus Assume Role In the previous architecture, users would be required to assume a role in the `identity` account, and then Terraform would automatically assume the role in the target account. This meant for every possible access pattern, there needed to be a corresponding role in the `identity` account and users needed to be granted access to that role, which would then be assumed in the target account. This was a complex and unwieldy configuration, and it had an additional downside: For users who were focused on working in a single account, they would have to switch back and forth, using the `identity` account role to perform Terraform operations in the target account, and then switching back to their original role to perform AWS CLI or `kubectl` operations in the target account. Now it is possible to grant users a single role in the target account, and enable that role to be used for all Terraform operations. This simplifies both role assignments and user workflows. ### Reduced Need for SuperAdmin and Other Special Configurations In both the current and the previous architecture, a special "SuperAdmin" role is created in the `root` (organization owner) account with full Admin privileges. This role is needed to create the infrastructure that supports creating the IAM roles that are used for ordinary operations. For example, SuperAdmin is required to create the accounts and initial IAM roles themselves. In the previous architecture, anything that ever needed to be done by the SuperAdmin role would _always_ need to be done by the SuperAdmin role. This caused a lot of friction, because the SuperAdmin role was accessed by static API keys which had to be protected, manually rotated, and manually activated, and there was always a greater risk of these keys being leaked, since they were long-lived. In the previous and current implementations, there was and is a single static role that is used to access the Terraform state backend, and all Terraform users *except* SuperAdmin were required to assume this role to access the Terraform state, a prerequisite for running Terraform. In the previous architecture, SuperAdmin was not allowed to assume this role because the role did not exist in the early stages of bringing up a set of accounts. This limitation required special configurations, such as `var.privileged` and `backend.s3.role_arn: null` in the Terraform configuration. In the new architecture, SuperAdmin can assume the same static role as everyone else, so these special configurations are only necessary for a small number of components, and to the extent an organization desires, nearly all operations can be performed by the `root-admin` role. Enabling the `root-admin` role to do all the things that previously required the SuperAdmin role converts the SuperAdmin role into more of a backup "break glass in case of emergency" role, which allows for tighter access control, including disabling or deleting access keys after use. Typically, whereas in the previous architecture we only had the `identity-admin` role and greatly restricted what the `root-admin` role could do, in the new architecture we add an `identity-devops` role that matches roughly what the old `identity-admin` role could do, and does not have access to the `root-admin` or `root-terraform` roles. This would be the role that would be used for day-to-day operations by senior DevOps engineers. In this setup, the `identity-admin` is removed, to reduce confusion with the `*-admin` roles in other accounts. A new role, `identity-managers`, is added for those people who are trusted with full access to the `root` account. From that role users would have access to the `root-terraform` role to be able to run Terraform in the `root` account, and can assume the`root-admin` role, which could be used for most purposes that previously required the SuperAdmin role. ## Configuration and Usage To understand how access control works using Dynamic Terraform Roles, see [AWS Access Control Architecture](/layers/identity/docs/aws-access-control-architecture), especially [the section on Terraform Support](/layers/identity/docs/aws-access-control-architecture#terraform-support). The instructions below provide a thorough overview of the migration steps, but you should always refer to the latest documentation for each component to see if anything has changed since this was written. ## Migrating to Dynamic Terraform Roles, pt. 1, preparation This section describes how to start using Dynamic Terraform Roles if you are already using the previous implementation. Customers who are new to Cloud Posse, or who started using Cloud Posse's reference architecture after the release of version 1.242.0 of the `terraform-aws-components` module, should not need to do anything, it should already be enabled. ### Phased Migration The migration to Dynamic Terraform Roles can be done in a single step, but doing so would have the effect of locking everyone (except SuperAdmin) out of all accounts, and then you would have to reconfigure your IdP to allow people to log into the new roles. This is a significant disruption, so we recommend a phased approach: 1. Ensure prerequisites are met 2. Add the new roles to the `aws-teams` configuration 3. Migrate users to the new roles 4. Remove the old roles from the `aws-teams` configuration and deploy `aws-team-roles` to the `identity` account This phased approach is laid out in the following sections. If you want to do it all at once, you can just go through all the phases without doing any commits or apply operations until the end, at which point, as SuperAdmin, apply the components in the following order: 1. `tfstate-backend` 2. `account-map` 3. `aws-teams` 4. `aws-team-roles` ### Prerequisites :::important Prerequisites This migration guide assumes your current configuration already meets the following prerequisites. If it does not, you may be able to find instructions adequate to upgrade your configuration to meet the prerequisites in the [AWS Access Control Evolution](/layers/identity/docs/aws-access-control-evolution) documentation and/or the change logs for the individual components. If you cannot find the information you need there, please contact [Cloud Posse Professional Services](https://cloudposse.com/professional-services/) for assistance. ::: You must be using version 1.242.0 or later of these `terraform-aws-components` components: * `account-map` * `aws-team-roles` * `aws-teams` * `tfstate-backend` * `rootfs/usr/local/bin/aws-config` You must also have upgraded the `providers.tf` file in all your Terraform components to the v1.238.0 version or later. This means if the component is older than that version, you can either upgrade it to that version or later, or you can just cherry-pick the `providers.tf` file (and any other providers for those components that have multiple providers configured) from the v1.238.0 version. All of the above components are backwards compatible with the old configurations via `legacy_*` feature flags that are set to "legacy mode" by default. This means you can upgrade the components first and then migrate your configuration at your own pace. You must have access to the `SuperAdmin` user. Although after the migration, you may have little use for `SuperAdmin`, two areas where it should always be used are when updating `tfstate-backend` and when updating the `aws-team-roles` in the `root` account. During the migration, all Terraform operations should be performed by `SuperAdmin`. You must have already configured your Atmos stacks to use a `backend.s3.role_arn` whose name ends in `tfstate` (or at least is not `terraform`), created by the `tfstate-backend` component. :::warning The root account "terraform" role's usage has radically changed In older versions of the Cloud Posse reference architecture, the `root-terraform` role was used (assumed) by all users to access the Terraform state, and had extremely limited permissions. In the new architecture, the `root-terraform` role has full AdminAccess permissions and is used to perform all Terraform operations in the `root` account, and therefore access to it should be highly restricted. It is therefore critical to ensure the following configuration is already in place: - Your `tfstate-backend` component and Atmos stacks use a `backend.s3.role_arn` that does not end in `terraform`. - If a `terraform` role exists in the `root` account, only individuals authorized to use Terraform to modify the `root` account have access to it. Failing to meet these requirements could result in users - losing access to the Terraform state, or - obtaining full AdminAccess to the `root` account depending on the specific configuration. ::: ## Migrating to Dynamic Terraform Roles, pt. 2, Add the New Stuff ### Configure the new teams :::important Update Atmos stacks first Because of some automatic configuration done in the new architecture, you need to update all of your `aws-teams` and `aws-team-roles` stacks before applying any Terraform changes. ::: #### Refresher: What are "teams" and "team roles"? Cloud Posse, based on recommendations from Google and AWS, uses a hub-and-spoke architecture for access control. In this architecture, we create a hub account we call `identity`, and spoke accounts, which are all the other accounts in the organization. There are no AWS Resource deployed in or owned by the `identity` account other than IAM resources. Through various mechanisms, users and automated services are granted IAM Roles in the `identity` account. We call these roles "teams", and they are analogous to "groups" in conventional access control systems. We call the IAM Roles in the spoke accounts "team roles", and they grant access to the resources in those accounts. Teams are granted or denied access to accounts by configuring the team roles in those accounts to allow or deny the team to assume them. ```mermaid flowchart LR T[""developers" team"] -->|"Assume "admin" role allowed"| D[[" `dev` account         -------------- aws-team-roles:   admin:     trusted_teams:     - "devops"     - "developer" "]] T --x|"Assume "admin" role denied"| P[[" `prod` account         -------------- aws-team-roles:  admin:   trusted_teams:   - "devops"   # - "developer" omitted "]] classDef code text-align:left,font-family:Courier New, monospace class D,P code ``` #### Add the new teams to `aws-teams` :::caution This configuration enables "managers" to have full Admin access to the "root" account. The configuration described below enables "managers" to have full Admin access to the `root` account, largely replacing the SuperAdmin user, meaning that access to the role should be as highly restricted as was access to the SuperAdmin role. The configuration adds a "devops" team that has admin access to all accounts other than `root` and `identity`, to take the place of `identity-admin` in the previous configuration (which had limited access to `root` and `identity` via other restrictions). This is the configuration most popular with our customers, but obviously, other configurations are reasonable, trading off convenience for security. Adjust your configuration to suit your organization's needs. ::: :::tip AWS Identity Center (previously AWS SSO) support In the configuration below, we include `trusted_permission_sets` for organizations using AWS Identity Center (previously AWS SSO). If you are not using the Cloud Posse `aws-sso` component, just remove the `trusted_permission_sets` entries. In much older versions of the Cloud Posse architecture, using the component `iam-primary-roles` rather than `aws-teams`, the Permission Sets intended to correspond to what we now call "teams" had names that ended with `RoleAccess`. In the new architecture, these Permission Sets have names that end with `TeamAccess`. We recommend you use the new names, but if you find that such migration is too difficult, you can continue to use the old names. To use the earlier convention, create a `account-map/modules/roles-to-principles/variables_override.tf` file in which you set variable `overridable_team_permission_set_name_pattern` to default to `"Identity%sRoleAccess"` ::: As a first step, we add the new teams and roles to the stacks. In the `aws-teams` configuration, add the new teams: ```yaml managers: <<: *user-template role_description: "Team with AdministratorAccess to all accounts **including** `root`" role_policy_arns: - "arn:aws:iam::aws:policy/AdministratorAccess" aws_saml_login_enabled: true trusted_teams: [ "managers" ] trusted_permission_sets: [ "IdentityManagersTeamAccess" ] # The devops team is the normal team for DevOps personnel. # It has full access to all accounts except `iam` and `root`. devops: <<: *user-template role_description: "Team with PowerUserAccess permissions in `identity` and AdministratorAccess to all other accounts except `root`" # Limit `devops` to Power User to prevent accidentally destroying the `devops` role itself # Use SuperAdmin to administer IAM access role_policy_arns: - "arn:aws:iam::aws:policy/PowerUserAccess" aws_saml_login_enabled: true # list of roles in primary that can assume into this role in delegated accounts # primary admin can assume delegated admin trusted_teams: [ "managers", "devops" ] trusted_permission_sets: [ "IdentityManagersTeamAccess", "IdentityDevopsTeamAccess" ] ``` In the old architecture, "teams" and "team roles" had the same name, and were both referred to as roles. Access to a role in any account was automatically granted by having access to the role of the same name in the `identity` account. This was done to simplify configuration and implementation, before we had [Atmos](https://atmos.tools), but it was very confusing and inflexible, and violated our preference that the same role name should mean the same thing in every account. In the new architecture, teams have names distinct from team roles, (team names are plural, while team role names are singular), and team roles get deployed to the `identity` account via `aws-team-roles`, just like every other account. In the first phase of our multiphase migration, you leave the team roles in `aws-teams` and do not deploy `aws-team-roles` to `identity`. This preserves the ability to use the old roles for a transition period. However, if you can afford the disruption and can run as `SuperAdmin`, you can remove the team roles from `aws-teams` and deploy `aws-team-roles` to `identity` at the same times as you make all the other changes. :::info Remove the old roles from `aws-teams` At some point (see warning below) you want to remove these roles from the `aws-teams` configuration: - `admin` - `poweruser` - `observer` - `reader` - `support` - `billing` - `billing_admin` - `helm` - `terraform` Most of those roles will be restored, because in the new architecture, we deploy `aws-team-roles` to the `identity` account, too. For now, in the phased migration approach, you should leave them in the configuration. Especially, you should leave the `terraform` role in the `identity` account with full AdminAccess, and enable the new `managers` team (and whatever other teams you want to be able to use Terraform to make changes in the `identity` account) to assume it. ::: :::warning EVERYONE will lose access to ALL accounts if you remove these roles first If you remove these roles from the `aws-teams` configuration before deploying the new teams and giving people access to them, everyone but SuperAdmin will lose access to all accounts. ::: You may want to add one or more teams for users that need some AWS access in some accounts, but not full Admin Access everywhere. As an example, here is how to create a new team for developers, who will have less access than DevOps personnel: ```yaml developers: <<: *user-template role_description: "Team with view permissions in `identity` and limited access to all other accounts" role_policy_arns: - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" - team_role_access aws_saml_login_enabled: true trusted_teams: - managers - devops - developers trusted_permission_sets: - IdentityManagersTeamAccess - IdentityDevopsTeamAccess - IdentityDevelopersTeamAccess ``` The extra entries under `trusted_teams` and `trusted_permission_sets` allow people on the `managers` and `devops` teams to assume the `developers` role in `identity`, thereby allowing them to test out the permissions that developers will have in other accounts. :::info Restrictions in the `identity` account You may have been wondering about the `team_role_access` entry under `role_policy_arns` in the `developers` team configuration above. The `identity` account is an account that has no resources other than IAM roles and policies. It is used to manage access to other accounts. Therefore, there is not much that is needed to be done and few people need IAM permissions to do anything in this account. We recommend in general minimizing permissions in the `identity` account. However, the prime function of the "teams" (implemented as IAM Roles) in the `identity` account is to allow users to assume roles in other accounts. This requires they have the IAM permissions to assume roles. Therefore, the `aws-teams` component ships with a `team_role_access` policy that allows this. Formally, all teams should have this policy, but in practice, it is redundant for some users and can be omitted. ::: We recommend you add, at a minimum, these teams to the `aws-teams`: - `managers`: Full AdminAccess to all accounts. This is the role that mostly replaces the SuperAdmin role. You may want to give this role to just one or two people, or you may want to disable access to this role entirely, enabling access only when needed. In the `identity` account, this role has full `AdministratorAccess`. - `devops`: Terraform `planner` (`ReadOnlyAccess`) access in the `root` and the `identity` account and AdminAccess to all other accounts. This is the role that most DevOps engineers will use for day-to-day operations. In the `identity` account, this role has `PowerUserAccess`, which is access to almost anything other than IAM. You may want to add these teams: - `developers`: (See above) Configured via `aws-team-roles` to have whatever access you want developers to have in whatever accounts. In `identity`, we recommend `ViewOnlyAccess` and `team_role_access`. - `helpdesk`: A team that has access to the `support` role in all accounts. This is useful for troubleshooting and for performing routine maintenance tasks. - `bookkepers`: A team that has access to the `billing` role in all accounts. This is useful for reading billing invoice and tracking payments. - `controllers`: A team that has access to the `billing_admin` role in all accounts. This is useful for managing billing, budgets, and creating reports. ### Configure the new roles: pt. 1: The defaults You should have default settings for `aws-team-roles` in your stacks. Review the default settings and make the following changes. #### Add the `planner` role In the `aws-team-roles` configuration, add a `planner` role: ```yaml # The planner role allows use of `terraform plan` but does not allow any changes to be made planner: # This assumes you have &user-template defined earlier in the YAML file <<: *user-template enabled: true role_description: "Role for running terraform plan" role_policy_arns: - "arn:aws:iam::aws:policy/ReadOnlyAccess" - "eks_viewer" # You may want to add other teams here, e.g. helpdesk # If you have a CI/CD team, you may want to add it here, too trusted_teams: [ "managers", "devops", "developers" ] trusted_permission_sets: [ "IdentityManagersTeamAccess", "IdentityDevopsTeamAccess", "IdentityDevelopersTeamAccess" ] ``` :::tip Naming roles `terraform` and `planner` This guide assumes you are using Cloud Posse's naming conventions for roles, specifically, `terraform` for Terraform-managed infrastructure modifications and `planner` for Terraform plan-only access. You can use different names if you want to, but you will need to create roles with some name that fulfill these purposes, and you will need to configure `account-map` (specifically, `terraform_role_name_map`) with the names you choose. ::: ##### Configure access to the `planner` role One major point of this whole migration is to enable users to run `terraform plan` without being able to make changes. Give those users access to the `planner` role via `trusted_teams`, `trusted_permission_sets`, or both. ##### Further limit the `planner` role (optional, not recommended) You can further restrict the `planner` role, for example by removing the `eks_viewer` policy, and replacing `ReadOnlyAccess` with a more restrictive policy such as `arn:aws:iam::aws:policy/job-function/ViewOnlyAccess` in certain accounts. However, this will probably cause `terraform plan` to fail for some components. This may be acceptable if you are allowing Spacelift or some other automated process access to the full `terraform` role in those accounts, but if you are restricting those, too, (as you should for the `root` and `identity` accounts), then you will lose the ability to perform drift detection on those components. You might want to make such changes in only a subset of accounts, such as `security` or `audit` accounts, or production accounts. Although this goes against the rule of having the same role mean the same thing in every account, it is a tradeoff the architecture designers made in order to have consistent automatic access to the appropriate role via Terraform. #### Remove outdated roles You might want to remove outdated roles such as `helm` and `cicd` if you have them in your configuration. You can add them back later if you need them. At the time of this writing, the standard set of roles from Cloud Posse is: - `admin`: Full access to all resources - `billing`: Read-only access to AWS billing information - `billing_admin`: Full access to AWS billing information - `support`: Access to AWS support resources - `poweruser`: Full access to all resources except IAM - `observer`: View-only access to non-sensitive resources - `reader`: Read-only access to all resources - `terraform`: Used by Terraform to make changes to the infrastructure via `apply` operations - `planner`: Used by Terraform when the user does not have access to the `terraform` role, with the intention that this role be allowed to perform plan operations without being able to make changes We also include a disabled role called `template` that is used to establish the default settings for roles to help keep things DRY. **Outdated** Cloud Posse roles include: - `helm`: Previously we used `helm` and `helmfile` for deploying Helm charts to Kubernetes. We now use Terraform exclusively (using the `helm` and `kubernetes` providers). If you are still using Helm, you may want to keep this role. - `cicd` and `spacelift`: These were the roles used by automated processes. We now recommend they assume a team in `identity`, and that team be allowed to assume the roles in other accounts as usual. You are more than welcome to add additional roles that suit your purposes, but if you find a role that is not in this list and not one you added, such as `helm` or `spacelift`, then you can delete it now. #### Add custom roles For all accounts, including `root` and `identity`, add your standard set of team roles and remove outdated roles. Remember, we strongly recommend that the roles in each account have exactly the same permissions, so that the same role name means the same thing everywhere. However, you can and probably should have different `trusted_teams` and `trusted_permission_sets` for the same role in different accounts. For example, in a "sandbox" account, you might be OK with giving developers access to the `admin` role, but in a production account, you might want to restrict them to the `planner` role or even the `observer` role. This difference in who can assume the role in different accounts is the chief benefit of the new architecture. By changing who has access to the `terraform` or `planner` role in each account, you can restrict who can do what via Terraform in each account. We recommend that only `managers` should have access to `terraform` and `admin` in `root` and `identity`. As you update the team roles, you should review and adjust who has access to them in each account. If you need a role in some account with different permissions, give it a different name. For example, maybe give Developers a `developer` role in some accounts where `admin` or `poweruser` is not appropriate, but only deploy that role in those accounts where you want developers to have that level of access, and in other accounts, create a `triage` role or use the `support` role so that developers can have some more restricted access. Do not create a `developer` role in every account and then change what IAM policies are attached to that role in each account, because we have seen in practice that this leads to confusion and mistakes. #### Enable access to the roles from the new teams In the `aws-team-roles` configuration, add the new teams to the `trusted_teams` list for all the roles, as appropriate. For example, add `managers` to all the roles, maybe add `devops` to all the roles (to be overridden in the `root` and `identity` accounts), and add `helpdesk` to the `support` role Later, you will remove the old roles from `trusted_teams`, but you want to wait until everyone is using the new teams before you do that. ### Configure the new roles: pt. 2: The exceptions You should have customized settings for `aws-team-roles` for the `root` and `identity` accounts. Update the settings for those accounts as follows. #### Fully normalize the `terraform` role, pt. 1 As discussed under [Prerequisites](#prerequisites) above, the `terraform` role in much older versions of the Cloud Posse reference architecture was used to access the Terraform state backend, and had extremely limited permissions. You should have already updated your configuration to use a `backend.s3.role_arn` that does not end in `terraform`, but it is possible you simply removed or disabled the `terraform` role from the `root` account and left the configuration using the `admin` role to perform Terraform operations in the `root` and `identity` accounts. We will prepare to undo this wart in the configuration by giving the `terraform` role in `root` and `identity` full AdminAccess, and restricting access to them to the `managers` team. **In the `root` account:** In the `aws-team-roles` stack configuration for the `root` account, ensure that the `terraform` role is enabled and has full AdminAccess. This is the default configuration for `aws-team-roles`, so it may have already been inherited by default and then simply disabled, in which case you can just enable it now. You will, however, want to restrict access to the `terraform` role to the `managers` team. This is done by setting the `trusted_teams` list to `["managers"]`. (In Atmos, maps are merged, but lists are replaced.) While you are there, you may want to modify the `planner` role, giving the `devops` team access to it, and perhaps including the CI/CD team(s). **In the `identity` account:** In the `identity` account, the `terraform` role should currently be controlled by the `aws-teams` configuration. You should have already added `managers` the `trusted_teams` list for the `terraform` role via `aws-teams`, but double-check now. Later, you will deploy the `aws-team-roles` stack to the `identity` account, and move control of the role from `aws-teams` to `aws-team-roles`. At that point you will want to ensure the same non-default access controls for `identity` that you established above for `root`. #### Migrate your current customizations You may have customized access to any of the `aws-team-roles` roles in certain accounts. We have already discussed `root` and `identity`, but you may have also customized the roles in production accounts, or security or audit or other sensitive accounts. You should review and update these customizations to ensure that the new teams (`managers`, `devops`, `developers`, etc.) have the appropriate access to the roles in those accounts. ### Apply phase 1 changes to `account-map` At this point, your `aws-teams` and `aws-team-roles` configurations should be updated to include the new teams and roles, and the new teams should be given access to the roles in the `aws-team-roles` configuration. Update your `account-map` configuration to toggle the features: ```yaml legacy_terraform_uses_admin: true terraform_dynamic_role_enabled: true ``` Optionally, have Atmos validate the syntax of your configuration: ```bash atmos validate stacks ``` Fix any issues, then apply the changes (you will need to be SuperAdmin to do this). Adjust the stack name as necessary to match your configuration. ```bash atmos terraform apply account-map -s core-gbl-root ``` ### Update `tfstate-backend` Your configuration for the `tfstate-backend` component should already be set up to manage the role that is used to access the Terraform state and specified in your backend.s3.role_arn configuration. Update that configuration to include the new teams, plus `root-admin` and SuperUser. ```yaml access_roles: default: &access-template write_enabled: true allowed_roles: # Everyone who uses Terraform, even for planning, needs read/write access. # Add the new teams to the list, e.g "managers", "devops", "developers" # that currently looks something like this: core-identity: ["admin", "cicd", "ops", "platform", "poweruser", "spacelift"] # Add a new entry to allow the root-admin role to access the Terraform state core-root: ["admin"] # Add a new entry to allow the SuperUser role to access the Terraform state # Replace existing `allowed_principal_arns` empty list with a list containing # the ARN of the SuperUser role, substituting your root account ID for allowed_principal_arns: [arn:aws:iam:::user/SuperAdmin] ``` If you have a read-only role, make the same additions, being careful not to remove any access. Apply that configuration, adjusting the stack name as necessary to match your configuration. Note that unlike the other components we are changing, whose stacks are in the global `gbl` "region", `tfstate-backend` is deployed to your primary region, so you will need to adjust that part of the stack name accordingly. Note that you will need to be SuperAdmin to do this. ```bash atmos terraform apply tfstate-backend -s core-usw2-root ``` ### Apply phase 1 changes to `aws-teams` and `aws-team-roles` Now you are ready to apply the changes to `aws-teams` and `aws-team-roles`. Deploy the `aws-teams` stack to the `identity` account: ```bash atmos terraform apply aws-teams -s core-gbl-identity ``` Deploy the `aws-team-roles` stack all the other accounts except `identity`: ```bash atmos terraform apply aws-team-roles -s -gbl- ``` ### Update `aws-config-teams` Check your Dockerfile to see if you are using the `aws-config-teams` file to configure the AWS SDK. Look for this line: ```Dockerfile ENV AWS_CONFIG_FILE=/etc/aws-config/aws-config-teams ``` If you are not using that exact configuration, consider starting to use it now (more below). It is managed by the `aws-config` script, which is installed in your Git repository in the `rootfs/usr/local/bin` directory. You can install or update it from [`terraform-aws-components`](https://github.com/cloudposse/terraform-aws-components/blob/main/rootfs/usr/local/bin/aws-config). You can then update the `aws-config-teams` file by running the `aws-config` script: ```bash aws-config teams > rootfs/etc/aws-config/aws-config-teams ``` Note that the script depends on files in the `components/terraform/account-map/account-info/` and `components/terraform/aws-team-roles/iam-role-info/` directories that are generated by `account-map` and `aws-team-roles` (plus `aws-teams`), respectively. Make sure all the files in those directories are checked into Git and updated. If you did not have the `aws-config` script installed before, you will probably need to add these files to your repository. If you are using something other than `aws-config-teams` to configure the AWS SDK, now would be a good time to start using it. Unlike earlier solutions, `aws-config` is automatically maintained by the Terraform components, so after making changes to the `aws-teams` and `aws-team-roles`, you just need to run `account-map` and then `aws-config` to update the AWS configuration. The `aws-config` script also can generate the configuration for the browser plugin [AWS Extend Switch Roles](https://github.com/tilfinltd/aws-extend-switch-roles), via the `switch-roles` subcommand. If you are using that plugin, update that configuration now, too: ```bash aws-config switch-roles > rootfs/etc/aws-config/aws-extend-switch-roles ``` You may also want to generate a configuration for Spacelift or other CICD. ```bash aws-config spacelift > rootfs/etc/aws-config/aws-config-spacelift ``` The `spacelift` subcommand generates a configuration appropriate for Spacelift, but it is likely to be a useful starting for other CICD systems, too. All you likely need to do is change the `role_arn` and `role_session_name` for the `` profile. Run `aws-config` with no arguments to list all the subcommands. ## Migrating to Dynamic Terraform Roles, pt. 3, switch over to the new stuff Somehow you are currently giving users access to roles in the `identity` account, perhaps via AWS Identity Center (previously AWS SSO) Permission Sets, perhaps via a SAML IdP. However you are doing that, you need to switch users over to using the new teams: `managers`, `devops`, `developers`, etc. We leave those details up to you. Once you have everyone switched over to using the new teams, you can move on to the next step. ## Migrating to Dynamic Terraform Roles, pt. 4, clean up ### Remove the old teams from `aws-teams` Remove all the old teams from the `aws-teams` configuration. :::caution This will temporarily disable ordinary Terraform operations in `identity` Once you remove the `terraform` role from the `aws-teams` configuration, ordinary Terraform usage in the `identity` account will be broken. However, the goal of the `identity` is to have no resources other than IAM roles and thus no ordinary Terraform operations should be necessary. The SuperUser will continue to be able to deploy `aws-teams` and `aws-team-roles`, and after making the changes, ordinary Terraform operations will be restored. ::: Remove the following teams from the `aws-teams` configuration: - `admin` - `billing` - `billing_admin` - `support` - `poweruser` - `observer` - `reader` - `terraform` - `helm` You may also want to remove teams like `ops`, `cicd`, and `spacelift`, but check first to make sure they are not still needed. Apply the changes to the `aws-teams` stack in the `identity` account: ```bash atmos terraform apply aws-teams -s core-gbl-identity ``` ### Remove the old access from `aws-team-roles` Go through the `aws-team-roles` configurations and remove the old teams (like `admin`) from the `trusted_teams` list for all the roles. Note that in the new configuration, no team roles have the same name as teams, so if you see a team role's own name in the `trusted_teams` list (previously standard configuration, now outdated), you should remove it. #### Fully normalize the `terraform` role, pt. 2 In earlier versions of the Cloud Posse reference architecture, the `terraform` role in the `identity` account was managed by `aws-teams`, and `aws-team-roles` was not deployed to the `identity` account. Now that you have cleaned out the old teams from `aws-teams`, you can and should deploy `aws-team-roles` to the `identity` account. It should be configured in the same way you have `aws-team-roles` configured for the `root` account. ### Deploy `aws-team-roles` to the all accounts Deploy the `aws-team-roles` stack to all the accounts, _including_ `identity`: ```bash atmos terraform apply aws-team-roles -s -gbl- ``` ### Apply final changes to `account-map` Update your `account-map` configuration to toggle the features: ```yaml legacy_terraform_uses_admin: false terraform_dynamic_role_enabled: true ``` Update `account-map` (Adjust the stack name as necessary to match your configuration.) ```bash atmos terraform apply account-map -s core-gbl-root ``` ### Normalize Terraform State Backend Access Search your stacks for `backend` and `remote_state_backend` configurations that have `role_arn: null` or `var.privileged: true`. You can remove these settings, and if those are the only `backend` or `remote_state_backend` settings being overridden, remove the entire configuration section. ### Update config files (optional) If you want to, you can use the `aws-config` script to update the various files generated earlier. This is optional, as all it will do is remove the old teams and roles no longer in use. --- ## FAQ(Identity) import Intro from '@site/src/components/Intro'; Frequently asked questions about identity and authentication with Cloud Posse's reference architecture. ## How do I create new teams? Follow the docs in the readme of the [`aws-teams` component](/components/library/aws/aws-teams/). ## How do I set up Leapp? See [How to Log into AWS](/layers/identity/how-to-log-into-aws/) ## Known Leapp Limitation with Windows and WSL Leapp sets AWS credentials to the home directory wherever it is running. So if you run Leapp on Windows, that will be set to the Windows home directory. Then when you start WSL2, that will have a different home directory. The workaround is to copy AWS credentials from the Windows home directory over to the WSL2 home directory. [ref](https://github.com/Noovolari/leapp/issues/153) ```bash cp -r /mnt/c/Users//.aws ~/.aws ``` ## How can I set varying permission for a role in several accounts? Keep in mind that Team Roles are designed to identical in all delegated accounts. However, you can still modify permission on an account-by-account basis if required. Override your [`aws-team-roles` component](/components/library/aws/aws-team-roles/) stack configuration to include different permissions when deployed to a given account. ## Why not use AWS Control Tower? AWS Control Tower until recently, did not support Terraform. Additionally, using Terraform, we have complete control over the Organization, Organizational Unit, and Account provisioning process. ## Why not just use AWS SSO? The primary reason to not rely solely on AWS SSO, is that cross-account terraform is not supported with AWS SSO Permission Sets. We cannot apply Terraform across the account hierarchy with a single Permission Set, so instead we combine AWS SSO with our AWS Teams and Team Roles. AWS SSO is fine for business users but less so for DevOps teams. --- ## How to Log into AWS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Note from '@site/src/components/Note'; We use Leapp to facilitate logging into AWS. Leapp is a tool that allows you to authenticate with your organization's Identity Provider (IdP) and then assume an IAM Role in AWS. This allows you to use your organization's SSO to authenticate with AWS. ## Requirements ### Install Leapp Install Leapp following [Leapp documentation](https://www.leapp.cloud/download/desktop-app) or with brew: ```console brew install --cask leapp ``` ### Install AWS Session Manager (If required) ```console brew install --cask session-manager-plugin ``` ### Launch Leapp ![Leapp Search](/assets/refarch/leapp-search.png) ## Setup The following steps are required only for initial setup. ### Launch Leapp ### Create new Integration ![Leapp Integration](/assets/refarch/leapp-integration.png) ### Fill out Single Sign-On configuration ```console Alias: acme # This can be whatever you would like to label the Integration in Leapp Portal URL: https://d-1111aa1a11.awsapps.com/start/ # Set this to your SSO Launch URL AWS Region: us-east-1 # Your primary region Auth. Method: In-browser # Optional ``` ### Click Integration “dots” and select “Login”. This should launch a tab in your web browser. ![Leapp Integration Dots](/assets/refarch/leapp-integration-dots.png) ![Leapp Integration Login](/assets/refarch/leapp-integration-login.png) ### Log into your IdP Log into your IdP for your Organization and “Allow” Authorization request ### Create a “Chained Session” from `core-identity` Create a “Chained Session” from the `core-identity` account with the `IdentityDevopsTeamAccess` Role This Permission Set will match the given Team name. For example, Developers will use `IdentityDevelopersTeamAccess` and DevOps will use `IdentityDevopsTeamAccess`. ![Leapp Chained Session](/assets/refarch/leapp-chained-session.png) ### Fill out the Chained Session configuration Fill out the Chained Session configuration for connecting to `core-identity` ```yaml Named profile: acme-identity # This must match the profile name given in AWS config Session Alias: acme-identity # Optional AWS Region: us-east-1 # This must be your primary region Role ARN: arn:aws:iam::666666666666:role/acme-core-gbl-identity-devops # This ARN depends on the given team. This example uses the "devops" team Role Session Name: acme-identity # Optional Assumer Session: core-identity # This must match the name of the identity account, almost always "core-identity" ``` ![Leapp Chained Session Configuration](/assets/refarch/leapp-chained-session-configuration.png) ### (Optional) Pin the new `acme-identity` IAM Role Chained Session This makes it easier to filter to the primary session we will be used for connecting to AWS - Go to All Sessions - Find the new IAM Role Chained Session for `acme-identity` or whatever value you used for `Session Alias` - Click the dots on the IAM Role Chained Session - Select Pin Session ### Connect to `acme-identity` IAM Role Chained Session - Select the Session - Click Start Session ![Leapp Start Session](/assets/refarch/leapp-start-session.png) ### Rebuild Geodesic Open your terminal of choice, navigate to the `infrastructure` repository, and launch Geodesic ```console make all ``` ### Use AWS in Geodesic You're done! You can now use AWS from with in Geodesic. ![Geodesic Check](/assets/refarch/geodesic-check.png) ## Usage After initial setup, quickly connect to AWS with the following steps: 1. Launch Leapp 2. Connect to `acme-identity` IAM Role Chained Session ![Leapp Start Session](/assets/refarch/leapp-start-session.png) 3. Open your terminal of choice, navigate to the `infrastructure` repository, and launch Geodesic ```console make run ``` 4. Done! ![Geodesic Check](/assets/refarch/geodesic-check.png) --- ## Identity and Authentication import ReactPlayer from "react-player"; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; There are several methods available to sign in to AWS through a browser or terminal, such as AWS SAML or IAM Identity Center (successor to AWS Single Sign-On). Various tooling has been tried to support AWS SAML, like "saml2aws," and others to support AWS SSO, such as "aws sso login" and "aws configure," to manage terminal access. However, these tools often had issues, such as not handling all methods of login with different IdPs and using screenscraping, which caused anti-bot measures to block them, for example, Google Workspaces. Cloud Posse's approach involves using IAM roles that provide highly precise control to different user groups. These groups are assigned to users through AWS SAML Apps or AWS SSO Groups ("Groups") and enable the user to assume roles into any allowed accounts. A significant advantage of this approach is the availability of an easy-to-use tool that works with all AWS sign-in methods. In the following pages, the Cloud Posse strategy will be explained in detail, demonstrating how it can be used to establish fine-grained access control for an entire organization.
AI generated voice
## Our Requirements Let’s start by identifying the minimum requirements for an identity and authentication system. ### Accessing AWS as a human or as a machine user. First we need to implement a system that is easy for both humans and machines to access securely, following the principle of least privilege ### Centralized management of user permissions Plus, all users and permissions must be centrally managed and integrated into an identity platform. We don’t want to copy and paste permissions across accounts or rely on any manual processes ### Tight control over user groups Next, we need fine grained access control for user groups. Then assign users to one or more groups depending on what they need access to. It needs to be easy to understand for both users and administrators. ### Apply Terraform for many accounts across an Organization With Terraform, we need to manage resources concurrently across multiple accounts. We don’t want to put this burden on the operator to constantly switch roles, so Terraform needs to do this for us. ### Switch roles into other accounts easily both in the UI and locally Finally for engineers, we want to quickly jump between accounts, access the “AWS” web console, or run “AWS” “CLI” commands without having to think how to do it every time. ## Problem We've Encountered Now you may be asking some questions. There are plenty of existing solutions out there for authentication and identity. How did Cloud Posse arrive at their solution? What’s wrong with the alternatives? ### AWS Control Tower lacked APIs First off, you might notice we don’t use AWS Control Tower. That’s because until recently, Control Tower didn’t have an API available. So we couldn’t programmatically manage it with Terraform. Of course Cloud Posse does everything with infrastructure as code, so that was a hard stop for us. We’re planning to add support for Control Tower now that it’s available, once it matures. ### AWS IAM Identity Center is only for humans The ideal way to access AWS for humans is with Identity Center, formally called “AWS SSO”, and this is included in our reference architecture. But that doesn’t solve how we provide access to machines. For example, integrating with GitHub Actions or Spacelift. With Identity Center, a user assumes a single role for one account. That’s not going to work for us with Terraform if we’re trying to apply Terraform concurrently across accounts. For example, transit gateway architecture requires provisioning resources and connecting them across accounts. Identity Center is also limited because it only works with a single IdP. For larger enterprises, multiple IdPs might be used. In addition, for the duration of an engagement, Cloud Posse configures our own IdP to access your infrastructure so that it’s easy for you to revoke our access when we’re done. ### AWS IAM Roles with SAML is cumbersome We needed to find a solution for machine access and to apply Terraform across accounts. To do that, we can use IAM roles with a SAML provider and assume them in Terraform or third party integrations such as GitHub Actions. The challenge then becomes making it easy for users. AWS SAML provides low level controls and is a little more cumbersome to use, especially when you compare how easy it is for users to use the IAM Identity Center. How can we have a consistent solution that works for both? Ultimately, AWS does not provide a single solution that meets all our requirements. We need to combine the best of both worlds. Identity Center is great for human access, but it doesn’t work well for machines. On the other hand, AWS SAML is great for machines but is cumbersome for users to navigate the AWS web console without a third party tool. ## Our Solution ### Integrated with Single Sign On ```mermaid flowchart LR user["User"] --> idp["Identity Provider"] idp --> identity_center["AWS IAM\nIdentity Center"] identity_center --> aws_team["AWS Team"] aws_team --> aws_team_role["AWS\nTeam Role"] ``` We use Identity Center to easily manage users and groups. These are connected with your Identity provider, such as Okta or GSuite. Users can sign into Identity Center to access any account they have access to with fine grained access controls. Administrators have a single place to manage users and can use Terraform to define those fine grained access controls. Then we’ve bridged the gap between Identity Center and IAM roles ### Team Oriented ```mermaid flowchart LR subgraph identity_teams["Identity Teams"] devops["devops"] end subgraph dev_team_roles["Dev Team Roles"] terraform_dev["terraform"] end subgraph staging_team_roles["Staging Team Roles"] terraform_staging["terraform"] end subgraph prod_team_roles["Prod Team Roles"] terraform_prod["terraform"] end devops --> terraform_dev devops --> terraform_staging devops --> terraform_prod ``` To simplify managing IAM access roles, we’ve created two concepts. AWS Teams and AWS Team Roles. All access in AWS is “Team” oriented. A “Team” is a group of users that we deploy into a central “identity” account. Then we deploy a number of “Team Roles” into every single account. A “Team” can assume any “Team Role” across the organization. We’ll explain in detail in the following pages. ### Highly Accessible Then to make it incredibly easy for users to log in, we recommend using Leapp to manage local AWS sessions. Leap supports all Identity Providers and can automatically refresh credentials when they expire. Plus, it’s open source and free to use. ### Pre-Configured for your Team Finally, to get everyone up and running on your Team, we preconfigured a single, common AWS config for all users. All users connect to AWS using the same AWS profile name. Then we can use the same AWS config to assume any other role from that given AWS profile if their team has permission. This way, users can quickly apply Terraform across accounts and assume roles directly in the accounts they have access. :::tip Using Dynamic Terraform Roles With dynamic Terraform roles, we can choose to deploy a `terraform` and a `planner` role into any given account. The `terraform` role is used to apply changes, and the `planner` role is used to plan changes. For more on dynamic Terraform roles, see the [Dynamic Terraform Roles](/layers/identity/docs/dynamic-terraform-roles/) documentation. ::: ## Next Steps Next, we'll explain how we've implemented this solution in detail. If you're curious about the thought that went into this process, please review the design decisions documentation. Next Step Review Design Decisions --- ## Using AWS SAML to Access AWS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; The AWS SAML component allows you to authenticate with AWS via a federated identity. This is an alternative to using AWS SSO, that provides lower-level control over the authentication process and supports multiple concurrent IdPs, but with more complexity and a reduced user experience. As an alternative to AWS SSO, the AWS SAML creates an Identity Provider (IdP) to authenticate with AWS via a federated identity. You can use this federated identity to connect directly to a given AWS Team. # Export an IdP metadata file from the chosen provider. ## Export an IdP metadata file from the chosen provider. The creation of metadata files will be different for each IdP. Here are some example setup references: 1. Open the [AWS documentation for GSuite](https://aws.amazon.com/blogs/desktop-and-application-streaming/setting-up-g-suite-saml-2-0-federation-with-amazon-appstream-2-0/) 1. Follow Steps 1 through 7. This document refers to Appstream, but the process will be the same for AWS. 1. Once you have completed the setup, download the metadata file. 1. Create an "Amazon Web Services Account Federation" application in Okta. 1. Select "SAML 2.0" from the Sign-On Method. 1. View and download the identity provider (IdP) metadata file. For details, please see the official [Okta documentation](https://help.okta.com/en-us/Content/Topics/DeploymentGuides/AWS/aws-configure-identity-provider.htm) Follow the [JumpCloud documentation](https://support.jumpcloud.com/support/s/article/getting-started-applications-saml-sso2). Once you have completed the setup, download the metadata file. The setup for Microsoft Entra ID (formerly AzureAD) has a few issues that we've encountered. Please follow our [documentation on Microsoft Entra ID](/layers/identity/tutorials/how-to-setup-saml-login-to-aws-from-office-365/) to get the metadata file. ## Import the metadata file from the chosen provider. Download and save the metadata file with the `aws-saml` component directory. 1. Place this file inside the `aws-saml` component directory (`components/terraform/aws-saml/`) 1. The filename should match the variable configured in the `aws-saml` stack catalog (`stacks/catalog/aws-saml.yaml`). 1. Commit this to version control. Make sure the `var.saml_providers` map key ends with `-okta`. We filter by this suffix to determine whether or not to set up a dedicated user for Okta. This is only necessary for Okta. ```yaml saml_providers: acme-okta: "OktaIDPMetadata-acme.com.xml" ``` ## Deploy the SAML Integration Deploy the `aws-saml` component to your Identity account. ```bash atmos terraform apply aws-saml -s core-gbl-identity ``` ## Complete the Identity Provider (IdP) setup If necessary, complete the integration setup in your chosen IdP. This will vary depending on the provider. Follow the steps in the [official Okta documentation](https://help.okta.com/en-us/content/topics/deploymentguides/aws/aws-configure-aws-app.htm) to complete the setup. Please review the following tips, as we've encountered these issues in the past: - Deploying the `aws-saml` component will create an AWS IAM User, which Okta will be used to discover roles in AWS. This user's access key and secret key are stored in AWS SSM Parameter Store in the same account and (default) region as the `aws-saml` component. This is unique for Okta. - In the "Provisioning" tab for the integration in Okta, you must check the **"Update User Attributes"** box. This does not appear in documentation but is necessary for the roles to populate in Okta. ## (Optional) Use AWS Extend Switch Roles Browser Extension We suggest using the AWS Extend Switch Roles browser extension to simplify role-switching in the AWS Console. This is optional but particularly helpful if you’re not using AWS IAM Identity Center. Please see the [AWS Extend Switch Roles plugin](https://github.com/tilfinltd/aws-extend-switch-roles). Once you've downloaded the plugin, take the aws-config file from within the `rootfs/etc/aws-config` directory in your infrastructure repository. Paste this into the AWS Extend Switch Roles plugin configuration. --- ## Optional import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some of the optional components that might help you along with the associated identity layer for more complex use cases. --- ## Create Custom IAM roles import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem Customers would like to expand or customize IAM roles ## Solution We distinguish between two types of IAM roles: "teams" and "team-roles". Teams are just like "groups" in IAM and AWS SSO, but because they are deployed and managed differently (and are implemented as IAM roles), we give them a distinct name. They are used to define a set of permissions which can then be assigned to multiple people. Teams only exist in the "identity" account.
Team
A "team" is a group of people (or one or more services) who have the same pattern of permissions. If you want to add a new team, you add it via `aws-teams`. The permissions defined in `aws-teams` only apply to the team in the identity account, and usually "team_role_access" is all that is needed there.
Team roles
"Team roles" are IAM roles which are assumed by people and services to do work. They are deployed to every account with the same IAM Policies attached. What differs from account to account is the trust policy, which defines who can assume the role. Thus, the trust policies across all the accounts are what define the scope of a team.
SSO Permission sets
SSO "Permission sets" are a special kind of IAM role which are assumed by people to do work. Like team roles, they have their permissions defined once and are then deployed to every account. SSO "groups" function just like "teams": they define a collection of permission sets in each account which the group member can access.
## Considerations - If you want to give a new team access to an existing team role, you just modify the `trusted_teams` list for the role accordingly. - If you want to create a new role with a new set of permissions, you add it via `aws-team-roles` as explained under [SAML](#saml). - If you want to create a new SSO permission set, you add it via `aws-sso` as explained under [AWS SSO](#aws-sso). ### SAML Each role is granted permissions by attaching a list of IAM policies to the IAM role via its `role_policy_arns` list. You can configure AWS managed policies by entering the ARNs of the policies directly into the list, or you can create a custom policy as follows: 1. Give the policy a name, e.g. `eks-admin`. We will use `NAME` as a placeholder for the name in the instructions below. 2. Create a file in the `aws-team-roles` directory with the name `policy-NAME.tf`. 3. In that file, create a policy as follows: ```hcl data "aws_iam_policy_document" "NAME" { # Define the policy here } resource "aws_iam_policy" "NAME" { name = format("%s-NAME", module.this.id) policy = data.aws_iam_policy_document.NAME.json tags = module.this.tags } ``` 4. Create a file named `additional-policy-map_override.tf` in the `aws-team-roles` directory (if it does not already exist). This is a [terraform override file](https://developer.hashicorp.com/terraform/language/files/override), meaning its contents will be merged with the main terraform file, and any locals defined in it will override locals defined in other files. Having your code in this separate override file makes it possible for the component to provide a placeholder local variable so that it works without customization, while allowing you to customize the component and still update it without losing your customizations. 5. In that file, redefine the local variable `overridable_additional_custom_policy_map` map as follows: ```hcl locals { overridable_additional_custom_policy_map = { NAME = aws_iam_policy.NAME.arn } } ``` If you have multiple custom policies, add each one to the map in the form `NAME = aws_iam_policy.NAME.arn`. 6. With that done, you can now attach that policy by adding the name to the `role_policy_arns` list. For example: ```yaml role_policy_arns: - "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" - "NAME" ``` 7. Redeploy the component to all the accounts. Changing the permissions of a team in the identity account is virtually the same, just done in the `aws-teams` component and configuration. The configuration of team access is inverted: each team role in each account configures who can access it, via the `trusted_teams` list. So if you want to give a new team access to an existing team role, or an existing team access to a new team role, you control that via the role's `trusted_teams` list in each account. ### AWS-SSO Creating a new AWS SSO permission set is similar to creating a new SAML team role (above), but the configuration is done in the `aws-sso` component. See the `aws-sso` component's documentation for details. ### Commands ``` atmos terraform deploy aws-sso --stack gbl-root atmos terraform deploy aws-teams --stack gbl-identity atmos terraform deploy aws-team-roles --stack gbl- ``` --- ## How to create a Delegated role for assumption by one or more Primary roles import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Status **OUT OF DATE** ## Problem You want to configure a new set of fine-grained IAM roles (e.g. for an intern) using [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-primary-roles) and [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) and not sure how to go about it. ## Solution ### Plan the privileges of the new role A Delegated role grants a collection of privileges. Users associated with a Primary role allowed to assume the Delegated role can access these privileges. ### Implement the IAM role Not to be overlooked is the importance of choosing a good name. It should be informative, and for the sake of automation, consist solely of letters (no symbols). For this example, we will create the role `restricted`. Because we want `restricted` to be able to do more than an `observer` but less than a `poweruser`, we will need to create a new Delegated role. If we are creating this permission set for use with a Primary role that is not yet defined, we will need to create that too, but for the sake of this example, we will assume we have already created a Primary IAM role, `intern`, that will be granted permission to assume this new Delegated role. ``` delegated_roles_config: admin: ... restricted: # role_policy_arns is the list of IAM Policies to attach to this # role. You can give the full ARN of an existing policy, or you # can give the name of a custom policy you define elsewhere # in the project. Here, we use a custom policy name we have already # defined, to give the restricted role permission to assume roles # in other accounts. role_policy_arns: ["delegated_assume_role"] role_description: "Restricted permissions for AWS and kubernetes" # We would set this to True were this role to also be used as a Primary role, # but nobody should be logging in directly as `restricted` so here it is false. sso_login_enabled: false # trusted_primary_roles is the list of other roles that can assume this one. trusted_primary_roles: ["admin", "ops", "poweruser", "intern"] ``` If we wanted to create a new policy for the new role, we would follow the example of `delegated_assume_role`. Create a new file with the role name in `components/terraform/iam-primary-roles/`, create the policy in the file, and add the policy to the `custom_policy_map` in `components/terraform/iam-primary-roles/main.tf`, but we do not need to allow our interns anything special in the `identity` account. #### Configure or disable the Delegated role in the other accounts By default, delegated roles are created in every account. This may or may not be desired behavior. If there are any accounts in which your role is irrelevant or should not be used to grant privileges, disable it by adding it to the `exclude_roles` list in `stacks/gbl-$stage.yaml`. ##### Configure IAM policy in other accounts By default, Delegated roles will get the same-named policies in other accounts as they got in `identity`. In practice, this is rarely correct, but if it is correct in this case, then you need not take any of the following steps. Otherwise, you need to collect the policies to apply to the role and list them in the `role_policy_arns` map (which we will get to after collecting the entries). ###### Identify one or more AWS managed IAM policies to apply If you want, you can use [AWS managed policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_managed-vs-inline.html#aws-managed-policies) (although there is a [case to be made against it](https://summitroute.com/blog/2018/07/02/aws_managed_policies_are_an_anti_pattern/), which includes that they are not easily found or comprehensively documented). Their ARNs can be used directly in the `role_policy_arns` map. ###### Create a custom policy for the role Just as with the `iam-primary-roles` project, we can create a custom policy for the new role. We will create a restricted policy that [lets them modify resources that are tagged](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_tags.html#access_tags_control-resources) `restrictedModify: allowed`. We will combine this with the AWS managed `ViewOnly` policy so they can look at things. (The AWS `ReadOnly` policy is more permissive and would let them read secrets, so we do not want to use that.) 1. #### Create a new file with the policy name under `iam-delegated-roles` called `restricted-policy`. 2. #### Define the policy, something like (this has not been vetted/tested) ```hcl data "aws_iam_policy_document" "restricted" { statement { sid = "PowerWhenTaggedForRestricted" effect = "Allow" actions = [ "cloudwatch:*", "ec2:*", "ecr:*", "eks:*", # Lambda does not support tag based access "lambda:*", "rds:*", "s3:*", ] resources = ["*"] condition { test = "StringEquals" values = ["allowed"] variable = "aws:ResourceTag/restrictedModify" } } } ``` 3. #### Create the policy ```hcl resource "aws_iam_policy" "restricted" { name = format("%s-restricted", module.label.id) description = "Allow users to modify specially tagged resources" policy = data.aws_iam_policy_document.restricted.json } ``` 4. #### Add the policy to the `custom_policy_map` in `main.tf` ```hcl custom_policy_map = { "restricted" = aws_iam_policy.restricted.arn } ``` 5. #### Override the default policy assignment for the new role In `iam-delegated-roles/default.auto.tfvars` the `role_policy_arns` map is just that: a map whose keys are role names and whose values are lists of policy ARNs (or names in the `custom_policy_map`) to assign to that role, overriding the assignments made in `iam-primary-roles`. Add an entry for the new role. ```hcl role_policy_arns = { # Put the ARN first, it will make the Terraform output cleaner restricted = ["arn:aws:iam::aws:policy/job-function/ViewOnlyAccess", "restricted"] ... } ``` ##### Configure custom access to Delegated role We need to allow one or more Primary IAM roles to assume this Delegated IAM role in one or more stages of our infrastructure for it to have any effect. As noted above, we will assume here that we have created the `intern` Primary role. Further, let's assume we wish to allow our new intern access to the `restricted` role's permissions only in our `sbx` and `dev` stages. We can accomplish this by adding a line for our new role to `trusted_primary_role_overrides` for `sbx` and `dev` as shown below. In `stacks/gbl-sbx.yaml` add ```yaml projects: terraform: iam-delegated-roles: vars: trusted_primary_role_overrides: restricted: ["dev", "intern"] ``` In `stacks/gbl-dev.yaml` add ```yaml projects: terraform: iam-delegated-roles: vars: trusted_primary_role_overrides: restricted: ["intern"] ``` ##### Extra Credit: disable roles entirely in `root` and `audit` Because we have assigned this role the `delegated_assume_role` policy, it cannot get into `root` or `audit`, but to be extra careful, we can prevent this role from being created in those accounts in the first place. In `stacks/gbl-root.yaml` and in `stacks/gbl-audit.yaml` add "restricted" to the list of additionally excluded roles. ##### Run Terraform to actually provision the accounts and policies These steps must be run as admin in the `root` account. We must run plan and apply once for each account that uses delegated roles. Perform as below for each stage, replacing $stage with the literal stage name. ```console assume-role eg-gbl-root-admin bash -l # Perform the following for each stage atmos terraform plan iam-delegated-roles --stack=gbl-$stage atmos terraform apply iam-delegated-roles --stack=gbl-$stage # leave the root admin subshell exit ``` ### Implement the Kubernetes role EKS maps IAM _roles_ to Kubernetes _groups_. Selecting appropriate permissions for the `restricted` group in Kubernetes is beyond the scope of this article. We will go through the steps without specifics. #### Assign Kubernetes usernames and groups to the Delegated roles in each account The admin, ops, poweruser, and observer primary IAM roles and the same delegated roles plus the terraform and helm roles required for our automation to function are configured in `stacks/eks/default.auto.tfvars.` Additional roles can be configured in the same file for universal permissions, or for each account in the relevant account config file. In this example, we need to add a new Kubernetes user and group and assign the new Delegated role to it. We will use the `sbx` account and the `uw2` environment, and add the following to `stacks/uw2-sbx.yaml`. Edit `stacks/uw2-dev.yaml` and add an entry for the `restricted` role to the `projects:terraform:eks:vars:delegated_iam_roles` list: ```yaml projects: ... terraform: ... eks: vars: delegated_iam_roles: role: "restricted" groups: ["idp:restricted"] ``` Do this for every account. Be sure to use the correct stage name and account number in the role ARN and username. We use the `idp:` prefix on the Kubernetes groups we create in order to avoid conflict with any pre-defined groups or roles. "IDP" is for Identity Provider and was chosen because the Kubernetes group membership is assigned based on an identity provided by an outside identity provider. If you want to create a mapping from the Primary role in `identity` to a Kubernetes role, you can do that by adding an entry in the `primary_additional_iam_roles` map in the `defaults.auto.tfvars` file, but this is not recommended. That feature is provided as a shortcut for operations personnel, is not required for anything, and should not be extended to other roles because it can open up unintended privileges. Note that we assigned the role to the group "idp:restricted". That group does not exist yet; we must create it. However, you can run the Terraform to apply the changes now if you prefer, or you can wait until after the groups are created. #### Create Kubernetes roles and bindings in each account Kubernetes roles and role mappings are defined in `/projects/helmfiles/idp-roles/` The `helmfile.yaml` uses the Kubernetes "raw chart" to directly specify Kubernetes resources. You can follow the examples in the existing `helmfile.yaml` for creating ClusterRoles and ClusterRoleBindings, and create new ones as needed. At a minimum, you will need to create a new ClusterRoleBinding to bind the Group "idp:restricted" to a ClusterRole. The examples bind a User as well, but that has no practical effect. #### Run Terraform and Helm in each account Be sure to run both the Terraform under `eks` and the Helm under `helmfiles/idp-roles` for each account with an EKS cluster. These can be run the normal way, as `eg-gbl-identity-ops`. ### Update the company AWS `config` file with the new Delegated role If you created a new Delegated role, then people who want to use that role need to add it to their AWS `config` files, normally stored as `~/.aws/config` on individual's computers. We keep the shared configuration in the repo as `/aws-config`. To update the file, - Add "restricted" to the list of delegated_roles in `/rootfs/usr/local/bin/aws-config-gen` - Run the modified `aws-config-gen` and save the output in `/aws-config` Let people know to update their configuration files. --- ## How to Easily Switch IAM Roles Between Accounts import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; Learn how to switch between multiple AWS account roles using the `aws-extend-switch-roles` browser extension. Just follow the step-by-step instructions for installing the extension, configuring it with your AWS accounts, and using it to switch roles seamlessly. ## Problem Your company uses a dozen or more AWS accounts with multiple roles. The AWS Web Console only remembers the last 5 roles you used. This gets old really quick and you want a better UX switching roles into AWS accounts. ## Solution :::tip The [aws-extend-switch-roles](https://github.com/tilfinltd/aws-extend-switch-roles) browser extension can be used to switch roles across many accounts easily from the browser. ::: 1. Refer to the [aws-extend-switch-roles documentation](https://github.com/tilfinltd/aws-extend-switch-roles#install) to install the extension on the browser of your choice. 2. Follow the [AWS CLI Access](https://github.com/BrightDotAi/infrastructure/blob/main/docs/authentication/onboarding.md#aws-cli-access) procedure to set up your local workstation and access the Geodesic shell. 3. Enter the Geodesic Shell - If the option is available, run the `aws-accounts gen-switch-roles` command to save the `aws-extend-switch-roles` configuration to your home directory. ```shell ⨠ aws-accounts gen-switch-roles > /localhost/aws-extend-profiles ``` - If not, use this workaround ```shell ⨠ aws-accounts gen-saml | \ grep -v source_profile | \ grep admin -C 1 | \ grep -v '\-\-' > /localhost/aws-extend-profiles ``` - If no `aws-accounts` use this `aws-gen-config` ```shell ⨠ aws-gen-config | \ grep -v source_profile | \ grep admin -C 1 | \ grep -v '\-\-' > /localhost/aws-extend-profiles ``` 4. In your browser, left-click the `aws-extend-switch-roles` extension (light-blue icon with a key) and click on the `Configuration` option. 5. Open `~/aws-extend-profiles` in your home directory, copy its contents, and paste it into the form, then click `Save`. 6. Follow the [AWS Sign-In Procedure](https://github.com/BrightDotAi/infrastructure/blob/main/docs/authentication/onboarding.md#aws-console-sign-in-via-okta-apps-dashboard) to open the AWS Console. You should now be able to switch IAM roles across accounts by clicking the extension in your browser and selecting the desired IAM role. :::info See the [official extension docs](https://github.com/tilfinltd/aws-extend-switch-roles#configuration) for more configuration options. ::: --- ## How to Setup SAML Login to AWS from Office 365 import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem Office 365 is a common business suite that has an active director to manage users and permissions. We need to utilize this to login to AWS. ## Solution Azure Devops has **Enterprise Apps** that are used to sign into things from your O365 Account. ### AWS SAML 1. Under [https://aad.portal.azure.com/#allservices/category/All](https://aad.portal.azure.com/#allservices/category/All) go to **Enterprise Applications** 2. Click **New Application** 3. Choose the right application, for SAML it’s **AWS Single-Account Access** 4. Click **Create**, default options are fine 5. On the Left Panel Click **Single sign-on** 6. Choose **SAML** as the Select a single sign-on method. 7. You may be prompted to change or use the defaults, if you have many aws single account logins you will need to modify the defaults. 8. Ensure the Identifier(Entity ID) is set to something valid (Should be `https://signin.aws.amazon.com/saml` optionally add a `#identifier` 9. Download the XML File by Pressing the Button on **Step** **5** **Setup <App Name>**. This should download an XML File, please send this to the CloudPosse Team, this will be placed in your `aws-saml` component to add your login. ### Setting up Login Role The next steps determine which role you sign into from the app. By default we recommend this be the admin team role that has administrative in almost every account (besides creating roles in identity, nor organization permissions in root). If you want to use a different team role, please ensure you understand the team permissions. - [aws-team-roles](/components/library/aws/aws-team-roles/) - [aws-teams](/components/library/aws/aws-teams/) Under the apps **Single sign-on** configuration (where the last steps left off) **Step 2** has an attribute called Role. this is the Role given to the user to attempt to sign into. The AWS Docs ([https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml_assertions.html#saml_role-attribute](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_saml_assertions.html#saml_role-attribute)) require this to be of the format: ``` arn:aws:iam::account-number:role/role-name1,arn:aws:iam::account-number:saml-provider/provider-name ``` This means your **user.assignedroles** in azure should be `arn:aws:iam::account-number:role/role-name1,arn:aws:iam::account-number:saml-provider/provider-name` for example: `arn:aws:iam::00000000000:role/abc-core-gbl-identity-admin,arn:aws:iam::00000000000:saml-provider/abc-core-gbl-identity-provider-azure-ad`, these values are generated from the `aws-saml` component, and should be given by the CloudPosse Team. :::caution Changing a Users Assigned Roles requires an escalated level of Azure Active Directory. (Hidden behind a paywall) ::: :::tip If you are using the most basic plan you can get around this paywall with a regex expression. ::: ### Workaround Regex: - Step **2** edit the attributes and claims, edit specifically the **Role** - Set the Source to be a transformation of anything such as `RegexReplace (user.primaryauthoritativeemail)` - Set the `Regex pattern` to be `(.+)$` - Set the `Replacement pattern` to be your value `arn:aws:iam::account-number:role/role-name1,arn:aws:iam::account-number:saml-provider/provider-name` - Test the replacement and ensure you get a value like the `arn:aws:iam::account-number:role/role-name1,arn:aws:iam::account-number:saml-provider/provider-name`. ## Add The app to Specific Users Then give this app to specific users and let them login to the aws-team! --- ## How to Setup Leapp on Windows with WSL ## Problem You want to set up local AWS credentials but the primary documentation refers to MacOS and Homebrew. Leapp and Geodesic are supported in Windows using WSL. Please see the [Solution](#solution) below. ## Solution 1. Install latest version of Docker Desktop, Leapp, AWS CLI, WSL (Ubuntu 22) - [Docker Desktop](https://www.docker.com/products/docker-desktop/) - [Leapp](https://docs.leapp.cloud/latest/installation/install-leapp/) - [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) - [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) 2. Optional but useful to debug, install the AWS CLI in WSL 3. Start Docker Desktop 4. Launch Leapp UI and connect to Chained Role session Please see [How to Log into AWS](/layers/identity/how-to-log-into-aws#setup), skipping all requirement steps. After you've connected to your identity profile via the Chained Role session, continue with the remaining steps here. 5. Find your home directory for Windows. Most likely this is `C:\Users\YOUR_NAME`. Open WSL and run the following: ```powershell explorer.exe . ``` 6. Copy `.aws` from Windows Home to WSL home In Powershell copy `.\.aws\` to the your local WSL home path found above. For example: ```powershell PS C:\Users\dan> cp -r .\.aws\ \\wsl.localhost\Ubuntu-22.04\home\dan ``` 7. Open WSL and verify your identity ```bash dan@WSL:~$ aws sts get-caller-identity --profile acme-identity { "UserId": "xyz:acme-identity", "Account": "123", "Arn": "arn:aws:sts::111111111111:assumed-role/acme-core-gbl-identity-devops/acme-identity" } ``` 8. Clone and navigate to `infrastructure` directory 9. Build your Geodesic image: ```powershell make all ``` 10. Done! In Geodesic you should see the green checkmark next to your profile. You can also verify with `aws sts`: ```bash ⧉ acme √ : [acme-identity] (HOST) infrastructure ⨠ aws sts get-caller-identity { "UserId": "AROAXYKJQXXXXXXXXXXXX:acme-identity", "Account": "111111111111", "Arn": "arn:aws:sts::111111111111:assumed-role/acme-core-gbl-identity-devops/acme-identity" } ``` ## FAQ ### Can we configure Leapp to write credentials to the WSL home directory? No, Leapp does not support changing the `~/.aws` directory path. [Please see this issue](https://github.com/Noovolari/leapp/issues/153). However [thanks to the Leapp open source community](https://github.com/Noovolari/leapp/issues/153#issuecomment-1199014972), we have a workaround! One way to do it: 1. Install Leapp in Windows 2. Open WSL 3. In WSL backup your `~/.aws` 4. In WSL create symlink from `/mnt/c/Users//.aws` to `~/.aws` Example: ``` # It is ok, if the command returns cp: cannot stat '.aws': No such file or directory cp -av ~/.aws ~/.aws.bak ln -s /mnt/c/Users//.aws ~/.aws ``` Then WSL will follow chosen session in Windows Leapp. One caveat is that Leapp CLI doesn't work in WSL. ### Leapp fails to launch with AWS CLI Error After I downloaded Leapp, I see this error when trying to start the app: ```powershell Error: An error occurred getting AWS CLI.. ``` You must first install the AWS CLI. See [AWS CLI - Getting Started](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) or run this command in PowerShell: ```powershell msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi ``` Restart PowerShell and then verify that the AWS CLI installed successfully: ```powershell PS C:\Users\dan> aws --version aws-cli/2.15.31 Python/3.11.8 Windows/10 exe/AMD64 prompt/off ``` Now restart Leapp. ### Initializing Docker API Proxy Failed When trying to start Docker Desktop I am getting this error: ```powershell Something went wrong. Starting services: initializing Docker API Proxy: open \.\pipe\docker_engine: Access is denied. ``` See this issue: [docker/for-win/issues/13663](https://github.com/docker/for-win/issues/13663). Run `wsl --update` and restart Docker Desktop. ### `.local/bin` is not in my `PATH` When I try to run `make all`, I get this error: ```powershell Inadequate permissions to install to /usr/local/bin. Installing to /home/dan/.local/bin. # Installing acme from acme/infrastructure:latest... # Installed acme to /home/dan/.local/bin/acme hash # WARNING: It appears /home/dan/.local/bin is not on your PATH. Please add it. make: *** [Makefile:56: install] Error 1 ``` Add your `.local/bin` to your `PATH` ```powershell export PATH=$PATH:/home//.local/bin ``` Now try to run `make all` again --- ## How to Use Leapp to Authenticate with AWS Learn how to use Leapp to supply AWS credentials to tools used within Geodesic. ## Problem Your company uses AWS SSO, AWS Federated IAM, or AWS IAM Users and you need to login to AWS. Tools like `saml2aws` offer inconsistent login experiences depending on the SSO provider. You’re a developer and would like to keep your session active during your working day without constantly re-entering your MFA token or running cumbersome command-line tools. ## Solution :::tip Leapp App Use the Leapp desktop app to seamlessly login to AWS with any IdP and keep your session current ::: In this how-to, we will help you get started using Geodesic to work with AWS resources by helping you set up and use [Leapp](https://leapp.cloud/) to handle credentials and authentication. Leapp is an open-source tool that makes this easier. ### Prerequisites 1. Install Leapp CLI from [https://www.leapp.cloud/releases](https://www.leapp.cloud/releases) 2. Install the AWS CLI (e.g. `brew install awscli`); afterwards, you may need to symlink your homebrew AWS CLI installation to `/usr/local/bin` by running: `ln -s /opt/homebrew/bin/aws /usr/local/bin/` 3. Install the AWS Session Manager Plugin, see [https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-install-plugin.html) We expect you’ve gone through our guide on [Geodesic](/resources/legacy/fundamentals/geodesic) prior to using Leapp since that contains some important understanding of what Geodesic is, how it works, and what it’s doing. Your organization is already set up with either the [sso](/components/library/aws/identity-center/) or the [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-primary-roles) and [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) components. #### AWS Region It will be helpful to know which AWS Region you will be primarily working in. This should be fairly common knowledge among people with AWS access at your company and documented as part of the ADR for [Decide on Primary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region). If you are doing this on your own, choose the region closest to you geographically. If you are still in doubt, pick - `ca-central-1` if in Canada, - `us-east-2` if in the United States east of the Mississippi, - `us-west-2` if anywhere else in North America or anywhere in Central America, - `sa-east-1` if in South America, - `eu-north-1` if in Europe, - `af-south-1` if in Africa, - `ap-south-1` if in Asia, or - `ap-southeast-2` if in Australia or Antarctica. #### Credentials You will have to have some way of authenticating to AWS already set up, and you need to know the nature of your credentials. Whoever gave you access to AWS should be able to tell you what kind of access you have. It should be one of the following: - **Federated Role** (recommended). This means you log into something you use to identify yourself to a lot of places (usually referred to as Single Sign-On or SSO), and it is set up to log you into AWS with a specific IAM Role as well. At a company, this is usually Google Workspaces (formerly GSuite), Okta, or Active Directory. This is also known as a SAML IdP. - **AWS SSO**. This means your company has configured AWS as a Single Sign-On provider. This is AWS _as_ a Single Sign-On provider, allowing you to access multiple _permission sets_ within AWS, not using some other Single Sign-On provider to sign in to AWS as a single IAM Role. Please note that even if your company has set up AWS _as_ a Single Sign-On provider, you still may be using your company’s primary SSO provider to authenticate to AWS SSO. - **AWS IAM User**. This is the older way of authenticating to AWS, with a basic username and password to log into the AWS console and a long-lived “Access Key” for API access. If you are going to use this method, we strongly recommend that you enable multi-factor authentication (MFA). Based on which kind of credentials you have, you will need to gather different information with which to configure Leapp. Whoever is in charge of setting up your access to AWS should be able to give you the information you need. ##### Federated Role If using Federated Login, you will need - Your IdP Single Sign-On URL. - For Google Workspaces, it looks something like `https://accounts.google.com/o/saml2/initsso?idpid=C0asdfasdfal&spid=12344321`. - For Okta, it looks something like `https://company.okta.com/app/company_samlidp_1/Hka1abcke6h4P1WQr5d7/sso/saml`. - Most importantly,it should not be confused with the AWS Single Sign-On URLs like these (**do not use these**) - `https://signin.aws.amazon.com/saml` (this is used by your IdP admin) - `https://my-sso-portal.awsapps.com/start` (this means you are using AWS SSO) - The AWS Identity Provider ARN is assigned by AWS to your IdP. Something like - `arn:aws:iam::123434211234:saml-provider/company-name` - Your assigned/authorized AWS IAM Role ARN. Something like - `arn:aws:iam::123434211234:role/role-name` ##### AWS SSO If using AWS SSO, you will need: - Your AWS SSO “start URL”, also known as your “portal URL”. It should be very close to: - `https://something.awsapps.com/start` - The region in which AWS SSO has been deployed. This may or may not be the same region you will be working in. ##### AWS IAM User If you are a regular IAM User who can log into the AWS Console, you should log into the AWS Console while setting up Leapp. Choose “My Security Credentials” from the Account drop-down menu: - Copy and paste your MFA ARN or Serial Number from the “Assigned MFA device” - Click on “Create access key” to create a new access key and copy the Access Key ID and Secret Access Key from the console and paste directly into the appropriate fields in Leapp. ### Step-by-Step Process #### Start Geodesic If you are a Cloud Posse client, you will have a customized version of Geodesic that sets an initial value for the environment variable `AWS_PROFILE`. Alternatively, you may have customized it yourself, or you may want to. In any case, we will go with what you have. So start Geodesic, and at the command prompt, type ``` echo $AWS_PROFILE ``` This is the value you will give to the profile in Leapp when you configure it. If the output of `echo` is blank, as would be expected if you are running our public tutorial image, use “default” for the profile name. Verify you do not have any credentials set via your environment variables. The following command should have no output: ``` declare -p | grep -E '(AWS_ACCESS_KEY_ID|AWS_SECRET_ACCESS_KEY|AWS_SESSION_TOKEN)=' ``` If any values are output by the `declare -p | grep...` command, you need to find a way to prevent those environment variables from being set when you start Geodesic. Most likely they were set via some customization you have installed, or by running something like `aws-vault`. Fix this before going forward, as having those environment variables set will prevent Leapp from being effective. #### Install Leapp Please refer to the official Leapp documentation for the latest instructions on installing and configuring Leapp. Now that you are armed with the information from the previous steps, it should be pretty easy. - Visit the [Leapp website](https://leapp.cloud/) - Download and install Leapp as instructed by the website - If using homebrew, you can `brew install --cask leapp` - Follow the instructions (under “Docs”) for configuring AWS #### Configure Leapp using the cli [https://www.npmjs.com/package/@noovolari/leapp-cli](https://www.npmjs.com/package/@noovolari/leapp-cli) `brew install Noovolari/brew/leapp-cli` Setup some environment variables, use your `LEAPP_IDP_URL` and ``` # your IDP URL by right clicking on the Google AWS icon and clicking "Copy Link" export LEAPP_IDP_URL="https://accounts.google.com/o/saml2/initsso?idpid=snip&spid=snip&forceauthn=false" export LEAPP_IDP_ARN="arn:aws:iam::1234567890:saml-provider/identity" export LEAPP_ROLE_ARN="arn:aws:iam::1234567890:role/identity" export LEAPP_PROFILE_NAME="my-identity-profile" export LEAPP_PROFILE_ALIAS="my-profile-alias" export LEAPP_REGION="us-east-2" ``` Create the idp url only once ``` ✗ leapp idp-url create --idpUrl $LEAPP_IDP_URL identity provider URL created ``` For each customer ``` ✗ leapp profile create --profileName $LEAPP_PROFILE_NAME \ && leapp session add \ --providerType "aws" \ --sessionType "awsIamRoleFederated" \ --sessionName $LEAPP_PROFILE_ALIAS \ --roleSessionName $LEAPP_PROFILE_ALIAS \ --region $LEAPP_REGION \ --roleArn $LEAPP_ROLE_ARN \ --idpUrl $LEAPP_IDP_URL \ --idpArn $LEAPP_IDP_ARN \ --profileId $(leapp profile list -x | grep $LEAPP_PROFILE_NAME | awk '{ print $1 }' | head -1) profile created session added ✗ LEAPP_SESSION_ID=$(leapp session list -x --csv | grep $LEAPP_PROFILE_ALIAS | cut -d',' -f1) ✗ leapp session start --sessionId $LEAPP_SESSION_ID session started ``` #### Configure Leapp using the app Below is some guidance to the Leapp documentation that applies as of the time of this writing. By the time you read this it may be out of date, but hopefully will still be of help in guiding you through the Leapp site. - The “AWS Profile” setting in Leapp must match _exactly_ the value of `$AWS_PROFILE` you found in Geodesic in the earlier step. - The “AWS Region” you set in the Leapp session should be the AWS Region you most often use, as discussed [above](/resources/legacy/howto/geodesic/authenticate-with-leapp#aws-region-and-credentials). - The “Session Alias” is completely up to you: it is the name for this set of credentials that you will see in the UI. The Leapp documentation is at [https://docs.leapp.cloud/](https://docs.leapp.cloud/) and the best set of instructions to follow are the ones under a sub-menu on the left side of the page: **Tutorials > AWS** - If you have a **Federated Login**, pick “AWS IAM Federated Role”. Most of the tutorial is about how to configure Federated Login itself, and you can skip all that. Just follow the last step: “Setup in Leapp”. - Otherwise, pick “AWS SSO” or “AWS IAM User” and follow the steps. #### Log in Using Leapp Leapp provides two UIs for logging into or out of a session. There is a system-wide menu item in the taskbar on Windows systems or as a “status menu” on the Mac menu bar. Click on it and a menu will appear with all your configured session names and their corresponding profile names. An icon for each session will appear either in gray if logged out or orange if logged in (at least those are the current defaults for the “Light” color scheme on macOS). Just select the menu item to toggle the state. Alternately, you can select “Show” from the menu and be shown a richer UI where you can do more, but still, the main thing is that you click on a session name to change its state and the indicator shows gray for logged out and orange for logged in. If you are not using IAM user access keys then in order to access AWS you will have to log in to your “identity provider” (e.g. Okta, Google) like you do for access to other resources. Therefore, when you try to activate a session in Leapp, it may open a small web browser window popup or open a window or tab in your default browser so you can complete the login flow, including supplying a second factor if you are using MFA, and perhaps solving a CAPTCHA puzzle. This is normal and expected behavior, not a virus or hacking attempt. When you finish logging in via the web browser, Leapp uses the credentials provided by your identity provider to authenticate to AWS, just as if you were logging into the AWS web console or SSO portal. Like your web browser, Leapp can store cookies and other information, and in addition, Leapp is able to use your system keychain for secure storage of other things like your Secret Access Key. Because of this, and depending on your identity provider and their settings, simply being logged into your computer may be enough authentication for Leapp to grant you access when you enable a session without asking you for anything. However, AWS requires periodic refreshing of session keys and if you are using MFA with AWS, you should enable Leapp notifications so you can receive prompts when your AWS session expires and Leapp needs a new MFA token in order to start a new session for you. #### It Should “Just Work” Once you log in to AWS using Leapp, go back into Geodesic and simply hit return at the prompt. The prompt should change from having a red ✗ and `[none]` to having a green √ and `[default]` or whatever your profile name is. This is Geodesic’s way of letting you know you are authorized with AWS, and what profile you have active. You can always confirm your current authentication status by running: ``` aws sts get-caller-identity ``` When you are properly authenticated, it will output your IAM role and other information. When you are logged out, it will print some kind of error message, usually: ``` Unable to locate credentials. You can configure credentials by running "aws configure". ``` ### FAQ #### “Named Profile” does not show my profile in the dropdown Click the box under “Named Profile” and start typing to add a new list item. Your text should replace what is in the box, i.e. “default”, and you will get an option to add a new list item. #### Getting Leapp integrated with Windows This presents a unique challenge as `docker` does not understand Windows block devices. If you try to simply mount the `.aws` from your Windows-native Leapp into a Geodesic container, the volume mount will fail to follow a symlink. It also will not share the Windows device. To get around this, there are two options: 1. Set up a cronjob that regularly syncs the `.aws` directory from your Windows native Leapp into your WSL2 home directory. This is the easiest option, but it means you'll have to wait for a few seconds depending on how often you run the cronjob. 2. Set up a FIFO pipe using `mkfifo` that will allow you to stream the output of `leapp` into the `.aws/credentials` file. If your `.aws/config` changes a lot, you'd also need a separate file. This is the more complicated option, but you can run `leapp` and have it immediately available in your Geodesic container. A simple example follows: ```bash #!/bin/bash mkfifo ~/.aws/credentials (while :; do cat /mnt/c/Users//.aws/credentials > ~/.aws/credentials; done) & ``` Because there has to be a reader in order to write, the while loop will block until something tries to read from the pipe. At worst, you may need to enter Geodesic twice to ensure the latest credentials are used. NOTE: This is just a mundane script. Sophistication can be added to support closing the pipe on exit, detecting if the pipe is already open, etc. --- ## Tutorials(6) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the associated identity layey components. --- ## How to Monitor Everything with Datadog import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Admonition from '@theme/Admonition'; This outlines Cloud Posse's practical approach to monitoring with Datadog, focusing on defining reusable Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for consistent implementation across customer environments. It aims to help businesses streamline monitoring and incident management by aligning technical performance with business goals. Our goal with this document is to identify the reusable, standard SLI/SLO for our customers that we can readily implement time and time again. This document goes very deep into the theory and practice of monitoring with Datadog. If you are looking for a quick start guide, see [our Setup Guide for DataDog](/layers/monitoring/datadog/setup/). If you are trying to solve a specific problem, checkout [our tutorials for DataDog](/layers/monitoring/datadog/tutorials/). ## Problem A typical business operating in AWS has literally tens of thousands of data points that leads to analysis paralysis. If we sit down and try to make sense of it all, we realize knowing what is critical and when is non-trivial. Modern advances in monitoring platforms have brought Artificial Intelligence and Anomaly Detection, but those are not foolproof. There is no silver bullet. AI can not tell us what is mission-critical for _our_ business to succeed. That’s because they only see part of the story. They have no visibility into our finances or business objectives. Defining Service Level Objectives, therefore, becomes paramount and while mathematically trivial to compute, but defining what are services in the context of your organization and what your objectives are, is still very subjective. There’s an overwhelming amount of theory on what we should do, but not that much prescriptive step-by-step advice on how to actually implement it using IaC resulting in a fully functioning system. ## Solution - Define a process for us to help our customers easily define SLOs that are special to their business - Define a process for us to identify generalized SLOs for every one of our customers that have a very common stack - Use SLOs to determine the health of a service, where a service is not always what end-users of the _business,_ but might be internal customers - Use SLOs to reduce the number of alerts and concentrate them on when specifically the objectives have been violated ## Definitions
Service
Any group of one or more applications (or "components") with a shared purpose for some customer (either internal to some team or other service or external with end-users).
Service Level Indicator (SLI)
A quantitative measurement of a service's performance or reliability (e.g. the amount of time a transaction took). SLIs may or may not be expressed as percentages. In Datadog, the SLI is implemented as a metric, synthetic or aggregation of one or more monitors. There's nothing called an SLI in Datadog.
Service Level Objective (SLO)
A target percentage for an SLI over a specific period of time. It's always expressed as a percentage. In Datadog it's a specific resource type defined with a numerator and denominator. A score of 100% is excellent; 0% is dead. Datadog has native support for SLOs as a resource.
Service Level Agreement (SLA)
An explicit (e.g. contractual agreement) or implicit agreement between a client and service provider stipulating the client's reliability expectations and service provider's consequences for not meeting them.
Error Budget
The allowed amount of unreliability derived from an SLO's target percentage (100% - target percentage) that is meant to be invested into product development. There's a corresponding Burn Rate associated with an Error Budget that is equally important to understand because it's the rate of change. Datadog automates burnrate as part of the widget.
MTTF (mean time to failure)
The outage frequency.
MTTR (mean time to restore)
The outage duration and is defined as it is experienced by users: lasting from the start of a malfunction until normal behavior resumes.
Availability
A percentage defined as (uptime)/(total length of time), using appropriate units (e.g. seconds), or (1 - (MTTR/MTTF)) x 100%.
RED method
(Rate, Errors, and Duration) focuses on monitoring your services, leaving their infrastructure aside and giving you an external view of the services themselves—in other words, from the client's point of view.
USE method
(Utilization, Saturation, and Errors) focuses on the utilization of resources to quickly identify common bottlenecks; however, this method only uses request errors as an external indicator of problems and is thus unable to identify latency-based issues that can affect your systems as well.
Some definitions borrowed from Datadog's [Key Terminology](https://docs.datadoghq.com/monitors/service_level_objectives/#key-terminology). ## Theory ### Golden Signals (SLIs) Golden Signals are a form of telemetry that applies to anything with throughput. Anything that’s not a golden signal is considered general telemetry.
Latency
The time it takes to service a request. It’s important to distinguish between the latency of successful requests and the latency of failed requests.
Traffic
A measure of how much demand is being placed on your system, measured in a high-level system-specific metric. For a web service, this measurement is usually HTTP requests per second.
Errors
The rate of requests that fail, either explicitly (e.g., HTTP 500s), implicitly (for example, an HTTP 200 success response, but coupled with the wrong content), or by policy (for example, "If you committed to one-second response times, any request over one second is an error").
Saturation
How "full" your service is. A measure of your system fraction, emphasizing the resources that are most constrained (e.g., in a memory-constrained system, show memory; in an I/O-constrained system, show I/O).
These golden signals are closely related to the [RED metrics](https://www.weave.works/blog/the-red-method-key-metrics-for-microservices-architecture/) for microservices: rate, errors, and duration, and the older [USE method](https://www.weave.works/blog/the-red-method-key-metrics-for-microservices-architecture/) focusing on utilization, saturation, and errors. These signals are used to calculate the service level objectives (SLOs). ### Other Signals Here are other useful metrics that are examples that don’t necessarily fit into the Golden Signals. #### Pull Requests
Time to Merge
The duration between the creation of a pull request and when it's merged. This metric reflects the efficiency of the review and merging process.
Lead Time
The total time from when work on a pull request starts until it's merged. It measures the overall speed of the development process.
Size (LOC)
The number of lines of code (LOC) changed in a pull request. This metric helps gauge the complexity and potential impact of the changes.
Flow Ratio
The ratio of the total number of pull requests opened to those closed in a day. It indicates whether the team's workflow is balanced and sustainable.
Discussions & Comments
The number of comments and discussions on a pull request. This metric shows the level of collaboration and code review quality.
Force Merged by Admins (bypassing approvals)
The number of pull requests merged by admins without the usual approval process. This metric highlights instances where standard review procedures were skipped.
#### Code Quality
Code Coverage
The percentage of your codebase covered by automated tests. This metric helps ensure that your code is thoroughly tested.
Static Code Analysis
The number of issues found by static code analysis tools. This metric helps identify potential bugs and security vulnerabilities.
Defect Escape Rate
The percentage of defects found in production that weren't caught by automated tests. This metric indicates the effectiveness of your testing strategy.
#### Customer Experience
Page Load Time
The time it takes for a web page to load. This metric is crucial for ensuring a positive user experience.
Browser Interaction Time
The time it takes for a user to interact with a web page. This metric helps gauge the responsiveness of your application.
Server Error Rate
The percentage of server errors returned to users. This metric indicates the reliability of your backend infrastructure.
JS Errors
The number of JavaScript errors encountered by users. This metric helps identify issues that impact the user experience.
### Service-Oriented View #### What is a Service? - A service is anything that warrants having an SLO - A service has a contract with internal parties or external parties to provide a “service” with certain guarantees. Guarantees with a contractual commitment are SLAs. - We Should be able to look at a business service and drill down to the alerts affecting that service #### Questions to ask: - What are the services we provide? e.g. API - Who are the consumers of those services? e.g. internal users or customers - Who is the support team of those services? ### Error Budget The Error Budget determines risk tolerance. It is usually a calculation of the remaining time in a given period that the SLO can be violated before the SLA is violated. When the budget is depleted (nearing 0%), reliability should be prioritized, which often results in: - freeze feature releases - prioritize postmortem items - improve automation over human toil - improve monitoring and observability - consult/collaborate with SRE When an Error Budget is abundant (remaining error budget > 0%), velocity should be prioritized, which results in: - release new features - expected system changes - try risky (but valuable) experiments ## Our Strategy 1. #### Decide on what qualifies as a “service” for the business 1. #### For every service, decide on how to quantify the 4 [_golden signals_](#golden-signals-slis), **these are your SLI**s - [Latency](#latency) - [Traffic](#traffic) - [Errors](#errors) - [Saturation](#saturation) 1. #### Determine how to create **SLO’s** from these SLIs over time. This should be a simple math formula over time. 1. #### Incidents are created **ONLY** in response to violations of the SLO for some period of time _t._ 1. #### There should be a 24 hour, 7 day, 30 day and 1 year SLO (for customers). 1. #### All alerts that relate to some SLO and Incident share same common tag convention. 1. #### Our SLO Dashboards should be by Service, coalescing into Team Dashboards, which further group into Organization Dashboards. 1. #### Organization/Team Dashboards should include a list of services (as we can only use DD’s APM Service for things that execute - such as containers, lambdas, and hosts) 1. #### Each service should have enough monitors to accurately describe when it is operating **correctly** and when it is not, no more. **Ideally** **1-3 SLIs per user journey** [[source](https://www.datadoghq.com/videos/solving-reliability-fears-with-service-level-objectives/#focus-on-one-to-three-slis)].
**As Simple as Possible, No Simpler** [From Google's SRE Book](https://sre.google/sre-book/monitoring-distributed-systems/):
To avoid an overly complex, and eventually unmaintainable monitoring system we propose avoiding unnecessary complexity, such as: Alerts on different latency thresholds, at different percentiles, on all kinds of different metrics Extra code to detect and expose possible causes Associated dashboards for each of these possible causes The sources of potential complexity are never-ending. Like all software systems, monitoring can become so complex that it’s fragile, complicated to change, and a maintenance burden. Therefore, design your monitoring system with an eye toward simplicity. In choosing what to monitor, keep the following guidelines in mind: The rules that catch real incidents most often should be as simple, predictable, and reliable as possible. Data collection, aggregation, and alerting configuration that is rarely exercised (e.g., less than once a quarter for some SRE teams) should be up for removal. Signals that are collected, but not exposed in any prebaked dashboard nor used by any alert, are candidates for removal.
## Theory into Practice ### SLIs We should create monitors for the [_Golden Signals_](#golden-signals-slis) for a given service. These are the SLIs. We can make SLIs a percentage out of 100%. Percentages are easy to understand by the widest audience, for example, 0% is bad, 100% is good. In Datadog, we can create a monitor for each SLI. We can also create a Datadog SLO for each SLI. Every SLI may or may not be a Datadog SLO. An SLI may be part of one or more SLOs (e.g. By Monitor Uptime). [See Datadog Docs for details](https://docs.datadoghq.com/service_management/service_level_objectives/metric/#define-queries). ### SLAs SLAs should be thought of like SLOs but on an annual basis and tied to a contractual commitment (expectations, impacts, and consequences) with customers. These usually have penalties associated with them. Additionally, SLAs are typically on a calendar year, and not a rolling window of 365 days. ### SLOs With well-defined SLOs we can ensure that everyone from the business side to engineering is aligned on what’s important and what the goals are for the business at large. - There is no one SLO. Each service gets its own SLO. - Combine significant SLIs for a given capability into a single SLO for that capability - A dashboard must exist that aggregates all SLOs in one place. - Technically, SLOs are not restricted to just what’s in Datadog, however, that’s just what we’re able to monitor. - Not every SLO is associated with an SLA SLOs will not necessarily be the same per business. #### Goals - As few SLOs as possible, but no fewer - Aggregate as much as possible into an SLO, but for the _right_ audience #### Examples | **Business Impact** | **Good examples of SLOs** | **Bad Examples of SLOs** | |-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------| | Our SEO experts require that pages respond within 300ms or we will drop in rankings. The website receives most of it’s traffic from natural search. | 99.95% of pages respond with a 200 response code in less than 300ms | CPU utilization is less than 95%SQL Database Read I/O is at 90% utilization | | As an e-commerce business, users must be able to add products to their cart and checkout or have no revenue. | 99.99% of cart transactions succeed without error. 99.99% of checkout transactions succeed without error. | | | As a online store, customers are much more likely to purchase products with images than with placeholder images. | 99.99% of images served are actual product images (e.g.99.99% of images are served within a latency of 100ms99.99% of images are served with a status code of 2XX | | #### Implementation [Datadog's SLO Page](https://app.datadoghq.com/slo/manage) and [Terraform resource](https://registry.terraform.io/providers/DataDog/datadog/latest/docs/reference/service_level_objective) We should have multiple early indicators (aka thresholds) for when the SLO is in jeopardy: e.g. when 10%, 25%, 50%, 75% of the error budget is exhausted.
**Target**: 99.9% Uptime **Error Budget**: 0.10 **“...at this rate, you’ll exhaust the error budget (violate the SLO) in X time.”**
Every monitor (SLI) that influences a given SLO should share a common tag value (E.g. `slo:myapp`) with the SLO. In other words, the SLO should be tagged `slo:myapp` and the monitors (SLI) corresponding to that (the _golden signals)_ should also then be tagged `slo:myapp`. :::caution Datadog SLOs are implemented on a rolling window, while SLAs are may be tied to a calendar period (e.g. the calendar year). ::: ```hcl # Create a new Datadog service level objective resource "datadog_service_level_objective" "foo" { name = "Example Metric SLO" type = "metric" description = "My custom metric SLO" query { numerator = "sum:my.custom.count.metric{type:good_events}.as_count()" denominator = "sum:my.custom.count.metric{*}.as_count()" } # 7 day rolling window thresholds { timeframe = "7d" target = 99.9 warning = 99.99 target_display = "99.900" warning_display = "99.990" } # 1 month rolling window thresholds { timeframe = "30d" target = 99.9 warning = 99.99 target_display = "99.900" warning_display = "99.990" } # 1 year rolling window (SLA) thresholds { timeframe = "365d" target = 99.9 warning = 99.99 target_display = "99.900" warning_display = "99.990" } tags = ["foo:bar", "baz"] } ``` Questions: ### Dashboards - Only SLOs are on the SLO dashboard - A dashboard must exist that aggregates all SLOs in one place, group logically by some service - Each widget displays an SLO with the remaining error budget. ## FAQ - **How to visualize error budgets and burn rate?** - [Using Datadogs SLO Summary](https://docs.datadoghq.com/dashboards/widgets/slo/#setup) - [Datadog Error Budget Alerts](https://docs.datadoghq.com/monitors/service_level_objectives/error_budget/#overview) - [SLO Checklist](https://docs.datadoghq.com/monitors/guide/slo-checklist/) ## References ### Datadog: - [Track the status of all your SLOs in Datadog](https://www.datadoghq.com/blog/slo-monitoring-tracking/) - [Best practices for managing your SLOs with Datadog](https://www.datadoghq.com/blog/define-and-manage-slos/) - [Datadog Picking good SLIs](https://www.datadoghq.com/blog/establishing-service-level-objectives/#picking-good-slis) - [DataDog Learning Center](https://learn.datadoghq.com/) - Video: [Error budgets for SLOs](https://www.datadoghq.com/videos/solving-reliability-fears-with-service-level-objectives/#error-budgets-for-slos) ### Google SRE: - [Monitoring Distributed Systems](https://sre.google/sre-book/monitoring-distributed-systems/) - [Setting SLOs: a step-by-step guide](https://cloud.google.com/blog/products/management-tools/practical-guide-to-setting-slos) - [SRE at Google: Our complete list of CRE life lessons](https://cloud.google.com/blog/products/devops-sre/sre-at-google-our-complete-list-of-cre-life-lessons) ### Blogs: - [SREs: Stop Asking Your Product Managers for SLOs](https://devops.com/sres-stop-asking-your-product-managers-for-slos/?utm_source=pocket_mylist) - [SRE fundamentals: SLIs, SLAs and SLOs](https://cloudplatform.googleblog.com/2018/07/sre-fundamentals-slis-slas-and-slos.html?utm_source=pocket_mylist) - [Microservice Observability, Part 1: Disambiguating Observability and Monitoring](https://bravenewgeek.com/microservice-observability-part-1-disambiguating-observability-and-monitoring/?utm_source=pocket_mylist) - [5 metrics Engineering Managers can extract from Pull Requests](https://sourcelevel.io/blog/5-metrics-engineering-managers-can-extract-from-pull-requests) - [Metrics to Improve Continuous Integration Performance](https://harness.io/blog/continuous-integration-performance-metrics/) - [SLIs and Error Budgets: What These Terms Mean and How They Apply to Your Platform Monitoring Strategy](https://tanzu.vmware.com/content/blog/slis-and-error-budgets-what-these-terms-mean-and-how-they-apply-to-your-platform-monitoring-strategy) - [SLOs and SLIs best practices for systems](https://newrelic.com/blog/best-practices/best-practices-for-setting-slos-and-slis-for-modern-complex-systems) ## Appendix #### Golden Signals Workbook | | **Golden Signals** | | | | |------------------------------------|--------------------------------------------------------------------------|--------------------------------------------------------------------------|----------------------------------------------------------------|----------------------------------------------------------------------------------------------| | **Services** | **Latency** | **Traffic** | **Errors** | **Saturation** | | | | | | | | **Platform Owners** | | | | | | Kubernetes Cluster: Pod Scheduling | | | | | | Kubernetes Cluster: Capacity | | Healthy Nodes / Total Nodes online | | | | Application Deployments | Time to Deploy / Total time of | Number of Successful Deployments / Total number of Deployments | 1 - (Unsuccessful Deployments / Total Deployments) | Total Time to Deploy in Seconds per Duration / Seconds in Duration (e.g. 86400 = 1 day) | | AWS Spend | | | | | | | | | | | | **Security & Compliance** | | | | | | | Time to Acknowledge Time to fix | | False PositivesNumber of Security Vulnerabilities | | | | | | | | | **Development & PM** | | | | | | Pull Request Throughput | 1 - (Time to Close or Merge PR / 1 Sprint) | 1 - (Number of Open PRs / Total Number of PRs) | PRs Open with Tests Passing / Total PRs Open | 1 - (PRs Open / Max Number of PRs Acceptable) | | Sprint Throughput | Total Number of Sprints to Complete Issues / Total Number of Issues | Issues Transitioned to Done / Total Issues in Sprint | Bugs Added to Active Sprint / Total Issues in Sprint | Total Issues Not Completed / Total Issues in Sprint | | | | | | | | **Microservice / Web Application** | | | | | | HTTP Requests | | | | Notes: INFO, DEBUG log level alerts relative to all alerts. | | Transactions | | | | | | Synthetic Requests | | | | | | | | | | | | **Customer Experience** | | | | | | | | | | | | **CI/CD and Release Management** | | | | | | Lead Time To Deploy | | | | | | Code Coverage | | | | | | Test Coverage | | | | | | GitHub Action Runs | Time to start build | | | | --- ## Setup Datadog import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Admonition from '@theme/Admonition'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; You need to set up monitoring for all of your newly deployed accounts. Datadog setup can get started after the accounts have been provisioned, though it won't be incredibly useful until your plat is in place, usually EKS or ECS. ## Prepare Datadog You'll need a datadog account and to generate an app key and api key for that datadog account and place them in SSM of your `auto` account. These should be placed under `datadog/default/datadog_app_key` and `datadog/default/datadog_api_key` respectively. To generate these keys we recommend using a Datadog Service Account. This allows you to create a service account with limited permissions to your Datadog account. This is useful for security and auditing purposes. This also allows any admins to rotate the keys without having to go through the account owner. ## Creating a Datadog Service Account 1. Go to your Organizations settings page 2. Click on the `Service Accounts` tab 3. Click on `New Service Account` 4. Give the service account a name and an email address 5. Give the service account the `Datadog Admin Role` (can be refined later) 6. Click `Create Service Account` 7. Click the created service account 8. Under Application Keys, click `New Key` 9. Give the Application Key a name (we recommend something like `terraform`) and click `Create Key` 10. Copy the `Application Key` for later. This is your `datadog_app_key` 11. Under Organization Settings, click `API Keys` 12. Click `New Key` 13. Give the API Key a name (we recommend something like `terraform`) 14. Click `Create Key` 15. Copy the API Key for later. This is your `datadog_api_key` ## Short Version There are two core components to the Datadog implementation 1. [**datadog-configuration**](https://docs.cloudposse.com/components/library/aws/datadog-credentials) 2. [**datadog-integration**](https://docs.cloudposse.com/components/library/aws/datadog-integration) Both are deployed to every account except `identity` and `root`. They are deployed to the global stack as they are done once per account. Once those are setup, we can begin deploying other components, such as - [**Monitors**](https://docs.cloudposse.com/components/library/aws/datadog-monitor) - [**Lambda Log Forwarders**](https://docs.cloudposse.com/components/library/aws/datadog-lambda-forwarder) - [**Datadog Log Archives**](https://docs.cloudposse.com/components/library/aws/datadog-logs-archive) We then deploy a setup for monitoring applications based on whether you use EKS or ECS. For **EKS** - [**Datadog Agent**](https://docs.cloudposse.com/components/library/aws/eks/datadog-agent) - [**Datadog Private Locations**](https://docs.cloudposse.com/components/library/aws/datadog-synthetics-private-location) For **ECS** - [**ECS-Service**](https://docs.cloudposse.com/components/library/aws/ecs-service) has a [datadog file](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/ecs-service/datadog-agent.tf) that manages all of datadog agent configuration for a service (Datadog as a sidecar) - [**ECS Private Locations**](https://docs.cloudposse.com/components/library/aws/datadog-private-location-ecs) ## Step by Step You should have a workflow to vendor in your components. This workflow can be run with the following command. Otherwise vendor in each component individually. ## Datadog Configuration This component handles the creation and duplication of Datadog API and APP keys. This component specifies a source account (usually `auto`) and a format for copying keys. You specify a source and destination format and a key store. This allows you to use separate keys for each account, tenant, or anything in between. We recommend either a single set of keys per Organization or tenant. This component also handles default configurations such as Datadog URL and provides a default configuration for other components to utilize via its submodule `datadog_keys`. Use a configuration similar to the following but check the [`README.md`](https://docs.cloudposse.com/components/library/aws/datadog-credentials/) for exact input references. ```yaml components: terraform: datadog-configuration: settings: spacelift: workspace_enabled: true vars: enabled: true name: datadog-configuration datadog_secrets_store_type: SSM datadog_secrets_source_store_account_stage: auto datadog_secrets_source_store_account_region: "us-east-2" datadog_site_url: us5.datadoghq.com ``` The most important variables are the key patterns to determine how keys are placed and the Datadog site URL configuration which should match how you signed up with Datadog. ## Datadog Integration Vendor in this component with `atmos vendor pull -c datadog-integration`. This component configures the integrations you have between Datadog and your AWS Accounts. This component is deployed to every account (except `root` and `identity`) to allow data from everywhere. This component is used by other components as this component creates the Datadog role for your account. Deploy this with `atmos terraform deploy datadog-integration -s ${tenant}-gbl-${stage}` alternatively ```shell atmos terraform deploy datadog-integration -s core-gbl-artifacts atmos terraform deploy datadog-integration -s core-gbl-audit atmos terraform deploy datadog-integration -s core-gbl-auto atmos terraform deploy datadog-integration -s core-gbl-dns atmos terraform deploy datadog-integration -s core-gbl-network atmos terraform deploy datadog-integration -s core-gbl-security atmos terraform deploy datadog-integration -s plat-gbl-sandbox atmos terraform deploy datadog-integration -s plat-gbl-dev atmos terraform deploy datadog-integration -s plat-gbl-staging atmos terraform deploy datadog-integration -s plat-gbl-prod ``` ## Datadog Monitors The `datadog-monitor` component creates monitors for Datadog. It contains a catalog of monitor entries that are deployed by default to every account this is deployed to. This component is deployed _globally_ as it is only deployed once per account. By default, we only apply this to `auto` and `plat` accounts. However, it can be added to more accounts as necessary for monitoring. Monitors are cataloged through YAML files and perform substitution through Terraform syntax, for example `${stage}`. It is important to note that this is different from Datadog syntax which is `{{ stage }}`. Anything in Datadog syntax will be inserted into the monitor as is, whereas Terraform will be substituted. That way we can deploy the same monitors across accounts and filter by stage or variable known to Terraform. In order to add new monitors, simply add a yaml file to `components/terraform/datadog-monitor/catalog/monitors/`. By default, the component includes a global collection of monitors: ```bash components/terraform/datadog-monitor/catalog/monitors/ ├── README.md ├── catalog │   └── monitors │   ├── aurora.yaml │   ├── ec2.yaml │   ├── efs.yaml │   ├── elb.yaml │   ├── host.yaml │   ├── k8s.yaml │   ├── lambda-log-forwarder.yaml │   ├── lambda.yaml │   ├── rabbitmq.yaml │   └── rds.yaml ├── component.yaml ├── context.tf ├── main.tf ├── outputs.tf ├── provider-datadog.tf ├── providers.tf ├── variables.tf └── versions.tf ``` Alternatively, we can add an additional level of nesting to the `datadog-monitor` catalog to categorize monitors by account. By arranging the catalog as follows, we can distinguish which monitors are deployed to a given stack with `local_datadog_monitors_config_paths`. This allows us to specify entirely unique monitor paths for each stage. ```bash components/terraform/datadog-monitor/catalog/monitors/ ├── README.md ├── catalog │   └── monitors │   ├── _defaults │   │   └── example.yaml │   ├── plat │   │   ├── dev │   │   │   └── example.yaml │   │   ├── staging │   │   └── prod └── ... ``` ```yaml # stacks/org/acme/plat/dev/monitoring.yaml components: terraform: datadog-monitor: vars: local_datadog_monitors_config_paths: - catalog/monitors/_defaults/*.yaml - catalog/monitors/plat/*.yaml - catalog/monitors/plat/dev/*.yaml ``` ```yaml # stacks/org/acme/plat/prod/monitoring.yaml components: terraform: datadog-monitor: vars: local_datadog_monitors_config_paths: - catalog/monitors/_defaults/*.yaml - catalog/monitors/plat/*.yaml - catalog/monitors/plat/prod/*.yaml ``` Each monitor is then defined in `components/terraform/datadog-monitors/catalog/monitors/_defaults/` categorized by component. It can then be extended into other stages, where the later in the array (`local_datadog_monitors_config_paths`) the higher precedence it takes in merging. Please see [datadog-monitor](https://docs.cloudposse.com/components/library/aws/datadog-monitor/) for more information. ## Lambda Log Forwarders This component is pretty straightforward to vendor and deploy. The important variables of note are ```yaml forwarder_rds_enabled: false forwarder_log_enabled: false forwarder_vpc_logs_enabled: false ``` as these variables determine which logs are forwarded to Datadog. The main implication of this decision is the cost, as VPC Flow logs can become incredibly expensive. ## Datadog Logs Archive This component is also relatively simple to deploy as well. Simply vendor in the component and deploy it. ```shell atmos vendor pull -c datadog-logs-archive ``` Use the configuration in the component readme as the stack/catalog entry. ```shell atmos terraform deploy datadog-logs-archive -s core-gbl-auto ``` ## EKS ### Datadog Agent For EKS deployments we need to deploy the [Datadog-Agent](https://docs.cloudposse.com/components/library/aws/eks/datadog-agent), this component deploys the helm chart for the Datadog Agent, it allows the Datadog Agent to be fully customized and also provides a format to support cluster-checks, which are a cheaper version of synthetic checks (though less feature-rich). Vendor in the component and begin deploying. This component is deployed to every region and account where you have an EKS Cluster. The component allows customizing values passed to the helm chart, this can be useful when passing variables to support features such as `IMDSV2` ```yaml components: terraform: datadog-agent: vars: enabled: true name: "datadog" description: "Datadog Kubernetes Agent" kubernetes_namespace: "monitoring" create_namespace: true repository: "https://helm.datadoghq.com" chart: "datadog" chart_version: "3.6.7" timeout: 1200 wait: true atomic: true cleanup_on_fail: true cluster_checks_enabled: true helm_manifest_experiment_enabled: false tags: team: sre service: datadog-agent app: monitoring # datadog-agent shouldn't be deployed to the Fargate nodes values: agents: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: eks.amazonaws.com/compute-type operator: NotIn values: - fargate datadog: env: - name: DD_EC2_PREFER_IMDSV2 value: "true" ``` This component should be highly customized to meet your needs. Please read through the Datadog Dogs to determine the best configuration for your setup. #### References - [Configure the Datadog Agent on Kubernetes](https://docs.datadoghq.com/containers/kubernetes/configuration?tab=helm) - [Duplicate hosts with Kubernetes on AWS (EC2 or EKS)](https://docs.datadoghq.com/containers/troubleshooting/duplicate_hosts/) - [Datadog Agent Helm Chart Values Reference](https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml "https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml") - [Cluster Agent Commands and Options](https://docs.datadoghq.com/containers/cluster_agent/commands/#cluster-agent-options) - [Kubernetes Trace Collection](https://docs.datadoghq.com/containers/kubernetes/apm/?tab=helm) ### Datadog Private Locations (Optional) This component is the Datadog Helm chart for deploying synthetic private locations to EKS. This is useful when you want Datadog Synthetic Checks to be able to check the health of pods inside your cluster, which is private behind a VPC. This component is straight forward and requires little to no stack customization. Use the catalog entry included with the [datadog-synthetics-private-location documentation](https://docs.cloudposse.com/components/library/aws/datadog-synthetics-private-location) to get started. ## ECS ### ECS-Service This primary component should be familiar as it deploys your applications. It also has several variables with hooks to deploy the Datadog Agent as a sidecar container (useful for fargate). to get started simply add the following variables to your ECS Service: ```yaml datadog_agent_sidecar_enabled: true datadog_log_method_is_firelens: true datadog_logging_default_tags_enabled: true # in addition set your service logging method to awsfirelens containers: service: log_configuration: logDriver: awsfirelens options: {} ``` This will add The Datadog Agent sidecar to your service, add default tags, and add Firelens as the logging method which ships logs directly to Datadog. ### `datadog-private-location-ecs` This component deploys an ECS task that handles private locations for ECS. This is the counterpart to the Eks version. To get started simply vendor in and use the stack catalog entry in the readme. --- ## Datadog Log Filtering import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; This document presents an overview of options regarding filtering logs sent to Datadog. ### Datadog's recommended practice Datadog positions their capability as [Logging Without Limits™](https://www.datadoghq.com/blog/logging-without-limits). They charge only [$0.10 per GB of log data ingested](https://www.datadoghq.com/pricing/?product=log-management#products) and recommend ingesting all logs, so that data is available on demand. Their more significant charge is based on the number of log events retained and the duration of the retention period (approx $1-2.50 per million events, depending on retention period). **TL;DR** Ingest all logs, and use dynamic processing on the server side to determine which logs to process. ### Preserve All Data, Just in Case By sending all log data to Datadog, you get 2 major benefits. First, you can use their [Live Tail](https://docs.datadoghq.com/logs/explorer/live_tail/) feature to view the current log stream, after processing but before indexing, so that all log data, including data excluded from indexing by filters, is available for real-time troubleshooting. Second, all logs ingested can be saved, with tags to support later filtering, to your own S3 bucket at no additional charge from Datadog. These logs can later be re-ingested (["rehydrated"](https://docs.datadoghq.com/logs/log_configuration/rehydrating)) for detailed analysis of an event, creating an historical view for on-demand retrospective analysis. ### Save Money with Server-Side Filtering The primary log data-processing charge from Datadog is for indexing logs. By filtering logs out of indexing, you save on the (substantial) indexing and retention costs, but, as explained above, the logs remain available via your archive storage for later analysis if needed. An additional benefit of excluding logs via server-side filtering is that the filter can be easily modified or temporarily disabled during an incident to quickly provide additional information, and then restored when the incident is resolved. - For documentation, see [Log Configuration -> Indexes -> Exclusion filters](https://docs.datadoghq.com/logs/log_configuration/indexes/#exclusion-filters) #### Infrastructure as Code for Server-side Filtering Datadog has moderate support for configuring server-side filtering via Terraform. There is unexplored complexity in the fact that the order of pipelines can be significant, but there is limited support for querying or managing the automatically provisioned integration pipelines. This area requires further investigation. Meanwhile, configuration via the Datadog web UI is going to be the easiest way to begin in any case. ### Source Filtering If you are sure you want to filter logs out at the source and not send them to Datadog, you can add filters on any application, and on the log forwarder. Filters are limited to regular expression matches, and can either exclude matches or exclude non-matches (include matches). #### Application Log Filtering For Kubernetes pods, you can filter out logs [via annotations](https://docs.datadoghq.com/agent/logs/advanced_log_collection?tab=kubernetes). > To apply a specific configuration to a given container, Autodiscovery identifies containers by name, NOT image. It tries to match `` to `.spec.containers[0].name`, not `.spec.containers[0].image`. To configure using Autodiscovery to collect container logs on a given `` within your pod, add the following annotations to your pod’s log_processing_rules: ```yaml apiVersion: apps/v1 kind: ReplicaSet metadata: name: cardpayment spec: selector: matchLabels: app: cardpayment template: metadata: annotations: ad.datadoghq.com/.logs: >- [{ "source": "java", "service": "cardpayment", "log_processing_rules": [{ "type": "exclude_at_match", "name": "exclude_datadoghq_users", "pattern": "\\w+@datadoghq.com" }] }] labels: app: cardpayment name: cardpayment spec: containers: - name: '' image: cardpayment:latest ``` #### Filtering Logs via the Datadog Agent The Datadog Agent can be configured with processing rules as well, via the `DD_LOGS_CONFIG_PROCESSING_RULES` environment variable. Unlike the pod annotation, which only applies to the specified Docker container, rules configured at the Datadog Agent apply to all services for which the agent forwards logs. Example (untested) entry in `values.yaml` (not stacks): ```yaml datadog: envDict: DD_LOGS_CONFIG_PROCESSING_RULES: >- [{ "type": "exclude_at_match", "name": "exclude_datadoghq_users", "pattern": "\\w+@datadoghq.com" }] ``` #### Filtering Logs via the Log Forwarder In a similar fashion, the Lambda Log Forwarder can be configured to filter out logs via the `EXCLUDE_AT_MATCH` and `INCLUDE_AT_MATCH` environment variables. Unlike the other options which allow you to provide a list of rules, the Lambda only accepts a single regular expression. Also unlike the other options, backslashes do not need to be escaped in the regex string in the environment variable. Example (untested) in the stack for `datadog-lambda-forwarder`: ```yaml vars: datadog_forwarder_lambda_environment_variables: EXCLUDE_AT_MATCH: '\w+@datadoghq.com' ``` ### Custom and Pre-defined Log Enhancement via Pipelines Datadog supports data transformation [pipelines](https://docs.datadoghq.com/logs/log_configuration/pipelines) to transform and filter logs on the server side. They provide [numerous pre-defined pipelines](https://app.datadoghq.com/logs/pipelines/pipeline/library) (_Datadog login required_) and allow you to create your own as well. Your pipelines can extract standard or custom fields, and generate custom metrics. Custom metrics generated by logs are unaffected by index filtering. While not required, this can be used to create additional attributes used in deciding whether or not to index a log entry or not. ### Additional Resources - [Log Configuration](https://docs.datadoghq.com/logs/log_configuration/) documentation covers pipelines, processors, log parsers, attributes and aliasing, indexing, archiving, and generating custom metrics. - Datadog offers free online training courses. [Going Deeper with Logs Processing](https://learn.datadoghq.com/courses/going-deeper-with-logs-processing) covers pipelines, processors, log parsers, and standard attributes for log processing. --- ## How to create a Synthetic and SLO import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem After adding a service you often have synthetic checks to determine its health and check KPIs. We also might have a business need for a specific uptime or value of a KPI. In order to do that we need to create synthetics and SLOs. ## Solution :::tip [datadog-synthetics](/components/library/aws/datadog-synthetics/) and datadog-slo Components make it easy to deploy new checks via yaml. ::: We can use the [datadog-synthetics](/components/library/aws/datadog-synthetics/) and datadog-slo components to deploy a synthetic and a SLO (or many) for your new service. To create a new Synthetic or SLO, determine which environments it should be deployed to and add the yaml definition for the test. Refer to the component documentation on how to setup and write the yaml configuration. ### Synthetics Datadog has 2 types of Synthetic Checks `API` or `browser`, decide which type you need to test your application using this documentation [https://docs.datadoghq.com/synthetics/](https://docs.datadoghq.com/synthetics/). ### SLOs SLOs should be “monitors” of business impacting KPIs or Events. In datadog they are defined through metrics or a collection of monitors. If you wish to make an SLO of your Synthetic Checks Datadog exposes a metric `synthetics.test_runs{*}.as_count()` which you can use to check the success rate of synthetic checks. ## References Datadog SLO Monitor Documentation: [https://docs.datadoghq.com/monitors/service_level_objectives/monitor/#overview](https://docs.datadoghq.com/monitors/service_level_objectives/monitor/#overview) Datadog SLO Metric Documentation: [https://docs.datadoghq.com/monitors/service_level_objectives/metric/#overview](https://docs.datadoghq.com/monitors/service_level_objectives/metric/#overview) Datadog Synthetic Documentation: [https://docs.datadoghq.com/synthetics/](https://docs.datadoghq.com/synthetics/) --- ## How to Monitor a new Service import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem When onboarding a new service we need some monitors to ensure its in a healthy status. ## Solution :::tip **TL;DR** Use Cluster Checks or Datadog Synthetics ::: Depending on how the service is exposed we can use cluster checks or synthetics. If a simple check will suffice we recommend using Cluster Checks: - [How to Setup Datadog Cluster Checks and Network Monitors for External URLs of Applications](/layers/monitoring/datadog/tutorials/how-to-setup-datadog-cluster-checks-and-network-monitors-for-ext) - [Datadog Cluster Checks](/components/library/aws/eks/datadog-agent/#adding-cluster-checks) However if multiple steps (such as login) is required we recommend Datadog Synthetics: - [How to create a Synthetic and SLO](/layers/monitoring/datadog/tutorials/how-to-create-a-synthetic-and-slo) These Checks are in addition to the default kubernetes checks we have that monitor for crashing pods, imagepullbackoff and other generic kubernetes issues. ## Next Steps After setting up health check monitors we should decide if we need APM metrics for this service. [https://docs.datadoghq.com/tracing/#send-traces-to-datadog](https://docs.datadoghq.com/tracing/#send-traces-to-datadog) --- ## How to Pass Tags Along to Datadog import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Tags are a great approach to describing who owns what services, and who should respond to different incidents related to those services. In order for to be able to act on the information, we need to ensure it’s passed along to datadog. Depending on if something is running on Kubernetes or infrastructure like RDS or an ALB, the way we tag (or label) things will differ. ## Solution :::tip **TL;DR** **AWS** The Datadog integration for AWS allows us to specify tags we apply to the integration, since this is per account we can ensure tags are applied for everything imported from that integration. **Kubernetes** The datadog agent has configuration that allows us to map labels or annotations to specific datadog tags. Those labels can even be set on the namespace so that they apply to all services within the namespace. ::: There are two main ways that events get generated: 1. AWS 2. Kubernetes Clusters ### AWS With several different AWS accounts, we want to make sure that every resource monitored through datalog has the right tags per account, such as `stage`, `environment`, and possibly `tenant`. Default tags fetched are found [here](https://docs.datadoghq.com/integrations/amazon_web_services/?tab=roledelegation#tags). Our [datadog-integration](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/datadog-integration) component also provides a variable `host_tags` which allows us to specify additional tags per account to help ensure all tags are assigned, such as `tenant`, `stage`, and `environment`. E.g. ``` components: terraform: datadog-integration: [...] vars: host_tags: - "stage:dev" - "tenant:platform" ``` ### Kubernetes With Kubernetes, we need to make sure that the right tags are fetched from either the namespace it was deployed or the labels attached to individual pods. As we may want one app to be monitored by team-a and another app in the same cluster to be monitored by team-b. By default, the datadog agent does not map kubernetes labels and annotations to datadog tags. We recommend setting default tags that you can add to your apps to map to datadog tags. this allows either namespace bound tags or app-specific tags to be set. These [values](https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml#L147-L167) in the datadog helm chart can be set to create a mapping to your datadog account. Take a look at Datadogs documentation for further details: [https://docs.datadoghq.com/agent/kubernetes/tag/?tab=containerizedagent](https://docs.datadoghq.com/agent/kubernetes/tag/?tab=containerizedagent) . --- ## How to Provision and Tune Datadog Monitors by Stage import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Datadog is a powerful platform with many ways of being configured. No one size fits all. Some companies choose to run with multiple datadog organizations, while others choose to consolidate in one account. Some companies want to adjust the thresholds at a service level, while others at the stage level. Multiple configurations exist and choosing the right one depends on what you want to accomplish. ## Solution :::tip Provision monitors by layer and vary the configurations by stage. ::: Monitoring happens at every layer of your infrastructure. We should strive to push as many monitors as possible to the lower levels so that the benefits are realized by all higher layers. The manner of monitoring each layer varies. For example, the _Infrastructure_ layer is best monitored using the [datadog-integration](/components/library/aws/datadog-integration/) provisioned per AWS account. While the _Application_ layer may be better suited by provisioning monitors defined within the application repo itself or using custom resources to manage the monitors via Kubernetes. ### Application Monitors Application monitors should be provisioned to monitor anything not caught by the underlying layers (e.g. application-specific behavior). See [How to Use Multiple Infrastructure Repositories with Spacelift?](/resources/deprecated/spacelift/tutorials/how-to-use-multiple-infrastructure-repositories-with-spacelift) for one approach to manage monitors using terraform. See [https://github.com/FairwindsOps/astro](https://github.com/FairwindsOps/astro)by Fairwinds, for a Kubernetes approach using an Operator and Custom Resources. Note, the Cloud Posse YAML format for monitors provisioned with terraform was directly inspired by `astro` and share almost the same schema. [https://github.com/FairwindsOps/astro/blob/master/conf-example.yml](https://github.com/FairwindsOps/astro/blob/master/conf-example.yml) ### Platform Monitors by Stage In general, we should strive for delivering platform-level monitors that apply to all services operating on the platform, rather than one-off monitors for individual applications. Of course, there will always be exceptions - just use this as a guideline. Here’s an example of setting up alerts for a production-tier and non-production-tier. ```yaml components: terraform: datadog-monitor-nonprod: component: datadog-monitor # Use the shared base component for `datadog-monitor` settings: spacelift: workspace_enabled: true vars: secrets_store_type: SSM datadog_api_secret_key: datadog/dev/datadog_api_key datadog_app_secret_key: datadog/dev/datadog_app_key datadog_monitors_config_paths: - catalog/monitors/nonprod/*.yaml # Specify the path to the configs relative to the base component. datadog_synthetics_config_paths: [] datadog-monitor-prod: component: datadog-monitor settings: spacelift: workspace_enabled: true vars: secrets_store_type: SSM datadog_api_secret_key: datadog/prod/datadog_api_key datadog_app_secret_key: datadog/prod/datadog_app_key datadog_monitors_config_paths: - catalog/monitors/prod/*.yaml datadog_synthetics_config_paths: [] ``` --- ## How to Setup Datadog Cluster Checks and Network Monitors for External URLs of Applications import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; ## Problem We often want a lightweight way to ensure our endpoints remain healthy. In kubernetes this requires that a loadbalancer be setup with the right annotations and certs to allow traffic to your application. This creates dependencies on cert-manager and other platform tools. The Health Check of a kubernetes app will always use the local IP, which doesn’t really test your networking. We need a way to test that your apps are still ready to receive requests. ## Solution Use **Cluster Network Checks** to test your endpoints. These will test external URLs which helps ensure endpoints are healthy. Cluster Checks are configured on the datadog agent, by specifying agent configuration you can set these checks to run once per cluster instead of once per node agent. These Checks test the validity of externally accessible URLs hosted in Kubernetes. To get started follow this guide: [https://docs.datadoghq.com/agent/cluster_agent/clusterchecks/?tab=helm](https://docs.datadoghq.com/agent/cluster_agent/clusterchecks/?tab=helm) For helm that means ensuring the following values are set ```yaml datadog: clusterChecks: enabled: true # (...) clusterAgent: enabled: true ``` ### External URLs We then need to set particular helm values for each installation of the agent in each cluster. the external URL checks must be written into the agent configuration and cannot be dynamically loaded by annotations. CloudPosse Datadog Agent now supports **Cluster Checks** this via `this pr` Upgrade to the latest and add your network checks as yaml. This follows the same configuration as monitors. Where checks are deep merged and templated, they can be configured per environment. ## Datadog Monitors After your Cluster Checks are setup we need to create monitors for them. :::info Http check will verify successful HTTP checks on a URL SSL check will verify the certificate of your URL ::: ```yaml https-checks: name: "(Network Check) ${stage} - HTTPS Check" type: service check query: | "http.can_connect".over("stage:${stage}").by("instance").last(2).count_by_status() message: | HTTPS Check failed on {{instance.name}} in Stage: {{stage.name}} escalation_message: "" tags: managed-by: Terraform notify_no_data: false notify_audit: false require_full_window: true enable_logs_sample: false force_delete: true include_tags: true locked: false renotify_interval: 0 timeout_h: 0 evaluation_delay: 0 new_host_delay: 0 new_group_delay: 0 no_data_timeframe: 2 threshold_windows: { } thresholds: critical: 1 warning: 1 ok: 1 ``` --- ## How to Sign Up for Datadog? import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem You’ve been asked to sign up for DataDog, but you're not quite sure which plans and features you need. ## Solution Review the [https://www.datadoghq.com/pricing/](https://www.datadoghq.com/pricing/) :::tip We recommend the Enterprise Plan if you will need Custom Roles, otherwise the Pro Plan should suffice. ::: After signing up, if you require child organizations, make sure to request it be enabled on your account by your Datadog Account Manager. [https://docs.datadoghq.com/account_management/multi_organization/](https://docs.datadoghq.com/account_management/multi_organization/) (see [Decide on How to Restrict Access to Metrics and Logs in Datadog](/layers/monitoring/design-decisions/decide-on-how-to-restrict-access-to-metrics-and-logs-in-datadog)) ### Recommended Features - [https://www.datadoghq.com/blog/private-synthetic-monitoring/](https://www.datadoghq.com/blog/private-synthetic-monitoring/) - [https://docs.datadoghq.com/logs/](https://docs.datadoghq.com/logs/) - [https://docs.datadoghq.com/monitors/service_level_objectives/monitor/](https://docs.datadoghq.com/monitors/service_level_objectives/monitor/) - [https://docs.datadoghq.com/tracing/](https://docs.datadoghq.com/tracing/) - [https://docs.datadoghq.com/synthetics/](https://docs.datadoghq.com/synthetics/) - [https://docs.datadoghq.com/real_user_monitoring/](https://docs.datadoghq.com/real_user_monitoring/) - [https://docs.datadoghq.com/database_monitoring/](https://docs.datadoghq.com/database_monitoring/) ### Pricing Gotchas - Commit to a certain number of hosts per month for cheaper pricing - If using spot instances, you may be charged for multiple hosts. For instance, you may use a spot instance for an EKS node, the node is replaced with another spot instance, and Datadog now charges for 2 hosts instead of 1 host for that month. ### Enterprise Plan Many features are restricted to Enterprise Plans. :::caution Custom Roles are an enterprise only feature. [https://docs.datadoghq.com/account_management/rbac/?tab=datadogapplication#custom-roles](https://docs.datadoghq.com/account_management/rbac/?tab=datadogapplication#custom-roles) ::: - Custom Roles - Watchdog: Automated insights - Correlations - Anomaly Detection - Forecast Monitoring - Live Processes - Advanced Administrative Tools ### Pro Plan The Pro Plan is the minimum acceptable plan that adds Alerts, Container Monitoring, SAML and Custom Metrics. - Unlimited Alerts - Unlimited Container Monitoring - 10 per host included - 20 per host included. Customizable. - Custom Metrics - Single Sign-On with SAML - Outlier Detection ## Related Design Decisions - [Decide on External Monitoring Solution](/layers/monitoring/design-decisions/decide-on-external-monitoring-solution) - [Decide on Log Retention and Durability Architecture](/layers/security-and-compliance/design-decisions/decide-on-log-retention-and-durability-architecture) - [Decide on How to Restrict Access to Metrics and Logs in Datadog](/layers/monitoring/design-decisions/decide-on-how-to-restrict-access-to-metrics-and-logs-in-datadog) --- ## How to use Datadog Metrics for Horizontal Pod Autoscaling (HPA) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem The native Kubernetes Horizontal Pod Autoscaler has only the ability to scale pods horizontally based on primitive metrics such as CPU and Memory. Your application is more complex and needs to scale based on other dimensions of data, such as queue depth. You have the data, but it’s in Datadog and you need some way for Kubernetes to scale based on complex insights. ## Solution :::tip **TL;DR**: Ensure the Datadog Cluster Agent Metrics Server is enabled, enable Kubernetes Integrations Autodiscovery as needed, then create `HorizontalPodAutoscaler` manifests referencing `DatadogMetric` objects or the Datadog query directly. ::: The Datadog Cluster Agent has a metrics server feature which, as of Kubernetes v1.10, can be used for Horizontal Pod Autoscaling. This means that metrics automatically collected by Datadog Cluster Agent can be leveraged in HorizontalPodAutoscaler k8s objects. Take for example the following HorizontalPodAutoscaler manifest, which leverages the `nginx.net.request_per_s` metric automatically collected by Datadog Cluster Agent: ```yaml apiVersion: autoscaling/v2beta1 kind: HorizontalPodAutoscaler metadata: name: nginxext spec: minReplicas: 1 maxReplicas: 5 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: nginx metrics: - type: External external: metricName: nginx.net.request_per_s metricSelector: matchLabels: kube_container_name: nginx targetAverageValue: 9 ``` This leverages the already-existing metrics collection plane for out-of-the-box Kubernetes metrics (see: [https://docs.datadoghq.com/agent/kubernetes/data_collected/](https://docs.datadoghq.com/agent/kubernetes/data_collected/) for the list of native Kubernetes metrics automatically collected by the Datadog Cluster Agent) This also opens up the possibility for Horizontal Pod Autoscaling capabilities for homegrown application Deployments, where metrics exported via Prometheus / OpenMetrics will be collected by the Datadog Cluster Agent and leveraged in HorizontalPodAutoscaler definitions. Lastly, Datadog metrics integrations exist for applications such as Redis, Nginx, which are auto-discoverable via annotations (see: [https://docs.datadoghq.com/getting_started/integrations/](https://docs.datadoghq.com/getting_started/integrations/) ) 1. Use `eks-datadog` component (already created by Cloud Posse) and override [https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml#L551](https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml#L551) to enable the metrics server ([https://github.com/DataDog/helm-charts/blob/dae884481c5b3c9b67fc8dbd69c944bf3ec955e9/charts/datadog/values.yaml#L1318](https://github.com/DataDog/helm-charts/blob/dae884481c5b3c9b67fc8dbd69c944bf3ec955e9/charts/datadog/values.yaml#L1318) must be set to true as well, which is already the case by default). Also, ensure Prometheus scraping is enabled ([https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml#L425](https://github.com/DataDog/helm-charts/blob/main/charts/datadog/values.yaml#L425) ) 2. For home grown applications, ensure Prometheus metrics are being exported by the application via an exported within the Kubernetes Pod. 3. For home grown applications, ensure the appropriate Datadog Prometheus / OpenMetrics collection configuration is present within the Pod annotations in the Deployment manifest ([https://docs.datadoghq.com/agent/kubernetes/prometheus/#simple-metric-collection](https://docs.datadoghq.com/agent/kubernetes/prometheus/#simple-metric-collection) ) 4. For off-the-shelf applications, configure the additional values required to insert the Datadog annotations into the Kubernetes manifests templated by the public Helm Chart — for example adding the Redis integration auto-discovery annotations into the Pod manifest ([https://github.com/bitnami/charts/blob/ba9f72954d8f21ff38018ef250477d159378e8f7/bitnami/redis/values.yaml#L275](https://github.com/bitnami/charts/blob/ba9f72954d8f21ff38018ef250477d159378e8f7/bitnami/redis/values.yaml#L275) ) 5. Optional but recommended: create `DatadogMetric` objects with the queries you will use to base HPA on. 6. Create appropriate `HorizontalPodAutoscaler` manifests referencing the `DatadogMetric` objects within the same namespace as the HPA (or alternatively the query directly, if not using `DatadogMetric` objects). ### References - [https://www.datadoghq.com/blog/autoscale-kubernetes-datadog/](https://www.datadoghq.com/blog/autoscale-kubernetes-datadog/) - [https://docs.datadoghq.com/agent/kubernetes/integrations/?tab=kubernetes#pagetitle](https://docs.datadoghq.com/agent/kubernetes/integrations/?tab=kubernetes#pagetitle) - [https://github.com/DataDog/helm-charts/tree/main/charts/datadog](https://github.com/DataDog/helm-charts/tree/main/charts/datadog) --- ## Tutorials(7) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with managing Datadog. --- ## Decide on Datadog Account Strategy import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Context and Problem Statement Datadog supports multiple organizations much like AWS (note Datadog calls them “organizations” instead of “accounts”). Using organizations is a way to tightly restrict access to monitoring data but introduces limitations. Managed Service Providers typically use this feature with customers who do not have access to each others’ data. For example, if you have a multi-tenant use case where you need to provide access to Datadog to your customers, then this may be the best way to go. In this model, users can be added to the parent organization and/or multiple child organizations and switch between them from the [user account settings menu](https://docs.datadoghq.com/account_management/#managing-your-organizations). The parent organization can view the usage (not the underlying metrics collected) of individual child organizations, allowing them to track trends in usage. Account settings, such as allow-listed IP addresses, are not inherited by child organizations from their parent organization. The Multi-organization Account feature is not enabled by default. Contact [Datadog support](https://docs.datadoghq.com/help/) to have it enabled. [https://docs.datadoghq.com/account_management/multi_organization/](https://docs.datadoghq.com/account_management/multi_organization/) ## Considered Options ### Option 1: Use Single Datadog Organization :::tip This is our recommended approach as it’s the easiest to implement, supports tracing across all your accounts and ensures you don’t need to switch organizations to view dashboards. ::: Cloud Posse will need access to your current production Datadog organization if we go this route. Also, see: [Decide on How to Restrict Access to Metrics and Logs in Datadog](/layers/monitoring/design-decisions/decide-on-how-to-restrict-access-to-metrics-and-logs-in-datadog) ### Option 2: Use Multiple Datadog Child Organizations :::danger We do not recommend this approach because you cannot do cross-account tracing. Datadog alert email notifications do not include the account information which is problematic when using multiple accounts. ::: :::caution Child organizations are an optional feature and has to be requested from Datadog support. ::: [Datadog supports organizations](https://docs.datadoghq.com/account_management/multi_organization/) the way AWS supports organizations of member accounts. When created, each child organization has a default API and app keys. The original organization can remain AS IS, untouched. For example, if you want to create a partition between your current environments and the new ones we’re provisioning, this would be the way to go. There is no way to aggregate metrics across organizational boundaries. #### Child org per AWS account Each AWS account gets its own Datadog child organization. e.g. ``` acme-plat-gbl-dev acme-plat-gbl-prod ``` #### Groups of child orgs (i.e. prod and non-prod) Singleton and dev/sandbox/staging AWS account can be placed under `non-prod` child org. Prod would be placed under `prod` child org. e.g. ``` acme-plat-prod acme-plat-nonprod ``` Or perhaps a different grouping? ## Other Considerations ### Key storage #### Option 1: Shared in Automation Account (Recommended) In this model, we store the keys in a shared account (e.g. automation) using SSM parameterized. For a single organization, the SSM parameters might look like this: ``` /datadog/DD_API_KEY /datadog/DD_APP_KEY ``` Or, for a multi-child organization, each child org would be differentiated by the SSM parameter. ``` /datadog//datadog_api_key /datadog//datadog_app_key ``` #### Option 2: Shared in Each Respective Child Organization In each AWS account, respective child org creds can be stored like this: ``` /datadog/datadog_api_key /datadog/datadog_app_key ``` The reason to copy the keys into each AWS account from the shared account is limited access to the SSM cross-account. Various services will only allow access to SSM in the same as account as the service e.g. ECS task definitions. ### Authentication #### Option 1: Use SAML This is the recommended long-term solution. #### Option 2: Invite users This is the fastest way to get up and running. SAML can always be added later if it’s needed. ## References - [Decide on How to Restrict Access to Metrics and Logs in Datadog](/layers/monitoring/design-decisions/decide-on-how-to-restrict-access-to-metrics-and-logs-in-datadog) - [https://docs.datadoghq.com/account_management/multi_organization/](https://docs.datadoghq.com/account_management/multi_organization/) --- ## Decide on Datadog Log Forwarding Requirements import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Context and Problem Statement Datadog supports log ingestion, but [it can be costly.](https://www.datadoghq.com/pricing/?product=log-management#log-management) Some companies prefer to use in-place tooling like Splunk or Sumologic instead. ## Considered Options ### Option 1 (Recommended) - Use Datadog :::tip Our Recommendation is to use Option 1 because you get a single pane of glass view into all operations ::: ### Option 2 - Other #### Pros - Tightly integrated with your existing systems - Possible lower cost to operate that Datadog #### Cons - We cannot assist with the implementation aside from forwarding logs using something like `fluentd` or `fluentbit` --- ## Decide on Datadog Private Locations import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Datadog Private Locations are a feature that needs to be enabled in your datadog account that allows monitoring applications that aren’t accessible to the open internet. To enable private locations, we need to: - Enable the feature in each datadog account. - Deploy the Private Location Docker Image. We can deploy the private location container to EKS. This leads to another decision: Do we deploy it once with the capability to ping the rest of the clusters' internal addresses, or do we deploy it to every cluster? ## Cost According to [https://www.datadoghq.com/pricing/?product=synthetic-monitoring#synthetic-monitoring-is-there-any-extra-charge-for-using-private-locations](https://www.datadoghq.com/pricing/?product=synthetic-monitoring#synthetic-monitoring-is-there-any-extra-charge-for-using-private-locations), registering a new Private Location is not at an additional cost; the regular costs for synthetics still apply. > [**Is there any extra charge for using private locations?**](https://www.datadoghq.com/pricing/?product=synthetic-monitoring#synthetic-monitoring-is-there-any-extra-charge-for-using-private-locations) > No. There are no additional costs to set up a private location. All test runs to a private location are billed just as they are to a managed location. ## Solution ### Option 1: Enable Private Locations, and deploy to every cluster (Recommended) :::tip Our Recommendation is to use Option 1 because it enables private location features and provides a consistent way to scale. ::: :heavy_plus_sign: Private Location Monitoring :heavy_plus_sign: One helm chart per cluster via a component installation :heavy_minus_sign: Must run an additional container per cluster ### Option 2: Don’t Use Private Locations :heavy_plus_sign: Ever So Slightly Cheaper (We don’t run the container) :heavy_minus_sign: Monitoring Only Publicly accessible services ## References - [https://www.datadoghq.com/pricing/?product=synthetic-monitoring#synthetic-monitoring-is-there-any-extra-charge-for-using-private-locations](https://www.datadoghq.com/pricing/?product=synthetic-monitoring#synthetic-monitoring-is-there-any-extra-charge-for-using-private-locations) - [https://docs.datadoghq.com/synthetics/private_locations/?tab=helmchart#overview](https://docs.datadoghq.com/synthetics/private_locations/?tab=helmchart#overview) --- ## Decide on External Monitoring Solution import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem We’ll want some sort of external monitoring solution, ideally provided by an external third-party SaaS. ## Solution Use a third-party monitoring solution, ideally one that supports more advanced synthetics as well as behind-the-firewall monitoring checks. :::tip We recommend you stick with the synthetic monitoring provided by Datadog which supports both public and private endpoints via the private locations. ::: | **Service** | **Pricing Page** | | -------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | **Pingdom** | [https://www.pingdom.com/pricing/](https://www.pingdom.com/pricing/) | | **UptimeRobot** | [https://uptimerobot.com/pricing/](https://uptimerobot.com/pricing/) | | **NewRelic Synthetic Monitors** | [https://docs.newrelic.com/docs/synthetics/synthetic-monitoring/using-monitors/intro-synthetic-monitoring/#types-of-synthetic-monitors](https://docs.newrelic.com/docs/synthetics/synthetic-monitoring/using-monitors/intro-synthetic-monitoring/#types-of-synthetic-monitors) | | **StatusCake** | [https://www.statuscake.com/pricing/](https://www.statuscake.com/pricing/) | | **Datadog Synthetic Monitoring** | [https://www.datadoghq.com/product/synthetic-monitoring/](https://www.datadoghq.com/product/synthetic-monitoring/) [https://docs.datadoghq.com/getting_started/synthetics/private_location/](https://docs.datadoghq.com/getting_started/synthetics/private_location/) (We have full support for this in our [https://github.com/cloudposse/terraform-datadog-platform](https://github.com/cloudposse/terraform-datadog-platform) module) | :::caution Datadog Synthetics can get very pricey ::: --- ## Decide on How to Restrict Access to Metrics and Logs in Datadog import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Restricting access to metrics and logs concerns organizations subject to benchmark compliance. There are a few ways this can be done with various tradeoffs. ## Solution ### Option 1: RBAC With RBAC, Roles can be used to categorize users and define what account permissions those users have (read, modify) on resources. Any user who is associated with one or more roles receives all permissions granted by their associated roles. The more roles a user is associated with, the more access they have within a Datadog account. [https://docs.datadoghq.com/account_management/rbac/permissions/](https://docs.datadoghq.com/account_management/rbac/permissions/) #### Built-in Roles (Recommended) By default, Datadog offers three roles, - Datadog Admin - Datadog Standard - Datadog Read-Only #### Custom Roles You can create [custom roles](https://docs.datadoghq.com/account_management/rbac/?tab=datadogapplication#custom-roles) to define a better mapping between your users and their permissions. :::note If you use a SAML identity provider, you can integrate it with Datadog for authentication, and you can map identity attributes to Datadog default and custom roles. For more information, see [Single Sign On With SAML](https://docs.datadoghq.com/account_management/saml/). ::: :::caution Creating and modifying custom roles is an **opt-in** Enterprise feature. Contact Datadog support to get it enabled for your account. ::: [https://docs.datadoghq.com/account_management/rbac/?tab=datadogapplication](https://docs.datadoghq.com/account_management/rbac/?tab=datadogapplication) ### Option 2: Datadog Child Organizations :::danger We do not recommend this approach because you cannot do cross-account tracing. Datadog alert email notifications do not include the account information which is problematic when using multiple accounts. ::: See [Decide on Datadog Account Strategy](/layers/monitoring/design-decisions/decide-on-datadog-account-strategy) ## References - [https://docs.datadoghq.com/account_management/rbac/permissions/](https://docs.datadoghq.com/account_management/rbac/permissions/) --- ## Decide on whether to use Datadog roles import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Roles can be used to restrict access to dashboards and monitors. ## Solution ### Option 1: Default Roles (Recommended) :::tip Stick with the Default roles unless you need the granularity ::: Datadog ships with the standard roles of admin, standard, and read only ### Option 2: Custom Roles Custom roles give you the ability to define a more granular access model. To enable custom roles, datadog support must be contacted. ## References - https://docs.datadoghq.com/account_management/rbac/ --- ## Design Decisions(5) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for how you'll gather telemetry and logs for your applications. --- ## Monitoring FAQ ### How do I add a new monitor? The easiest way to get started with an IaC monitor is to create it by hand in Datadog! While this may seem counterintuitive, seeing live graphs of your data and being able to view available metrics makes it much easier to figure out what is really important to look at. Once you have a monitor configured in Datadog, you can export it to JSON, convert it to YAML, and then add it to your catalog of monitors. Don't forget to replace hardcoded variables with Terraform variables or Datadog variables, as both interpolations will work. ### What is Datadog Interpolation vs Terraform Interpolation? When looking at a Datadog monitor, anything with `${foo}` is Terraform interpolation; this will be substituted before being sent to Datadog. `{{bar}}` is Datadog interpolation. This will be used by datadog when **the event comes in**. This is useful for things like tags where you want to tag the event with the name of the cluster, but you don't know the name of the cluster when you create the monitor. ### I'm not receiving the metrics I need, what do I do? First off, we need to figure out where **should** the metrics be coming from. If you are trying to receive metrics or logs from an EKS deployment or service, check if the [`datadog-agent`](/components/library/aws/eks/datadog-agent/) is deployed and its logs look healthy. If you are trying to receive metrics or logs from an ECS service, check the datadog agent sidecar container is deployed to your [`ecs-service`](/components/library/aws/ecs-service/). If you are trying to receive metrics from an AWS Service, first check if the [Datadog AWS Integration](https://app.datadoghq.com/integrations/amazon-web-services) is deployed via [`datadog-integration`](/components/library/aws/datadog-integration/) and that the tile is working. Then check if the Datadog AWS Integration is enabled for the service you are trying to monitor. You can often find the metric is under the integration tiles' **metrics** tab. If it is enabled and working and you are not receiving metrics, check the integration role has the right permissions. If you are trying to receive logs from an AWS Service, check the [`datadog-lambda-forwarder`](/components/library/aws/datadog-lambda-forwarder/) is deployed and working --- ## How to Setup Amazon Managed Grafana import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; The Amazon Managed Grafana is a fully managed service for Grafana used to query, visualize, and alert on your metrics, logs, and traces. Grafana provides a centralized dashboard where we can add many data sources. ## Deployment ### Collecting Metrics Once Prometheus is fully functional on its own, then we add the HTTP endpoint for Prometheus as a data source for Amazon Managed Grafana, where we can centralize, visualize, query, and alert on those metrics. The Prometheus workspace is fully managed by AWS so therefore is not deployed to an EKS cluster. Deploy the Amazon Managed Prometheus workspace with the `managed-prometheus/workspace` component to each platform account and/or any account where you'd like to collect metrics. Define a stack catalog as follows: ```yaml components: terraform: prometheus: metadata: component: managed-prometheus/workspace vars: enabled: true name: prometheus # Create cross-account role for core-auto to access AMP grafana_account_name: core-auto ``` Then import this stack catalog file anywhere you want to deploy Prometheus. For example, all platform accounts. Then deploy the workspace into each stack: ```console atmos terraform apply prometheus -s plat-use2-sandbox atmos terraform apply prometheus -s plat-use2-dev atmos terraform apply prometheus -s plat-use2-staging atmos terraform apply prometheus -s plat-use2-prod ``` Once you have the workspace provisioned, then add a collector. There are a number of collectors that can be set up with Prometheus, but we primarily use the Amazon managed collector for EKS, commonly referred to as a "scraper". The scraper is deployed alongside an EKS cluster and is granted permission to read metrics for that EKS cluster. That scraper then forwards logs to Amazon Managed Prometheus. Deploy the managed collected with the `eks/prometheus-scraper` component to any account with Prometheus where you'd like to collect metrics from EKS. Define a stack catalog as follows: ```yaml components: terraform: eks/prometheus-scraper: vars: enabled: true name: prometheus-scraper prometheus_component_name: prometheus ``` Then import this stack catalog file anywhere you want to deploy Prometheus. For example, all platform accounts. Then deploy the workspace into each stack: ```console atmos terraform apply eks/prometheus-scraper -s plat-use2-sandbox atmos terraform apply eks/prometheus-scraper -s plat-use2-dev atmos terraform apply eks/prometheus-scraper -s plat-use2-staging atmos terraform apply eks/prometheus-scraper -s plat-use2-prod ``` Finally after the scraper is deployed, we have to finish the Cluster Role Binding configuration with the EKS cluster's auth map. Note the `scraper_role_arn` and `clusterrole_username` outputs from the `eks/prometheus-scraper` component and set them to `rolearn` and `username` respectively with the `map_additional_iam_roles` input for `eks/cluster`. ```yaml components: terraform: eks/cluster: vars: map_additional_iam_roles: # this role is used to grant the Prometheus scraper access to this cluster. See eks/prometheus-scraper - rolearn: "arn:aws:iam::111111111111:role/AWSServiceRoleForAmazonPrometheusScraper_111111111111111" username: "acme-plat-ue2-sandbox-prometheus-scraper" groups: [] ``` Then reapply each given cluster component. ### Scraping Logs Logs are collected with Loki and Promtail by Grafana. Grafana Loki is a set of resources that can be combined into a fully featured logging stack. Unlike other logging systems, Loki is built around the idea of only indexing metadata about your logs: labels (just like Prometheus labels). Log data itself is then compressed and stored in chunks in object stores such as S3 or GCS, or even locally on a filesystem. Whereas Promtail is an agent which ships the contents of local logs to Loki. Promtail scrapers logs from an EKS cluster, and can be enabled to receive logs on its own via an API server. Both Loki and Promtail are deployed to EKS via Helm charts. Deploy these with the `eks/loki` and `eks/promtail` components respectively. First deploy `eks/loki`. Add the `eks/loki` component and stack catalog as such: ::::tip Internal ALBs We recommend using an internal ALB for logging services. You must connect to the private network to access the Loki endpoint. :::: ```yaml components: terraform: eks/loki: vars: enabled: true name: loki alb_controller_ingress_group_component_name: eks/alb-controller-ingress-group/internal ``` Then deploy the `eks/promtail` component with an example stack catalog as follows: ```yaml components: terraform: eks/promtail: vars: enabled: true name: promtail ``` Import both into any account where you have an EKS cluster, and deploy then in order. For example `plat-use2-dev`. ```console atmos terraform apply eks/loki -s plat-use2-dev atmos terraform apply eks/promtail -s plat-use2-dev ``` ### Amazon Managed Grafana Workspace Now that we have metrics and logs collected in each platform account, we want to create a central "hub" for accessing that data. That hub is Grafana. The primary component of Amazon Managed Grafana is the workspace. The Amazon Managed Grafana workspace is the logically isolated Grafana server, where we can create Grafana dashboards and visualizations to analyze your metrics, logs, and traces without having to build, package, or deploy any hardware to run your Grafana servers. Deploy the centralized Amazon Managed Grafana workspace to `core-auto` with the `managed-grafana/workspace` component. For example ```yaml components: terraform: grafana: metadata: component: managed-grafana/workspace vars: enabled: true name: grafana private_network_access_enabled: true sso_role_associations: - role: "ADMIN" group_ids: - "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # This grafana workspace will be allowed to assume the cross # account access role from these prometheus components. # Add all plat accounts after deploying prometheus in those accounts prometheus_source_accounts: - component: prometheus tenant: plat stage: sandbox - component: prometheus tenant: plat stage: dev ``` Import this component in `core-use2-auto` (your primary region), and then deploy this component with the following: ```bash atmos terraform apply grafana -s core-use2-auto ``` ### Managing Terraform We have fully Terraformed this Grafana-based monitoring system using the [Grafana Terraform Provider](https://registry.terraform.io/providers/grafana/grafana/latest). We deploy an API Key after creating the workspace and then use that API key to create all necessary Grafana sub components, including all data sources and dashboards. Create that API key with the `managed-grafana/api-key` component. ```yaml components: terraform: grafana/api-key metadata: component: managed-grafana/api-key vars: enabled: true grafana_component_name: grafana ``` Then deploy it in the same account as the Grafana workspace. ```console atmos terraform apply grafana/api-key -s core-use2-auto ``` ::::info API Key Rotation By default, this Grafana API key will expire after 30 days (max). The component is configured to automatically suggest replacing API key after that expiration date, but Terraform will need to be reapplied to refresh that key. :::: Now other Grafana sub components will be able to pull that API key from AWS SSM and use it to access the Grafana workspace. ### Adding Data Sources In order to visualize and query metrics and logs, we need to add each as a data source for the centralized Amazon Managed Grafana workspace. We have created a data source component for each type. Use the `managed-grafana/data-source/managed-prometheus` component to add the Managed Prometheus workspace as a data source for Grafana. Add the following stack catalog: ```yaml components: terraform: grafana/datasource/defaults: metadata: component: managed-grafana/data-source/managed-prometheus type: abstract vars: enabled: true grafana_component_name: grafana grafana_api_key_component_name: grafana/api-key prometheus_component_name: prometheus grafana/datasource/plat-sandbox-prometheus: metadata: component: managed-grafana/data-source/managed-prometheus inherits: - grafana/datasource/defaults vars: name: plat-sandbox-prometheus prometheus_tenant_name: plat prometheus_stage_name: sandbox grafana/datasource/plat-dev-prometheus: metadata: component: managed-grafana/data-source/managed-prometheus inherits: - grafana/datasource/defaults vars: name: plat-dev-prometheus prometheus_tenant_name: plat prometheus_stage_name: dev # Plus all other Prometheus deployments ... ``` Then deploy the components into the same stack as Grafana. For example `core-use2-auto`: ``` atmos terraform apply grafana/datasource/plat-sandbox-prometheus -s core-use2-auto atmos terraform apply grafana/datasource/plat-dev-prometheus -s core-use2-auto atmos terraform apply grafana/datasource/plat-staging-prometheus -s core-use2-auto atmos terraform apply grafana/datasource/plat-prod-prometheus -s core-use2-auto ``` Use the `managed-grafana/data-source/loki` component to add Grafana Loki as a data source for Grafana. Add the following stack catalog to the same catalog you used for the Prometheus data sources. ```yaml components: terraform: ... # These use the same default data source component defined for the prometheus # data source components, since the inputs and structure are the mostly the same grafana/datasource/plat-sandbox-loki: metadata: component: managed-grafana/data-source/loki inherits: - grafana/datasource/defaults vars: name: plat-sandbox-loki loki_tenant_name: plat loki_stage_name: sandbox grafana/datasource/plat-dev-loki: metadata: component: managed-grafana/data-source/loki inherits: - grafana/datasource/defaults vars: name: plat-dev-loki loki_tenant_name: plat loki_stage_name: dev # Plus all other Loki deployments ... ``` Then deploy the components into the same stack as Grafana. For example `core-use2-auto`: ```console atmos terraform apply grafana/datasource/plat-sandbox-loki -s core-use2-auto atmos terraform apply grafana/datasource/plat-dev-loki -s core-use2-auto atmos terraform apply grafana/datasource/plat-staging-loki -s core-use2-auto atmos terraform apply grafana/datasource/plat-prod-loki -s core-use2-auto ``` ### Creating Dashboards We fully support Terraformed Grafana dashboards with the `managed-grafana/dashboard` component. Search the [Grafana Dashboard Library](https://grafana.com/grafana/dashboards/) to find the dashboards that best suite your requirements. Once you've found a dashboard, copy the dashboard URL from "Download JSON". Right click "Download JSON" and select "Copy Link Address". This is the dashboard URL we need. Now create a catalog entry. For example, see the stack catalog below where we create a dashboard _for each_ of our data sources defined earlier. When you import a dashboard in the Grafana UI, you can specify the dashboard inputs after importing. For these components, we instead specify the inputs that we want to replace before creating the dashboard. We do that with `var.config_input`. This map variable will take a specific string as the map key and replace all occurrences of that string with the given value. However to know what that input value is, you will need to open the dashboard JSON and find any value in `${ }` format; although these can usually be logically determined by the type of the data source prefixed with `DS_`. For example a Prometheus data source would likely be `${DS_PROMETHEUS}` and a Loki data source would likely be `${DS_LOKI}`. Be sure to include `${ }` in the map key; we want to replace it entirely in the rendered JSON. ```yaml components: terraform: grafana/dashboard/defaults: metadata: component: managed-grafana/dashboard type: abstract vars: enabled: true grafana_component_name: grafana grafana_api_key_component_name: grafana/api-key grafana/dashboard/plat-sandbox-prometheus: metadata: component: managed-grafana/dashboard inherits: - grafana/dashboard/defaults vars: dashboard_name: acme-plat-ue2-sandbox-prometheus dashboard_url: "https://grafana.com/api/dashboards/315/revisions/3/download" config_input: "${DS_PROMETHEUS}": "acme-plat-ue2-sandbox-prometheus" grafana/dashboard/plat-sandbox-loki: metadata: component: managed-grafana/dashboard inherits: - grafana/dashboard/defaults vars: dashboard_name: acme-plat-ue2-sandbox-loki dashboard_url: "https://grafana.com/api/dashboards/13639/revisions/2/download" config_input: "${DS_LOKI}": "acme-plat-ue2-sandbox-loki" grafana/dashboard/plat-dev-prometheus: metadata: component: managed-grafana/dashboard inherits: - grafana/dashboard/defaults vars: dashboard_name: acme-plat-ue2-dev-prometheus dashboard_url: "https://grafana.com/api/dashboards/315/revisions/3/download" config_input: "${DS_PROMETHEUS}": "acme-plat-ue2-dev-prometheus" grafana/dashboard/plat-dev-loki: metadata: component: managed-grafana/dashboard inherits: - grafana/dashboard/defaults vars: dashboard_name: acme-plat-ue2-dev-loki dashboard_url: "https://grafana.com/api/dashboards/13639/revisions/2/download" config_input: "${DS_LOKI}": "acme-plat-ue2-dev-loki" # Plus all other data sources in staging, prod, etc ... ``` Now import this stack file into the same stack as Grafana, for example `core-use2-auto`, and deploy those components: ```console atmos terraform apply grafana/dashboard/plat-sandbox-prometheus -s core-use2-auto atmos terraform apply grafana/dashboard/plat-sandbox-loki -s core-use2-auto atmos terraform apply grafana/dashboard/plat-dev-prometheus -s core-use2-auto atmos terraform apply grafana/dashboard/plat-dev-loki -s core-use2-auto ``` And that's it! Validate the set up in Grafana. Open the Grafana workspace, select the menu in the top left, click "Dashboards". Choose any of your newly deployed dashboards. ## References - [AWS Documentation on Managed Collectors for EKS](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-collector-how-to.html) - [AWS Documentation on Connecting Grafana Data sources via a private network (VPC)](https://docs.aws.amazon.com/grafana/latest/userguide/AMG-configure-vpc.html) - [AWS FAQ on using VPC with Amazon Managed Grafana](https://docs.aws.amazon.com/grafana/latest/userguide/AMG-configure-vpc-faq.html) - [Grafana Terraform Provider](https://registry.terraform.io/providers/grafana/grafana/latest) - [Grafana Loki Setup Docs](https://grafana.com/docs/loki/latest/setup/install/) - [Grafana Dashboard Library](https://grafana.com/grafana/dashboards/) --- ## Setup AWS Managed Grafana import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Admonition from '@theme/Admonition' import Note from '@site/src/components/Note' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Grafana is built around the Amazon managed services for Grafana and Prometheus. At this time, we have implemented only the EKS integrations with AWS Managed Grafana. We are open to adding ECS support. These instructions include the steps for integrated EKS with AWS Managed Grafana. ## Vendor Components Vendor all required components ## Deploy Grafana and the Prometheus Scraper Please see [How to Setup Grafana](/layers/monitoring/grafana/) for in depth documentation. However, if you can find the summary of steps in Atmos Workflows here. You can choose to run these workflows one-by-one, or run them altogether with the following: ## Grant the Prometheus Scraper Access to EKS After deploying the `eks/prometheus-scraper` component, you will need to reapply the `eks/cluster` component with an update to `var.map_additional_iam_roles`. ```console atmos terraform output eks/prometheus-scraper -s plat-use1-dev atmos terraform output eks/prometheus-scraper -s plat-use1-staging atmos terraform output eks/prometheus-scraper -s plat-use1-prod ``` Note the `scraper_role_arn` and `clusterrole_username` outputs and set them to `rolearn` and `username` respectively with the `map_additional_iam_roles` input for `eks/cluster`. ```yaml # stacks/orgs/acme/plat/STAGE/us-east-1/eks.yaml components: terraform: eks/cluster: vars: map_additional_iam_roles: # this role is used to grant the Prometheus scraper access to this cluster. See eks/prometheus-scraper - rolearn: "arn:aws:iam::111111111111:role/AWSServiceRoleForAmazonPrometheusScraper_111111111111111" username: "acme-plat-ue2-sandbox-prometheus-scraper" groups: [] ``` ## Reapply EKS Cluster Then reapply `eks/cluster`: ```console atmos terraform apply eks/cluster -s plat-use1-dev atmos terraform apply eks/cluster -s plat-use1-staging atmos terraform apply eks/cluster -s plat-use1-prod ``` ## Accessing Grafana We would prefer to have a custom URL for the provisioned Grafana workspace, but at the moment it's not supported natively and implementation would be non-trivial. We will continue to monitor that Issue and consider alternatives, such as using Cloudfront. [Issue #6: Support for Custom Domains](https://github.com/aws/amazon-managed-grafana-roadmap/issues/6) You can access Grafana from the Grafana workspace endpoint that is output by the `grafana` component: ```console atmos terraform output grafana -s core-use1-auto ``` Or you can open your AWS Single-Sign-On page, navigate to the "Applications" tab, and then select "Amazon Grafana". [https://d-1111aa1a11.awsapps.com/start/](https://d-1111aa1a11.awsapps.com/start/) --- ## Implement Telemetry import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from 'react-player'; import CategoryList from '@site/src/components/CategoryList'; Monitoring is a key component of any production system. It is important to have visibility into the health of your system and to be able to react to issues before they become problems.
AI generated voice
## The Problem Monitoring is a difficult problem to solve. There are many different tools and services that can be used to monitor your system. It is important to have a consistent approach to monitoring that can be applied across all of your systems. There is often a tradeoff between the cost of monitoring and the value it provides. It is important to have a monitoring solution that is cost effective and provides value to your organization. Another problem is when monitoring is configured incorrectly and causes more problems than it solves, usually seen through ignored alerts or no alerts at all. ## Our Solution We have developed a set of Terraform modules that can be used to deploy a monitoring solution for your system. These modules are designed to be used with Datadog. Datadog is a monitoring service that provides a wide range of features and integrations with other services. We have broken down the monitoring solution into several components to make it easier to deploy and manage. ### Implementation #### Foundation - [`datadog-configuration`](/components/library/aws/datadog-credentials/): This is a **utility** component. This component expects Datadog API and APP keys to be stored in SSM or ASM, it then copies the keys to SSM/ASM of each account this component is deployed to. This is for several reasons: 1. Keys can be easily rotated from one place 2. Keys can be set for a group and then copied to all accounts in that group, meaning you could have a pair of api keys and app keys for production accounts and another set for non-production accounts. This component is **required** for all other components to work. As it also stores information about your Datadog account, which other components will use, such as your Datadog site url, along with providing an easy interface for other components to configure the Datadog provider. - [`datadog-integration`](/components/library/aws/datadog-integration/): This component is the core component binding Datadog to AWS, this component is deployed to every account and sets up all the Datadog Integration tiles with AWS. This is what provides the majority of your metrics to AWS! - [`datadog-lambda-forwarder`](/components/library/aws/datadog-lambda-forwarder/): This component is an AWS Lambda function that ships logs from AWS to Datadog. Details of it can be found [here](https://docs.datadoghq.com/logs/guide/forwarder/?tab=terraform) - [`datadog-monitor`](/components/library/aws/datadog-monitor/): This component deploys monitors via yaml configuration. When you [vendor](https://atmos.tools/cli/commands/vendor/usage/#docusaurus_skipToContent_fallback) in this component you will find [our catalog of pre-built monitors](https://github.com/cloudposse/terraform-datadog-platform/tree/main/catalog/monitors). We deploy this component to every account, our monitors have Terraform interpolation to allow you to set the thresholds for each monitor. This allows you to set different thresholds per stage using the same monitors but different configurations using familiar atmos inheritance. #### EKS - [`datadog-agent`](/components/library/aws/eks/datadog-agent/): This component deploys the Datadog agent on EKS, it also deploys the [Datadog Cluster Agent](https://docs.datadoghq.com/agent/cluster_agent/), the agent is a daemonset that runs on every node in your cluster (with the exception of fargate (serverless) nodes). This component handles sending Kubernetes metrics, logs, and events to Datadog. This component also can deploy the [Datadog Cluster Checks](https://docs.datadoghq.com/containers/cluster_agent/clusterchecks/) which are a way to run checks on your cluster from within the cluster itself, this is often a cheaper way than [Synthetic Monitoring](https://docs.datadoghq.com/synthetics/) to monitor services in your cluster. - [`datadog-private-location-eks`](/components/library/aws/datadog-synthetics-private-location/): This component deploys a private location for [Synthetic Monitoring](https://docs.datadoghq.com/synthetics/) to your EKS cluster. This allows synthetic checks to run even inside a private cluster. #### ECS - [`ecs-service`](/components/library/aws/ecs-service/): This component contains variables that enable Datadog integration with ECS. For more information on how to deploy a service to ecs, see the [ecs-service](/components/library/aws/ecs-service/) component, specifically the [`datadog_agent_sidecar_enabled`](/components/library/aws/ecs-service/#input_datadog_agent_sidecar_enabled) variable. - [`datadog-private-location-ecs`](/components/library/aws/datadog-private-location-ecs/): This component deploys a private location for [Synthetic Monitoring](https://docs.datadoghq.com/synthetics/) to your ECS cluster. This allows synthetic checks to run against your ECS cluster. #### Additional - [`datadog-logs-archive`](/components/library/aws/datadog-logs-archive/): This component creates a single [log archive](https://docs.datadoghq.com/logs/log_configuration/archives/?tab=awss3) pipeline for each AWS account. Using this component you can setup [multiple logs archive rules](https://docs.datadoghq.com/logs/log_configuration/archives/?tab=awss3#multiple-archives). - [`datadog-synthetics`](/components/library/aws/datadog-synthetics/): This component deploys Datadog synthetic checks, which are external health checks for your services, similar to [Pingdom](https://www.pingdom.com/). ## References --- ## Accessing the Network import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; Lastly, configure the VPN. The VPN will be provisioned in the `network` account and will leverage Transit Gateway to connect various VPCs to the VPN client. VPN deployment consists of three parts: authentication, component deployment, and client setup. ## Set up authentication First, set up authentication. - We recommend [using AWS IAM Identity Center to authenticate users](https://aws.amazon.com/blogs/security/authenticate-aws-client-vpn-users-with-aws-single-sign-on/). - Follow only the first section included in the linked AWS blog, _Create and configure the Client VPN SAML applications in AWS IAM Identity Center_, through downloading the _AWS IAM Identity Center SAML metadata_. - Save that file under the `ec2-client-vpn` component (`components/terraform/ec2-client-vpn`) as "aws-sso-saml-app.xml". This should match the given document name for `saml_metadata_document` in the `ec2-client-vpn` stack catalog (`stacks/catalog/ec2-client-vpn.yaml`) ## Deploy the VPN Next, deploy the `ec2-client-vpn` component. This is done by running the following: Depending on the given network configuration, you may run out of available Client VPN routes. That error will look something like this: ```console ╷ │ Error: error creating EC2 Client VPN Route (cvpn-endpoint-0b7487fc0043a3df0,subnet-0b88f999578fd2340,10.101.96.0/19): ClientVpnRouteLimitExceeded: Limit exceeded │ status code: 400, request id: 779f977b-2b31-490a-a4b1-2c8cb1da068d │ │ with module.ec2_client_vpn.aws_ec2_client_vpn_route.default[40], │ on .terraform/modules/ec2_client_vpn/main.tf line 245, in resource "aws_ec2_client_vpn_route" "default": │ 245: resource "aws_ec2_client_vpn_route" "default" { │ ``` If this happens, you'll need to [increase the number of routes](https://console.aws.amazon.com/servicequotas/home/services/ec2/quotas/L-401D78F7) allowed for the Client VPN endpoint. That said, you should already have a quota increase request ready for this in `stacks/orgs/acme/core/network/global-region/baseline.yaml`. You can apply that quota using `atmos terraform apply account-quotas -s core-gbl-network`. ## Download & Install VPN Client - Finally, set up the AWS VPN Client to access the VPN. - [Download the AWS VPN Client](https://aws.amazon.com/vpn/client-vpn-download/) and or install it by running `brew install aws-client-vpn` in a regular terminal. Follow the [AWS Documentation](https://docs.aws.amazon.com/vpn/latest/clientvpn-user/connect-aws-client-vpn-connect.html) to complete the VPN setup. ## Configure VPN Client The Atmos Workflow `deploy/vpn` creates a local VPN configuration as `acme-core.ovpn` (`rootfs/etc/aws-config/acme-core.ovpn`) located in the aws-config dir of `rootfs/`. If it doesn't exist, create this file using the `client_configuration` output of the `ec2-client-vpn` component, and commit it to the repo under `rootfs/etc/aws-config/acme-core.ovpn` for future reference. ```shell atmos terraform output ec2-client-vpn -s core-use1-network ``` ## Connect to VPN Once you configure the AWS VPN Client, set the file as the config and connect. From there you should be able to access resources on any subnet in the VPCs you've provisioned. ### Optional: Bastion hosts If you'd like to set up bastion hosts, you can do so by running the following. This would let you further evaluate the VPN. By default, we deploy the bastion to all accounts connected to Transit Gateway. --- ## Establish Connectivity with Transit Gateway import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Deployment Next, set up AWS Transit Gateway (TGW) to connect each desired AWS account to the private network by running the following command: This command will deploy the TGW hub and spokes to connect the network to the AWS accounts. The TGW hub is deployed in the `core-network` account, and the TGW spokes are deployed in each AWS account with a VPC. --- ## Deploying the Network import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; The first step in deploying the network is to deploy the VPCs in each region. This will create the necessary foundation for the platform to run and includes the VPC, subnets, route tables, security groups, and VPC endpoints. :::tip Up to this point, we've used the `SuperAdmin` user for administrative access. With the Identity layer now deployed, switch to using your designated AWS Team credentials for local access and deployments. Using roles rather than users provides better security through temporary credentials and easier access management. Unless otherwise requested, assume all future deployments use your AWS Team. Please see [How to Log into AWS](/layers/identity/how-to-log-into-aws/) ::: ## Vendor the Networking components First, vendor the networking components by running the following: ## Deploy all VPCs Deploy all the VPCs in every configured region by running the following command: ## Decommission the default VPCs Once all VPCs are deployed, decomission the default VPC in each region by running the following command from within the Geodesic shell and while connected to you `core-identity` AWS profile: ```bash wipe-default-vpcs ``` --- ## Decide on AWS Account VPC Subnet CIDR Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We need to devise a subnet allocation scheme tolerant of multiple accounts operating in multiple regions that do not conflict with any other ranges which may need to be peered in the future. ## General Considerations - Having unique, non-overlapping VPC CIDRs makes connecting clusters to each other much easier - Each VPC must be subdivided into several non-overlapping subnet ranges to provide public and private address spaces across multiple availability zones - **ALBs need a minimum of 2 subnets allocated** ### EKS Considerations - Using Amazon’s CNI, each Kubernetes pod gets its own IP in the subnet, and additional IPs are reserved so they are immediately available for new pods when they are launched - You will need a lot more IPs than you anticipate due to performance optimizations in how CNIs are managed by EKS [https://betterprogramming.pub/amazon-eks-is-eating-my-ips-e18ea057e045](https://betterprogramming.pub/amazon-eks-is-eating-my-ips-e18ea057e045) [https://medium.com/codex/kubernetes-cluster-running-out-of-ip-addresses-on-aws-eks-c7b8e5dd8606](https://medium.com/codex/kubernetes-cluster-running-out-of-ip-addresses-on-aws-eks-c7b8e5dd8606) - AWS supports the `eksctl` tool (we do not). Their default recommendation is: > The default VPC CIDR used by `eksctl` is `192.168.0.0/16`. It is divided into 8 (`/19`) subnets (3 private, 3 public > & 2 reserved). - EKS clusters limit the number of pods based on the number of [ENIs available per instance type](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html#AvailableIpPerENI). - [Kubernetes has limits](https://kubernetes.io/docs/setup/best-practices/cluster-large/), but those are pretty high. **The reality is most clusters operate at a much smaller scale.** At v1.18, Kubernetes supports clusters with up to 5000 nodes. More specifically, we support configurations that meet _all_ of the following criteria: - No more than 5000 nodes - No more than 150000 total pods - No more than 300000 total containers - No more than 100 pods per node - EKS [will use either](https://github.com/aws/containers-roadmap/issues/216#issue-423314258) 10.100.0.0/16 or 172.20.0.0/16 for cluster Services, so avoiding those ranges will avoid some problems with inter-cluster routing :::caution Use CIDR ranges smaller than a `/19` at your own risk. Cloud Posse does not take responsibility for any EKS cluster issues related to underprovisioning CIDR ranges. ::: ## Our standard recommendation - Each account gets it's own `/16` (65,534 usable IPs) (or `/15` = 2 x `/16` for more than 4 total regions), consecutively numbered, starting with 10.101.0.0 - Each region in an account gets 1 x `/18` (16,382 usable IPs), usually allocated as 1 or 2 countries/legislative areas per account, each with 2 regions for DR/failover - Each region allocates 6 x `/21` (2,046 usable IPs) subnets (3 AZ \* (public + private)) for EKS. - Any additional “single purpose” subnets in a region should be `/24` (254 usable IPs) Further reading: - [https://aws.amazon.com/blogs/containers/eks-vpc-routable-ip-address-conservation/](https://aws.amazon.com/blogs/containers/eks-vpc-routable-ip-address-conservation/) - [https://medium.com/@jeremy.i.cowan/custom-networking-with-the-aws-vpc-cni-plug-in-c6eebb105220](https://medium.com/@jeremy.i.cowan/custom-networking-with-the-aws-vpc-cni-plug-in-c6eebb105220) - [https://tidalmigrations.com/subnet-builder/](https://tidalmigrations.com/subnet-builder/) ### CIDR Subnet Table | **Subnet Mask** | **CIDR Prefix** | **Total IP Addresses** | **Usable IP Addresses** | **Number of /24 networks** | | --------------- | --------------- | ---------------------- | ----------------------- | -------------------------- | | 255.255.255.255 | /32 | 1 | 1 | 1/256th | | 255.255.255.254 | /31 | 2 | 2\* | 1/128th | | 255.255.255.252 | /30 | 4 | 2 | 1/64th | | 255.255.255.248 | /29 | 8 | 6 | 1/32nd | | 255.255.255.240 | /28 | 16 | 14 | 1/16th | | 255.255.255.224 | /27 | 32 | 30 | 1/8th | | 255.255.255.192 | /26 | 64 | 62 | 1/4th | | 255.255.255.128 | /25 | 128 | 126 | 1 half | | 255.255.255.0 | /24 | 256 | 254 | 1 | | 255.255.254.0 | /23 | 512 | 510 | 2 | | 255.255.252.0 | /22 | 1,024 | 1,022 | 4 | | 255.255.248.0 | /21 | 2,048 | 2,046 | 8 | | 255.255.240.0 | /20 | 4,096 | 4,094 | 16 | | 255.255.224.0 | /19 | 8,192 | 8,190 | 32 | | 255.255.192.0 | /18 | 16,384 | 16,382 | 64 | | 255.255.128.0 | /17 | 32,768 | 32,766 | 128 | | 255.255.0.0 | /16 | 65,536 | 65,534 | 256 | | 255.254.0.0 | /15 | 131,072 | 131,070 | 512 | | 255.252.0.0 | /14 | 262,144 | 262,142 | 1024 | | 255.248.0.0 | /13 | 524,288 | 524,286 | 2048 | | 255.240.0.0 | /12 | 1,048,576 | 1,048,574 | 4096 | | 255.224.0 0 | /11 | 2,097,152 | 2,097,150 | 8192 | | 255.192.0.0 | /10 | 4,194,304 | 4,194,302 | 16,384 | | 255.128.0.0 | /9 | 8,388,608 | 8,388,606 | 32,768 | | 255.0.0.0 | /8 | 16,777,216 | 16,777,214 | 65,536 | | 254.0.0.0 | /7 | 33,554,432 | 33,554,430 | 131,072 | | 252.0.0.0 | /6 | 67,108,864 | 67,108,862 | 262,144 | | 248.0.0.0 | /5 | 134,217,728 | 134,217,726 | 1,048,576 | | 240.0.0.0 | /4 | 268,435,456 | 268,435,454 | 2,097,152 | | 224.0.0.0 | /3 | 536,870,912 | 536,870,910 | 4,194,304 | | 192.0.0.0 | /2 | 1,073,741,824 | 1,073,741,822 | 8,388,608 | | 128.0.0.0 | /1 | 2,147,483,648 | 2,147,483,646 | 16,777,216 | | 0.0.0.0 | /0 | 4,294,967,296 | 4,294,967,294 | 33,554,432 | --- ## Decide on CIDR Allocations import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Please also read the [design decision](/layers/network/design-decisions/decide-on-aws-account-vpc-subnet-cidr-strategy) for more information. --- ## Decide on Client VPN Options import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement You need to remotely access resource that reside in a private VPC. Different teams or individuals need access to different resources. ## Solution Use AWS Client VPN for remote user access. ## Considered Options Each option below can be integrated with AWS SSO. ### Option 1: Deploy 1 Client VPN in the Network Account :::tip Our Recommendation is to use Option 1 for customers who do not need fine-grained network access controls. Anyone on the VPN should have access to all network services via the Transit Gateway. ::: Ideal for companies where one team will require access to all accounts and there are no plans to introduce access for other teams. #### Pros - Anyone on the VPN has access to all network services via the Transit Gateway - Least expensive to operate - No need to switch networks once connected to VPN #### Cons - Total access to every account ### Option 2: Deploy Multiple Client VPNs Depending on Network Segments in the Network Account Ideal for companies where certain teams require segmented access to multiple accounts. We define these accounts as a segment. #### Pros - More access control options #### Cons - More expensive to operate - Deciding on how to segment the network can be a complex decision - Requires switching VPNs when accessing another account. This is more disruptive to developer workflows ### Option 3: Deploy Client VPN(s) Directly in the Accounts Needed This is a requirement when you know you need very granular access controls with restricting access to certain accounts. #### Pros - Highest level of access control to each account. #### Cons - Most expensive to operate and grows as more accounts are added - Requires switching VPNs when accessing another account. This is the most disruptive path to developer workflows. ## References - --- ## Decide on Hostname Scheme for Service Discovery import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ### Context and Problem Statement We need a consistent way of naming resources. Also please see related design-decision concerning DR implications. This is not an easily reversible decision once whatever convention is in use across services. ### Considered Options 1. Multi-cloud? e.g. AWS, GCP, Azure (we recommend baking the cloud into the service discovery domain. See [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain)) 2. Multi-region? [Decide on Primary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) 3. Pet or Cattle? → Blue/green or multi-generational 4. Short or Long region name? see [Decide on Regional Naming Scheme](/layers/project/design-decisions/decide-on-regional-naming-scheme) 5. Does it extend all the way down to the VPC? (we do not recommend this due to excessive subnet allocations and complications around network routing) 6. Too many DNS zone delegations add latency to DNS lookups due to having to jump between nameservers We typically use the following convention with tenants - `$service.$region.$account.$tenant.$tld` - e.g. `eks.us-east-1.prod.platform.ourcompany.com` where `platform` is the tenant Or without tenants: - `$service.$region.$account.$tld` - e.g. `eks.us-east-1.prod.ourcompany.com` without a tenant The question is now what to do for the `$service` name. Using `eks` is visually appealing but treating the cluster like a named pet. If in the future we want to support multiple generations of clusters, we may want to consider this in whatever convention. We may want to consider that following convention: - `$service-$color.$region.$account.$tld` with a `CNAME` of `$service.$region.$account.$tld` pointing to the live cluster - e.g. `eks-blue.us-east-1.prod.ourcompany.com` ## Related - [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain) --- ## Decide on How to Support TLS import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options ### Option 1 (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - #### Cons - ### Option 2 #### Pros - #### Cons - ### Option 3 #### Pros - #### Cons - ## References - Links to any research, ADRs or related Jiras --- ## Decide on IPv4 and IPv6 support import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options ### Option 1 (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - #### Cons - ### Option 2 #### Pros - #### Cons - ### Option 3 #### Pros - #### Cons - ## References - Links to any research, ADRs or related Jiras --- ## Decide on Opting Into Non-default Regions import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; If a Region is disabled by default, you must enable it before you can create and manage resources. It would be a pre-requisite to deploying anything in the region. The following Regions are disabled by default: - Africa (Cape Town) - Asia Pacific (Hong Kong) - Asia Pacific (Jakarta) - Europe (Milan) - Middle East (Bahrain) When you enable a Region, AWS performs actions to prepare your account in that Region, such as distributing your IAM resources to the Region. This process takes a few minutes for most accounts, but this can take several hours. You cannot use the Region until this process is complete. Source: [https://docs.aws.amazon.com/general/latest/gr/rande-manage.html](https://docs.aws.amazon.com/general/latest/gr/rande-manage.html) ## Procedure for enabling a region If we need to enable the regions, it needs to be done as a manual step and is convenient to do at the same time we set up MFA for the root user of the account. We also at the same time need to edit the STS Global endpoint settings to generate credentials valid in all regions instead of just the default regions. When you enable the regions in the AWS console, you are prompted to do this, so just follow the prompt. ## Related - [Decide on Primary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) --- ## Decide on Organization Supernet CIDR Ranges import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem - We need to record all existing and provisioned CIDR ranges as a system of record, as well as any additional context as necessary (E.g. what the CIDRs are used for). - We need to decide on the all-encompassing CIDR for this organization for contiguous networks. It’s not a requirement, but a strong recommendation. - All VPCs subnets should be carved out of this supernet. [Decide on AWS Account VPC Subnet CIDR Strategy](/layers/network/design-decisions/decide-on-aws-account-vpc-subnet-cidr-strategy) ## Solution - Document the CIDR ranges provisioned for all the accounts in ADR so we know what is in use today - Add any other known CIDR ranges (e.g. from other accounts not under this AWS organization) - Take into account any multi-cloud, multi-region strategies. - [https://tidalmigrations.com/subnet-builder/](https://tidalmigrations.com/subnet-builder/) ### Example ## Pro Tip Use the [https://tidalmigrations.com/subnet-builder/](https://tidalmigrations.com/subnet-builder/) with an additional overlay from CleanshotX. --- ## Decide on Primary AWS Region import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; While the company might operate in multiple regions, one region should be selected as the primary region. There are certain resources that will not be geographically distributed and these should be provisioned in this default region. When starting from scratch with a new AWS account, it's a good time to revisit decisions that might have been made decades ago. There are many new AWS regions that might be better suited for the business. ### Customer Proximity One good option is picking a default region that is closest to the where the majority of end-users reside. ### Business Headquarters One good option is picking a default region that is closest to where the majority of business operations take place. This is especially true if most of the services in the default region will be consumed by the business itself. ### Stability When operating on AWS, selecting a region other than `us-east-1` is advisable as this is the default region (or used to be) for most AWS users. It has historically had the most service interruptions presumably because it is one of the most heavily-used regions and operates at a scale much larger than other AWS regions. Therefore we advise using `us-east-2` over `us-east-1` and the latencies between these regions is very minimal. ### High Availability / Availability Zones Not all AWS regions support the same number of availability zones. [Many regions only offer (2) availability zones](https://howto.lintel.in/list-of-aws-regions-and-availability-zones/) when a minimum of (3) is recommended when operating kubernetes to avoid "split-brain" problems. ### Cost Not all regions cost the same to operate. ### Service Availability Not all regions offer the full suite of AWS services or receive new services at the same rate as others. Other times, certain regions receive platform infrastructure updates slower than others. Also, recently AWS launched [Local Zones](https://aws.amazon.com/about-aws/global-infrastructure/regions_az/#AWS_Local_Zones) (e.g. `us-west-2-lax-1a`) which operate a subset of AWS services. ### Instance Types Not all instance types are available in all regions ### Latency Latency between v1 infrastructure and v2 infrastructure could be a factor. See [cloudping.co/grid](https://www.cloudping.co/grid) for more information. ### References - [https://www.geekwire.com/2017/analysis-rethinking-cloud-architecture-outage-amazon-web-services/](https://www.geekwire.com/2017/analysis-rethinking-cloud-architecture-outage-amazon-web-services/) - [https://www.concurrencylabs.com/blog/choose-your-aws-region-wisely/](https://www.concurrencylabs.com/blog/choose-your-aws-region-wisely/) - [https://www.concurrencylabs.com/blog/choose-your-aws-region-wisely/](https://www.concurrencylabs.com/blog/choose-your-aws-region-wisely/) --- ## Decide on Service Discovery Domain import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import ReactPlayer from "react-player"; It's important to distinguish between branded/vanity domains (e.g. `cloudposse.com`, `slack.cloudposse.com`) used by customers and your infrastructure service discovery domains (e.g. `cloudposse.net`) used by services or internal consumers. For example, a product might have dozens of branded domains for SEO and marketing purposes, but you'll only have one infrastructure powering it. The service discovery domain is only for internal consumption. We get to define the conventions for this, not marketing. 😉 The service discovery domain will always be hosted on Route53, while the vanity domain can be hosted anywhere. The "service discovery domain" will be further subdivided by delegating a dedicated zone to each AWS account. For example, we don’t share DNS zones between production and staging. Therefore each account has its own service discovery domain (E.g. `prod.example.net`). See [Decide on Hostname Scheme for Service Discovery](/layers/network/design-decisions/decide-on-hostname-scheme-for-service-discovery) for more context. This is a non-reversible decision, so we recommend taking the time to discuss with the team what they like the best. ## Considerations ### Length of Domain :::tip Our recommendation is to keep it short and simple. ::: The length of the domain doesn’t technically matter, but your engineers will be typing this out all the time. ### Buy New or Reuse :::tip We usually recommend registering a net-new domain (e.g. on route53) rather than repurposing an existing one. Domains are too inexpensive these days to worry about the cost. ::: The "service discovery domain" does not need to be associated with the company’s brand identity and can be something completely separate from the company itself. If you prefer to repurpose an existing one, then we recommend a TLD which has no existing resource records. :::caution We do not recommend using the service discovery domain for AWS account addresses due to the cold start problem. You cannot provision the accounts without the email & domain, and you cannot provision the email & domain in the new accounts since they do not yet exist. ::: ### Registrar :::tip We recommend using the Route53 Registrar from the `dns` account ::: When registering a new domain, we have the option of using Route53’s built-in registrar or using your existing registrar. Many enterprise-scale organizations use MarkMonitor to manage their domain portfolio. Our convention is to use the `dns` account (see [REFARCH-55 - Decide on AWS Account Flavors and Organizational Units](https://docs.cloudposse.com/layers/accounts/design-decisions/decide-on-aws-account-flavors-and-organizational-units)) as the registrar. Note, the AWS Route53 Registrar cannot be automated with terraform and ClickOps is still required for domain registration. [https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/registrar.html](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/registrar.html) We recommend checking with your legal department on where they want to consolidate domain ownership. It has larger ramifications as to IP/trademark defense. ### Choose Top-level Domain Wisely The `.com`, `.net`, or `.org`domains are what we typically recommend due to the maturity of the TLDs. :::caution Newer, trendier TLDs like `.io`, `.ninja`, `.sh`, etc. have unproven long-term viability. The `.io` domain, trendy amongst startups is actually the registrar for the Indian Ocean and has been the subject of much scrutiny. [https://thehackerblog.com/the-io-error-taking-control-of-all-io-domains-with-a-targeted-registration/](https://thehackerblog.com/the-io-error-taking-control-of-all-io-domains-with-a-targeted-registration/) [https://fortune.com/2020/08/31/crypto-fraud-io-domain-chagos-islands-uk-colonialism-cryptocurrency/](https://fortune.com/2020/08/31/crypto-fraud-io-domain-chagos-islands-uk-colonialism-cryptocurrency/) [https://www.spamhaus.org/statistics/tlds/](https://www.spamhaus.org/statistics/tlds/) ::: TLDs operated by Google (`.dev`, `.app`, Et al.) have mandatory HSTS (TLS) enabled in Chrome and browsers which adopt [https://hstspreload.org/](https://hstspreload.org/) . This means that you cannot access `http://` URLs by default, which is a security best-practice, but nonetheless inconsistent with other TLDs. [https://security.googleblog.com/2017/09/broadening-hsts-to-secure-more-of-web.html](https://security.googleblog.com/2017/09/broadening-hsts-to-secure-more-of-web.html) ### Multiple AWS Organizations For customers using the “Model Organization” pattern (see [Decide on AWS Organization Strategy](/layers/accounts/design-decisions/decide-on-aws-organization-strategy)) we recommend one TLD Service Discovery domain per AWS Organization. Organizations are a top-level construct for isolation, so we believe that extends all the way down to the Service Discovery domain. ### Multi-Cloud / On-prem If your organization plans to operate in multiple public clouds or on-prem, we recommend adopting a convention where each cloud gets its own service discovery domain, rather than sharing the domain across all clouds (e.g. by delegating zones). The primary reason is to reduce the number of zones delegated, but also to decouple cloud dependencies. See the related design decision on [Decide on Hostname Scheme for Service Discovery](/layers/network/design-decisions/decide-on-hostname-scheme-for-service-discovery) to understand our zone delegation strategy. e.g. Suppose you had to support AWS, GCP, Azure and On-prem, the convention could be: - `example-aws.net` - `example-gcp.net` - `example-azure.net` - `example-onprem.net` ### Internal/Public Route53 Zones :::tip We recommend using public DNS zones for service discovery ::: We generally prescribe using public DNS zones rather than internal zones. Security is all about Defense in Depth (DiD), and while this adds another layer between VPC, Private Subnets, Security Groups, Firewalls, Network ACLs and Shield that the added layer of obscurity has fewer benefits than detractions. The benefits of keeping the zones public are easier interoperability between networks that do not share a common DNS server, and the ability to expose services as necessary using the service discovery domain to third parties services (e.g. partners, vendors, integrations like Snowflake or Fivetran, etc). See also our related ADR [Proposed: Use Private and Public Hosted Zones](/resources/adrs/proposed/proposed-use-private-and-public-hosted-zones) for additional context. ### Dedicated TLD per Organization, Delegated DNS Zones per AWS Account Delegate one zone per AWS account name to each AWS account. For example `prod.example.net`, `staging.example.net`, and `corp.example.net`. :::tip We recommend delegating one zone per AWS account ::: ### Dedicated TLD per AWS Account Delegate one dedicated "Top Level Domain" to each account (or some subnet). For example, `example.qa` for staging and `example.com` for prod. The benefit of this approach is we truly share nothing between accounts. The downside is coming up with a scalable DNS naming convention. Thus a hybrid between DNS zone delegation and multiple-TLDs is recommended. We think this is overkill and instead, recommend the dedicated TLD per AWS Organization coupled and [Decide on Hostname Scheme for Service Discovery](/layers/network/design-decisions/decide-on-hostname-scheme-for-service-discovery) leveraging delegated zones by account. ## Related - [Decide on Hostname Scheme for Service Discovery](/layers/network/design-decisions/decide-on-hostname-scheme-for-service-discovery) - [Decide on Vanity (Branded) Domain](/layers/network/design-decisions/decide-on-vanity-branded-domain) - [https://youtu.be/ao-2mfA5OTE](https://youtu.be/ao-2mfA5OTE) --- ## Decide on Transit Gateway Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement The [AWS Transit Gateway](https://aws.amazon.com/transit-gateway/) connects VPCs located in any account or organization (and on-premises networks) through a centrally managed network hub. This simplifies the work of connecting networks and puts an end to complex VPC peering connections. Think of it like a cloud router, where each new connection is only made once. As you expand globally, inter-region peering connections, the AWS Transit Gateway helps establish a global network. All data is automatically encrypted and never travels over the public internet. With this in mind, the transit gateway needs to be configured to support the specific For example: - In which accounts will certain services live (e.g. Automation/Spacelift Runners, custom apps, etc)? - Where will the VPN solution be deployed, if there is one? - In which accounts will EKS clusters be deployed to? - Do certain stages need to communicate with one another (e.g. staging → prod and prod → staging)? ## Considered Options ### Option 1 (Recommended) :::tip Cloud Posse recommends Option 1 because it enables the use of automation to perform changes as well as any other business requirement ::: - Connect all accounts with the `auto` account to enable automation (Spacelift, GitHub Action Runners, etc) - Connect all accounts with the `network` account to use it as the entry-point for VPN connections - Any other requirements that are business driven (e.g. dev → staging, staging → prod, dev → dev, etc) #### Consequences - Can use automation (Spacelift) to handle changes to infrastructure - Can create private EKS clusters, accessible via automation and human (via VPN) ## References - [https://aws.amazon.com/transit-gateway/](https://aws.amazon.com/transit-gateway/) --- ## Decide on Vanity (Branded) Domains import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import ReactPlayer from "react-player"; ## Problem We need a domain that represents your branded domain in the live environment. This will be our synthetic production domain since we do not want to interfere with any current production environments. Also, we do not want to use any domains currently in use for any other purpose. :::caution **IMPORTANT** This is not the same as [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain). ::: - These domains are for public-facing endpoints - **Prevent devs from using logic jumps based on the domain instead of using feature flags** - **Maintain symmetry with production** (e.g. if we have `acme.com` in production redirect to `www.acme.com`, we should be able to test/validate identical behavior with a top-level domain in staging and dev) - `CNAME` **is not possible with a zone-apex, only** `A` **records**. (If we used `staging.acme.com`, we would technically want to use `www.staging.acme.com` for zone-apex parity and that just gets too long) - **Cookie domain scope should only be for staging or production but not both.** Separate domains prevent this from happening. For example, if you set `Domain=acme.com`, cookies are available on subdomains like `staging.acme.com` and `www.acme.com`. Therefore, we want to avoid this possibility. We have seen this affect the ability to properly QA features. - **CORS headers should prevent cross-origin requests from staging and product.** We want to prevent wildcards from permitting cross-staging requests which could lead to staging hammering production (vice versa) ``` Access-Control-Allow-Origin: https://*.acme.com ``` ## Considerations _Use one domain for each stage (prod, staging, dev, sandbox, etc)_ - This top-level domain will be delegated to each account as necessary. Typically including `prod`, `staging`, `dev`, and optionally `sandbox`. - Our standard recommendation is to acquire a new domain with a regular TLD. - One good convention is to use your namespace suffixed with `-prod` or `-staging`. e.g. `$namespace-$stage.com` would become `cpco-prod.com`, `cpco-staging.com`, `cpco-dev.com` :::info Remember, these are for synthetic testing of branded domain functionality, and not the _actual_ domains your customers will be using. ::: ## FAQ ### What are examples of vanity domains? Think of vanity domains as all of your publicly branded properties. E.g. `apple.com` and `www.apple.com` and `store.apple.com`. ### Why do we differentiate between vanity domains and service discovery domains? It’s not uncommon that vanity domains are controlled by a different entity in the organization and may not even be controlled using terraform or other IaC. Of course, we prefer to manage them with terraform using our [dns-primary](/components/library/aws/dns-primary/) component, it’s not a _technical_ requirement. ### What’s the difference between vanity domains and service discovery domains? Vanity domains are typically fronted by a CDN and then upstream to some load balancer (e.g. ALB). The load balancer on the other hand will typically have a service discovery domain associated with it (e.g. `lb.uw2.prod.acme.org`). The service discovery domain is a domain whose conventions we (operations teams) control (e.g. totally logical and hierarchical with multiple zone delegations). While the vanity domains are governed by a different set of stakeholders such as marketing, sales, legal, and SEO. You might have hundreds or thousands of vanity domains pointed to a single service discovery domain. ### Why don’t we just use `staging.acme.com` and `dev.acme.com` as our vanity domains? We want symmetry and this is not symmetrical to what you use currently in production. E.g. your production traffic doesn’t go to `prod.acme.com`. It goes to `acme.com` or `www.acme.com`; so for the same reason, we want to have something that symmetrical to `acme.com` for dev and staging purposes. Another example is a cookie set on `.acme.com` will work for both `staging.acme.com` and `prod.acme.com`, and that's a bad thing from a testing perspective. ## Related - [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain) - [https://youtu.be/ao-2mfA5OTE](https://youtu.be/ao-2mfA5OTE) --- ## Decide on VPC NAT Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; **DRAFT** ## Context and Problem Statement ## Considered Options ### Option 1 - One VPC per Region, Per Platform Account with Dedicated NAT Gateways per AZ (Recommended) :::tip Our Recommendation is to use Option 1 because it keeps the separation by stage and ensures all egress per stage originates from a specific set of IPs ::: #### Pros - Easily managed with terraform - Easier for third parties to restrict IPs for ingress traffic - Keep accounts symmetrical #### Cons - More expensive to operate as more NAT gateways are deployed (mitigated by reducing the number of gateways in lower stages) ### Option 2 - One VPC per Region, Per Platform Account with Centralized NAT Gateways per AZ in Network Account The Compliant Framework for Federal and DoD Workloads in AWS GovCloud (US) advocates for a strategy like this, whereby in the Network (transit) account, there will be a DMZ with a Firewall. #### Pros - Ideally suited for meeting specific compliance frameworks #### Cons - All traffic from all accounts egress through the same NAT IPs, making it hard for third-parties to restrict access (e.g. staging accounts can access third-party production endpoints) - Shared NAT gateways are “singletons” used by the entire organization; changes to these gateways are not be rolled out by stage. Risky to make changes - in the critical path of everything. ### Option 3 - Shared VPCs with Dedicated NAT Gateways #### Pros - Less expensive #### Cons - All traffic from all accounts egress through the same NAT IPs, making it hard for third-parties to restrict access (e.g. staging accounts can access third-party production endpoints) - Shared VPCs are “singletons” used by multiple workloads; changes to these VPCs are not be rolled out by stage. Risky to make changes. ## References - **Compliant Framework for Federal and DoD Workloads in AWS GovCloud (US)** [https://aws.amazon.com/solutions/implementations/compliant-framework-for-federal-and-dod-workloads-in-aws-govcloud-us/](https://aws.amazon.com/solutions/implementations/compliant-framework-for-federal-and-dod-workloads-in-aws-govcloud-us/) - Relates to [Decide on AWS Account VPC Subnet CIDR Strategy](/layers/network/design-decisions/decide-on-aws-account-vpc-subnet-cidr-strategy) --- ## Decide on VPC Network Traffic Isolation Policy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Describe why we are making this decision or what problem we are solving. ## Considered Options Both options support principles of least privilege. ### Option 1 - Use a Flat Network with two Subnets per AZ (Public, Private) (Recommended) :::tip Our Recommendation is to use Option 1 because it is the easiest to administer and reduces the complexity of the network architecture ::: #### Pros - Use Security Group ACLs to easily restrict service-to-service communication using Security Group IDs. - Elastic network that doesn’t require advanced insights into the size and growth of the workloads - #### Cons - Security Groups have limited flexibility across regions: e.g. Security Group ACLs only work with CIDRs across regions (and not by Security Group ID) - Harder to monitor traffic between workloads - ### Option 2 - Use a Custom Subnet Strategy Based on Workload #### Pros - More easily restrict network traffic across regions and data centers - Follows principles of Least-privilege - Also compatible with using Security Group ACLs for an additional layer of security - Easier to monitor traffic between workloads #### Cons - Requires advanced planning to identify and allocate all workloads and IP space - Harder to scale elastically - Puts a large burden on network administrators - Large route tables, complicated transit gateway rules - Requires active monitoring to ensure subnets are not at capacity ## References - Also relates to [Decide on AWS Account VPC Subnet CIDR Strategy](/layers/network/design-decisions/decide-on-aws-account-vpc-subnet-cidr-strategy) - Also relates to [Decide on VPC NAT Strategy](/layers/network/design-decisions/decide-on-vpc-nat-strategy) --- ## Decide on VPC Peering Requirements (e.g. to Legacy Env) import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement VPC peering is used when we need network connectivity between VPCs - possibly residing in different accounts underneath different organizations. ## Use cases - Peering with legacy/heritage accounts to facilitate migrations with minimal downtime. - Enablement of CI/CD to connect to clusters and databases within VPCs - Service migration - Connecting VPCs in multiple regions ## Considered Options ### VPC Peering :::tip Our recommendation is to use VPC peering mostly for connecting third-party networks or for cost optimization over transit gateways (when necessary). ::: This is where we would provision a vpc-peering component and require the legacy vpc id, account id, and an IAM role that can be assumed by the identity account. Direct VPC peering may reduce costs where there’s significant traffic going between two VPCs. [https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/](https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vpc-announces-pricing-change-for-vpc-peering/) ### Transit Gateway :::tip Our recommendation is to _always_ deploy a transit gateway so we can use it with Terraform automation to manage clusters and databases. This is regardless of whether or not we deploy VPC peering. ::: An alternative approach to VPC peering between accounts in AWS is also to leverage the transit gateway, which we usually deploy in most engagements to facilitate CI/CD with GitHub Actions and Spacelift automation. This would require a transit gateway already set up and configured in the legacy account so we can peer the v2 transit-gateway with the v1 infrastructure. [https://aws.amazon.com/transit-gateway/pricing/](https://aws.amazon.com/transit-gateway/pricing/) Be advised, that _excessive_ traffic over a transit-gateway will be costly. This is why there is a use-case to leverage both VPC peering and transit gateways. If the costs are significant between any two VPCs, direct VPC peering is a more cost-effective way to do it because the traffic doesn't egress to the transit gateway and then ingress back into the other account, effectively cutting transit costs in half. ### NAT Gateways If there are overlapping CIDR ranges in the VPCs, we’ll also need to consider deploying private NAT gateways to translate network addresses. [https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-scenarios.html#private-nat-overlapping-networks](https://docs.aws.amazon.com/vpc/latest/userguide/nat-gateway-scenarios.html#private-nat-overlapping-networks) --- ## Review Design Decisions(3) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for how you'll implement the network and DNS layer of your infrastructure. --- ## Setting up DNS import Note from '@site/src/components/Note'; import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Requirements Before deploying DNS, first purchase your chosen vanity and service domains in the `core-dns` account or in your chosen registrar. Refer back to the [Decide on Vanity (Branded) Domain](/layers/network/design-decisions/decide-on-vanity-branded-domain/) and [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain/) design decisions for more information. When registering a new domain, we have the option of using Route53’s built-in registrar or using an existing registrar. Many enterprise-scale organizations use MarkMonitor to manage their domain portfolio. Our convention is to use the `core-dns` account as the registrar. This allows us to use AWS IAM roles and policies to manage access to the registered domains and to centralized DNS management. the AWS Route53 Registrar cannot be automated with Terraform, so ClickOps is still required for domain registration. [Registering domain names using Amazon Route 53 - Amazon Route 53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/registrar.html) We recommend checking with your legal department on where they want to consolidate domain ownership. It has larger ramifications as to IP/trademark defense. ## Deploy DNS Components The DNS stacks are broken up into primary and delegated deployments. Primary DNS zones only start with an `NS` record among other defaults and expect the the owner of their associated domain to add these `NS` records to whatever console manages the respective domain. Consult the [dns-primary component documentation](/components/library/aws/dns-primary/) for more information. The delegated DNS zones insert their `NS` records into the primary DNS zone; thus they are mostly automated. Consult the [dns-delegated component documentation](/components/library/aws/dns-delegated/) for more information. To start the dns setup, run the following. This will go through creating primaries, and then follow up with establishing the delegates. ## Configure Registrar `NS` Records for Domain (Click Ops) In order to connect the newly provisioned Hosted Zone to the purchased domains, add the `NS` records to the chosen Domain Registrar. Retrieve these with the output of `dns-primary`. These will need to be manually added to the registered domain. - #### Delegate Shared Service Domain, `acme-svc.com` ```shell atmos terraform output dns-primary -s core-gbl-dns ``` - #### Delegate Platform Sandbox Vanity Domain, `acme-sandbox.com` ```shell atmos terraform output dns-primary -s plat-gbl-sandbox ``` - #### Delegate Platform Dev Vanity Domain, `acme-dev.com` ```shell atmos terraform output dns-primary -s plat-gbl-dev ``` - #### Delegate Platform Staging Vanity Domain, `acme-stage.com` ```shell atmos terraform output dns-primary -s plat-gbl-staging ``` - #### Delegate Platform Prod Vanity Domain, `acme-prod.com` ```shell atmos terraform output dns-primary -s plat-gbl-prod ``` [For more on `NS` records](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/SOA-NSrecords.html) ### ACM Each domain managed by the `dns-primary` component will create its own ACM certificate. However, we need additional ACM certificates to validate delegated domains. We use a separate instance of the `acm` component to provision each service domain certificate. We can deploy all required ACM certificates with the following command: --- ## FAQ(Network) import Intro from '@site/src/components/Intro'; import ReactPlayer from "react-player"; import Steps from '@site/src/components/Steps'; Frequently asked questions about network and DNS with Cloud Posse's reference architecture. ## What is the difference between a Vanity and a Service Domain? Service domains are fully automated constructions of host names without concern for marketing or branding. Although they are not secret, the public will never see them. We use these domains for logic driven service discovery of resources across the organization. On the other hand vanity domains are entirely up to the requirements of the business. Marketing may require hundreds or thousand of domains to be associated with an application, and these domains may not follow any naming pattern or hierarchy. These are the domains used by the customer. ## Other common DNS questions - [What are examples of vanity domains?](/layers/network/design-decisions/decide-on-vanity-branded-domain#what-are-examples-of-vanity-domains) - [Why do we differentiate between vanity domains and service discovery domains?](/layers/network/design-decisions/decide-on-vanity-branded-domain#why-do-we-differentiate-between-vanity-domains-and-service-discovery-domains) - [What’s the difference between vanity domains and service discovery domains?](/layers/network/design-decisions/decide-on-vanity-branded-domain#whats-the-difference-between-vanity-domains-and-service-discovery-domains) - [Why don’t we just use staging.acme.com and dev.acme.com as our vanity domains?](/layers/network/design-decisions/decide-on-vanity-branded-domain#why-dont-we-just-use-stagingacmecom-and-devacmecom-as-our-vanity-domains) ## Can we add additional VPCs? Yes you can create additional VPCs, although we recommend against it. By design, we implement account-level separation rather than VPC network data separation. So before creating a new VPC, ask yourself if the ultimate objective would be better accomplished by a new account. If you do want to continue with creating a new VPC, simply define a new instance of the `vpc` component in a given stack. Give that component a new name, such as `vpc/data-1`, and then inherit the default vpc settings. ## How can we add an additional region? In order to add a new network region: 1. Create a new mixin for the region: `stacks/mixins/{{ region }}/` 1. Define a new stack configuration for the region. The regions of any given account are defined by resources in the directories for the given region, `stacks/orgs/{{ namespace }}/{{ tenant }}/{{ stage }}/{{ region }}/` 1. Add the required resources to the stack file, `stacks/orgs/{{ namespace }}/{{ tenant }}/{{ stage }}/{{ region }}/network.yaml`. For example for networking, define a new VPC, connect Transit Gateway, and define Client VPN routes to the new regions. For more, see [How to Define Stacks for Multiple Regions](/learn/maintenance/tutorials/how-to-define-stacks-for-multiple-regions) ## How can we connect a legacy AWS account to our network? Connect a legacy AWS account with VPC Peering. For more, see the [`vpc-peering` component](/components/library/aws/vpc-peering/) ## Why not use `dns-delegated` for all vanity domains? The purpose of the `dns` account is to host root domains shared by several accounts (with each account being delegated its own subdomain) and to be the owner of domain registrations purchased from Amazon. The purpose of the `dns-primary` component is to provision AWS Route53 zones for the root domains. These zones, once provisioned, must be manually configured into the Domain Name Registrar's records as name servers. A single component can provision multiple domains and, optionally, associated ACM (SSL) certificates in a single account. Cloud Posse's architecture allows/requires that root domains shared by several accounts be provisioned in the `dns` account with `dns-primary` and delegated to other accounts with each account getting its own subdomain corresponding to a Route 53 zone in the delegated account. Cloud Posse's architecture requires at least one such domain, called "the service domain", be provisioned. The service domain is not customer facing and is provisioned to allow fully automated construction of host names without any concerns about how they look. Although they are not secret, the public will never see them. Root domains used by a single account are provisioned with the `dns-primary` component directly in that account. Cloud Posse calls these "vanity domains". These can be whatever the marketing or PR or other stakeholders want to be. **There is no support for `dns-primary` to provision root domains outside of the dns account that are to be shared by multiple accounts.** After a domain is provisioned in the `dns` account, the `dns-delegated` component can provision one or more subdomains for each account, and, optionally, associated ACM certificates. For the service domain, Cloud Posse recommends using the account name as the delegated subdomain (either directly, e.g. "plat-dev", or as multiple subdomains, e.g. "dev.plat") because that allows `dns-delegated` to automatically provision any required host name in that zone. So, the `dns` account gets a single `dns-primary` component deployed. Every other account that needs DNS entries gets a single `dns-delegated` component, chaining off the domains in the `dns` account. Optionally, accounts can have a single `dns-primary` component of their own, to have apex domains (which Cloud Posse calls "vanity domains"). Typically, these domains are configured with CNAME (or apex alias) records to point to service domain entries. The architecture does not support other configurations, or non-standard component names. ## Why should the `dns-delegated` component be deployed globally rather than regionally? The `dns-delegated` component is designed to manage resources across all regions within an AWS account, such as with Route 53 DNS records. Deploying it at the regional level can lead to conflicts because it implies multiple deployments per account, which would cause Terraform to fight for control over the same resources. Although the `gbl` (“global”) region is not a real AWS region, it is used as a placeholder to signify that resources are meant to be managed globally, not regionally. Deploying `dns-delegated` globally ensures there is a single source of truth for these DNS records within the account. Deploying this component regionally can cause issues, especially if multiple regional stacks try to manage the same DNS records. This creates an anti-pattern where resources meant to be global are unintentionally duplicated, leading to configuration drift and unexpected behavior. Please see the [global (default) region](/learn/conventions/#global-default-region) definition for more on `gbl` as a convention. ## How is the EKS network configured? EKS network is designed with this network and DNS architecture in mind, but is another complex topic. For more, see the following: - [EKS Fundamentals](/layers/eks) - [EKS - How To Setup Vanity Domains on an ALB](/layers/eks/tutorials/how-to-setup-vanity-domains-on-alb-eks) :::info Private subnets are kept large to account for EKS configurations that can consume a significant amount of IP addresses. ::: ## How is the ECS network configured? ECS connectivity is also designed with this network and DNS architecture in mind. For more, see the following: - [ECS Fundamentals](/layers/ecs) - [ECS - How To Setup Vanity Domains on an ALB](/layers/ecs/tutorials/how-to-setup-vanity-domains-on-alb-ecs) --- ## Network and DNS import ReactPlayer from "react-player"; import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Learn Cloud Posse’s approach to designing robust and scalable Network and DNS architectures on AWS. We discuss our solutions to common challenges in network design, focusing on maintainability, security, and scalability. We cover essential topics such as account isolation, connecting accounts using Transit Gateway, deploying AWS Client VPN for network access, and differentiating between service and vanity domains. This document is intended to present Cloud Posse's approach to designing Network and DNS architectures. The contents of this document assume that the reader is familiar with the basics of [networking and content delivery services in AWS](https://aws.amazon.com/products/networking/).
AI generated voice
## The Problem There is no single solution for Network and DNS architecture. Ultimately, the right network architecture may come down to your individual business needs. Yet often too much thought in design leads to snowflake designs that are specific to a given business but are entirely unique, overly complex, and difficult to maintain. However, all network designs have a fundamental set of requirements that we can define. All networks need some private subnets and some public subnets. All networks need to be able to restrict access and enforce boundaries externally and internally, and finally all networks need some way to discover services inside the network. When it comes to DNS, often there is no design consideration for domain management. Companies may have hundreds or thousands of marketing domains, e.g. for SEO, and yet have no sane method for services to discover each other using DNS. At the same time, DNS needs some of the same boundaries as the network: services should be isolated and secure. Furthermore, networking in a cloud environment is entirely software defined. We have the ability to do things that would be too tedious or too difficult to achieve in physical environments. Companies still largely rely on IPv4 networks, which have limited IP space, and we need to ensure that how we allocate networks can scale with your business and even integrate with other third party providers. Networking and content delivery is far from trivial. There are countless designs and architectures that accomplish similar outcomes. Ultimately our goal is to ensure that whatever we implement will enable your success. ## Our Solution As with all infrastructure design, Cloud Posse has an opinionated solution. We aim to reduce complexity where possible, while providing secure and robust networks that are maintainable and scalable. We have identified the most common and reusable pattern for network architecture, so that we can define reusable building blocks for a network. We've standardized the definition of a VPC, provided distinction between marketing and service domains for discovering services, and created a secure and reliable way for services to communicate with each other across the accounts. ### Account Isolation As a foundational design with the AWS Organization, we have already isolated resources into accounts. This separation creates a physical boundary between resources in AWS, including VPCs. Therefore, we can deploy a single or multiple VPCs in an account and guarantee that resources in those subnets will not be able to access resources in other accounts. Because of this design, we recommend deploying a single VPC per account (that needs a network). Production resources will only live in the VPC in the `plat-prod` account, and unless connected, no other VPC will be able to access those resources. Similarly, you could deploy a `data` account (or several) with a VPC to isolate data resources further. ### Connecting Accounts Now that we have separated networks in each account, we need to be able to connect the account networks as required. We do this with Transit Gateway. We deploy a Transit Gateway hub and route table to the central Network account, and then deploy Transit Gateway spokes to all other accounts. The route table in the Network account specifies which accounts are able to access others, and the Transit Gateway spokes provide that connection. ### Accessing the Network In order for a user to connect to the Network, we deploy an AWS Client VPN. This VPN is deployed to the Network account, which already has access to all other account networks. Then we define a set of rules for the VPN itself to specify where we want this VPN to be able to connect. ### Service Domains We recommend deploying a Service Domain in the DNS account and then connecting all app (platform) accounts to this service domain via subdomains. We delegate a single Host Zone for each account's subdomain. Since DNS is global, multi-resource records or resources in other regions will all be included in this same zone. This is why we consider the `dns` components as global. Furthermore, any services added has a logically defined record in the delegated zone. Consider the diagram below. Here `acme-svc.com` would be deployed with `dns-primary` in the DNS account, and all subdomains would be deployed in the respective app accounts with `dns-delegated` and use logically defined and hierarchical subdomains. For example, `echo.use1.dev.plat.acme-svc.com`, `echo.use1.prod.plat.acme-svc.com`, and `echo.use1.auto.core.acme-svc.com`. These domains are logically creating following the service, region, stage, tenant, and then finally the service domain. _The `echo.use1` resource record (CNAME) is created in the `dev.plat` hosted zone with `dns-delegated`, which is delegated from primary hosted zone, acme-svc.com, in the DNS account with `dns-primary`_ ### Vanity Domains Vanity domains are commonly referred to as "branded" or "marketing" domains and are used to meet the requirements of your individual business. A business may have any number of vanity domains as required, and that list of domains will grow as your business expands. Unlike service domains, we do not recommend delegatation on vanity domains. We do not want to share stages for vanity domains, because we must ensure total isolation between domains in each stage. To do this, we recommend deploying at least one domain per app account. By doing so, we can create symmetry between prod and non-prod environments and avoid cross-site scripting. This enables properly testing of an apex or root domain, so that we can fully validate a configuration before deploying to production. In the diagram below, we have many domains deploy across the organization. The DNS account holds only the root service domain deployed by `dns-primary`. All other stages control their respective vanity domains with a single `dns-primary` component. For example, we deploy a single vanity domain to dev, `acme-dev.com`, single vanity domain to staging, `acme-staging.com`, and many vanity domains to production, `acme.com`, `acme-prod.com`, `acme-marketing.com`, or even `alternate-brand.com`. These domains can be whatever your business requires. --- ## Monitor Everything import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; import Step from '@site/src/components/Step'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import ReactPlayer from 'react-player'; With so many moving pieces, it's crucial to monitor what's happening under the hood to understand what's going on. This includes gathering telemetry in the form of metrics and logs coming from your services and the underlying infrastructure. This data must be shipped somewhere to build dashboards and raise alerts that will escalate to the appropriate personnel. Depending on your business needs, you may also need to monitor for security and compliance against various technical benchmarks like PCI/DSS, CIS, ISO 27001, and others. ## Set up Telemetry Choose between Datadog or AWS-managed Prometheus and Grafana with Loki for gathering your telemetry. Datadog offers the most mature implementation, while AWS-managed Grafana and Prometheus provide lower-cost alternatives with various trade-offs, that make them a good fit for many organizations. Datadog is our most comprehensive observability solution, offering a monitoring-as-code approach using YAML configuration fully managed with Terraform. This includes Datadog monitors, custom RBAC roles, synthetic tests, child organizations, and other resources. We show how to define reusable Service Level Indicators (SLIs) and Service Level Objectives (SLOs) for consistent implementation, helping to reduce alert fatigue by focusing on critical business-specific metrics and leveraging Datadog's advanced capabilities. Get Started Amazon Managed Grafana is a fully managed service by AWS in collaboration with Grafana Labs. Although it's significantly less expensive than Datadog, it is also more barebones in comparison. - Managed Grafana allows you to query, visualize, and set alerts for your metrics, logs, and traces through a centralized dashboard where you can add multiple data sources. - AWS Managed Prometheus together with `promtail` collects and queries metrics from your containerized applications. - Deploy Loki for efficient log collection from containerized applications (for EKS users) Get Started
AI generated voice
## Monitor for Security & Compliance Monitoring for security and compliance is essential for organizations subject to industry regulations like HIPAA or for e-commerce companies aiming for PCI compliance. Our reference architecture includes comprehensive support for AWS's suite of security-oriented services, including: - [Security Hub](https://aws.amazon.com/security-hub/): Centralized security view - [GuardDuty](https://aws.amazon.com/guardduty/): Threat detection service - [Inspector](https://aws.amazon.com/inspector/): Automated security assessments - [Macie](https://aws.amazon.com/macie/): Data security and privacy - [AWS Config](https://aws.amazon.com/config/): Resource configuration tracking - [IAM Access Analyzer](https://aws.amazon.com/iam/features/analyze-access/): Policy monitoring and validation - [Shield](https://aws.amazon.com/shield/): DDoS protection - [Audit Manager](https://aws.amazon.com/audit-manager/): Continuous audit and compliance - [CloudTrail](https://aws.amazon.com/cloudtrail/): User activity and API usage - [WAF](https://aws.amazon.com/waf/): Web application firewall
--- ## Set Up Your Platform import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; import Step from '@site/src/components/Step'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import ReactPlayer from 'react-player'; Your platform ensures consistent service delivery every time. A well-designed platform seamlessly integrates with your monitoring, security, and compliance systems, building on your established foundation. Automated software delivery pipelines deploy new services quickly and easily. The reference architecture supports AWS EKS, Amazon ECS, and Lambda functions. ## Container Orchestration Choose a path for consistent delivery of your services. The reference architecture supports AWS EKS, Amazon ECS, and Lambda functions. Elastic Container Service (ECS) is a fully-managed container orchestration service provided by Amazon Web Services (AWS) that simplifies the process of deploying, managing, and scaling containerized applications. ECS makes it easy to run and manage Docker containers on AWS infrastructure, providing a secure and scalable platform for your applications. One of the major benefits of ECS over EKS, is that there is no need to upgrade the underlying platform. ECS is a managed service that is always up to date. This means that you can focus on your application and not the underlying platform.
AI generated voice
Get Started
Amazon EKS is a managed Kubernetes service that allows you to run Kubernetes in AWS cloud and on-premises data centers. AWS handles the availability and scalability of the Kubernetes control plane, which oversees tasks such as scheduling containers, managing application availability, and storing cluster data. While AWS manages control plane upgrades, users are responsible for the worker nodes and the workloads running on them, including operators, controllers, and applications. We use Karpenter for managing node pools and support spot instances to optimize costs. Be aware that you'll need to upgrade the cluster quarterly due to the significant pace of Kubernetes innovation. Although EKS has a steeper learning curve compared to ECS, it offers greater flexibility and control, making it ideal for organizations already utilizing Kubernetes. Get Started
## Configure GitHub Actions to enable CI/CD Deploy self-hosted runners to automate your software delivery pipelines, within private networks. Get Started ## Automate Terraform Your Terraform Deployments Use GitHub Actions to automate your Terraform deployments with Atmos, ensuring consistent infrastructure across your environments.
AI generated voice
Get Started
Once you're done setting up your platform, our attention will shift to how you ship your software by leveraging GitHub Actions and GitHub Action Workflows. --- ## Creating an Infrastructure repository import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Note from '@site/src/components/Note'; import Admonition from '@theme/Admonition'; Learn how to create a GitHub repository to host infrastructure tools and configurations. Then configure repository settings, enable branch protection, and add collaborators. ## Create a GitHub repository Create an empty GitHub repository to host infrastructure tools and configuration, and clone it to your host computer into a directory below your `$HOME` directory in the directory tree, for example, `/Users/morpheus/src/infrastructure`. ## Setup basic repository settings We recommend the following GitHub repository settings. 1. Under "Features", ensure "Issues" are enabled. We will use Issues with Atmos GitHub Actions for Terraform. 1. Under "Pull Requests", disable both "Allow merge commits" and "Allow rebase merging". We do this to create a clean commit history. Otherwise, the git history on the main branch will contain individual commits from each feature branch, which will result in a history which may contain _"dirty"_ commits. 1. Under "Pull Requests", check "Always suggest updating pull request branches" 1. Under "Pull Requests", check "Automatically delete head branches" ## Enable branch protection 1. Under branches, select "Add branch ruleset" 1. Name this ruleset whatever you'd like. For example, "Protected Branches" 1. Under "Bypass list", add your admin team. This team is able to skip the rules were are about to add, so this the bypass list should be sparingly granted 1. Under "Target branches", select "Add target" and "Include all default branches" 1. Then check the following rules: - [x] Restrict deletions - [x] Require a pull request before merging - [x] Block force pushes ## Add collaborators and teams Now add in your collaborators and teams. Under "Collaborators and teams", click "Add Teams". Generally most teams should have "Write" access, and "Admin" access should be granted sparingly. For engagements with Cloud Posse, please grant Cloud Posse "Write" access. ## Import the Reference Architecture With the GitHub repository prepared, we are now ready to import the Cloud Posse reference architecture. The contents of this repository are supplied as part of our [Quickstart](/quickstart) or [Jumpstart](/jumpstart) packages. For the remainder of this guide, we will assume you have access to the reference architecture configurations. Learn More With your repository set up, we need to address some of the prerequisites for your workstation. This includes building the toolbox image on your workstation. Next Step --- ## Decide on 1Password Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We need to determine the best strategy for using 1Password to securely share sensitive information, such as passwords and integration keys, with individuals and teams during engagements with Cloud Posse. This decision aims to ensure a secure and efficient method for exchanging secrets while considering compatibility with AWS root account credentials. ## Problem We need a secure (cryptographic) way to share sensitive information (e.g. passwords, integration keys, credit card numbers, etc) with individuals and teams. Ideally, the solution works with AWS so we can secure root account credentials. 1Password is a great choice for sharing secrets with teams. The downside is it doesn't support cryptographically secure means of sharing secrets with individuals. It also does not integrate with terraform. Please see [Decide on MFA Solution for AWS Root Accounts](/layers/accounts/design-decisions/decide-on-mfa-solution-for-aws-root-accounts) for additional context on why we recommend 1Password. ## Supported Options :::caution During the course of your engagement with Cloud Posse we require using 1Password as the secrets storage for exchanging secrets between teams. Customer is free to use whatever system internally and copy secrets out of 1Password. ::: ### Use Your 1Password (Recommended) You can share a private vault with our team for the duration of this engagement. ### Use Cloud Posse’s 1Password (Temporary Alternative) We can share a private vault with your team for the duration of this engagement. That way your company can work on procuring the best solution for your team. We recommend this approach if your team does not already have a viable solution and procurement of 1Password will delay the engagement. ## Excluded Options ### PGP / GPG / PKE Public Key Encryption is a great way to securely exchange secrets, but it's overly complicated for non-engineers. Anything that’s complicated or not the path-of-least-resistance tends to lose in the long run. ### Slack Slack does not provide any secure means of exchanging secrets. It should not be used. ### LastPass LastPass does not provide a means for shared TOTP, so we cannot work in a collaborative environment. --- ## Decide on ECR Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; This decision assumes that per the previous design decision, we’ll be using ECR to store docker images. There are a number of ways we can do this. Here are some considerations. Many of these concepts can be combined, so we’ll just list them out. ## Considerations - Do you have a monorepo with multiple containers built from different `Dockerfile`. We recommend no more than one `Dockerfile` per repo. - What is the naming convention for repositories? We recommend naming ECR repositories after the GitHub repo. - Lifecycle rules to restrict the number of images and avoid hard limits ## Architecture 1. We typically deploy a single ECR in the `artifacts` (or similar account like `automation`). This is our typical recommendation. Each service will have one docker repository. All images are pushed to this repo with commit SHAs. We'll use lifecycle rules on tags to ensure critical images are not deleted. There's no promotion of images between ECRs and all ECRs are typically read-only from any account. 2. We can deploy multiple ECRs per service (e.g. `myservice-prod`, `myservice-dev`. Then promote images between the ECRs. We’ve only done this once and honestly don’t like it because it adds a lot of complexity to the pipelines without much profit. 3. We can deploy one ECR per account or set of accounts. For example, we can have a production ECR and another one for everything else. We’ll need to orchestrate image promotion between ECRs, which is the reason we don’t usually recommend this. 4. Docker Lambas require ECR within the same account. For this, we’ll need to provision an additional ECR repo per account and recommend setting up replication from a centralized repo. ## Configuration We’ll need the repo in place before we can push docker images to it. When and how should we provision it? 1. Should each service define it’s own ECR in the microservice repository? - How should this be implemented? For example, if we’re practicing gitops for GitHub repository creation, then we can also provision the ECR at the same time. If we’re not, then we’ll need to tie this into the pipelines for the GitHub repository itself. 2. Should we centralize this in the foundational infrastructure? - If we centralize it, how should it be configured? - Provide a long static-list of repos - Use the `terraform-github-provider` to generate that list automatically for all repositories --- ## Decide on Infrastructure Repository Name import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We highly recommend using a mono-repo for your foundational infrastructure. This doesn’t preclude introducing other infrastructure repositories in the future. Suggestions: - `infrastructure` - `infra` - `$namespace-infra` (e.g. `cpco-infra`) - `cloud-infrastructure` - `ops` - `cloud-ops` If you already have a repo by any of these names, we suggest you create a new one so we can start with a clean history. --- ## Decide on Namespace Abbreviation import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Using a common prefix for all resource names will help establish a consistent naming convention. Certain resources in AWS are globally unique (e.g. for all customers). In order to maintain an (optimistically) unique naming convention, prefixing all resources with a namespace goes a long way to ensuring uniqueness. Shorter the better. Some AWS resource names like S3 bucket names and Elasticache Redis names are restricted to something like 65 characters. We recommend a namespace prefix of 2-4 characters. The longer, the more optimistic we can be about avoiding collisions with other global resources in AWS. Some strategies we’ve seen is removing all vowels from your company name, or taking the initials for longer company names. ## Examples
Intel
`intl`
Google
`ggl`
Cloud Posse
`cpco`
:::note It is advised to keep the namespace as short as possible (< 5 chars) because of resources with low max character limits [AWS Resources Limitations](/resources/legacy/aws-feature-requests-and-limitations) ::: ## References - [https://github.com/cloudposse/terraform-null-label](https://github.com/cloudposse/terraform-null-label) --- ## Decide on Regional Naming Scheme import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We need to decide how we’ll handle DR if that’s a requirement. It has far-reaching implications on naming conventions and is not an easily reversible decision. Our current best practice is to use the following convention: | **Field** | **Description** | **Example** | | ------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `namespace` | Something short that uniquely identifies your organization. This relates to [Decide on Namespace Abbreviation](/layers/project/design-decisions/decide-on-namespace-abbreviation) | `cpco``eg` | | `tenant` | | | | `environment` (aka abbreviated region) | Environment indicates which AWS region the resource is in, using one of 2 sets of abbreviations. We use `gbl` for resources that are not specific to any region, such as IAM Roles. You have a choice of 2 sets of abbreviations: `fixed` or `short`.The `fixed` abbreviations are- exactly 3 letters- short and consistent so lists stay aligned on semantic boundaries- The drawback is that AWS regions, have collisions when algorithmically reduced to 3 letters, so some regions (particularly in Asia) have non-obvious abbreviationsThe `short` abbreviations are- 4 or more letters- easier to understand- usually identical to the prefix AWS uses for Availability Zone IDs in the region- The drawback is that there is 1 or more additional characters which can lead closer to max character restraints (e.g. target groups have a max of 32 characters)We recommend using the `short` abbreviations, which more closely canonical zone ids by AWS.See [AWS Region Codes](/resources/adrs/adopted/use-aws-region-codes/#region-codes) for the full breakdown. | AWS region code → fixed abbreviation (3 letter) → short abbreviation (4 letter+)`us-east-1` → `ue1` → `use1``us-west-2` → `uw2` → `usw1``eu-west-3` → `ew3` → `euw3``ap-south-1` → `as0` → `aps1``af-south-1` → `fs1` → `afs1``cn-north-1` → `nn0` → `cnn1``us-gov-west-1` → `gw1` → `usgw1` | | `stage` (aka account) | The stage is where the resources operate. Our convention is to isolate every stage in a dedicated AWS member account (aka flavor), which is why we frequently call accounts stages. | `prod`, `at`, `network` | These field names correspond to the variable inputs of the `terraform-null-label` ([https://github.com/cloudposse/terraform-null-label](https://github.com/cloudposse/terraform-null-label)) used throughout all Cloud Posse terraform modules. Usage of this convention ensures consistency and reduces the likelihood of resource name collisions while maintaining human legibility. Using this convention, resource names look like this: `{namespace}-{tenant}-{environment}-{stage}-{name}-{attributes}` Here are some more examples to help understand the relationships. | **Inputs** | **Outputs** | | ---------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | |
namespace: acmeenvironment: ue2 # us-east-2stage: automationname: eks-cluster
| `acme-ue2-automation-eks-cluster` | |
namespace: acmeenvironment: gblstage: devname: eks-clusterattributes: ["test"]
| `acme-gbl-dev-eks-cluster-test` | |
namespace: acmetenant: vkngenvironment: ue2 # us-east-2stage: automationname: eks-cluster
| `acme-vkng-ue2-automation-eks-cluster` | Also, see the corresponding design decision for the [Decide on Hostname Scheme for Service Discovery](/layers/network/design-decisions/decide-on-hostname-scheme-for-service-discovery) as this will be impacted by whatever is chosen. --- ## Decide on Secrets Placement for Terraform import Intro from "@site/src/components/Intro"; We need to decide where to store secrets used by Terraform. We have two options: store secrets in each account or store them in a centralized account. ## Context Often we need to integrate with third-party services or internal services that require API keys or other secrets. We need to decide where to store these secrets so that Terraform can access them. There are two reasonable options for storing secrets in our AWS account architecture. We need to decide which one to use. ### Option 1: Store Secrets in each account The first option is to store the credential in the same account as the resource. For example, API keys scoped to `dev` would live in `plat-dev`. #### Pros - Accounts can easily access their given credentials - IAM level boundaries are enforced between accounts #### Cons - Secret administrators need to access many accounts to create those secrets - There is no centralized management for all secrets out there ### Option 2: Store Credentials in a Centralized Account The second option is to store the credentials in a centralized account, such as `corp` or `auto`. Now you would need to share those credentials with each account, for example with [AWS RAM](https://aws.amazon.com/ram/). #### Pros - Centralized secrets management - Secret administrators have a single place to manage secrets - Once shared, resources in a given account still access their given secrets from their own account. They do not need to reach out to another account #### Cons - Complexity with AWS RAM - Secret administrators must be careful to share secrets with the correct accounts - You need to decide what account to use as the centralized management account. We could deploy `corp` if you'd like for this or reuse `auto`. ## Decision We will use AWS SSM Parameter Store for all platform-level secrets used by `infrastructure` and `terraform`. ## Related - [Decide on Secrets Strategy for Terraform](/layers/project/design-decisions/decide-on-secrets-management-strategy-for-terraform/) --- ## Decide on Secrets Management Strategy for Terraform import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Deciding how to store secrets is crucial for securing both platform integration and application data when using Terraform. The appropriate secret store depends on the stack layer and must account for situations where other infrastructure might not yet be in place (e.g. Vault, Kubernetes, etc). We need to decide where secrets will be kept. We’ll need to be able to securely store platform integration secrets (e.g. master keys for RDS, HashiCorp Vault unseal keys, etc) as well as application secrets (any secure customer data). One consideration is that a self-hosted solution won’t be available during cold-starts, so a hosted/managed solution like ASM/SSM is required. - e.g. Vault deployed as helm chart in each tenant environment using KMS keys for automatic unsealing (this chart already exists) - SSM Parameter Store + KMS for all platform-level secrets used by `infrastructure` and Terraform - AWS Secrets Manager supports automatic key rotation which almost nothing other than RDS supports and requires applications to be modified in order to use it to the full extent. ## Recommendation We will use AWS SSM Parameter Store for all platform-level secrets used by `infrastructure` and Terraform. ## Related - [Use SSM over ASM for Infrastructure](/resources/adrs/adopted/use-ssm-over-asm-for-infrastructure) - [Decide on 1Password Strategy](/layers/project/design-decisions/decide-on-1password-strategy) --- ## Decide on Terraform Version import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Different versions of Terraform and OpenTofu offer varying features and compatibility. Terraform 1.x versions maintain backward compatibility within the series, providing stability for existing workflows. However, OpenTofu offers a fully open-source alternative that aligns with Cloud Posse's values and avoids potential legal risks introduced by Terraform's licensing changes. To ensure consistency and compatibility across modules and components, Cloud Posse recommends OpenTofu as the preferred choice for new projects and workflows. :::warning Disclaimer The content of this document is provided for informational purposes only and should not be construed as legal advice. Cloud Posse is not qualified to provide legal counsel, and any decisions related to the use of Terraform under the Business Source License (BSL) should be reviewed by professional legal advisors. OpenTofu is recommended based on technical and operational considerations, not legal advice. ::: ## Context Terraform is a popular infrastructure-as-code tool that allows you to define, provision, and manage cloud resources. Terraform is developed by HashiCorp. From inception to 1.5.7, all versions were permissively licensed under the OSI-approved MPL software license. All newer releases are available under the Business Source License (BSL). The BSL license imposes restrictions on the use of Terraform in certain scenarios, which may impact long-term use and compatibility with third-party tools and integrations. Subsequently, every major open-source OS distribution (e.g. [Debian](https://wiki.debian.org/DFSGLicenses#DFSG-compatible_Licenses), [Alpine](https://wiki.alpinelinux.org/wiki/Release_Notes_for_Alpine_3.19.0#HashiCorp_packages), [Homebrew](https://formulae.brew.sh/formula/terraform)) has removed Terraform from their registries due to the BSL license. [GitLab has also removed Terraform](https://docs.gitlab.com/ee/update/deprecations.html#deprecate-terraform-cicd-templates) from their CI/CD pipelines due to the BSL license. This has created a significant challenge for organizations that rely on Terraform for infrastructure automation. OpenTofu (previously named OpenTF) is a fork of Terraform 1.5.7 that was [accepted by the CNCF](https://www.linuxfoundation.org/press/announcing-opentofu) and is fully open-source under the MPL license. OpenTofu is designed to maintain compatibility with Terraform 1.x modules and components while providing a stable and open-source alternative to the BSL-licensed Terraform versions. :::important Terraform providers are not affected by this change. They are independently licensed and can be used with any version of Terraform and OpenTofu. While HashiCorp maintains some providers, the vast majority are not maintained by HashiCorp. Most importantly, the [`terraform-provider-aws`](https://github.com/hashicorp/terraform-provider-aws/blob/main/LICENSE) remains under the MPL license. ::: ### OpenTofu Supporters [![CNCF Landscape](https://img.shields.io/badge/CNCF%20Landscape-5699C6)](https://landscape.cncf.io/?item=provisioning--automation-configuration--opentofu) The project is backed by many organizations, including: - [CNCF](https://github.com/cncf/sandbox/issues/81) - [CloudFlare](https://blog.cloudflare.com/expanding-our-support-for-oss-projects-with-project-alexandria/) - [OpenStreet Maps](https://twitter.com/OSM_Tech/status/1745147427324133501) - [JetBrains](https://blog.jetbrains.com/idea/2024/11/intellij-idea-2024-3/) - [Cisco](https://blogs.cisco.com/developer/open-tofu-providers) - [Microsoft Azure](https://github.com/Azure/Azure-Verified-Modules/discussions/1512), [`microsoft/fabric`](https://github.com/opentofu/registry/issues/1004), [`terraform-provider-azapi`](https://github.com/opentofu/registry/issues/920) - [VMWare Tanzu](https://docs.vmware.com/en/Tanzu-Cloud-Service-Broker-for-AWS/1.10/csb-aws/GUID-index.html) - Cloud Posse - Mixpanel - Buildkite - ExpressVPN - Allianz - Harness - Gruntwork - Spacelift - Env0 - Digger - Terrateam - Terramate For the full list of supporters, see the [OpenTofu website](https://opentofu.org/supporters/). ## Problem Historically, Terraform versions pre-1.x were notoriously backward incompatible. This changed with Terraform 1.x releases, where backward compatibility is assured for all subsequent 1.x releases. While Terraform provides a stable experience, its recent shift to the BSL license introduces considerations for certain use cases, integrations, and compliance. OpenTofu is based on Terraform 1.5.7 (the last MPL-licensed version) and maintains compatibility with Terraform 1.x modules and continues to evolve as a fully open-source project under the stewardship of the CNCF. Cloud Posse modules and components are verified to work with OpenTofu as part of our test automation, but with hundreds of modules, there may be delays in verifying full support with every new release. OpenTofu has not been without controversy, with some organizations expressing concerns about the project's governance and sustainability. [HashiCorp sent a cease and desist](https://opentofu.org/blog/our-response-to-hashicorps-cease-and-desist/) to the project. However, the project has gained significant traction and support from the community, including key contributors from the original Terraform project. As a result, [it's sandbox application to the CNCF is delayed](https://github.com/cncf/sandbox/issues/81#issuecomment-2331714515) (as of 2024-09-05). ## Considerations Using OpenTofu ensures compatibility with third-party tools and integrations that are no longer supported with BSL-licensed Terraform versions. Furthermore, OpenTofu aligns with Cloud Posse's commitment to open-source principles and avoids potential compatibility and operational risks associated with BSL-licensed software. Cloud Posse only supports MPL-licensed versions of Terraform (Terraform 1.5.7 or older), and all versions of OpenTofu. Terraform 1.x remains backward compatible within the major version, but its BSL license imposes restrictions that may impact long-term use. ## Recommendation Cloud Posse recommends using the [latest OpenTofu release](https://github.com/opentofu/opentofu/releases) for all new projects and workflows. :::important Consult with Your Legal Team Cloud Posse cannot provide legal advice. Organizations should consult with their legal teams to understand the implications of the BSL license on their use of Terraform. - [HashiCorp BSL License](https://www.hashicorp.com/bsl) - [HashiCorp BSL FAQ](https://www.hashicorp.com/bsl-faq) ::: ## Latest Releases - **OpenTofu**: [https://github.com/opentofu/opentofu/releases](https://github.com/opentofu/opentofu/releases) - **Terraform**: [https://github.com/hashicorp/terraform/releases](https://github.com/hashicorp/terraform/releases) ## References - Mozilla Public License (MPL) applies to HashiCorp Terraform Versions 1.5.7 and earlier: [https://www.mozilla.org/en-US/MPL/](https://www.mozilla.org/en-US/MPL/) - Business Source License (BSL) applies to HashiCorp Terraform Versions 1.6.0 and later: [https://www.hashicorp.com/bsl](https://www.hashicorp.com/bsl) - Announcement of Terraform 1.6.0 and BSL License: [https://www.hashicorp.com/blog/announcing-hashicorp-terraform-1-6](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-1-6) - OpenTofu Project: [https://opentofu.io/](https://opentofu.io/) - [OpenTofu Announces General Availability](https://www.linuxfoundation.org/press/opentofu-announces-general-availability) 2024-01-10, and ready for production use. - [OpenTofu FAQ](https://opentofu.org/faq/) - [OpenTofu Migration Guide](https://opentofu.org/docs/intro/migration/) - [Atmos OpenTofu Configuration](https://atmos.tools/core-concepts/projects/configuration/opentofu) - [Spacelift OpenTofu Configuration with Atmos](https://atmos.tools/integrations/spacelift#opentofu-support) - [Martin Atkins](https://spacelift.io/blog/two-million-and-three-things-to-celebrate-in-the-opentofu-community) - Former core contributor of HashiCorp Terraform is now a core contributor to OpenTofu. --- ## Foundational Design Decisions import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Before deploying any infrastructure, there are some fundamental design decisions of our architecture. As you get started, be aware of these foundational choices. In our reference architecture, we've made some default decisions for you, but you may want to customize these based on your specific needs. ### Review Design Decisions Review each of the following design decisions and record your decisions now. You will need the results of these decisions going forward. :::tip When working with Cloud Posse as part of our [Jumpstart](/jumpstart) or [Quickstart](/quickstart), we will review each of these decisions with you. ::: --- ## Getting Started import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from "react-player"; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import CategoryList from '@site/src/components/CategoryList'; import TaskList from '@site/src/components/TaskList'; import useBaseUrl from '@docusaurus/useBaseUrl'; ## Problem As a new engineer joining the project, you are familiar with AWS, Terraform, Kubernetes, Docker, etc, but you're not familiar with the opinionated way that Cloud Posse does it — what we call the _SweetOps_ method. There are so many tools, conventions, components, stacks, that you don't know where to get started. ## Solution :::tip Review the documentation, then start by getting your hands dirty with your first project. Don't be afraid to reach out and ask for help, if you get stuck. You'll learn much faster this way and be less overwhelmed trying to master the concepts that have taken us the better part of 7 years to develop. ::: Here you will find a quick start document for each layer of infrastructure. These documents are intended to present a common problem and a Cloud Posse solution to that problem. Also included here are common tools for Cloud Posse as well as pertinent Design Decisions with context of every decision behind the Reference Architecture. # Checklist :::info Documentation is our top priority Please let us know if anything is missing or holding you up. We'll make sure to prioritize it. ::: This guide assumes you have the following: - Terraform experience working with modules, providers, public registry, state backends, etc - AWS experience including a firm understanding of IAM, the web console, etc - Comfortable using the command line, docker, git, terraform, etc - Starting from scratch, with a new AWS account, and that you have the root account credentials If this all sounds a little bit daunting, you may want to start by reviewing the [Learning Resources](/resources/legacy/learning-resources). ### Review Foundational Design Decisions [Review Design Decisions](/layers/project/design-decisions) and record your decisions now. You will need the results of these decisions going forward. ### Create a Repository Follow our guide on [Creating a Repository](/layers/project/create-repository) to set up your GitHub repository with the proper settings, branch protection, and team access. This repository will serve as the foundation for your infrastructure code and configurations. ### Set up your toolbox container Set up your development environment by following our [Prepare the Toolbox Image](/layers/project/toolbox/) guide. Geodesic is our infrastructure automation toolbox that packages all the necessary tools into a convenient Docker image. Let's get started by creating the repository and importing the configurations provided by Cloud Posse as part of the Quickstart. If you don't have a Quickstart, consider learning more about its benefits. Next Step Learn More --- ## Prepare the Toolbox Image import Intro from '@site/src/components/Intro'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import DismissibleDialog from '@site/src/components/DismissibleDialog'; import CodeBlock from '@theme/CodeBlock'; import CollapsibleText from '@site/src/components/CollapsibleText'; import PartialDockerfile from '@site/examples/snippets/Dockerfile'; import PartialMakefile from '@site/examples/snippets/Makefile'; import Note from '@site/src/components/Note'; Geodesic is a powerful Linux toolbox container designed to optimize DevOps workflows by providing essential dependencies for a DevOps toolchain, ensuring consistency and efficiency across development environments without additional software installation on your workstation. It can be extended and customized to fit specific needs by creating your own `Dockerfile` based on Geodesic, allowing you to add your favorite tools and share the container with your team for a unified working environment. Geodesic is similar in principle to [devcontainers](https://containers.dev/). However, being a container itself, Geodesic can run anywhere containers are supported—whether on your local workstation, remotely inside clusters, or on bastion hosts. Additionally, you can use Geodesic as the base image for a devcontainer.
Geodesic in action.
Where are the configs? The configurations are available via our Quickstart Try Quickstart ## Building the Toolbox Image Build a Geodesic infrastructure container. This container that has all the tools like terraform and atmos for building infrastructure. It's built from the `Dockerfile` and there are some predefined targets defined in the `Makefile` to make this easy. Customize these for your organization. Here are examples of both for reference. {PartialDockerfile} {PartialMakefile} The standard `Makefile` includes a number of commands. In order to build the initial, complete Geodesic image, run the following: ```bash make all ``` On future builds, use `make run` to use the cached image. :::tip Alias We install a wrapper script with `make all` to your chosen namespace. For example, simply enter for given namespace to start your Geodesic container once built: ```bash acme ``` See the `install` step of the `Makefile` for more details. ::: Build the toolbox image locally before continuing. Follow the [toolbox image setup steps in the How-to Get Started guide](/layers/project/#building-the-toolbox-image). In short, run `make all`. The container will have the given local home mapped, so you should be able to use aws normally inside it once you set a profile that has valid credentials. For instance, if I log in to the profile `acme` with [leapp](https://github.com/Noovolari/leapp), I can run `aws --profile acme sts get-caller-identity` and get a response. Once you've verified that the infra container has access to aws resources, we can move on to the next step. With your repository set up, workstation configured and toolbox in hand, you're ready to get to work provisioning your infrastructure with Atmos and Terraform. The next step is to learn how to provision AWS accounts. Next Step --- ## Getting started with Geodesic v4 import Intro from '@site/src/components/Intro'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one convenient image that you can use as your infrastructure automation toolbox. We call it [Geodesic](/learn/toolchain/#geodesic) and we use it as our DevOps automation shell and as the base Docker image for all of our DevOps scripting / CI jobs. In this tutorial, we'll walk you through how to use Geodesic to execute Terraform and other tooling. We'll be sure to talk about what is going on under the hood to ensure you're getting the full picture. Geodesic v4 is the current version This documentation is for Geodesic v4, which is the current version. While it is largely the same as earlier versions, there are some significant differences, and we have retained documentation on Geodesic v3 for those who have yet to make the switch. Please be aware of which version of Geodesic and which version of the documentation you are using in case you find inconsistencies. ## Prerequisites ### System Requirements To accomplish this tutorial, you'll need to have [Docker installed](https://docs.docker.com/get-docker/) on your local machine. **That's all**. Although Geodesic is supplied as a Docker image, it is best used by installing a wrapper shell script that configures the Docker container to mount directories and files from your local machine and support running multiple `bash` shells simultaneously. To install the wrapper script, you must have write access to either `/usr/local/bin` or `$HOME/.local/bin` on your local machine, and you must have the installed directory in your `$PATH`. ### Geodesic Usage Patterns Let's talk about a few of the ways that one can run Geodesic. Our toolbox has been built to satisfy many use-cases, and each result in a different pattern of invocation: ### Install Geodesic You can **install** Geodesic onto your local machine running `make install` with the [Makefile](https://github.com/cloudposse/geodesic/blob/main/Makefile) provided in the Geodesic repository. ### Build Your Own Toolbox You can **build your own toolbox** on top of Geodesic. This is what Cloud Posse generally recommends to practitioners. We do this when we want to provide additional packages or customization to our team while building on the foundation that Geodesic provides. This is relatively to do by using Geodesic as your base image (e.g. `FROM cloudposse/geodesic:latest-debian`) in your own `Dockerfile`, adding your own Docker `RUN` commands or overriding environment variables, and then customizing the [Geodesic Makefile](https://github.com/cloudposse/geodesic/blob/main/Makefile) with your own `DOCKER_ORG`, `DOCKER_IMAGE`, `DOCKER_FILE`, and `APP_NAME` variables. (There are other variables you can customize as well, but these are the most common ones.) Then you can run `make build` to create a new image, `make install` to install the wrapper script that will run it, and then run it via the `APP_NAME` you configured. If you like, you can do this all in one step by running `make all`. ### Quick Install You can skip using `make` and just install Geodesic Example: `docker run --rm cloudposse/geodesic:latest-debian init | bash` installs `/usr/local/bin/geodesic` (or `$HOME/.local/bin/geodesic`) on your local machine which you can execute repeatedly via simply typing `geodesic`. In this example, we're pinning the script to use the `cloudposse/geodesic:latest-debian` docker image, but we could also pin to our own image or to a specific version. ### Run Standalone You can **run standalone** Geodesic as a standard docker container using `docker run`, but in this mode, Geodesic will not have access to your local machine's files, so it is less useful. Some use cases are to provide tools to debug a Kubernetes cluster by installing Geodesic as a pod in the cluster, or to use it as a CI/CD tool where the tool takes care of mounting the required files and directories. ### Interactive Shell Example Example: `docker run -it --rm --volume $PWD:/workspace cloudposse/geodesic:latest-debian --login` opens a bash login shell (`--login` is our Docker `CMD` here; it's actually just [the arguments passed to the `bash` shell](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html) which is our `ENTRYPOINT`) in our Geodesic container. ### One-Off Command Example Example: `docker run --rm cloudposse/geodesic:latest-debian -c "terraform version"` executes the `terraform version` command as a one-off and outputs the result. In this tutorial, we'll be running the installed Geodesic `geodesic` to allow us to take advantage of the wrapper script's features. ## Tutorial ### Install the Geodesic Wrapper Script First, at your terminal, let's install the Geodesic shell! ```bash # Since the "latest" tag changes, ensure we do not have a stale image docker image rm cloudposse/geodesic:latest-debian # OK if image not found docker run --rm cloudposse/geodesic:latest-debian init | bash ``` The result of running this command should look something like this: ```bash # Installing geodesic from cloudposse/geodesic:latest-debian... # Installed geodesic to /usr/local/bin/geodesic ``` ### Start the Geodesic Shell You should now be able to launch a Geodesic shell just by typing `geodesic` at your terminal: ```bash geodesic ``` ![Geodesic Login Shell](/assets/geodesic-login-shell.png) Exit it for now by typing `exit` or pressing `logout`. ### Download our Tutorial Project Great -- we've started up Geodesic so now let's do something with it. How about we pull a terraform project and apply it? To accomplish this, let's do the following: ### TODO: Continue updates from here ```bash # Change to our /localhost directory so that we can pull our project's code to our # local machine as well as our docker container cd /localhost # Clone our tutorials repository git clone https://github.com/cloudposse/tutorials # Change to our tutorial code cd tutorials/01-geodesic ``` Easy! And since we changed into our `/localhost` directory inside Geodesic, the `tutorials` project that we git cloned is available both in the container that we're running our shell in **and** on our local machine in our `$HOME` directory. This enables us to share files between our local machine and our container, which should start to give you an idea of the value of mounting `$HOME` into Geodesic. ### Apply our Terraform Project Now that we've got some code to work with, let's apply it... ```bash # Setup our terraform project terraform init # Apply our terraform project terraform apply -auto-approve ``` Sweet, you should see a successful `terraform apply` with some detailed `output` data on the original star wars hero! 😎 Just to show some simple usage of another tool in the toolbox, how about we parse that data and get that hero's name? ### Read some data from our Outputs Let's utilize [`jq`](https://github.com/stedolan/jq) to grab some info from that terraform project's output: ```bash # Pipe our terraform project's output into jq so we can pull out our hero's name terraform output -json | jq .star_wars_data.value.name ``` Again, without having to install anything, we've grabbed a tool from our toolbox and were able to use it without a second thought. ## Conclusion The beautiful thing about all of this is that we didn't need to install anything except Docker on our local machine to make this happen. Tools like `git`, `terraform`(all versions), and `jq` all involve specific installation instructions to get up and running using the correct versions across various machines/teams, but by using Geodesic we're able to quickly skip over all of that and use a container that includes them out of the box alongside [dozens of other tools as well](https://github.com/cloudposse/packages/tree/master/vendor). And with the mounting of our `$HOME` directory to `/localhost` of the container, our Geodesic shell just ends up being an extension of our local machine. That is why we call it a toolbox as it enables consistent usage of CLI tools across your entire organization! If you want to see another usage of Geodesic, [read our next tutorial in the SweetOps series about one of our most important tools: `atmos`.](https://atmos.tools/quick-start/introduction) --- ## Poly-Repo Strategy with account-map import Intro from '@site/src/components/Intro'; import Note from '@site/src/components/Note'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; When managing multiple repositories with Terraform and Atmos, you need a strategy to handle `account-map` configurations effectively. This tutorial shows you how to implement a poly-repo strategy that maintains consistency across your infrastructure while leveraging `account-map` for dynamic role selection. ## Problem You want many repositories to deploy infrastructure as code with Terraform and Atmos. However, the standard `providers.tf` configuration with Cloud Posse components requires `account-map` to dynamically select Terraform roles and target accounts. ## Solution The poly-repo strategy with `account-map` involves maintaining a central infrastructure repository as the source of truth for account mappings, while configuring service repositories to use static `account-map` configurations. This approach enables team autonomy while ensuring consistent account mappings across your infrastructure. ## Use the infrastructure Repo for core services Use the primary infrastructure repository to deploy all accounts, Terraform state, and core services: - Deploy `account-map` here - Download the `account-map` Terraform output to use later - Optional, highly recommended: Use a custom atmos command to download the `account-map` output as an artifact ```yaml # atmos.yaml commands: - name: download-account-map description: This command downloads the Terraform output for account-map. env: - key: AWS_PROFILE value: cptest-core-gbl-root-admin # change this to a role you can assume with access to account-map steps: # initialize terraform and selects workspace - "atmos terraform workspace account-map core-gbl-root -s core-gbl-root" # download the terraform output for account-map. limit atmos to only the JSON output - "atmos terraform output account-map -s core-gbl-root --logs-level Off --redirect-stderr=/dev/null --skip-init -- -json | jq 'map_values(.value)' | yq -P -oy > components/terraform/account-map/account-map-output.{{ now | date \"2006-01-02\" }}.yaml" ``` You will need to update this artifact each time `account-map` changes. This is fairly infrequent, and there are potential solutions for optimization yet to be considered. ## Mock `account-map` in each poly-repos In each of the poly-repos, we need to configure `account-map` to use a static backend. We want remote-state to read a static YAML configuration when referring to `account-map`, rather than reading the terraform backend. 1. Download the `account-map` component (vendoring recommended). We do not need to apply this component, but we do need the submodules included with the component for the common `providers.tf` configuration in all other components 2. Create a component instance for `account-map` -- typically with the stack catalog `stacks/catalog/account-map.yaml` 3. Set `remote_state_backend_type: static` 4. Paste the `account-map` YAML output under `remote_state_backend.static`. I'd recommend using imports to manage the output artifact separately or other ways to make it easier to understand and update. Done! Now when remote-state refers to the `account-map` component, it will instead check the static remote state backend rather than S3. ```yaml components: terraform: account-map: remote_state_backend_type: static remote_state_backend: static: PASTE OR INCLUDE YAML HERE (make sure to set indentation) ``` ## Frequently Asked Questions This tutorial is based on a [GitHub discussion](https://github.com/orgs/cloudposse/discussions/49) about splitting up Atmos and Terraform into service repos. ### How do you run the components used within the non-infrastructure repos? Is the directory format the same? You can use components in the same way as the infrastructure repo, but this would be dependent on your atmos configuration. You will need to set the base paths for components and stacks, then you execute atmos as usual. ### Is it possible to reuse components in the infrastructure repo? At the moment no. However, we are developing just-in-time vendoring for components where you would be able to specify a remote source for a component in stack config. It's not generally available yet -- stay tuned. ### How does the GitHub action run? Does it assume the same terraform IAM role? We usually deploy an aws-teams role (typically called gitops) for GitHub actions. That team is able to plan and/or apply Terraform by assuming the standard terraform from aws-team-roles. The infrastructure repo assumes the gitops team using GitHub OIDC. You have a few options. You could reuse the same gitops role by adding the new repos to that allowed repos list. However, you may want finer scoped privileges in the alternate repos. In which case you could create an additional AWS Team with limited access. ### What about guardrails? How do you only allow updates to specific components or resources? Both by included components and AWS Team permission. The repo would only include components it is responsible for managing, and therefore would only be aware of that set of components. However, you should still scope the AWS Team to least privilege required for that set of resources. You could separate the Terraform state backend as well to create an additional boundary between the sets of resources. ### How do you prevent conflicts with the same component deployed in the same stack across multiple repos? We don't recommend deploying the same component from the same stack across repos. The idea with poly-repos is that an app team can manage their own infrastructure independently. We wouldn't recommend controlling the same infrastructure with 2 separate sources of code. --- ## Tutorials(8) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the foundational components. --- ## Decide on Infrastructure & Software Static Analysis Tools import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Infrastructure Considerations: (terraform, docker) - checkov (open source alternative by bridgecrew; works with github actions) - bridgecrew (managed service - acquired by Paloalto Networks) - tflint - tfsec - conftest ## Software Static Analysis - Sonatype - Sonarqube - Snyk - WhiteSource - JFrog --- ## Decide on Kubernetes Platform Compliance Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Decide on a strategy for CIS Compliance/hardening on EKS. ## Considerations - [https://github.com/aquasecurity/kube-bench](https://github.com/aquasecurity/kube-bench) (integrates with Security Hub [https://aws.amazon.com/about-aws/whats-new/2020/12/aws-security-hub-adds-open-source-tool-integration-with-kube-bench-and-cloud-custodian/](https://aws.amazon.com/about-aws/whats-new/2020/12/aws-security-hub-adds-open-source-tool-integration-with-kube-bench-and-cloud-custodian/)) - [https://aws.amazon.com/about-aws/whats-new/2022/01/amazon-guardduty-elastic-kubernetes-service-clusters/](https://aws.amazon.com/about-aws/whats-new/2022/01/amazon-guardduty-elastic-kubernetes-service-clusters/) - [https://snyk.io/](https://snyk.io/) - [https://www.rapid7.com/](https://www.rapid7.com/) --- ## Decide on Log Retention and Durability Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Not all logs were created equal. Some may contain PHI (Protected Health Information) or CHD (Card Holder Data) while others are simply HTTP request logs. Depending on regional jurisdiction (E.g. Europe), there can be other requirements (E.g. [GDPR on AWS](https://docs.aws.amazon.com/whitepapers/latest/navigating-gdpr-compliance/monitoring-and-logging.html)). We need to identify the log destinations to discuss how to handle them. ### Recommendations - Use 90 days unless the compliance framework stipulates differently (e.g. PCI/DSS) - Use 60 days in the standard S3 storage tier and then transfer to glacier for the last 30 days. ## Considerations ### Which Logs are in Scope? - **Cloud Trails** (AWS API logs) - **CloudWatch** Logs (platform logs) - **Datadog** Logs (logs stored in datadog) - **Web Access Logs** (e.g. ALBs) - **WAF** Logs - **Shield** Logs - **VPC flow logs** (these are huge - every packet that flows through the VPC) - **Application logs** (the events emitted from your applications) ### How are logs handled? For everything in scope, we need to address: - Where should logs be aggregated (e.g. S3 bucket in audit account, datadog)? - Do logs need to be replicated to any backup region? - Should logs be versioned? - Which logs should get forwarded to Datadog? (e.g. ALBs, flow logs, cloud trails, etc) - How long is the log data online (e.g. easily accessible vs cold storage like [Glacier](https://aws.amazon.com/s3/storage-classes/glacier/))? - Does any of the data contain PHI, PII, CHD, etc.? - Are there any data locality, data residency restrictions on logs? e.g. Can logs pass regional boundaries (E.g. for GDPR compliance logs may need to stay in EU)? - What’s the tolerance for the latency of accessing log events (e.g. can it be hours or needs to be within seconds)? ### References - [https://docs.aws.amazon.com/whitepapers/latest/navigating-gdpr-compliance/monitoring-and-logging.html](https://docs.aws.amazon.com/whitepapers/latest/navigating-gdpr-compliance/monitoring-and-logging.html) - [https://www.pcidssguide.com/what-are-the-pci-dss-log-retention-requirements/](https://www.pcidssguide.com/what-are-the-pci-dss-log-retention-requirements/) - [https://aws.amazon.com/s3/storage-classes/glacier/](https://aws.amazon.com/s3/storage-classes/glacier/) --- ## Decide on Strategy for Hardened Base AMIs import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem When critical CVEs come out, they should be promptly remediated. If we rely strictly on upstream AMIs, we have little control over the time to remediate as we’re dependent on third parties to remediate the vulnerabilities and their failure to do so will jeopardize our ability to meet commitments. Many compliance frameworks require host-level hardening and the ability to demonstrate something along the lines of [https://www.cisecurity.org/controls/continuous-vulnerability-management/](https://www.cisecurity.org/controls/continuous-vulnerability-management/). Laws, regulations, standards, or contractual agreements may require an even higher priority or shorter timeline for remediation. For example, to comply with the [Payment Card Industry Data Security Standard (PCI DSS)](https://www.pcisecuritystandards.org/document_library), vulnerabilities in any PCI environment: - CVSS scores of 4 or higher must be remediated within 30 days of notification. - CVSS scores of less than 4 must be remediated within two to three months. See [Decide on Technical Benchmark Framework](/layers/security-and-compliance/design-decisions/decide-on-technical-benchmark-framework) ## Solution We need a solution that covers both EKS (for customers using it) and for standalone EC2 instances where applicable. Additionally, regardless of the solution, we'll also need to instrument the process for rolling out the changes. See [How to Enable Spacelift Drift Detection](/resources/deprecated/spacelift/tutorials/how-to-enable-spacelift-drift-detection) for a nice way to automatically update AMIs using data sources. ### Use CIS or Not? > CIS benchmarks are internationally recognized as security standards for defending IT systems and data against cyberattacks. Used by thousands of businesses, they offer prescriptive guidance for establishing a secure baseline configuration.The CIS Foundation is the most recognized industry standard for hardening OS images, however, they have not yet published the CIS standard for container-optimized OS. The traditional CIS benchmarks are for full-blown OSs with a different set of concerns that do not apply to a container-optimized OS. What CIS has defined are [the best practices for hardening EKS as a platform](https://aws.amazon.com/de/blogs/containers/introducing-cis-amazon-eks-benchmark/) and that standard is covered by `kube-bench`. So by running `kube-bench` on a cluster we would be able to validate if Bottlerocket meets the CIS standard for nodes managed by Kubernetes. While this is not the same as "certification", it might be good enough for benchmark compliance. ### Use Existing Hardened Image - AWS does not provide turnkey CIS-compliant base AMIs (third-party vendors only). - Bottlerocket is more secure but is still not _technically_ CIS-compliant out of the box [https://github.com/bottlerocket-os/bottlerocket/issues/1297](https://github.com/bottlerocket-os/bottlerocket/issues/1297) - [https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami-bottlerocket.html](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami-bottlerocket.html) - [https://aws.amazon.com/about-aws/whats-new/2021/10/amazon-eks-nodes-groups-bottlerocket/](https://aws.amazon.com/about-aws/whats-new/2021/10/amazon-eks-nodes-groups-bottlerocket/) - [CIS provides marketplace images](https://aws.amazon.com/marketplace/seller-profile?id=dfa1e6a8-0b7b-4d35-a59c-ce272caee4fc), but these add $0.02/hour. ### DIY Hardened Images - Build our own AMIs based on something like Bottlerocket or Amazon Linux and do our own hardening. - Any hardening we do would necessitate the implementation of the packer configurations and pipelines for managing it. - Create GitHub Action pipeline to build packer images and distribute them to enabled regions ### Cloud-Init Patching With `cloud-init` we can patch the system at runtime. This has the benefit of not requiring us to manage any complicated factory for building AMIs for multiple regions but violates the principle of immutable infrastructure. ### AWS Systems Manager Patch Manager With AWS Systems Manager can apply patch documents to running systems based on policies, but violates the principle of immutable infrastructure. --- ## Decide on a Technical Benchmark Framework for Compliance import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Benchmark Considerations - SOC2 Type II - HIPAA - HITRUST - PCI/DSS - CIS - NIST - ISO27001 - AWS Well-Architected ## SOC2 Considerations SOC2 defines a set of high-level expectations, but it’s up to the responsible party (e.g. Customer) to assert what controls are in place for each pillar. 1. **Logical and physical access controls** 2. **System operations** 3. **Change management** 4. **Risk mitigation** Using a combination of one or more of the compliance standards such as CIS, HITRUST, NIST, ISO27001, etc is the typical approach. Organizationally, this is a decision that has both technical and procedural impacts. The Technical Benchmark Framework should satisfy the vast majority of requirements for both HIPAA and SOC2, which means most likely selecting more than one. ### Questions - Has the team already started mapping out any of SOC2 controls that would influence technical controls or configurations? --- ## Decide on WAF Requirements/Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem ALBs have very limited capabilities to fend off attacks by themselves. Using Security Groups is not a scalable solution. [The number of inbound/outbound rules is limited to a max of 120 (60 ea)](https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html#vpc-limits-security-groups). To thwart any sort of Denial of Service (DoS) attack, more tools are required. Moreover, not all attacks are as easily identified as DoS attacks. Other threat vectors include SQL injection, XSS, etc. The older your applications, the more external dependencies you have, the greater the attack surface area. ## Solution Deploy a Web Application Firewall (WAF) capable of performing Layer-7 inspection and mitigation. :::info Our recommendation is to deploy the AWS WAF with the AWS Managed Rules for the [https://owasp.org/www-project-top-ten/](https://owasp.org/www-project-top-ten/). [https://aws.amazon.com/marketplace/solutions/security/waf-managed-rules](https://aws.amazon.com/marketplace/solutions/security/waf-managed-rules) ::: ## Considerations - ALB/NLB won’t provide TLS in-transit with nitro instances - AWS WAF only works with ALBs - We need to terminate TLS at the Kubernetes ingress layer (e.g. with cert-manager and ZeroSSL) in order to deliver end to end encryption - Cloudflare Argo tunnel will work without exposing the cluster endpoints directly to the internet Our recommendation is to use AWS WAF with ALB load balancers, then use AWS Nitro instances for e2e encryption inside EKS, and self-signed certs between the ALB and the pods. ## References - [https://github.com/cloudposse/terraform-aws-waf](https://github.com/cloudposse/terraform-aws-waf) --- ## Design Decisions(6) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for how you'll monitor for security and compliance by leveraging the full suite of AWS security-oriented services. --- ## FAQ(Security-and-compliance) ## Q: Error: error disabling security hub control ```bash Error: error disabling security hub control arn:aws:securityhub:us-west-2:267314709865:control/aws-foundational-security-best-practices/v/1.0.0/EC2.17: InvalidAccessException: Account 267314709865 is not subscribed to AWS Security Hub │ { │ RespMetadata: { │ StatusCode: 401, │ RequestID: "53bc6538-18ff-4df0-9c14-57afe7b1d1a0" │ }, │ Code_: "InvalidAccessException", │ Message_: "Account 267314709865 is not subscribed to AWS Security Hub" │ } │ │ with awsutils_security_hub_control_disablement.ec2_multiple_enis[0], │ on main.tf line 138, in resource "awsutils_security_hub_control_disablement" "ec2_multiple_enis": │ 138: resource "awsutils_security_hub_control_disablement" "ec2_multiple_enis" { ``` A: None of the commands in the final step have been run. Check the commands and run them again. ## Q: Deploying `compliance` to `security` fails on Step 1 ```text │ Error: error designating security hub administrator account members: BadRequestException: │ status code: 400, request id: 7279a38b-7bff-48b5-834c-f8e3ab0d4bf7 │ │ with awsutils_security_hub_organization_settings.this[0], │ on main.tf line 72, in resource "awsutils_security_hub_organization_settings" "this": │ 72: resource "awsutils_security_hub_organization_settings" "this" { ``` or ```text │ Error: error updating security hub administrator account settings: InvalidAccessException: Account 174772947570 is not an administrator for this organization │ { │ RespMetadata: { │ StatusCode: 401, │ RequestID: "335a2c87-c5bf-4006-9017-50a4428fcc30" │ }, │ Code_: "InvalidAccessException", │ Message_: "Account 174772947570 is not an administrator for this organization" │ } ``` A: Please re-deploy and set the correct input vars to `false` , then deploy compliance-root, then redeploy with the correct input vars to `true` and it should work. ## Q: Deploying `compliance` to `security` fails on Step 3 `designating guardduty administrator account members` ```text │ Error: error designating guardduty administrator account members: BadRequestException: The request is rejected because the input detectorId is not owned by the current account. │ { │ RespMetadata: { │ StatusCode: 400, │ RequestID: "e8d39bc9-2f1f-4d95-8283-6193facba4d3" │ }, │ Message_: "The request is rejected because the input detectorId is not owned by the current account.", │ Type: "InvalidInputException" │ } ``` A: This error may not be related to GuardDuty delegation, but rather the `awsutils` provider configuration. For example: ```hcl provider "awsutils" { region = var.region profile = module.iam_roles.profiles_enabled ? coalesce(var.import_profile_name, module.iam_roles.terraform_profile_name) : null dynamic "assume_role" { for_each = module.iam_roles.profiles_enabled ? [] : ["role"] content { role_arn = coalesce(var.import_role_arn, module.iam_roles.terraform_role_arn) } } } ``` ## Q: Deploying `compliance` to `security` fails on Step 3 `designating security hub administrator account members` ```text │ Error: error designating security hub administrator account members: [{ │ AccountId: "1234567890", │ ProcessingResult: "Operation failed because your organization master must first enable SecurityHub to be added as a member" │ }] ``` A: Security Hub must be enabled for the Organization. Typically this is done with the `account` component by adding `securityhub.amazonaws.com` to `aws_service_access_principals`. Alternately, it can be manually enabled. Open the AWS console for the `root` account within the given Organization, go to Security Hub for the appropriate region, click “Enable Security Hub” under “AWS Integrations” ## Q: `Blank spaces are not acceptable for input parameter: policyARN` ```text │ Error: Error creating AWSConfig rule: Failed to create AWSConfig rule: InvalidParameterValueException: Blank spaces are not acceptable for input parameter: policyARN. │ │ with module.aws_config.aws_config_config_rule.rules["iam-policy-in-use"], │ on .terraform-mdev/modules/aws_config/main.tf line 42, in resource "aws_config_config_rule" "rules": │ 42: resource "aws_config_config_rule" "rules" {│ ``` A: The `support` role may not be deployed into the given account. Check `aws-team-roles` for the `support` role. --- ## Security and Compliance import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; ## Problem We need to ensure your AWS Organization meets certain benchmarks for compliance (e.g. HIPAA, PCI/DSS, etc). Your AWS accounts contain thousands of resources, making it impossible to audit without automation. Setting up the automation by hand is tedious and error-prone. ## Solution Deploy a set of Cloud Posse components to enable SecurityHub, GuardDuty, AWS Config, and more! :::caution It is very important that you read these docs and follow all of the steps carefully and in order. Failure to do so may result in a condition that needs to be manually cleaned up across tens of regions in each of your accounts. ::: ### Related Components - [aws-config](/components/library/aws/aws-config/) - [guardduty](/components/library/aws/guardduty/) - [security-hub](/components/library/aws/security-hub/) The older `compliance` and `compliance-root` components are deprecated as of July 2023. We recommend moving to our new [`aws-config`](/components/library/aws/aws-config/), [`guardduty`](/components/library/aws/guardduty/), and [`security-hub`](/components/library/aws/security-hub/) components. ### Background Info The AWS Config, Security Hub, and GuardDuty services must be configured in every enabled region in each of your AWS accounts. Unfortunately, [AWS does not support](https://docs.aws.amazon.com/general/latest/gr/rande-manage.html) disabling regions which were introduced prior to March 20, 2019. These regions are enabled by default. The table below lists the regions that cannot be disabled. :::info In addition to the regions below, the AWS services listed above must also be deployed to any regions you opted into. ::: | | | | | | -------------- | -------------- | --------- | --------- | | ap-northeast-1 | ap-southeast-2 | eu-west-2 | us-west-1 | | ap-northeast-2 | ca-central-1 | eu-west-3 | us-west-2 | | ap-northeast-3 | eu-central-1 | sa-east-1 | | | ap-south-1 | eu-north-1 | us-east-1 | | | ap-southeast-1 | eu-west-1 | us-east-2 | | The Cloud Posse AWS Config, Guard Duty, and Security Hub components (the "Components") are responsible for deploying these AWS services in a sane way. At the AWS Organizational level, the Components designate an account to be the primary account within the AWS Organization responsible for configuring the service. This is referred to as the Delegated Administrator account. In addition, where possible, the Components designate a single region to be the “central collection region” so that compliance information can be aggregated into a single region. In a typical REFARCH setup, the logs are written to the `audit` account, the AWS services (Config, Security Hub, and GuardDuty) are deployed to the `security` account, and the Organizational management account (`root` account) is used to delegate `security` as the Delegated Administrator account. The central collection region is usually the same as the primary region used in your platform accounts for hosting your application workloads. --- ## Setup Security and Compliance import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints' import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Admonition from '@theme/Admonition' import Note from '@site/src/components/Note' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; These are the setup instructions for Security and Compliance in your AWS Organization. ## Organization-Level Configuration The following steps are required to set up Security and Compliance in your AWS Organization. They only be completed once for the entire organization. ### Vendor Components Vendor all data components with the following workflow: ### Add Service Principals to the `account` component Add the following service principals to the `aws_service_access_principals` variable of the `account` in `stacks/catalog/account.yaml`: - `config.amazonaws.com` - `config-multiaccountsetup.amazonaws.com` - `guardduty.amazonaws.com` - `securityhub.amazonaws.com` The following command requires [SuperAdmin](/layers/accounts/tutorials/how-to-create-superadmin-user/). Ensure the `plan` output only touches service principals. ```bash atmos terraform plan account -s core-gbl-root ``` The output of `plan` should look similar to the following: ```1vRLXlhggjCF4VZ09qvOF9nLoxi__0EVFgDjgtfLlSRZfzPbd5ZhvzOLSVHECM015A7SCNPJoCeXPb3X # aws_organizations_organization.this[0] will be updated in-place ~ resource "aws_organizations_organization" "this" { ~ aws_service_access_principals = [ + "config-multiaccountsetup.amazonaws.com", + "config.amazonaws.com", # (8 unchanged elements hidden) ] id = "[random string]" # (9 unchanged attributes hidden) } Plan: 0 to add, 1 to change, 0 to destroy. ``` Apply the changes with: ```bash atmos terraform apply account -s core-gbl-root ``` ### Deploy `config-bucket` There should only be a single bucket that will act as a store for AWS Config data across all regions The `config-bucket` is required for storing `AWS Config` data and is a pre-requisite for deploying AWS Config. See [config-bucket](https://github.com/cloudposse/terraform-aws-config-storage) ```bash atmos terraform plan config-bucket --stack core-use1-audit atmos terraform apply config-bucket --stack core-use1-audit ``` ### Deploy `cloudtrail-bucket` There should only be a single bucket that will act as a store for AWS CloudTrail data across all regions Deploying the `cloudtrail-bucket` to the Audit account allows the Organization to isolate audit data and permissions from other environments (production, staging, development) and is a requirement for deploying Security Hub. See [cloudtrail-bucket](https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket) This bucket has likely been provisioned by the [Cold Start](/layers/accounts/account-baseline/). Run the following `terraform plan` to ensure the bucket exists. If it doesn't, create it with `terraform apply`. ```bash atmos terraform plan cloudtrail-bucket --stack core-use1-audit atmos terraform apply cloudtrail-bucket --stack core-use1-audit ``` ### Deploy IAM Role for CIS Benchmark Part of the benchmark rules for CIS AWS Foundations includes deploying a support role to manage incidents with AWS Support. See [CIS Benchmark 1.20](https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-cis-controls.html#securityhub-cis-controls-1.20) and [IAM.18](https://docs.aws.amazon.com/securityhub/latest/userguide/iam-controls.html#iam-18). These roles are managed from the [Identity Workflow](/layers/identity/deploy/) using `aws-teams` and `aws-team-roles` components. ## AWS Config :::info If a step in one of the following workflows fails you can restart from that failed step by using the following command: ```bash atmos workflow deploy/aws-config/global-collector -f compliance --from-step step4 ``` ::: ### Set up AWS Config globally Deploy AWS Config to each region in order to collect data for global resources such as IAM. This command requires [`SuperAdmin`](/layers/accounts/tutorials/how-to-create-superadmin-user/). ### Set up AWS Config for SuperAdmin accounts Deploy AWS Config into accounts that require SuperAdmin to apply. ## Security Hub ### Set up the Delegated Administrator account First, deploy to each region of the Delegated Administrator account. ### Set up the Organization Management account Next, using [SuperAdmin](/layers/accounts/tutorials/how-to-create-superadmin-user/), deploy to the Organization Management (root) account in order to designate the `security` account as the Organization Delegated Administrator account. ### Assume the identity role `assume-role acme-identity` ### Configure Security Hub organization-wide Finally, deploy the `security-hub/org-settings` component to the `security` account in order to enable and configure Security Hub in all other accounts and regions. ## GuardDuty ### Set up the Delegated Administrator account First, deploy to each region of the Delegated Administrator account. ### Set up the Organization Management account Next, deploy to the Organization Management (root) account in order to designate the `security` account as the Organization Delegated Administrator account. ### Configure GuardDuty organization-wide Finally, deploy to the `security` account in order to enable and configure GuardDuty in all other accounts and regions. ## Route53 DNS Resolver Firewall ### Set up DNS Firewall buckets Deploy the required S3 buckets for Route53 DNS Resolver Firewall logging. ### Configure the DNS Firewall Deploy and configure the Route53 DNS Resolver Firewall. ## AWS Shield ### Set up AWS Shield Advanced :::info An [AWS Shield Advanced subscription](https://docs.aws.amazon.com/waf/latest/developerguide/enable-ddos-prem.html) is required in each `plat` AWS account before running this workflow. ::: Deploy AWS Shield Advanced protection. ## AWS Inspector v2 ### Set up the Delegated Administrator account Delegate Administration account for [AWS Inspector v2](https://docs.aws.amazon.com/inspector/latest/user/what-is-inspector.html) to `core-security` for all regions. ### Configure Inspector organization-wide Enable Inspector in all regions across accounts. --- ## Enable GuardDuty for EKS Protection import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; ## Problem The new feature, GuardDuty for EKS Protection, expands coverage to continuously monitor and profile Amazon Elastic Kubernetes Service (EKS) workload activity to identify malicious or suspicious behavior representing potential threats to container workloads. ## Solution Enable GuardDuty for EKS Protection in the `security` account via ClickOps. Under Guard Duty in the AWS Console, go to “Kubernetes Protection” and enable “Kubernetes Audit Logs Monitoring” for both (1) this account **and** for (2) all your active member accounts - - - --- ## Tutorials(9) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the associated Security & Compliance components. --- ## Decide How to distribute Docker Images import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Choosing the right method to distribute Docker images is crucial for efficient deployment and management.There are various options, including AWS ECR, GitHub Container Registry, DockerHub, Artifactory/Nexus, and self-hosted registries, with multiple advantages and drawbacks. #### Use AWS ECR This is by far the most common approach we see taken. Our typical implementation includes a single ECR in the automation account. Read-only access granted to other accounts as necessary. Push images with commit sha’s and stage tag. Use lifecycle rules on stage tag to avoid eviction. The main downside with ECR, is each image repository in the registry must be explicitly provisioned. If we decide to go with ECR, we’ll also want to [Decide on ECR Strategy](/layers/project/design-decisions/decide-on-ecr-strategy) . #### Use GitHub Container Registry [https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-docker-registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-docker-registry) #### Use Dockerhub DockerHub is well suited for public images because it’s the default registry, however, images are aggressively rate limited for anonymous pulls, and no longer recommended. Additionally, as a private registry, it’s a bit dated and requires static credentials, unlike with ECR. One nice thing DockerHub is repositories do not need to be explicitly created. #### Use Artifactory/Nexus/etc This is more common for traditional artifact storage in Java shops. We don’t see this typically used with Docker, but it is supported. #### Self-hosted Registries (e.g. Quay, Docker Registry, etc) We don’t recommend this approach because, at the very least, we’ll need to use something else like ECR for bootstrapping. --- ## Decide on Argo CD Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Deciding on the architecture for Argo CD involves considering multiple clusters, plugin management, and Kubernetes integration. This document presents recommended strategies, potential risks, and common deployment patterns for Argo CD. ## Context Argo CD is a specialized tool for continuous delivery to Kubernetes, akin to how Terraform Cloud focuses on Terraform deployments. Argo CD does not support deployments outside of Kubernetes (e.g., uploading files to a bucket). While it supports plugins, these are not intended to extend its capabilities beyond Kubernetes. ## Considerations - Deploy multiple Argo CD instances across clusters to facilitate systematic upgrades in different environments. - Argo CD operates as a single pod, requiring disruptive restarts to add or upgrade plugins. - Restarts of Argo CD are disruptive to deployments. - Increasing the number of Argo CD servers complicates visualizing the delivery process. - Each Argo CD server must integrate with every cluster it deploys to. - Argo CD can deploy to the local cluster by using a service account. ### Pros - Simplifies dependency management across components. - Protects the KubeAPI by reducing public access requirements. - Provides a powerful CD tool for Kubernetes with multiple pod rollout strategies. - Offers a user-friendly UI and supports diverse deployment toolchains within the Argo CD Docker image. - Enables faster deployments and "backup Kubernetes cluster" capabilities. - Establishes a consistent framework for continuous deployment independent of the CI platform. ### Cons - Asynchronous deployments can break the immediate feedback loop from commit to deployment. - Application CRDs must reside in the namespace where Argo CD runs. - Application names must be unique per Argo CD instance. - Custom toolchains require custom Docker images, necessitating Argo CD redeployment. - Redeploying Argo CD can disrupt active deployments. - Plugin updates require redeployment since tools must be included in the Docker image. - Access management involves multiple levels (e.g., GitHub repo access, Argo CD projects, RBAC), introducing complexity. - Requires additional self-hosted solutions compared to simpler CI-based deployments with Helm 3. - Repository management for private repos in Argo CD lacks a declarative approach, needing research for potential patterns. - Argo CD's lifecycle becomes part of the critical path for deployments. ## Recommendations - **Deploy one Argo CD instance per cluster** to simplify upgrades and manage disruptions effectively. - **Use a single Argo CD instance for all namespaces within a cluster** to centralize deployment management and reduce complexity. - **Adopt a dedicated repository strategy** managed by Terraform via the GitHub Terraform provider: - One repository for production environments. - One repository for non-production environments. - One repository for preview environments. - **Avoid using plugins**: - Commit raw manifests (e.g., rendered from Helm templates or Kustomize) directly to the repository. - Shift manifest rendering to CI to ensure predictable, verifiable deployments. - This approach simplifies troubleshooting, avoids plugin upgrade issues, and ensures complete visibility into what is deployed. - **Deploy operators that require IAM roles and backing services with Terraform**, not Argo CD, to ensure proper role management and infrastructure provisioning. - **Use Argo CD for application deployments** in combination with GitHub Actions to streamline deployment pipelines and align with CI/CD best practices. - **Use Helm to Provision Argo CD** with Terraform --- ## Decide on ArgoCD Deployment Repo Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ### Context ArgoCD synchronizes the state of Kubernetes with some repo/branch/directory in your VCS system. There’s no canonical way to do it. There are many variations as well. ## Examples - 1 repo per cluster - 1 repo for multiple clusters, multiple repos for multiple groups of clusters - 1 branch per cluster - 1 directory per cluster ## Considerations The more repos, the harder to update multiple clusters at a time The more clusters in one repo, the larger the git commit history With one repo per cluster, every time a new clsuter is created, a new repo needs to be created as well. With fewer repos, the more contention working with Git. Git sucks for high throughput. ## Recommendation Our recommendation is ~3 repos, with multiple clusters in each: - prod (all production clusters) - use main branch to represent deployed state for all clusters - use one directory per cluster, namespace - use branch protections to restrict commits - non-prod (all non-production clusters) - use main branch to represent deployed state for all clusters - use one directory per cluster, namespace - use branch protections to restrict commits - preview (all preview environments) - avoid polluting the git history - no branch protections ## Related - [Decide on ArgoCD Architecture](/layers/software-delivery/design-decisions/decide-on-argocd-architecture) --- ## Decide on Branching Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement ## Problem Statement A branching strategy is crucial in software development for facilitating team collaboration, maintaining code stability, enabling Continuous Integration/Continuous Deployment (CI/CD), ensuring code quality through reviews, simplifying change tracking and rollbacks, supporting parallel development, and managing different product releases. It provides individual workspaces for developers, segregates code based on development stages, automates testing and deployment processes, allows for scrutinized merges into the main codebase, and supports simultaneous development and maintenance of multiple product versions. Essentially, an effective branching strategy underpins a stable, manageable, and efficient development environment. ## Options ## Gitflow [Gitflow](https://nvie.com/posts/a-successful-git-branching-model/) has gained popularity as it provides an efficient framework for collaborative development and scaling development teams. The Gitflow model offers robust control mechanisms and facilitates collaborative development at a price of a slowdown development speed and higher numbers of merge conflicts. Teams working on projects with a massive codebase, low level of automation, and minimum test code coverage and teams with a high percentage of junior devs all benefit from the Gitflow strategy. However, Gitflow may not be suitable for startups where development speed is the priority. ```mermaid --- title: Gitflow --- gitGraph commit tag: "0.1.0" branch hotfix branch develop order: 2 commit commit checkout develop branch featureA order: 2 branch featureB order: 2 checkout main checkout featureA commit commit checkout hotfix commit checkout main merge hotfix tag: "0.1.1" checkout featureB commit commit checkout develop merge featureA branch release order: 1 commit commit checkout develop merge release checkout release commit checkout featureB commit checkout develop merge release checkout main checkout release checkout main merge release tag: "0.2.0" checkout develop merge featureB checkout main merge develop tag: "0.3.0" checkout develop commit ``` ## GitHub Flow (Trunk-based) [Github Flow](https://docs.github.com/en/get-started/quickstart/github-flow) is a lightweight branching strategy where developers introduce changes as Pull Requests from short-lived, ephemeral feature branches. Finally, the PRs merged into the single "trunk" branch that is the source of truth. GitHub Flow works particularly well with a team of experienced developers, facilitating the quick introduction of improvements without unnecessary bureaucracy. It fits with high automation of release engineering processes, microservices architecture, and mature engineering culture. ```mermaid --- title: GitHub Flow --- gitGraph commit tag: "0.1.0" branch release/0.1.0 order: 4 commit type: HIGHLIGHT id: "0.1.0" checkout main commit checkout main branch feature order: 1 checkout main commit tag: "0.2.0" branch release/0.2.0 order: 4 commit type: HIGHLIGHT id: "0.2.0" branch hotfix order: 4 checkout hotfix commit checkout feature commit commit checkout main checkout feature commit checkout hotfix commit checkout release/0.2.0 merge hotfix tag: "0.2.1" checkout main merge feature checkout main merge release/0.2.0 checkout main commit tag: "0.3.0" branch release/0.3.0 order: 5 commit type: HIGHLIGHT id: "0.3.0" checkout main commit ``` ## Recommendation We recommend using Github Flow as a basic branching strategy in conjunction with СI/CD workflows and [branches rulesets](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-rulesets/about-rulesets) for better quality and control over changes. We highly recommend avoiding long-living branches, especially for software that does not have to support multiple versions of the software running in the wild. The branching strategy does not have to be the same across the whole organization - different teams can use different flows. However, that often leads to complexity in the CI/CD pipelines and reduces reusability. For these reasons, we recommend a consistent branching strategy, at a minimum on a team level. ## References - [A successful Git branching model](https://nvie.com/posts/a-successful-git-branching-model/) - [Trunk-Based Development vs Git Flow: When to Use Which Development Style](https://blog.mergify.com/trunk-based-development-vs-git-flow-when-to-use-which-development-style/) - [Long-lived branches with Gitflow](https://www.thoughtworks.com/radar/techniques/long-lived-branches-with-gitflow) --- ## Decide on Customer Apps for Migration import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We need to identify an application and its associated services that are suitable as the first candidate for migration to the new platform. This is an application that will be targeted by all the release engineering work. We can pretty much handle anything, but ideal candidates have these characteristics: [https://cloudposse.com/12-factor-app/](https://cloudposse.com/12-factor-app/) :::tip Our recommendation is to start with a model application template repository so we do not impact any current CI/CD processes during the development of GitHub Action workflows. ::: Apps that do not have these characteristics may require more engineering effort. Using any existing repository will pose a risk of triggering GitHub events (E.g. pull requests, releases, etc) that other existing CI/CD systems (e.g. Jenkins, CircleCI, etc) will respond to. Furthermore, several GitHub Action events only work on the default branch (e.g. `main`) and for this reason, we will need to merge to PRs to test the end-to-end process. For this reason, we recommend starting with a model application template repository that your team can use to document and train others on your CI/CD process. Completing the migration workbook will help identify suitable applications. Our workbook template is here [https://docs.google.com/spreadsheets/d/1CDcJosaqoby2Fq2AmZnf-xRizI4pcc-sqpi04ggHqSI/edit#gid=863544204](https://docs.google.com/spreadsheets/d/1CDcJosaqoby2Fq2AmZnf-xRizI4pcc-sqpi04ggHqSI/edit#gid=863544204) and can be copied and shared. Our goal is to migrate a couple of apps within the allotted Sprint(s), however, we highly recommend leaving some for homework. --- ## Decide on Database Seeding Strategy for Ephemeral Preview Environments import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We need to decide how to provision databases for use with preview environments. These databases must come up very quickly (e.g. 10-20 seconds rather than 20-30 minutes it takes for RDS). Once these databases come online, we need to have data staged for them to be useful. Restoring very large database dumps can be very slow and we need to update database dumps and scrub them. We typically cannot (and should not) use snapshots directly from production due to constraints around how we must handle PII, PHI, CDH, etc. :::caution As a general best practice, we should never use production data in non-production environments to avoid accidental leakage or usage of data. ::: ## Considerations We prefer to include the DBA in these conversations. Suggested requirements: - They should come online very fast, so the process of bringing up new environments is not slowed down. - They should be easily destroyed - They should be inexpensive to operate because there will be many of them - They should have realistic data, so the environments are testing something closer to staging/production ## Considered Options **Option 1:** Seed data (fixtures) - **recommended** - Most database migration tools support something like this (e.g. `rake db:fixtures:load`) - This is the easiest to implement **Option 2:** Docker Image with Preloaded Dataset - Advisable if the dataset is large enough that loading dump would take too long, but the dataset isn’t so large that sticking it in a docker image is not unreasonable - Implementation will require additional scope for automating the creation of the docker image **Option 3:** Shared RDS cluster, Preloaded Shared database - A shared database preloaded with sanitized seeded data can be shared across preview environments - No ability to test migrations using this approach **Option 4:** Shared RDS cluster, one database per env, with seed data - Greater economies of scale are achieved by sharing the database - Custom process of hydrating the database for each preview environment will need to be implemented **Option 5:** Dedicated cluster (not advised) - Too slow to launch (e.g. +30-40 minutes), expensive, complicated to implement --- ## Decide on GitHub Actions Workflow Organization Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem GitHub Actions Workflow files scattered in repositories throughout the GitHub organization can quickly violate the DRY principle if they contain repeated code. ## Considerations ### Standardize Workflows Across the Organization ### Metrics and Observability With appropriate metrics, you’ll be able to answer questions like: - Are we deploying faster? ...or slowing down? - What is the stability of our deployments? See [https://www.datadoghq.com/blog/datadog-github-actions-ci-visibility/](https://www.datadoghq.com/blog/datadog-github-actions-ci-visibility/) [leanix-poster_17_metrics_to_help_build_better_software-en.pdf](/assets/refarch/leanix-poster_17_metrics_to_help_build_better_software-en.pdf) [https://www.leanix.net/en/wiki/vsm/dora-metrics](https://www.leanix.net/en/wiki/vsm/dora-metrics) ### Public Actions Trusted organizations for public actions Verified organizations for public actions ### Private Actions Private actions technically require GitHub Enterprise. We can do a workaround for non-enterprise organizations: an explicit `git clone` of a private actions repo. ### Shared Workflows ### Code Reusability [Composite Actions](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action) can be leveraged to solve that problem. Composite Actions are very similar to GHA workflow files in that they contain multiple steps, some of which can reference open-source Actions. Still, they are not individual workflows but rather actions that another workflow can reference. These Composite Actions thus reduce code repetition within the organization. ### GitHub Script :::caution Usage of inline GitHub Scripts are difficult to maintain. They are acceptable inside of dedicated actions, but not recommended as part of workflows. ::: Some composite actions may utilize the [github-script](https://github.com/actions/github-script) action, resulting in inline Node.js code that lacks any syntax highlighting and can make the composite action YAML file unnecessarily long. A solution for this is to create separate Node.js modules, invoke them within the inline code supplied to `github-script`, and supply the references created by `github-script` and contexts available in GHA workflows to those modules: ``` const actionContext = require('./actions/lib/actioncontext.js')(this, context, core, github, ${{ toJSON(github) }}, ${{ toJSON(inputs) }}, ${{ toJSON(steps) }}) const deployment = require('./actions/lib/deployment.js')(actionContext) deployment.newDeployment(JSON.parse(`${{ inputs.stages }}`)) ``` ## Recommendation - Use a private repository for reusable GitHub workflows (e.g. `acme/github-workflows`) - Use GitHub Enterprise to support approval steps - Use Organization `acme/.github` repository with starter workflows - Use Cloud Posse’s existing workflows to get started quickly - Use Cloud Posse’s public github actions - Use a private GitHub actions repository specific to your organization (e.g. `acme/github-actions`) - Use a private template repository to make it easy for developers to initialize new projects - Adjust `webhook_startup_timeout` in the chart. This setting is used for automatically scaling back replicas. The recommended default is 30 minutes, but no one size fits all. Here's further documentation for your consideration: [scaling runners](https://github.com/actions/actions-runner-controller/blob/master/docs/automatically-scaling-runners.md) ## Related - [Decide on Strategy for Continuous Integration](/layers/software-delivery/design-decisions/decide-on-strategy-for-continuous-integration) - [Decide on Self-Hosted GitHub Runner Strategy](/layers/software-delivery/design-decisions/decide-on-self-hosted-github-runner-strategy) - [GitHub Actions](/learn/tips-and-tricks/github-actions) --- ## Decide on Hot-fix or Rollback Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Releasing a change is when it’s made available to a user or subset of users. In an ideal world, deployments should not equal releases. ## Considerations #### Feature Flagging This strategy involves using a feature-flagging feature such as LaunchDarkly in order to be able to toggle features without redeploying code to an environment. If a new feature does not work, it can be toggled off using the LaunchDarkly API without having to author a new release and have it redeployed using the CD pipeline. If the feature flags are configured properly, this can be an effective solution as the changes do not have to be authored and passed through the CD pipeline, hence shortening the mean time to restore. #### Rolling Forward This strategy involves authoring a patch release in order to disable a problematic feature or author a bug-fix, using the CD pipeline. This can lead to a longer mean time to restore when compared to feature flagging, since fixes or feature disablements need to be authored and passed through the CD pipeline. #### Release branches If release branches are utilized, then any bug-fix commits need to be pushed to the release branch, indicating a new patch semantic version corresponding to the minor semantic version corresponding to that release branch (for example if the release branch is 1.1.x and the 1.1.0 tag had a bug, then a bug-fix commit can be pushed to that release branch and tagged at 1.1.1). These changes then need to pass through the CD pipeline and deployed to the live environment. This is very similar to the _Rolling Forward_ strategy, with the only difference being that releases are not cut directly from the trunk. ## Related - [Decide on Release Promotion Strategy](/layers/software-delivery/design-decisions/decide-on-release-promotion-strategy) --- ## Decide on how ECS apps are deployed import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We need to decide on what methodology to use when deploying applications to ECS. Think of Helm Charts in Kubernetes as similar to using Terraform Modules for ECS Tasks #### Deploy using Spacelift - Github action that triggers Spacelift run which runs atmos and terraform - Spacelift makes use of rego policies - Optional manual confirmation #### Deploy using Github Actions without Spacelift - Github action runs atmos and terraform directly - Auto deploy on merges - Auto deploy on manual cut releases --- ## Decide on Kubernetes Application Artifacts import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; We prefer strategies that ship the Application code (e.g. docker images) with the Application configuration (E.g. everything needed to run the application on the platform, such as manifests, IAM roles, etc.) **Application-specific Infrastructure Considerations** - IAM roles, SNS topics, - Does the terraform code live alongside the apps or in the infrastructure mono repo (our preference is alongside the apps). - When should these infrastructure changes rollout? e.g. before or after application changes. - If the resources will be shared amongst services, then we should probably not do this for those dependencies and instead move them to shared infrastructure since their lifecycle is not coupled to one application. **Application Configuration Considerations** - Raw manifests - Helm charts - Kustomize --- ## Decide on Maintenance Page Solution import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem When your sites go down, we need to be able to communicate more graciously to customers that you’re having issues than a generic “502 Bad Gateway” message. :::info Specifically, this decision relates to services behind an ALB. CloudFront and S3 are out of scope. ::: ## Solution We recommend deploying a static maintenance page. The industry best practice is to host the downtime page on a cloud provider that does not share infrastructure with your primary cloud provider. E.g. S3 is not recommended, as even S3 has gone down. That said, using a separate cloud provider is a micro-optimization for a very narrow set of failure scenarios. Some related considerations are how the maintenance page will be activated. ### Considered Options There are a few options: ### Option 1 Use route53 health checks. Cloud Posse does not recommend it because poorly implemented DNS-clients clients may cache the downtime host. ### Option2 Use CloudFront dynamically redirect to downtime page using an Origin Group with fail-over. Here’s a simple example using `terraform` to provision a maintenance page on Cloud Flare. - [https://github.com/adinhodovic/terraform-cloudflare-maintenance](https://github.com/adinhodovic/terraform-cloudflare-maintenance) ### Option 3 (Recommended) Use ALB with [fixed response](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-listeners.html#fixed-response-actions) - Use `fixed-response` with iframe (not ideal) to S3 static site - Use `fixed-response` populated with HTML from `file` with inline CSS, SVGs, etc. and no external dependencies (if possible) - Add GA code for analytics --- ## Decide on Pipeline Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement ## Problem Statement Teams need a release engineering process to help QA and developer teams operate efficiently. Namely, QA needs a way to validate changes in QA environments before releasing them to staging or production. Changes to production require approval gates, so only authorized persons can release to production. And if changes need to be made to the running production release, those need to be performed via hotfixes that need a special CI/CD and release workflow. The more service you operate, the more important it is that workflows are very DRY and are not copied between all repositories, making maintenance difficult. ## Prerequisites Before implementation on the pipeline strategy, the following should be in place - An inventory of the applications for migration to the new pipelines - Cloud Posse access to the repositories - All the GitHub Action runners deployed ## High-level Approach :::info The following is our Kubernetes-centric approach with GitHub Actions. Similar strategies can be implemented for other platforms, but would require different techniques for integration testing and deployment. ::: Cloud Posse’s turn-key implementation is an approach that provides QA environments, approval gates, release deployments, and hotfixes in a way that applications can utilize with minimal effort and minimal duplication. Predefined workflows
Feature branch workflow
Triggered on changes in a pull request that target the `main` branch. It will perform CI (build and test) and CD (deploy into _Preview_ and/or _QA_ environments) jobs.
Main branch workflow
Triggered on commit into the `main` branch to integrate the latest changes and create/update the next draft release. It will perform CI (build and test) and CD (deploy into `Dev` environment) jobs.
Release workflow
Triggered when a new release is published. The workflow will promote artifacts (docker image) that was built by the “_Feature branch workflow_“ to the release version and deploy them to the `Staging` and `Production` environments with approval gates. In addition, the workflow will create a special `release/ {version}` branch that is required for the hotfixes workflow.
Hot Fix Branch workflow
Triggered on changes in a pull request that target any `release/{version}` branch. It will perform CI (build and test) and CD (deploy into `Hotfix` environment) jobs.
Hot Fix Release workflow
Triggered on commit into the `release/{version}` branch to integrate new hotfix changes. It will perform CI (build and test) and CD (deploy into the `Production` environment with approval gates) jobs. In addition, it will create a new release with incremented patch version and create a regular PR target `main` branch to integrate the hotfix with the latest code.
The implementation should use custom GitHub actions and reusable workflows to have DRY code and a clear definition for each `workflow/job/step/action`. Integrate with Github UI to visualize the release workflow in-process and in-state. ## **Goals** The top 3 goals of our approach is to... 1. Make it very easy for developers to onboard new services 2. Ensure it’s easy for developers to understand the workflow and build failures 3. Leverage GitHub UI, so it’s easy to understand what software is released by an environment ## **Key Features & Use Cases** What we implement as part of our approach and the specific use cases we address is explained below. ### CI testing based on the Feature branch workflow - A developer creates a PR target to the `main` branch. GHA will perform build and run test on each commit. The developer should have ability to deploy/undeploy the changes to `Preview` and/or `QA` environment by adding/removing specific labels in PR Gihub UI. When PR merged or closed GHA should undeploy the code from `Preview`/`QA` environments where it is deployed to. ### CI Preview Environments - Preview environments are unlimited ephimerial environments running on Kubernetes. When a PR with a target of the `main` branch is labeled with the `deploy` label, it will be deployed into a new preview environment. If developer needs to test the integration between several services they can deploy those apps into the same preview environment by creating PRs using the same named branch (e.g. `feature/add-widgets`). - Preview environments by convention expect that all third party services (databases, messaging bus, cache and etc) are deployed from scratch in Kubernetes as a part of the environment and removed on PR close. - The developer is responsible for defining third party services and to orchestrate them in Kubernetes (e.g. with [Operators](https://operatorhub.io/)). ### CI QA Environments - QA environments are a discrete set of static environments running on Kubernetes with preprovisioned third party services. They are similar to preview environments, except that environments are shared by QA engineers to verify PR changes in “close to real live” environment. QA engineer can deploy/undeploy PR changes to one of the QA environments by adding or removing the `deploy/qa{number}` label. - If several PRs of one repo have `deploy/qa{number}` label then the latest deployment (commit & push) will override each other. - It is responsibility of QA engineers to avoid this conflict. GitHub environments UI is useful for seeing what is deployed. ### Test commits into the main branch - On each commit into the `main` branch, the “_Main branch workflow_” triggers. It will build and test the latest code from the `main` branch, create or update the latest draft release and deploy the code to `Dev` environment. - If the commit was done by merging a PR then the PR title/description would be added to the release changelog. ### Bleeding Edge Environment on Dev - The “dev” environment is a single environment with provisioned third-party services. The environment should be approximately equivalent to `Staging` and `Production` environments. Developers and QA engineers need it to perform integration testing and validate the interaction between the latest version of applications and services before cutting a release. This is why it’s called the “bleeding edge.” ### Automatic Draft Releases Following Semver Convention - On commit in the `main` branch GHA should create new draft release or update it. The release should have auto generated changelog based on commit comment messages and PRs title/descriptions. - Developer can manage sections of the changelog by adding specific labels to the PR. - Also labels are used to define the release major/minor semver increment (minor increments by default) ### Automated Releases with Approval Gates - When a Developer (or Release Manager) decides to issue a new release they need just to publish the _Draft Release_ that will trigger the “_Release workflow_“. The workflow should create a new “Release branch” `release/{version}`, promote docker image with release version and consequently deploy it to `Staging` and `Production` environments with approval gates. Developer need to approve deployment on `Staging` environment, wait the deployment would be successfully completed and then repeat the same for `Production` environment. ### Staging Environment - `Staging` is a single environment with provisioned third-party services. The environment should be approximately equivalent to `Production` environment. Developers and QA engineers use it to perform integration testing, run migrations, test deploy procedures and interactions of the latest released versions. So while the the `Dev` environment operates on the latest commit into `main`, the `Staging` environment operates on the latest release. ### Production Environment - `Production` is a single environment with provisioned third party services used by real users. It operates on releases that have been promoted from `Staging` after approval. ### Hotfix Pull Request workflow - In the case when there is a bug in the application that runs in the `Production` environment, the Developer needs to create a Hotfix PR. - Hotfix PR should target to “_Release branch_” `release/{version}`. GHA should perform build and run tests on each commit. The developer should have ability to deploy/undeploy the changes to `Hotfix` environment by adding/removing specific labels in PR Gihub UI. When PR merged or closed GHA should undeploy the code from `Hotfix` environment. ### Hotfix Environment - `Hotfix` is a single environment with provisioned third-party services. The environment should be approximately equivalent to `Production` environment. Developers and QA engineers need it to perform integration testing, migrations, deploy procedures and interactions of the hotfix with other services. - If there are several `hotfix` PRs in one repo deployments to `Hotfix` environment will be conflicting. The latest deploy will be running on `Hotfix` environment. - This is responsibility of Developers and QA engineers to avoid that conflicts. ### Hotfix Release workflow - On each commit into a “_Release branch_” `release/{version}` “_Hotfix release workflow_” triggers. It will build and test the latest code from the branch, create a new release with increased patched version and deploy it with approval gate to the `Production` environment. - Developer should also take care of the hotfix to the `main` branch, for which a reintegration PR will be created automatically. ### Deployments - All deployments are by default performed with `helmfile` on Kubernetes clusters. ### Reusable workflows and GHA - All workflows and custom github actions should be reusable and have not specific repository references. - “Reusable workflows in private repo“ pattern - Reusable worklows should be stored in separate repo and copied on change across all repositories by special workflow - according to `Reusable workflow in private organization repositories` pattern. ## Considerations The following considerations are required before we can begin implementing the turnkey GitHub Action workflows. ### Supported Environments The following key decisions need to be made as part of this design decision: - Which environments are relevant to your organization? (e.g. do you need the Preview/QA environments or is Dev/Staging/Prod sufficient?) - Preview environments (not all applications are suitable for this) - QA environments - Dev/Staging/Production environments - Hotfix environment ### Approval Gate Strategy GitHub Enterprise is required to support native approval gates on deployments to environments. Approval gates support a permissions model to restrict who is allowed to approve a deployment. Without GitHub Enterprise, we’ll need to use an alternative strategy using [workflow_dispatch](https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow) to manually trigger deployments using the GitHub UI. ### GitHub Repo Strategy for Applications We’ll need to know what strategy you use for your applications: e.g. monorepo, polyrepo, or a hybrid approach involving multiple monorepos. ### GitHub Repo for Shared Workflows What repo do you want to use to store the shared GitHub action workflows? e.g. we recommend calling it `github-action-workflows` GitHub Enterprise users will have a native ability to use private-shared workflows. Non-GitHub Enterprise users will need to use a workaround, which involves cloning the shared workflows repo before using them. ### GitHub Repo for Private GitHub Actions What repo do you want to use for your private GitHub actions? For GitHub Enterprise users we recommend using one repo per private GitHub Action so that they can be individually versioned. We’ll need to know what convention to use. Cloud Posse uses `github-action-$name` while we’ve seen some organizations use patterns like `$name.action` and `action-$name`. We like the `github-action-$name` convention because it follows the Terraform convention for modules and providers (e.g. `terraform-provider-aws`) We recommend a monorepo for non-GitHub enterprise users. If we take this approach, we’ll need to clone the private GitHub Actions repo as part of each workflow. We’ll need to know what this repo is called. We recommend calling it `github-actions`. Alternatively, if your company uses a monorepo strategy for ## **Out of Scope**
Automated Rollbacks
Automated triggering of rollbacks is not supported. Manually initiated, automatic rollbacks are supported, but should be triggered by reverting the pull request and using the aforementioned release process.
Provision environments
Provision k8s clusters, third party services for any environments should be performed as separate mile stone. We expect already have K8S credentials for deployments
Define Docker based third party services
Third party services running in docker should be declared individually per application. This is Developers field of work.
Key Metrics & Observability
Monitoring CI pipelines and tests for visibility (e.g. with with Datadog CI) is not factored in but can be added at a later time. [https://www.datadoghq.com/blog/datadog-ci-visibility/](https://www.datadoghq.com/blog/datadog-ci-visibility/)
## **Open Issues & Key Decisions** [Decide on Database Seeding Strategy for Ephemeral Preview Environments](/layers/software-delivery/design-decisions/decide-on-database-seeding-strategy-for-ephemeral-preview-enviro) [Decide on Customer Apps for Migration](/layers/software-delivery/design-decisions/decide-on-customer-apps-for-migration) [Decide on Seeding Strategy for Staging Environments](/layers/software-delivery/design-decisions/decide-on-seeding-strategy-for-staging-environments) ## **Design and Explorations Research** Links to any supporting documentation or pages, if any - [Continuous Delivery: Understand your Value Stream - Step 1](https://medium.com/@yaravind/continuous-delivery-understand-your-value-stream-step-1-e2955eaeba95) - [Value Stream Management: Treat Your Pipeline as Your Most Important Product](https://devops.com/value-stream-management-treat-your-pipeline-as-your-most-important-product/) - [Deployments approval gates with Github](https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments) - [Release workflow POC (Cloud Posse version)](https://github.com/cloudposse/example-github-action-release-workflow/pull/45) - [Using environments for deployment on Github](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment) - [https://github.community/t/reusable-workflows-in-private-organization-repositories/215009/43](https://github.community/t/reusable-workflows-in-private-organization-repositories/215009/43) - [https://medium.com/@er.singh.nitin/how-to-share-the-github-actions-workflow-in-an-organization-privately-c3bb3e0deb3](https://medium.com/@er.singh.nitin/how-to-share-the-github-actions-workflow-in-an-organization-privately-c3bb3e0deb3) - [GitHub Actions](/learn/tips-and-tricks/github-actions) is Cloud Posse’s own reference documentation which includes a lot of our learnings ## **Security Risk Assessment** The release engineering system consists of two main components - _Github Action Cloud_ (a.k. _GHA_) and _Github Action Runners_ (a.k. _GHA-Runners_). The _GHA-Runners_ can be ‘Cloud provided' or 'Self-hosted’. ‘_Self-hosted GHA-Runners_' are executed on EC2 instances under the control of the autoscaling group in the dedicated '_Automation’_ AWS account. On an EC2 instance, bootstrap GHA-Runner registers itself on Github with a _**Registration token**_ **(1)**. From that moment _GHA_ can run workflows on it. When a new `_Workflow Run_` is initialized, GHA issues a new unique _**Default token (2)**_. That token is used to authenticate on Github API and interact with it. For example, `_Workflow Run_` uses it to pull source code from a Github repository **(3)**. _**Default token**_ scoped to a repository (or another Github resource) that was the source of the triggered event. On the provided diagram, it is the _Application Repository._ If a workflow needs to pull source code from another repository, we have to use _Personal Access Token (****PAT****),_ which had to be issued preliminarily. On the diagram, this is ‘**PAT PRIVATE GHA' (4)** that we use to pull the organization's private actions used as steps in GHA workflows. In a moment GHA-Runner pulled the ‘_Application_’ source code and ‘_Private Actions_’ it is ready to perform real work - build docker images, run tests, deploy to specific environments and interact with Github for a better developer experience. To interact with AWS services `_Workflow Run_` assumes **CICD (5)** **IAM role** that grants permissions to work with ECR and to assume **Helm (5)** **IAM roles** from another account. The **'Helm' IAM role** is useful to **Authenticate (6)** on a specific EKS cluster and to deploy there. Assuming **CICD IAM role** is possible only on '\_Self-hosted GHA-\_Runners’ as EC2 Instance credentials used for initial interaction with AWS. _**Default token**_ fits all needs except one - creating a _Hotfix Reintegration Pull Request._ for that functionally we need to implement a workaround. On the diagram provided one of the possible workarounds - using _**PAT to Create PRs (7)**_ with wider permissions***.*** ### Registration token Registration token required only to register/deregister ‘_Self-hosted GHA-Runner_' on Github. The token allows attaching '_Self-hosted GHA-Runner_' to the organization or a single repository scope. If '_Self-hosted GHA-Runner_' scoped to the organization level, any repository in the org can run its workflows on the ‘_Self-hosted GHA-Runner_'. - [github-runners](/components/library/aws/github-runners/) - [https://docs.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners](https://docs.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners) ### Default Github Token The token is generated on '_Workflow Run_' initialization. So it is unique per '_Workflow Run_'. The token is scoped to the repository, that triggered the '_Workflow Run_'. By default, the token can have `permissive` or `restricted` scopes granted. The difference between declared in the table below. You can select which of the default scopes would be used. For settings per repo - follow [this documentation](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#setting-the-permissions-of-the-github_token-for-your-repository), for setting for all repositories in the organization - follow [this documentation](https://docs.github.com/en/organizations/managing-organization-settings/disabling-or-limiting-github-actions-for-your-organization#setting-the-permissions-of-the-github_token-for-your-organization). | Scope | Default access(permissive) | Default access(restricted) | | ------------------- | ------------------------------- | ------------------------------- | | actions | read/write | none | | checks | read/write | none | | contents | read/write | read | | deployments | read/write | none | | id-token | none | none | | issues | read/write | none | | metadata | read | read | | packages | read/write | none | | pages | read/write | none | | pull-requests | read/write | none | | repository-projects | read/write | none | | security-events | read/write | none | | statuses | read/write | none | We recommend using the `restricted` scope by default. GHA workflows can [explicitly escalate permissions](https://docs.github.com/en/actions/security-guides/automatic-token-authentication#how-the-permissions-are-calculated-for-a-workflow-job) if that’s required for the process. All workflows implemented in POC explicitly request escalation of permission from the `restricted` scope. Please check the following table. | Scope | Default access(restricted) | Pull Request Workflow | Bleeding edge Workflow | Release Workflow | Hotfix Pull Request Workflow | Hotfix workflow | | ------------- | ------------------------------- | --------------------- | ---------------------- | ---------------- | ---------------------------- | --------------- | | contents | read | read | read/write | read/write | read | read/write | | deployments | none | read/write | none | none | read/write | none | | metadata | read | read | read | read | read | read | | pull-requests | none | read/write | none | none | read/write | read/write | - [https://docs.github.com/en/actions/security-guides/automatic-token-authentication](https://docs.github.com/en/actions/security-guides/automatic-token-authentication) - [github-runners](/components/library/aws/github-runners/) - [https://docs.github.com/en/organizations/managing-organization-settings/disabling-or-limiting-github-actions-for-your-organization#setting-the-permissions-of-the-github_token-for-your-organization](https://docs.github.com/en/organizations/managing-organization-settings/disabling-or-limiting-github-actions-for-your-organization#setting-the-permissions-of-the-github_token-for-your-organization) - [https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#setting-the-permissions-of-the-github_token-for-your-repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#setting-the-permissions-of-the-github_token-for-your-repository) - [https://docs.github.com/en/rest/overview/permissions-required-for-github-apps#permission-on-repository-hooks](https://docs.github.com/en/rest/overview/permissions-required-for-github-apps#permission-on-repository-hooks) ### Private Github Actions PAT Having additional PAT is a necessary evil to share the Private Github Actions library. The only way to use private GitHub action is to pull it from a private repository and reference it with the local path. It is impossible to use the _‘Default Github token’_ as it is scoped to one repo - [read more](https://github.com/actions/checkout#checkout-multiple-repos-private) To get this PAT with minimal required permissions follows these steps: 1. Create a technical user on Github ( like `bot+private-gha@example.com` ) 2. Added the user to the `Private Actions` repository with 'read-only' permissions (`https://github.com/{organization}/{repository}/settings/access`) 3. Generate a PAT for the technical user with that level of permissions [https://github.com/settings/tokens/new](https://github.com/settings/tokens/new) 4. Save the PAT as organization secret with name `GITHUB_PRIVATE_ACTIONS_PAT` (`https://github.com/organizations/{organization}/settings/secrets/actions`) - [https://github.com/actions/checkout#checkout-multiple-repos-private](https://github.com/actions/checkout#checkout-multiple-repos-private) - [https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/](https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/) - [https://github.com/marketplace/actions/private-actions-checkout#github-app](https://github.com/marketplace/actions/private-actions-checkout#github-app) ### AWS Assume Role Sessions Detailed description interaction with AWS API is out of the scope of this POC. Just want to mention that by default ‘_Self-hosted GHA-Runner_' have the same access to AWS resources as the Instance profile role attached to the ‘GHA-Runners' EC2 instances. The minimal requirement is granted to assume the ‘CICD' role and through it assume any 'Helm’ roles to get access to EKS clusters for deployment. ### Authentication on EKS with IAM Detailed description authentication on EKS with IAM is out of the scope of this POC. The only thing we’d like to mention is that we will have the same level of permissions on EKS as the 'Helm' role do. ### `Create PR` Problem The final step in `Hotfix` workflow is to create PR into the `main` branch to reintegrate the hotfix changes with the latest code in the `main`. The problem is that `Creating and approving PR` is separate permission that is disabled by default. And it seems to be a best practice to leave it as is. That permission can be granted on the same with default scopes for '_Default token_' pages ([repo](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#preventing-github-actions-from-creating-or-approving-pull-requests) or [org](https://docs.github.com/en/organizations/managing-organization-settings/disabling-or-limiting-github-actions-for-your-organization#preventing-github-actions-from-creating-or-approving-pull-requests) level). #### Workarounds: 1. Enabled `Creating and approving PR` on the repo or even org level and used 'Default Github Token' to create a PR 2. Create a new technical GitHub user, permit it to create PRs, issue PAT under the user, and use it for PR creation. This is close to what we did for '_Private Actions_' but with much wider access. 3. Skip the automatic PR creation feature and rely on developers to create PRs from Github UI ### Learn more: - https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions#stealing-the-jobs-github_token --- ## Decide on Release Engineering Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem A consistent branching strategy is needed to ensure that a consistent workflow is shared across all projects. ## Solution ### GitFlow Strategy Gitflow is a branching strategy that allows for parallel development by creating separate branches for features and releases. This strategy is considered a bit complicated and advanced but can be beneficial for larger, more complex projects. The Gitflow branching model consists of the following branches: - **Master Branch:** Represents the production-ready code and is typically only updated when a new release is made. - **Develop Branch:** Represents the latest development code and serves as a parent branch for feature branches. - **Feature Branches:** Created from the develop branch, feature branches are used to develop new features or functionality. Once the feature is complete, it is merged back into the develop branch. - **Release Branches:** Created from the develop branch, release branches are used to prepare for a new production release. Any bug fixes and final testing are done on this branch before being merged back into both the develop and master branches. - **Hotfix Branches:** Similar to release branches, hotfix branches are created from the master branch to address any critical bugs or issues discovered in the production code. Once the hotfix is complete, it is merged back into both the master and develop branches. The benefit of Gitflow is that it provides a clear path for changes to be made to the codebase, ensuring that production-ready code is only released from the master branch. It also allows for multiple developers to work on features and bug fixes in parallel without disrupting the development workflow. ### Trunk-Based Strategy Trunk-based development is a branching strategy that allows for continuous integration and deployment. This strategy is considered simpler and more lightweight than Gitflow, but may not be suitable for larger, more complex projects. The trunk-based branching model consists of the following branches: - **Main (_Trunk_) Branch:** Represents the latest development code and serves as a parent branch for feature branches. - **Feature Branches:** Created from the _Trunk_ branch, feature branches are used to develop new features or functionality. Once the feature is complete, it is merged back into the develop branch. --- ## Decide on Release Promotion Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ### Problem and Context We need to control when a release is deployed to a given stage (E.g. dev, staging, production). We must decide how releases will be promoted from staging to production. How that will be accomplished will depend on whether or not GitHub Enterprise features are available and whether or not it’s possible to use semantic versioning or not. ### Assumptions - Auto deployment to the `dev` stage will be triggered upon every commit to the default branch (e.g. `main`) ### Options #### Option A: Automatically Deploy to Staging on Every Release, Use GitHub Approval Steps for Production ##### Pros - Natively supported by GitHub - Environment protection rules ensure RBAC restricts who can approve deployments ##### Cons - Requires GitHub Enterprise, as GitHub Approvals, GitHub Environment protection rules (and Environment Secrets) are only available in GitHub Enterprise. #### Option B: Automatically Deploy to Staging on Every Release, Use Manual GitHub Action Workflow to Production Deployments ##### Pros - Does not require GitHub Enterprise - Staging always represents the latest release ##### Cons - No environment protection rules; anyone who can run the workflow can deploy. Mitigated by customizing the workflow with business logic to restrict it, but not supported by Cloud Posse today. #### Option C: Use Manual GitHub Action Workflow for Staging and Production Deployments ##### Pros - Does not require GitHub Enterprise - Full control over when every stage is updated ##### Cons - More manual operations to promote a release - No environment protection rules; anyone who can run the workflow can deploy. Mitigated by customizing the workflow with business logic to restrict it, but not supported by Cloud Posse today. ### Out of Scope - Tightly coupled multi-service application deployments ### Related Design Decisions - [Decide on Database Schema Migration Strategy](/layers/data/design-decisions/decide-on-database-schema-migration-strategy) --- ## Decide on Repositories Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement ## Problem Statement Deciding repository strategies for your codebase is a crucial choice because it can significantly impact your development processes, collaboration effectiveness, tooling, and architectural decisions. There are two main strategies for organizing source code repositories: [monorepo](https://en.wikipedia.org/wiki/Monorepo) and `polyrepo`. In a middle there can be a hybrid strategies: - `multi monorepos` - `monorepo as read-only proxy` - `polyrepos & monorepos` The hybrid strategies inherit gains and loss of the main two. That's why focus on pros and const for the main repository structures. ## Poly repo In a polyrepo structure, each project, service, or library has its own repository. ### Benefits #### Isolation Each project is isolated from others, reducing the risk of a change in one project inadvertently affecting others. #### Scalability Polyrepos can scale more effectively as each repository can be managed separately. #### Simple and fast CI/CD Pipelines CI/CD pipelines contains less logic and works faster because it only has to process the relevant parts of the codebase. ### Challenges #### Code Duplication Code that's shared between projects might have to be duplicated in each repository. #### Increased Management Overhead Managing multiple repositories can be more complex and time-consuming. #### Complex Dependency Management If libraries have interdependencies, it can be harder to manage versioning across multiple repositories. ## Monorepo Monorepos hold all of an organization's code in a single repository. All projects, libraries, and services live together. ### Benefits #### Code Sharing and Reuse With all the code in one place, it's easy to share and reuse code across multiple projects. This can lead to more consistent code, reduce duplication, and enhance productivity. #### Unified Versioning A monorepo has a single source of truth for the current state of the system. #### Collaboration and Code Review Developers can work together on code, have visibility of changes across the entire project, and perform code reviews more effectively. #### Simplified Dependency Management All projects use the same version of third-party dependencies, which can make managing those dependencies easier. ### Challenges #### Scalability As a codebase grows, it can become more challenging to manage and navigate a monorepo. #### Complex and slower CI/CD Pipelines Continuous integration and deployment can become slower as your codebase grows because the pipeline may need to compile and test the entire codebase for every change. CI/CD pipelines for monorepo are complex and required special tooling such as [Bazel](https://bazel.build/), [Pants](https://www.pantsbuild.org/), [Please](https://please.build/) or [Buck](https://buck2.build/). #### Risk of Breaking Changes A small change in one part of the codebase might break something else unexpectedly since everything is interconnected. #### Dummy Versioning Whenever the entire monorepo is tagged, it automatically assigns this new tag to all code inside, including all hosted libraries. This could lead to the release of all these libraries under the new version number, even if many of these libraries have not been updated or modified in any way. ## Recommendation We recommend using `Polyrepo` as a basic repository organization strategy because it leads to faster development cycle, simplify CI/CD pipelines, do not require additional tooling. Active usage of preview (ephemeral) and QA environments, testing automation, deployment on multiple stages (dev -> staging -> prod) during release workflow and implementing of a particular deployment patterns (Blue/Green, Canary, Rolling) allow catching integration issues before the code goes on production. The repository strategy does not have to be the same across the whole organization - different teams can use different patterns, but that lead to the complexity of the CI/CD pipelines and reduce reusability. That's why we recommend having a consistent repository strategy, at least on a team level. ## References - [Monorepo vs. polyrepo](https://github.com/joelparkerhenderson/monorepo-vs-polyrepo) - [From a Single Repo, to Multi-Repos, to Monorepo, to Multi-Monorepo](https://css-tricks.com/from-a-single-repo-to-multi-repos-to-monorepo-to-multi-monorepo/) - [Monorepo vs Polyrepo](https://earthly.dev/blog/monorepo-vs-polyrepo/) - [Polyrepo vs. Monorepo - How does it impact dependency management?](https://www.endorlabs.com/blog/polyrepo-vs-monorepo-how-does-it-impact-dependency-management) - [Monorepo Vs Polyrepo Architecture: A Comparison For Effective Software Development](https://webo.digital/blog/monorepo-vs-polyrepo-architecture/) - --- ## Decide on Seeding Strategy for Staging Environments import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem Longer-lived staging environments need a dataset that closely resembles production. If this dataset becomes stale, we’ll not be effectively testing releases before they hit production. Restoring snapshots from production is not recommended. ## Considerations - Should contain anonymized users, invalid email addresses - No CHD, PHI, PII must be contained in the database - The scale of data should be close to the production database - Snapshots from production are dangerous if not anonymized/scrubbed (imagine the risk of sending emails to everyone from your staging env) - Fixtures are not recommended (scale of data for fixtures usually does not represent production) - We recommend including the DBA in these conversations. - QA teams want stable data so that they can run through their test scenarios ## Recommendations :::caution Cloud Posse does not have a turnkey solution for seeding staging environments ::: - ETL pipeline scrubs the data and refreshes the database weekly or monthly. (e.g. AWS Glue, GitHub Action Schedule Job) --- ## Decide on Self-Hosted GitHub Runner Strategy import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem GitHub Actions enables organizations to run their own runners (aka workers) free-of-charge. Only problem is we have to host them and there’s a couple of ways of doing it, each with their own pros and cons. For the most part, we don’t consider it optional to deploy runners, if using GitHub Actions. ## Considerations We have prior art for the following strategies. The strategies are not mutually exclusive, but most often we see companies only implement one solution. ## Considerations GitHub Actions is the CI/CD platform offered by GitHub. Its main advantages are that, firstly, it comes at no additional costs on top of the cost of seats for users within the GitHub organization, when using self-hosted Runners. Secondly, there is a thriving ecosystem of open source Actions that can be used within GitHub Action workflows. ### Self-hosted Runners on EC2 GitHub Actions Runners can be self-hosted on EC2 or on Kubernetes clusters. This is probably the easiest to deploy and understand how it works, but it’s the least optimal way of managing runners. On EC2, the [Runner executable](https://github.com/actions/runner) can be installed using a user-data script, or baked into the AMI of the instance to be deployed to EC2. The runner can be deployed to an Auto Scaling Group (ASG), which usually presents a de-registration problem when the ASG scales in, however EventBridge and AWS Systems Manager can be utilized in tandem to have the runner de-register prior to being terminated (see: [Cloud Posse's github-runners component](https://github.com/cloudposse/terraform-aws-components/tree/95ade5b36b61d2432179399bd0e9fa8639eeb899/modules/github-runners) which has this implementation). It’s also possible to use Spot Instances. Easily use AWS SSM to connect to runners. Autoscale anytime sustained CPU capacity > 5% for ~5 minutes (e.g. doing anything). Scale down when CPU capacity is < 2% for 45 minutes (e.g. doing nothing). Requires a minimum of 1 node online. This is a good route if builds need to run on multiple kinds of architectures (e.g. ARM64 for M1), for which operating the kubernetes node pools would be cumbersome. If we go this route, we’ll also want to determine if we should deploy the Datadog Monitoring Agent on the nodes. [https://github.com/cloudposse/terraform-aws-ec2-autoscale-group](https://github.com/cloudposse/terraform-aws-ec2-autoscale-group) ### Self-hosted Runners on Kubernetes :::tip This is our recommended approach ::: Deploying these Runners on Kubernetes is possible using [actions-runner-controller](https://github.com/actions-runner-controller/actions-runner-controller). With this controller, a small-to-medium-sized cluster can house a large number of Runners (depending on their requested Memory and CPU resources), and these Runners can scale automatically using the controller’s `HorizontalRunnerAutoscaler` CRD. This has the benefit that it can scale to zero and leverages all the monitoring we have on the platform. This solution also allows for using a custom runner image without having to rebuild an AMI or modify a user-data script and re-launch instances, which would be necessary when deploying the Runners to EC2. `actions-runner-controller` also supports several various mechanisms for scaling the number of Runners: `PercentageRunnersBusy` simply scales the Runners up or down based on how many of them are currently busy, without having to maintain a list of repositories used by the Runners, which would be the case in `TotalNumberOfQueuedAndInProgressWorkflowRuns`. The most efficient and recommended option for horizontal auto-scaling using the `actions-runner-controller`, however, is to [enable the controller’s webhook server](https://github.com/actions-runner-controller/actions-runner-controller#webhook-driven-scaling) and configure the `HorizontalRunnerAutoscaler` to scale on GitHub webhook events (for event name: `check_run`, type: `created`, status: `queued`). Note that the `actions-runner-controller` does not have any logic to automatically create the webhook configuration in GitHub, and hence, the webhook server needs to be exposed and configured manually in GitHub or using the GitHub API. If using `aws-load-balancer-controller`, ensure that within `actions-runner-controller` Helm chart, `githubWebhookServer.ingress.enabled` is set to `true`, and if using `external-dns`, set `githubWebhookServer.ingress.annotations` to include an `external-dns.alpha.kubernetes.io/alias`. Finally, configure the webhook in GitHub to match the hostname and port of the endpoint corresponding to the newly-created Ingress object. In general, Cloud Posse recommends using `actions-runner-controller` over EC2-based Runners due to the flexibility in runner sizing, choice of container image, and advanced horizontal scaling options. If however, EC2 Runners need to be utilized due to specific requirements such as a build environment on ARM-based instances, then that option is recommended as well. [https://github.com/summerwind/actions-runner-controller](https://github.com/summerwind/actions-runner-controller) ### Repository-wide or Organization-wide Runners Self-hosted GitHub Actions Runners can be made to be either repository-wide or organization-wide. Runners registered for a specific repository can only run for workflows corresponding to that repository, while Runners registered for an organization can run for any workflow for any repository in an organization, provided that the labels selected by the `runs-on` attribute in the workflow definition match the labels corresponding to the runner. Repo-level runners have the befit of reduced scope for PAT, however, pools are not shared across repos so there are wasted resources. In general, Cloud Posse recommends choosing Organization-wide Runners and ensuring horizontal scaling is configured to adequately respond to an influx of queued runs (see the previous section). [https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups](https://docs.github.com/en/actions/hosting-your-own-runners/managing-access-to-self-hosted-runners-using-groups) ### Labeling Runners Whenever a GitHub Actions Runner is registered, it provides a list of labels to GitHub. Then, workflow definitions can specify which Runners to run on. For example, if the workflow syntax specifies `runs-on: [self-hosted, linux]`, then the runner must be registered with the label `linux`. In another example, if two Runners are registered, one with the labels `linux`, `ubuntu`, and `medium`, and one with the labels `linux`, `ubuntu`, and `large`, and workflow A specifies `runs-on: [self-hosted, ubuntu]` and workflow B specifies `runs-on: [self-hosted, ubuntu, large]`, then: - Workflow A can run on both the first runner and the second runner. It’ll run on whichever is available. - Workflow B can only run on the second runner. Some advanced configurations may involve creating multiple `RunnerDeployment` CRDs (`actions-runner-controller`) which use different container images with different linux distributions or with different packages installed, then naming the labels accordingly. In general, Cloud Posse’s recommendation is to create meaningful runner labels that can be later referenced by developers writing GHA Workflow YAML files. ### Integration with AWS The GitHub Actions Runners often need to perform continuous integration tasks such as write to S3 or push container images to ECR. With GitHub-hosted Runners this has historically been difficult but is now made possible [using GitHub’s support for OIDC](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect), which allows for creating a trust relationship between GitHub-hosted Runners and an IAM role in one of the organization’s AWS account (preferably a dedicated `automation` account). For `actions-runner-controller`, this has been historically possible for a longer time now on self-hosted GitHub Actions Runners running on `actions-runner-controller` using EKS cluster’s OIDC provider (see: [IRSA](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-enable-IAM.html)). If using GHA Runners on EC2, an EC2 instance profile can be created, allowing the instances to assume an IAM role. ## Related Decisions - [Decide on IAM Roles for GitHub Action Runners](/resources/legacy/design-decisions/decide-on-iam-roles-for-github-action-runners) - [Decide on Self-Hosted GitHub Runner Strategy](/layers/software-delivery/design-decisions/decide-on-self-hosted-github-runner-strategy) - [Decide on Strategy for Continuous Integration](/layers/software-delivery/design-decisions/decide-on-strategy-for-continuous-integration) - [Decide on GitHub Actions Workflow Organization Strategy](/layers/software-delivery/design-decisions/decide-on-github-actions-workflow-organization-strategy) --- ## Decide on Strategy for Continuous Integration import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; # Decide on Strategy for Continuous Integration ### Considerations A strategy for Continuous Integration — i.e. container image builds, and single-page applications need to be adopted. There are different levels of testing. - Unit Tests - Integration Tests - Linting/Static Analysis Tests - Security Tests Centralized storage for test reports ## Options for Unit Tests ## Options for Integration Tests The options available for integration testing will depend to some degree on the technology. For example, single-page applications that are typically deployed to S3/CloudFront, for integration testing purposes might be still tested as dockerized apps. ### Option 1: Docker Composition with Test Script ### Option 2: Deployment to Cluster with Test Script Deploy a preview environment and then test it. Note: not all services are suitable for previews. ### Option 3: Test script ## Options for Linting/Static Analysis Tests - Superlinter ## Options for Security Tests --- ## Decide on Strategy for Developer Environments import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Versioning Considerations - Use latest in the default branch? - Use the latest release? - Use any developer-specified release? - How do overwrite it? ## Strategy Considerations - Local - Do your developer workstations/laptops have sufficient resources to build and launch all dependent services? - Remote - Hybrid ## Tool Considerations 1. Garden 2. Skaffold 3. Docker Compose --- ## Decide on Strategy for Managing and Orchestrating Secrets import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement We’ll inevitably need some way to store confidential key material and then access that from applications. Many options exist, and the best option depends on the circumstance. An often overlooked piece of the puzzle is how the key material will be created and updated, and how that ties into the broader release engineering workflow. ## Considerations ### Exfiltration From a security perspective, we ideally want to avoid direct access to application-level secrets in the CI/CD pipeline. This is achieved using two techniques: encrypted files or kubernetes operators and direct platform integration with the secrets store. When not using Kubernetes the operator approach is not suitable. ### Secrets for Applications - Secrets storage (e.g. SSM, ASM, Vault, Encrypt Files) - Secrets retrieval - Operators (e.g. external-secrets-operator) - Application Code Changes - Environment variables - Secrets orchestration (CRUD) ## Considered Options for GitHub Actions For all options, we assume secrets will be manually created and updated. ### Option A: GitHub Secrets During the CI or CD pipeline execution, it may be necessary to have access to secrets. For example, integration tokens with third-party vendors, or tokens to retrieve files (e.g. from VCS or S3). - GitHub Secrets Environment variables ### Option B: AWS Secrets (SSM, ASM) - GitHub Action reads from (SSM, ASM) storage leveraging GitHub OIDC to access secrets in AWS - IAM Roles & Permissions for applications - IAM Roles & permissions for teams ## Considered Options for ECS ### Option A: Use SSM Natively supported by ECS. The `chamber` tool is convenient for updating values. ### Option B: Use ASM Natively supported by ECS. ASM supports lambda hooks that can rotate secrets. Not as easy to manage on the command-line as using `chamber. ### Option C: Use S3 Story secrets in a KMS encrypted file in a private S3 bucket. Fetch the file as part of the container entrypoint script. This is only recommended if the number of secrets is so large we exceed the max document size of a container definition. We have run into this when migrating highly parameterized applications. ## Considered Options for EKS ### Option 1: Use SSM + External Secrets Operator (Recommended for EKS) :::tip We recommend this option because SSM is a centralized source of truth and well understood. ::: [https://external-secrets.io/](https://external-secrets.io/) #### Pros - Applications automatically updated when SSM values change - Easier to rotate secrets without CI/CD application deployments (a new `ReplicaSet` is created) #### Cons - es If many secrets change at around the same time, it can be disruptive to the application as each change causes kubernetes to create a new `ReplicaSet` as part of the Kubernetes `Deployment` ### Option 2: Use Sops Secrets Operator + KMS [https://github.com/isindir/sops-secrets-operator](https://github.com/isindir/sops-secrets-operator) #### Pros - Easily rollout secrets along side application deployments - Secrets are protected by KMS using IAM #### Cons - Secrets rotation requires application deployment - Mozilla SOPS project (despite being used by thousands of projects) lack maintainers. [https://github.com/mozilla/sops/discussions/927](https://github.com/mozilla/sops/discussions/927) ## References - [Decide on Secrets Management Strategy for Terraform](/layers/project/design-decisions/decide-on-secrets-management-strategy-for-terraform) --- ## Decide on Strategy for Preview Environments (e.g. Review Apps) import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Considerations - Use as few AWS proprietary services (E.g DynamoDB, SNS, SQS) because provisioning terraform for ephemeral environments is very slow, complicated and therefore not recommended - Use instead as many tools that have managed service equivalents in AWS (DocumentDB~MongoDB container, MSK~Kafka container, MySQL, Postgres) - Usage of API Gateway will require running terraform and will complicate preview environments - Preview environments are poor substitutions for remote development environments due to the slow feedback loop (e.g. commit, push, build, deploy) - Preview environments are not a replacement for staging and QA environments, which should have configurations that more likely resemble production - Multiple microservices are much harder to bring up if any sort of version pinning is required ### Use helmfile with raw chart :::caution We do not recommend this approach. ::: #### Pros - _Very rapid to prototype;_ no need to commit to any chart or convention - Repos/apps share nothing, so they won’t be affected by breaking changes in a shared chart (mitigated by versioning charts) #### Cons - This is the least DRY approach and the manifest for services are not reusable across microservices - Lots of manifests everywhere leads to inconsistency between services; adding some new convention/annotation requires updating every repo - No standardization of how apps are described for kubernetes (e.g. what we get with a custom helm chart) ### Use helmfile with custom chart Slight improvement over using `helmfile` with the `raw` chart. #### Pros - Reusable chart between services; very DRY - Chart can be tested and standardized to reduce the variations of applications deployed (e.g. NodeJS chart, Rails chart) - More conventional approach used by community at large (not cloudposse, but everyone using helm and helmfile) #### Cons - Using `helmfile` is one more tool; for new comers they often don’t appreciate the value it brings - CIOps is slowly falling out of favor for GitOps ### Use GitHub Actions directly with helm #### Pros - Very easy to understand what is happening with “CIOps” - Very easy to implement #### Cons - No record of the deployed state for preview environments in source control - Requires granted direct Kubernetes administrative access GitHub Action runners in order to deploy helm charts - GitHub Action runners will need direct access to read any secrets needed to deploy the helm releases. (mitigation is to use something like `sops-operator` or `external-secrets` operator) ### Use GitHub actions with ArgoCD and helm For some additional context on ArgoCD [Decide on ArgoCD Architecture](/layers/software-delivery/design-decisions/decide-on-argocd-architecture) ## Requirements - How quickly should environments come online? e.g. 5 minutes or less - How many backing services are required to validate your one service? - Do you need to pin dependent services at specific versions for previews? - if so, rearchitect how we do this - How should we name DNS for previews? - The biggest limitation is ACM and wildcard certs, so we’ll need a flat namespace - `https://pr-123-new-service.dev.acme.org`. (using the `*.dev.acme.org` ACM certificate) - URLs will be posted to GitHub Status API to that environments are directly reachable from PRs - Do we need to secure these environments? We recommend just locking down the ALB to internal traffic and using VPN - How will we handle databases for previews? How will we seed data. [Decide on Database Seeding Strategy for Ephemeral Preview Environments](/layers/software-delivery/design-decisions/decide-on-database-seeding-strategy-for-ephemeral-preview-enviro) - What is the effort to implement ArgoCD? - Very little, we have all the terraform code to deploy ArgoCD - We need to change the GitHub Actions to: - build the docker image (like to do) - render the helm chart to the raw k8s manifests - commit the manifests to deployment repo when ready to deploy - What is the simplest path we could take to implement and that developers will have the easiest time understanding? ## Patterns of Microservices This is more of a side note that not all microservices organizations are the same. If you’re using microservices, please self-identify with some of these patterns as it will be helpful in understanding the drivers behind them and how they are implemented. 1. As a result of acquisitions 2. As a result of architecture design from the beginning (this premature) 3. As a result of needing to use different languages for specific purposes 4. As a result of seeing performance needs to scale 5. If this is the case, we _technically_ don’t need to do microservices; we just need to be able to control the entry point & routing (e.g. a “Microservices Monolith”) 6. For this to work, the monolith needs to be able to communicate with itself as a service (e.g. gRPC) for local development. We see this with Go microservices; this can be done when it’s necessary as a pattern to scale endpoints 7. Preview environments can still use the gRPC but over localhost 8. As a result of wanting to experiment with multiple versions of the same service (E.g. using a service mesh) ## Related Design Decisions [Decide on Strategy for Preview Environments (e.g. Review Apps)](/layers/software-delivery/design-decisions/decide-on-strategy-for-preview-environments-e-g-review-apps) :::caution Internal preview environments cannot accept webhook callbacks from external services like twilio ::: --- ## Decide on Terraform Configuration Pattern for Application Repositories import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement The infrastructure monorepo that exists within an organization is responsible for configuring the core infrastructure of the organization: AWS accounts, VPCs, Kubernetes clusters, Route53, etc. However, AWS resources and/or other dependencies specific to a single application — such as a single S3 bucket — is not in the scope of the infrastructure monorepo, and should be managed externally, such that developers responsible for the application in question can manage its dependencies via infrastructure-as-code. ## Considered Options ### In-repo Terraform A Terraform configuration can be placed within the application repository and automated using `atmos`. This Terraform configuration requires the `infra-state.mixin.tf` mixin in order to be able to read the state of components in the infrastructure monorepo, for example from the `eks` component. #### Implementation This implementation is described in detail in the following guide: [How to Manage Terraform Dependencies in Micro-service Repositories](/learn/maintenance/tutorials/how-to-manage-terraform-dependencies-in-micro-service-repositori) . #### Scope The Terraform configuration within the application repository should have resources pertaining specifically to that application, specifically for the regional stack configured by `atmos` (see previous section). This includes: - An IAM Role for a ServiceAccount for that application (see: [IRSA](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-enable-IAM.html)) - An S3 bucket for the application - An SNS topic for the application - etc. These Terraform resources are not limited to the AWS provider. Other valid types of resources include: - LaunchDarkly Feature Flags - Datadog Monitors ## References - [How to Manage Terraform Dependencies in Micro-service Repositories](/learn/maintenance/tutorials/how-to-manage-terraform-dependencies-in-micro-service-repositori) --- ## Design Decisions(7) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for how you'll implement CI/CD for your applications. --- ## ECS with ecspresso import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CollapsibleText from '@site/src/components/CollapsibleText'; We use the [`ecspresso`](https://github.com/kayac/ecspresso) deployment tool for Amazon ECS to manage ECS services using a code-driven approach, alongside reusable GitHub Action workflows. This setup allows tasks to be defined with Terraform within the infrastructure repository, and task definitions to reside alongside the application code. Ecspresso provides extensive configuration options via YAML, JSON, and Jsonnet, and includes plugins for enhanced functionality such as Terraform state lookups. ```mermaid --- title: Ecspresso Deployment Lifecycle --- sequenceDiagram actor dev as Developer participant commit as Application box GitHub Action Workflow participant ci as CI participant deploy as CD end box ECS participant service as ECS Service participant deployment as ECS Deployment participant cluster as ECS Cluster end activate cluster dev ->>+ commit: Create Commit commit ->>+ ci: Trigger ci ->>+ deploy: Trigger deactivate ci deactivate commit deploy ->>+ service: RenderTask Definition loop deploy --> commit: WaitService Status end service ->>+ deployment: UpdateTask Definition deactivate service loop deployment ->> cluster: Removeold tasks deployment ->> cluster: Addnew tasks end deactivate deployment ``` ### Github Action Workflows The basic deployment flow is for feature branches. You can use the following sample workflow to add pull request deploys to your application repository: :::tip Latest Examples Check out our [example app-on-ecs](https://github.com/cloudposse-examples/app-on-ecs) for the latest example of how to use `ecspresso` with GitHub Actions. ::: ```yaml title=".github/workflows/feature-branch.yaml" name: 1 - Feature Branch on: pull_request: branches: [ main ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] permissions: pull-requests: write deployments: write id-token: write contents: read concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: false jobs: monorepo: uses: cloudposse/github-actions-workflows/.github/workflows/controller-monorepo.yml@main with: file: ./deploy/config.yaml ci: uses: cloudposse/github-actions-workflows/.github/workflows/ci-dockerized-app-build.yml@main needs: [ monorepo ] with: organization: "cloudposse" repository: ${{ github.event.repository.name }} secrets: ecr-region: ${{ secrets.ECR_REGION }} ecr-iam-role: ${{ secrets.ECR_IAM_ROLE }} registry: ${{ secrets.ECR_REGISTRY }} secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} cd: uses: cloudposse/github-actions-workflows/.github/workflows/cd-preview-ecspresso.yml@main needs: [ ci, monorepo ] if: ${{ always() && needs.monorepo.outputs.apps != '[]' }} strategy: matrix: app: ${{ fromJson(needs.monorepo.outputs.apps) }} with: image: ${{ needs.ci.outputs.image }} tag: ${{ needs.ci.outputs.tag }} repository: ${{ github.event.repository.name }} app: ${{ matrix.app }} open: ${{ github.event.pull_request.state == 'open' }} labels: ${{ toJSON(github.event.pull_request.labels.*.name) }} ref: ${{ github.event.pull_request.head.ref }} exclusive: true enable-migration: ${{ contains(fromJSON(needs.monorepo.outputs.migrations), matrix.app) }} settings: ${{ needs.monorepo.outputs.settings }} env-label: | qa1: deploy/qa1 secrets: secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} ``` ```yaml title=".github/workflows/main-branch.yaml" name: 2 - Main Branch on: push: branches: [ main ] permissions: contents: write id-token: write pull-requests: read concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: false jobs: monorepo: uses: cloudposse/github-actions-workflows/.github/workflows/controller-monorepo.yml@main with: file: ./deploy/config.yaml ci: uses: cloudposse/github-actions-workflows/.github/workflows/ci-dockerized-app-build.yml@main needs: [ monorepo ] with: organization: "cloudposse" repository: ${{ github.event.repository.name }} secrets: ecr-region: ${{ secrets.ECR_REGION }} ecr-iam-role: ${{ secrets.ECR_IAM_ROLE }} registry: ${{ secrets.ECR_REGISTRY }} secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} cd: uses: cloudposse/github-actions-workflows/.github/workflows/cd-ecspresso.yml@main needs: [ ci, monorepo ] strategy: matrix: app: ${{ fromJson(needs.monorepo.outputs.apps) }} with: image: ${{ needs.ci.outputs.image }} tag: ${{ needs.ci.outputs.tag }} repository: ${{ github.event.repository.name }} app: ${{ matrix.app }} environment: dev enable-migration: ${{ contains(fromJSON(needs.monorepo.outputs.migrations), matrix.app) }} settings: ${{ needs.monorepo.outputs.settings }} secrets: secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} release: uses: cloudposse/github-actions-workflows/.github/workflows/controller-draft-release.yml@main needs: [ cd ] ``` ```yaml title=".github/workflows/release.yaml" name: 3 - Release on: release: types: [published] permissions: id-token: write contents: write concurrency: group: ${{ github.workflow }} cancel-in-progress: false jobs: monorepo: uses: cloudposse/github-actions-workflows/.github/workflows/controller-monorepo.yml@main with: file: ./deploy/config.yaml ci: uses: cloudposse/github-actions-workflows/.github/workflows/ci-dockerized-app-promote.yml@main needs: [ monorepo ] with: organization: "cloudposse" repository: ${{ github.event.repository.name }} version: ${{ github.event.release.tag_name }} secrets: ecr-region: ${{ secrets.ECR_REGION }} ecr-iam-role: ${{ secrets.ECR_IAM_ROLE }} registry: ${{ secrets.ECR_REGISTRY }} secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} cd: uses: cloudposse/github-actions-workflows/.github/workflows/cd-ecspresso.yml@main needs: [ ci, monorepo ] strategy: matrix: app: ${{ fromJson(needs.monorepo.outputs.apps) }} with: image: ${{ needs.ci.outputs.image }} tag: ${{ needs.ci.outputs.tag }} repository: ${{ github.event.repository.name }} app: ${{ matrix.app }} environment: "staging" enable-migration: ${{ contains(fromJSON(needs.monorepo.outputs.migrations), matrix.app) }} settings: ${{ needs.monorepo.outputs.settings }} secrets: secret-outputs-passphrase: ${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }} ``` ## References - [Ecspresso](https://github.com/kayac/ecspresso) : Tool repo - [example-app-on-ecs](https://github.com/cloudposse/example-app-on-ecs): Example app - [github-action-deploy-ecspresso](https://github.com/cloudposse/github-action-deploy-ecspresso): Base action - [`cd-ecspresso`](https://github.com/cloudposse/github-actions-workflows/blob/main/.github/workflows/cd-ecspresso.yml): Primary workflow - [`cd-preview-ecspresso`](https://github.com/cloudposse/github-actions-workflows/blob/main/.github/workflows/cd-preview-ecspresso.yml): feature branch workflow --- ## ECS Partial Task Definitions import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; This document describes what partial task definitions are and how we can use them to set up ECS services using Terraform and GitHub Actions. ## The Problem Managing ECS Services is challenging. Ideally, we want our services to be managed by Terraform so everything is living in code. However, we also want to update the task definition via GitOps as through the GitHub release lifecycle. This is challenging because Terraform can create the task definition, but if updated by the application repository, the task definition will be out of sync with the Terraform state. Managing it entirely through Terraform means we cannot easily update the newly built image by the application repository unless we directly commit to the infrastructure repository, which is not ideal. Managing it entirely through the application repository means we cannot codify the infrastructure and have to hardcode ARNs, secrets, and other infrastructure-specific configurations. ## Introduction ECS Partial task definitions is the idea of breaking the task definition into smaller parts. This allows for easier management of the task definition and makes it easier to update the task definition. We do this by setting up Terraform to manage a portion of the task definition, and the application repository to manage another portion. The Terraform (infrastructure) portion is created first. It will create an ECS Service in ECS, and then upload the task definition JSON to S3 as `task-template.json`.The application repository will have a `task-definition.json` git controlled, during the development lifecycle, the application repository will download the task definition from S3, merge the task definitions, then update the ECS Service with the new task definition. Finally, GitHub actions will update the S3 bucket with the deployed task definition under `task-definition.json`. If Terraform is planned again, it will use the new task definition as the base for the next deployment, thus not resetting the image or application configuration. ### Pros The **benefit** to using this approach is that we can manage the task definition portion in Terraform with the infrastructure, meaning secrets, volumes, and other ARNs can be managed in Terraform. If a filesystem ID updates we can re-apply Terraform to update the task definition with the new filesystem ID. The application repository can manage the container definitions, environment variables, and other application-specific configurations. This allows developers who are closer to the application to quickly update the environment variables or other configuration. ### Cons The drawback to this approach is that it is more complex than managing the task definition entirely in Terraform or the application repository. It requires more setup and more moving parts. It can be confusing for a developer who is not familiar with the setup to understand how the task definition is being managed and deployed. This also means that when something goes wrong, it becomes harder to troubleshoot as there are more moving parts. ### Getting Setup #### Pre-requisites - Application Repository - [Cloud Posse Example ECS Application](https://github.com/cloudposse-examples/app-on-ecs) - Infrastructure Repository - ECS Cluster - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/ecs/) - [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs). - `ecs-service` - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/ecs-service/) - [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs-service). - **Must** use the Cloud Posse Component. - [`v1.416.0`](https://github.com/cloudposse/Terraform-aws-components/releases/tag/1.416.0) or later. - S3 Bucket - [Cloud Posse Docs](https://docs.cloudposse.com/components/library/aws/s3-bucket/) - [Component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/s3-bucket). #### Steps 1. Set up the S3 Bucket that will store the task definition. This bucket should be in the same account as the ECS Cluster.
S3 Bucket Default Definition ```yaml components: terraform: s3-bucket/defaults: metadata: type: abstract vars: enabled: true account_map_tenant_name: core # Suggested configuration for all buckets user_enabled: false acl: "private" grants: null force_destroy: false versioning_enabled: false allow_encrypted_uploads_only: true block_public_acls: true block_public_policy: true ignore_public_acls: true restrict_public_buckets: true allow_ssl_requests_only: true lifecycle_configuration_rules: - id: default enabled: true abort_incomplete_multipart_upload_days: 90 filter_and: prefix: "" tags: {} # Move to Glacier after 2 years transition: - storage_class: GLACIER days: 730 # Never expire expiration: {} # Versioning isn't enabled, but these default values are still required noncurrent_version_transition: - storage_class: GLACIER days: 90 noncurrent_version_expiration: {} ```
```yaml import: - catalog/s3-bucket/defaults components: terraform: s3-bucket/ecs-tasks-mirror: #NOTE this is the component instance name. metadata: component: s3-bucket inherits: - s3-bucket/defaults vars: enabled: true name: ecs-tasks-mirror ``` 2. Create an ECS Service in Terraform Set up the ECS Service in Terraform using the [`ecs-service` component](https://github.com/cloudposse/Terraform-aws-components/tree/main/modules/ecs-service). This will create the ECS Service and upload the task definition to the S3 bucket. To enable Partial Task Definitions, set the variable `s3_mirror_name` to be the component instance name of the bucket to mirror to. For example `s3-bucket/ecs-tasks-mirror` ```yaml components: terraform: ecs-services/defaults: metadata: component: ecs-service type: abstract vars: enabled: true ecs_cluster_name: "ecs/cluster" s3_mirror_name: s3-bucket/ecs-tasks-mirror ``` 3. Set up an Application repository with GitHub workflows. An example application repository can be found [here](https://github.com/cloudposse-examples/app-on-ecs). Two things need to be pulled from this repository: - The `task-definition.json` file under `deploy/task-definition.json` - The GitHub Workflows. An important note about the GitHub Workflows, in the example repository they all live under `.github/workflows`. This is done so development of workflows can be fast, however we recommend moving the shared workflows to a separate repository and calling them from the application repository. The application repository should only contain the workflows `main-branch.yaml`, `release.yaml` and `feature-branch.yml`. To enable Partial Task Definitions in the workflows, the call to [`cloudposse/github-action-run-ecspresso` (link)](https://github.com/cloudposse-examples/app-on-ecs/blob/main/.github/workflows/workflow-cd-ecspresso.yml#L133-L147) should have the input `mirror_to_s3_bucket` set to the S3 bucket name. the variable `use_partial_taskdefinition` should be set to `'true'`
Example GitHub Action Step ```yaml - name: Deploy uses: cloudposse/github-action-deploy-ecspresso@0.6.0 continue-on-error: true if: ${{ steps.db_migrate.outcome != 'failure' }} id: deploy with: image: ${{ steps.image.outputs.out }} image-tag: ${{ inputs.tag }} region: ${{ steps.environment.outputs.region }} operation: deploy debug: false cluster: ${{ steps.environment.outputs.cluster }} application: ${{ steps.environment.outputs.name }} taskdef-path: ${{ inputs.path }} mirror_to_s3_bucket: ${{ steps.environment.outputs.s3-bucket }} use_partial_taskdefinition: "true" timeout: 10m ```
## Operation Changes through Terraform will not immediately be reflected in the ECS Service. This is because the task template has been updated, but whatever was in the `task-definition.json` file in the S3 bucket will be used for deployment. To update the ECS Service after updating the Terraform for it, you must deploy through GitHub Actions. This will then download the new template and create a new updated `task-definition.json` to store in s3. --- ## Setting up ecspresso import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Note from '@site/src/components/Note' import Admonition from '@theme/Admonition' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; This setup guide will help you get started with [ecspresso](https://github.com/kayac/ecspresso). It features an example app, which demonstrates how your GitHub Actions work with your infrastructure repository. | Steps | Actions | | ---------------------------------------------------- | --------------------------------------------------------------------------------- | | 1. Create a repository from the Example App template | [cloudposse-examples/app-on-ecs](https://github.com/cloudposse-examples/app-on-ecs) | | 2. Update and reapply `ecr` | `atmos terraform apply ecr -s core-use1-artifacts` | | 3. Validate the environment configuration | Click Ops | | 4. Create a GitHub PAT | Click Ops | | 5. Set all Example App repository secrets | Click Ops | | 6. Deploy the shared ECS Task Definition S3 Bucket | `atmos apply s3-bucket/ecs-tasks-mirror -s < YOUR STACK >` | | 7. Deploy the example ECS services | `atmos workflow deploy/app-on-ecs -f quickstart/app/app-on-ecs` | We recommend moving all workflows with `ecspresso` and `workflow` **prefixes** to a shared workflow repository that can be used by the rest of your organization. We do not recommend keeping all shared workflows in the same repository as in this example, because it defeats the reusability. We've included all workflows in the example app repositories to make it easier to follow along and document. ### Create the Example App repository This step requires access to the GitHub Organization. Customers will need to create this GitHub repository in Jumpstart engagements. Cloud Posse deploys an example application with this new repository. This is separate from the infrastructure repository and can be used as reference for future applications. Cloud Posse also maintains a public example of this app repository with [cloudposse/example-app-on-ecs](https://github.com/cloudposse/example-app-on-ecs/). This is a GitHub repository template, meaning it can be used to create new repositories with predefined content. 1. Create a new repository in your organization from the [cloudposse/example-app-on-ecs](https://github.com/cloudposse/example-app-on-ecs/) template. 2. Choose any name for the repository. For example, we might call this repo `acme/example-app-on-ecs`. 3. Grant Cloud Posse `admin` access to this repository. 4. If necessary, install the self-hosted runner GitHub App to this new repository. ### Create Image and GitHub OIDC Access Roles for ECR The Example App will build and push an image to the ECR registry. Create that image with the `ecr` component if not already created. The Example App GitHub Workflows will also need to be able to access that registry, and to do so, we deploy GitHub OIDC roles with the same `ecr` component. Add the following snippet in addition to any other repositories or images already included in these lists: ```yaml components: terraform: ecr: vars: github_actions_allowed_repos: - acme/example-app-on-ecs # ECR must be all lowercase images: - acme/example-app-on-ecs ``` Reapply the `ecr` component with the following: ```console atmos terraform apply ecr -s core-use1-artifacts ``` ### Configure the Environment We use the [cloudposse/github-action-interface-environment](https://github.com/cloudposse/github-action-interface-environment) GitHub Composite Action to read environment configuration from a private location. By default, we use the infrastructure repository as that private location and save the configuration to `.github/environments/ecspresso.yaml`. This action stores metadata about the environments we want to deploy to. It is the binding glue between our GHA, GitHub environments, and our infrastructure. When this action is called, an `environment` input is passed in. We then look up in the map below information about that environment, that information is stored as an output to be used by the rest of the GitHub actions. For more on GitHub Composite Actions, please see the [official GitHub documentation](https://docs.github.com/en/actions/creating-actions/creating-a-composite-action). Create or confirm the configuration in `.github/environments/ecspresso.yaml` in the `acme/infra-acme` repository now. If the file doesn't exist, here's the template: The `role` defined in this configuration may not exist yet. This role will be created by the given `ecs-service` component with the GitHub OIDC mixin. Once completing the [Deploy the Example App ECS Service](#deploy-the-example-app-ecs-service) step, please verify this role is correct.
Copy, paste, and edit this in ./.github/environments/ecspresso.yaml ```yaml name: 'Environments' description: 'Get information about cluster' inputs: environment: description: "Environment name" required: true namespace: description: "Namespace name" required: true repository: description: "Repository name" required: false application: description: "Application name" required: false attributes: description: "Comma separated attributes" required: false outputs: name: description: "Environment name" value: ${{ steps.result.outputs.name }} region: description: "AWS Region" value: ${{ steps.result.outputs.region }} role: description: "IAM Role" value: ${{ steps.result.outputs.role }} cluster: description: "Cluster" value: ${{ steps.result.outputs.cluster }} namespace: description: "Namespace" value: ${{ steps.result.outputs.namespace }} ssm-path: description: "SSM path" value: ${{ steps.result.outputs.ssm-path }} s3-bucket: description: "S3 Bucket name" value: ${{ steps.result.outputs.s3-bucket }} account-id: description: "AWS account id" value: ${{ steps.result.outputs.aws-account-id }} stage: description: "Stage name" value: ${{ steps.result.outputs.stage }} runs: using: "composite" steps: - uses: cloudposse/github-action-yaml-config-query@0.1.0 id: suffix with: query: .${{ inputs.application == '' }} config: | true: suffix: ${{ inputs.repository }} false: suffix: ${{ inputs.repository }}-${{ inputs.application }} - uses: cloudposse/github-action-yaml-config-query@0.1.0 id: result with: query: .${{ inputs.environment }} config: | qa1: cluster: acme-plat-${{ steps.region.outputs.result }}-dev-ecs-platform name: acme-plat-${{ steps.region.outputs.result }}-dev-${{ steps.name.outputs.name }}-qa1 role: arn:aws:iam::101010101010:role/acme-plat-${{ steps.region.outputs.result }}-dev-${{ steps.name.outputs.name }}-qa1 ssm-path: /ecs-service/${{ steps.name.outputs.name }}/url/0 region: us-east-1 qa2: cluster: acme-plat-${{ steps.region.outputs.result }}-dev-ecs-platform name: acme-plat-${{ steps.region.outputs.result }}-dev-${{ steps.name.outputs.name }}-qa2 role: arn:aws:iam::101010101010:role/acme-plat-${{ steps.region.outputs.result }}-dev-${{ steps.name.outputs.name }}-qa2 ssm-path: /ecs-service/${{ steps.name.outputs.name }}/url/0 region: us-east-1 dev: cluster: acme-plat-use1-dev-ecs-platform name: acme-plat-use1-dev-${{ steps.suffix.outputs.suffix }} role: arn:aws:iam::101010101010:role/acme-plat-use1-dev-${{ steps.suffix.outputs.suffix }} ssm-path: /ecs-service/${{ steps.suffix.outputs.suffix }}/url/0 region: us-east-1 s3-bucket: acme-plat-use1-dev-ecs-tasks-mirror aws-account-id: 101010101010 stage: dev prod: cluster: acme-plat-use1-prod-ecs-platform name: acme-plat-use1-prod-${{ steps.suffix.outputs.suffix }} role: arn:aws:iam::202020202020:role/acme-plat-use1-prod-${{ steps.suffix.outputs.suffix }} ssm-path: /ecs-service/${{ steps.suffix.outputs.suffix }}/url/0 region: us-east-1 s3-bucket: acme-plat-use1-prod-ecs-tasks-mirror aws-account-id: 202020202020 stage: prod sandbox: cluster: acme-plat-use1-sandbox-ecs-platform name: acme-plat-use1-sandbox-${{ steps.suffix.outputs.suffix }} role: arn:aws:iam::303030303030:role/acme-plat-use1-sandbox-${{ steps.suffix.outputs.suffix }} ssm-path: /ecs-service/${{ steps.suffix.outputs.suffix }}/url/0 region: us-east-1 s3-bucket: acme-plat-use1-sandbox-ecs-tasks-mirror aws-account-id: 303030303030 stage: sandbox staging: cluster: acme-plat-use1-staging-ecs-platform name: acme-plat-use1-staging-${{ steps.suffix.outputs.suffix }} role: arn:aws:iam::404040404040:role/acme-plat-use1-staging-${{ steps.suffix.outputs.suffix }} ssm-path: /ecs-service/${{ steps.suffix.outputs.suffix }}/url/0 region: us-east-1 s3-bucket: acme-plat-use1-staging-ecs-tasks-mirror aws-account-id: 404040404040 stage: staging ```
Then the Example App, verify that the target environment is correct. This should be in the `.github/configs/environment.yaml` file in Example App repository. ```yaml ## file: .github/configs/environment.yaml # assumes the same organization environment-info-repo: infrastructure implementation_path: .github/environments implementation_file: ecspresso.yaml implementation_ref: main ```
### Create a GitHub PAT This step requires access to the GitHub Organization. Customers will need to create this PAT in Jumpstart engagements. In order for the Example App workflows to read the private environment configuration, we need to pass a token to the Composite Action. 1. Create a fine-grained PAT. Please see [Creating a fine-grained personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-fine-grained-personal-access-token). 2. Name this PAT whatever you would like. We recommend calling it `PRIVATE_CONFIG_READ_ACCESS` 3. Grant this PAT `read` permission on the `acme/infra-acme` repository: ```diff Repository + Contents: Read-only + Metadata: Read-only ``` 4. Upload this PAT to 1Password, and Cloud Posse will add it as a GitHub repository secret. Or you can create an organization secret now that you can reuse in future application repositories. ### Add the Example App Secrets The GitHub Action workflows expect a few GitHub Secrets to exist to build images in AWS ECR. Add each of the following secrets to the Example App repository:
`PRIVATE_CONFIG_READ_ACCESS`
This is the PAT we created above in Create a GitHub PAT.
`ECR_REGISTRY`
This is your ECR Registry, such as `111111111111.dkr.ecr.us-east-1.amazonaws.com`.
`ECR_REGION`
This is the AWS region where the `ecr` component is deployed. For example, `us-east-1`
`ECR_IAM_ROLE`
This is the GitHub OIDC role created by the `ecr` component for accessing the registry. For this organization, this would be `arn:aws:iam::111111111111:role/acme-core-use1-artifacts-ecr-gha`. Verify this value by checking the output of the `ecr` component.
`GHA_SECRET_OUTPUT_PASSPHRASE`
This is a random string used to encrypt and decrypt sensitive image names and tags. This can be anything. For example, generate this with the following: ```console openssl rand -base64 24 ```
### Configure the S3 Mirror Bucket, if not already configured If you haven't already configured the S3 mirror bucket, deploy and configure the shared S3 bucket for ECS tasks definitions now. Follow the [ECS Partial Task Definitions guide](/layers/software-delivery/ecs-ecspresso/ecs-partial-task-definitions/#steps) ### Deploy the Example App ECS Service Ensure you have stacks configured for the Example App in every stage of your platform. This task definition uses the `latest` ECR image for the Example App, which is built by the CI steps of the release pipelines. However, that step hasn't been run yet! You will need to first trigger the `main-branch` CI steps for the Example App, ignore the failure in the deploy step, and then deploy these components.
Catalog entry for the Example App ```yaml import: - catalog/ecs-services/defaults components: terraform: ecs-services/example-app-on-ecs: metadata: component: ecs-service inherits: - ecs-services/defaults vars: name: example-app-on-ecs ssm_enabled: true github_actions_iam_role_enabled: true github_actions_iam_role_attributes: [ "gha" ] github_actions_ecspresso_enabled: true github_actions_allowed_repos: - acme/example-app-on-ecs cluster_attributes: [platform] alb_configuration: "private" use_lb: true unauthenticated_paths: - "/" - "/dashboard" containers: service: name: app image: 111111111111.dkr.ecr.us-east-1.amazonaws.com/example-app-on-ecs:latest log_configuration: logDriver: awslogs options: {} port_mappings: - containerPort: 8080 hostPort: 8080 protocol: tcp task: desired_count: 1 task_memory: 512 task_cpu: 256 ignore_changes_desired_count: true ignore_changes_task_definition: true ```
Finally, apply the `ecs-services/example-app-on-ecs` component to deploy the Example App ECS service.
## Triggering Workflows Now that all requirements are in place, validate all workflows. 1. ### Clone the Example App locally ```bash git clone git@github.com:acme/example-app-on-ecs.git ``` 2. ### Change the demo color in `main.go` ```go func main() { c := os.Getenv("COLOR") if len(c) == 0 { c = "red" // change this color to something else, such as "blue" } ``` 3. ### Create a Pull Request Creating a PR will trigger the CI build and test workflows and the QA cleanup workflows. Ensure these all pass successfully. 4. ### Add the `deploy/qa1` label Adding this label will kickoff a new workflow to build and test once again and then deploy to the `qa1` environment. Ensure this workflow passes successfully and then validate the "Deployment URL" returned. Private endpoints require the VPN. If you're deploying a private endpoint, connect to the VPN in order to access the deployment URL. 5. ### Merge the Pull Request Merging the PR will trigger two different workflows. The Feature Branch workflow will be triggered to clean up and release the QA environment, and the Main Branch workflow will be triggered to deploy to `dev` and draft a release. Once both workflows pass, check that the QA environment is no longer active and then validate the dev URL. Finally, make sure a draft release was successfully created. 6. ### Publish a Release Using the draft release created by the Main Branch workflow, click Edit and then Publish. This will kick off the Release workflow and deploy to `staging` and then to `prod`. Once this workflow finishes, validate both endpoints. ## Next Steps Workflows with `ecspresso` and `workflow` **prefixes** should be moved to a shared workflow repository that can be used by the rest of your organization. ## FAQ ### Adding Additional Applications This setup is a one time setup. You can add as many applications as you want to your platform. You can also add as many environments as you want to your platform. To add additional applications: 1. Ensure the `ecspresso` and `workflow` **prefixes** are moved to a shared workflow repository that can be used by the rest of your organization. 2. Create a new repository from one of the example app templates. 3. Create your Example app Configuration file in the new repository. 4. Ensure your infrastructure is deployed. ## References - [Ecspresso](https://github.com/kayac/ecspresso) : Tool repo - [example-app-on-ecs](https://github.com/cloudposse/example-app-on-ecs): Example app - [github-action-deploy-ecspresso](https://github.com/cloudposse/github-action-deploy-ecspresso): Base action - [`cd-ecspresso`](https://github.com/cloudposse/github-actions-workflows/blob/main/.github/workflows/cd-ecspresso.yml): Primary workflow - [`cd-preview-ecspresso`](https://github.com/cloudposse/github-actions-workflows/blob/main/.github/workflows/cd-preview-ecspresso.yml): feature branch workflow --- ## EKS with ArgoCD import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Note from '@site/src/components/Note'; import CollapsibleText from '@site/src/components/CollapsibleText'; Argo CD is an open-source declarative, GitOps continuous delivery tool for Kubernetes applications. It enables developers to manage and deploy applications on Kubernetes clusters using Git repositories as the source of truth for configuration and definitions. Argo CD follows the GitOps methodology, which means that the entire application configuration, including manifests, parameters, and even application state, is stored in a Git repository. #### SAML Security Considerations SAML is an industry-standard but security concerns have been raised by Dex, Mastadon, and others, due to the inherent difficulty of validating XML documents and inconsistent handling by SAML libraries in various languages. Our ArgoCD implementation by default uses SAML authentication with Dex and ArgoCD. For more information, please see: - [SAML is insecure by design](https://joonas.fi/2021/08/saml-is-insecure-by-design/) - [SAML Raider - SAML2 Burp Extension](https://github.com/CompassSecurity/SAMLRaider) - [Proposal: deprecate the SAML connector](https://github.com/dexidp/dex/discussions/1884) - [Mattermost blog post of July 28, 2021 where `@jupenur`](https://mattermost.com/blog/securing-xml-implementations-across-the-web/) states: > If you maintain an application in Ruby, JavaScript, .NET, or Java and rely on SAML or other security-critical XML > use-cases, the question burning in the back of your mind should be: "How do I patch this?" The good news is that you > should already be patched if you use Ruby or JavaScript and update your dependencies regularly. And if you use .NET > or Java, there's probably nothing to worry about. ### Overview Argo CD simplifies the deployment and management of applications on Kubernetes by leveraging GitOps principles, providing a clear separation between the desired state of applications and the operational state of the cluster. This approach enhances collaboration, repeatability, and traceability in the deployment process. ```mermaid --- title: ArgoCD Deployment Lifecycle --- sequenceDiagram actor dev as Developer participant commit as Application box GitHub Action Workflow participant ci as CI participant deploy as CD end participant repo as ArgoCD Repo box EKS participant argocd as ArgoCD participant k8s as K8S API end activate argocd loop Lookup changes argocd --> repo: Pull Desired State end dev ->>+ commit: Create Commit commit ->>+ ci: Trigger ci ->>+ deploy: Trigger deactivate ci deactivate commit deploy ->>+ deploy: Render Manifest deploy ->>+ repo: Commit Desired State deactivate deploy loop deploy --> commit: Wait Commit Status end opt New Desired State Found argocd ->> repo: Pull Desired State deactivate repo activate argocd argocd ->> k8s: Reconcile State argocd ->>+ commit: Set Commit Status end deactivate argocd loop deploy ->> commit: Wait Commit Status commit ->>- deploy: Commit Status Success end deactivate deploy deactivate argocd ``` ### Deployment Application repository will create a deployment when a workflow is triggered and call the relevant shared workflow. :::tip Latest Examples Check out our [example app-on-eks-with-argocd](https://github.com/cloudposse-examples/app-on-eks-with-argocd) for the latest example of how to use ArgoCD with GitHub Actions. ::: ```yaml title=".github/workflows/feature-branch.yaml" name: Feature Branch on: pull_request: branches: [ 'main' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] permissions: pull-requests: write deployments: write id-token: write contents: read jobs: do: uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/feature-branch.yml@main with: organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" open: ${{ github.event.pull_request.state == 'open' }} labels: ${{ toJSON(github.event.pull_request.labels.*.name) }} ref: ${{ github.event.pull_request.head.ref }} secrets: github-private-actions-pat: "${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}" registry: "${{ secrets.ECR_REGISTRY }}" secret-outputs-passphrase: "${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}" ecr-region: "${{ secrets.ECR_REGION }}" ecr-iam-role: "${{ secrets.ECR_IAM_ROLE }}" ``` ```yaml title=".github/workflows/main-branch.yaml" name: Main Branch on: push: branches: [ main ] permissions: contents: write id-token: write jobs: do: uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/main-branch.yml@main with: organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" secrets: github-private-actions-pat: "${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}" registry: "${{ secrets.ECR_REGISTRY }}" secret-outputs-passphrase: "${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}" ecr-region: "${{ secrets.ECR_REGION }}" ecr-iam-role: "${{ secrets.ECR_IAM_ROLE }}" ``` ```yaml title=".github/workflows/release.yaml" name: Release on: release: types: [published] permissions: id-token: write contents: write jobs: perform: uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/release.yml@main with: organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" version: ${{ github.event.release.tag_name }} secrets: github-private-actions-pat: "${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}" registry: "${{ secrets.ECR_REGISTRY }}" secret-outputs-passphrase: "${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}" ecr-region: "${{ secrets.ECR_REGION }}" ecr-iam-role: "${{ secrets.ECR_IAM_ROLE }}" ``` That workflow calls a Reusable Workflow, `cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd`, that is designed to deploy a dockerized application from ECR to EKS using ArgoCD specifically. ### Hotfix Workflows Hotfix workflows are designed to push changes directly to a released version in production. Ideally we want any change to move through the standard release lifecycle, but in reality there are times when we need the ability to push a hotfix directly to production. In order to enable hotfix workflows, create two additional workflows and modify the existing release workflow. See each of the following workflows: Before running any hotfix workflows, we must first create release branches with any release. Modify the existing release workflow to include the `hotfix` job below. ```yaml title=".github/workflows/release.yaml" name: Release on: release: types: [published] permissions: id-token: write contents: write jobs: perform: ... hotfix: name: release / branch uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/hotfix-mixin.yml@main with: version: ${{ github.event.release.tag_name }} ``` This `hotfix-branch.yaml` workflow will deploy a duplicate app in the _production_ cluster to a new namespace. We need to deploy to production to validate a hotfix directly for production. Deploy this workflow by creating a Pull Request into the a release branch and adding the `deploy` label. ```yaml title=".github/workflows/hotfix-branch.yaml" name: Hotfix Branch on: pull_request: branches: [ 'release/**' ] types: [opened, synchronize, reopened, closed, labeled, unlabeled] permissions: pull-requests: write deployments: write id-token: write contents: read jobs: do: uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/hotfix-branch.yml@main with: organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" open: ${{ github.event.pull_request.state == 'open' }} labels: ${{ toJSON(github.event.pull_request.labels.*.name) }} ref: ${{ github.event.pull_request.head.ref }} path: deploy secrets: github-private-actions-pat: "${{ secrets.PRIVATE_REPO_ACCESS_TOKEN }}" registry: "${{ secrets.ECR_REGISTRY }}" secret-outputs-passphrase: "${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}" ecr-region: "${{ secrets.ECR_REGION }}" ecr-iam-role: "${{ secrets.ECR_IAM_ROLE }}" ``` Once we've validated a Pull Request for a given hotfix, we can merge that change into the release branch. When changes are pushed to a release branch, the "Hotfix Release" workflow is triggered. _This workflow will deploy the given change directly to production_. Before deploying, the workflow will create a minor version release and test it. After the deployment, it will create a reintegration pull request to bring the hotfix back into the main branch and lower environments. In order to enable the "Hotfix Release" workflow, add the following: ```yaml title=".github/workflows/hotfix-release.yaml" name: Hotfix Release on: push: branches: [ 'release/**' ] permissions: contents: write id-token: write jobs: do: uses: cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd/.github/workflows/hotfix-release.yml@main with: organization: "${{ github.event.repository.owner.login }}" repository: "${{ github.event.repository.name }}" path: deploy secrets: github-private-actions-pat: "${{ secrets.PRIVATE_REPO_ACCESS_TOKEN }}" registry: "${{ secrets.ECR_REGISTRY }}" secret-outputs-passphrase: "${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}" ecr-region: "${{ secrets.ECR_REGION }}" ecr-iam-role: "${{ secrets.ECR_IAM_ROLE }}" ``` These workflows also call the same Reusuable Workflow repository, `cloudposse/github-actions-workflows-docker-ecr-eks-helm-argocd`, as well as several of the same Reusuable Workflows called from that repository. For example, `cloudposse/github-actions-workflows` and `cloudposse/actions-private`. :::tip Verify environment configs carefully Be sure the environment configuration mapping includes `hotfix`. This typically lives with your private configuration repository, for example `cloudposse/actions-private`, and is called by the `cloudposse/github-action-interface-environment` action. For example, add the following: ```yaml runs: using: "composite" steps: - name: Environment info uses: cloudposse/github-action-yaml-config-query@0.1.0 id: result with: query: .${{ inputs.environment }} config: | ... hotfix: cluster: https://github.com/GH_ORG/argocd-deploy-prod/blob/main/plat/use2-prod/apps cluster-role: arn:aws:iam::PRODUCTION_ACCOUNT_ID:role/acme-plat-use2-prod-eks-cluster-gha namespace: ${{ inputs.namespace }} ssm-path: platform/acme-plat-use2-prod-eks-cluster ``` ::: ### Implementation - [`eks/argocd`](/components/library/aws/eks/argocd/): This component is responsible for provisioning [ArgoCD](https://argoproj.github.io/cd/). - [`argocd-repo`](/components/library/aws/argocd-github-repo/): This component is responsible for creating and managing an ArgoCD desired state repository. - [`sso-saml-provider`](/components/library/aws/sso-saml-provider/): This component reads sso credentials from SSM Parameter store and provides them as outputs ## References - [ArgoCD Setup](/layers/software-delivery/eks-argocd/setup) - [Decide on Pipeline Strategy](/layers/software-delivery/design-decisions/decide-on-pipeline-strategy) --- ## Setup Argo CD import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Admonition from '@theme/Admonition' import TaskList from '@site/src/components/TaskList' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; This setup guide will walk you through the process of setting up Argo CD in your environment. ## Requirements In order to deploy Argo CD, EKS must be fully deployed and functional. In particular, the user deploying the cluster must have a working VPN connection to the targeted account. See [the EKS documentation](/layers/eks/deploy-clusters/) for details. All deployment steps below assume that the environment has been successfully set up with the following steps. ### Authentication 1. Sign into AWS via Leapp 1. Connect to the VPN 1. Open Geodesic ## Setup Steps ## Vendor Argo CD components First vendor all related components for the Argo CD layer: ## Create Argo CD GitHub Repositories - Create the two required Argo CD GitHub repos: - [acme/argocd-deploy-non-prod](https://github.com/acme/argocd-deploy-non-prod) - [acme/argocd-deploy-prod](https://github.com/acme/argocd-deploy-prod) ## Prepare Authentication Argo CD can be integrated with GitHub using either GitHub Apps (recommended) or Personal Access Tokens (PATs). GitHub Apps provide more granular permissions, better security, and improved audit capabilities. Argo CD requires several different types of GitHub authentication for various components and workflows. While these could be combined, we follow the principle of least privilege by creating separate authentication credentials for each specific purpose. The following authentication methods are required: 1. #### Terraform `argocd-repo` Access First we will need to apply the Argo CD desired state repositories configuration with Terraform. By default, we use local access to apply the component. This requires an engineer to locally authenticate with GitHub and apply this component locally. Since this component is rarely updated, this can be a reasonable trade-off. 2. #### Argo CD Instance Next, we need a GitHub App for Terraform and the `eks/argocd` component. This app is used to register the webhook in GitHub for the Argo CD Application created with this given component. After creating the GitHub App, add the app's private key to AWS SSM Parameter Store in each account with Argo CD, typically the `plat-dev`, `plat-staging`, and `plat-prod` accounts, and add the App ID to the stack catalog. Detailed instructions linked below. 3. #### Argo CD Desired State Repository Access (2) We will need two more GitHub Apps for accessing the ArgoCD desired state repositories _from GitHub Actions_. GitHub Actions running for an application repositories will build and update application manifests in the Argo CD desired state repositories and therefore will need write access to that respective non-prod or prod repository. This GitHub App does not need to be added to the stack catalog or SSM since it will be used by GitHub Actions, not Terraform. 4. #### Argo CD GitHub Notification Access The last GitHub App is used by the Argo CD notifications system to update the GitHub commit status on deployments. This is stored in SSM and pulled by the `eks/argocd` component. That component will pass the ID and private key to the Argo CD instance in the given EKS cluster. That Argo CD instance uses that app _only when synchronous mode is enabled_. After creating the GitHub App, add the app's private key to AWS SSM Parameter Store in each account with Argo CD, typically the `plat-dev`, `plat-staging`, and `plat-prod` accounts, and add the App ID to the stack catalog. Detailed instructions linked below. Follow the instructions in [Argo CD Integrations: How to set up Authorization for Argo CD with GitHub Apps](/layers/software-delivery/eks-argocd/tutorials/github-apps) to create and configure all GitHub Apps for Argo CD. Once completed, you should have 4 GitHub Apps: - `Argo CD Instance` - `Argo CD Deploy Non-Prod` - `Argo CD Deploy Prod` - `Argo CD Notifications` ## Deploy the Argo CD Desired State Repositories Deploy the Argo CD configuration for the two Argo CD desired state GitHub repositories with the following workflow: Once this finishes, review the two repos in your GitHub Organization. These should both be fully configured at this point. - [acme/argocd-deploy-non-prod](https://github.com/acme/argocd-deploy-non-prod) - [acme/argocd-deploy-prod](https://github.com/acme/argocd-deploy-prod) ## Create AWS Identity Center Applications In order to authenticate with Argo CD, we recommend using an AWS IAM Identity Center SAML Application. These apps can use existing Identity Center groups that we've already setup as part of the [Identity layer](/layers/identity/). Please see [Argo CD Integrations: How to create an AWS Identity Center Application](/layers/software-delivery/eks-argocd/tutorials/identity-center-apps) and follow all steps. ## Deploy the Argo CD Instances to each EKS Cluster Once the GitHub repositories are in place and the SAML applications have been created and configuration uploaded to SSM, we're ready to deploy Argo CD to each cluster. Deploy `eks/argocd` to each cluster with the following workflow: ## Validation Once all deployment steps are completed, Argo CD should be accessible at the following URLs. Please note that you must be able to authenticate with AWS Identity Center to access any given app. - https://argocd.use1.dev.plat.acme-svc.com - https://argocd.use1.staging.plat.acme-svc.com - https://argocd.use1.prod.plat.acme-svc.com ## Next Steps Assuming login goes well, here's a checklist of GitHub repos needed to connect Argo CD: - [ ] `acme/infra-acme` repo (Should already exist!) - [ ] `acme/infra-acme/.github/environments` private workflows. This directory stores private environment configurations. Primarily, that is the [`cloudposse/github-action-yaml-config-query`](https://github.com/cloudposse/github-action-yaml-config-query) action used to get role, namespace, and cluster mapping for each environment. - [ ] (2) Argo CD deploy nonprod and prod (Should already be created by `argocd-repo` component in earlier step) - [ ] `argocd-deploy-non-prod` - [ ] `argocd-deploy-prod` - [ ] `acme/example-app` repo should be private repo generated from the [app-on-eks-with-argocd](https://github.com/cloudposse-examples/app-on-eks-with-argocd) template :::info Sensitive Log Output Note that all of these workflow runs run from within your private app repo, so any sensitive log output will not be public. ::: ### Environment Configuration Update the `cloudposse/github-action-interface-environment` action to point to your infrastructure repository. 1. Set `implementation_repository` to `acme/infra-acme` 2. Verify `implementation_path`, `implementation_file`, and `implementation_ref` match your local configuration. [Example app reference](https://github.com/cloudposse-examples/app-on-eks-with-argocd/blob/1abe260c7f43dde1c6610845e5a64a9d08eb8856/.github/workflows/workflow-cd-preview-argocd.yml#L167-L178) ### Verify GitHub OIDC Access Roles The IDP permissions in IAM will be sensitive to capitalization, and yet the docker image must -not- have uppercase letters! Make sure that your repo is allowed to assume roles for all relevant clusters and ECR repos: 1. Update the `github_actions_allowed_repos` variable in `ecr`, `eks/cluster`, or any other relevant components with GitHub OIDC access. 2. If your GitHub Organization has mixed capitalization cases, make sure these entries are case-sensitive ### GitHub Environment Secrets Add each of the following secrets to the `acme/example-app` repo: 1. `github-private-actions-pat`: `${{ secrets.PUBLIC_REPO_ACCESS_TOKEN }}` 2. `registry`: `${{ secrets.ECR_REGISTRY }}` 3. `secret-outputs-passphrase`: `${{ secrets.GHA_SECRET_OUTPUT_PASSPHRASE }}` 4. `ecr-region`: `${{ secrets.ECR_REGION }}` 5. `ecr-iam-role`: `${{ secrets.ECR_IAM_ROLE }}` ### Specify Ingress Group 1. Update the `deploy/releases/app.yaml` 2. Make sure the ingress is not set to `default`. It should likely be `alb-controller-ingress-group`. you can read more about this [from our docs on the alb controller component](/layers/eks/faq/#how-does-the-alb-controller-ingress-group-determine-the-name-of-the-alb) 3. Set the domain accordingly. Each environment will need the service domain + environment.stage.tenant (ie. `use2.staging.plat.acme-svc.com` ) 4. If your organization has mixed case, you'll need to edit the `organization` parameter to be lowercased in the GitHub workflows: `feature-branch.yml`, `main-branch.yaml`, and `release.yaml` ## FAQ ### GitHub Apps vs Personal Access Tokens We recommend using GitHub Apps for Argo CD integration with GitHub. GitHub Apps offer several advantages over Personal Access Tokens: 1. **Granular Permissions**: GitHub Apps can be granted access to specific repositories rather than requiring organization-wide access. 2. **Better Security**: GitHub Apps use JWT authentication and short-lived tokens, reducing the risk of token exposure. 3. **Improved Audit Capabilities**: Actions performed by GitHub Apps are clearly identified in audit logs. 4. **Rate Limiting**: GitHub Apps have their own rate limits, separate from user-based limits. 5. **Webhook Support**: GitHub Apps can receive webhooks for events in repositories they have access to. 6. **Multiple Installations**: The same GitHub App can be installed on different repositories with different permissions. For more information on setting up Argo CD with GitHub Apps, see [Argo CD Integrations: How to set up Authorization for Argo CD with GitHub Apps](/layers/software-delivery/eks-argocd/tutorials/github-apps). --- ## How to set up Authorization for Argo CD with GitHub Apps import Admonition from '@theme/Admonition' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Steps from '@site/src/components/Steps' import TaskList from '@site/src/components/TaskList' GitHub Apps is now the preferred method for Argo CD integration with GitHub. GitHub Apps provide more granular permissions, better security, and improved audit capabilities compared to Personal Access Tokens (PATs). This guide will walk you through setting up Argo CD with GitHub Apps. - GitHub Apps provide more granular permissions than PATs - GitHub Apps can be installed on specific repositories - GitHub Apps have built-in rate limiting and audit capabilities - GitHub Apps need the ability to bypass branch protection rules ## Required GitHub Apps Argo CD integration requires multiple GitHub Apps, each with specific permissions and repository scopes: ### 1. Argo CD Instance - **Actor**: Argo CD Deployment in Kubernetes - **Use cases and permissions**: - Allow Argo CD to read from Desired State Repositories: - `repository.contents`: Read-Only - `repository.metadata`: Read-Only - Webhooks for Desired State Repositories: - `repository.webhooks`: Read and Write - `repository.metadata`: Read-Only - **Repository scope**: - `acme/argocd-deploy-non-prod` - `acme/argocd-deploy-prod` ### 2. Argo CD Deploy Non-prod - **Actor**: GitHub App IAT supplied to GitHub Actions Workflows - **Use cases and permissions**: - Push commits to Desired State Repositories - `repository.contents`: Read and Write - `repository.metadata`: Read-Only - **Repository scope**: - `acme/argocd-deploy-non-prod` ### 3. Argo CD Deploy Prod - **Actor**: GitHub App IAT supplied to GitHub Actions Workflows - **Use cases and permissions**: - Push commits to Desired State Repositories - `repository.contents`: Read and Write - `repository.metadata`: Read-Only - **Repository scope**: - `acme/argocd-deploy-prod` ### 4. Argo CD Notifications - **Use cases and permissions**: - Commit status API (relay commit statuses from Desired State Repositories to app repositories via notification templates in `eks/argocd`): - `repository.commit_statuses`: Write - **Repository scope**: - All of the app repositories ## Deployment ### Create GitHub Apps First, you need to create the required GitHub Apps in your organization: 1. Go to your GitHub organization settings 2. Navigate to "GitHub Apps" and click "New GitHub App" 3. Create the following GitHub Apps with their respective permissions: #### Argo CD Instance - Name: `Argo CD Instance` - Homepage URL: Your organization's homepage - Webhook: Disabled - Permissions: - Allow Argo CD to read from Desired State Repositories: - `repository.contents`: Read-Only - `repository.metadata`: Read-Only - Webhooks for Desired State Repositories: - `repository.webhooks`: Read and Write - `repository.metadata`: Read-Only #### Argo CD Deploy Non-prod - Name: `Argo CD Deploy Non-prod` - Homepage URL: Your organization's homepage - Webhook: Disabled - Permissions: - Push commits to Desired State Repositories: - `repository.contents`: Read and Write - `repository.metadata`: Read-Only #### Argo CD Deploy Prod - Name: `Argo CD Deploy Prod` - Homepage URL: Your organization's homepage - Webhook: Disabled - Permissions: - Push commits to Desired State Repositories: - `repository.contents`: Read and Write - `repository.metadata`: Read-Only #### Argo CD Notifications - Name: `Argo CD Notifications` - Homepage URL: Your organization's homepage - Webhook: Disabled - Permissions: - Commit status API (relay commit statuses from Desired State Repositories to app repositories via notification templates in `eks/argocd`): - `repository.commit_statuses`: Write ### Generate and Store GitHub App Credentials After creating each GitHub App, you need to generate and store credentials: 1. For each GitHub App: 1. On the GitHub App page, scroll down to "Private keys" and click "Generate a private key" 1. Download the private key file 1. Store the App ID, Installation ID, and private key securely in 1Password 2. Upload these credentials to AWS SSM Parameter Store - Upload the `Argo CD Instance` private key to `/argocd/argo_cd_instance/app_private_key` in `core-auto`: ```bash # Replace acme with your namespace or assume the role separately. # Your default region should be the same as your primary region. assume-role acme-core-gbl-auto-admin chamber write argocd argo_cd_instance/app_private_key \ "$(cat /path/to/argocd-deploy-non-prod.private-key.pem)" ``` - Upload the `Argo CD Notifications` private key to `/argocd/argo_cd_notifications/app_private_key` to `core-auto`: ```bash assume-role acme-core-gbl-auto-admin chamber write argocd argo_cd_notifications/app_private_key \ "$(cat /path/to/argocd-notifications.private-key.pem)" ``` ### Install the GitHub Apps Install each GitHub App on its required repositories: 1. For `Argo CD Instance`: - Go to the GitHub App settings page - Click "Install App" in the sidebar - Select the repositories: - `acme/argocd-deploy-non-prod` - `acme/argocd-deploy-prod` - Complete the installation 2. For `Argo CD Deploy Non-Prod`: - Go to the GitHub App settings page - Click "Install App" in the sidebar - Select the repository: - `acme/argocd-deploy-non-prod` - Complete the installation 3. For `Argo CD Deploy Prod`: - Go to the GitHub App settings page - Click "Install App" in the sidebar - Select the repository: - `acme/argocd-deploy-prod` - Complete the installation 4. For `Argo CD Notifications`: - Go to the GitHub App settings page - Click "Install App" in the sidebar - Select all app repositories, such as `acme/example-app-on-eks` - Complete the installation ### Configure Branch Protection Rules If branch protection rules are enabled in your GitHub Organization, you'll need to configure exceptions for the ArgoCD GitHub Apps. This allows ArgoCD to update repositories while still maintaining security. The GitHub Apps must be able to bypass branch protection rules in order for ArgoCD's automated deployments to work correctly. Branch rulesets may be configured both or either at an organization and repository level. Check the enabled rulesets in both ArgoCD desired state repositories under "Code, planning, and automation" > "Branch rules". Add the ArgoCD GitHub Apps to the bypass list of any ruleset that prevents changes to the main branch. ## Configure Argo CD Desire State Repositories to Use GitHub Apps This step should be pre-configured for Reference Architecture users. Update your Argo CD desired state repository configuration to use the GitHub App: ```yaml components: terraform: argocd-repo: vars: # 1. Use local access to apply this component rather than a PAT # https://registry.terraform.io/providers/integrations/github/latest/docs#github-cli use_local_github_credentials: true # 2. If synchronous mode is enabled, set the notifications to send to "github" and not to the "webhook" github_notifications: - "notifications.argoproj.io/subscribe.on-deploy-started.github: \"\"" - "notifications.argoproj.io/subscribe.on-deploy-succeeded.github: \"\"" - "notifications.argoproj.io/subscribe.on-deploy-failed.github: \"\"" # 3. Optional, disable the SSH deploy keys to use a GitHub App # for the Argo CD instance to authenticate with the desired state repository deploy_keys_enabled: false ``` ### Configure Argo CD to Use GitHub Apps Reference Architecture users will need to update both GitHub App and Installation IDs. Update your Argo CD configuration to use the GitHub Apps by modifying the component configuration shown below. You can find the Installation ID by going to your GitHub Organization settings, selecting "GitHub Apps", clicking on your app, then selecting "Install App". The Installation ID will be in the URL, e.g. https://github.com/organizations/acme/settings/installations/44444444. ```yaml # stacks/catalog/eks/argocd/defaults.yaml components: terraform: eks/argocd: vars: # GitHub App (Argo CD Instance) # This GitHub App is used for the Argo CD instance to manage webhooks and read from the desired state repository. # ie https://github.com/acme/argocd-deploy-non-prod github_app_enabled: true github_app_id: "1234567" github_app_installation_id: "44444444" # The SSM parameter must exist in the account and region where Argo CD is deployed. ssm_github_app_private_key: "/argocd/argo_cd_instance/app_private_key" # Optional, disable the SSH deploy keys to use this GitHub App # for the Argo CD instance to authenticate with the desired state repository github_deploy_keys_enabled: false # GitHub App (Argo CD Notifications) # This GitHub App is used for the Argo CD instance to send commit status updates back to each ap repository. # This is only required if synchronous mode is enabled. # ie https://github.com/acme/example-app-on-eks github_notifications_app_enabled: true github_notifications_app_id: "8901235" github_notifications_app_installation_id: "55555555" # The SSM parameter must exist in the account and region where Argo CD is deployed. ssm_github_notifications_app_private_key: "/argocd/argo_cd_notifications/app_private_key" ``` ### Deploy the updated configuration Redeploy both the `argocd-repo` component for both nonprod and prod. Then redeploy all instances of `eks/argocd` ### Configure GitHub Actions Workflows Update your GitHub Actions workflows to use the appropriate GitHub App. Set the following GitHub environment variables for the application repositories: 1. Set `ARGO_CD_DEPLOY_NONPROD_APP_ID` in both `preview` and `dev` 2. Set `ARGO_CD_DEPLOY_PROD_APP_ID` in `staging` and `prod`. Then set the following secrets: 1. Add `ARGO_CD_DEPLOY_NONPROD_APP_PRIVATE_KEY` to `preview` and `dev`. 2. Add `ARGO_CD_DEPLOY_PROD_APP_PRIVATE_KEY` to `staging` and `prod`. \**Add QA environments if necessary* Please be sure to update your GitHub Workflows to support GitHub App authentication. If you are unsure, please reach out to Cloud Posse. ## References - [Setting up Argo CD](/layers/software-delivery/eks-argocd/setup/) - [GitHub Apps Documentation](https://docs.github.com/en/developers/apps) - [GitHub Apps Permissions](https://docs.github.com/en/developers/apps/building-github-apps/setting-permissions-for-github-apps) --- ## How to setup Synchronous Notifications for Argo CD with GitHub Commit Statuses import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' Synchronous notifications are used in Argo CD release engineering workflows to notify an application workflow of a successful deployment. The application repo deploys an updated app manifest to the Argo CD deployment repo. Then the Argo CD app in the EKS cluster pulls and deploys that updated application. Depending on the result of that deployment, Argo CD triggers a notifier. Our implementation of Argo CD breaks up notifications into "notifiers", "templates", and "triggers". You will see these keywords referenced frequently in the `eks/argocd` component. ## Create Notifiers and Webhook Tokens A notifier is the top level resource for any notification. If you wish to set up any notification, you must first create a notifier to react to a given event. Furthermore, this is where we set up authorization for webhooks. By default, we create 2 notifier webhooks: `app-repo-github-commit-status` and `argocd-repo-github-commit-status`, both of which use the `common_github-token` secret as an authorization token. That authorization token is programmatically pulled from AWS SSM using the path defined by `var.notifications_notifiers.ssm_path_prefix`, which is typically `/argocd/notifications/notifiers`. Using this prefix, the `/argocd/notifications/notifiers/common/github-token` parameter value is given to the `common_github-token` secret. You may add additional notifiers as follows. In this use case, `var.notifications_notifiers` is deep merged with the 2 default notifiers for `app-repo-github-commit-status` and `argocd-repo-github-commit-status`. This allows us to use different authorization tokens for this given webhook than the default `$common_github-token` ```yaml components: terraform: eks/argocd: vars: notifications_notifiers: webhook: foo-repo-github-commit: url: "https://api.github.com" headers: - name: "Authorization" value: "Bearer $webhook_foo-repo-github-commit_github-token" ``` Similarly, authorization token is programmatically pulled from AWS SSM using the path defined by `var.notifications_notifiers.ssm_path_prefix` _for any `webhook` notifier given_. Therefore, if you add an SSM parameter such as `/argocd/notifications/notifiers/foo-repo-github-commit/github-token`, the component will create the `webhook_foo-repo-github-commit_github-token` secret. ## Define Notification Templates A template defines the event structure for a notification. This is the message and webhook. Again, by default we set up `app-repo-github-commit-status` and `argocd-repo-github-commit-status` templates. ```yaml templates: template.app-deploy-failed: | "alertmanager": null "message": "Application {{ .app.metadata.name }} failed deploying new version." "webhook": "app-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"error\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" "argocd-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"error\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.spec.source.repoURL}}/statuses/{{.app.status.operationState.operation.sync.revision}}" template.app-deploy-started: | "alertmanager": null "message": "Application {{ .app.metadata.name }} is now running new version of deployments manifests." "webhook": "app-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"pending\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" "argocd-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"pending\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.spec.source.repoURL}}/statuses/{{.app.status.operationState.operation.sync.revision}}" template.app-deploy-succeeded: | "alertmanager": null "message": "Application {{ .app.metadata.name }} is now running new version of deployments manifests." "webhook": "app-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"success\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" "argocd-repo-github-commit-status": "body": "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"success\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" "method": "POST" "path": "/repos/{{call .repo.FullNameByRepoURL .app.spec.source.repoURL}}/statuses/{{.app.status.operationState.operation.sync.revision}}" ``` In order to add additional templates, use `var.notifications_templates`. This value is again deep merged with `app-repo-github-commit-status` and `argocd-repo-github-commit-status`. ```yaml components: terraform: eks/argocd: vars: notifications_templates: app-deploy-succeeded: message: "Application {{ .app.metadata.name }} is now running new version of deployments" webhook: foo-repo-github-commit: body: "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"success\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" method: "POST" path: "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" app-deploy-started: message: "Application {{ .app.metadata.name }} is now running new version of deployments" webhook: foo-repo-github-commit: body: "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"pending\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" method: "POST" path: "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" app-deploy-failed: message: "Application {{ .app.metadata.name }} failed deploying new version." webhook: foo-repo-github-commit: body: "{\"context\":\"continuous-delivery/{{.app.metadata.name}}\",\"description\":\"ArgoCD\",\"state\":\"error\",\"target_url\":\"{{.context.argocdUrl}}/applications/{{.app.metadata.name}}\"}" method: "POST" path: "/repos/{{call .repo.FullNameByRepoURL .app.metadata.annotations.app_repository}}/statuses/{{.app.metadata.annotations.app_commit}}" ``` ## Configure Triggers Finally, a trigger determines when these notifications are sent. By default we set up `app-repo-github-commit-status` and `argocd-repo-github-commit-status` triggers. ```yaml triggers: trigger.on-deploy-failed: | - "oncePer": "app.status.sync.revision" "send": - "app-deploy-failed" "when": "app.status.operationState.phase in ['Error', 'Failed' ] or ( app.status.operationState.phase == 'Succeeded' and app.status.health.status == 'Degraded' )" trigger.on-deploy-started: | - "oncePer": "app.status.sync.revision" "send": - "app-deploy-started" "when": "app.status.operationState.phase in ['Running'] or ( app.status.operationState.phase == 'Succeeded' and app.status.health.status == 'Progressing' )" trigger.on-deploy-succeeded: | - "oncePer": "app.status.sync.revision" "send": - "app-deploy-succeeded" "when": "app.status.operationState.phase == 'Succeeded' and app.status.health.status == 'Healthy'" ``` These triggers may trigger _multiple templates_. For example `trigger.on-deploy-succeeded` triggers both `template.app-deploy-succeeded.webhook.app-repo-github-commit-status` and `template.app-deploy-succeeded.webhook.argocd-repo-github-commit-status`. ## References - [Setting up ArgoCD](/layers/software-delivery/eks-argocd/setup/) - [Argo CD Notifications (official)](https://argocd-notifications.readthedocs.io/en/stable/) - [GitHub Commit statuses API](https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#create-a-commit-status) --- ## How to create an AWS Identity Center Application for ArgoCD import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' In order to authenticate with ArgoCD, we recommend using an AWS IAM Identity Center SAML Application. These apps can use existing Identity Center groups that we've already setup as part of the [Identity layer](/layers/identity). ## Create AWS Identity Center Applications 1. For each `dev`, `staging`, and `prod` in the `plat` tenant, create an [IAM Identity Center Application](https://docs.aws.amazon.com/singlesignon/latest/userguide/samlapps.html). 2. Use the 'callback' url of `eks/argocd` for both the ACS URL and the SAML Audience fields. For example, `https://argocd.use1.dev.plat.acme-svc.com/api/dex/callback`. This should be your _service domain_. 3. Next, update the custom SAML application attributes: | Name | Value | Type | | :-------- | :---------------- | :------------ | | `Subject` | `${user:subject}` | `persistent` | | `email` | `${user:email}` | `unspecified` | | `groups` | `${user:groups}` | `unspecified` | 4. Now assign AWS Identity Center groups to the SAML app. If you ever recreate the groups, you'll need to go back to the SAML application and remove/re-add the group. 5. Record the IDs of each group you assigned. If you've recently updated the groups, you'll likely need to redo this step as group IDs change on any significant updates. 6. Update the config for `eks/argocd` to use the given AWS Identity Center groups groups: ```console components: terraform: eks/argocd: vars: # Note: the IDs for AWS Identity Center groups will change if you alter/replace them: argocd_rbac_groups: - group: deadbeef-dead-beef-dead-beefdeadbeef role: admin - group: aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee role: reader ``` 7. Finally, for each stage run `atmos terraform deploy sso-saml-provider -s plat-use1-{ stage }` :::info Tip If you get any errors using AWS SSO, make sure the `Subject` attribute is set to `persistent` and connect to the cluster with `set-cluster plat-{ region }-{ stage } admin && kubens argocd` and then delete the dex pod to reset it. ::: # References - [Setting up ArgoCD](/layers/software-delivery/eks-argocd/setup/) --- ## How to set up Authorization for ArgoCD with GitHub PATs import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps' import Step from '@site/src/components/Step' import StepNumber from '@site/src/components/StepNumber' import Admonition from '@theme/Admonition' import Note from '@site/src/components/Note' import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; The deployment process for ArgoCD includes setting up access tokens for a number of responsibilities. We will need to create the desire state repositories with necessary access, create Webhooks for these repos, grant the app in the EKS cluster permission to send notifications, and grant access for GitHub workflows. :::tip Fine-grained Personal Access Tokens (PAT) Fine-grained PATs are preferred to classic PATs. All PATs except the Notifications GitHub PAT will be fine-grained PATs. See [Managing your personal access tokens](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens). ::: ## Establish a Bot User We deploy a number of GitHub Personal Access Tokens (PATs) as part of the EKS with ArgoCD application. By default each PAT is given the least-access required for the given job. Each one of these PATs will be associated with a given user. We recommend creating or using an existing "bot" user. For example, at Cloud Posse we have the "cloudpossebot" GitHub user. This user has its own email address and GitHub account, is accessible from our internal 1Password vault for all privileged users, and has all access keys and tokens stored with it in 1Password. This bot user will need permission to manage a few repositories in your Organization. If you wish to simplify deployment, you can grant this user permission to create repositories. See [Can we use the Bot user to create the ArgoCD repos](#can-we-use-the-bot-user-to-create-the-argocd-repos). Use this bot user for all access tokens in the remainder of this guide. ## Create ArgoCD GitHub Repositories Create the two required ArgoCD GitHub repos: - [acme/argocd-deploy-non-prod](https://github.com/acme/argocd-deploy-non-prod) - [acme/argocd-deploy-prod](https://github.com/acme/argocd-deploy-prod) Then grant the bot user `Admin` access to these two repositories. ## Create the first GitHub PAT In order for Terraform to manage these two GitHub repositories for ArgoCD, we must deploy our first GitHub PAT ([follow this manual to create a PAT](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token)). All tokens created in this setup guide use a Cloud Posse naming convention. If a different pattern makes more sense for your organization, feel free to change it up! See [FAQ: What is the Cloud Posse naming convention for PATs?](#what-is-the-cloud-posse-naming-convention-for-pats). This token needs permission to manage the ArgoCD deployment repositories. 1. Name this PAT `argocd/terraform/argocd-repo`. 2. For repository access, select "Only select repositories" and choose both `acme/argocd-deploy-non-prod` and `acme/argocd-deploy-prod`. 3. Grant this PAT the following permissions ```diff # Repository permissions + Administration: Read and write + Contents: Read and write + Metadata: Read-only # Organization permissions + Members: Read-only ``` Use the following workflow to upload this PAT to AWS SSM: Or manually save this PAT to AWS SSM at `argocd/github/api_key` in the `core-auto` account. ## Deploy ArgoCD GitHub Repositories Configuration Deploy the ArgoCD configuration for the two GitHub repos with the following workflow: Once this finishes, review the two repos in your GitHub Organization. These should both be fully configured at this point. - [acme/argocd-deploy-non-prod](https://github.com/acme/argocd-deploy-non-prod) - [acme/argocd-deploy-prod](https://github.com/acme/argocd-deploy-prod) Now that the ArgoCD deployment repos are configured, we need to create GitHub PATs for ArgoCD. ## Create the Webhook GitHub PATs The next two PATs created will be used by Terraform with the `eks/argocd` component; one for `argocd-deploy-non-prod` and one for `argocd-deploy-prod`. Each of these PATs is used to register the webhook in GitHub for the ArgoCD Application created with this given component. Terraform will pull that PAT from SSM typically using the `argocd/github` path in `plat-dev`, `plat-staging`, and `plat-prod` accounts. You may notice these PATs use the same SSM path as PAT #1 yet is deployed to a different account. We intentionally separate these PATs in order to adhere to least-privilege principle. This way, each account can pull a PAT from the same SSM path in the same account with the minimal set of permission that this account requires. These PATs can be combined into a single PAT if preferred. Create two PATs with the following allowed permissions. First nonprod: 1. Name this PAT `argocd/terraform-webhooks/nonprod` 2. Limit this PAT to `acme/argocd-deploy-non-prod` 3. Grant the following permission: ```diff Repository: + Webhooks: Read and write + Metadata: Read-only ``` 4. Use the following workflow to upload this PAT to AWS SSM: Or manually save this PAT to AWS SSM at `argocd/github/api_key` in the `plat-dev` and `plat-staging` accounts. Now repeat the same process for production: 1. Name this PAT `argocd/terraform-webhooks/prod` 2. Limit this PAT to `acme/argocd-deploy-prod` 3. Grant the following permission (again): ```diff Repository: + Webhooks: Read and write + Metadata: Read-only ``` 4. Use the following workflow to upload this PAT to AWS SSM: Or manually save this PAT to AWS SSM at `argocd/github/api_key` in the `plat-prod` account. ## Create the Notifications GitHub PAT The next PAT is used by the ArgoCD notifications system to set the GitHub to commit status on successful deployments. This PAT is stored in SSM and pulled by the `eks/argocd` component. That component will pass the token to the ArgoCD application in the given EKS cluster. That ArgoCD Application uses that PAT only when synchronous mode is enabled. As of January 2023, GitHub does not support fine-grained PATs with the [commit statuses API](https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#create-a-commit-status). Therefore, we must create a _classic_ PAT for the bot user. 1. Name this _classic_ PAT `ARGOCD_APP_NOTIFICATIONS` 2. Grant the following permission: ```diff + repo:status ``` 3. Then check that the bot user has access to the _application_ repo. For example for Cloud Posse, that is [cloudposse-examples app-on-eks-with-argo](https://github.com/cloudposse-examples/app-on-eks-with-argocd). Use the following workflow to upload this PAT to AWS SSM: Or manually save this PAT to AWS SSM at `argocd/notifications/notifiers/common/github-token` in the `plat-dev`, `plat-staging`, and `plat-prod` accounts. ## Create the Workflows GitHub PATs The final two PATs are used in the release engineering workflows; again one for nonprod and one for prod. Each PAT will need access to two repos. First, it needs read access to the private environment configuration. By default, this is the `infra-repo` repository. Second, it needs write access to the given ArgoCD deploy repository in order to update the deployment configuration for new applications. For the nonprod PAT: 1. Name this PAT `argocd/github/nonprod` 2. Limit this PAT to `acme/argocd-deploy-non-prod` and `acme/infra-repo` 3. Grant this PAT the following permissions: ```diff Repository + Contents: Read and write + Metadata: Read-only ``` This PAT _does not_ need to be uploaded to AWS SSM, and instead store this PAT for reference in 1Password. We will upload this PAT as a GitHub secret for the release workflows, typically with the `ARGOCD_GITHUB_NONPROD` secret. Now for the prod PAT: 1. Name this PAT `argocd/github/prod` 2. Limit this PAT to `acme/argocd-deploy-prod` and `acme/infra-repo` 3. Grant this PAT the following permissions: ```diff Repository + Contents: Read and write + Metadata: Read-only ``` Again store this PAT for reference in 1Password and upload this PAT as a GitHub secret for the release workflows, typically with the `ARGOCD_GITHUB_PROD` secret. ## FAQ ### What is the Cloud Posse naming convention for PATs You can name your PATs however you prefer, but for the sake of consistency, we recommend establishing a naming convention for PATs. At Cloud Posse we prefer to use the following pattern: `//` However for *classic* PATs, secret names can only contain alphanumeric characters ([a-z], [A-Z], [0-9]) or underscores (\_). Spaces are not allowed. Must start with a letter ([a-z], [A-Z]) or underscores (\_). So for *classic* PATs use all Caps and underscores. For example: ```console # 1. Terraform access for argocd-repo. Requires access to apply both prod and nonprod argocd/terraform/argocd-repo # needs read on org members, write admin and code on both argocd repos # 2. Terraform access for eks/argocd webhooks argocd/terraform-webhooks/nonprod # needs permission to write repository hooks on argocd-deploy-nonprod argocd/terraform-webhooks/prod # needs permission to write repository hooks on argocd-deploy-prod # 3. ArgoCD access for app in cluster ARGOCD_APP_NOTIFICATIONS # needs permission to write commit statuses on any application repo # 4. GitHub Workflow access argocd/github/nonprod # needs write access to argocd-deploy-nonprod and read for infra argocd/github/prod # needs write access to argocd-deploy-prod and read for infra ``` ### Can we use the Bot user to create the ArgoCD repos? By default, we do not require that the bot user creates the ArgoCD deployment repositories. However, the component does support enabling that option. If you wish to allow the bot user to both create and manage the ArgoCD deployment repos, grant the bot user permission in your Organization, and then set `var.create_repo` to `true` in `stacks/catalog/argocd-repo/defaults.yaml`. ### Resource not accessible by personal access token ```console { "message": "Resource not accessible by personal access token", "documentation_url": "https://docs.github.com/rest/commits/statuses#create-a-commit-status" } ``` You may see this message if you attempt to use a fine-grained PAT to set a GitHub commit status. As of January 2023, GitHub does not support fine-grained PATs with the [commit statuses API](https://docs.github.com/en/rest/commits/statuses?apiVersion=2022-11-28#create-a-commit-status). Therefore, we must create a _classic_ PAT for the bot user. ### Forbids access via a personal access token (classic). ```console { "message": "`acme` forbids access via a personal access token (classic). Please use a GitHub App, OAuth App, or a personal access token with fine-grained permissions.", "documentation_url": "https://docs.github.com/rest/commits/statuses#create-a-commit-status" } ``` You must enable classic PATs for the GitHub Organization. Under the GitHub Organization settings, go to `Personal access tokens` > `Settings` > `Personal access token (classic)` and select `Allow access via personal access tokens (classic)`. ### Why not use a GitHub App? At the time this component was developed, GitHub Apps were not fully supported. However, we plan to update our recommendation to a GitHub App soon! Please check with Cloud Posse on the latest status. ## References - [Setting up ArgoCD](/layers/software-delivery/eks-argocd/setup/) --- ## Tutorials(10) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; Here are some additional tutorials that will help you along in your usage of Argo CD. --- ## Implementing CI/CD import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from 'react-player'; import Note from '@site/src/components/Note'; This document assumes the reader understands the importance of CI/CD and DevOps as a concept. For more, see [What is DevOps](https://aws.amazon.com/devops/what-is-devops/).
AI generated voice
## The Problem At this point, we have defined a complete Infrastructure as Code (IaC) environment, yet we still need some way to deliver our application to users. Release Engineering is that process. Primarily, this includes Continuous Integration (CI) and Continuous Delivery (CD). CI/CD acts as the glue between our IaC and the delivery of the software to customers and is how an app consumes its platform. Historically, developers would define a CI/CD process for a given application, typically with Jenkins, and then duplicate that process for each application. Quickly we would have many different pipelines for our library of apps, each with an individual purpose specific to that given app. As pipelines grew in numbers and complexity, entire teams would be hired solely to manage these systems, and CI/CD became the scapegoat system that everyone loves to hate. For years CI/CD meant the same thing. Now CI is separate from CD. CI is process of building an artifacts, and CD is the process of taking that artifact to delivery. There are many methods of CI and of CD. For example, Spacelift is CD for Terraform; ArgoCD is CD for Kubernetes. Similarly there are many tools for CI. Jenkins, GitHub Actions, CircleCI, and countless others all offer solutions. Ultimately, any solution must be codified and create a standard pattern of software delivery. It needs to support many languages and many frameworks. All Git workflows need to be supported for these different languages and frameworks in a consistent way, such that we do not create snowflakes. Yet one size will not fit all, so we need the ability to break glass without throwing out the whole solution. Modern day Release Engineering is complex. Pipelines grow exponentially and often cannot be tested. Companies rely on CI/CD to ship software, yet have no way to test CI/CD _itself_. ## Our Solution ### Concept Release engineering process is critical for successful software development. On the one hand, it is responsible for the continuous resilient delivery of the software at a consistent quality. On the other hand, the process can be treated as a part of the [organization's value stream](https://www.thoughtworks.com/radar/techniques/path-to-production-mapping), can highlight specific organizational [structure](https://en.wikipedia.org/wiki/Conway%27s_law), and demonstrate the maturity of the engineering culture. Release engineering pipelines are required to measure [the organization's performance](https://dora.dev/). At Cloud Posse, we consider CI/CD pipelines as [software](https://www.thoughtworks.com/radar/techniques/pipelines-as-code), and an automated part of the release engineering process. Developing CI/CD pipelines we use practices and design principles well-established in software engineering, including but not limited to * [Separation of concern](https://en.wikipedia.org/wiki/Separation_of_concerns) * [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) * [Tests automation](https://en.wikipedia.org/wiki/Test_automation) * [Versioning](https://en.wikipedia.org/wiki/Software_versioning) * [Design by contract](https://en.wikipedia.org/wiki/Design_by_contract) * [Convention over configuration](https://en.wikipedia.org/wiki/Convention_over_configuration) and [code](https://en.wikipedia.org/wiki/Convention_over_Code) While there is value in flexibility by creating custom pipelines in each repo, we value the convention across pipelines to improve maintainability and consistency across organizations. This interface should be standardarized regardless of what is being delivered. In order to minimize boilerplate in pipelines, we create shared workflows with Github Actions to define how to handle specific aspects of the Release Engineering process. These workflows use a combination of Reusuable Workflows, Composite Actions, and regular Actions (both public or private). These layers have multiple levels of abstraction and allow us to define configuration per environment and create exceptions at any given point. Much of the common boilerplate patterns can be combined into reusable steps broken down into each level of abstraction. Composite Actions consolidate common steps into a single, modular action that can be documented, parameterized, and tested. Reusable Workflows combine these tested Composite Actions and regular Actions to into common processes. Composite Actions can be anywhere, public or private, but Reusable Workflows must be specific to an Organization. Moreover, a Reusable Workflow can have multiple jobs that can run together. Yet Composite Actions cannot have multiple jobs but can have multiple steps. Reusable Workflows can call other Reusable Workflows and Composite Actions can call other Composite Actions. Both Reusable Workflows and Composite Actions do not have a trigger. Both are functions that take inputs and produce outputs and therefore can and and should be documented and tested. Cloud Posse defines common patterns across customers and offers several solutions. We have many workflows for many purposes. For example you could have a shared CI workflow to provide linting, testing, and validation. Or you could have several CD workflows: CD to deploy an app to EKS with ArgoCD, CD to deploy code to a Lambda function, or a CD to deploy a Docker image to ECR. All these workflows are stored in YAML files and follow a common convention. Finally, they are organized consistently so that we are able to introduce additional interfaces down the road. ## Workflows ```mermaid --- title: Deployment Lifecycle --- stateDiagram-v2 direction LR [*] --> pr : Create Pull Request pr --> main : merge main --> release : Create Release state "Pull Request" as pr { [*] --> label label --> preview preview --> [*] state "Add deploy Label" as label state "Deploy to Preview" as preview } state "Main Branch" as main { [*] --> dev dev --> [*] state "Deploy to Dev" as dev } state "Release" as release { [*] --> staging staging --> confirm confirm --> prod prod --> [*] state "Deploy to Staging" as staging state "Confirm" as confirm state "Deploy to Prod" as prod } ``` Create a Pull Request with changes to the application. Add the "deploy" label to the PR, which will trigger a deployment to the Preview environment. Validate your changes and approve the PR. When the PR is merged into main, a deployment to Dev will be triggered next. When ready to cut a release, create a Release with GitHub. This will trigger another workflow to first deploy to Staging and then will wait for manual confirmation. Once manually approved, the workflow will continue and deploy to Production. ### Release Engineering Flavors Refer to our stack specific implementations for more details: - [**Dockerized App on EKS with ArgoCD**](/layers/software-delivery/eks-argocd/) - [**Dockerized App on ECS with Ecspresso**](/layers/software-delivery/ecs-ecspresso/) - [**Lambda App**](/layers/software-delivery/lambda) ## FAQ ### I cannot assume the AWS roles from GitHub Workflows The following error commonly occurs when setting up GitHub OIDC roles and permission: ``` Error: Could not assume role with OIDC: Not authorized to perform sts:AssumeRoleWithWebIdentity ``` To resolve this error, make sure your workflow has appropriate permission to assume GitHub OIDC roles. ```yaml permissions: id-token: write # This is required for requesting the JWT contents: read # This is required for actions/checkout ``` ### How does GitHub OIDC work with AWS? Please see [How to use GitHub OIDC with AWS](/layers/github-actions/github-oidc-with-aws) --- ## Lambda with GitHub Workflows import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import CollapsibleText from '@site/src/components/CollapsibleText'; Deploy Lambda functions using GitHub Workflows with a code-driven approach. The build process updates S3 with assets and SSM with the new version, requiring a Terraform run for promotion. GitHub Workflows manage the entire lifecycle, from building and packaging Lambda functions to deploying them with reusable workflows. ### Overview ```mermaid --- title: Lambdas with GitHub Workflows --- flowchart LR subgraph core-auto publish["publish"] promote["promote"] end subgraph core-artifacts artifacts_bucket["@Bucket lambda artifacts"] end subgraph "SSM parameters" subgraph "plat-dev" dev_lambda_tag["@SystemsManager /lambda/hello/tag"] end subgraph "plat-staging" staging_lambda_tag["@SystemsManager /lambda/hello/tag"] end subgraph "plat-prod" prod_lambda_tag["@SystemsManager /lambda/hello/tag"] end end pr["@PullRequest PR #1234 "] --> publish --> artifacts_bucket publish --> dev_lambda_tag push["@GitBranch push → main"] --> publish --> artifacts_bucket publish --> staging_lambda_tag release["@GitHubRelease release"] --> promote promote --> artifacts_bucket promote --> prod_lambda_tag artifacts_bucket --> staging_lambda_tag ``` ### Build and Deployment Application repository updates S3 with build assets, then updates SSM with the new version. Each SSM update is basically a promotion, and requires a Terraform run to realize the change. ```yaml title=".github/workflows/reusable-publish-lambda-zip.yaml" name: Publish Lambda Function on: workflow_call: inputs: function-name: required: true type: string source-folder: required: true type: string artifacts-bucket-and-prefix: required: true type: string aws-region: required: true type: string secrets: cicd-role-arn: required: true permissions: id-token: write contents: read jobs: publish: runs-on: self-hosted steps: - name: Configure AWS credentials uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: ${{ inputs.cicd-role-arn }} aws-region: ${{ inputs.aws-region }} - name: Checkout uses: actions/checkout@v4 - name: Package Lambda run: | cd ${{ inputs.source-folder }} && zip ${{ github.sha }}.zip * - name: Push Lambda run: | aws s3 cp ${{ inputs.source-folder }}/${{ github.sha }}.zip s3://${{ inputs.artifacts-bucket-and-prefix }}/${{ inputs.function-name }}/ --sse - name: Write tag to SSM run: | aws ssm put-parameter --name /lambda/${{ inputs.function-name}}/tag --type String --value ${{ github.sha }} --overwrite ``` ```yaml title=".github/workflows/reusable-promote-lambda-zip.yaml" name: Publish Lambda Function on: workflow_call: inputs: function-name: required: true type: string artifacts-bucket-and-prefix: required: true type: string aws-region: required: true type: string secrets: cicd-role-arn: required: true staging-role-arn: required: true prod-role-arn: required: true permissions: id-token: write contents: read jobs: publish: runs-on: self-hosted steps: - name: Configure AWS credentials for 'cicd' role uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: ${{ inputs.cicd-role-arn }} aws-region: ${{ inputs.aws-region }} - name: Configure AWS credentials for source stage uses: aws-actions/configure-aws-credentials@v4 with: aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }} aws-session-token: ${{ env.AWS_SESSION_TOKEN }} role-duration-seconds: 3000 role-skip-session-tagging: true role-to-assume: ${{ inputs.staging-role-arn }} aws-region: ${{ inputs.aws-region }} - name: Checkout uses: actions/checkout@v4 - name: Get tag from SSM id: get-tag-from-ssm run: | TAG=`aws ssm get-parameter --name /lambda/${{ inputs.function-name }}/tag | jq -r .Parameter.Value` echo "tag=$TAG" >> $GITHUB_OUTPUT - name: Copy Lambda to local run: | aws s3 cp s3://${{ inputs.artifacts-bucket-and-prefix }}/${{ inputs.function-name }}/${{ steps.get-tag-from-ssm.outputs.tag }}.zip . - name: Configure AWS credentials for 'cicd' role uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: ${{ inputs.cicd-role-arn }} aws-region: ${{ inputs.aws-region }} - name: Configure AWS credentials for destination stage uses: aws-actions/configure-aws-credentials@v4 with: aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }} aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }} aws-session-token: ${{ env.AWS_SESSION_TOKEN }} role-duration-seconds: 3000 role-skip-session-tagging: true role-to-assume: ${{ inputs.prod-role-arn }} aws-region: ${{ inputs.aws-region }} - name: Copy Lambda to destination bucket run: | aws s3 cp ${{ steps.get-tag-from-ssm.outputs.tag }}.zip \ s3://${{ inputs.artifacts-bucket-and-prefix }}/${{ inputs.function-name }}/ --sse - name: Write tag to SSM run: | aws ssm put-parameter --name /lambda/${{ inputs.function-name}}/tag --type String --value ${{ steps.get-tag-from-ssm.outputs.tag }} --overwrite ``` ```yaml title=".github/workflows/reusable-promote-lambda-zip.yaml" name: Deploy Lambda via Spacelift on: workflow_call: inputs: function-name: required: true type: string stack: required: true type: string secrets: spacelift-api-key-id: required: true spacelift-api-key-secret: required: true jobs: deploy: runs-on: self-hosted container: 123456789012.dkr.ecr.us-east-2.amazonaws.com/acme/infra:latest steps: - name: Trigger Spacelift Stack Execution env: SPACELIFT_API_ENDPOINT: https://acme.app.spacelift.io SPACELIFT_API_KEY_ID: ${{ secrets.spacelift-api-key-id }} SPACELIFT_API_KEY_SECRET: ${{ secrets.spacelift-api-key-secret }} run: | spacectl stack deploy --id ${{ inputs.stack }}-lambda-${{ inputs.function-name}} --tail ``` ### Implementation - [`lambda`](/components/library/aws/lambda/): This component is responsible for creating the Lambda function. After promotion, the Lambda function is updated with the new version. ## References - [Lambda Setup](/layers/software-delivery/lambda) - [Foundation Release Engineering](/layers/software-delivery/lambda/) - [Decide on Pipeline Strategy](/layers/software-delivery/design-decisions/decide-on-pipeline-strategy) --- ## Software Delivery import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; import Step from '@site/src/components/Step'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import ReactPlayer from 'react-player'; Software delivery is the process of moving your applications from development to production. This involves building, testing, deploying, and promoting them through environments like dev, staging, and production, with approval gates at each stage. Whether you're using EKS, ECS, or Lambdas, the solutions may vary slightly, but we maintain a consistent, reusable pattern across all applications. ## Deploy all Backing Services & Databases Ensure all the backing services that your applications depend on are deployed and running. This includes databases, caches, and message queues, etc. Get Started ## Implement CI/CD Choose a path for delivery of your services with GitHub Actions. The reference architecture supports deployment to AWS EKS, Amazon ECS, and Lambda functions. We use the `ecspresso` deployment tool for Amazon ECS to manage ECS services using a code-driven approach, alongside reusable GitHub Action workflows. This setup allows tasks to be defined with Terraform within the infrastructure repository, and task definitions to reside alongside the application code. Get Started Argo CD is an open-source declarative, GitOps continuous delivery tool for Kubernetes applications. It enables developers to manage and deploy applications on Kubernetes clusters using Git repositories as the source of truth for configuration and definitions. Our Argo CD implementation follows the GitOps methodology and integrates with GitHub Actions, ensuring that the entire application configuration, including manifests, parameters, and even application state, is stored in a Git repository. Get Started Deploy Lambda functions using GitHub Workflows with a code-driven approach. The build process updates S3 with assets and SSM with the new version, requiring a Terraform run for promotion. GitHub Workflows manage the entire lifecycle, from building and packaging Lambda functions to deploying them, with reusable workflows. Get Started
AI generated voice
Once you're done deploying your apps, it's time to start monitoring everything. We'll show you how to do that next. Our Examples ### Reusable Workflows We've consolidated all the workflows into the example applications, including the GitHub reusable workflows. We've done this to make it easier for Developers to understand how the example leverages all the workflows. In practice, we recommend moving the reusable workflows into a centralized repository, where they can be shared by other application repositories. For example, we would recommend moving all the `ecspresso-*` and all `workflow-*` workflow files to a centralized repository (e.g. a repository named `github-action-workflows`, alternatively the `infrastructure` repository directly). The best solution will depend on your GitHub Organization structure and team size. Pick what works for you and your team. When your workflows are consolidated, you will need only 3 inside an application repository: 1. `feature-branch.yaml` 2. `main-branch.yaml` 3. `release.yaml` 4. (optional) `hotfix-branch.yaml` 5. (optional) `hotfix-enabled.yaml` 6. (optional) `hotfix-release.yaml` The remaining workflows are the reusable/shared implementation. This approach makes it easier to define a standardized CI/CD interface for all of your services. ```console .github ├── configs/ │ ├── draft-release.yml │ └── environment.yaml └── workflows/ ├── ecspresso-feature-branch.yml ├── ecspresso-hotfix-branch.yml ├── ecspresso-hotfix-mixin.yml ├── ecspresso-hotfix-release.yml ├── ecspresso-main-branch.yml ├── ecspresso-release.yml ├── feature-branch.yml ├── main-branch.yaml ├── release.yaml ├── workflow-cd-ecspresso.yml ├── workflow-cd-preview-ecspresso.yml ├── workflow-ci-dockerized-app-build.yml ├── workflow-ci-dockerized-app-promote.yml ├── workflow-controller-draft-release.yml ├── workflow-controller-hotfix-reintegrate.yml ├── workflow-controller-hotfix-release-branch.yml └── workflow-controller-hotfix-release.yml ``` After moving to a centralized workflow repository, you should have a setup like the following: ```console Application Repository ├── .github │ ├── configs/ │ │ └── draft-release.yml │ └── workflows/ │ ├── feature-branch.yml │ ├── main-branch.yaml │ └── release.yaml └── ... github-action-workflows ├── .github/ │ └── workflows │ ├── ecspresso-feature-branch.yml │ ├── ecspresso-hotfix-branch.yml │ ├── ecspresso-hotfix-mixin.yml │ ├── ecspresso-hotfix-release.yml │ ├── ecspresso-main-branch.yml │ ├── ecspresso-release.yml │ ├── workflow-cd-ecspresso.yml │ ├── workflow-cd-preview-ecspresso.yml │ ├── workflow-ci-dockerized-app-build.yml │ ├── workflow-ci-dockerized-app-promote.yml │ ├── workflow-controller-draft-release.yml │ ├── workflow-controller-hotfix-reintegrate.yml │ ├── workflow-controller-hotfix-release-branch.yml │ └── workflow-controller-hotfix-release.yml └── ... ``` --- ## How to Create a Migration Checklist import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem ## Solution :::warning This didn't export cleanly and needs to be reworked. ::: ### Pre-cutover Tasks - [ ] Update EKS Task with ElasticSearch URL - [ ] Ensure ALB ingress can support 2 active vanity domains - [ ] Create ACM component - [ ] Audit Security Groups - [ ] Implement Database Seeding & Migration Strategy - [ ] Create `release` pipeline - [ ] Redeploy bastion on public and private subnets - [ ] SSH access via bastion host - [ ] Restore most recent snapshot from production to staging - [ ] Attach additional scratch space to EKS tasks for web app - [ ] Bastion Host user-data/ACL updates - [ ] Investigate 502s - [ ] Tune EKS web app and tasks - [ ] Implement Lambda Log Parser for Cloudwatch Logs - [ ] Update Spacelift to Trigger on Changes to TF Var files - [ ] Rename SSM Parameters for AWS_* to LEGACY_AWS_* - [ ] Add feature flag to disable scheduled tasks for Preview - [ ] Provision Read Replica For Production Database (used by Redshift) - [ ] Implement Scheduled Tasks - [ ] Implement Pipeline to Create Preview Environments - [ ] Decide on Cut Over Plan - [ ] Deploy sidekiq workers (high priority) - [ ] Containers should log to stdout - [ ] Decide on Pipeline Strategy - [ ] Implement `registry/terraform/eks-web-app` module - [ ] Deploy acme to Production ([app.acme.com](http://app.acme.com)) - [ ] Integrate EKS Web App with Cloudwatch Logs - [ ] Implement Vanity DNS with EKS Tasks - [ ] Deploy [http://acme.com](http://acme.com) to Staging ([app.acme.com](http://app.acme.com)) - [ ] Deploy [http://acme.com](http://acme.com) as EKS Task with Spacelift - [ ] Create `build` pipeline - [ ] Reduce Scope of IAM Grants for GitHub Runners - [ ] Create `deploy` pipeline - [ ] ETL Postgres Databases to Bastion Instance - [ ] Import Staging Database to All RDS Clusters for Testing - [ ] Update Spacelift Config to Assume Role before Apply - [ ] Implement Preview Environment Destroy Pipeline - [ ] Increase GitHub Runners volume sizes - [ ] Make sure all required backing services are provisioned on *acme accounts - [ ] Setup [http://acme.com](http://acme.com) staging domain - [ ] Move aurora-potsgres from *acme accounts to *acme - [ ] Setup [http://acme.com](http://acme.com) temp vanity domain - [ ] Deploy bastion to corp account - [ ] Update RDS Maintenance Window - [ ] Provision ECS Bastion Instance with SSM Agent - [ ] Decide How to Run Database Migrations - [ ] Decide on Database Seeding Strategy - [ ] Decide on deployment strategy for `repository` - [ ] Decide on Log Group Architecture - [ ] Implement `cloudposse/terraform-aws-code-deploy` module - [ ] Add Instance Profile to GitHub Runners to Support Pushing to ECR - [ ] Use Postgres terraform provider to manage users - [ ] Deploy self-hosted GitHub Action Runners with Terraform - [ ] Proposal: Implement GitOps-driven Continuous Delivery Pipeline for Microservices and Preview Environments - [ ] Decide on RDS Maintenance Window - [ ] Move remaining child modules from acme-com to infrastructure registry ### Cutover Plan ##### Rollback Plan - [ ] Verify Backup Integrity and Recency - [ ] Ensure ability to perform software rollbacks with automation (E.g. CI/CD) - [ ] Prepare step-by-step plan to rollback to Heroku ##### External Availability Monitoring - [ ] Enable “Real User Monitoring” (RUM). Establish a 1-2 week baseline before launch - [ ] Enable external synthetic tests 2-4 weeks before launch to identify any potential stability problems (e.g. during deployments) ##### Exception Logging - [ ] Ensure you have frontend/javascript exception logging enabled in Datadog ##### QA - [ ] Test & Time Restore Process (x minutes) - [ ] Audit errors/warnings from pg_restore to ensure they are acceptable - [ ] Coordinate with QA team on acceptance testing - [ ] Ensure robots.txt blocks crawlers on non-prod environments ##### Load Tests - [ ] Replicate production workloads to ensure systems handle as expected - [ ] Tune EKS Autoscaling - [ ] Verify Alert Escalations ##### Reduce DNS TTLs - [ ] Set all SOAs for TLDs (e.g. `acme.com`) to 60 seconds to mitigate effects of negative DNS caching - [ ] Set TTLs to 60 seconds on branded domains (E.g. `acme.com`) ##### Security - [ ] Audit Security Groups (EKS & RDS) ##### Schedule Cut Over - [ ] Identify all relevant parties, stakeholders - [ ] Communicate scope of migration and any expected downtime ##### Prepare Maintenance Page - [ ] Provide a means to display a maintenance page (if necessary) - [ ] Should be a static page (e.g. hosted on S3) - [ ] Update copy as necessary to communicate extent of the outage our downtime ##### Perform End-to-End Tests - [ ] Verify deployments are working - [ ] Verify software rollbacks are working - [ ] Verify auto-scaling is working (pods and nodes) - or we can over-provision for go-live - [ ] Verify TLS certificates are in working order (non-staging) - [ ] Verify logs are flowing to cloudwatch and Datadog - [ ] Verify TLD redirects are working ##### Perform Cut-Over - [ ] [Choose time] Activate Maintenance Page - [ ] Delegate [http://acme.com](http://acme.com) zone to new account - [ ] Take Fresh Production Database Dump on Bastion - [ ] Load Database Dump - [ ] Update env vars in Production SSM to use prod settings from 1password - [ ] Disable Heroku deployments - [ ] Perform ACM flip for [http://acme.com](http://acme.com) - [ ] Disable monitoring? - [ ] Merge/Rebase main into acme-master - [ ] Open PR for acme-master into main - [ ] replace `acme-master` with `master` in github - [ ] Merge the PR to master - [ ] Merge the auto-generated PR in `infra` - [ ] Confirm ALL deployments in spacelift - [ ] Instruct QA team to commence testing on `app.acme.com` - [ ] Flip CNAME for [http://acme.com](http://acme.com) to [http://acme.com](http://acme.com) in legacy account - [ ] Manual TLS validation for [http://acme.com](http://acme.com) ACM - [ ] Instruct QA team to commence testing on `app.acme.com` - [ ] Enable monitoring - [ ] Deactivate Maintenance Page (happens automatically by flipping DNS) ##### Post-Cut-over Checklist - [ ] Verify ability to deploy - [ ] Monitor customer support tickets - [ ] re-enable scheduled EKS tasks for production - [ ] Review exception logs - [ ] Review Slow Query Logs - [ ] Monitor non-200 status codes for anomalies - [ ] Check Real End User Data - [ ] Audit Errors/Warnings after loading - [ ] Ensure `robots.txt` is permitting indexing in production (SEO) ### Post Cutover Tasks - [ ] Ensure Idempotent Plan for Scheduled EKS Tasks - [ ] Rename acme component to `acme-com` - [ ] Configure auto-scaling - [ ] Fix Bastion host to access Redis - [ ] Tune Healthcheck Settings - [ ] Automatically add `migrate` label - [ ] Improve Automated PR Descriptions - [ ] Clean up acme Artifacts In Spacelift (no longer needed after move to acme-com) - [ ] Update Spacelift for acme - [ ] Remove unneeded resources from data accounts ##### Someday - [ ] Prepare `acme.com` vanity domain in `prod` and all DNS records (do not delegate NS) --- ## Tutorials(11) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; These are some additional tutorials that will help you along with the software-delivery components. --- ## Component Development import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from "react-player"; import TaskList from '@site/src/components/TaskList'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Admonition from '@theme/Admonition'; # The Problem While all companies are unique, their infrastructure doesn't need to be. Well-built infrastructure consists of reusable building blocks that implement all the standard components like servers, clusters, load balancers, etc. Rather than building everything from scratch “the hard way”, there's an easier way. Using our “reference architecture” and its service catalog of all the essential pieces of infrastructure, everything a business needs can be composed together as an architecture using “Stack” configurations. Best of all, it's all native terraform.
AI generated voice
# Our Solution Cloud Posse defines components. Components are opinionated, self-contained building blocks of Infrastructure-as-Code (IAC) that solve one specific problem or use-case. Components are similar to a Terraform root module and define a set of resources for any deployment. - Terraform components live under the `components/terraform` directory. - Cloud Posse maintains a collection of public components with [`terraform-aws-components`](https://github.com/cloudposse/terraform-aws-components) - The best components are generic enough to be reused in any organization, but there's nothing wrong with writing specialized components for your company. - Detailed documentation for using components with Atmos can be found under [atmos.tools Core Concepts](https://atmos.tools/core-concepts/components/) :::info Pro tip! We recommend that you always check first with Cloud Posse to see if we have an existing component before writing your own. Sometimes we have work that has not yet been upstreamed to our public repository. ::: ## Prerequisites In order to be able to create a new component, this document assumes the developer has the following requirements: - [ ] Authentication to AWS, typically with Leapp - [ ] The infrastructure repository cloned locally - [ ] Geodesic up and running - [ ] A basic understanding of Atmos - [ ] An intermediate understanding of Terraform ## Create the component in Terraform - Make a new directory in `components/terraform` with the name of the component - Add the files that should typically be in all components: ```console . ├── README.md ├── component.yaml # if vendoring from cloudposse ├── context.tf ├── main.tf ├── outputs.tf ├── providers.tf ├── remote-state.tf ├── variables.tf └── versions.tf ``` All the files above should look familiar to Terraform developers, except for a few of the following.
`context.tf`
Cloud Posse uses `context.tf` to consistently set metadata across all resources. The `context.tf` is always identical. Copy it exactly from [here](https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf). ```bash curl -sL https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/exports/context.tf -o context.tf ```
`providers.tf`
- For `providers.tf`, if we are just using AWS providers only, copy it from [our common files (commonly referred to as mixins) folder](https://github.com/cloudposse/terraform-aws-components/blob/master/mixins/providers.depth-1.tf). - If we are using Kubernetes, then you may need an additional providers file for Helm and Kubernetes providers. Also copy this file from [the mixins folder](https://github.com/cloudposse/terraform-aws-components/blob/master/mixins/provider-helm.tf).
`remote-state.tf`
By convention, we use this file when we want to pull Terraform Outputs from other components. See [the `remote-state` Module](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state).
`component.yaml`
The component manifest is used for vendoring the latest version from Cloud Posse. More on this later.
## Add Terraform Modules or Resources - In your `main.tf`, or other file names of your choosing, add configurations of Terraform modules and/or resources directly. - When you use a Cloud Posse module, you should **pass the context metadata into the module**, like [this](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/s3-bucket/main.tf#L35). All Cloud Posse modules have the variable `context`, which you pass `module.this.context` - You could also use other external modules that are not provided by Cloud Posse. - Use `module.this.tags` when you want to pass a list of tags to a resources or module not provided by Cloud Posse. Tags are already included with `var.context` for any Cloud Posse module. Cloud Posse has a lot of open source modules, so [check here first](https://registry.terraform.io/namespaces/cloudposse) to avoid repeating existing effort. - Handle the variable `module.this.enabled`, so that resources are not created when `var.enabled` is set to `false`. Cloud Posse modules will do this automatically when passed `var.context`. When adding a resource or using a non-Cloud Posse module, then configure enabled with a count, for example `count = module.this.enabled ? 1 : 0` - Use [`remote-state`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/latest/submodules/remote-state) to read Terraform Output from other components. [For example the `eks/alb-controller` component](https://github.com/cloudposse/terraform-aws-components/blob/master/modules/eks/alb-controller/remote-state.tf) ## Configure Stacks ### Directory Structure - Put all Stack configurations in the `stacks` directory ``` stacks/ ├── catalog/ ├── mixins/ ├── orgs/ └── workflows/ ``` - Default configurations for a component live in the `catalog` directory, and configurations for deployed accounts live in the `orgs` directory. - All files in the `catalog` and `orgs` directories are **stack configuration files**. Read on for more information. ### Define defaults for a component in the `catalog` directory - The Stack Catalog is used to define a component's default configuration for a specific organization. Define variables that would not be shared in an Open Source setting here. - By convention, name the configuration the same as your component. For example, if your component is `components/terraform/foobar` then the file would be named `stacks/catalog/foobar.yaml` ```yaml components: terraform: foobar: ``` - Above, `foobar` is the name of the component. - Pass variables into Terraform like this: ```yaml components: terraform: foobar: vars: sample_variable_present_in_variables_tf_of_component: "hello-world" ``` ### Component Types - Atmos supports component types with the `metadata.type` parameter - There are two types of components:
real
is a "concrete" component instance
abstract
a component configuration, which cannot be instantiated directly. The concept is borrowed from [abstract base classes](https://en.wikipedia.org/wiki/Abstract_type) of Object Oriented Programming.
- By default, all components are `real` - Define an `abstract` component by setting `metadata.type` to `abstract`. See the following example ```yaml components: terraform: foobar/defaults: # We can name this anything metadata: type: abstract # This is what makes the component `abstract` component: foobar # This needs to match exactly an existing component name vars: tags: team: devops ``` For more details, see [atmos.tools](https://atmos.tools/core-concepts/components/inheritance) - With an `abstract` component default, we can inherit default settings for any number of derived components. For example: ```yaml components: terraform: foobar: # Since this component name matches exactly, we do not need to add `metadata.component` metadata: type: real # This is the default value and is only added for visibility inherits: - foobar/defaults # The name of the `abstract` component vars: sample_variable_present_in_variables_tf_of_component: "hello-world" ``` Now `foobar` will uses the same configuration as `foobar/defaults` but may describe additional variables.
## Add Component Imports - In a stack configuration file, we can import other stack configuration files with `import` - When a file is imported, the YAML from that file is deep merged on top of earlier imports. This is the same idea as merging two dictionaries together. - Stack configurations can import each other as needed, and there can be multiple layers or different hierarchies of configurations ## Deploy Components with a Stack - In the directory corresponding to the environment you want to deploy in, for example `stacks/orgs/acme/plat/sandbox/us-east-1/`, add a new file (or adding to an existing file) your component by importing it from the catalog. ```yaml import: # These two imports add default variables - orgs/acme/plat/sandbox/_defaults - mixins/region/us-east-1 # This imports a real component, which will deploy even if we do not # inherit from it or override any values. - catalog/foobar ``` - In the above example, we have imported the `foobar` catalog configuration into the `plat-use1-sandbox` environment into a new YAML file of any name. For example `foobar.yaml` ```yaml import: - orgs/acme/plat/sandbox/_defaults - mixins/region/us-east-1 - catalog/foobar components: terraform: foobar: vars: sample_variable_present_in_variables_tf_of_component: "env-specific-config" ``` ## Deploy Now that the component is [(1) defined in Terraform](#create-the-component-in-terraform), [(2) created in Atmos](#catalog-stacks), and [(3) imported in the target Stack](#deploy-components-with-a-stack), now deploy the component with Atmos. ``` atmos terraform apply foobar -s plat-use1-sandbox ```
## Vendoring Atmos supports component vendoring. We use vendor to pull a specific version of the component from [the upstream library](https://github.com/cloudposse/terraform-aws-components). When vendoring a component, 1. Create a branch of your repository 2. Add a `component.yaml` file to the components directory ```yaml apiVersion: atmos/v1 kind: ComponentVendorConfig spec: source: uri: github.com/cloudposse/terraform-aws-components.git//modules/foobar?ref={{ .Version }} version: 1.160.0 included_paths: - "**/**" excluded_paths: [] ``` 3. Fill out the `component.yaml` with the latest version from [the upstream library](https://github.com/cloudposse/terraform-aws-components) 4. Run the vendor commands: `atmos vendor pull --component foobar` 5. Create a Pull Request to check for changes against any existing component. Keep in mind vendoring will overwrite any custom changes to existing files upstream. ## Next Steps At this point, your component is complete in code, but there is still more to do! 1. Run precommit Terraform docs and linting against the new component 2. Add your new component to your GitOps automation tooling, such as Spacelift 3. Configure `CODEOWNERS` for the new component, if necessary 4. Documentation! # References - [Cloud Posse's Library of Terraform Components](https://github.com/cloudposse/terraform-aws-components) - [Cloud Posse's Library of Terraform Modules](https://github.com/orgs/cloudposse/repositories?q=terraform-aws&type=all&language=&sort=) - [Atmos Core Concepts](https://atmos.tools/core-concepts/components/#types-of-components) --- ## Exercises import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; Exercises are a great way to get hands-on with the code. None of these are necessary, but if you would like to take the initiative, feel free to complete any or all of these exercises. If you have questions or would like Cloud Posse to review your solution, please reach out! ## Exercise: Add a component from Cloud Posse to your library This exercise is intended to demonstrate how to pull an existing component that is already upstream and supported by Cloud Posse into your environment. For the sake of this exercise, let's create an SQS Queue component. This component is supported by Cloud Posse, find and deploy that component into your sandbox environment. When this component is deployed, you should have a single new Spacelift stack (if you're using Spacelift) and have created a new SQS queue in your sandbox environment. This queue should be logically named following Cloud Posse naming standards, and that name should be easily retrievable with Terraform state. ## Exercise: Create a new custom component not supported by Cloud Posse This exercise is intended to practice creating a new component that is not supported by Cloud Posse. Cloud Posse does not currently have a component for a static website. For this exercise, use the [`cloudposse/cloudfront-s3-cdn/aws` module](https://github.com/cloudposse/terraform-aws-cloudfront-cdn) to deploy a basic static website to your sandbox environment. When this exercise is complete, you should have a new component, a new component catalog, and an import into sandbox. This component should follow Cloud Posse convention outlined above and be maintained with GitOps (if applicable). --- ## Frequently Asked Questions(Component-development) ### When should we write a new component? Developing a new component may be necessary when: 1. Existing components do not provide the required functionality and the existing component cannot be easily extended. 2. [Cloud Posse's Library of Terraform Components](https://github.com/cloudposse/terraform-aws-components) does not already provide the component. We recommend asking Cloud Posse in your shared Slack channel with us if we happen to have the component not yet upstreamed. 3. You have some existing Terraform code that you would like to import into the Atmos framework. ### Can components include other components? Components should not include other components but instead can refer to another component's Terraform Output with `remote-state`. Components can however refer to any Terraform module ### What is the purpose of `context.tf`? Cloud Posse uses [`context.tf`](/resources/legacy/learning-resources#the-contexttf-pattern) to consistently set metadata across all modules and resources. ### Where should the `context.tf` file be copied from? The `context.tf` file should be copied exactly from https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/exports/context.tf. ### How can Terraform modules or resources be added to a component? Terraform modules or resources can be added directly to the `main.tf` file or other files of your choosing. **Please be advised** that modifying any of the vendored components may break the ability to pull down upstream changes. If modifying an existing component to preserve vendoring capabilities, name your files something like `customizations.extended.tf`, so that it won't collide with upstream files (e.g. using `.extended.tf` as the extension). Similarly, if needing to modify core functionality, consider using [Terraform overrides](https://developer.hashicorp.com/terraform/language/files/override). ### What is an abstract component? An [abstract component](https://atmos.tools/core-concepts/components#types-of-components) is a component configuration that cannot be instantiated directly. ### Does Cloud Posse have automated testing for components? We do not have automated testing for our components at this time. However, GitOps pipelines with Spacelift can automate component deployment into lower environments so that we can validate any code changes in each stage before reaching production. Also, we do have automated tests for all Terraform modules the components built off of ([rds-cluster example](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/master/test)) and static code linting for Terraform and Atmos (with a local `.github/workflows/pre-commit.yml`). --- ## Count vs For Each import Intro from '@site/src/components/Intro'; import Note from '@site/src/components/Note'; import PillBox from '@site/src/components/PillBox'; Terraform in Depth This article is part of our "Terraform in Depth" series, where we dive into the details of Terraform that require a deeper understanding and longer explanation than are required for our other Terraform articles. When you are dynamically creating multiple instances of a resource in Terraform, you have two options: `count` and `for_each`. Both of these options allow you to create multiple instances of a resource, but they have different use cases and different implications for your Terraform code. ![Terraform](/assets/08bcd99-terraform.png) There are 2 key considerations when using `count` or `for_each`: 1. **Addressing**: Terraform must be able to determine the "address" of each resource instance during the planning phase. This is discussed in the next section. 2. **Stability**: When using `count`, resources whose configuration has not changed can nevertheless be destroyed and recreated by Terraform because they have moved to a new address. Using `for_each` usually avoids this issue. This is discussed further below. Use `for_each` when possible, and `count` when you can't use `for_each`. ### Background: Terraform Resource Addressing During the planning phase, Terraform must be able to determine the ["address"](https://developer.hashicorp.com/terraform/cli/state/resource-addressing) of each resource instance. The address of a resource instance is a unique identifier that Terraform uses to track the state of the resource. The address is a combination of the resource type, the resource name, and, when that is not unique due to `count` or `for_each`, the index or key of the resource instance, possibly along with other information. For example: ```hcl locals { availability_zone_ids = ["usw2a", "usw2b"] } resource "aws_eip" "pub" { count = length(local.availability_zone_ids) } ``` will generate resources with addresses like `aws_eip.pub[0]` and `aws_eip.pub[1]`. ```hcl resource "aws_eip" "pub" { for_each = toset(local.availability_zone_ids) } ``` will generate resources with addresses like `aws_eip.pub["usw2a"]` and `aws_eip.pub["usw2b"]`. The values supplied to for each (either the strings in a set of strings, or the keys of a map) are used as the keys in the addresses. :::important Although documentation and commentary often refer to the requirement that Terraform must know at plan time how many instances of a resource to create, it is more accurate to say that Terraform must know at plan time the address of each instance of a resource under its management. This is because the address is used to as the key to the data structure that stores the state of the resource, and Terraform must be able to access that data during the plan phase to compare it to the desired state of the resource and compute the necessary changes. If some address cannot be determined at plan time, `terraform plan` will fail with an error. This issue is discussed in greater detail in [Error: Values Cannot Be Determined Until Apply](/learn/component-development/terraform-in-depth/terraform-unknown-at-plan-time). The main reason not to use `for_each` is that the values supplied to it would not be known until apply time. ::: ### Count is Easier to Determine, but Less Stable The `count` option operates on simple integers: you specify the number of resource instances you want to create (`n`), and Terraform will create that many (0 to `n-1`). Because you can often know at plan time how many instances of a resource you will need without knowing exact details of each instance, `count` is almost always easier to use than `for_each`. However, `count` is less stable than `for_each`, which makes it less desirable. #### Use `count` for Simple Optional Cases When you have a simple case where you know you want to create zero or one instance of a resource, particularly as the result of a boolean input variable, `count` is the best choice. For example: ```hcl resource "aws_instance" "bastion" { count = var.bastion_enabled ? 1 : 0 # ... } ``` None of the drawbacks of `count` versus `for_each` apply when you are never creating more than one instance, so the advantages of `count` versus `for_each` favor using `count` in this case. This is, in fact, the most common usage of `count` in Cloud Posse's Terraform modules. Similarly, to avoid using two variables for an optional resource, for example `vpc_enabled` and `vpc_ipv4_cidr_block`, you can use a single variable and toggle the option based on whether or not it is supplied. :::caution Do not condition the creation of a resource on the _value_ of a variable. Instead, place the value in a list and condition the creation of the resource on the length of the list. This is discussed in greater detail below. ::: It can be tempting, and indeed early Cloud Posse modules did this, to use the value of a variable to determine whether or not to create a resource. ```hcl # DO NOT DO THIS variable "vpc_ipv4_cidr_block" { type = string default = null } resource "aws_vpc" "vpc" { # This fails when var.vpc_ipv4_cidr_block is computed during the apply phase count = var.vpc_ipv4_cidr_block == null ? 0 : 1 cidr_block = var.vpc_ipv4_cidr_block } ``` The problem with this approach is that it requires the value of `var.vpc_ipv4_cidr_block` to be known at plan time, which is frequently not the case. Often, the value supplied will be generated or computed during the apply phase, and this whole construct fails in this scenario. The recommended way to toggle an option by supplying a value is to supply the value inside a list, and toggle the option based on the length of the list. ```hcl variable "vpc_ipv4_cidr_block" { type = list(string) default = [] # Accepting the value as a list can lead a casual user to think that # they can supply multiple values, and that each value will be used # somehow, perhaps to create multiple VPCs. To prevent confusion or surprise, # add a validation rule. Without this kind of validation rule, the user # will not get any feedback that their additional list items are being ignored. validation { condition = length(var.vpc_ipv4_cidr_block) < 2 error_message = <<-EOT The list should contain at most 1 CIDR block. If the list is empty, no VPC will be created. EOT } } resource "aws_vpc" "vpc" { count = length(var.vpc_ipv4_cidr_block) cidr_block = var.vpc_ipv4_cidr_block[count.index] } ``` This allows the user to choose the option without having to supply the value as configuration available at plan time. #### The Instability of `count` The problem with `count` is that when you use it with a list of configuration values, the resource instances are configured according to their index in the list. If you add or remove an item from the list, the index of other items in the list will change, and so even though a resource configuration has not changed in any fundamental way, the configuration will now apply to a different instance of the resource, effectively causing Terraform to destroy it in one index and recreate it in another. For example, consider the case where you want to create a set of IAM Users. We will illustrate the problem with `count` here, and then show how to use `for_each` to avoid the problem in the next section. To create a reusable module that creates IAM users, you might do something like this: ```hcl variable "users" { type = list(string) } resource "aws_iam_user" "example" { count = length(var.users) name = var.users[count.index] } output "users" { value = aws_iam_user.example } ``` Say you first deploy this configuration with the following input: ```hcl module "users" { source = "./iam_users" users = ["Dick", "Harry"] } output "ids" { value = { for v in module.users.users : v.name => v.id } } ``` You will get a plan like this (many elements omitted): ```hcl # module.users.aws_iam_user.example[0] will be created + resource "aws_iam_user" "example" { + name = "Dick" } # module.users.aws_iam_user.example[1] will be created + resource "aws_iam_user" "example" { + name = "Harry" } Changes to Outputs: + ids = { + Dick = "Dick" + Harry = "Harry" } ``` This is all fine, until you realize you left out "Tom". You revise your root module like this: ```hcl module "users" { users = ["Tom", "Dick", "Harry"] } ``` You will get a plan like this (many elements omitted): ```hcl # module.users.aws_iam_user.example[0] will be updated in-place ~ resource "aws_iam_user" "example" { id = "Dick" ~ name = "Dick" -> "Tom" tags = {} # (5 unchanged attributes hidden) } # module.users.aws_iam_user.example[1] will be updated in-place ~ resource "aws_iam_user" "example" { id = "Harry" ~ name = "Harry" -> "Dick" tags = {} # (5 unchanged attributes hidden) } # module.users.aws_iam_user.example[2] will be created + resource "aws_iam_user" "example" { + id = (known after apply) + name = "Harry" } Plan: 1 to add, 2 to change, 0 to destroy. Changes to Outputs: ~ ids = { ~ Dick = "Dick" -> "Harry" ~ Harry = "Harry" -> (known after apply) + Tom = "Dick" } ``` Note that because "Tom" was inserted at the beginning of the list, all the other elements moved to new addresses, so all 3 users are going to be modified. In most cases, the existing 2 resources would be destroyed and 3 new resources would be created. That would be bad enough if, for example, the resources were NAT gateways or VPCs. In this particular case, it is even worse! In this case (and for some other resources), existing resources will be updated in place, potentially causing serious problems when those resources are referenced by ID. If you are lucky, you will get an error message like this: ``` Error: updating IAM User (Harry): EntityAlreadyExists: User with name Dick already exists. ``` If you are unlucky (or if you run `terraform apply` 3 times), the change will go through, and user "Dick" will be renamed user "Tom", meaning that whatever access Dick had, Tom now gets. Likewise, user Dick is renamed Harry, getting Harry's access, and Harry get the newly created user. For example, Tom can now log in with user name "Tom" using Dick's password, while Harry will be locked out as a new user. This nightmare scenario has a lot to do with peculiarities of the implementation of IAM principals, but gives you an idea of what can happen when you use `count` with a list of resource configurations. Note: The above behavior was actually observed using Terraform v1.5.7 and AWS provider v5.38.0. Hopefully something less dangerous and confusing happens with the current versions of the tools when you try this yourself, but nevertheless be prepared for behavior like this. :::note All of this instability is a direct consequence of resource configuration being address by its position in a list of configurations. When items are added to or deleted from the list, or when the list provided in a random order (as used to happen with many AWS data sources), resources may be needlessly affected. The answer to this is `for_each`, but that is not without its own limitations. ::: ### For Each is Stable, But Not Always Feasible to Use #### The Stability of `for_each` In large part to address the instability of `count`, Terraform introduced the ability to use `for_each` to create multiple instances of a resource. Where `count` takes a single integer, `for_each` takes a set of strings, either explicitly or as the keys of a map. When you use `for_each`, the instance addresses are the string values of the set of strings passed to `for_each`. We can rewrite the IAM User example using `for_each` like this: ```hcl variable "users" { type = set(string) } resource "aws_iam_user" "example" { for_each = var.users name = each.key } ``` Now, if deploy the code with Dick and Harry, and then you add Tom to the list of users, the plan will look like this (many elements omitted): ```hcl # module.users.aws_iam_user.example["Tom"] will be created + resource "aws_iam_user" "example" { + name = "Tom" } Changes to Outputs: ~ ids = { + Tom = (known after apply) # (2 unchanged attributes hidden) } ``` This is what we want! Nothing has changed in the code regarding Dick or Harry, so nothing has changed in the infrastructure regarding Dick or Harry. ##### The problems with `for_each` If `for_each` is so much better, why is it not used by everybody all the time? The answer is because the keys supplied to `for_each` must be known at plan time. It used to be the case that data sources were not read during the plan phase, so something like: ``` data "aws_availability_zones" "available" { state = "available" } resource "aws_subnet" "zone" { for_each = data.aws_aws_availability_zones.available.names } ``` would fail because the zone names would not be available during the planning phase. Hashicorp has worked to make more data available during planning to mitigate such problems, with one major improvement being reading data sources during planning. Thus you will see a lot of old code using `count` that could now be rewritten to use `for_each` and require a recent version of Terraform. A still current issue would be trying to create a dynamic number of compute instances and assign IP addresses to them. The most obvious way to do this would be to use `for_each` with a map of instance IDs to IP addresses, but the instance IDs are not known until the instances are created, so that would fail. In some cases, possibly such as this one, where the configuration for all the resources is the same, you can generate keys using `range()` so that the association between a compute instance and an IP address remains stable and is not dependent on the order in which instance IDs are returned by the cloud provider. In other cases, such as when the configurations vary, using proxy keys like this has all the same problems as `count`, in which case using `count` is better because it is simpler and all of the issues with `count` are already understood. ::: note Another limitation, though not frequently encountered, is that "sensitive" values, such as sensitive input variables, sensitive outputs, or sensitive resource attributes, cannot be used as arguments to `for_each`. As stated previously, the value supplied to `for_each` is used as part of the resource address, and as such, it will always be disclosed in UI output, which is why sensitive values are not allowed. Attempts to use sensitive values as `for_each` arguments will result in an error. ::: Ideally, as we saw with IAM users in the examples above, the user would supply static keys in the initial configuration, and then they would always be known and usable in `for_each`, while allowing the user to add or remove instances without affecting the others. The main obstacle to this is when the user does not know how many instances they will need. For example, if they need one for each availability zone, the number they need will depend on the region they are deploying to, and they may want to adjust the configuration for each region in that way. ### Conclusion In conclusion, use `for_each` when you can, and `count` when you must. Finding suitable keys to use with `for_each` can be a challenge, but it is often worth the effort to avoid the instability of `count`. Module authors should be sensitive to the needs of their users and provide `for_each` where possible, but consider using `count` where it seems likely the user may have trouble providing suitable keys. One possible solution for module authors (though generally not advisable due to the complexity it introduces) is to accept a list of object that have an optional `key` attribute, and use that attribute as the key for `for_each` if it is present, and use `count` if it is not. This presents a consistent interface to the user, and allows them to use `for_each` when they can, and `count` when they must. It does introduce complexity and new failure modes, such as when some elements have keys and others do not, or when duplicate keys are present, or again if the keys are not known at plan time, so this particular solution should be approached with caution. Weigh the consequences of the complexity against the benefits of the stability of `for_each`. For many kinds of resources, having them be destroyed and recreated is of little practical consequence, so the instability of `count` is not worth the added complexity and potential for failure that `for_each` introduces. ### Further Reading - [Error: Values Cannot Be Determined Until Apply](/learn/component-development/terraform-in-depth/terraform-unknown-at-plan-time). - [Terraform Best Practices](/best-practices/terraform) --- ## Terraform in Depth import Intro from '@site/src/components/Intro' import DocCardList from '@theme/DocCardList' In this section, we dive into advanced details of Terraform that require a deeper understanding of Terraform and longer explanation than are required for the other articles. We provide a lot of information about how to use Terraform and write Terraform code elsewhere on this site: --- ## Error: Values Cannot Be Determined Until Apply import Intro from '@site/src/components/Intro'; import Note from '@site/src/components/Note'; import PillBox from '@site/src/components/PillBox'; Terraform in Depth This article is part of our [Terraform in Depth](/learn/component-development/terraform-in-depth) series, where we dive into advanced details of Terraform that require a deeper understanding of Terraform and longer explanation than are required for our other Terraform articles. ## Terraform Errors When Planning: Values Cannot Be Determined Until Apply One of the more frustrating errors you can encounter when using Terraform is an error message referring to a value "that cannot be determined until apply". These are often referred to as "unknown at plan time" errors, in part because they show up when running `terraform plan`. ``` Error: Invalid count argument The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. ``` These errors always occur in the context of creating a variable number of resources, and they can be confusing, because it can appear to the user that the value in question should be known at plan time. In fact, the same code will sometimes work and sometimes not, depending on the specific details of how it is used and the state of the infrastructure. This is a particular problem for authors of reusable Terraform modules, because they need to be aware of potential problems that may occur when their module is used in contexts they did not anticipate. In this article, we will explain what these errors mean, why they occur, and how to avoid them. ### The Two-Phase Execution Model To begin with, Terraform implements a [two-phase execution model](https://developer.hashicorp.com/terraform/learn/core-workflow). 1. The first phase is the "plan" phase, where Terraform determines what changes are necessary to achieve the desired state. 2. The second phase is the "apply" phase, where Terraform makes the changes determined to be required during the plan phase. The rationale for this two-phase model is to allow Terraform to show you what changes it will make before it makes them. Terraform is designed so that it make no changes during the plan phase, making it always safe to run `terraform plan`. Then, during the apply phase, it will only make the changes you approved from the plan phase. The error message above only occurs during the plan phase, and it means that some value that Terraform needs to know in order to plan the changes is not known while executing the plan phase. It implies that the value is properly defined, but that it depends on some value that will be generated during the apply phase. ### When Does an Unknown Value Cause a Plan to Fail? Terraform always requires you to approve any changes before it makes them, but it does not always show you the exact details of the changes it will make. #### Unknown Individual Attribute Values are Allowed in a Plan It is impractical for Terraform to compute every detail of the changes it will make during the apply phase, and therefore some details can be declared "unknown" at plan time but still allow the plan to succeed and be approved. In general, the value of a resource attribute is allowed to be unknown, and the plan will show that the attribute will change, but show that the new value is unknown, or, more specifically: `(known after apply)`. For example, consider the case where you want to create a new compute instance and then add it as a target to a load balancer. Terraform will not know the specific IP address of the compute instance until it is created, so it cannot show you the exact details of how it will be added as a target to the load balancer. Instead, it will show you that it will add a target to the load balancer, and you will have to approve that change without knowing the exact details. (The alternative would be to require that you create the compute instance in one configuration, obtain its IP address from that configuration, and then use that IP address in a second configuration to add it as a target to the load balancer. If you want that level of control, you can set up your configurations to work that way, but in most cases, people prefer to manage as much as possible with a single configuration.) #### Resource Creation and Destruction Always Require Explicit Approval Terraform always requires explicit approval to create or destroy a resource. It also allows you to create a number of resources of the same type using a parameter with `count` or `for_each` to determine how many to create. For example: ```hcl resource "aws_instance" "bastion" { count = length(var.availability_zones) # ... } ``` This feature is also used in practice to create a resource conditionally: ```hcl resource "aws_instance" "bastion" { count = var.bastion_enabled ? 1 : 0 # ... } ``` Terraform needs to know how many resources to create during the plan phase or the plan will fail with an error message like the one at the beginning of this article. ### What is Known and Unknown at Plan Time, Part 1: The Obvious In the planning phase, Terraform knows the current state of the infrastructure, and some information provided to it via variables and data sources, but it does not know the future state of the infrastructure. (The exact amount of data available at plan time, particularly from data sources, and the freshness of the data from data sources, has varied over time as Terraform has matured, with the general direction being that more data is available at plan time and less data remains unknown. For example, you can tell Terraform to create a new compute instance, and you can tell it what IP address to assign to that instance, at which point Terraform with know the IP of the instance at plan time. Alternatively, you can not supply an IP address, and the cloud provider will assign one, but then Terraform does not know the IP address at plan time, either. In either case, Terraform will not know the specific instance ID until it creates the instance. It is important to note that: - In terms of known versus unknown, `null` was not a special value [prior to Terraform version 1.6](https://github.com/hashicorp/terraform/issues/26755#issuecomment-1771450399). If you create a resource, its ID is not known at plan time. Even if you know that successful creation of the resource will result in a non-null ID, Terraform may not, and a test like `id == null` may fail as being unknown at plan time. - Because this behavior is changing in Terraform, but some people are still using older versions or switching to open source forks due to licensing issues, it is important for authors of reusable modules to be aware that this limitation may exist for many users but be invisible to the module author because they are testing their code with a newer version of Terraform. - Passing values into or out of a module usually does affect whether Terraform knows the value is `null` or not at plan time. For example, if you had a module that would create an EBS volume when an instance is created, you might have a snippet like this: ```hcl resource "aws_ebs_volume" "example" { count = module.ec2_instance.bastion.id != null ? 1 : 0 # ... } ``` In this case, Terraform will not know the value of `module.ec2_instance.bastion.id` at plan time. Passing the instance ID into a module does not change that. (You could, in Terraform version 1.6, declare the input non-nullable (`nullable = false`), and then Terraform would know that the value is not `null` at plan time, but if you did that, then you would never get the `id == null` condition and always create the EBS volume, so that is not a real solution, if you want to make the creation of the EBS volume conditional) It is on Hashicorp's roadmap to allow resource providers to declare attributes to be null or non-null at plan time, but you should not rely on that. Rather, you should be aware that your testing may not turn up this issue because you are using later versions of Terraform and providers, but you should still guard against it. :::tip Using the _value_ of a module input to conditionally create resources is a common source of issues in a reusable module. When the module is called with a configured value, as can be common when testing, the module works fine, but if the value is not known at plan time, which is common in actual use when the value is computed from other resources, the module will fail. Using a `random_integer` with a `keeper` of `timestamp()` can help you simulate the behavior of a value that is not known at plan time during testing and catch these kinds of issues ahead of time. Best practice is to either use a separate boolean input (e.g. `ebs_volume_enabled`) to condition the resource creation, or to take the optional value as an element of a list and use the length of the list to determine whether to create the resource. See [Use feature flags or lists to control optional behaviors](/best-practices/terraform#use-feature-flags-to-enabledisable-functionality) for more information. ::: ### What is Known and Unknown at Plan Time, Part 2: The Less Obvious #### The State of the Infrastructure Is Known at Plan Time Hopefully it is obvious to see why Terraform would complain about not knowing how many null resources to create in the following example: ```hcl resource "random_integer" "example" { min = 1 max = 2 } resource "null_resource" "example" { count = random_integer.example.result } ``` (Here we use `random_integer` to represent a computed value unknown at plan time, and `null_resoure` to represent a dependent resource, so that you can easily try these examples on your own.) Terraform does not know if it should create one or two null resources until it knows the value of `random_integer.example.result`. However, if you run ``` terraform apply -target=random_integer.example -auto-approve ``` so that the `random_integer.example` resource is created, it becomes part of the infrastructure, and Terraform has a known value for `random_integer.example.result` at plan time. Therefore, if you run `terraform apply` after that, it will succeed. This is both the good news and the bad news: - The good news is that Terraform will not complain about theoretically unknown values in most cases where it can figure out the value during the plan phase. (Not in all cases, though, [as we will see below](#the-results-are-the-same-but-the-path-to-get-there-is-different).) This means that if you use a module that uses an input variable to determine how many resources to create, and you provide a value for that input variable that is known at plan time, then Terraform will not complain. - The bad news is that Terraform will not complain about theoretically unknown values in most cases where it can figure out the value during the plan phase. This means that you can write Terraform code that will work in some cases and not in others, and you may not realize it because it works when you try it. #### Known Values Can Be Transformed Into Unknown Values in Non-Obvious Ways ##### The Results are the Same, but the Path to Get There is Different The following example is a bit more subtle. Using the same `random_integer` resource as above, say we want to choose from 1 of 2 configurations, rather than directly affect the number of resources created. For example, we want to configure subnet IDs based on whether the resource is in public or private subnets. Consider the following code: ```hcl locals { visibility = random_integer.example.result == 1 ? "public" : "private" config_map = { public = { subnets = ["subnet-0abcd1234efgh5678", "subnet-1abcd1234efgh5679"] } private = { subnets = ["subnet-2abcd1234efgh5678", "subnet-3abcd1234efgh5679"] } } } resource "null_resource" "example" { count = length(local.config_map[local.visibility].subnets) } ``` This will fail, even though the value of `random_integer.example.result` is irrelevant to the number of resources created. Regardless of the value of `random_integer.example.result`, Terraform should create 2 resources. However, Terraform is not so sophisticated that it can figure out that the keys of the possible maps yield lists of the same length, regardless of which map is chosen. Instead, it will say that the value of `random_integer.example.result` is unknown, so the element of `local.config_map` is unknown, and so on. This is what we referred to above as where Terraform will complain about theoretically unknown values in some cases where it actually could figure out the value during the plan phase. ###### One Solution to This Particular Problem ```hcl locals { visibility = random_integer.example.result == 1 ? "public" : "private" config_map = { public = { subnets = ["subnet-0abcd1234efgh5678", "subnet-1abcd1234efgh5679"] } private = { subnets = ["subnet-2abcd1234efgh5678", "subnet-3abcd1234efgh5679"] } } subnets = [local.config_map[local.visibility].subnets[0], local.config_map[local.visibility].subnets[1]] } resource "null_resource" "example" { count = length(local.subnets) } ``` :::tip We saw before that Terraform could not deduce that the length of `local.config_map[local.visibility].subnets` was 2 regardless of the value of `local.visibility`. However, by explicitly creating a list with 2 entries, Terraform knows the length of the list, and the plan will succeed. This is not the only way to get around the problem, but it is a common one. ::: ##### Explicit Transformations of Lists One good thing about using the length of a list to determine the count is that the length of the list can be known even if the values of the list are not. As we saw in the previous example, the length of `local.subnets` was known even if the subnet IDs in the list were not, ant that was sufficient. On the other hand, transformations of a list with unknown values can make the length of the list once again unknown. - `compact` will remove `null` values from a list and `flatten` will remove empty nested lists (`length(flatten(1, [], 2)` is 2), so the length of the list will become unknown unless all of the values are known, even if you as a human can tell there are no `null` values or empty nested lists. - `distinct` and `toset()` will remove duplicate values from a list, so again the length of the list will become unknown unless all of the values are known. - `sort`, prior to Terraform version 1.6, [would make the length of the list unknown](https://github.com/hashicorp/terraform/issues/31035) unless all of the values were known. :::tip When providing a list that will be used to determine the number of resources to create, it is important to avoid using any transformations that can cause the length of the list to become unknown. ::: ##### Implicit Transformations of Maps For reasons detailed in [Terraform Count vs For Each](/learn/component-development/terraform-in-depth/terraform-count-vs-for-each), it is usually preferable to use `for_each` rather than `count` to create multiple resources. However, when using `for_each`, it is required that all of the keys be known at plan time. If you use a list of strings to make the keys, such as via `zipmap` or a `for` expression, then the list is implicitly transformed into a set via the equivalent of `distinct(compact(list))`. As explained [above](#explicit-transformations-of-lists), this will make the length of the list unknown unless all of the values are known at plan time. In general, keys to map inputs should be user-supplied configuration values given as inputs, and not computed values. Most of the benefits of using maps over lists are lost if you use computed values as keys, so only use lists where it is likely that users can supply the keys. --- ## Learn the Concepts import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import StepNumber from '@site/src/components/StepNumber'; import Step from '@site/src/components/Step'; import ReactPlayer from 'react-player'; Your platform ensures consistent service delivery every time. A well-designed platform seamlessly integrates with your monitoring, security, and compliance systems, building on your established foundation. Automated software delivery pipelines deploy new services quickly and easily. The reference architecture supports AWS EKS, Amazon ECS, and Lambda functions.
AI generated voice
## Setup Prerequisites ## Onboard Yourself ## Review the Toolchain ## Familiarize Yourself with our Conventions ## Develop Your Own Components Once you're done setting up your platform, our attention will shift to how you ship your software by leveraging GitHub Actions and GitHub Action Workflows. --- ## Conventions import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import ReactPlayer from 'react-player' Here’s a summary of all of our conventions across Terraform, Stacks, Catalogs, etc. ## SweetOps Conventions SweetOps is built on top of a number of high-level concepts and terminology that are critical to understanding prior to getting started. In this document, we break down these concepts to help you better understand our conventions as we introduce them. ### Components [Components](/components) are opinionated, self-contained units of infrastructure as code that solve one, specific problem or use-case. SweetOps has two flavors of components: 1. **Terraform:** Stand-alone root modules that implement some piece of your infrastructure. For example, typical components might be an EKS cluster, RDS cluster, EFS filesystem, S3 bucket, DynamoDB table, etc. You can find the [full library of SweetOps Terraform components on GitHub](https://github.com/cloudposse/terraform-aws-components). We keep these types of components in the `components/terraform/` directory within the infrastructure repository. 2. **Helmfiles**: Stand-alone, applications deployed using `helmfile` to Kubernetes. For example, typical helmfiles might deploy the DataDog agent, cert-manager controller, nginx-ingress controller, etc. Similarly, the [full library of SweetOps Helmfile components is on GitHub](https://github.com/cloudposse/helmfiles). We keep these types of components in the `components/helmfile/` directory within the infrastructure repository. One important distinction about components that is worth noting: components are opinionated “root” modules that typically call other child modules. Components are the building blocks of your infrastructure. This is where you define all the business logic for how to provision some common piece of infrastructure like ECR repos (with the [ecr](/components/library/aws/ecr/) component) or EKS clusters (with the [eks/cluster](/components/library/aws/eks/cluster/) component). Our convention is to stick components in the `components/terraform` directory and to use a `modules/` subfolder to provide child modules intended to be called by the components. :::caution We do not recommend consuming one terraform component inside of another as that would defeat the purpose; each component is intended to be a loosely coupled unit of IaC with its own lifecycle. Further more, since components define a state backend, it’s not supported in terraform to call it from other modules. ::: #### Additional Considerations - Components should be opinionated. They define how _your_ company wants to deliver a service. - Components should try to not rely on more than 2 providers, in order to have the most modular configuration. Terraform does not support passing a list of providers via variable, instead, all the providers should be statically listed inside the module. So Using 1-2 providers ensures a simple way exists to create any number of architectures with a given component (e.g. “primary” and “delegated” resources). There are few if any architectures with ternary/quaternary/etc relationships between accounts, which is why we recommend sticking to 1-2 providers. [https://github.com/hashicorp/terraform/issues/24476](https://github.com/hashicorp/terraform/issues/24476) - Components should not have a configuration setting in their names (e.g. `components/terraform/eks-prod` is poor convention). Prod is a type of configuration and the component shouldn’t differ by stage, only by configuration. The acceptable exception to the convention is naming conventions `...-root` which can only be provisioned in the root account (E.g. AWS Organizations). - Components should try to expose the same variables as the upstream child modules unless it would lead to naming conflicts. - Components should use `context.tf` [Terraform](/resources/legacy/fundamentals/terraform). - Components should have a `README.md` with sample usage - Components should be well formatted (e.g. `terraform fmt`) - Components should use `remote-state` where possible to obtain values automatically from other components. All `remote-state` lookups belong in the `remote-state.tf` file. See [How to Use Terraform Remote State](/learn/maintenance/tutorials/how-to-use-terraform-remote-state). - Components should try to upstream as much business logic as possible to child modules to promote reuse. - Components should use strict version pinning in components and lower-bound pinning in terraform modules. See [our best practice for this](/best-practices/terraform#use-miminum-version-pinning-on-all-providers). See [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) after pinning. See [Proposed: Use Strict Provider Pinning in Components](/resources/adrs/proposed/proposed-use-strict-provider-pinning-in-components) for more context. ### Stacks We use [Stacks](/resources/legacy/fundamentals/stacks) to define and organize configurations. We place terraform “root” modules in the `components/terraform` directory (e.g. `components/terraform/s3-bucket`). Then we define one or more catalog archetypes for using the component (e.g. `catalog/s3-bucket/logs.yaml` and `catalog/s3-bucket/artifacts`). Stacks are a way to express the complete infrastructure needed for an environment using a standard YAML configuration format that has been developed by Cloud Posse. Stacks consist of components and the variables inputs into those components. For example, you configure a stack for each AWS account and then reference the components which comprise that stack. The more modular the components, the easier it is to quickly define a stack without writing any new code. Here is an example stack defined for a Dev environment in the us-west-2 region: ```yaml # Filename: stacks/uw2-dev.yaml import: - eks/eks-defaults vars: stage: dev terraform: vars: {} helmfile: vars: account_number: "1234567890" components: terraform: dns-delegated: vars: request_acm_certificate: true zone_config: - subdomain: dev zone_name: example.com vpc: vars: cidr_block: "10.122.0.0/18" eks: vars: cluster_kubernetes_version: "1.19" region_availability_zones: ["us-west-2b", "us-west-2c", "us-west-1d"] public_access_cidrs: ["72.107.0.0/24"] aurora-postgres: vars: instance_type: db.r4.large cluster_size: 2 mq-broker: vars: apply_immediately: true auto_minor_version_upgrade: true deployment_mode: "ACTIVE_STANDBY_MULTI_AZ" engine_type: "ActiveMQ" helmfile: external-dns: vars: installed: true datadog: vars: installed: true datadogTags: - "env:uw2-dev" - "region:us-west-2" - "stage:dev" ``` Great, so what can you do with a stack? Stacks are meant to be a language and tool agnostic way to describe infrastructure, but how to use the stack configuration is up to you. We provide the following ways to utilize stacks today: 1. [atmos](https://github.com/cloudposse/atmos): atmos is a command-line tool that enables CLI-driven stack utilization and supports workflows around `terraform`, `helmfile`, and many other commands 2. [terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils): is our terraform provider for consuming stack configurations from within HCL/terraform. 3. [Spacelift](https://spacelift.io/): By using the [terraform-spacelift-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) you can configure Spacelift continuously deliver components. Read up on why we [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) . ### Global (Default) Region The global region, annotated `gbl`, is an environment or region we use to deploy unique components. A component may be deployed in the `gbl` region for any of the following reasons: 1. The AWS Service itself is global (e.g. S3 bucket, Cloudfront) 1. The AWS Service is forced into a specific region (IAM, Route 53 - similar to AWS declaring something as "global") 1. The AWS Service should only be deployed exactly once across regions (AWS Identity Center) 1. The resource isn't in AWS, Kubernetes, or anywhere we can reasonably assign to a region. This is common with third-party providers such as Spacelift. However, the AWS provider still needs a region to be defined. We set the global region to the primary region as default. This is intended to cause the least confusion when looking for resources, yet the "global region" can be any region. ### Catalogs Catalogs in SweetOps are collections of sharable and reusable configurations. Think of the configurations in catalogs as defining archetypes (a very typical example of a certain thing) of configuration (E.g. `s3/public` and `s3/logs` would be two kinds of archetypes of S3 buckets). They are also convenient for managing [Terraform](/resources/legacy/fundamentals/terraform). These are typically YAML configurations that can be imported and provide solid baselines to configure security, monitoring, or other 3rd party tooling. Catalogs enable an organization to codify its best practices of configuration and share them. We use this pattern both with our public terraform modules as well as with our stack configurations (e.g. in the `stacks/catalog` folder). SweetOps provides many examples of how to use the catalog pattern to get you started. Today SweetOps provides a couple important catalogs: 1. [DataDog Monitors](https://github.com/cloudposse/terraform-datadog-monitor/tree/master/catalog/monitors): Quickly bootstrap your SRE efforts by utilizing some of these best practice DataDog application monitors. 2. [AWS Config Rules](https://github.com/cloudposse/terraform-aws-config/tree/master/catalog): Quickly bootstrap your AWS compliance efforts by utilizing hundreds of [AWS Config](https://aws.amazon.com/config/) rules that automate security checks against many common services. 3. [AWS Service Control Policies](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/master/catalog): define what permissions in your organization you want to permit or deny in member accounts. In the future, you’re likely to see additional open-source catalogs for OPA rules and tools to make sharing configurations even easier. But it is important to note that how you use catalogs is really up to you to define, and the best catalogs will be specific to your organization. ### Collections Collections are groups of stacks. ### Segments Segments are interconnected networks. For example, a production segment connects all production-tier stacks, while a non-production segment connects all non-production stacks. ### Primary vs Delegated Primary vs Delegated is a common implementation pattern in SweetOps. This is most easily described when looking at the example of domain and DNS usage in a multi-account AWS organization: SweetOps takes the approach that the root domain (e.g. `example.com`) is owned by a **primary** AWS account where the apex zone resides. Subdomains on that domain (e.g. `dev.example.com`) are then **delegated** to the other AWS accounts via an `NS` record on the primary hosted zone which points to the delegated hosted zone’s name servers. You can see examples of this pattern in the [dns-primary](/components/library/aws/dns-primary/), [dns-delegated](/components/library/aws/dns-delegated/) and [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-primary-roles) / [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) components. ### Live vs Model (or Synthetic) Live represents something that is actively being used. It differs from stages like “Production” and “Staging” in the sense that both stages are “live” and in-use. While terms like “Model” and “Synthetic” refer to something which is similar, but not in use by end-users. For example, a live production vanity domain of `acme.com` might have a synthetic vanity domain of `acme-prod.net`. ### Docker Based Toolbox (aka Geodesic) In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one convenient image that you can use as your infrastructure automation toolbox. We call it [Geodesic](/resources/legacy/fundamentals/geodesic) and we use it as our DevOps automation shell and as the base Docker image for all of our DevOps tooling. Geodesic is a DevOps Linux Distribution packaged as a Docker image that provides users the ability to utilize `atmos`, `terraform`, `kubectl`, `helmfile`, the AWS CLI, and many other popular tools that compromise the SweetOps methodology without having to invoke a dozen `install` commands to get started. It’s intended to be used as an interactive cloud automation shell, a base image, or in CI/CD workflows to ensure that all systems are running the same set of versioned, easily accessible tools. ### Vendoring Vendoring is a strategy of importing external dependencies into a local source tree or VCS. Many languages (e.g. NodeJS, Golang) natively support the concept. However, there are many other tools which do not address how to do vendoring, namely `terraform`. There are a few reasons to do vendoring. Sometimes the tools we use do not support importing external sources. Other times, we need to make sure to have full-control over the lifecycle or versioning of some code in case the external dependencies go away. Our current approach to vendoring of thirdparty software dependencies is to use [vendir](https://github.com/vmware-tanzu/carvel-vendir) when needed. Example use-cases for Vendoring: 1. Terraform is one situation where it’s needed. While terraform supports child modules pulled from remote sources, components (aka root modules) cannot be pulled from remotes. 2. GitHub Actions do not currently support importing remote workflows. Using `vendir` we can easily import remote workflows. ### Generators Generators in SweetOps are the pattern of producing code or configuration when existing tools have shortcomings that cannot be addressed through standard IaC. This is best explained through our use-cases for generators today: 1. In order to deploy AWS Config rules to every region enabled in an AWS Account, we need to specify a provider block and consume a compliance child module for each region. Unfortunately, [Terraform does not currently support the ability loop over providers](https://github.com/hashicorp/terraform/issues/19932), which results in needing to manually create these provider blocks for each region that we’re targeting. On top of that, not every organization uses the same types of accounts so a hardcoded solution is not easily shared. Therefore, to avoid tedious manual work we use the generator pattern to create the `.tf` files which specify a provider block for each module and the corresponding AWS Config child module. 2. Many tools for AWS work best when profiles have been configured in the AWS Configuration file (`~/.aws/config`). If we’re working with dozens of accounts, keeping this file current on each developer’s machine is error prone and tedious. Therefore we use a generator to build this configuration based on the accounts enabled. 3. Terraform backends do not support interpolation. Therefore, we define the backend configuration in our YAML stack configuration and use `atmos` as our generator to build the backend configuration files for all components. ### The 4-Layers of Infrastructure We believe that infrastructure fundamentally consists of 4-layers of infrastructure. We build infrastructure starting from the bottom layer and work our way up. Each layer builds on the previous one and our structure is only as solid as our foundation. The tools at each layer vary and augment the underlying layers. Every layer has it’s own SDLC and is free to update independently of the other layers. The 4th and final layer is where your applications are deployed. While we believe in using terraform for layers 1-3, we believe it’s acceptable to introduce another layer of tools to support application developers (e.g. Serverless Framework, CDK, etc) are all acceptable since we’ve built a solid, consistent foundation. ## Terraform ### Mixins Terraform does not natively support the object-oriented concepts of multiple inheritances or [mixins](https://en.wikipedia.org/wiki/Mixin), but we can simulate by using convention. For our purposes, we define a mixin in terraform as a controlled way of adding functionality to modules. When a mixin file is dropped into a folder of a module, the code in the mixin starts to interact with the code in the module. A module can have as many mixins as needed. Since terraform does not directly, we instead use a convention of exporting what we want to reuse. We achieve this currently using something we call an `export` in our terraform modules, which publish some reusable terraform code that we copy verbatim into modules as needed. We use this pattern with our `terraform-null-label` using the `context.tf` file pattern (See below). We also use this pattern in our `terraform-aws-security-group` module with the [https://github.com/cloudposse/terraform-aws-security-group/blob/main/exports/security-group-variables.tf](https://github.com/cloudposse/terraform-aws-security-group/blob/main/exports/security-group-variables.tf). To follow this convention, create an `export/` folder with the mixin files you wish to export to other modules. Then simply copy them over (E.g. with `curl`). We recommend calling the installed files something `.mixin.tf` so it’s clear it's an external asset. ### Resource Factories Resource Factories provide a custom declarative interface for defining multiple resources using YAML and then terraform for implementing the business logic. Most of our new modules are developed using this pattern so we can decouple the architecture requirements from the implementation. See [https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c) for a related discussion. To better support this pattern, we implemented native support for deep merging in terraform using our [https://github.com/cloudposse/terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils) provider as well as implemented a module to standardize how we use YAML configurations [https://github.com/cloudposse/terraform-yaml-config](https://github.com/cloudposse/terraform-yaml-config). Examples of modules using Resource Factory convention: - [https://github.com/cloudposse/terraform-aws-service-control-policies](https://github.com/cloudposse/terraform-aws-service-control-policies) - [https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) - [https://github.com/cloudposse/terraform-datadog-platform](https://github.com/cloudposse/terraform-datadog-platform) - [https://github.com/cloudposse/terraform-opsgenie-incident-management](https://github.com/cloudposse/terraform-opsgenie-incident-management) - [https://github.com/cloudposse/terraform-aws-config](https://github.com/cloudposse/terraform-aws-config) ### Naming Conventions (and the `terraform-null-label` Module) Naming things is hard. We’ve made it easier by defining a programmatically consistent naming convention, which we use in everything we provision. It is designed to generate consistent human-friendly names and tags for resources. We implement this using a terraform module which accepts a number of standardized inputs and produces an output with the fully disambiguate ID. This module establishes the common interface we use in all of our terraform modules in the Cloud Posse ecosystem. Use `terraform-null-label` to implement a strict naming convention. We use it in all of our [Components](/components) and export something we call the `context.tf` pattern. [https://github.com/cloudposse/terraform-null-label](https://github.com/cloudposse/terraform-null-label) Here’s video where we talk about it. There are 6 inputs considered "labels" or "ID elements" (because the labels are used to construct the ID): 1. `namespace` 2. `tenant` 3. `environment` 4. `stage` 5. `name` 6. `attributes` This module generates IDs using the following convention by default: `{namespace}-{environment}-{stage}-{name}-{attributes}`. However, it is highly configurable. The delimiter (e.g. `-`) is configurable. Each label item is optional (although you must provide at least one). #### Tenants `tenants` are a Cloud Posse construct used to describe a collection of accounts within an Organizational Unit (OU). An OU may have multiple tenants, and each tenant may have multiple AWS accounts. For example, the `platform` OU might have two tenants named `dev` and `prod`. The `dev` tenant can contain accounts for the `staging`, `dev`, `qa`, and `sandbox` environments, while the `prod` tenant only has one account for the `prod` environment. By separating accounts into these logical groupings, we can organize accounts at a higher level, follow AWS Well-Architected Framework recommendations, and enforce environment boundaries easily. ### The `context.tf` Mixin Pattern Cloud Posse Terraform modules all share a common `context` object that is meant to be passed from module to module. A `context` object is a single object that contains all the input values for `terraform-null-label` and every `cloudposse/terraform-*` module uses it to ensure a common interface to all of our modules. By convention, we install this file as `context.tf` which is why we call it the `context.tf` pattern. By default, we always provide an instance of it accessible via `module.this`, which makes it always easy to get your _context._ 🙂 Every input value can also be specified individually by name as a standard Terraform variable, and the value of those variables, when set to something other than `null`, will override the value in the context object. In order to allow chaining of these objects, where the context object input to one module is transformed and passed on to the next module, all the variables default to `null` or empty collections. ### Atmos CLI We predominantly call `terraform` from `atmos`, however, by design all of our infrastructure code runs without any task runners. This is in contrast to tools like `terragrunt` that manipulate the state of infrastructure code at run time. See [How to use Atmos](/learn/maintenance/tutorials/how-to-use-atmos) ## Helm ### Charts as an Interface Typically, in a service-oriented architecture (SOA) aka microservices architecture, there will be dozens of very similar services. Traditionally, companies would develop a “unique” helm chart for each of these services. In reality, the charts were generated by running the `helm create` ([https://helm.sh/docs/helm/helm_create/](https://helm.sh/docs/helm/helm_create/) ) command that would generate all the boilerplate. As a result, the services would share 99% of their DNA with each other (e.g. like monkeys and humans), and 1% would differ. This led to a lot of tech debt, sprawl, and copy & paste 🍝 mistakes. For proprietary apps deployed by your organization, we recommend taking a different tactic when developing helm charts. Instead, treat charts like an interface - the way you want to deploy apps to Kubernetes. Develop 1-2 charts based on the patterns you want your developers to use (e.g. microservice, batchjob, etc). Then parameterize things like the `image`, `env` values, `resource` limits, `healthcheck` endpoints, etc. Think of charts like developing your own Heroku-like mechanism to deploy an application. Instead of writing 2 dozen charts, maintain one. Make your apps conform to the convention. Push back on changes to the convention unless necessary. **What if we need more than one deployment (or XYZ) in a chart?** That’s easy. You have a few options: a) Deploy the same chart twice; b) Decide if as an organization you want to support that interface and then extend the chart; c) Develop an additional chart interface. **What if we want to add a feature to our chart and don’t want to disrupt other services?** No problem. Charts are versioned. So think of the version of a chart as the version of your interface. Every time you change the chart, bump the version. Ensure all your services pin to a version of the chart. Now when you change the version of the chart in your service, you know that your upgrading your interface as well. **What if we need some backing services?** Good question. You can still use the features of umbrella charts, and even feature flag common things like deploying a database backend for development environments by using a `condition` in the `requirements.yaml` that can be toggled in the `values.yaml`. _Pro-tip:_ Use [https://artifacthub.io/](https://artifacthub.io/) to find ready-made charts you can use. ```yaml - name: elasticsearch version: ^1.17.0 repository: https://kubernetes-charts.storage.googleapis.com/ condition: elasticsearch.enabled - name: kibana version: ^1.1.0 repository: https://kubernetes-charts.storage.googleapis.com/ condition: kibana.enabled ``` --- ## Upgrade & Maintain import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## The Problem Even if we codify our infrastructure, that doesn't mean our job is done. Time needs to be spent updating components, adding features, and fixing bugs. Over time this churn can create significant tech debt or worse, stagnation. Moreover, open source software isn't a silver bullet of 'free updates' ( ie. projects can become stale or abandoned ). ## Our Solution We discuss many of the common tasks, best-practices, and the means of automation, along with guidance on how to lean on that automation without overdoing it early on. The trick with ownership is to take things in stride, starting slow and building up automation. ## What needs to be maintained ### Making decisions Read through your ADR documents and make sure that they are up to date. When you make a deliberate effort to change something, try in earnest to document those changes. This doesn't mean you need to write up components, but certainly express when one technology should be preferred over another, and discuss patterns that your team should adopt and reherse. Good ADR Examples: - Technology choices (adding, removing) - Scaling up infrastructure (new orgs, new regions, etc) - Analysis, direction, or principles of patterns ### Creating components We have a [separate guide on authoring components](/learn/component-development) that can guide you through the methods, but we should also talk about the impact on maintenance separately. [Make sure your component doesn't already exist](https://github.com/cloudposse/terraform-aws-components) and that includes [checking for PRs](https://github.com/cloudposse/terraform-aws-components/pulls). Maximize the use of modules by searching the [Terraform Registry](https://registry.terraform.io/) and remember that there is a high cost to components: - Keep your providers up to date: - GitOps and Spacelift solutions will warn you about deprecation in their logs. Read them and deligently create issues on them. - Dependencies should be carefully considered: - Avoid mixing global and regional resources. Two smaller components will compose better - If you need to make many instances of a resource, consider drying that up with Atmos - i.e. if you need to make 4 VPCs, then make 4 instances of a component that produces the VPC - It is significantly easier to use Atmos for DRYing configuration rather than Terraform - Maintenance will include disabling/enabling components. Make sure that your component respects this flag or it could be very difficult to update and extend. - Consider versioning and maintaining components outside of your infrastructure repo. If you plan for other organizations to use your component, make sure you practice vendoring. ### Updating components Components in Atmos support vendoring. This means you can version them independently of your infrastructure to best manage the operational cost of updating them. Make sure you [read up on how vendoring works in Atmos](https://atmos.tools/core-concepts/components/vendoring) and carefully read [release information](https://github.com/cloudposse/terraform-aws-components/releases) for risks and breaking changes. ### Updating infrastructure When you are working on altering your `stacks` folder, Atmos has several features to help manage the sprawl. [Be sure to read up on how Atmos manages stacks](https://atmos.tools/core-concepts/stacks). Some key patterns for success while maintaining stacks: - [Validate stacks often while configuring them](https://atmos.tools/core-concepts/stacks/validation) - [Use the `describe` command to look at imported files](https://atmos.tools/core-concepts/stacks/describing) - Try to dry up catalog entries after `atmos terraform plan` is working, not before - Often, catalog patterns emerge once your components are configured in many environments - Mixin and layer patterns emerge over many PRs and with maturity. Rushing them can lead to significant tech debt ### Operational Headaches Some situations you should plan for include: - Expect `atmos terraform destroy` to fail. Test with `enabled=false`, then destroy. - Updating runners for things like GitOps, GitHub Actions, or Spacelift can be a catch-22. Carefully consider that while you replace them, they could destroy themselves or otherwise mess up state locks. - What order of operations does a set of infrastructure pieces take? - Document all required clickops. Many apis like AWS still have these - Tools like [Spacelift](https://docs.spacelift.io/concepts/stack/stack-dependencies) understand dependencies. [You can use Atmos to make sure they are tracked](https://atmos.tools/cli/commands/describe/dependents) - Consider [Atmos Workflows](https://atmos.tools/cli/commands/workflow) when steps need to manage resources in peculiar fashions such as using the [Terraform `-target` flag](https://developer.hashicorp.com/terraform/cli/commands/plan#target-address) - It's always easier to add/remove than to mutate. Prefer replacing components whenever you are making complex changes. - If availability or global dependencies are a concern: - ADR Docs should be present to discuss risks and describe how they are mitigated - Consider using a new stage. The [AWS Well-Architected](https://aws.amazon.com/architecture/well-architected/) and [12-factor](https://12factor.net/) patterns go over the patterns of a good platform. _You are maintaining a platform_. ### Secrets rotation Simply put, SSM Parameter Store is very helpful, but it won't let you know about rotation and drift. - Consider [using developer automation features in 1Password](https://1password.com/developers/secrets-management) to help with secret rotation - You can also use an ADR to document credentials that should expire and when - [Atmos workflows](https://atmos.tools/cli/commands/workflow) can be used to rotate secrets - [Terraform has the `time_rotating` resource](https://registry.terraform.io/providers/hashicorp/time/latest/docs/reference/rotating.html) - Make sure you are using bot accounts or applications for GitHub secrets. If any of the tokens in your 1Password vault are personal, there will be foreseeable problems. You can use the `gh auth status` command from the `gh` cli to verify the user of each token. ## Automation and tooling ### Renovate Renovate is a swiss-army knife for keeping abreast of changes in open source software. Some leading patterns and best practices include: - [Renovate can watch for releases and notify you on a dashboard](https://docs.renovatebot.com/key-concepts/dashboard/) - [Renovate can watch Dockerfiles](https://docs.renovatebot.com/modules/datasource/docker/) - [Renovate can notify EndOfLife cycles](https://docs.renovatebot.com/modules/datasource/endoflife-date/) - Make sure you consider your platform, such as Kubernetes or AMI distributions - Consider the aforementioned "dashboard" feature so you avoid alert fatigue - [Renovate can watch terraform modules](https://docs.renovatebot.com/modules/datasource/terraform-module) - [Renovate can watch terraform providers](https://docs.renovatebot.com/modules/datasource/terraform-provider) Since it can be daunting to configure Renovate for everything, we recommend starting with only the basic and most crucial sources of tech debt: - Make sure Geodesic updates create PRs - You'll get a lot of automated updates from this alone, including patches to Terraform and `aws-cli` - Create module and provider rules for custom components ### UpdateCLI [UpdateCLI](https://www.updatecli.io/) is a tool that can be used to update many different types of software and can implement auto-discovery. While the configuration is more complex than Renovate, it can be customized to do much more in-depth automation. Considerations: - Auto-discovery quickly leads to alert fatigue. Consider it for high churn - Updating stacks is possible, and you can even update AMI searches or db versions, but make sure you have a good understanding of the impact of the change before you automate it. ## Atmos Component Updater Atmos has a [Component Updater](https://atmos.tools/integrations/github-actions/component-updater) which can be enabled as a GitHub action. The Atmos Component Updater will automatically suggest pull requests in your new repository. To do so, we need to create and install a GitHub App and allow GitHub Actions to create and approve pull requests within your GitHub Organization. For more on the Atmos Component Updater, see [atmos.tools](https://atmos.tools/integrations/github-actions/component-updater). 1. Ensure [all requirements are met](https://atmos.tools/integrations/github-actions/component-updater/#requirements). 1. Set up a [Github App](https://atmos.tools/integrations/github-actions/component-updater/#using-a-github-app) with permission to create Pull Requests. We use a GitHub App because Pull Requests will only trigger other GitHub Action Workflows if the Pull Request is created by a GitHub App or PAT. 1. Create a [GitHub Environment](https://atmos.tools/integrations/github-actions/component-updater/#using-github-environments). With environments, the Atmos Component Updater workflow will be required to follow any branch protection rules before running or accessing the environment's secrets. Plus, GitHub natively organizes these Deployments separately in the GitHub UI ## Maturing infrastructure Many of the topics above concern maturing infrastructure. As you grow, you will find many patterns in how your platform responds to business needs. This takes time. Make sure that you retro your platform regularly. Patterns to consider for maturing your infrastructure include: - Monthly meetings to sync on tech debt, outages, and vulnerabilities - Rotating ownership of components - Reviewing telemetry and auditing PRs ## References - [Renovate](https://www.mend.io/renovate/) - [UpdateCLI](https://www.updatecli.io/) - [Atmos](https://atmos.tools/) - [AWS Well-Architected](https://aws.amazon.com/architecture/well-architected/) - [12-factor](https://12factor.net/) ## FAQs ### How can I quickly patch a newly vendored component? We recommend using [Terraform override files](https://developer.hashicorp.com/terraform/language/files/override) to quickly patch a component. ### What if the resources in my component need to move after vendoring? Consider using [Terraform `moved` configuration](https://developer.hashicorp.com/terraform/tutorials/configuration-language/move-config#move-your-resources-with-the-moved-configuration-block), understanding that the state commands can also be codified in Atmos workflows. ### Should I teach my infrastructure to update itself? It's best to first do as much manual work as possible. Once you feel like you have a well analyzed pattern, consider making a PR to add an ADR and discuss. If the ADR holds up to criticism, it should encapsulate what you plan to automate. ### Developers want to iterate on infrastructure. How do I manage this? If developers want to use your platform in a way that affects terraform state: - Can these resources be released from state? Then give developers access with `aws-teams` - If developers want to codify their own infrastructure outside of your platform: - Do they just need extra environments? Codify them using [Atmos template imports](https://atmos.tools/core-concepts/stacks/imports/#go-templates-in-imports) - Can they manage components in a separate repo? Then vendor their component repo. They can use the `sandbox` account to test their components. Mostly, the platform you make will need room to iterate, but this can get costly quickly. Make sure to start small and set goals to drive when you can increase the cost of the platform. --- ## Customize the Geodesic Shell import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Geodesic solves a lot of problems teams have, but it’s opinionated and probably doesn’t work exactly the way you like your shell to work. Maybe you would like more `alias` definitions, or maybe you don’t like our `alias` definitions. Maybe you want to change the colors of `ls` output (e.g. `DIR_COLORS`) or you just can’t stand using `vim` as the default editor. ## Solution Our own developers couldn’t agree on what the best look-and-feel was for `geodesic` so we added support for customizations. This is how to customize Geodesic at launch time. :::tip Several features of Geodesic can be customized at launch time (rather than during the build of the Docker image) so that people can share an image yet still have things set up the way they like. This document describes how to configure the customization. ::: :::caution ### WARNING One of the key benefits of Geodesic is that it provides a consistent environment for all users regardless of their local machine. It eliminates the "it works on my machine" excuse. While these customization options can be great productivity enhancements as well as provide the opportunity to install new features to try them out before committing to installing them permanently, they can also create the kind of divergence in environments that brings back the "it works on my machine" problem. Therefore, we have included an option to disable the customization files: the preferences, the overrides, and the docker environment files. Simply set and export the host environment variable `$GEODESIC_CUSTOMIZATION_DISABLED` to any value other than "false" before launching Geodesic. ::: The way it works is users can place bash shell scripts on their host computer, which are read in either at the start of the `bash` profile script processing or at the end of it. These shell scripts can set up environment variables, command aliases, shell functions, etc., and through setting environment variables, can cause Geodesic to enable or disable certain features. Users can also choose whether to have a single `bash` history file for all containers or to have separate history files. This is convenient if working with multiple geodesic containers. ### Root Directory for Configuration All configuration files are stored under `$GEODESIC_CONFIG_HOME`, which defaults to `/localhost/.geodesic`. At this time, `/localhost` is mapped to the host `$HOME` directory and this cannot be configured yet, so all configuration files must be under `$HOME`, but within that limitation, they can be placed anywhere. So if you set `$GEODESIC_CONFIG_HOME` to `/localhost/work/config/geodesic`, then files would go in `~/work/config/geodesic/` and below on your Docker host machine. ### Resources There are currently 3 resources used for configuration: - **Preferences**, which are shell scripts loaded very early in the launch of the Geodesic shell. - **Overrides**, which are shell scripts loaded very late in the launch of the Geodesic shell. - `Bash` **History Files**, which store `bash` command line history. Additionally, when Geodesic exits normally, it will run the host command `geodesic_on_exit` if it is available. This is intended to be a script that you write and install anywhere on your PATH to do whatever cleanup you want. For example, change the window title. Both preferences and overrides can be either a single file, named `preferences` and `overrides` respectively, or can be a collection of files in directories named `preferences.d` and `overrides.d`. If they are directories, all the visible files in the directories will be sourced, except for hidden files and files with names matching the `GEODESIC_AUTO_LOAD_EXCLUSIONS` regex, which defaults to `(~|.bak|.log|.old|.orig|.disabled)$`. `bash` history is always stored in a single file named `history`, never a directory of files nor files with any other name. If you want to use a separate history file for one Geodesic-based Docker image not shared by other Geodesic-based Docker images, you must create an empty `history` file in the image-specific configuration directory (see below). ### Configuration by File Placement Resources can be in several places and will be loaded from most general to most specific, according to the name of the docker container image. - The most general resources are the ones directly in `$GEODESIC_CONFIG_HOME`. These are applied first. To keep the top-level directory less cluttered and to avoid name clashes, you can put them in a subdirectory named `defaults`. If that subdirectory exists, then `GEODESIC_CONFIG_HOME` itself is not searched. - The `DOCKER_IMAGE` name is then parsed. Everything before the final `/` is considered the "company" name and everything after is, following the Cloudposse reference architecture, referred to as the "stage" name. So for the `DOCKER_IMAGE` name `cloudposse/geodesic`, the company name is `cloudposse` and the stage name is `geodesic` - The next place searched for resources is the directory with the same name as the "company". In our example, that would be `~/.geodesic/cloudposse`. Resources here would apply to all containers from the same company. - The next place searched for resources is the directory with the same name as the "stage", which is generally the name of the project. In our example, that would be `~/.geodesic/geodesic`. Resources here would apply to all containers with the same base name, perhaps various forks of the same project. - The final place searched is the directory with the full name of the Docker image: `$GEODESIC_CONFIG_HOME/$DOCKER_IMAGE`, i.e. `~/.geodesic/cloudposse/geodesic`. Files here are the most specific to this container. By loading them in this order, you can put your defaults at one level and then override/customize them at another, minimizing the amount of duplication needed to customize a wide range of containers. ### Usage details Preferences and Overrides are loaded in the order specified above and all that are found are loaded. For history files, only the last one found is used. To start keeping separate history, just create an empty history file in the appropriate place. While Preferences and Override files themselves must be `bash` scripts and will be directly loaded into the top-level Geodesic shell, they can of course call other programs. You can even use them to pull configurations out of other places. Symbolic links must be relative if you want them to work both inside Geodesic and outside of it. Symbolic links that reference directories that are not below `$HOME` on the host will not work. When possible, Geodesic mounts the host `$HOME` directory as `/localhost` and creates a symbolic link from `$HOME` to `/localhost` so that files under `$HOME` on the host can be referenced by the exact same absolute path both on the host computer and inside Geodesic. For example, if the host `$HOME` is `/Users/fred`, then `/Users/fred/src/example.sh` will refer to the same file both on the host and from inside the Geodesic shell. In general, you should put most of your customization in the Preferences files. Geodesic (usually) takes care to respect and adapt to preferences set before it starts adding on top of them. The primary use for overrides is if you need the results of the initialization process as inputs to your configuration, or if you need to undo something Geodesic does not yet provide a configuration option for not doing in the first place. ### Example: Adding Aliases and Environment Variables Add the following to `~/.geodesic/defaults/preferences` ``` # Add an alias for `kubectl` alias kc='kubectl' alias ll='ls -al' # Add an alias to easily run `geodesic` inside of kubernetes alias debugpod='kubectl run remote-shell-example --image=public.ecr.aws/cloudposse/geodesic:latest-debian --rm=true -i -t --restart=Never --env="BANNER=Geodesic" -- -l' export AWS_ASSUME_ROLE_TTL=8h export AWS_CHAINED_SESSION_TOKEN_TTL=8h if [[ "$USE_AWS_VAULT" = "true" ]] ; then export AWS_VAULT_SERVER_ENABLED=true export AWS_VAULT_ASSUME_ROLE_TTL=8h # Install the Debian package from cloudposse/packages for `aws-vault` apt-get install -y aws-vault fi ``` ### Example: Customize the command prompt You can set each of the 4 glyphs used by the command line prompt, plus the host file system designator, individually: - `ASSUME_ROLE_ACTIVE_MARK` is the glyph to show when you have AWS credentials active. Defaults to a green, bold, '√' SQUARE ROOT (looks like a check mark): `$'\u221a'` - `ASSUME_ROLE_INACTIVE_MARK` is the glyph to show when you do not have AWS credentials active. Defaults to a red, bold, '✗' BALLOT X: `$'\u2717'` - `BLACK_RIGHTWARDS_ARROWHEAD` is the glyph at the end of the prompt. The troublesome default is '⨠' Z NOTATION SCHEMA PIPING: `$'\u2a20'` - `BANNER_MARK` is the glyph at the start of the first line of a 2-line prompt that introduces the `BANNER` text. Defaults to '⧉', TWO JOINED SQUARES: `$'\u29c9'` - `PROMPT_HOST_MARK` is added to the command line when the current working directory is on the host computer (via a bind mount) and not in the container. Defaults to '(HOST)' with "HOST" in red bold. Disable this feature by setting `export PROMPT_HOST_MARK=""`. The default `BLACK_RIGHTWARDS_ARROWHEAD` is from the Unicode Supplemental plane and therefore may be missing from some systems. You can use `->` instead by adding ``` export BLACK_RIGHTWARDS_ARROWHEAD="->" ``` to your `~/.geodesic/defaults/preferences` file. _Cautions_: - You can set these variables to multiple characters, and use ANSI escape codes to change their colors, but in order for command line editing to continue to work properly, any non-printing characters must be escaped. We have had the best luck with starting the escape with `$'\x01'` and ending it with `$'\x02'`, but there a couple of other, similar options. If you fail to do this, or do it incorrectly, your cursor will be incorrectly positioned when you edit a command line from your history and what you see will not be what is executed. - Command line editing can also be affected by ["ambiguous width characters"](https://www.unicode.org/reports/tr11/tr11-39.html). Unicode characters can be "narrow" or "wide" or "ambiguous". In practice in Roman (e.g. English) scripts, a "wide" character takes up the space of 2 "narrow" characters, and the standard Roman letters are all narrow. In older versions of Unicode, many Emoji are ambiguous width. If you use an ambiguous width character that prints wide but is interpreted as narrow, command line editing will suffer from incorrect cursor placement, as with non-printing characters above. This can be worked around by finding and selecting a preference such as "Treat ambiguous characters as wide" (if you can find such a preference), but we recommend just avoiding characters that cause problems. ### Troubleshooting If customizations are not being found or are not working as expected, you can set the host environment variable `$GEODESIC_TRACE` to "custom" before launching Geodesic and a trace of the customization process will be output to the console. --- ## How to Define Stacks for Multiple Regions? import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem We want to deploy a component or set of to multiple regions. The components might need specific settings depending on the region. We want to be as DRY as possible but not compromise on the customization of the configuration. ## Solution First, make sure you’re familiar with [Stacks](/resources/legacy/fundamentals/stacks) and [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks) with [Components](/components) Inheritance. :::tip Define one stack configuration for every region and simply import the catalog configuration with all components you want to reuse per region. ::: Let’s say we want to deploy the [vpc](/components/library/aws/vpc/) into the AWS `us-east-1` and `us-west-2` regions in the `dev` account. We’ll want to customize the CIDR block, region, and availability zones used. Here’s how to do it... 1. Define a catalog entry for the common `vpc` configuration. This is where we can define our organizations best-practices for a VPC. ```yaml # stacks/catalog/vpc.yaml components: terraform: vpc: backend: s3: workspace_key_prefix: vpc settings: spacelift: workspace_enabled: true vars: enabled: true subnet_type_tag_key: acme.net/subnet/type vpc_flow_logs_enabled: true #vpc_flow_logs_bucket_environment_name: uw2 vpc_flow_logs_bucket_stage_name: audit vpc_flow_logs_bucket_tenant_name: mgmt vpc_flow_logs_traffic_type: ALL ``` 2. Now define a stack configuration for the `us-east-1` region. ```yaml # stacks/ue1-dev.yaml import: - catalog/vpc # Define the global variables for this region vars: region: us-east-1 environment: ue1 components: terraform: vpc: vars: cidr_block: 10.1.0.0/18 vpc_flow_logs_bucket_environment_name: ue1 availability_zones: - "us-east-1a" - "us-east-1b" - "us-east-1c" ``` 3. Then repeat the process and define a stack configuration for the `us-west-2` region. ```yaml # stacks/uw2-dev.yaml import: - catalog/vpc # Define the global variables for this region vars: region: us-west-2 environment: uw2 components: terraform: vpc: vars: cidr_block: 10.2.0.0/18 vpc_flow_logs_bucket_environment_name: uw2 availability_zones: - "us-west-2a" - "us-west-2b" - "us-west-2c" ``` Now use the standard [Atmos](/resources/legacy/fundamentals/atmos) commands to plan and apply the stack configurations. --- ## How to Document a New Design Decision import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem During the course of building and managing infrastructure, lots of decisions are made along the way frequently through informal/casual conversations via Slack, Zoom calls, and whiteboarding. Persons involved in making the decisions may come and go. When new team members are onboarded, they lack all the context from previous decisions and need a way to quickly come up to speed. Plus, with so many decisions it’s hard sometimes to remember why a certain decision was made a the time. Usually, we make the best decisions based on the information, best practices, and options available at the time. However, as technology evolves, these options change and it might not be obvious anymore why a particular decision was made or even recalling what options were considered or ruled out. ## Solution Design Decisions are anything we need to confirm architecturally or strategically before performing the implementation. They should include as much context as possible about what options are considered or ruled out. As part of this process, we’ll want to ask the right questions so we gather the necessary requirements for implementation. Once a decision is made, an should be written to capture the decision. Learn [How to write ADRs](/learn/maintenance/tutorials/how-to-write-adrs). ## Process 1. **Identify the layer that the Design Decision is associated with.** 1. Review the other decisions to make sure there’s not one that is similar enough. In that case, we should enrich the context of that decision, rather than create a new one. 1. **Create the Design Decision.** 1. Title/Summary _must_ always begin with “Decide on” so that our automation will automatically recognize it as a Design Decision 2. Add the following 3 sections: (see template) ```markdown ## Status Undecided ## Decision * ## Consequences * ``` 1. Store this document with the other Design Decisions in your repository, typically under `docs/adrs/`. ### Step 2: Create a Document The document should contain all the context regarding the design decision. The context includes what options should be considered or ruled out, any pertinent questions, risks and limitations. Review [our catalog of design decisions](/tags/design-decision) for inspiration. Below are some examples. ## References - https://adr.github.io/madr/ --- ## How to Load Test in AWS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem You’re tasked with load testing a service in AWS. You need to understand what can adversely affect your testing and anything to be aware of before beginning your tests. ## Solution :::caution Load testing without adequate observability in place is not advised. ::: ### Load Testing Methods and Utilities Cloud Posse’s recommended load-testing tool for AWS is [k6](https://k6.io/). k6 is a CLI-driven load-testing-as-code tool from Grafana Labs with a Javascript DSL. Cloud Posse prefers using k6 for load-testing for the following reasons: - **Open Source** - well documented (see [docs](https://docs.k6.io/docs)) and with simple command line usage - **Synthetic Testing** - allows to easily create load test scenarios based on virtual users and simulated traffic configurations - **Small Footprint** - implemented in [Go](https://golang.org/), which has excellent support for concurrency and small memory footprint - **JavaScript DSL** - scenario scripting in `JavaScript` ES2015/ES6 with support for local and remote modules - **Testing as Code** - test logic and configuration options are both in JS for version control friendliness - **Command-line Driven** - can be run from the command line with command & control through CLI - **Compatible with** [**HAR**](http://www.softwareishard.com/blog/har-12-spec/) - has a built-in [HAR](http://www.softwareishard.com/blog/har-12-spec/) converter that will read HAR files and convert them to `k6` scripts (see [session recording](https://docs.k6.io/docs/session-recording-har-support)) - **Automated Testing** - can be easily integrated into CI/CD pipelines - **Comprehensive Metrics** - provides a comprehensive set of built-in [metrics](https://docs.k6.io/docs/result-metrics) - **Beautiful Visualizations** - can stream metrics into [InfluxDB](https://www.influxdata.com/) for storage and visualization with [Grafana](https://grafana.com/) (see [influxdb-grafana](https://docs.k6.io/docs/influxdb-grafana)) A Cloud Posse repository with code snippets and a container image definitions for automated testing via k6 exists [here](https://github.com/cloudposse/load-testing). Other alternatives for load-testing on AWS include: - [gatling.io](https://gatling.io/) — another CLI-driven load-testing-as-code tool with Scala, Java and Kotlin DSLs. Gatling does not have a built-in HAR converter, but has a built-in _Gatling Recorder_ which records HTTP requests and responses for later use in load-testing. - [locust.io](https://locust.io/) — a web-UI-driven load-testing framework written in Python. Predates both Gatling and k6, but at the same time may be considered outdated. ### Important Considerations #### Notifying AWS Before Commencing Network Stress Tests Most customer testing is testing workloads for performance (E.g. MTTR). These load tests do not generate traffic levels that qualify them as network stress tests. If on the other hand you’re planning on testing network performance, be advised that AWS has guidelines on how to go about it. Read more here: [https://aws.amazon.com/ec2/testing/](https://aws.amazon.com/ec2/testing/) > This policy only applies when a customer's network stress test generates traffic from their Amazon EC2 instances which meets one or more of the following criteria: sustains, in aggregate, for more than 1 minute, over 1 Gbps (1 billion bits per second) or 1 Gpps (1 billion packets per second); generates traffic that appears to be abusive or malicious; or generates traffic that has the potential for impact to entities other than the anticipated target of the testing (such as routing or shared service infrastructure). Customers will need to ensure the target endpoint has authorized the testing and understands the expected volumes. Some external endpoints or AWS services may have lower than expected thresholds for certain testing scenarios. We understand that many of our large customers generate more than 1 Gbps or 1 Gpps of traffic in normal production mode regularly, which is completely normal and not under the purview of this policy, unless specifically done for the purpose of network stress testing.Stress tests that will exceed these guidelines require that you fill out an Amazon EC2 Network Stress Test intake form, which can be obtained by [completing the form](https://console.aws.amazon.com/support/contacts#/simulated-events). #### AWS Load-testing Limitations - Load testing from outside of AWS is not advised as you’ll hit the limits of all the transit connections. - Load testing from within containers might be limited by the resource constraints of the container - EC2 Network Bandwidth is tied to the instance size. [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-network-bandwidth.html](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-network-bandwidth.html) - RDS clusters recently restored from snapshots will be much less performant as [blocks are lazy-loaded in the background](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_RestoreFromSnapshot.html#:~:text=The%20DB%20instance%20continues%20to%20load%20data%20in%20the%20background.%20This%20is%20known%20as%20lazy%20loading.) and caches are not yet hydrated #### Common AWS Load Testing Pitfalls Review the [AWS Common pitfalls](https://aws.amazon.com/articles/best-practices-in-evaluating-elastic-load-balancing/#:~:text=Common%20Pitfalls%20When%20Testing%20Elastic%20Load%20Balancing) when load testing. #### Horizontal Pod Autoscaling :::caution Horizontal Pod Autoscaling requires the `metrics-server` component be deployed ::: [https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) #### Pre-warming AWS Resources Certain resources in AWS need to be pre-warmed. Most notably, [Elastic Load Balancers still require prewarming](https://aws.amazon.com/articles/best-practices-in-evaluating-elastic-load-balancing/#pre-warming). This includes ALBs, however, we do not have any links to documentation to back that up. Reach out to Amazon to ask them to pre-warm your load balancers. Alternatively, make sure you scale your tests progressively, as ELBs scale best with a gradual increase in traffic load. ELBs do not respond well to spiky loads, and you’ll see 503s if too much traffic is directed their way too quickly. [NLBs do not need prewarming.](https://aws.amazon.com/blogs/aws/new-network-load-balancer-effortless-scaling-to-millions-of-requests-per-second/) NLB is designed to handle tens of millions of requests per second while maintaining high throughput at ultra-low latency, with no effort on the customer's part. As a result, no pre-warm is needed. EKS Clusters and Kubernetes Pods, and ECS tasks do not scale instantaneously. Autoscaling is designed for real-world scenarios where traffic progressively increases. Therefore make sure you either overscale your clusters and containers, or ensure tests are progressively scaled up to allow for autoscaling. Also, consider pre-warming all database caches (e.g. RDS and Elasticache). #### Scaling AWS Resources - Make sure to raise account limits for resources as needed. - [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-limits.html](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-limits.html) - If using SpotInst, make sure to raise the limits of AWS spot instances as well as adjust Budgets in [Spot.io](https://console.spotinst.com/spt/budget/aws) by NetApp. - [https://spot.io/blog/elastigroup-budgets-track-and-administer-your-cloud-compute-spendings-in-real-time/](https://spot.io/blog/elastigroup-budgets-track-and-administer-your-cloud-compute-spendings-in-real-time/) --- ## How to Manage Terraform Dependencies in Micro-service Repositories import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem The organization’s infrastructure monorepo (commonly located in GitHub at `[organization]/infrastructure`, `[organization]/infra`, etc.) is responsible for managing the organization’s foundational infrastructure, such as: - AWS Accounts - VPCs - EKS Clusters - Backing Services However, hosting certain microservice-specific dependencies in the infrastructure monorepo for all Terraform configurations may not be ideal, as their lifecycle is more closely coupled to the microservice than to the general infrastructure. This is particularly true when application-specific Terraform providers are used to manage resources such as Datadog monitors, LaunchDarkly feature flags, application-specific S3 buckets, IRSA IAM Roles, etc. That being said, a Terraform configuration created in a repository outside the infrastructure monorepo is not going to work out of the box with the same features as if it were to exist inside the monorepo. This is primarily because `atmos` is not used outside the monorepo. ## Solution The following is a guide on how to create a Terraform configuration inside a microservice repo that can be used with Spacelift, and which retains some of the features and patterns stemming from the infrastructure monorepo, despite `atmos` not being involved. ### Create the Terraform Configuration Directory in the Microservice Repository First, the Terraform configuration directory needs to be created. This directory should ideally exist in `terraform` in order for it to be easily identified. This directory _cannot_ be `.terraform` (note the preceding period) because that is a designated directory managed by Terraform, even when invoking Terraform from that directory (Terraform creates a `.terraform` directory inside the context from which it is invoked, so `.terraform/.terraform` should not be problematic technically, however Terraform is strict about the presence of any such directory). Because `atmos` is not involved inside the Microservice repository, we are presented with a couple of technical problems: - Cloud Posse’s [remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state) module expects a `stacks` directory, which will not be present in the microservice repository. - Cloud Posse’s `providers.tf` pattern requires an invocation of the `account-map` component’s `iam-roles` submodule, which is not present in the microservice repository. To circumvent both of these issues, we clone the infrastructure monorepo using a `module` block. Ideally, a git ref should be used in order to avoid potentially different results when the Terraform configuration is run. Thus, authoring a release in the infrastructure monorepo is warranted. After this is done, we can drop in the following file into the microservice repo to resolve these two issues: ``` # # This file allows a Terraform config outside the organization's infrastructure repository in order to circumvent the # following two issues: # # 1. Cloud Posse’s 'remote-state' module expects a 'stacks' directory, which will not be present in this repository. # 2. Cloud Posse's providers.tf pattern requires an invocation of the 'account-map' component’s 'iam-roles' submodule, # which is not present in this repository. # # The source attribute in the monorepo and iam_roles module invocations cannot be interpolated. So the git ref at the end # of these source URIs should be kept up to date with the latest release of the monorepo. This can be automated with # the use of Renovate (https://github.com/renovatebot/renovate). # module "monorepo" { source = "git::ssh://git@github.com/ACME/infrastructure.git?ref=v0.0.1" } locals { monorepo_local_path = "${path.module}/.terraform/modules/monorepo" stack_config_local_path = "${local.monorepo_local_path}/stacks" } provider "aws" { region = var.region # `terraform import` will not use data from a data source, so on import we have to explicitly specify the profile profile = coalesce(var.import_profile_name, module.iam_roles.terraform_profile_name) } module "iam_roles" { # https://www.terraform.io/docs/language/modules/sources.html#modules-in-package-sub-directories source = "git::ssh://git@github.com/ACME/infrastructure.git//components/terraform/account-map/modules/iam-roles?ref=v0.0.1" stack_config_local_path_override = local.stack_config_local_path context = module.this.context } variable "import_profile_name" { type = string default = null description = "AWS Profile to use when importing a resource" } variable "region" { description = "AWS Region." type = string } ``` After this file is dropped in, there is no need to create a `providers.tf` file. This file should have a consistent name when being dropped into all of the microservice repositories. For example, `infra-state.mixin.tf`, or something to that effect, so that it can be easily identified across all such repositories. Lastly, ensure that within the `account-map` component in the infrastructure monorepo, the `iam-roles` submodule has an override for `stack_config_local_path` when it is invoking the `account-map` module: ``` module "account_map" { source = "cloudposse/stack-config/yaml//modules/remote-state" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" component = "account-map" privileged = var.privileged environment = var.global_environment_name stack_config_local_path = coalesce(var.stack_config_local_path_override, "../../../stacks") stage = var.root_account_stage_name tenant = var.root_account_tenant_name context = module.always.context } ``` The override variable can be declared as follows: ``` variable "stack_config_local_path_override" { description = <<-EOT An override for the `stack_config_local_path` when invoking the `account-map` module. This is useful when invoking this module from another repository, which may not have a `stacks` directory. Leave this as `null` when not performing such an invocation. EOT default = null } ``` The `terraform/` directory should be structured similar to the following: ``` terraform/ ├── backend.tf.json ├── context.tf ├── default.auto.tfvars ├── drop-in.tf ├── main.tf ├── remote-state.tf └── variables.tf ``` `backend.tf.json` should share the same format as files with the same name in the `components/[component]` directories in the infrastructure monorepo. In particular, `workspace_key_prefix` should be updated to reflect the name of the microservice repository: ``` { "terraform": { "backend": { "s3": { "acl": "bucket-owner-full-control", "bucket": "acme-mgmt-uw2-root-tfstate", "dynamodb_table": "acme-mgmt-uw2-root-tfstate-lock", "encrypt": true, "key": "terraform.tfstate", "profile": "acme-mgmt-gbl-root-terraform", "region": "us-west-2", "workspace_key_prefix": "example-nestjs-api-server" } } } } ``` ### Give the Spacelift GitHub App Access to the Microservice Repository Spacelift needs access to the microservice repository in question in order to create the Spacelift Stack. In order to allow for this, perform the following actions: 1. Visit `https://github.com/organizations/[YOUR ORGANIZATION]/settings/profile` 2. Click `Installed GitHub Apps` 3. Select `spacelift.io`, then `Configure` 4. Select the microservice repositories that will be deployed by Spacelift, in addition to the infrastructure monorepo. ### Create a Spacelift Stack in the Infrastructure Monorepo pointing to the Microservice Repository ``` components: terraform: example-nestjs-api-server: backend: s3: workspace_key_prefix: example-nestjs-api-server settings: spacelift: workspace_enabled: true repository: example-nestjs-api-server branch: main component_root: terraform vars: enabled: true ``` At this point, your stack should be running without issues in Spacelift. ### How to Develop Terraform Configuration Locally In order to develop the Terraform Configuration Locally, you will need the `tfvars` JSON generated by Spacelift when the stack is created. In order to do this, create a very minimal Terraform configuration using the `infra-state.mixin.tf` (and maybe one or two resources). After the Spacelift stack is created using the administrative stack, navigate to the newly-created stack and to the `Env` tab in Spacelift: Copy the contents of `spacelift.auto.tfvars.json` and populate a file with the same name in the Terraform directory in the application repository. You will then be able to run `terraform init` and `terraform plan`. --- ## How to rollback Terraform state for a component import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem If Terraform state for a component gets corrupted or lost for some reason, it’s possible to rollback the terraform state to a previous version since we use S3 bucket versioning on all state objects. ## Solution Components represent terraform “root” modules and their Terraform state is stored in an s3 state bucket. Rolling back to that state requires a series of commands to be executed under the appropriate IAM role. 1. Assume the `root-admin` IAM role to be able to access s3 buckets in the `root` account. From `geodesic`, run the following command (replace `` with your company namespace): ``` assume-role -gbl-root-admin bash -l ``` 2. Run the command `aws s3 ls` to list all the buckets in the `root` account. You should see a state bucket with a name similar to `--root-tfstate` , where `` is your company namespace and `` is the region abbreviation (e.g. `uw2` or `ue2`). 3. Run the following command to list the versions of the state file for a component: ``` aws s3api list-object-versions --bucket --prefix --max-items 10 ``` where `` is the name of the bucket in the format `--root-tfstate` `` is the path to the state file for the component and is usually in the format `/-/terraform.tfstate` You should get a list of state file versions for the component similar to this: ``` { "Versions": [ { "ETag": "\"af821255ecd29b03f5ef538ffc9ded4b\"", "Size": 180384, "StorageClass": "STANDARD", "Key": "/-/terraform.tfstate", "VersionId": "j1hisFJ91fUxQM2N1FyMmNtsMMBJE5NP", "IsLatest": true, "LastModified": "2021-04-30T18:23:24+00:00" }, { "ETag": "\"9053b298ae1aea5e79080d3eba067c4d\"", "Size": 172389, "StorageClass": "STANDARD", "Key": "/-/terraform.tfstate", "VersionId": "ff9uG.N8Iwoe97Bfo1eOD7Ki8p7N1snX", "IsLatest": false, "LastModified": "2021-04-22T15:26:19+00:00", } ], } ``` Choose the `"VersionId"` you want to restore to. 4. Restore the state file to the selected previous version. You do this by using the `copy-object` command. You must copy the previous version of the object into the same bucket, using the same object key. For more information, see [copy-object](https://docs.aws.amazon.com/cli/latest/reference/s3api/copy-object.html) in the _AWS CLI Command Reference_. Run the following command to copy the previous version of the state file into the same bucket: ``` aws s3api copy-object --copy-source "/?versionId=" --key --bucket --server-side-encryption aws:kms ``` where `` is the ID of the previous version that you want to restore to (you can get it from the list returned from the `aws s3api list-object-versions` command) 5. In a separate geodesic run `atmos terraform plan` for the stack you were recovering. You will be prompted with an error: ``` update the Digest value stored in the DynamoDB table to the following value: ``` Copy that digest value. 6. From the `-gbl-root-admin` shell, update DynamoDB digest to the one you were prompted with ``` aws dynamodb update-item --table-name ---tfstate-lock --key '{"LockID": {"S": "//-/terraform.tfstate-md5"}}' --attribute-updates '{"Digest": {"Value": {"S": ""},"Action": "PUT"}}' --return-values UPDATED_NEW | jq '.Attributes.RuleSetVersion.S' ``` [https://docs.aws.amazon.com/AmazonS3/latest/userguide/RestoringPreviousVersions.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RestoringPreviousVersions.html) [https://docs.aws.amazon.com/AmazonS3/latest/userguide/RetrievingObjectVersions.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RetrievingObjectVersions.html) [https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) --- ## How to Rotate Kops Master Node Certificates import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Old Kubernetes clusters (deployed prior to Kubernetes 1.16.2) may require manual certificate rotation. When the certificates expire, the master nodes can no longer communicate and the cluster becomes destabilized. ## Solution If the master nodes are all still operating normally, then upgrading to Kubernetes 1.16.2 or later should fix the issue and prevent it from recurring. However, if master nodes are already off line due to expired certificates, you will need to rotate them manually. :::tip Use ssh, Lens, Teleport, or other means to open a shell on each master node and rotate certificates. ::: ### Confirm certificates expiring ``` echo | openssl s_client -connect localhost:2380 2>/dev/null | openssl x509 -noout -dates ``` ### Backup existing certificates ``` sudo find /mnt/ -name server.key | sudo xargs -I'{}' cp '{}' '{}'.bak.20200412 sudo find /mnt/ -name server.crt | sudo xargs -I'{}' cp '{}' '{}'.bak.20200412 sudo find /mnt/ -name me.key | sudo xargs -I'{}' cp '{}' '{}'.bak.20200412 sudo find /mnt/ -name me.crt | sudo xargs -I'{}' cp '{}' '{}'.bak.20200412 sudo find /mnt/ -name '*.bak.*' ``` ### Delete certificates ``` sudo find /mnt/ -name server.key | xargs -I {} sudo rm {} sudo find /mnt/ -name server.crt | xargs -I {} sudo rm {} sudo find /mnt/ -name me.key | xargs -I {} sudo rm {} sudo find /mnt/ -name me.crt | xargs -I {} sudo rm {} ``` ### Restart etcd-manager to generate new certificates ``` sudo docker ps -f name='etcd-manager' -q | sudo xargs docker kill ``` ### Verify new certs ``` echo | openssl s_client -connect localhost:2380 2>/dev/null | openssl x509 -noout -dates ``` ### Repeat this entire process on all master nodes ### Verify cluster healthy Wait for pods to show as `Ready` ``` kubectl get nodes ``` --- ## How to run Docker-in-Docker with Geodesic? import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Using the a `geodesic` based toolbox, sometimes it would be helpful to be able to leverage all the commands that ship with the image together with other containers. Out-of-the-box we do not ship docker as it adds a lot of unnecessary cruft to the base image. ## Solution :::tip Simply invoke the `geodesic` wrapper script with `--with-docker` or set the environment variable `WITH_DOCKER=true` ::: ### Debian Instructions Here are the steps to reproduce for Debian. First, install the `geodesic` shell. :::info If you’re running from an infrastructure repository, you should run `make install` instead, which will install the appropriate image and wrapper script. ::: Here’s an example of doing it more generally, simply using the geodesic base image that is publicly available. ``` docker run -it public.ecr.aws/cloudposse/geodesic:latest-debian -c init | bash ``` Start the shell, which was just installed (usually to `/usr/local/bin/something`). This will ensure the docker socket is mounted into the container. **Pro tip:** `/usr/loca/bin/geodesic` is a simple shell script. Inspect it to understand how it works. ``` geodesic --with-docker ``` After running this command, you’re dropped into an interactive shell within the container. Then install `docker` as you would normally. :::caution The docker package goes by many different names depending on the OS distribution. ::: ``` apt-get -y install docker.io ``` Validate docker is working: ``` docker ps ``` --- ## How to support GovCloud and Other AWS Partitions with Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem AWS has several distinct types of cloud regions that range from the commercial regions (what most people are familiar with), to the GovCloud regions and China regions. In AWS parlance, these are called partitions. Not all services are supported in these regions due to regulatory requirements and each partition has its own [ARN format](https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/using-govcloud.html). Terraform modules may not support all partitions if they were not written with it in mind. ## Solution :::tip Most Cloud Posse terraform modules should work in these partitions. ::: For services available in the China regions, see [https://www.amazonaws.cn/en/about-aws/regional-product-services/](https://www.amazonaws.cn/en/about-aws/regional-product-services/). For services available in GovCloud, consult [https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/using-services.html](https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/using-services.html). In order to support multiple partitions in HCL, simply use the following convention leveraging the `aws_partition` data source anywhere you need to construct an ARN. ``` # Look up the current AWS partition data "aws_partition" "current" {} data "aws_iam_policy_document" "bucket_policy" { statement { sid = "ExampleUsingPartitions" effect = "Deny" actions = ["s3:PutObject"] # Construct the ARN using the appropriate partition. resources = ["arn:${data.aws_partition.current.partition}:s3:::${join("", aws_s3_bucket.default.*.id)}/*"] principals { identifiers = ["*"] type = "*" } } } ``` --- ## How to Switch Versions of Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; ## Problem You need to use (or want to upgrade to) the latest version of terraform for a component you’re working on, but do not want to affect other components. ## Solution #### Install Desired Version of Terraform First, make sure you’ve installed the latest version of `terraform` in [Geodesic](/resources/legacy/fundamentals/geodesic) by following our guide [How to Upgrade or Install Versions of Terraform](/learn/maintenance/upgrades/how-to-upgrade-or-install-versions-of-terraform). #### Switch Terraform Versions (Local Geodesic Shell) :::tip You can pin the version of `terraform` using `update-alternatives` as well as in the component configuration. ::: To see Versions supported Check either the _Dockerfile_, or run `ls /usr/share/terraform/` on your geodesic shell ``` update-alternatives --set terraform /usr/share/terraform/${Major.Minor}/bin/terraform ``` ### Terraform Releases [https://github.com/hashicorp/terraform/releases](https://github.com/hashicorp/terraform/releases) #### Switch Terraform Versions for a Component on a stack Update the terrafrom component’s settings in `/stacks/globals.yaml` to set the default atmos terraform version and the spacelift version. an example configuration would look like ``` components: terraform: : settings: spacelift: terraform_version: "0.14" # Version of terraform used by Spacelift command: "/usr/bin/terraform-0.14" # Version of terraform used by Atmos on the command line ``` This would then set the spacelift version and local atmos version to use a specific terraform Major.Minor version for that component. the `major.minor.patch` versions are defined (and installed) in the Dockerfile of the Project. :::caution Spacelift installs the version of terraform specified in `terraform_version` and does not use the package distributed in the container. The `terraform_version` must be defined in the `spacelift/default.auto.tfvars` file. ::: #### Spacelift Version Mapping By convention, we have components pin only to _major_ versions of terraform and not patch releases. The precise version of terraform used is configured in the `spacelift/default.auto.tfvars` file. Note, that the `spacelift` component is not yet configurable via stack configurations and all tuning is done via the `default.auto.tfvars` file. :::caution Spacelift does not always support the bleeding edge patch releases of terraform. Reach out to spacelift if the version you need is not available and they’ll let you know the ETA. ::: ``` terraform_version = "0.14.11" # Default version of terraform to use terraform_version_map = { # Map major versions to precise versions of terraform "0.14" = "0.14.11" "0.15" = "0.15.5" "1.0" = "1.0.9" } ``` ## Related - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) --- ## How to terraform non-AWS infrastructure? import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Most of the infrastructure found inside of the infrastructure repository is for AWS, but companies rely on other platforms like Datadog, Snowflake, etc. If the `infrastructure` repository is just for AWS, then where do we put the rest of the infrastructure code for these systems? ## Solution :::tip The `infrastructure` repository is for any Infrastructure as Code, regardless of the platform or service. Simply use a stack name that confers which cloud or region. ::: The `infrastructure/` repository has a `stacks/` folder. Every stack by convention is named something like `$env-$name`, and we use custom AWS short-codes to denote the `$env`. However, another way to think of this is as a region in a cloud, and few services other than AWS have multiple regions exposed. For example, if you wanted to terraform `datadog`, you could consider that an environment or `snowflake` could be another environment. In the latest version of `atmos`, we also now support stack configurations organized into folder structures. This means you could organize configurations like `stacks/datadog/legacy.yaml` and `stacks/snowflake/us.yaml` ## Other Considerations - Even though you’re terraforming some service outside of AWS, does that service directly relate to some service in AWS? Maybe then it should be associated with the AWS stack. - You could also host the infrastructure code in a separate repository and point spacelift to that repository. See [How to Use Multiple Infrastructure Repositories with Spacelift?](/resources/deprecated/spacelift/tutorials/how-to-use-multiple-infrastructure-repositories-with-spacelift) --- ## How to update components.yaml to new organization import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Cloud Posse migrated each component in the `cloudposse/terraform-aws-components` repository to individual repositories under a new [`cloudposse-terraform-components`](https://github.com/orgs/cloudposse-terraform-components) GitHub organization. This change aims to improve the stability, maintainability, and usability of our components. Now all components are available in the new organization, and we need to update the references in the `component.yaml` files ## Solution Updating `component.yaml` is as easy as changing `source.uri` to the new component repo and `source.version` to the latest release version of `cloudposse/terraform-aws-components` - `v1.532.0`. You can do that both manually or automatically using the `component-updater` GitHub Action. :::caution There are two components that have special migration procedure - `spacelift` and `tgw`. Please check the [migration guide](#special-migration-procedure) for more details. ::: ### Automatic update If you are using the [`component-updater`](https://github.com/cloudposse/github-action-atmos-component-updater) GitHub Action you will get `component.yaml` files updated automatically. 1. Run Github action workflow that uses `component-updater` action 2. Review new PRs and merge them ### Manual update 1. For each component in your repository open the `component.yaml` file 2. Find the `source` section and update the `version` field to `1.532.0` 3. Find the `source` section and update the `uri` field to the new component repo according to this table | Component | Old URI | New URI | |-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------| | access-analyzer | github.com/cloudposse/terraform-aws-components.git//modules/access-analyzer?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-access-analyzer//src?ref=\{\{.Version\}\} | | account | github.com/cloudposse/terraform-aws-components.git//modules/account?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-account//src?ref=\{\{.Version\}\} | | account-map | github.com/cloudposse/terraform-aws-components.git//modules/account-map?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-account-map//src?ref=\{\{.Version\}\} | | account-quotas | github.com/cloudposse/terraform-aws-components.git//modules/account-quotas?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-account-quotas//src?ref=\{\{.Version\}\} | | account-settings | github.com/cloudposse/terraform-aws-components.git//modules/account-settings?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-account-settings//src?ref=\{\{.Version\}\} | | acm | github.com/cloudposse/terraform-aws-components.git//modules/acm?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-acm//src?ref=\{\{.Version\}\} | | alb | github.com/cloudposse/terraform-aws-components.git//modules/alb?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-alb//src?ref=\{\{.Version\}\} | | amplify | github.com/cloudposse/terraform-aws-components.git//modules/amplify?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-amplify//src?ref=\{\{.Version\}\} | | api-gateway-account-settings | github.com/cloudposse/terraform-aws-components.git//modules/api-gateway-account-settings?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-api-gateway-account-settings//src?ref=\{\{.Version\}\} | | api-gateway-rest-api | github.com/cloudposse/terraform-aws-components.git//modules/api-gateway-rest-api?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-api-gateway-rest-api//src?ref=\{\{.Version\}\} | | argocd-repo | github.com/cloudposse/terraform-aws-components.git//modules/argocd-repo?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-argocd-github-repo//src?ref=\{\{.Version\}\} | | athena | github.com/cloudposse/terraform-aws-components.git//modules/athena?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-athena//src?ref=\{\{.Version\}\} | | aurora-mysql | github.com/cloudposse/terraform-aws-components.git//modules/aurora-mysql?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-aurora-mysql//src?ref=\{\{.Version\}\} | | aurora-mysql-resources | github.com/cloudposse/terraform-aws-components.git//modules/aurora-mysql-resources?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-aurora-mysql-resources//src?ref=\{\{.Version\}\} | | aurora-postgres | github.com/cloudposse/terraform-aws-components.git//modules/aurora-postgres?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-aurora-postgres//src?ref=\{\{.Version\}\} | | aurora-postgres-resources | github.com/cloudposse/terraform-aws-components.git//modules/aurora-postgres-resources?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-aurora-postgres-resources//src?ref=\{\{.Version\}\} | | auth0/app | github.com/cloudposse/terraform-aws-components.git//modules/auth0/app?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-auth0-app//src?ref=\{\{.Version\}\} | | auth0/connection | github.com/cloudposse/terraform-aws-components.git//modules/auth0/connection?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-auth0-connection//src?ref=\{\{.Version\}\} | | auth0/tenant | github.com/cloudposse/terraform-aws-components.git//modules/auth0/tenant?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-auth0-tenant//src?ref=\{\{.Version\}\} | | aws-backup | github.com/cloudposse/terraform-aws-components.git//modules/aws-backup?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-backup//src?ref=\{\{.Version\}\} | | aws-config | github.com/cloudposse/terraform-aws-components.git//modules/aws-config?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-config//src?ref=\{\{.Version\}\} | | aws-inspector | github.com/cloudposse/terraform-aws-components.git//modules/aws-inspector?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-inspector//src?ref=\{\{.Version\}\} | | aws-inspector2 | github.com/cloudposse/terraform-aws-components.git//modules/aws-inspector2?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-inspector2//src?ref=\{\{.Version\}\} | | aws-saml | github.com/cloudposse/terraform-aws-components.git//modules/aws-saml?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-saml//src?ref=\{\{.Version\}\} | | aws-shield | github.com/cloudposse/terraform-aws-components.git//modules/aws-shield?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-shield//src?ref=\{\{.Version\}\} | | aws-sso | github.com/cloudposse/terraform-aws-components.git//modules/aws-sso?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-identity-center//src?ref=\{\{.Version\}\} | | aws-ssosync | github.com/cloudposse/terraform-aws-components.git//modules/aws-ssosync?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ssosync//src?ref=\{\{.Version\}\} | | aws-teams | github.com/cloudposse/terraform-aws-components.git//modules/aws-teams?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-teams//src?ref=\{\{.Version\}\} | | aws-team-roles | github.com/cloudposse/terraform-aws-components.git//modules/aws-team-roles?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-team-roles//src?ref=\{\{.Version\}\} | | bastion | github.com/cloudposse/terraform-aws-components.git//modules/bastion?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-bastion//src?ref=\{\{.Version\}\} | | cloudmap-namespace | github.com/cloudposse/terraform-aws-components.git//modules/cloudmap-namespace?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-cloudmap-namespace//src?ref=\{\{.Version\}\} | | cloudtrail | github.com/cloudposse/terraform-aws-components.git//modules/cloudtrail?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-cloudtrail//src?ref=\{\{.Version\}\} | | cloudtrail-bucket | github.com/cloudposse/terraform-aws-components.git//modules/cloudtrail-bucket?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-cloudtrail-bucket//src?ref=\{\{.Version\}\} | | cloudwatch-logs | github.com/cloudposse/terraform-aws-components.git//modules/cloudwatch-logs?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-cloudwatch-logs//src?ref=\{\{.Version\}\} | | cognito | github.com/cloudposse/terraform-aws-components.git//modules/cognito?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-cognito//src?ref=\{\{.Version\}\} | | config-bucket | github.com/cloudposse/terraform-aws-components.git//modules/config-bucket?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-config-bucket//src?ref=\{\{.Version\}\} | | datadog-configuration | github.com/cloudposse/terraform-aws-components.git//modules/datadog-configuration?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-credentials//src?ref=\{\{.Version\}\} | | datadog-integration | github.com/cloudposse/terraform-aws-components.git//modules/datadog-integration?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-integration//src?ref=\{\{.Version\}\} | | datadog-lambda-forwarder | github.com/cloudposse/terraform-aws-components.git//modules/datadog-lambda-forwarder?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-lambda-forwarder//src?ref=\{\{.Version\}\} | | datadog-logs-archive | github.com/cloudposse/terraform-aws-components.git//modules/datadog-logs-archive?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-logs-archive//src?ref=\{\{.Version\}\} | | datadog-monitor | github.com/cloudposse/terraform-aws-components.git//modules/datadog-monitor?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-monitor//src?ref=\{\{.Version\}\} | | datadog-private-location-ecs | github.com/cloudposse/terraform-aws-components.git//modules/datadog-private-location-ecs?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-private-location-ecs//src?ref=\{\{.Version\}\} | | datadog-synthetics | github.com/cloudposse/terraform-aws-components.git//modules/datadog-synthetics?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-synthetics//src?ref=\{\{.Version\}\} | | datadog-synthetics-private-location | github.com/cloudposse/terraform-aws-components.git//modules/datadog-synthetics-private-location?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-datadog-synthetics-private-location//src?ref=\{\{.Version\}\} | | dms/endpoint | github.com/cloudposse/terraform-aws-components.git//modules/dms/endpoint?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dms-endpoint//src?ref=\{\{.Version\}\} | | dms/iam | github.com/cloudposse/terraform-aws-components.git//modules/dms/iam?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dms-iam//src?ref=\{\{.Version\}\} | | dms/replication-instance | github.com/cloudposse/terraform-aws-components.git//modules/dms/replication-instance?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dms-replication-instance//src?ref=\{\{.Version\}\} | | dms/replication-task | github.com/cloudposse/terraform-aws-components.git//modules/dms/replication-task?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dms-replication-task//src?ref=\{\{.Version\}\} | | dns-delegated | github.com/cloudposse/terraform-aws-components.git//modules/dns-delegated?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dns-delegated//src?ref=\{\{.Version\}\} | | dns-primary | github.com/cloudposse/terraform-aws-components.git//modules/dns-primary?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dns-primary//src?ref=\{\{.Version\}\} | | documentdb | github.com/cloudposse/terraform-aws-components.git//modules/documentdb?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-documentdb//src?ref=\{\{.Version\}\} | | dynamodb | github.com/cloudposse/terraform-aws-components.git//modules/dynamodb?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-dynamodb//src?ref=\{\{.Version\}\} | | ec2-client-vpn | github.com/cloudposse/terraform-aws-components.git//modules/ec2-client-vpn?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ec2-client-vpn//src?ref=\{\{.Version\}\} | | ec2-instance | github.com/cloudposse/terraform-aws-components.git//modules/ec2-instance?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ec2-instance//src?ref=\{\{.Version\}\} | | ecr | github.com/cloudposse/terraform-aws-components.git//modules/ecr?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ecr//src?ref=\{\{.Version\}\} | | ecs | github.com/cloudposse/terraform-aws-components.git//modules/ecs?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ecs//src?ref=\{\{.Version\}\} | | ecs-service | github.com/cloudposse/terraform-aws-components.git//modules/ecs-service?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ecs-service//src?ref=\{\{.Version\}\} | | efs | github.com/cloudposse/terraform-aws-components.git//modules/efs?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-efs//src?ref=\{\{.Version\}\} | | eks/actions-runner-controller | github.com/cloudposse/terraform-aws-components.git//modules/eks/actions-runner-controller?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-actions-runner-controller//src?ref=\{\{.Version\}\} | | eks/alb-controller | github.com/cloudposse/terraform-aws-components.git//modules/eks/alb-controller?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-alb-controller//src?ref=\{\{.Version\}\} | | eks/alb-controller-ingress-class | github.com/cloudposse/terraform-aws-components.git//modules/eks/alb-controller-ingress-class?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-alb-controller-ingress-class//src?ref=\{\{.Version\}\} | | eks/alb-controller-ingress-group | github.com/cloudposse/terraform-aws-components.git//modules/eks/alb-controller-ingress-group?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-alb-controller-ingress-group//src?ref=\{\{.Version\}\} | | eks/argocd | github.com/cloudposse/terraform-aws-components.git//modules/eks/argocd?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-argocd//src?ref=\{\{.Version\}\} | | eks/aws-node-termination-handler | github.com/cloudposse/terraform-aws-components.git//modules/eks/aws-node-termination-handler?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-node-termination-handler//src?ref=\{\{.Version\}\} | | eks/cert-manager | github.com/cloudposse/terraform-aws-components.git//modules/eks/cert-manager?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-cert-manager//src?ref=\{\{.Version\}\} | | eks/cluster | github.com/cloudposse/terraform-aws-components.git//modules/eks/cluster?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-cluster//src?ref=\{\{.Version\}\} | | eks/datadog-agent | github.com/cloudposse/terraform-aws-components.git//modules/eks/datadog-agent?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-datadog-agent//src?ref=\{\{.Version\}\} | | eks/echo-server | github.com/cloudposse/terraform-aws-components.git//modules/eks/echo-server?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-echo-server//src?ref=\{\{.Version\}\} | | eks/external-dns | github.com/cloudposse/terraform-aws-components.git//modules/eks/external-dns?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-external-dns//src?ref=\{\{.Version\}\} | | eks/external-secrets-operator | github.com/cloudposse/terraform-aws-components.git//modules/eks/external-secrets-operator?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-external-secrets-operator//src?ref=\{\{.Version\}\} | | eks/github-actions-runner | github.com/cloudposse/terraform-aws-components.git//modules/eks/github-actions-runner?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-github-actions-runner//src?ref=\{\{.Version\}\} | | eks/idp-roles | github.com/cloudposse/terraform-aws-components.git//modules/eks/idp-roles?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-idp-roles//src?ref=\{\{.Version\}\} | | eks/karpenter | github.com/cloudposse/terraform-aws-components.git//modules/eks/karpenter?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-karpenter-controller//src?ref=\{\{.Version\}\} | | eks/karpenter-node-pool | github.com/cloudposse/terraform-aws-components.git//modules/eks/karpenter-node-pool?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-karpenter-node-pool//src?ref=\{\{.Version\}\} | | eks/keda | github.com/cloudposse/terraform-aws-components.git//modules/eks/keda?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-keda//src?ref=\{\{.Version\}\} | | eks/loki | github.com/cloudposse/terraform-aws-components.git//modules/eks/loki?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-loki//src?ref=\{\{.Version\}\} | | eks/metrics-server | github.com/cloudposse/terraform-aws-components.git//modules/eks/metrics-server?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-metrics-server//src?ref=\{\{.Version\}\} | | eks/prometheus-scraper | github.com/cloudposse/terraform-aws-components.git//modules/eks/prometheus-scraper?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-prometheus-scraper//src?ref=\{\{.Version\}\} | | eks/promtail | github.com/cloudposse/terraform-aws-components.git//modules/eks/promtail?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-promtail//src?ref=\{\{.Version\}\} | | eks/redis | github.com/cloudposse/terraform-aws-components.git//modules/eks/redis?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-redis//src?ref=\{\{.Version\}\} | | eks/redis-operator | github.com/cloudposse/terraform-aws-components.git//modules/eks/redis-operator?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-redis-operator//src?ref=\{\{.Version\}\} | | eks/reloader | github.com/cloudposse/terraform-aws-components.git//modules/eks/reloader?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-reloader//src?ref=\{\{.Version\}\} | | eks/storage-class | github.com/cloudposse/terraform-aws-components.git//modules/eks/storage-class?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-storage-class//src?ref=\{\{.Version\}\} | | eks/spacelift-worker-pool-controller | github.com/cloudposse/terraform-aws-components.git//modules/eks/spacelift-worker-pool-controller?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-spacelift-worker-pool-controller//src?ref=\{\{.Version\}\} | | eks/spacelift-worker-pool | github.com/cloudposse/terraform-aws-components.git//modules/eks/spacelift-worker-pool?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-spacelift-worker-pool//src?ref=\{\{.Version\}\} | | eks/tailscale | github.com/cloudposse/terraform-aws-components.git//modules/eks/tailscale?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eks-tailscale//src?ref=\{\{.Version\}\} | | elasticache-redis | github.com/cloudposse/terraform-aws-components.git//modules/elasticache-redis?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-elasticache-redis//src?ref=\{\{.Version\}\} | | elasticsearch | github.com/cloudposse/terraform-aws-components.git//modules/elasticsearch?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-elasticsearch//src?ref=\{\{.Version\}\} | | eventbridge | github.com/cloudposse/terraform-aws-components.git//modules/eventbridge?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-eventbridge//src?ref=\{\{.Version\}\} | | github-action-token-rotator | github.com/cloudposse/terraform-aws-components.git//modules/github-action-token-rotator?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-github-action-token-rotator//src?ref=\{\{.Version\}\} | | github-oidc-provider | github.com/cloudposse/terraform-aws-components.git//modules/github-oidc-provider?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-github-oidc-provider//src?ref=\{\{.Version\}\} | | github-oidc-role | github.com/cloudposse/terraform-aws-components.git//modules/github-oidc-role?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-github-oidc-role//src?ref=\{\{.Version\}\} | | github-runners | github.com/cloudposse/terraform-aws-components.git//modules/github-runners?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-github-runners//src?ref=\{\{.Version\}\} | | github-webhook | github.com/cloudposse/terraform-aws-components.git//modules/github-webhook?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-github-webhook//src?ref=\{\{.Version\}\} | | global-accelerator | github.com/cloudposse/terraform-aws-components.git//modules/global-accelerator?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-global-accelerator//src?ref=\{\{.Version\}\} | | global-accelerator-endpoint-group | github.com/cloudposse/terraform-aws-components.git//modules/global-accelerator-endpoint-group?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-global-accelerator-endpoint-group//src?ref=\{\{.Version\}\} | | glue/catalog-database | github.com/cloudposse/terraform-aws-components.git//modules/glue/catalog-database?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-catalog-database//src?ref=\{\{.Version\}\} | | glue/catalog-table | github.com/cloudposse/terraform-aws-components.git//modules/glue/catalog-table?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-catalog-table//src?ref=\{\{.Version\}\} | | glue/connection | github.com/cloudposse/terraform-aws-components.git//modules/glue/connection?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-connection//src?ref=\{\{.Version\}\} | | glue/crawler | github.com/cloudposse/terraform-aws-components.git//modules/glue/crawler?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-crawler//src?ref=\{\{.Version\}\} | | glue/iam | github.com/cloudposse/terraform-aws-components.git//modules/glue/iam?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-iam//src?ref=\{\{.Version\}\} | | glue/job | github.com/cloudposse/terraform-aws-components.git//modules/glue/job?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-job//src?ref=\{\{.Version\}\} | | glue/registry | github.com/cloudposse/terraform-aws-components.git//modules/glue/registry?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-registry//src?ref=\{\{.Version\}\} | | glue/schema | github.com/cloudposse/terraform-aws-components.git//modules/glue/schema?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-schema//src?ref=\{\{.Version\}\} | | glue/trigger | github.com/cloudposse/terraform-aws-components.git//modules/glue/trigger?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-trigger//src?ref=\{\{.Version\}\} | | glue/workflow | github.com/cloudposse/terraform-aws-components.git//modules/glue/workflow?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-glue-workflow//src?ref=\{\{.Version\}\} | | guardduty | github.com/cloudposse/terraform-aws-components.git//modules/guardduty?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-guardduty//src?ref=\{\{.Version\}\} | | iam-role | github.com/cloudposse/terraform-aws-components.git//modules/iam-role?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-iam-role//src?ref=\{\{.Version\}\} | | iam-service-linked-roles | github.com/cloudposse/terraform-aws-components.git//modules/iam-service-linked-roles?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-iam-service-linked-roles//src?ref=\{\{.Version\}\} | | ipam | github.com/cloudposse/terraform-aws-components.git//modules/ipam?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ipam//src?ref=\{\{.Version\}\} | | kinesis-stream | github.com/cloudposse/terraform-aws-components.git//modules/kinesis-stream?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-kinesis-stream//src?ref=\{\{.Version\}\} | | kms | github.com/cloudposse/terraform-aws-components.git//modules/kms?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-kms//src?ref=\{\{.Version\}\} | | lakeformation | github.com/cloudposse/terraform-aws-components.git//modules/lakeformation?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-lakeformation//src?ref=\{\{.Version\}\} | | lambda | github.com/cloudposse/terraform-aws-components.git//modules/lambda?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-lambda//src?ref=\{\{.Version\}\} | | macie | github.com/cloudposse/terraform-aws-components.git//modules/macie?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-macie//src?ref=\{\{.Version\}\} | | managed-grafana/api-key | github.com/cloudposse/terraform-aws-components.git//modules/managed-grafana/api-key?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-grafana-api-key//src?ref=\{\{.Version\}\} | | managed-grafana/dashboard | github.com/cloudposse/terraform-aws-components.git//modules/managed-grafana/dashboard?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-grafana-dashboard//src?ref=\{\{.Version\}\} | | managed-grafana/data-source/loki | github.com/cloudposse/terraform-aws-components.git//modules/managed-grafana/data-source/loki?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-grafana-data-source-loki//src?ref=\{\{.Version\}\} | | managed-grafana/data-source/managed-prometheus | github.com/cloudposse/terraform-aws-components.git//modules/managed-grafana/data-source/managed-prometheus?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-grafana-data-source-managed-prometheus//src?ref=\{\{.Version\}\} | | managed-grafana/workspace | github.com/cloudposse/terraform-aws-components.git//modules/managed-grafana/workspace?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-grafana-workspace//src?ref=\{\{.Version\}\} | | managed-prometheus/workspace | github.com/cloudposse/terraform-aws-components.git//modules/managed-prometheus/workspace?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-managed-prometheus-workspace//src?ref=\{\{.Version\}\} | | memorydb | github.com/cloudposse/terraform-aws-components.git//modules/memorydb?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-memorydb//src?ref=\{\{.Version\}\} | | mq-broker | github.com/cloudposse/terraform-aws-components.git//modules/mq-broker?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-mq-broker//src?ref=\{\{.Version\}\} | | msk | github.com/cloudposse/terraform-aws-components.git//modules/msk?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-msk//src?ref=\{\{.Version\}\} | | mwaa | github.com/cloudposse/terraform-aws-components.git//modules/mwaa?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-mwaa//src?ref=\{\{.Version\}\} | | network-firewall | github.com/cloudposse/terraform-aws-components.git//modules/network-firewall?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-network-firewall//src?ref=\{\{.Version\}\} | | opsgenie-team | github.com/cloudposse/terraform-aws-components.git//modules/opsgenie-team?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-opsgenie-team//src?ref=\{\{.Version\}\} | | philips-labs-github-runners | github.com/cloudposse/terraform-aws-components.git//modules/philips-labs-github-runners?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-philips-labs-github-runners//src?ref=\{\{.Version\}\} | | rds | github.com/cloudposse/terraform-aws-components.git//modules/rds?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-rds//src?ref=\{\{.Version\}\} | | redshift | github.com/cloudposse/terraform-aws-components.git//modules/redshift?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-redshift//src?ref=\{\{.Version\}\} | | redshift-serverless | github.com/cloudposse/terraform-aws-components.git//modules/redshift-serverless?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-redshift-serverless//src?ref=\{\{.Version\}\} | | route53-resolver-dns-firewall | github.com/cloudposse/terraform-aws-components.git//modules/route53-resolver-dns-firewall?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-route53-resolver-dns-firewall//src?ref=\{\{.Version\}\} | | runs-on | github.com/cloudposse/terraform-aws-components.git//modules/runs-on?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-runs-on//src?ref=\{\{.Version\}\} | | s3-bucket | github.com/cloudposse/terraform-aws-components.git//modules/s3-bucket?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-s3-bucket//src?ref=\{\{.Version\}\} | | security-hub | github.com/cloudposse/terraform-aws-components.git//modules/security-hub?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-security-hub//src?ref=\{\{.Version\}\} | | ses | github.com/cloudposse/terraform-aws-components.git//modules/ses?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ses//src?ref=\{\{.Version\}\} | | sftp | github.com/cloudposse/terraform-aws-components.git//modules/sftp?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-sftp//src?ref=\{\{.Version\}\} | | site-to-site-vpn | github.com/cloudposse/terraform-aws-components.git//modules/site-to-site-vpn?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-site-to-site-vpn//src?ref=\{\{.Version\}\} | | snowflake-account | github.com/cloudposse/terraform-aws-components.git//modules/snowflake-account?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-snowflake-account//src?ref=\{\{.Version\}\} | | snowflake-database | github.com/cloudposse/terraform-aws-components.git//modules/snowflake-database?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-snowflake-database//src?ref=\{\{.Version\}\} | | sns-topic | github.com/cloudposse/terraform-aws-components.git//modules/sns-topic?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-sns-topic//src?ref=\{\{.Version\}\} | | spa-s3-cloudfront | github.com/cloudposse/terraform-aws-components.git//modules/spa-s3-cloudfront?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-spa-s3-cloudfront//src?ref=\{\{.Version\}\} | | spacelift/admin-stack | github.com/cloudposse/terraform-aws-components.git//modules/spacelift/admin-stack?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-spacelift-admin-stack//src?ref=\{\{.Version\}\} | | spacelift/spaces | github.com/cloudposse/terraform-aws-components.git//modules/spacelift/spaces?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-spacelift-spaces//src?ref=\{\{.Version\}\} | | spacelift/worker-pool | github.com/cloudposse/terraform-aws-components.git//modules/spacelift/worker-pool?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-spacelift-worker-pool-asg//src?ref=\{\{.Version\}\} | | sqs-queue | github.com/cloudposse/terraform-aws-components.git//modules/sqs-queue?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-sqs-queue//src?ref=\{\{.Version\}\} | | ssm-parameters | github.com/cloudposse/terraform-aws-components.git//modules/ssm-parameters?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-ssm-parameters//src?ref=\{\{.Version\}\} | | sso-saml-provider | github.com/cloudposse/terraform-aws-components.git//modules/sso-saml-provider?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-sso-saml-provider//src?ref=\{\{.Version\}\} | | strongdm | github.com/cloudposse/terraform-aws-components.git//modules/strongdm?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-strongdm//src?ref=\{\{.Version\}\} | | tfstate-backend | github.com/cloudposse/terraform-aws-components.git//modules/tfstate-backend?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-tfstate-backend//src?ref=\{\{.Version\}\} | | tgw/cross-region-hub-connector | github.com/cloudposse/terraform-aws-components.git//modules/tgw/cross-region-hub-connector?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-tgw-hub-connector//src?ref=\{\{.Version\}\} | | tgw/hub | github.com/cloudposse/terraform-aws-components.git//modules/tgw/hub?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-tgw-hub//src?ref=\{\{.Version\}\} | | tgw/spoke | github.com/cloudposse/terraform-aws-components.git//modules/tgw/spoke?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-tgw-spoke//src?ref=\{\{.Version\}\} | | vpc | github.com/cloudposse/terraform-aws-components.git//modules/vpc?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-vpc//src?ref=\{\{.Version\}\} | | vpc-flow-logs-bucket | github.com/cloudposse/terraform-aws-components.git//modules/vpc-flow-logs-bucket?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-vpc-flow-logs-bucket//src?ref=\{\{.Version\}\} | | vpc-peering | github.com/cloudposse/terraform-aws-components.git//modules/vpc-peering?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-vpc-peering//src?ref=\{\{.Version\}\} | | waf | github.com/cloudposse/terraform-aws-components.git//modules/waf?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-waf//src?ref=\{\{.Version\}\} | | zscaler | github.com/cloudposse/terraform-aws-components.git//modules/zscaler?ref=\{\{.Version\}\} | github.com/cloudposse-terraform-components/aws-zscaler//src?ref=\{\{.Version\}\} | ### Special Migration Procedures The following components have been split into multiple repos by subcomponents. Here's what you need to know to update your configurations. #### Spacelift Components 1. Copy the `component.yaml` file from `spacelift` component to `spacelift/admin-stack`, `spacelift/spaces` and `spacelift/worker-pool` subcomponents dirs. 2. Remove `component.yaml` file from `spacelift` component 3. For each subcomponent in `component.yaml` update `source.version` to `1.532.0` 4. For each subcomponent in `component.yaml` update `source.uri` - replace `github.com/cloudposse/terraform-aws-components.git//modules/spacelift?ref={{.Version}}` with | Subcomponent | New URI | |-------------------------|------------------------------------------------------------------------------------------------------| | `spacelift/admin-stack` | ```github.com/cloudposse-terraform-components/aws-spacelift-admin-stack//src?ref={{.Version}}``` | | `spacelift/spaces` | ```github.com/cloudposse-terraform-components/aws-spacelift-spaces//src?ref={{.Version}}``` | | `spacelift/worker-pool` | ```github.com/cloudposse-terraform-components/aws-spacelift-worker-pool-asg//src?ref={{.Version}}``` | #### Transit Gateway Components (TGW) 1. Copy the `component.yaml` file from `tgw` component to `tgw/cross-region-hub-connector`, `tgw/hub` and `tgw/spoke` subcomponents dirs. 2. Remove `component.yaml` file from `tgw` component 3. For each subcomponent in `component.yaml` update `source.version` to `1.532.0` 4. For each subcomponent in `component.yaml` update `source.uri` - replace `github.com/cloudposse/terraform-aws-components.git//modules/tgw?ref={{.Version}}` with | Subcomponent | New URI | |----------------------------------|----------------------------------------------------------------------------------------------| | `tgw/cross-region-hub-connector` | ```github.com/cloudposse-terraform-components/aws-tgw-hub-connector//src?ref={{.Version}}``` | | `tgw/hub` | ```github.com/cloudposse-terraform-components/aws-tgw-hub//src?ref={{.Version}}``` | | `tgw/spoke` | ```github.com/cloudposse-terraform-components/aws-tgw-spoke//src?ref={{.Version}}``` | --- ## How to Use Atmos with Existing Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Not everyone beginning to implement Atmos is starting with a clean slate, and many users would like to put Atmos to use with their existing Terraform infrastructure. However, the SweetOps approach is highly opinionated and as a result, can be intimidating for new users to begin to apply the practices to existing infrastructure. How can we leverage Atmos to incrementally build off our existing Terraform infrastructure and begin to DRY variable configurations? ## Solution Applying Atmos to existing infrastructure can be an easy process if broken into a few parts. The following sections will outline each key concept with Atmos and the steps to apply each to your existing Terraform code. :::tip **TL;DR:** Follow the 5 steps below for a detailed explanation of each step. 1. Copy all terraform components to `components/terraform/`. See [Components](/components/) 2. Create each stack under `stacks/`. See [Stacks](/resources/legacy/fundamentals/stacks) 3. Define each component within `stacks/catalog/` and import into stacks. See [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks) 4. Copy existing Terraform variable definitions into the component defaults, catalog definitions, or stack configuration. 5. Import existing state using Atmos. See [Atmos](/resources/legacy/fundamentals/atmos) ::: ### **Part 1**: Organization The first and foremost key to success with Atmos is organization. Organization of Terraform components, variables, and environment configuration naturally lead to DRY code, easier understanding, and less room for errors. With Atmos, we organize all `components` ([Components](/components)) and variables configuration, referred to as `stacks` ([Stacks](/resources/legacy/fundamentals/stacks)), into respective folders. An example layout would look like the following: ``` infrastructure/ ├── ... ├── components/ │ ├── terraform/ │ │ ├── ec2-instance/ │ │ └── vpc/ └── stacks/ ``` #### Steps 1. Decide on a unique name to give your components. Make sure if follows our [Components](/components). 2. Create the folder in `components/terraform` with the given name of the component 3. Move all existing Terraform code to the path `components/terraform`. ### **Part 2**: Stacks Stacks are what Atmos uses to define configuration and variables for each component in any environment. Our standard convention is that each `stack` operates in a given region and environment; for example, `ue1-nonprod.yaml` defines the configuration for `nonprod` operating in the `us-east-1` region. Anything included in these stacks should be **unique** to that given region and environment or be imported by a global configuration file. Read more about [Stacks](/resources/legacy/fundamentals/stacks) ``` infrastructure/ ├── ... ├── components/ └── stacks/ ├── use1-nonprod.yaml ├── usw1-nonprod.yaml ├── use1-prod.yaml └── usw1-prod.yaml ``` #### Steps 1. Create a `yaml` file for each environment and region 2. Use the following as an example to start each stack. Only include unique configuration for the given environment and region. ``` vars: stage: nonprod terraform: vars: {} components: terraform: {} ``` ### **Part 3**: Catalog The `catalog` defines every component available to use. Each component is given a configuration file within the `catalog` where common config is set. For example, a basic configuration for a component called `ec2-instance` would look like the following: ``` components: terraform: ec2-instance: backend: s3: workspace_key_prefix: ec2-instance vars: enabled: true ``` When we want to use a `component` with a `stack`, we can import that `component` from the `catalog`. Then that given `stack` can define unique variables for that `component`. Read more about the `catalog` and how to use imports under [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks) #### Steps 1. Create the `catalog` folder under `stacks/` 2. Create a `yaml` configuration file for each component. 3. Import the `component` into any `stack` file you want the component deployed, using the path to the catalog entry. For example: ``` import: - catalog/ec2-instance ``` ### **Part 4**: DRY Components Another key concept with Atmos is to avoid all variable repetition; any non-unique variable should only be specified a single time. With Atmos, they are usually three places to define a variable: #### Component variables There are variables defined with the Terraform component in `components/terraform` and are nonspecific to your organization. Typically default values are included with `defaults.tfvars`. For example: ``` # components/terraform/ec2-instance/default.auto.tfvars enabled = false ``` #### Catalog variables Variables defined within the catalog can be specific to your organization but should be nonspecific to any region or environment. For example, a catalog configuration might look like the following: ``` # stacks/catalog/ec2-instance.yaml components: terraform: ec2-instance: backend: s3: workspace_key_prefix: ec2-instance vars: enabled: true instance_type: t3.xlarge name: example ``` #### Stack variables Stack variables include any variables defined specifically for an environment, region, or a combination of the two. Define these in the most optimal way to avoid any repetition between stacks, and use imports as necessary to pull shared configuration. In this example below, the us-east-1 sandbox stack pulls the shared configuration for us-east-1, imports the `ec2-instance` component, and defines unique variables for the `vpc` component. ``` # stacks/use1-sbx01.yaml import: - use1-globals - catalog/ec2-instance vars: stage: sbx01 terraform: vars: {} components: terraform: vpc: vars: cidr_block: 10.88.0.0/18 ``` #### Steps 1. Copy any unique variables from the Terraform component into the highest stack level possible 2. Remove any non-general variables from the Terraform component ### **Part 5**: Importing State Atmos supports all built-in [Terraform Subcommands](https://www.terraform.io/docs/cli/commands/index.html), including `import`. Once the components and stacks are properly created, running `atmos terraform import -s ` will pull the existing state for any resource. Each resource in the existing Terraform state will need to be imported into the new Atmos-created Terraform state. See the [Terraform documentation on import](https://www.terraform.io/docs/cli/import/index.html) for additional details. ## Summary At this point, all Terraform components should be defined under `components/terraform` and all variables configurations listed in `stacks/catalog`. Any deployment into a given environment and region is listed within the given `stack` with an import. For example: ``` infrastructure/ ├── ... ├── components/ │ ├── terraform/ │ │ └── ec2-instance/ └── stacks/ ├── catalog │ └── ec2-instance.yaml ├── use1-nonprod.yaml ├── usw1-nonprod.yaml ├── use1-prod.yaml └── usw1-prod.yaml ``` #### Next Steps Now that your environment is properly configured, follow the [Atmos](/resources/legacy/fundamentals/atmos) documentation for running commands. --- ## How to use Atmos import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Getting Started To use [Atmos](https://atmos.tools) we recommend interacting with the `atmos` command via `geodesic`. :::info Run all the following commands in the geodesic container from the Project root. e.g the _infrastructure_ repository. To run the geodesic shell run `make run` from the project root. ::: The following assumes `` is a folder that exists in `/components/terraform/`that contains .tf files to be applied. e.g. `ecr`, `dns-primary`, `accounts` are examples of **components**. And that a `` is the name of a stack that exists in ` -s ``` #### terraform apply ``` atmos terraform apply -s ``` #### terraform destroy ``` atmos terraform destroy -s ``` #### terraform force-unlock ``` cd components/terraform/ terraform force-unlock -force cd ../../../ ``` #### terraform apply -auto-approve This command is used for planning and applying in the same command. ``` atmos terraform deploy -s ``` ## Related Guides - [How to Upgrade Atmos](/learn/maintenance/upgrades/how-to-upgrade-atmos) - [How to Upgrade or Install Versions of Terraform](/learn/maintenance/upgrades/how-to-upgrade-or-install-versions-of-terraform) - [How to Manage Terraform Dependencies in Micro-service Repositories](/learn/maintenance/tutorials/how-to-manage-terraform-dependencies-in-micro-service-repositori) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to Use Terraform Remote State](/learn/maintenance/tutorials/how-to-use-terraform-remote-state) - [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [How to support GovCloud and Other AWS Partitions with Terraform](/learn/maintenance/tutorials/how-to-support-govcloud-and-other-aws-partitions-with-terraform) - [How to terraform non-AWS infrastructure?](/learn/maintenance/tutorials/how-to-terraform-non-aws-infrastructure) - [How to use Atmos](/learn/maintenance/tutorials/how-to-use-atmos) --- ## How to Use Imports and Catalogs in Stacks import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem As infrastructure grows, we end up with hundreds or thousands of settings for components and stack configurations. If we copy and paste these settings everywhere, it’s error-prone and not DRY. What we really want to do is to define a sane set of defaults and override those defaults when we need them to change. ## Solution :::tip Use the `import` together with the `catalog` convention to write DRY configurations you can reuse any number of times. ::: Using a component catalog, you can define the default values for all instances of a component across your stacks. It is also important to note that you can provide default values for multiple instances of a component in a single stack using the component inheritance pattern via the `component` attribute. ```yaml # stacks/catalog/s3/bucket.yaml components: terraform: s3-bucket: vars: # By convention, leave the component disabled in the catalog configuration (you can enable each inherited components individually) enabled: false user_enabled: false acl: private grants: null versioning_enabled: true ``` :::info The `import` works by deep merging all files in the order they are defined using the [Mergo](https://github.com/imdario/mergo) library. ::: Then inside of a stack configuration (`uw2-dev` for example), we can `import` the `catalog/s3/bucket` component configuration into our stack and use the `s3-bucket` component defined therein. ```yaml # stacks/uw2-dev.yaml import: - catalog/s3/bucket components: terraform: public-images: component: s3-bucket # Inherit all the settings from the `s3-bucket` component vars: enabled: true # Enable the component acl: public # Override the ACL and make this a public bucket name: public-images # Give it a unique name private-artifacts: component: s3-bucket # Inherit all the settings from the `s3-bucket` component vars: enabled: true # Enable the component name: private-artifacts # Give it a unique name # ... ``` In the above example, we’re able to utilize the default settings provided via the `s3-bucket` base component from the catalog, while also creating multiple instances of the same component and providing our own overrides. This enables maximum reuse of global component configuration with minimal configuration. --- ## How to Use Terraform Remote State import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem One of the well-established best-practices for managing large-scale infrastructures with terraform is to break the state into multiple smaller “root modules” each having their own terraform state and independent SDLC, what we call [Components](/components). But the problem then arises that you cannot directly access the properties of a resource that was provisioned (e.g. the `vpc_id`). One option is just to copy and paste 🍝 the settings from component to component, but this is error-prone and far from DRY. To solve this, Terraform has the concept of “remote state” which can be accessed using the `terraform_remote_state` [data source](https://www.terraform.io/docs/language/state/remote-state-data.html). All a root module needs to do is add an `output { ... }` for every thing it wants to export. However, knowing how to access the remote state in S3 requires a deep understanding of the [Structure of Terraform S3 State Backend Bucket](/layers/accounts/tutorials/terraform-s3-state) used by terraform to manage state. It’s easy to get it wrong, and the consequences of referencing the values from the wrong state can be catastrophic. ## Solution Cloud Posse maintains a “remote state” module that is 💯 aware of all [Stacks](/resources/legacy/fundamentals/stacks) and is, therefore, able to abstract all this complexity down to a simple module invocation. Using this module, it’s almost as convenient as using the native resources directly. :::tip Use the Cloud Posse `remote-state` module to seamlessly access the remote state of any component from any stack in any region or account. ::: The `remote-state` submodule of the `terraform-yaml-stack-config` module accepts a `component` and a `stack` name and returns remote state outputs for the component. The module supports `s3`, `remote` (Terraform Cloud), and `static` backends. ### Usage The following example passes a `component` config and returns remote state outputs from the `s3` backend for the `vpc` Terraform component in the current stack. Passing the `stack` argument, we could request the outputs from any other stack. :::info Our convention is to put all usage of `remote-state` into a single file called `remote-state.tf` so it’s clear what are the external dependencies of the module ::: ```hcl # remote-state.tf module "vpc" { source = "cloudposse/stack-config/yaml//modules/remote-state" version = "0.21.1" component = "vpc" stack = "" context = module.this.context } ``` ### State Backend Configuration The backend type (`s3`) and backend configuration for the components are defined in the stack YAML config files. A global backend configuration is typically defined in the `globals.yaml` file and then overridden in each components' catalog configuration to define the `workspace_key_prefix`. ```yaml # globals.yaml terraform: backend_type: s3 # s3, remote, static, vault, etc. backend: s3: encrypt: true bucket: "acme-mgmt-uw2-root-tfstate" key: "terraform.tfstate" dynamodb_table: "acme-mgmt-uw2-root-tfstate-lock" profile: "acme-mgmt-gbl-root-terraform" acl: "bucket-owner-full-control" region: "us-west-2" remote: {} vault: {} static: {} ``` ## References - [https://www.hashicorp.com/reference/evolving-infrastructure-terraform-opencredo](https://www.hashicorp.com/reference/evolving-infrastructure-terraform-opencredo) - [https://www.terraform.io/docs/language/state/remote-state-data.html](https://www.terraform.io/docs/language/state/remote-state-data.html) - [https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state) --- ## How to Version Pin Components in Stack Configurations import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem One of the common best practices in Infrastructure as Code is to use strict version pinning. But before we get into the details on how to do this, let’s note that in all of our components and modules, we refer to a module by specific version (e.g. `1.2.3`) or some semver expression (e.g. `~> 1.2.0`). However, in most of our stack configs, you’ll generally see that all stacks refer to the same exact version of a component (e.g. the same exact directory). Doesn’t this violate the principle of version pinning? Not exactly. Rather than pin each environment to a specific version, we’re saying we want to pin all environments to the same version (commit SHA), and instead, release to environments at the cadence we want. The driver behind this is we want all environments to converge on the same version and we want our tools (e.g. Spacelift) to tell us when they diverge. If on the other hand, we were version pinning each environment, then we would need to devise some other strategy to ensure that environments do not diverge. With that said, there are still some situations where we want to forcibly diverge environments for an extended period of time. For example, perhaps some environment will be sunset in the near future, and keeping it current is not worth the effort. In this case, we might want to diverge so we can release hotfixes without needing to upgrade. ## Solution :::tip Simply version the directories of components by version and use the component inheritance documented in [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks). ::: 1. Version the component you want to diverge, e.g. rename `components/terraform/rds` to `components/terraform/rds@1.2.3` 2. Create a new version of the component, e.g. `components/terraform/rds@2.0.0`; we strongly recommend trying to keep an identical module interface (e.g. variable and outputs) between versions to reduce upgrade pains 3. Update the stack configuration for the environments to pin to a specific version of the component: ``` components: terraform: rds: # Keep our component name as `rds` so we can eventually upgrade to `2.0.0` and maintain state component: rds@1.2.3 # Use the component from `components/terraform/rds@1.2.3` vars: enabled: true # Enable the component ``` What we like about this is it makes it exceptionally clear that there are now 2 (or more) versions of the same component. Also, it shows that you're incurring tech debt by taking on maintenance two similar but diverging components. Thus our recommendation is to diverge for only short periods of time. :::info There’s nothing special about the `rds@1.2.3` component name. You could use `rds-1.2.3` or `rds_1`, etc. ::: Other hacks like Git submodules can be used to accomplish a similar outcome. We just cannot stress enough that practicing version pinning of components by the environment will lead to serious drift and offload the burden to keep the environments updated to engineers, rather than automation. ## Related - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) --- ## How to write ADRs import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Note from '@site/src/components/Note'; Architectural Design Records (ADRs) are how we capture the decision from our customers and record it in the customer's repository. Once a Jira issue is moved to _Pending Review_ and the decision documented in the issue, then it’s time to record the decision by opening a PR against the customer’s infrastructure repository. Once the PR is merged, then the issue is considered _Done._ ## Process 1. Start a new branch for the ADR **(Open one PR per ADR)** 2. Create a new ADR (typically in the `docs/adr` folder). - The filename should begin with the index number (e.g. `0005` would be the 5th ADR in the directory), followed by a short description. - The short description is _ideally_ formatted as the decision, however, that’s frequently too long. In that case, summarize the decision title. 3. Record the decision and the consequences and map them to the corresponding fields in the ADR template 4. Commit the changes, open the PR. 5. Add a link to the PR in the Jira issue to document the work product 6. Request customer and cloud posse team to review the PR 7. Once the PR is merged, move the corresponding _Design Decision_ issue to _Done_. 8. Ensure that corresponding implementation tasks exist and are linked to the _Design Decision_ task. :::caution If the ADR outcome requires some work to be performed or implemented, make sure that the corresponding ticket exists. We do not always have a matching implementation task for each design decision predefined in our reference architecture. ::: ## Related Documents - [How to Document a New Design Decision](/learn/maintenance/tutorials/how-to-document-a-new-design-decision) ## FAQ ### How to generate `docs/adr/README.md` index for ADRs? We recommend using `adr-tools` to generate the `README.md` Install `adr-tools` on OSX with Homebrew: ``` brew install adr-tools echo "docs/adr" > .adr-dir adr generate toc > docs/adr/README.md ``` If using `prettier` in `.pre-commit.yaml` then add the toc to the file exclusion ``` files: ^(?!(components\/terraform|docs\/adr)\/.*README).*md$ ``` ### What is the overall process for making and documenting a Design Decision? 1. On our weekly calls review the Design Decisions in the **BACKLOG** 2. Make a decision and document it in the Jira ticket’s description. Once it’s decided, move it to **READY TO IMPLEMENT** 3. Open a Pull Request with the formalized ADR and comment on the Jira with a link to the Pull Request, then move Jira to **PENDING REVIEW** 4. Once the Pull Request for the ADR is approved and merged, move it to **DONE** ### What is the Source of Truth The source of truth for ADRs is primarily the corresponding Design Decisions. We can also borrow from our ADRs to our repo [https://github.com/cloudposse/adr](https://github.com/cloudposse/adr) and the Design Decisions context from our Reference Architecture. ### Why do we use a sequential numbering system? The sequential numbering system is to clearly depicts the order in which the ADRs were reached. Keep in mind that decisions can be reverted in subsequent ADRs. when many people are contributing ADRs at the same time, the indexes might end up conflicting. See RFC below. ### Why don’t we use ticket numbers as the index? Ticket numbers do not necessarily connote the order of decisions. We generate tickets programmatically from our Reference Architecture and the ticket numbers are based on that. The order in which we decide issues has nothing to do with the order we created them, but if we used ticket numbers, the order would be of creation and not decisions. Note, we’re proposing a change to this: [Proposed: Use ISO-8601 Date Index for ADRs](/resources/adrs/proposed/proposed-use-iso-8601-date-index-for-adrs) ## RFC Below are some comments from others on how we can improve the process. :::caution **IMPORTANT**: These are not yet decided. Do not incorporate these into ADRs today. ::: ### Suggestions & Considerations - **Status [Dropdown]** - A decision may be "proposed" if the project stakeholders haven't agreed with it yet, or "accepted" once it is agreed. If a later ADR changes or reverses a decision, it may be marked as "deprecated" or "superseded" with a reference to its replacement. - **Context [Text Field]** - This section describes the forces at play, including technological, political, social, and project local. These forces are probably in tension, and should be called out as such. The language in this section is value-neutral. It is simply describing facts. - **Decision [Text Field]** - This section describes our response to these forces. It is stated in full sentences, with active voice. "We will ..." - **Consequences [Text Field]** - This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and project in the future. - [Proposed: Use ISO-8601 Date Index for ADRs](/resources/adrs/proposed/proposed-use-iso-8601-date-index-for-adrs) Use date-based indexes (e.g. `2021-09-24-decided-to-use-dates-in-adrs.md`) instead of sequentially incrementing indexes. The combination of date and title should never conflict (even when creating ADR that supersede previous decisions in ADR) and it also gives a better idea of when decisions were made. @Steven Hopkins - Avoid the use the ADR number in the markdown title. as it's a pain when we need to resolve number sequencing issues when we have to update filename and title. - We discussed opening a PR for each Design Decision during project kickoffs, I see this causing issues for others working in the same repo and wanting to add ADR, they would need to search through all open PR for ADR to find the correct number. - Dictate how long an ADR can be subject to changes after being merged to the main branch @Yonatan Koren - Are ADRs immutable and need to be supplanted by another ADR as soon as they’re merged to the main branch? - Is there a period when they are still “warm” or “wet”? i.e. one month? one sprint? anytime before the technology in scope of the ADR is used in production? --- ## Tutorials(12) import DocCardList from '@theme/DocCardList'; import Intro from '@site/src/components/Intro'; Tutorials are step-by-step guides that help you learn how to use the reference architecture. --- ## How to Keep Everything Up to Date import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem **DRAFT** Your infrastructure is composed of dozens, if not hundreds, of components and applications. Their configurations span multiple tools (`docker`, `helm`, `packages`) and configuration file formats (e.g. `Dockerfile`, `stacks/catalog/cert-manager.yaml` and `default.auto.tfvars`). Industry best practice is to use strict version pinning in order to maintain stable infrastructure. However, this shifts the burden of keeping everything current onto the developer. So, we need some way to automate the update process, in order to reduce the developer’s burden and, as a result, reduce long-term tech debt by staying current. ## Solution :::tip Use [Renovatebot](https://docs.renovatebot.com/) to automatically open Pull Requests when new versions of dependencies are available. Rennovatebot is a free and fully hosted solution that is much more powerful than [https://github.com/dependabot](https://github.com/dependabot) by GitHub. ::: First, install the `renovate.json` config file into your repo: TODO Second, add `# renovate` annotations to your code as detailed below in the `Examples` section. Although annotations may not be necessary for some dependencies (our `renovate.json` file does try to identify common dependency patterns, even in the absence of an annotation), we do recommend that you annotate where possible. For conventions on keeping packages from specific data sources (e.g., Helm releases, or Docker images) updated, please see the `Data Sources` section below. For details on pinning package versions to a pre-specified range, see the `Version Pinning` section below. ## Examples ### Dockerfile Automatically update base images in Dockerfiles. ``` # renovate: datasource=docker depName=cloudposse/geodesic registryUrl=public.ecr.aws versioning=docker ARG GEODESIC_VERSION=0.146.5 ARG OS=debian FROM public.ecr.aws/cloudposse/geodesic:$GEODESIC_VERSION-$OS ``` Automatically update `ARG` and `ENV` statements for package pinning in Dockerfiles. ```dockerfile # renovate: datasource=github-releases depName=cloudposse/atmos versioning="regex:(?0)\\.(?21)\\.(?\\d+)" ARG ATMOS_VERSION=1.51.0 # renovate: datasource=github-releases depName=cloudposse/atmos ARG TERRAFORM_VERSION=1.0.4 # renovate: datasource=docker depName=cloudposse/awesome-image versioning=docker registryUrl=docker.io/images ENV DOCKER_VERSION=19.03.1 # renovate: datasource=helm depName=echo-server registryUrl=https://helm.sh/charts ENV ECHO_SERVER_VERSION=1.19.1 ``` Automatically update packages based on `cargo` versioning scheme. [https://docs.renovatebot.com/modules/versioning/#cargo-versioning](https://docs.renovatebot.com/modules/versioning/#cargo-versioning) ```dockerfile # Install terraform # Install specific versions of Terraform. Must match versions in Spacelift terraform_version_map # in components/terraform/spacelift/default.auto.tfvars # renovate: datasource=github-releases depName=hashicorp/terraform versioning="regex:v?(?0)\\.(?12)\\.(?\\d+)" ARG TF_12_VERSION=0.12.30 # renovate: datasource=github-releases depName=hashicorp/terraform cargo=">=0.13.0 <0.14.0" ARG TF_13_VERSION=0.13.6 # renovate: datasource=github-releases depName=hashicorp/terraform cargo=">=0.14.0 <0.15.0" ARG TF_14_VERSION=0.14.7 # renovate: datasource=github-releases depName=hashicorp/terraform cargo=">=0.15.0 <0.16.0" ARG TF_15_VERSION=0.15.4 # renovate: datasource=github-releases depName=hashicorp/terraform cargo=">=1.0.0 <1.1.0" ARG TF_1_0_VERSION=1.0.2 RUN apt-get update && apt-get install -y -u --allow-downgrades \ terraform-0.12="${TF_12_VERSION}-*" terraform-0.13="${TF_13_VERSION}-*" \ terraform-0.14="${TF_14_VERSION}-*" terraform-0.15="${TF_15_VERSION}-*" \ terraform-1="${TF_1_0_VERSION}-*" ``` ### Terraform `.tfvars` Automatically update helm charts in terraform var files. ```hcl enabled = true name = "datadog" description = "Datadog Kubernetes Agent" kubernetes_namespace = "monitoring" create_namespace = true repository = "https://helm.datadoghq.com" chart = "datadog" # renovate: datasource=helm depName=datadog registryUrl=https://helm.datadoghq.com chart_version = "2.22.10" timeout = 180 wait = true atomic = true cleanup_on_fail = true ``` ### Stack Configurations Automatically update docker images. ```yaml components: terraform: metrics-server: settings: spacelift: workspace_enabled: true vars: enabled: true values: image: repository: "k8s.gcr.io/metrics-server/metrics-server" # renovate: datasource=docker depName=metrics-server/metrics-server versioning=docker registryUrl=k8s.gcr.io tag: "v0.3.7" pullPolicy: IfNotPresent ``` Automatically update the `cluster_kubernetes_version` ```yaml components: terraform: eks: settings: spacelift: workspace_enabled: true vars: # renovate: datasource=github-releases depName=kubernetes/kubernetes cluster_kubernetes_version: "1.20" node_groups: main: # values of `null` will be replaced with default values # availability_zones = null will create 1 auto scaling group in each availability zone of region availability_zones: null desired_group_size: 3 # number of instances to start ``` ## Data Sources Supported conventions: ### GitHub Releases ```yaml # renovate: datasource=github-releases depName=cloudposse/atmos cargo=">=1.2.3 <2.0.0" ``` ### Helm Releases ```yaml # renovate: datasource=github-releases depName=cloudposse/atmos cargo=">=1.2.3 <2.0.0" ``` ### Docker Tags ```yaml # renovate: datasource=github-releases depName=cloudposse/atmos cargo=">=1.2.3 <2.0.0" ``` ### Terraform Modules (terraform modules are handled automatically without our rules) ### Terraform Components :::caution Terraform components cannot be updated at this time using this strategy; however, the terraform modules referenced in your components will be automatically updated. ::: ### GitHub Actions [https://docs.renovatebot.com/modules/manager/github-actions/](https://docs.renovatebot.com/modules/manager/github-actions/) ## Version Pinning To pin a package to a given version range, simply add a field of the form `version="regex:v?(?\\d+)\\.(?\\d+)\\.(?\\d+)"` to that package’s renovate annotation, specifying the desired major/minor versions. For example, to pin a package to the `1.x.x` release range, you would use `version="regex:v?(?1)\\.(?\\d+)\\.(?\\d+)"`; similarly, to pin a package to the `2.1` release range, you would use `version="regex:v?(?2)\\.(?1)\\.(?\\d+)"`. --- ## How to Move `atmos-gitops-config.yaml` to `atmos.yaml` import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem We want to reduce config files related to GitOps in the `infrastructure` repository and move the `atmos-gitops-config.yaml` file to `atmos.yaml`. ## Solution :::tip TL;DR Create the changes according to the [Move atmos-gitops-config.yaml to atmos.yaml](https://github.com/cloudposse-examples/infra-demo-gitops/pull/6) PR. Set [GitHub action variables](https://github.com/cloudposse-examples/infra-demo-gitops/settings/variables/actions) `ATMOS_VERSION` and `ATMOS_CONFIG_PATH` | variable name | value | | ------------------- | ------------------------------- | | `ATMOS_VERSION` | `1.63.0` | | `ATMOS_CONFIG_PATH` | `./rootfs/usr/local/etc/atmos/` | ::: ### Updating Create changes following the [Move atmos-gitops-config.yaml to atmos.yaml](https://github.com/cloudposse-examples/infra-demo-gitops/pull/6) PR as an example. The following configuration fields now moved to [GitHub action variables](https://github.com/cloudposse-examples/infra-demo-gitops/settings/variables/actions) `ATMOS_VERSION` and `ATMOS_CONFIG_PATH` | name | variable name | value | | ------------------- | ------------------- | ------------------------------- | | `atmos-version` | `ATMOS_VERSION` | `1.63.0` | | `atmos-config-path` | `ATMOS_CONFIG_PATH` | `./rootfs/usr/local/etc/atmos/` | The following configuration fields moved to the `atmos.yaml` configuration file. | name | YAML path in `atmos.yaml` | | ------------------------ | ---------------------------------------------------- | | `aws-region` | `integrations.github.gitops.artifact-storage.region` | | `terraform-state-bucket` | `integrations.github.gitops.artifact-storage.bucket` | | `terraform-state-table` | `integrations.github.gitops.artifact-storage.table` | | `terraform-state-role` | `integrations.github.gitops.artifact-storage.role` | | `terraform-plan-role` | `integrations.github.gitops.role.plan` | | `terraform-apply-role` | `integrations.github.gitops.role.apply` | | `terraform-version` | `integrations.github.gitops.terraform-version` | | `enable-infracost` | `integrations.github.gitops.infracost-enabled` | | `sort-by` | `integrations.github.gitops.matrix.sort-by` | | `group-by` | `integrations.github.gitops.matrix.group-by` | For example, to migrate to the new version, you should have something similar to the following in your `atmos.yaml`: `./rootfs/usr/local/etc/atmos/atmos.yaml` ```yaml # ... your existing configuration integrations: github: gitops: terraform-version: 1.4.5 infracost-enabled: False artifact-storage: region: us-east-1 bucket: acme-core-use1-auto-gitops table: acme-core-use1-auto-gitops-plan-storage role: arn:aws:iam::111111111111:role/acme-core-use1-auto-gha-iam-gitops-gha role: plan: arn:aws:iam::222222222222:role/acme-core-gbl-identity-gitops apply: arn:aws:iam::222222222222:role/acme-core-gbl-identity-gitops matrix: sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` This corresponds to the previous configuration `./.github/config/atmos-gitops.yaml` file (deprecated) like this: ```yaml atmos-version: 1.63.0 atmos-config-path: ./rootfs/usr/local/etc/atmos/ terraform-state-bucket: acme-core-use1-auto-gitops terraform-state-table: acme-core-use1-auto-gitops-plan-storage terraform-state-role: arn:aws:iam::111111111111:role/acme-core-use1-auto-gha-iam-gitops-gha terraform-plan-role: arn:aws:iam::222222222222:role/acme-core-gbl-identity-gitops terraform-apply-role: arn:aws:iam::222222222222:role/acme-core-gbl-identity-gitops terraform-version: 1.4.5 aws-region: us-east-1 enable-infracost: False sort-by: .stack_slug group-by: .stack_slug | split("-") | [.[0], .[2]] | join("-") ``` --- ## How to Upgrade Atmos import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Note from '@site/src/components/Note'; import Steps from '@site/src/components/Steps'; ## Problem You are running an out-of-date version of atmos and need to upgrade it to leverage some new functionality. ## Solution Update the `ATMOS_VERSION` in the [Geodesic](/resources/legacy/fundamentals/geodesic) `Dockerfile` of your `infrastructure` repository. ### Upgrading from 0.x.x → 1.x :::note To reduce the number of variables in case something breaks, please make sure all spacelift stacks are in a confirmed state. ::: :::note Terraform workspaces no longer are tied to root stack names and instead are created from context variables. For the rare scenarios where root stacks do not correlate to their pre-existing terraform workspaces and backwards compatibility is needed, `metadata.terraform_workspace` (1.3.20+) yaml key would need to be set to the name of the file (without .yaml) for each component in that root stack to prevent a destroy and recreate on each resource. ::: :::caution The administrative stack must have its runner image set to custom Geodesic image built by the monorepo and not the default Spacelift runner image. This is because upgrading to the latest `cloud-infrastructure-automation` module introduces the Cloud Posse’s `utils` provider, which requires `atmos.yaml`. ::: 1. Update the `ATMOS_VERSION` in the [Geodesic](/resources/legacy/fundamentals/geodesic) `Dockerfile` of your `infrastructure` repository (see “Atmos Versions”) 2. Remove `variant` installation from the Dockerfile 3. Add `atmos.yaml` to `rootfs/usr/local/etc/atmos/atmos.yaml` - See [Atmos](/resources/legacy/fundamentals/atmos) for example. 4. In `rootfs/usr/local/etc/atmos/atmos.yaml` file, review the following config: `excluded_paths` is a list of path globs that are NOT top-level stacks and should be excluded from searching by `atmos` (note that those stack config files are usually imported into top-level stacks). If, for example, the `catalog` folder is not used, but some other folder(s) are used to keep the imported files, remove the `"catalog/**/*"` item from the list and add the other folder(s) to exclude them. ```yaml excluded_paths: - "globals/**/*" - "catalog/**/*" - "**/*globals*" ``` 5. Review all other settings in `rootfs/usr/local/etc/atmos/atmos.yaml`. For example, if `tenant` is not in use, modify `helm_aws_profile_pattern`, `cluster_name_pattern` and `name_pattern` settings. 6. Upgrade [spacelift automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) module to latest version in the `spacelift` component - Note that the version below shows 0.45.0 but this may not be the latest version - Remove the variable `var.stack_config_path` from `components/terraform/spacelift/variables.tf` - Remove the following two fields from the Spacelift module invocation and remove `locals` ```hcl stacks = local.stacks stack_config_path = var.stack_config_path locals { config_filenames = fileset(var.stack_config_path, "*.yaml") stack_config_files = [for f in local.config_filenames : f if(replace(f, "globals", "") == f)] stacks = [for f in local.stack_config_files : trimsuffix(basename(f), ".yaml")] } ``` 7. Update the file `components/terraform/spacelift/context.tf` to the latest version (take it from [https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf](https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf)) 8. Update the file `.spacelift/config.yml` with the following content ```yaml version: "1" stack_defaults: before_init: - spacelift-write-vars - spacelift-tf-workspace before_plan: [] before_apply: [] environment: AWS_SDK_LOAD_CONFIG: true AWS_CONFIG_FILE: /etc/aws-config/aws-config-cicd # replace with your namespace and remove this comment AWS_PROFILE: -gbl-identity ATMOS_BASE_PATH: /mnt/workspace/source stacks: infrastructure: before_init: [] before_plan: [] before_apply: [] ``` 9. Update the file `rootfs/usr/local/bin/spacelift-tf-workspace` with the following content ```bash #!/bin/bash # Add -x for troubleshooting set -ex -o pipefail terraform init -reconfigure echo "Selecting Terraform workspace..." echo "...with AWS_PROFILE=$AWS_PROFILE" echo "...with AWS_CONFIG_FILE=$AWS_CONFIG_FILE" atmos terraform workspace "$ATMOS_COMPONENT" --stack="$ATMOS_STACK" # Remove -x for security set -e +x ``` 10. Update the file `rootfs/usr/local/bin/spacelift-write-vars` to look like the following ```bash #!/bin/bash # Add -x for troubleshooting set -ex -o pipefail function main() { if [[ -z $ATMOS_STACK ]] || [[ -z $ATMOS_COMPONENT ]]; then echo "Missing required environment variable" >&2 echo " ATMOS_STACK=$ATMOS_STACK" >&2 echo " ATMOS_COMPONENT=$ATMOS_COMPONENT" >&2 return 3 fi echo "Writing Stack variables to spacelift.auto.tfvars.json for Spacelift..." atmos terraform write varfile "$ATMOS_COMPONENT" \ --stack="$ATMOS_STACK" -f spacelift.auto.tfvars.json > /dev/null jq . remote-state-files cat remote-state-files | \ while read remotestatefile; do \ hcledit block list --file $remotestatefile | \ while read block; do \ echo "hcledit attribute set $block.version '\"0.22.3\"' --file $remotestatefile -u"; \ echo "hcledit attribute rm $block.stack_config_local_path --file $remotestatefile -u"; \ done; done > update-remotestate.sh chmod +x update-remotestate.sh ./update-remotestate.sh ``` 13. Bump Geodesic to the latest like `1.2.1`. This is so the `ATMOS_BASE_PATH` env var no longer needs to be set 14. Run `make all` to build a new container 15. Change directory into the root path of your repository within `/localhost` 16. Set the `export ATMOS_BASE_PATH=$(pwd)`env var if it’s not already set 17. Run `atmos terraform plan vpc --stack ` and other atmos commands as a test 18. Open a pull request with the changes described above 19. After approval, merge the PR into the default branch 20. Rebuild the Docker image and push the latest version to the Docker registry used by Spacelift (usually ECR) with the tag `latest`. This is required for the changes to the Spacelift files in `rootfs/usr/local/bin/` to take effect, and is usually done by automation like calling a GitHub Actions or executing a Codefresh pipeline. 21. Confirm that the `infrastructure` stack and components' stacks are operational in Spacelift ### Troubleshooting #### Failed local runs ```console ╷ │ Error: │ No stack config files found in the provided paths: │ - /localhost/git/infrastructure/components/terraform/compliance/stacks/**/* ``` This means that the `ATMOS_BASE_PATH` (different for everyone, should be set to the absolute path of the repo root) is most likely unset or an older version of `atmos` is used. ```console $ atmos version 1.3.17 $ echo $ATMOS_BASE_PATH /localhost/git/infrastructure/ ``` If it still fails, try running an `atmos terraform clean` command to remove any stale data like a stale `.terraform.lock.hcl` file and rerun. #### Failed Spacelift runs Any stack failures may be due to stacks pinned to a specific old hash even if it points to the default branch. Each failed stack should point to the latest HEAD because the latest will contain the `ATMOS_BASE_PATH` env var set in the `.spacelift/config`. To do this, for each failed stack 1. Navigate to the stack 2. Go to stack settings 3. Change branch to something else 4. Change it back to the original branch (like `main` or whatever the default branch is) 5. Retrigger and it should show up as “Unconfirmed” or “No changes” as expected. ## Atmos Versions [https://github.com/cloudposse/atmos/releases](https://github.com/cloudposse/atmos/releases) ## Remote state module Versions [https://github.com/cloudposse/terraform-yaml-stack-config/releases](https://github.com/cloudposse/terraform-yaml-stack-config/releases) ## Open Issues [https://github.com/cloudposse/atmos/issues](https://github.com/cloudposse/atmos/issues) --- ## How to Upgrade EKS Cluster Addons import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem EKS clusters support “Addons” that can be automatically installed on a cluster, but they also need to be kept up to date. AWS officially supports 3 cluster add-ons and they are not automatically updated when updating the EKS cluster: - Amazon VPC CNI - CoreDNS - Kube-Proxy See [https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html](https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html), for more. ## Solution :::tip Ensure all `addons` for the component use an `addon_version` that corresponds to the `cluster_kubernetes_version` using the `aws eks describe-addon-versions` command. ::: :::caution - Requires EKS Cluster Version **1.18** or greater. See [How to Upgrade EKS](/learn/maintenance/upgrades/how-to-upgrade-eks) On Upgrading your cluster! - Requires `terraform-aws-eks-cluster` module version **0.43.0** or greater. [https://github.com/cloudposse/terraform-aws-eks-cluster](https://github.com/cloudposse/terraform-aws-eks-cluster) which adds supports for `addons` - Requires [eks/cluster](/components/library/aws/eks/cluster/)) component which maps `addons` to `var.addons` ::: Adding the `addons` variable will enable the addons to your cluster. However, determining the `addon_version` can be difficult. It can be done through ClickOps, or through CLI. When running `aws eks describe-addon-versions` make sure to specify the `--kubernetes-version` which corresponds to the `cluster_kubernetes_version` in the stack configuration. See the “Troubleshooting” section, if you run into problems. ``` # Get all Addon Versions for All Plugins for kubernetes version 1.20 aws eks describe-addon-versions --kubernetes-version 1.20 | jq # Get Latest Version for all Plugins for kubernetes version 1.20 aws eks describe-addon-versions --kubernetes-version 1.20 | jq '.addons[] | "\(.addonName) \(.addonVersions[0].addonVersion)"' ``` Unfortunately, at the time of this writing, it does not appear that there’s any `data` source in `terraform` to retrieve the latest version of addons. The `eks_addons` [data source](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_addon) will only return information about the addons currently installed in an EKS cluster. Note that the addon version may or may not include the kubernetes version. E.g. `kube-proxy` versions their releases according to kubernetes releases, but appends the `eksbuild.2` which will vary. The `vpc-cni`, however, has it’s own versioning cadence. ```yaml components: terraform: eks: vars: cluster_kubernetes_version: "1.20" addons: - addon_name: "kube-proxy" addon_version: "v1.20.4-eksbuild.2" resolve_conflicts: "NONE" service_account_role_arn: null - addon_name: "vpc-cni" addon_version: "v1.9.1-eksbuild.1" resolve_conflicts: "NONE" service_account_role_arn: null - addon_name: "coredns" addon_version: "v1.8.3-eksbuild.1" resolve_conflicts: "NONE" service_account_role_arn: null ``` ## Troubleshooting ### Error: `unexpected EKS Add-On (eg-uw2-tools-eks-cluster:kube-proxy) state returned during creation: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'` ``` │ Error: unexpected EKS Add-On (eg-uw2-tools-eks-cluster:kube-proxy) state returned during creation: unexpected state 'CREATE_FAILED', wanted target 'ACTIVE'. last error: 1 error occurred: │ * : ConfigurationConflict: Apply failed with 1 conflict: conflict with "before-first-apply" using v1: .data.config ``` :::tip The AWS API will run a first time script! This allows the addons to be created but will **FAIL** the first attempt through _spacelift_. If this happens Retry. ::: :::note This leaves the cluster addons in a `Create Failed` State, which does not appear to impact services ::: --- ## How to Upgrade EKS import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem EKS Clusters need to be upgraded to stay up to date with the newest features. AWS only supports the latest 3 major releases, so staying up to date is important. Upgrading may require not only updating the cluster, but the client tools, operators, API versions, addons, annotations, and AMIs. ## Prerequisites :::caution Before performing any kubernetes upgrade, **make sure:** - There are no breaking changes such as API deprecations that affect your deployed services. - Ensure there are no unconfirmed Spacelift runs relating to the EKS component, services running on top of EKS, or anything else EKS depends on (e.g., VPC, Global Accelerator, WAF) - Ensure all pods are healthy (e.g., not in a crash loop) before upgrading ::: A list of deprecations by version is available in the official Kubernetes [https://kubernetes.io/docs/reference/using-api/deprecation-guide/](https://kubernetes.io/docs/reference/using-api/deprecation-guide/). You can use tools like `pluto` by Fairwinds or `kube-no-trouble` [https://github.com/FairwindsOps/pluto](https://github.com/FairwindsOps/pluto) [https://github.com/doitintl/kube-no-trouble](https://github.com/doitintl/kube-no-trouble) Here’s an example of Pluto: ([See docs for more details](https://pluto.docs.fairwinds.com/quickstart/#file-detection-in-a-directory)) ``` pluto detect-helm -owide NAME NAMESPACE KIND VERSION REPLACEMENT DEPRECATED DEPRECATED IN REMOVED REMOVED IN cert-manager/cert-manager-webhook cert-manager MutatingWebhookConfiguration admissionregistration.k8s.io/v1beta1 admissionregistration.k8s.io/v1 true v1.16.0 false v1.19.0 ``` ## Solution :::tip Simply update the `cluster_kubernetes_version` and `addons`, the apply the changes with Spacelift or Atmos. ::: #### Decide on Kubernetes Version Kubernetes releases new minor versions (e.g. 1.21), as generally available approximately every three months. Each minor version is supported for approximately twelve months after its first release. Amazon adds support for the latest version very quickly after release, but typically after the first patch version release (e.g. 1.XX.**1**)**,** but sometimes later. [https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html](https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html) Also, understand [Amazon's EKS version lifecycle](https://aws.amazon.com/blogs/compute/updates-to-amazon-eks-version-lifecycle/) :::caution Note the latest Kubernetes release may not yet be available for EKS. ::: [https://github.com/kubernetes/kubernetes/releases](https://github.com/kubernetes/kubernetes/releases) #### Upgrade `kubectl` Version We need to install the version of `kubectl` that is at least one minor version away from the `cluster_kubernetes_version` in [Geodesic](/resources/legacy/fundamentals/geodesic). Installation instructions will vary depending on if you’re using Alpine, Ubuntu, or CentOS. For the most current list of `kubectl-1.x` versions, check out our `cloudposse/packages` repository [https://github.com/cloudposse/packages/tree/master/vendor](https://github.com/cloudposse/packages/tree/master/vendor). We publish special packages for `kubectl` to support installing multiple versions simultaneously. We support package pinning going back as far as `kubectl-0.13` 😵 ). You’ll want to install the version of `kubectl` by adding it to the `Dockerfile` for your infrastructure repository. :::caution The version of `kubectl` CLI needs to be within one minor version of the version of Kubernetes Version you are using. You can use `kubectl` v1.18 with Kubernetes 1.17, 1.18, and 1.19, but should not use it with 1.20 ::: | | | | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- | | `kubectl-1.13` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.13](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.13) | **Ubuntu**: `apt-get install -y kubectl-1.13`**Alpine**: `apk add kubectl-1.13@cloudposse` | | `kubectl-1.14` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.14](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.14) | **Ubuntu**: `apt-get install -y kubectl-1.14`**Alpine**: `apk add kubectl-1.14@cloudposse` | | `kubectl-1.15` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.15](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.15) | **Ubuntu**: `apt-get install -y kubectl-1.15`**Alpine**: `apk add kubectl-1.15@cloudposse` | | `kubectl-1.16` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.16](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.16) | **Ubuntu**: `apt-get install -y kubectl-1.16`**Alpine**: `apk add kubectl-1.16@cloudposse` | | `kubectl-1.17` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.17](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.17) | **Ubuntu**: `apt-get install -y kubectl-1.17`**Alpine**: `apk add kubectl-1.17@cloudposse` | | `kubectl-1.18` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.18](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.18) | **Ubuntu**: `apt-get install -y kubectl-1.18`**Alpine**: `apk add kubectl-1.18@cloudposse` | | `kubectl-1.19` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.19](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.19) | **Ubuntu**: `apt-get install -y kubectl-1.19`**Alpine**: `apk add kubectl-1.19@cloudposse` | | `kubectl-1.20` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.20](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.20) | **Ubuntu**: `apt-get install -y kubectl-1.20`**Alpine**: `apk add kubectl-1.20@cloudposse` | | `kubectl-1.21` | [https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.21](https://github.com/cloudposse/packages/tree/master/vendor/kubectl-1.21) | **Ubuntu**: `apt-get install -y kubectl-1.21`**Alpine**: `apk add kubectl-1.21@cloudposse` | #### Switch `kubectl` Versions (Local Geodesic Shell) To see Versions supported Check either the `Dockerfile`, or run `ls /usr/share/kubectl/` on your geodesic shell ``` update-alternatives --set kubectl /usr/share/kubectl/${Major.Minor}/bin/kubectl ``` #### Upgrade EKS Cluster Version :::caution You cannot upgrade an EKS cluster more than 1 minor patch level at a time. To get from 1.17 to 1.20 requires 3 separate upgrades: 1.17 → 1.18, 1.18 → 1.19, 1.19 → 1.20, 1.20 → 1.21 ::: 1. Open a Pull Request to update the EKS Component settings. 1. Update the variable `cluster_kubernetes_version` in the stack configuration 2. Update the `addons` that correspond to the `cluster_kubernetes_version` (see [How to Upgrade EKS Cluster Addons](/learn/maintenance/upgrades/how-to-upgrade-eks-cluster-addons) ) 3. Update the `Dockerfile` to ensure the version of `kubectl` installed corresponds to the `cluster_kubernetes_version` 2. Merge the Pull Request 3. Confirm the changes in Spacelift (or `atmos terraform apply` the changes). 4. Verify cluster is operational afterward by running: 1. Node Check `kubectl get nodes` 2. Pod Check `kubectl get pods --all-namespaces` 3. Check the AWS Console for EKS :::info This can take between 20-40 minutes depending on the size of the cluster! ::: ##### What to do if Spacelift Runner Crashes while updating: The spacelift worker occasionally crashes. if it does so during the long update of EKS Cluster version. Check AWS, you should find that the cluster is in the `Updating` state. If it is **do nothing**, this will allow the cluster to finish updating. Once the Cluster is updated to your targeted version you can re-trigger the stack. It is highly likely that in the event of a crash that terraform state will be locked, follow the steps below to unlock your terraform state. After unlocking your state, re-trigger the spacelift stack and confirm the changes. this should apply successfully. #### Unlocking Terraform State To unlock the state go to your **infrastructure** repository, and run ``` atmos terraform plan eks -s ``` Then `cd` into the directory of the component, such as ``` cd components/terraform/eks ``` and run ``` terraform force-unlock -force ``` #### Update EKS Node Pool AMI See [https://docs.aws.amazon.com/eks/latest/userguide/update-managed-node-group.html](https://docs.aws.amazon.com/eks/latest/userguide/update-managed-node-group.html) #### Override EKS Node Pool AMI Type See [https://docs.aws.amazon.com/eks/latest/APIReference/API_Nodegroup.html](https://docs.aws.amazon.com/eks/latest/APIReference/API_Nodegroup.html) Use `ami_type` to set a valid value. Current supported types are `AL2_x86_64 | AL2_x86_64_GPU | AL2_ARM_64` There is a [pending PR](https://github.com/cloudposse/terraform-aws-eks-node-group/pull/93) for `BOTTLEROCKET_ARM_64 | BOTTLEROCKET_x86_64` ami types #### Override EKS Node Pool AMI Custom Usually this shouldn’t need to be used since we support amazon’s `ami_type` argument but if overriding the image with a custom AMI is desired then follow the below steps. The upstream module is [https://github.com/cloudposse/terraform-aws-eks-node-group](https://github.com/cloudposse/terraform-aws-eks-node-group) 1. Set `ami_image_id` in `eks/modules/node_groups_by_az` on the `cloudposse/eks-node-group/aws` module 2. Set `after_cluster_joining_userdata = ["ls"]` so the custom AMI will need to be used ## References - [https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html](https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html) - [How to Upgrade EKS Cluster Addons](/learn/maintenance/upgrades/how-to-upgrade-eks-cluster-addons) --- ## How to Upgrade or Install Versions of Terraform import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem You want to install or upgrade to a specific version of terraform, ideally, without affecting anything that needs newer or older versions of terraform. ## Solution #### Install Desired Version of Terraform First, make sure you’ve installed the latest version of `terraform` in [Geodesic](/resources/legacy/fundamentals/geodesic). Installation instructions will vary depending on if you’re using Alpine, Ubuntu, or CentOS. We publish special [packages](https://github.com/cloudposse/packages) for terraform to support installing multiple versions simultaneously. We support package pinning going back as far as Terraform 0.11 (remember those days? 😵 ). :::info We support installing multiple versions of terraform **without** using a version manager like `tfenv`. ::: You’ll want to install the desired version of terraform by adding it to the `Dockerfile` for your infrastructure repository. E.g. ``` RUN apt-get install -y terraform-1 ``` [https://github.com/cloudposse/packages](https://github.com/cloudposse/packages) ### Terraform Releases [https://github.com/hashicorp/terraform/releases](https://github.com/hashicorp/terraform/releases) Here’s a list of our supported versions of terraform (these are updated nightly by a [github action](https://github.com/cloudposse/packages/tree/master/.github/workflows)) | | | | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | Terraform 1.x | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-1](https://github.com/cloudposse/packages/tree/master/vendor/terraform-1) | **Ubuntu**: `apt-get install -y terraform-1`**Alpine**: `apk add terraform-1@cloudposse` | | Terraform 0.15 | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.15](https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.15) | **Ubuntu**: `apt-get install -y terraform-0.15`**Alpine**: `apk add terraform-0.15@cloudposse` | | Terraform 0.14 | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.14](https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.14) | **Ubuntu**: `apt-get install -y terraform-0.14`**Alpine**: `apk add terraform-0.14@cloudposse` | | Terraform 0.13 | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.13](https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.13) | **Ubuntu**: `apt-get install -y terraform-0.13`**Alpine**: `apk add terraform-0.13@cloudposse` | | Terraform 0.12 | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.12](https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.12) | **Ubuntu**: `apt-get install -y terraform-0.12`**Alpine**: `apk add terraform-0.12@cloudposse` | | Terraform 0.11 | [https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.11](https://github.com/cloudposse/packages/tree/master/vendor/terraform-0.11) | **Ubuntu**: `apt-get install -y terraform-0.11`**Alpine**: `apk add terraform-0.11@cloudposse` | :::tip You can pin the version of `terraform` using `update-alternatives` as well as in the component configuration. ::: After installing the desired version of terraform, make sure you know [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform), and don’t forget to update the component’s configuration in the stack for both Spacelift and command line. ## Related - [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) --- ## Upgrades import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import DocCardList from '@theme/DocCardList'; Upgrading tools like Amazon EKS, Atmos, and Terraform involves updating to newer versions, which include enhancements, bug fixes, and security patches. Our upgrade guides offer detailed instructions for each step of the process, along with troubleshooting tips, best practices, and advice on testing upgrades in non-production environments to minimize impact on live systems. --- ## Onboarding as a New Developer import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import TaskList from '@site/src/components/TaskList'; import ReactPlayer from 'react-player'; This guide is intended to help new developers get up to speed with the Cloud Posse reference architecture. It covers the basics of the architecture, how to get started, and how to contribute. We assume you have a basic understanding of Terraform and AWS. We assume you are joining a team that is already using the Cloud Posse reference architecture that's been fully implemented in your AWS organization. This guide assumes you have the following: - Terraform experience working with modules, providers, public registry, state backends, etc - AWS experience including a firm understanding of general concepts, IAM, the web console, etc - Existing AWS environment utilizing Cloud Posse's reference architecture If this all sounds a little bit daunting, you may want to start by reviewing the [Learning Resources](/resources/legacy/learning-resources). If you're new to Terraform, we recommend starting with the [Terraform documentation](https://learn.hashicorp.com/collections/terraform/aws-get-started). ## Quickstart 1. Review the Toolchain 2. AWS Access 3. Setup Leapp 4. Repository Access 5. Configure Local Workstation 7. Checkout Repository 8. Build Geodesic 9. Run the Container 10. Test Atmos ## Getting Started ### Learn the Cloud Posse Toolchain
AI generated voice
- [ ] Review the Introduction to Toolset video and become familiar with the concepts of [Components](/components), [Stacks](https://atmos.tools/core-concepts/stacks/), and [Stack Catalogs](https://atmos.tools/core-concepts/stacks/catalogs) - [ ] Review [Geodesic](https://github.com/cloudposse/geodesic/) docker toolbox and [How to Customize the Geodesic Shell](/learn/maintenance/tutorials/how-to-customize-the-geodesic-shell) to your liking. - [ ] Review [atmos.tools](https://atmos.tools/) - [ ] Review our [Conventions](/learn/conventions) (there are a lot of them) - [ ] Review the Architectural Design Records (ADRs) for your organization; these are typically in `docs/adr` within the infrastructure repository - [ ] Review Cloud Posse’s [Architectural Design Records (ADRs)](/resources/adrs), which may not be the same as your organization's practices - [ ] Check out the [Learning Resources](/resources/legacy/learning-resources) that have been curated from our weekly office hours
### Review our AWS Toolchain - [ ] Review [Leapp](/layers/identity/tutorials/leapp) and [How to Use Leapp to Authenticate with AWS](/layers/identity/tutorials/leapp) - [ ] Review why we [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) ### Review our Kubernetes Toolchain (for EKS users) - [ ] [Helm](/resources/legacy/helm) - [ ] [Kubernetes](/resources/legacy/kubernetes) - [ ] [Lens](/resources/legacy/lens) ### How we Use Terraform (for developers) - [ ] Review [Terraform](/best-practices/terraform/) conventions used by Cloud Posse - [ ] We store all the terraform states in S3. Make sure you understand the [Structure of Terraform S3 State Backend Bucket](/layers/accounts/tutorials/terraform-s3-state) - [ ] Learn [How to Use Terraform Remote State](/learn/maintenance/tutorials/how-to-use-terraform-remote-state) - we make it easy to read the state of any component in any account. - [ ] Review [Component Development](/learn/component-development/). - [ ] [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [ ] [How to Use Atmos with Existing Terraform](/learn/maintenance/tutorials/how-to-use-atmos-with-existing-terraform) ### Try a Training Exercise - [ ] Clone your company’s infrastructure repository - [ ] Review the components in your company’s infrastructure repository (e.g. `infrastructure/components/terraform`) - [ ] Review the stack configurations in your company’s infrastructure repository (e.g. `infrastructure/stacks/`) - [ ] Review the catalog configurations in your company’s infrastructure repository (e.g. `infrastructure/stacks/catalog`)
## Developer Workflow 1. Start a new feature branch 2. Develop and test your changes locally in the sandbox account 3. Push your changes to the feature branch 4. Create a pull request 5. Review the pull request 6. Wait for tests to pass (or fix them) 7. Seek approval from the team 8. Merge the pull request 9. Deploy the changes --- ## Prerequisites import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import TaskList from '@site/src/components/TaskList'; Before provisioning infrastructure, make sure you have the necessary cloud access and that your local workstation has the required software installed. ## Obtain Cloud Access If you're working within an existing organization, ensure you have sufficient access to the accounts you'll be managing. If you're provisioning the infrastructure from scratch, you'll need the AWS root account credentials as part of the bootstrapping process before AWS Single Sign-On is set up. - [ ] AWS SSO Access (talk to your AWS administrator) - [ ] Ensure you can log in to AWS. See [How to Use Leapp to Authenticate with AWS](/layers/identity/tutorials/leapp) to get started. ## Setup Your Workstation Prerequisites for your local host computer: - [ ] Docker installed (to run our tools) - [ ] Slack installed (to communicate with our team) - [ ] `make` installed, preferably GNU Make - [ ] `git` installed - [ ] [Leapp](/layers/identity/tutorials/leapp) for authentication ### Homebrew Users We typically ship a `Brewfile` with a list of local dependencies. Make sure you have [Homebrew](https://brew.sh) and use `brew bundle` to install dependencies from the `Brewfile` 1. First, make sure you’ve [cloned the `infrastructure` repository](/layers/project/create-repository) for your organization. 2. Just run `brew bundle install` to install all localhost dependencies. Here's an example `Brewfile` we typically include in the infrastructure repository. ```shell brew "atmos" brew "coreutils" brew "pre-commit" brew "terraform-docs" brew "opentofu" brew "tflint" ``` 3. Next, you'll need docker set up. There are a variety of ways to do this. Review the [Docker for Desktop](https://docs.docker.com/desktop/install/mac-install/) installation guide for macOS. ### Docker on Windows ##### Setting up containers on Windows 1. [Install Docker Desktop](https://docs.docker.com/docker-for-windows/install/) or install [Rancher Desktop](https://rancherdesktop.io/) and configure it to use `docker` instead of `nerdctl`. 1. If you picked Docker Desktop, follow [instructions on WSL setup](https://docs.docker.com/desktop/wsl/), otherwise follow [Rancher's setup docs](https://docs.rancherdesktop.io/ui/preferences/wsl). ##### Using Ubuntu on Windows 1. First, [setup WSL2](https://docs.microsoft.com/en-us/windows/wsl/install-win10) and then configure WSL to use version `2` by default with this command: `wsl --set-default-version 2` 1. [Install Ubuntu for Windows](https://ubuntu.com/tutorials/install-ubuntu-on-wsl2-on-windows-10#1-overview) 1. [Install Windows terminal](https://apps.microsoft.com/store/detail/windows-terminal/9N0DX20HK701) 1. Launch Ubuntu shell from Windows Terminal 1. [Add user to Docker](https://docs.docker.com/engine/install/linux-postinstall/) 1. If you chose to use Rancher, just install `docker`. If you chose to use Docker Desktop, you'll need to go into settings and enable your Ubuntu distro. Once applied, you'll see `docker` available in your Ubuntu shell `PATH`. ##### Running Geodesic on Windows 1. You'll need an editor that supports WSL2. [VSCode](https://code.visualstudio.com/) and most JetBrains IDEs support WSL2 using remote development plugins. 1. Clone this repo into your WSL2 environment 1. Install `curl` and `build-essential` with `apt-get` 1. Launch your editor's remote development mode and open the repo 1. You should now be able to run `make all` and have it build the container and launch a shell inside it #### Docker on Linux Systems You should be able to use `docker` as normal. Launch docker before running the `Makefile`. Linux users should run Docker in ["rootless"](https://docs.docker.com/engine/security/rootless/) mode. In this mode, the Docker daemon runs as the host user (rather than as root) and files created by the root user in Geodesic are owned by the host user on the host. Not only does this configuration solve [this issue](https://github.com/cloudposse/geodesic/issues/594), but it provides much better system security overall. ## Clone Infrastructure Repository The infrastructure is where you'll find all the code to provision the reference architecture. You'll need to clone this repository to your local workstation. ```shell git clone https://github.com/$org/$repo.git ``` ## Build the Geodesic Toolbox Container Geodesic is a toolbox container that provides an interactive and consistent environment for executing infrastructure commands, similar to a devcontainer. It includes essential tools like `terraform`, `atmos`, and AWS CLI. Build this container before provisioning infrastructure. ```shell make all ``` This will build the container and launch a shell inside it. You can now start provisioning infrastructure. With your repository set up and toolbox container. Next Step --- ## Docker Tips & Tricks import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Here's a collection of some nice little hacks for docker. A lot of them are related to house keeping. ## Docker Stats Produce console stats for all running containers (e.g. like `top`): ```shell docker stats $(docker ps --format='{{.Names}}' ``` ## Remove all stopped containers ```shell docker rm $(docker ps -a -q) ``` ## Remove all untagged images ```shell docker images -q --filter "dangling=true" | xargs docker rmi ``` ## Prune everything The docker system prune command is a shortcut that prunes images, containers, and networks. :::info - In Docker `17.06.0` and earlier, volumes are also pruned. - In Docker `17.06.1` and higher, you must specify the `--volumes` flag for docker system prune to prune volumes. ::: ```shell $ docker system prune WARNING! This will remove: - all stopped containers - all networks not used by at least one container - all dangling images - all build cache Are you sure you want to continue? [y/N] y ``` If you are on Docker `17.06.1` or higher and want to also prune volumes, add the `--volumes` flag: ```shell $ docker system prune --volumes WARNING! This will remove: - all stopped containers - all networks not used by at least one container - all volumes not used by at least one container - all dangling images - all build cache Are you sure you want to continue? [y/N] y ``` By default, you are prompted to continue which can be bypassed by adding the `-f` or `--force` flag. ## Simulate Multiple Inheritance Docker doesn't technically support multiple-inheritance, whereby an image can automatically merge multiple images using `FROM`. It does, however, support [multi-stage builds](/best-practices/docker#multi-stage-builds) that can be used to effectively achieve the same result. --- ## GitHub Actions Tips & Tricks import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Related Components - [github-runners](/components/library/aws/github-runners/) ## Security [https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository) [https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions) ## Tips & Tricks ### Write Complex Steps in `github-script` Rather than try some impossible concoction of YAML, use typescript to define your CI/CD logic. [https://github.com/actions/github-script](https://github.com/actions/github-script) ### Easily Write Files in Workflows The `1arp/create-a-file-action` action is convenient for writing files. ``` - name: Create ${{ steps.vars.outputs.argocd_app_root }}/config.yaml uses: 1arp/create-a-file-action@0.2 with: path: "some/dir" file: config.yaml content: | name: "${{ github.event.deployment.payload.app }}" manifests: "${{ steps.vars.outputs.argocd_app_manifests }}" namespace: "${{ inputs.namespace }}" ``` ### Checkout and Run Private GitHub Actions :::info **GitHub Enterprise** users can now use private actions natively within the organization. [https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/](https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/) ::: ``` - name: Checkout Shared Actions uses: actions/checkout@v2 with: repository: acme/actions path: ./.github/actions token: ${{ secrets.CROSS_REPO_TOKEN }} - name: Hello World uses: ./.github/actions/hello-world id: hello-world ``` ### Use Empty Commits to Trigger Actions ``` git commit --allow-empty --message 'bump' ``` ### Use `workflow_dispatch` to Manually Trigger Workflows In this example, the workflow will trigger on `workflow_dispatch` and prompt the user to enter the required input for `delete`. Note, `delete` is just an example; the input parameters can be whatever you want. ``` on: # Enable manual runs workflow_dispatch: inputs: delete: description: 'Set to "true" to actually delete stuff' required: true default: 'false' ``` ## Known Limitations There are a lot of _non-obvious_ limitations when working with GitHub Actions. Here are the ones we’ve been bit by in developing workflows. Also, make sure to check out the [Public Roadmap](https://github.com/orgs/github/projects/4247/views/2?filterQuery=label%3Aactions) for GHA. ### Roadmap Items These are some roadmap items we’re excited to see implemented: - [https://github.com/github/roadmap/issues/52](https://github.com/github/roadmap/issues/52) - [https://github.com/github/roadmap/issues/575](https://github.com/github/roadmap/issues/575) - [https://github.com/github/roadmap/issues/199](https://github.com/github/roadmap/issues/199) - [https://github.com/github/roadmap/issues/503](https://github.com/github/roadmap/issues/503) ### General - Set in GitHub Actions running on self-hosted runner hangs indefinitely. Nothing meaningful in logs. The issue may be due to higher resource requirements for the `summerwind/actions-runner-dind` GHA runner. Increasing the resources may resolve the issue. - Not all event types will run off of branches. Learn more here [https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows](https://docs.github.com/en/actions/learn-github-actions/events-that-trigger-workflows) - A single workflow cannot build and then `use` the derived docker image. Docker image must exist before workflow is started. - Workflow approval steps are a GitHub Enterprise feature. [https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments](https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments) - GitHub Actions can only be used from public repos unless using GitHub Enterprise with an “enterprise” account to support `internal` (not “private”) repositories. To use in private repos, you need to `git clone` the repo and `use` the local path to the action. [https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/](https://github.blog/changelog/2022-01-21-share-github-actions-within-your-enterprise/) [https://github.com/github/roadmap/issues/254](https://github.com/github/roadmap/issues/254) [https://github.com/github/roadmap/issues/74](https://github.com/github/roadmap/issues/74) - GitHub Actions cron jobs automatically disable after 60 days of inactivity (no new commits) [https://docs.github.com/en/actions/managing-workflow-runs/disabling-and-enabling-a-workflow](https://docs.github.com/en/actions/managing-workflow-runs/disabling-and-enabling-a-workflow) - The `GITHUB_TOKEN` is scoped to only permit operations on the _current_ repo of the GitHub Action workflow - No ternary operator [https://github.com/actions/runner/issues/409](https://github.com/actions/runner/issues/409) - No way to restrict who can edit workflows to a subset of users with write permissions[https://github.com/actions/runner/issues/494](https://github.com/actions/runner/issues/494) ### Composite Actions - ~~Composite actions do not support~~ `if` conditional. [https://github.com/actions/runner/issues/834](https://github.com/actions/runner/issues/834) Update! Released 2021-11-09 [https://github.blog/changelog/2021-11-09-github-actions-conditional-execution-of-steps-in-actions/](https://github.blog/changelog/2021-11-09-github-actions-conditional-execution-of-steps-in-actions/) - Composite actions do not support post-run capabilities [https://github.community/t/no-post-run-capability-for-composite-actions/139046/4](https://github.community/t/no-post-run-capability-for-composite-actions/139046/4) - ~~Composite actions cannot call other composite actions~~ [https://github.com/actions/runner/issues/862](https://github.com/actions/runner/issues/862) - [https://github.com/actions/runner/issues/665](https://github.com/actions/runner/issues/665) - [https://github.com/actions/runner/issues/1348](https://github.com/actions/runner/issues/1348) ### Shared Workflows - GitHub Actions shared workflows can only be private in GitHub Enterprise using an `internal` (not `private`) repo.. [https://docs.github.com/en/actions/learn-github-actions/reusing-workflows](https://docs.github.com/en/actions/learn-github-actions/reusing-workflows) - Private reusable workflows is in the GitHub Roadmap [https://github.com/github/roadmap/issues/51](https://github.com/github/roadmap/issues/51) - Reusable workflows can't call other reusable workflows. - Reusable workflows stored within a private repository can only be used by workflows within the same repository (unless using GitHub Enterprise). - Any environment variables set in an `env` context defined at the workflow level in the caller workflow are not propagated to the called workflow. For more information about the `env` context, see "[Context and expression syntax for GitHub Actions](https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#env-context)." - You can't set the concurrency of a called workflow from the caller workflow. For more information about `jobs..concurrency`, see "[Workflow syntax for GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/workflow-syntax-for-github-actions#jobsjob_idconcurrency)." - The `strategy` property is not supported in any job that calls a reusable workflow. ## References - Helpful github actions by ReviewDog [https://github.com/reviewdog/reviewdog](https://github.com/reviewdog/reviewdog) - `deployment` documentation [https://docs.github.com/en/rest/reference/deployments](https://docs.github.com/en/rest/reference/deployments) , [https://docs.github.com/en/rest/reference/deployments#create-a-deployment](https://docs.github.com/en/rest/reference/deployments#create-a-deployment) - [GitHub Actions Public Roadmap](https://github.com/orgs/github/projects/4247/views/2?filterQuery=label%3Aactions) --- ## Terraform Tips & Tricks import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## S3 Bucket Lifecycle Rules ```hcl title="Example: S3 Bucket Lifecycle Rules" module "assets_bucket_label" { source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.3.3" namespace = "eg" stage = "dev" name = "assets" } resource "aws_s3_bucket" "assets" { bucket = "${module.assets_bucket_label.id}" tags = "${module.assets_bucket_label.tags}" acl = "private" region = "us-west-2" force_destroy = false lifecycle_rule { id = "${module.assets_bucket_label.id}" enabled = true prefix = "" tags = "${module.assets_bucket_label.tags}" noncurrent_version_expiration { days = "90" } noncurrent_version_transition { days = "60" storage_class = "GLACIER" } transition { days = "30" storage_class = "STANDARD_IA" } transition { days = "60" storage_class = "GLACIER" } expiration { days = "180" } } } ``` For an example of how we use it, check out our [`terraform-aws-s3-log-storage`](https://github.com/cloudposse/terraform-aws-s3-log-storage) module. :::info - https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html - https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#using-object-lifecycle ::: ## Encrypted S3 Buckets ```hcl title="Example: Encrypted S3 Bucket" module "assets_bucket_label" { source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.3.3" namespace = "eg" stage = "dev" name = "assets" } resource "aws_s3_bucket" "assets" { bucket = "${module.assets_bucket_label.id}" tags = "${module.assets_bucket_label.tags}" acl = "private" region = "us-west-2" force_destroy = false server_side_encryption_configuration { rule { apply_server_side_encryption_by_default { sse_algorithm = "AES256" } } } } ``` For an example of how we use it, check out our [`terraform-aws-s3-log-storage`](https://github.com/cloudposse/terraform-aws-s3-log-storage) module. ## Use Pre Commit Hooks for Linting We strongly urge that all code be linted prior to checking into Git. Running `terraform fmt` on the codebase before committing will accomplish this. To set this up so that it happens automatically prior to any commit, configure `git` pre-commit hooks using the `pre-commit` utility. ### OSX Installation ```shell brew install pre-commit ``` Then run `pre-commit install` in a given terraform repo to configure the hooks. ### .pre-commit-config.yaml ```yaml - repo: git://github.com/antonbabenko/pre-commit-terraform sha: v1.45.0 hooks: - id: terraform_fmt - id: terraform_validate ``` After setting this up, every time you commit, the `terraform fmt` command will be run to canonicalize your files and a basic smoke test to validate all configurations without requiring required variables to be set. Any time your commit affects any `*.tf` files, the validator will ensure well-formed terraform code. --- ## Tips and Tricks import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a collection of tips and tricks we've learned over the years implementing our reference architectures. --- ## Toolchain import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import ReactPlayer from 'react-player'; Learn about the essential tools Cloud Posse uses to manage infrastructure as code. This guide covers the Geodesic Toolbox Container for standardizing development environments, the Atmos framework for implementing conventions and workflows, Terraform for managing cloud infrastructure, and GitHub Actions for CI/CD automation. ## Geodesic Toolbox Container for DevOps We run most of our tools inside of this container. It's a great way to standardize your development environment and tooling across your team. ## Atmos Framework Atmos brings the framework and conventions. Learn more about Atmos, by visiting the [Atmos Documentation](https://atmos.tools/). - Components - Stacks - Vendoring - Workflows ## Terraform Terraform is the infrastructure as code tool used to manage your cloud infrastructure, called by Atmos. - Terraform Providers - Terraform Modules - Terraform "Root Modules" (aka "Components") ## GitHub Actions GitHub Actions is the CI/CD tool that we use to automate everything. - Self-hosted Runners - GitHub Actions - GitHub Actions Workflows --- ## Artifactory import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Explore our collection of reusable Terraform modules for managing and automating Artifactory configurations. These modules help streamline your Artifactory setup and maintenance using infrastructure as code. --- ## kv-store # Module: `kv-store` This module is a key/value store for storing configuration data that should be shared among terraform root modules. The backend for this key/value store is a generic Artifactory repository. ## Introduction ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-artifactory-kv-store/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-artifactory-kv-store/tree/main/test). ```hcl module "example" { source = "cloudposse/kv-store/artifactory" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" set = { "key1" = {value = "value1", sensative = false } "key2" = {value = "value2", sensative = true } } context = module.label.this } ``` ## Quick Start Here's how to get started... ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-artifactory-kv-store/) - complete example of using this module ## Variables ### Required Variables
`artifactory_auth_token` (`string`) required
The authentication token to use when accessing artifactory. Getting this value from the environment is supported with JFROG_ACCESS_TOKEN or ARTIFACTORY_ACCESS_TOKEN variables.
`artifactory_base_uri` (`string`) required
The base URI for artifactory.
`artifactory_repository` (`string`) required
The name of the artifactory repository.
`key_label_order` (`list(string)`) required
The order in which the labels (ID elements) appear in the full key path. For example, if you want a key path to look like /\{namespace\}/\{tenant\}/\{stage\}/\{environment\}/name, you would set this varibale to ["namespace", "tenant", "stage", "environment", "name"].
### Optional Variables
`get` optional
A map of keys to read from the key/value store. The key_path, namespace, tenant, stage, environment, and name are derived from context by default, but can be overridden by specifying a value in the map. **Type:** ```hcl map(object( { key_path = optional(string), properties = optional(map(string)) } ) ) ``` **Default value:** `{ }`
`get_by_path` optional
A map of keys to read from the key/value store. The key_path, namespace, tenant, stage, environment, and name are derived from context by default, but can be overridden by specifying a value in the map. **Type:** ```hcl map(object( { key_path = optional(string), properties = optional(map(string)) } ) ) ``` **Default value:** `{ }`
`key_prefix` (`string`) optional
The prefix to use for the key path. This is useful for storing all keys for a module under a common prefix. **Default value:** `""`
`set` optional
A map of key-value pairs to write to the key/value store. The key_path, namespace, tenant, stage, environment, and name are derived from context by default, but can be overridden by specifying a value in the map. **Type:** ```hcl map(object( { key_path = optional(string), value = string, sensitive = bool, properties = optional(map(string)) } ) ) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Outputs
`values`
the values retrieved with the kv store
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `artifactory`, version: `>= 10` - `context`, version: `>= 0.0.0` - `restapi`, version: `>= 1` ### Providers - `artifactory`, version: `>= 10` - `context`, version: `>= 0.0.0` - `restapi`, version: `>= 1` ## Resources The following resources are used by this module: - [`restapi_object.write_kv_pair`](https://registry.terraform.io/providers/Mastercard/restapi/latest/docs/resources/object) (resource) ## Data Sources The following data sources are used by this module: - [`artifactory_file.read_kv_pair`](https://registry.terraform.io/providers/jfrog/artifactory/latest/docs/data-sources/file) (data source) - [`artifactory_file.read_kv_path_file`](https://registry.terraform.io/providers/jfrog/artifactory/latest/docs/data-sources/file) (data source) - [`artifactory_file_list.read_kv_path`](https://registry.terraform.io/providers/jfrog/artifactory/latest/docs/data-sources/file_list) (data source) - [`context_config.this`](https://registry.terraform.io/providers/cloudposse/context/latest/docs/data-sources/config) (data source) --- ## acm-request-certificate # Module: `acm-request-certificate` Terraform module to request an ACM certificate for a domain and add a CNAME record to the DNS zone to complete certificate validation ## Usage This example will request an SSL certificate for `example.com` domain ```hcl module "acm_request_certificate" { source = "cloudposse/acm-request-certificate/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" domain_name = "example.com" process_domain_validation_options = true ttl = "300" } ``` This example will request an SSL certificate for `example.com` domain and all its subdomains `*.example.com` ```hcl module "acm_request_certificate" { source = "cloudposse/acm-request-certificate/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" domain_name = "example.com" process_domain_validation_options = true ttl = "300" subject_alternative_names = ["*.example.com"] } ``` ## Variables ### Required Variables
`domain_name` (`string`) required
A domain name for which the certificate should be issued
### Optional Variables
`certificate_authority_arn` (`string`) optional
ARN of an ACM PCA **Default value:** `null`
`certificate_export` (`bool`) optional
Specifies whether the certificate can be exported **Default value:** `false`
`certificate_transparency_logging_preference` (`bool`) optional
Specifies whether certificate details should be added to a certificate transparency log **Default value:** `true`
`key_algorithm` (`string`) optional
Specifies the algorithm of the public and private key pair that your Amazon issued certificate uses to encrypt data **Default value:** `null`
`process_domain_validation_options` (`bool`) optional
Flag to enable/disable processing of the record to add to the DNS zone to complete certificate validation **Default value:** `true`
`subject_alternative_names` (`list(string)`) optional
A list of domains that should be SANs in the issued certificate **Default value:** `[ ]`
`ttl` (`string`) optional
The TTL of the record to add to the DNS zone to complete certificate validation **Default value:** `"300"`
`validation_method` (`string`) optional
Method to use for validation, DNS or EMAIL **Default value:** `"DNS"`
`wait_for_certificate_issued` (`bool`) optional
Whether to wait for the certificate to be issued by ACM (the certificate status changed from `Pending Validation` to `Issued`) **Default value:** `false`
`zone_id` (`string`) optional
The zone id of the Route53 Hosted Zone which can be used instead of `var.zone_name`. **Default value:** `null`
`zone_name` (`string`) optional
The name of the desired Route53 Hosted Zone **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the certificate
`domain_validation_options`
CNAME records that are added to the DNS zone to complete certificate validation
`id`
The ID of the certificate
`validation_certificate_arn`
Certificate ARN from the `aws_acm_certificate_validation` resource
`validation_id`
The ID of the certificate validation
## Dependencies ### Requirements - `terraform`, version: `>= 0.14` - `aws`, version: `>= 6.4.0` ### Providers - `aws`, version: `>= 6.4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_acm_certificate.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate) (resource) - [`aws_acm_certificate_validation.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate_validation) (resource) - [`aws_route53_record.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) ## Data Sources The following data sources are used by this module: - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## alb(Alb) # Module: `alb` Terraform module to create an ALB, default ALB listener(s), and a default ALB target and related security groups. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-alb/tree/main/examples/complete). For automated test of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-alb/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name delimiter = var.delimiter attributes = var.attributes ipv4_primary_cidr_block = "10.0.0.0/16" assign_generated_ipv6_cidr_block = true tags = var.tags } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false tags = var.tags } module "alb" { source = "cloudposse/alb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter vpc_id = module.vpc.vpc_id security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.public_subnet_ids internal = var.internal http_enabled = var.http_enabled http_redirect = var.http_redirect access_logs_enabled = var.access_logs_enabled cross_zone_load_balancing_enabled = var.cross_zone_load_balancing_enabled http2_enabled = var.http2_enabled idle_timeout = var.idle_timeout ip_address_type = var.ip_address_type deletion_protection_enabled = var.deletion_protection_enabled deregistration_delay = var.deregistration_delay health_check_path = var.health_check_path health_check_timeout = var.health_check_timeout health_check_healthy_threshold = var.health_check_healthy_threshold health_check_unhealthy_threshold = var.health_check_unhealthy_threshold health_check_interval = var.health_check_interval health_check_matcher = var.health_check_matcher target_group_port = var.target_group_port target_group_target_type = var.target_group_target_type stickiness = var.stickiness alb_access_logs_s3_bucket_force_destroy = var.alb_access_logs_s3_bucket_force_destroy alb_access_logs_s3_bucket_force_destroy_enabled = var.alb_access_logs_s3_bucket_force_destroy_enabled tags = var.tags } ``` ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
A list of subnet IDs to associate with ALB
`vpc_id` (`string`) required
VPC ID to associate with ALB
### Optional Variables
`access_logs_enabled` (`bool`) optional
A boolean flag to enable/disable access_logs **Default value:** `true`
`access_logs_prefix` (`string`) optional
The S3 log bucket prefix **Default value:** `""`
`access_logs_s3_bucket_id` (`string`) optional
An external S3 Bucket name to store access logs in. If specified, no logging bucket will be created. **Default value:** `null`
`additional_certs` (`list(string)`) optional
A list of additonal certs to add to the https listerner **Default value:** `[ ]`
`alb_access_logs_s3_bucket_force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the ALB access logs S3 bucket so that the bucket can be destroyed without error **Default value:** `false`
`allow_ssl_requests_only` (`bool`) optional
Set to true to require requests to use Secure Socket Layer (HTTPS/SSL) on the access logs S3 bucket. This will explicitly deny access to HTTP requests **Default value:** `false`
`certificate_arn` (`string`) optional
The ARN of the default SSL certificate for HTTPS listener **Default value:** `""`
`client_keep_alive` (`number`) optional
Client keep alive value in seconds. The valid range is 60-604800 seconds. The default is 3600 seconds. **Default value:** `3600`
`cross_zone_load_balancing_enabled` (`bool`) optional
A boolean flag to enable/disable cross zone load balancing **Default value:** `true`
`default_target_group_enabled` (`bool`) optional
Whether the default target group should be created or not. **Default value:** `true`
`deletion_protection_enabled` (`bool`) optional
A boolean flag to enable/disable deletion protection for ALB **Default value:** `false`
`deregistration_delay` (`number`) optional
The amount of time to wait in seconds before changing the state of a deregistering target to unused **Default value:** `15`
`drop_invalid_header_fields` (`bool`) optional
Indicates whether HTTP headers with header fields that are not valid are removed by the load balancer (true) or routed to targets (false). **Default value:** `false`
`enable_glacier_transition` (`bool`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files **Default value:** `true`
`expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to expunge the objects **Default value:** `90`
`glacier_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to move the data to the Glacier Flexible Retrieval storage tier **Default value:** `60`
`health_check_healthy_threshold` (`number`) optional
The number of consecutive health checks successes required before considering an unhealthy target healthy **Default value:** `2`
`health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `15`
`health_check_matcher` (`string`) optional
The HTTP response codes to indicate a healthy check **Default value:** `"200-399"`
`health_check_path` (`string`) optional
The destination for the health check request **Default value:** `"/"`
`health_check_port` (`string`) optional
The port to use for the healthcheck **Default value:** `"traffic-port"`
`health_check_protocol` (`string`) optional
The protocol to use for the healthcheck. If not specified, same as the traffic protocol **Default value:** `null`
`health_check_timeout` (`number`) optional
The amount of time to wait in seconds before failing a health check request **Default value:** `10`
`health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before considering the target unhealthy **Default value:** `2`
`http2_enabled` (`bool`) optional
A boolean flag to enable/disable HTTP/2 **Default value:** `true`
`http_enabled` (`bool`) optional
A boolean flag to enable/disable HTTP listener **Default value:** `true`
`http_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in HTTP security group **Default value:** ```hcl [ "0.0.0.0/0", "::/0" ] ```
`http_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to HTTP ingress security group **Default value:** `[ ]`
`http_ingress_security_group_ids` (`list(string)`) optional
List of security group IDs to allow access to HTTP ingress security group **Default value:** `[ ]`
`http_port` (`number`) optional
The port for the HTTP listener **Default value:** `80`
`http_redirect` (`bool`) optional
A boolean flag to enable/disable HTTP redirect to HTTPS **Default value:** `false`
`https_enabled` (`bool`) optional
A boolean flag to enable/disable HTTPS listener **Default value:** `false`
`https_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in HTTPS security group **Default value:** ```hcl [ "0.0.0.0/0", "::/0" ] ```
`https_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to HTTPS ingress security group **Default value:** `[ ]`
`https_ingress_security_group_ids` (`list(string)`) optional
List of security group IDs to allow access to HTTPS ingress security group **Default value:** `[ ]`
`https_port` (`number`) optional
The port for the HTTPS listener **Default value:** `443`
`https_ssl_policy` (`string`) optional
The name of the SSL Policy for the listener **Default value:** `"ELBSecurityPolicy-TLS13-1-2-2021-06"`
`idle_timeout` (`number`) optional
The time in seconds that the connection is allowed to be idle **Default value:** `60`
`internal` (`bool`) optional
A boolean flag to determine whether the ALB should be internal **Default value:** `false`
`ip_address_type` (`string`) optional
The type of IP addresses used by the subnets for your load balancer. The possible values are `ipv4` and `dualstack`. **Default value:** `"ipv4"`
`lifecycle_configuration_rules` optional
A list of S3 bucket v2 lifecycle rules, as specified in [terraform-aws-s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket)" These rules are not affected by the deprecated `lifecycle_rule_enabled` flag. **NOTE:** Unless you also set `lifecycle_rule_enabled = false` you will also get the default deprecated rules set on your bucket. **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = any expiration = any transition = list(any) noncurrent_version_expiration = any noncurrent_version_transition = list(any) })) ``` **Default value:** `[ ]`
`lifecycle_rule_enabled` (`bool`) optional
DEPRECATED: Defaults to `false`, use `lifecycle_configuration_rules` instead. When `true`, configures lifecycle events on this bucket using individual (now deprecated) variables." **Default value:** `false`
`listener_additional_tags` (`map(string)`) optional
The additional tags to apply to all listeners **Default value:** `{ }`
`listener_http_fixed_response` optional
Have the HTTP listener return a fixed response for the default action. **Type:** ```hcl object({ content_type = string message_body = string status_code = string }) ``` **Default value:** `null`
`listener_https_fixed_response` optional
Have the HTTPS listener return a fixed response for the default action. **Type:** ```hcl object({ content_type = string message_body = string status_code = string }) ``` **Default value:** `null`
`listener_https_redirect` optional
Have the HTTPS listener return a redirect response for the default action. **Type:** ```hcl object({ host = optional(string) path = optional(string) port = optional(string) protocol = optional(string) query = optional(string) status_code = string }) ``` **Default value:** `null`
`load_balancer_name` (`string`) optional
The name for the default load balancer, uses a module label name if left empty **Default value:** `""`
`load_balancer_name_max_length` (`number`) optional
The max length of characters for the load balancer. **Default value:** `32`
`load_balancing_algorithm_type` (`string`) optional
Determines how the load balancer selects targets when routing requests. Only applicable for Application Load Balancer Target Groups **Default value:** `"round_robin"`
`load_balancing_anomaly_mitigation` (`string`) optional
Determines whether to enable target anomaly mitigation. Only supported by the weighted_random load balancing algorithm type. Valid values are 'on' or 'off'. **Default value:** `"off"`
`noncurrent_version_expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies when non-current object versions expire (in days) **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies (in days) when noncurrent object versions transition to Glacier Flexible Retrieval **Default value:** `30`
`preserve_host_header` (`bool`) optional
Indicates whether the Application Load Balancer should preserve the Host header in the HTTP request and send it to the target without any change. **Default value:** `false`
`reserved_capacity_units` (`number`) optional
The number of capacity units reserved for the load balancer **Default value:** `null`
`security_group_enabled` (`bool`) optional
Enables the security group **Default value:** `true`
`security_group_ids` (`list(string)`) optional
A list of additional security group IDs to attach to the ALB **Default value:** `[ ]`
`slow_start` (`number`) optional
The amount of time (30-900 seconds) until a healthy target receives its full share of requests from the load balancer. 0 to disable. **Default value:** `null`
`standard_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`stickiness` optional
Target group sticky configuration **Type:** ```hcl object({ cookie_duration = number enabled = bool }) ``` **Default value:** `null`
`target_group_additional_tags` (`map(string)`) optional
The additional tags to apply to the target group **Default value:** `{ }`
`target_group_name` (`string`) optional
The name for the default target group, uses a module label name if left empty **Default value:** `""`
`target_group_name_max_length` (`number`) optional
The max length of characters for the target group. **Default value:** `32`
`target_group_port` (`number`) optional
The port for the default target group **Default value:** `80`
`target_group_protocol` (`string`) optional
The protocol for the default target group HTTP or HTTPS **Default value:** `"HTTP"`
`target_group_protocol_version` (`string`) optional
The protocol version for the default target group HTTP1 or HTTP2 or GRPC **Default value:** `"HTTP1"`
`target_group_target_type` (`string`) optional
The type (`instance`, `ip` or `lambda`) of targets that can be registered with the target group **Default value:** `"ip"`
`xff_header_processing_mode` (`string`) optional
Determines how the load balancer modifies the X-Forwarded-For header in the HTTP request before sending the request to the target. The possible values are append, preserve, and remove. Only valid for Load Balancers of type application. The default is append **Default value:** `"append"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_logs_bucket_id`
The S3 bucket ID for access logs
`alb_arn`
The ARN of the ALB
`alb_arn_suffix`
The ARN suffix of the ALB
`alb_dns_name`
DNS name of ALB
`alb_name`
The ARN suffix of the ALB
`alb_zone_id`
The ID of the zone which ALB is provisioned
`default_target_group_arn`
The default target group ARN
`default_target_group_arn_suffix`
The default target group ARN suffix
`http_listener_arn`
The ARN of the HTTP forwarding listener
`http_redirect_listener_arn`
The ARN of the HTTP to HTTPS redirect listener
`https_listener_arn`
The ARN of the HTTPS listener
`listener_arns`
A list of all the listener ARNs
`security_group_id`
The security group ID of the ALB
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.99` ### Providers - `aws`, version: `>= 5.99` ### Modules Name | Version | Source | Description --- | --- | --- | --- `access_logs` | 0.19.0 | [`cloudposse/lb-s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/lb-s3-bucket/aws/0.19.0) | n/a `default_load_balancer_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `default_target_group_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lb.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb) (resource) - [`aws_lb_listener.http_forward`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) (resource) - [`aws_lb_listener.http_redirect`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) (resource) - [`aws_lb_listener.https`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) (resource) - [`aws_lb_listener_certificate.https_sni`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_certificate) (resource) - [`aws_lb_target_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_target_group) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.http_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.http_ingress_from_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.https_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.https_ingress_from_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: --- ## alb-ingress # Module: `alb-ingress` Terraform module to provision an HTTP style ALB ingress based on hostname and/or path. ALB ingress can be provisioned without authentication, or using Cognito or OIDC authentication. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-alb-ingress/tree/main/examples/complete). For automated test of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-alb-ingress/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name delimiter = var.delimiter attributes = var.attributes cidr_block = var.vpc_cidr_block tags = var.tags } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false tags = var.tags } module "alb" { source = "cloudposse/alb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter vpc_id = module.vpc.vpc_id security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.public_subnet_ids internal = var.internal http_enabled = var.http_enabled access_logs_enabled = var.access_logs_enabled alb_access_logs_s3_bucket_force_destroy = var.alb_access_logs_s3_bucket_force_destroy cross_zone_load_balancing_enabled = var.cross_zone_load_balancing_enabled http2_enabled = var.http2_enabled idle_timeout = var.idle_timeout ip_address_type = var.ip_address_type deletion_protection_enabled = var.deletion_protection_enabled deregistration_delay = var.deregistration_delay health_check_path = var.health_check_path health_check_timeout = var.health_check_timeout health_check_healthy_threshold = var.health_check_healthy_threshold health_check_unhealthy_threshold = var.health_check_unhealthy_threshold health_check_interval = var.health_check_interval health_check_matcher = var.health_check_matcher target_group_port = var.target_group_port target_group_target_type = var.target_group_target_type tags = var.tags } module "alb_ingress" { source = "cloudposse/alb-ingress/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter vpc_id = module.vpc.vpc_id authentication_type = var.authentication_type unauthenticated_priority = var.unauthenticated_priority unauthenticated_paths = var.unauthenticated_paths slow_start = var.slow_start stickiness_enabled = var.stickiness_enabled default_target_group_enabled = false target_group_arn = module.alb.default_target_group_arn unauthenticated_listener_arns = [module.alb.http_listener_arn] tags = var.tags } ``` ## Variables ### Required Variables
`vpc_id` (`string`) required
The VPC ID where generated ALB target group will be provisioned (if `target_group_arn` is not set)
### Optional Variables
`authenticated_hosts` (`list(string)`) optional
Authenticated hosts to match in Hosts header **Default value:** `[ ]`
`authenticated_listener_arns` (`list(string)`) optional
A list of authenticated ALB listener ARNs to attach ALB listener rules to **Default value:** `[ ]`
`authenticated_paths` (`list(string)`) optional
Authenticated path pattern to match (a maximum of 1 can be defined) **Default value:** `[ ]`
`authenticated_priority` (`number`) optional
The priority for the rules with authentication, between 1 and 50000 (1 being highest priority). Must be different from `unauthenticated_priority` since a listener can't have multiple rules with the same priority **Default value:** `null`
`authentication_cognito_on_unauthenticated_request` (`string`) optional
Cognito unauthenticated behavior, deny, allow, or authenticate **Default value:** `"authenticate"`
`authentication_cognito_request_extra_params` (`map(string)`) optional
Cognito query parameters to include in redirect request **Default value:** `null`
`authentication_cognito_scope` (`string`) optional
Cognito scope, which should be a space separated string of requested scopes (see https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims) **Default value:** `null`
`authentication_cognito_user_pool_arn` (`string`) optional
Cognito User Pool ARN **Default value:** `""`
`authentication_cognito_user_pool_client_id` (`string`) optional
Cognito User Pool Client ID **Default value:** `""`
`authentication_cognito_user_pool_domain` (`string`) optional
Cognito User Pool Domain. The User Pool Domain should be set to the domain prefix (`xxx`) instead of full domain (https://xxx.auth.us-west-2.amazoncognito.com) **Default value:** `""`
`authentication_oidc_authorization_endpoint` (`string`) optional
OIDC Authorization Endpoint **Default value:** `""`
`authentication_oidc_client_id` (`string`) optional
OIDC Client ID **Default value:** `""`
`authentication_oidc_client_secret` (`string`) optional
OIDC Client Secret **Default value:** `""`
`authentication_oidc_issuer` (`string`) optional
OIDC Issuer **Default value:** `""`
`authentication_oidc_on_unauthenticated_request` (`string`) optional
OIDC unauthenticated behavior, deny, allow, or authenticate **Default value:** `"authenticate"`
`authentication_oidc_request_extra_params` (`map(string)`) optional
OIDC query parameters to include in redirect request **Default value:** `null`
`authentication_oidc_scope` (`string`) optional
OIDC scope, which should be a space separated string of requested scopes (see https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims, and https://developers.google.com/identity/protocols/oauth2/openid-connect#scope-param for an example set of scopes when using Google as the IdP) **Default value:** `null`
`authentication_oidc_token_endpoint` (`string`) optional
OIDC Token Endpoint **Default value:** `""`
`authentication_oidc_user_info_endpoint` (`string`) optional
OIDC User Info Endpoint **Default value:** `""`
`authentication_type` (`string`) optional
Authentication type. Supported values are `COGNITO` and `OIDC` **Default value:** `""`
`default_target_group_enabled` (`bool`) optional
Enable/disable creation of the default target group **Default value:** `true`
`deregistration_delay` (`number`) optional
The amount of time to wait in seconds while deregistering target **Default value:** `15`
`health_check_enabled` (`bool`) optional
Indicates whether health checks are enabled. Defaults to `true` **Default value:** `true`
`health_check_healthy_threshold` (`number`) optional
The number of consecutive health checks successes required before healthy **Default value:** `2`
`health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `15`
`health_check_matcher` (`string`) optional
The HTTP response codes to indicate a healthy check **Default value:** `"200-399"`
`health_check_path` (`string`) optional
The destination for the health check request **Default value:** `"/"`
`health_check_port` (`string`) optional
The port to use to connect with the target. Valid values are either ports 1-65536, or `traffic-port`. Defaults to `traffic-port` **Default value:** `"traffic-port"`
`health_check_protocol` (`string`) optional
The protocol to use to connect with the target. Defaults to `HTTP`. Not applicable when `target_type` is `lambda` **Default value:** `"HTTP"`
`health_check_timeout` (`number`) optional
The amount of time to wait in seconds before failing a health check request **Default value:** `10`
`health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before unhealthy **Default value:** `2`
`listener_http_header_conditions` optional
A list of http header conditions to apply to the listener. **Type:** ```hcl list(object({ name = string value = list(string) })) ``` **Default value:** `[ ]`
`load_balancing_algorithm_type` (`string`) optional
Determines how the load balancer selects targets when routing requests. Only applicable for Application Load Balancer Target Groups. The value is round_robin, least_outstanding_requests or weighted_random. The default is round_robin. **Default value:** `"round_robin"`
`load_balancing_anomaly_mitigation` (`string`) optional
Determines whether to enable target anomaly mitigation. Only supported by the weighted_random load balancing algorithm type. Valid values are 'on' or 'off'. **Default value:** `"off"`
`port` (`number`) optional
The port for the created ALB target group (if `target_group_arn` is not set) **Default value:** `80`
`protocol` (`string`) optional
The protocol for the created ALB target group (if `target_group_arn` is not set) **Default value:** `"HTTP"`
`protocol_version` (`string`) optional
Only applicable when protocol is `HTTP` or `HTTPS`. The protocol version. Specify GRPC to send requests to targets using gRPC. Specify HTTP2 to send requests to targets using HTTP/2. The default is `HTTP1`, which sends requests to targets using HTTP/1.1 **Default value:** `"HTTP1"`
`slow_start` (`number`) optional
The amount of time for targets to warm up before the load balancer sends them a full share of requests. The range is 30-900 seconds or 0 to disable. The default value is `0` seconds **Default value:** `0`
`stickiness_cookie_duration` (`number`) optional
The time period, in seconds, during which requests from a client should be routed to the same target. After this time period expires, the load balancer-generated cookie is considered stale. The range is 1 second to 1 week (604800 seconds). The default value is 1 day (86400 seconds) **Default value:** `86400`
`stickiness_cookie_name` (`string`) optional
Name of the application based cookie. AWSALB, AWSALBAPP, and AWSALBTG prefixes are reserved and cannot be used. Only needed when `stickiness_type` is app_cookie **Default value:** `null`
`stickiness_enabled` (`bool`) optional
Boolean to enable / disable `stickiness`. Default is `true` **Default value:** `true`
`stickiness_type` (`string`) optional
The type of sticky sessions. The possible values are `lb_cookie` or `app_cookie` **Default value:** `"lb_cookie"`
`target_group_arn` (`string`) optional
Existing ALB target group ARN. If provided, set `default_target_group_enabled` to `false` to disable creation of the default target group **Default value:** `""`
`target_group_name` (`string`) optional
Override the target group name **Default value:** `""`
`target_type` (`string`) optional
The type (`instance`, `ip` or `lambda`) of targets that can be registered with the target group **Default value:** `"ip"`
`unauthenticated_hosts` (`list(string)`) optional
Unauthenticated hosts to match in Hosts header **Default value:** `[ ]`
`unauthenticated_listener_arns` (`list(string)`) optional
A list of unauthenticated ALB listener ARNs to attach ALB listener rules to **Default value:** `[ ]`
`unauthenticated_paths` (`list(string)`) optional
Unauthenticated path pattern to match (a maximum of 1 can be defined) **Default value:** `[ ]`
`unauthenticated_priority` (`number`) optional
The priority for the rules without authentication, between 1 and 50000 (1 being highest priority). Must be different from `authenticated_priority` since a listener can't have multiple rules with the same priority **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`target_group_arn`
ALB Target Group ARN
`target_group_arn_suffix`
ALB Target Group ARN suffix
`target_group_name`
ALB Target Group name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `target_group` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lb_listener_rule.authenticated_hosts_cognito`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.authenticated_hosts_oidc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.authenticated_hosts_paths_cognito`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.authenticated_hosts_paths_oidc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.authenticated_paths_cognito`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.authenticated_paths_oidc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.unauthenticated_hosts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.unauthenticated_hosts_paths`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_listener_rule.unauthenticated_paths`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_lb_target_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_target_group) (resource) ## Data Sources The following data sources are used by this module: - [`aws_lb_target_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lb_target_group) (data source) --- ## alb-target-group-cloudwatch-sns-alarms # Module: `alb-target-group-cloudwatch-sns-alarms` Terraform module for creating alarms for tracking important changes and occurrences from ALBs. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms/tree/main/examples/complete). For automated tests of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-alb-target-group-cloudwatch-sns-alarms/tree/main/test). ```hcl module "alb_alarms" { source = "cloudposse/alb-target-group-cloudwatch-sns-alarms/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "staging" name = "app" alb_arn_suffix = "XXXXXXXXX" target_group_arn_suffix = "YYYYYYYYY" } ``` ## Variables ### Required Variables
`alb_arn_suffix` (`string`) required
The ARN suffix of ALB
`target_group_arn_suffix` (`string`) required
The ARN suffix of ALB Target Group
### Optional Variables
`alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when this alarm transitions into an ALARM state from any other state. If set, this list takes precedence over notify_arns **Default value:** ```hcl [ "" ] ```
`elb_5xx_count_threshold` (`number`) optional
The maximum count of ELB 5XX requests over a period. A negative value will disable the alert **Default value:** `25`
`evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`httpcode_alarm_description` (`string`) optional
The string to format and use as the httpcode alarm description **Default value:** `"HTTPCode %v count for %v over %v last %d minute(s) over %v period(s)"`
`insufficient_data_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when this alarm transitions into an INSUFFICIENT_DATA state from any other state. If set, this list takes precedence over notify_arns **Default value:** ```hcl [ "" ] ```
`notify_arns` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when this alarm transitions into ANY state from any other state. May be overridden by the value of a more specific \{alarm,ok,insufficient_data\}_actions variable. **Default value:** ```hcl [ "" ] ```
`ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when this alarm transitions into an OK state from any other state. If set, this list takes precedence over notify_arns **Default value:** ```hcl [ "" ] ```
`period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`target_3xx_count_threshold` (`number`) optional
The maximum count of 3XX requests over a period. A negative value will disable the alert **Default value:** `25`
`target_4xx_count_threshold` (`number`) optional
The maximum count of 4XX requests over a period. A negative value will disable the alert **Default value:** `25`
`target_5xx_count_threshold` (`number`) optional
The maximum count of 5XX requests over a period. A negative value will disable the alert **Default value:** `25`
`target_response_time_alarm_description` (`string`) optional
The string to format and use as the target response time alarm description **Default value:** `"Target Response Time average for %v over %v last %d minute(s) over %v period(s)"`
`target_response_time_threshold` (`number`) optional
The maximum average target response time (in seconds) over a period. A negative value will disable the alert **Default value:** `0.5`
`treat_missing_data` (`string`) optional
Sets how alarms handle missing data points. Values supported: missing, ignore, breaching and notBreaching **Default value:** `"missing"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`httpcode_elb_5xx_count_cloudwatch_metric_alarm_arn`
ELB 5xx count CloudWatch metric alarm ARN
`httpcode_elb_5xx_count_cloudwatch_metric_alarm_id`
ELB 5xx count CloudWatch metric alarm ID
`httpcode_target_3xx_count_cloudwatch_metric_alarm_arn`
Target Group 3xx count CloudWatch metric alarm ARN
`httpcode_target_3xx_count_cloudwatch_metric_alarm_id`
Target Group 3xx count CloudWatch metric alarm ID
`httpcode_target_4xx_count_cloudwatch_metric_alarm_arn`
Target Group 4xx count CloudWatch metric alarm ARN
`httpcode_target_4xx_count_cloudwatch_metric_alarm_id`
Target Group 4xx count CloudWatch metric alarm ID
`httpcode_target_5xx_count_cloudwatch_metric_alarm_arn`
Target Group 5xx count CloudWatch metric alarm ARN
`httpcode_target_5xx_count_cloudwatch_metric_alarm_id`
Target Group 5xx count CloudWatch metric alarm ID
`target_response_time_average_cloudwatch_metric_alarm_arn`
Target Group response time average CloudWatch metric alarm ARN
`target_response_time_average_cloudwatch_metric_alarm_id`
Target Group response time average CloudWatch metric alarm ID
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `httpcode_elb_5xx_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `httpcode_target_3xx_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `httpcode_target_4xx_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `httpcode_target_5xx_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `target_response_time_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.httpcode_elb_5xx_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.httpcode_target_3xx_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.httpcode_target_4xx_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.httpcode_target_5xx_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.target_response_time_average`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) ## Data Sources The following data sources are used by this module: --- ## amplify-app # Module: `amplify-app` Terraform module to provision AWS Amplify apps, backend environments, branches, domain associations, and webhooks. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-amplify-app/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-amplify-app/tree/main/test). ```hcl data "aws_ssm_parameter" "github_pat" { name = var.github_personal_access_token_secret_path with_decryption = true } module "amplify_app" { source = "cloudposse/amplify-app/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" access_token = data.aws_ssm_parameter.github_pat.value description = "Test Amplify App" repository = "https://github.com/cloudposse/amplify-test2" platform = "WEB" enable_auto_branch_creation = false enable_branch_auto_build = true enable_branch_auto_deletion = true enable_basic_auth = false iam_service_role_enabled = true iam_service_role_actions = [ "logs:CreateLogStream", "logs:CreateLogGroup", "logs:DescribeLogGroups", "logs:PutLogEvents" ] auto_branch_creation_patterns = [ "*", "*/**" ] auto_branch_creation_config = { # Enable auto build for the created branches enable_auto_build = true } # The build spec for React build_spec = <<-EOT version: 0.1 frontend: phases: preBuild: commands: - yarn install build: commands: - yarn run build artifacts: baseDirectory: build files: - '**/*' cache: paths: - node_modules/**/* EOT custom_rules = [ { source = "/<*>" status = "404" target = "/index.html" } ] environment_variables = { ENV = "test" } environments = { main = { branch_name = "main" enable_auto_build = true backend_enabled = false enable_performance_mode = true enable_pull_request_preview = false framework = "React" stage = "PRODUCTION" } dev = { branch_name = "dev" enable_auto_build = true backend_enabled = false enable_performance_mode = false enable_pull_request_preview = true framework = "React" stage = "DEVELOPMENT" } } domains = { "test.net" = { enable_auto_sub_domain = true wait_for_verification = false sub_domain = [ { branch_name = "main" prefix = "" }, { branch_name = "dev" prefix = "dev" } ] } } context = module.label.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-amplify-app/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`access_token` (`string`) optional
The personal access token for a third-party source control system for the Amplify app. The personal access token is used to create a webhook and a read-only deploy key. The token is not stored. Make sure that the account where the token is created has access to the repository. **Default value:** `null`
`auto_branch_creation_config` optional
The automated branch creation configuration for the Amplify app **Type:** ```hcl object({ basic_auth_credentials = optional(string) build_spec = optional(string) enable_auto_build = optional(bool) enable_basic_auth = optional(bool) enable_performance_mode = optional(bool) enable_pull_request_preview = optional(bool) environment_variables = optional(map(string)) framework = optional(string) pull_request_environment_name = optional(string) stage = optional(string) }) ``` **Default value:** `null`
`auto_branch_creation_patterns` (`list(string)`) optional
The automated branch creation glob patterns for the Amplify app **Default value:** `[ ]`
`basic_auth_credentials` (`string`) optional
The credentials for basic authorization for the Amplify app **Default value:** `null`
`build_spec` (`string`) optional
The [build specification](https://docs.aws.amazon.com/amplify/latest/userguide/build-settings.html) (build spec) for the Amplify app. If not provided then it will use the `amplify.yml` at the root of your project / branch. **Default value:** `null`
`custom_headers` (`string`) optional
The custom headers for the Amplify app, allows specifying headers for every HTTP response. Must adhere to AWS's format: https://docs.aws.amazon.com/amplify/latest/userguide/custom-headers.html **Default value:** `null`
`custom_rules` optional
The custom rules to apply to the Amplify App **Type:** ```hcl list(object({ condition = optional(string) source = string status = optional(string) target = string })) ``` **Default value:** `[ ]`
`description` (`string`) optional
The description for the Amplify app **Default value:** `null`
`domain_config` optional
DEPRECATED: Use the `domains` variable instead. Amplify custom domain configuration. **Type:** ```hcl object({ domain_name = string enable_auto_sub_domain = optional(bool, false) wait_for_verification = optional(bool, false) sub_domain = list(object({ branch_name = string prefix = string })) }) ``` **Default value:** `null`
`domains` optional
Amplify custom domain configurations **Type:** ```hcl map(object({ enable_auto_sub_domain = optional(bool, false) wait_for_verification = optional(bool, false) sub_domain = list(object({ branch_name = string prefix = string })) })) ``` **Default value:** `{ }`
`enable_auto_branch_creation` (`bool`) optional
Enables automated branch creation for the Amplify app **Default value:** `false`
`enable_basic_auth` (`bool`) optional
Enables basic authorization for the Amplify app. This will apply to all branches that are part of this app. **Default value:** `false`
`enable_branch_auto_build` (`bool`) optional
Enables auto-building of branches for the Amplify App **Default value:** `true`
`enable_branch_auto_deletion` (`bool`) optional
Automatically disconnects a branch in the Amplify Console when you delete a branch from your Git repository **Default value:** `false`
`environment_variables` (`map(string)`) optional
The environment variables for the Amplify app **Default value:** `{ }`
`environments` optional
The configuration of the environments for the Amplify App **Type:** ```hcl map(object({ branch_name = optional(string) basic_auth_credentials = optional(string) backend_enabled = optional(bool, false) environment_name = optional(string) deployment_artifacts = optional(string) stack_name = optional(string) display_name = optional(string) description = optional(string) enable_auto_build = optional(bool) enable_basic_auth = optional(bool) enable_notification = optional(bool) enable_performance_mode = optional(bool) enable_pull_request_preview = optional(bool) environment_variables = optional(map(string)) framework = optional(string) pull_request_environment_name = optional(string) stage = optional(string) ttl = optional(number) webhook_enabled = optional(bool, false) })) ``` **Default value:** `{ }`
`iam_service_role_actions` (`list(string)`) optional
List of IAM policy actions for the AWS Identity and Access Management (IAM) service role for the Amplify app. If not provided, the default set of actions will be used for the role if the variable `iam_service_role_enabled` is set to `true`. **Default value:** `[ ]`
`iam_service_role_arn` (`list(string)`) optional
The AWS Identity and Access Management (IAM) service role for the Amplify app. If not provided, a new role will be created if the variable `iam_service_role_enabled` is set to `true`. **Default value:** `[ ]`
`iam_service_role_enabled` (`bool`) optional
Flag to create the IAM service role for the Amplify app **Default value:** `false`
`oauth_token` (`string`) optional
The OAuth token for a third-party source control system for the Amplify app. The OAuth token is used to create a webhook and a read-only deploy key. The OAuth token is not stored. **Default value:** `null`
`platform` (`string`) optional
The platform or framework for the Amplify app **Default value:** `"WEB"`
`repository` (`string`) optional
The repository for the Amplify app **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amplify App ARN
`backend_environments`
Created backend environments
`branch_names`
The names of the created Amplify branches
`default_domain`
Amplify App domain (non-custom)
`domain_associations`
Created domain associations
`id`
Amplify App Id
`name`
Amplify App name
`webhooks`
Created webhooks
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `role` | 0.18.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.18.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_amplify_app.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/amplify_app) (resource) - [`aws_amplify_backend_environment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/amplify_backend_environment) (resource) - [`aws_amplify_branch.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/amplify_branch) (resource) - [`aws_amplify_domain_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/amplify_domain_association) (resource) - [`aws_amplify_webhook.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/amplify_webhook) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## api-gateway # Module: `api-gateway` Terraform module to provision API Gatway resources. The root module creates an API Gateway [REST API](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-rest-api.html) along with configuring tracing, logging, and metrics. The module also consists of the following submodules: - [account-settings](https://github.com/cloudposse/terraform-aws-api-gateway/tree/main/modules/account-settings) - to provision account-level settings for logging and metrics for API Gateway ## Introduction A set of modules for configuring an API Gateway ## Usage Setup the account-level settings for logging and metrics for API Gateway: ```hcl module "api_gateway_account_settings" { source = "cloudposse/api-gateway/aws//modules/account-settings" # version = "x.x.x" context = module.this.context } ``` ## Examples Review the [examples](https://github.com/cloudposse/terraform-aws-api-gateway/tree/main/examples) folder to see how to use the API Gateway modules. ## Variables ### Required Variables
### Optional Variables
`access_log_format` (`string`) optional
The format of the access log file. **Default value:** `" {\n\t\"requestTime\": \"$context.requestTime\",\n\t\"requestId\": \"$context.requestId\",\n\t\"httpMethod\": \"$context.httpMethod\",\n\t\"path\": \"$context.path\",\n\t\"resourcePath\": \"$context.resourcePath\",\n\t\"status\": $context.status,\n\t\"responseLatency\": $context.responseLatency,\n \"xrayTraceId\": \"$context.xrayTraceId\",\n \"integrationRequestId\": \"$context.integration.requestId\",\n\t\"functionResponseStatus\": \"$context.integration.status\",\n \"integrationLatency\": \"$context.integration.latency\",\n\t\"integrationServiceStatus\": \"$context.integration.integrationStatus\",\n \"authorizeResultStatus\": \"$context.authorize.status\",\n\t\"authorizerServiceStatus\": \"$context.authorizer.status\",\n\t\"authorizerLatency\": \"$context.authorizer.latency\",\n\t\"authorizerRequestId\": \"$context.authorizer.requestId\",\n \"ip\": \"$context.identity.sourceIp\",\n\t\"userAgent\": \"$context.identity.userAgent\",\n\t\"principalId\": \"$context.authorizer.principalId\",\n\t\"cognitoUser\": \"$context.identity.cognitoIdentityId\",\n \"user\": \"$context.identity.user\"\n}\n"`
`data_trace_enabled` (`bool`) optional
Whether data trace logging is enabled for this method, which effects the log entries pushed to Amazon CloudWatch Logs. **Default value:** `false`
`endpoint_type` (`string`) optional
The type of the endpoint. One of - PUBLIC, PRIVATE, REGIONAL **Default value:** `"REGIONAL"`
`iam_tags_enabled` (`string`) optional
Enable/disable tags on IAM roles and policies **Default value:** `true`
`logging_level` (`string`) optional
The logging level of the API. One of - OFF, INFO, ERROR **Default value:** `"INFO"`
`metrics_enabled` (`bool`) optional
A flag to indicate whether to enable metrics collection. **Default value:** `false`
`openapi_config` (`any`) optional
The OpenAPI specification for the API **Default value:** `{ }`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the IAM role **Default value:** `""`
`private_link_target_arns` (`list(string)`) optional
A list of target ARNs for VPC Private Link **Default value:** `[ ]`
`rest_api_policy` (`string`) optional
The IAM policy document for the API. **Default value:** `null`
`stage_name` (`string`) optional
The name of the stage **Default value:** `""`
`throttling_burst_limit` (`number`) optional
The API request burst limit **Default value:** `-1`
`throttling_rate_limit` (`number`) optional
The API request rate limit **Default value:** `-1`
`vpc_endpoints` (`list(string)`) optional
List of VPC Endpoint IDs to attach to the API Gateway **Default value:** `null`
`xray_tracing_enabled` (`bool`) optional
A flag to indicate whether to enable X-Ray tracing. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the REST API
`created_date`
The date the REST API was created
`execution_arn`
The execution ARN part to be used in lambda_permission's source_arn when allowing API Gateway to invoke a Lambda function, e.g., arn:aws:execute-api:eu-west-2:123456789012:z4675bid1j, which can be concatenated with allowed stage, method and resource path.The ARN of the Lambda function that will be executed.
`id`
The ID of the REST API
`invoke_url`
The URL to invoke the REST API
`log_group_name`
The ARN of the Cloudwatch log group
`root_resource_id`
The resource ID of the REST API's root
`stage_arn`
The ARN of the gateway stage
## Dependencies ### Requirements - `terraform`, version: `>= 0.13` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_log_group` | 0.6.8 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.8) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_api_gateway_deployment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_deployment) (resource) - [`aws_api_gateway_method_settings.all`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_method_settings) (resource) - [`aws_api_gateway_rest_api.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_rest_api) (resource) - [`aws_api_gateway_rest_api_policy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_rest_api_policy) (resource) - [`aws_api_gateway_stage.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_stage) (resource) - [`aws_api_gateway_vpc_link.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/api_gateway_vpc_link) (resource) ## Data Sources The following data sources are used by this module: --- ## account-settings(Account-settings) # API Gateway Account Settings Module This module allows you set the global, regional settings required to allow API Gateway to write to CloudWatch logs. Every AWS region you want to deploy an API Gateway to must be configured with an IAM Role that gives API Gateway permissions to create and write to CloudWatch logs. Without this configuration, API Gateway will not be able to send logs to CloudWatch. This configuration is done once per region regardless of the number of API Gateways deployed in that region. This module creates an IAM role, assigns it the necessary permissions to write logs and sets it as the "CloudWatch log role ARN" in the API Gateway configuration. ## Usage **IMPORTANT:** The `main` branch is used in `source` just as an example. In your code, do not pin to `main` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-config/releases). For a complete example, see [examples/account-settings](https://github.com/cloudposse/terraform-aws-api-gateway/tree/main/modules/account-settings/../../examples/account-settings). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest)(which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-api-gateway/tree/main/modules/account-settings/../../test). ```hcl module "account_settings" { source = "cloudposse/api-gateway/aws//modules/account-settings" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } ``` --- ## athena(Athena) # Module: `athena` Terraform module to deploy an instance of [Amazon Athena](https://aws.amazon.com/athena/) on AWS. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-athena/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-athena/tree/main/test). ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "athena" { source = "cloudposse/athena/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.label.this } ``` ## Variables ### Required Variables
`data_catalogs` (`map(any)`) required
Map of Athena data catalogs and related configuration.
`databases` (`map(any)`) required
Map of Athena databases and related configuration.
`named_queries` (`map(map(string))`) required
Map of Athena named queries and related configuration.
### Optional Variables
`athena_kms_key` (`string`) optional
Use an existing KMS key for Athena if `create_kms_key` is `false`. **Default value:** `null`
`athena_kms_key_deletion_window` (`number`) optional
KMS key deletion window (in days). **Default value:** `7`
`athena_s3_bucket_id` (`string`) optional
Use an existing S3 bucket for Athena query results if `create_s3_bucket` is `false`. **Default value:** `null`
`bytes_scanned_cutoff_per_query` (`number`) optional
Integer for the upper data usage limit (cutoff) for the amount of bytes a single query in a workgroup is allowed to scan. Must be at least 10485760. **Default value:** `null`
`create_kms_key` (`bool`) optional
Enable the creation of a KMS key used by Athena workgroup. **Default value:** `true`
`create_s3_bucket` (`bool`) optional
Enable the creation of an S3 bucket to use Athena query results **Default value:** `true`
`enforce_workgroup_configuration` (`bool`) optional
Boolean whether the settings for the workgroup override client-side settings. **Default value:** `true`
`publish_cloudwatch_metrics_enabled` (`bool`) optional
Boolean whether Amazon CloudWatch metrics are enabled for the workgroup. **Default value:** `true`
`requester_pays_enabled` (`bool`) optional
Boolean whether to enable Requester Pays option on the Athena workgroup. **Default value:** `false`
`s3_output_path` (`string`) optional
The S3 bucket path used to store query results. **Default value:** `""`
`workgroup_description` (`string`) optional
A description the of Athena workgroup. **Default value:** `""`
`workgroup_encryption_option` (`string`) optional
Indicates whether Amazon S3 server-side encryption with Amazon S3-managed keys (SSE_S3), server-side encryption with KMS-managed keys (SSE_KMS), or client-side encryption with KMS-managed keys (CSE_KMS) is used. **Default value:** `"SSE_KMS"`
`workgroup_force_destroy` (`bool`) optional
The option to delete the workgroup and its contents even if the workgroup contains any named queries. **Default value:** `false`
`workgroup_state` (`string`) optional
State of the workgroup. Valid values are `DISABLED` or `ENABLED`. **Default value:** `"ENABLED"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`data_catalogs`
List of newly created Athena data catalogs.
`databases`
List of newly created Athena databases.
`kms_key_arn`
ARN of KMS key used by Athena.
`named_queries`
List of newly created Athena named queries.
`s3_bucket_id`
ID of S3 bucket used by Athena.
`workgroup_id`
ID of newly created Athena workgroup.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_athena_data_catalog.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_data_catalog) (resource) - [`aws_athena_database.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_database) (resource) - [`aws_athena_named_query.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_named_query) (resource) - [`aws_athena_workgroup.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/athena_workgroup) (resource) - [`aws_kms_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) ## Data Sources The following data sources are used by this module: --- ## AWS import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Discover a wide range of reusable Terraform modules tailored for managing AWS infrastructure. These modules simplify the deployment and management of AWS services through consistent and scalable code. --- ## backup # Module: `backup` Terraform module to provision [AWS Backup](https://aws.amazon.com/backup), a fully managed backup service that makes it easy to centralize and automate the back up of data across AWS services such as Amazon EBS volumes, Amazon EC2 instances, Amazon RDS databases, Amazon DynamoDB tables, Amazon EFS file systems, and AWS Storage Gateway volumes. :::note The syntax of declaring a backup schedule has changed as of release `0.14.0`, follow the instructions in the [0.13.x to 0.14.x+ migration guide](https://github.com/cloudposse/terraform-aws-backup/tree/main/docs/migration-0.13.x-0.14.x+.md). ::: :::warning The deprecated variables have been fully deprecated as of `1.x.x`. Please use the new variables as described in the [0.13.x to 0.14.x+ migration guide](https://github.com/cloudposse/terraform-aws-backup/tree/main/docs/migration-0.13.x-0.14.x+.md). ::: ## Usage For a complete example on how to backup an Elastic File System (EFS), see [examples/complete](https://github.com/cloudposse/terraform-aws-backup/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [`test`](https://github.com/cloudposse/terraform-aws-backup/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes tags = var.tags delimiter = var.delimiter ipv4_primary_cidr_block = "172.16.0.0/16" } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes tags = var.tags delimiter = var.delimiter availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false } module "efs" { source = "cloudposse/efs/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes tags = var.tags delimiter = var.delimiter region = var.region availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id subnets = module.subnets.private_subnet_ids allowed_security_group_ids = [module.vpc.vpc_default_security_group_id] } module "backup" { source = "cloudposse/backup/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes tags = var.tags delimiter = var.delimiter plan_name_suffix = var.plan_name_suffix vault_enabled = var.vault_enabled iam_role_enabled = var.iam_role_enabled plan_enabled = var.plan_enabled selection_tags = var.selection_tags backup_resources = [module.efs.arn] not_resources = var.not_resources rules = var.rules kms_key_arn = var.kms_key_arn advanced_backup_setting = var.advanced_backup_setting backup_vault_lock_configuration = var.backup_vault_lock_configuration } ``` In the above example `var.rules` could be defined as follows: ```hcl rules = [ { name = "${module.this.name}-daily" schedule = var.schedule start_window = var.start_window completion_window = var.completion_window lifecycle = { cold_storage_after = var.cold_storage_after delete_after = var.delete_after } } ] ``` or as yaml ```yaml rules: - name: "plan-daily" schedule: "cron(0 5 ? * * *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes delete_after: 35 # 7 * 5 ``` ## Variables ### Required Variables
### Optional Variables
`advanced_backup_setting` optional
An object that specifies backup options for each resource type **Type:** ```hcl object({ backup_options = map(string) resource_type = string }) ``` **Default value:** `null`
`backup_resources` (`list(string)`) optional
An array of strings that either contain Amazon Resource Names (ARNs) or match patterns of resources to assign to a backup plan **Default value:** `[ ]`
`backup_vault_lock_configuration` optional
The backup vault lock configuration, each vault can have one vault lock in place. This will enable Backup Vault Lock on an AWS Backup vault it prevents the deletion of backup data for the specified retention period. During this time, the backup data remains immutable and cannot be deleted or modified." `changeable_for_days` - The number of days before the lock date. If omitted creates a vault lock in `governance` mode, otherwise it will create a vault lock in `compliance` mode. **Type:** ```hcl object({ changeable_for_days = optional(number) max_retention_days = optional(number) min_retention_days = optional(number) }) ``` **Default value:** `null`
`iam_role_enabled` (`bool`) optional
Should we create a new Iam Role and Policy Attachment **Default value:** `true`
`iam_role_name` (`string`) optional
Override target IAM Role Name **Default value:** `null`
`kms_key_arn` (`string`) optional
The server-side encryption key that is used to protect your backups **Default value:** `null`
`not_resources` (`list(string)`) optional
An array of strings that either contain Amazon Resource Names (ARNs) or match patterns of resources to exclude from a backup plan **Default value:** `[ ]`
`permissions_boundary` (`string`) optional
The permissions boundary to set on the role **Default value:** `null`
`plan_enabled` (`bool`) optional
Should we create a new Plan **Default value:** `true`
`plan_name_suffix` (`string`) optional
The string appended to the plan name **Default value:** `null`
`rules` optional
A list of rule objects used to define schedules in a backup plan. Follows the following structure: ```yaml rules: - name: "plan-daily" schedule: "cron(0 5 ? * * *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes delete_after: 35 # 7 * 5 # days - name: "plan-weekly" schedule: "cron(0 5 ? * SAT *)" start_window: 320 # 60 * 8 # minutes completion_window: 10080 # 60 * 24 * 7 # minutes delete_after: 90 # 30 * 3 ``` **Type:** ```hcl list(object({ name = string schedule = optional(string) enable_continuous_backup = optional(bool) start_window = optional(number) completion_window = optional(number) lifecycle = optional(object({ cold_storage_after = optional(number) delete_after = optional(number) opt_in_to_archive_for_supported_resources = optional(bool) })) copy_action = optional(object({ destination_vault_arn = optional(string) lifecycle = optional(object({ cold_storage_after = optional(number) delete_after = optional(number) opt_in_to_archive_for_supported_resources = optional(bool) })) })) })) ``` **Default value:** `[ ]`
`selection_conditions` optional
An array of conditions used to specify a set of resources to assign to a backup plan **Type:** ```hcl object({ string_equals = optional(list(object({ key = string value = string })), []) string_like = optional(list(object({ key = string value = string })), []) string_not_equals = optional(list(object({ key = string value = string })), []) string_not_like = optional(list(object({ key = string value = string })), []) }) ``` **Default value:** `{ }`
`selection_tags` optional
An array of tag condition objects used to filter resources based on tags for assigning to a backup plan **Type:** ```hcl list(object({ type = string key = string value = string })) ``` **Default value:** `[ ]`
`vault_enabled` (`bool`) optional
Should we create a new Vault **Default value:** `true`
`vault_name` (`string`) optional
Override target Vault Name **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`backup_plan_arn`
Backup Plan ARN
`backup_plan_version`
Unique, randomly generated, Unicode, UTF-8 encoded string that serves as the version ID of the backup plan
`backup_selection_id`
Backup Selection ID
`backup_vault_arn`
Backup Vault ARN
`backup_vault_id`
Backup Vault ID
`role_arn`
The Amazon Resource Name (ARN) specifying the role
`role_name`
The name of the IAM role created
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label_backup_role` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_backup_plan.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/backup_plan) (resource) - [`aws_backup_selection.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/backup_selection) (resource) - [`aws_backup_vault.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/backup_vault) (resource) - [`aws_backup_vault_lock_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/backup_vault_lock_configuration) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_backup_vault.existing`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/backup_vault) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_role.existing`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_role) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## budgets # Module: `budgets` Terraform module to create [AWS Budgets](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html) and an associated SNS topic and Lambda function to send notifications to Slack. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-budgets/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-budgets/tree/main/test). ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } locals { budgets = [ { name = "budget-ec2-monthly" budget_type = "COST" limit_amount = "1200" limit_unit = "USD" time_period_end = "2087-06-15_00:00" time_unit = "MONTHLY" cost_filter = { Service = ["Amazon Elastic Compute Cloud - Compute"] } notification = { comparison_operator = "GREATER_THAN" threshold = "100" threshold_type = "PERCENTAGE" notification_type = "FORECASTED" } }, { name = "100-total-monthly" budget_type = "COST" limit_amount = "100" limit_unit = "USD" time_unit = "MONTHLY" }, { name = "s3-3GB-limit-monthly" budget_type = "USAGE" limit_amount = "3" limit_unit = "GB" time_unit = "MONTHLY" } ] } module "budgets" { source = "cloudposse/budgets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" budgets = local.budgets # create an SNS topic and lambda for Slack notifications notifications_enabled = true slack_webhook_url = "https://slack-webhook-url" slack_username = "AWS Budgets" slack_channel = "notifications" # encrypt SNS topic, this also creates a KMS CMK that allows `budgets.amazonaws.com` to use it encryption_enabled = true context = module.label.this } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-budgets/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`budgets` (`any`) optional
A list of Budgets to be managed by this module, see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/budgets_budget#argument-reference for a list of possible attributes. For a more specific example, see `examples/complete/fixtures.us-east-2.tfvars` in this repository. **Default value:** `[ ]`
`encryption_enabled` (`bool`) optional
Whether or not to use encryption. If set to `true` and no custom value for KMS key (kms_master_key_id) is provided, a KMS key is created. **Default value:** `true`
`kms_enable_key_rotation` (`bool`) optional
Specifies whether key rotation is enabled **Default value:** `true`
`kms_key_deletion_window_in_days` (`number`) optional
Duration in days after which the key is deleted after destruction of the resources **Default value:** `7`
`kms_master_key_id` (`string`) optional
The ID of a KMS CMK to be used for encryption (see https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-sns-policy.html#protect-sns-sse for appropriate key policies). **Default value:** `null`
`notifications_enabled` (`bool`) optional
Whether or not to setup Slack notifications. Set to `true` to create an SNS topic and Lambda function to send alerts to Slack. **Default value:** `false`
`slack_channel` (`string`) optional
The name of the channel in Slack for notifications. Only used when `notifications_enabled` is `true` **Default value:** `""`
`slack_emoji` (`string`) optional
A custom emoji that will appear on Slack messages **Default value:** `":amazon-aws:"`
`slack_username` (`string`) optional
The username that will appear on Slack messages. Only used when `notifications_enabled` is `true` **Default value:** `""`
`slack_webhook_url` (`string`) optional
The URL of Slack webhook. Only used when `notifications_enabled` is `true` **Default value:** `""`
`vpc_security_group_ids` (`list(string)`) optional
List of security group ids when the notifying Lambda Function should run in the VPC. **Default value:** `null`
`vpc_subnet_ids` (`list(string)`) optional
List of subnet ids when the notifying Lambda Function should run in the VPC. Usually private or intra subnets. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`budgets`
List of Budgets that are being managed by this module
`kms_key_arn`
ARN of the KMS CMK that was created specifically for budget notifications
`kms_key_id`
ID of the KMS CMK that is used for SNS notifications
`lambda_cloudwatch_log_group_arn`
The ARN of the Log Group used by the Slack Notify Lambda
`lambda_function_arn`
The ARN of the Lambda function
`lambda_function_invoke_arn`
The ARN to be used for invoking lambda function from API Gateway
`lambda_function_name`
The name of the Lambda function
`lambda_iam_role_arn`
The ARN of the IAM role used by Lambda function
`sns_topic_arn`
ARN of the SNS topic created for notifications
`sns_topic_name`
The name of the SNS topic created for notifications
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `kms_key` | 0.12.1 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.1) | n/a `slack_notify_lambda` | 0.9.2 | [`cloudposse/sns-lambda-notify-slack/aws`](https://registry.terraform.io/modules/cloudposse/sns-lambda-notify-slack/aws/0.9.2) | n/a `sns_topic` | 0.21.0 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.21.0) | see https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-sns-policy.html `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_budgets_budget.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/budgets_budget) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.kms_key_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## cicd # Module: `cicd` Terraform module to create AWS [`CodePipeline`](https://aws.amazon.com/codepipeline/) with [`CodeBuild`](https://aws.amazon.com/codebuild/) for [`CI/CD`](https://en.wikipedia.org/wiki/CI/CD) This module supports three use-cases: 1. **GitHub -> S3 (build artifact) -> Elastic Beanstalk (running application stack)**. The module gets the code from a ``GitHub`` repository (public or private), builds it by executing the ``buildspec.yml`` file from the repository, pushes the built artifact to an S3 bucket, and deploys the artifact to ``Elastic Beanstalk`` running one of the supported stacks (_e.g._ ``Java``, ``Go``, ``Node``, ``IIS``, ``Python``, ``Ruby``, etc.). - http://docs.aws.amazon.com/codebuild/latest/userguide/sample-maven-5m.html - http://docs.aws.amazon.com/codebuild/latest/userguide/sample-nodejs-hw.html - http://docs.aws.amazon.com/codebuild/latest/userguide/sample-go-hw.html 2. **GitHub -> ECR (Docker image) -> Elastic Beanstalk (running Docker stack)**. The module gets the code from a ``GitHub`` repository, builds a ``Docker`` image from it by executing the ``buildspec.yml`` and ``Dockerfile`` files from the repository, pushes the ``Docker`` image to an ``ECR`` repository, and deploys the ``Docker`` image to ``Elastic Beanstalk`` running ``Docker`` stack. - http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html 3. **GitHub -> ECR (Docker image)**. The module gets the code from a ``GitHub`` repository, builds a ``Docker`` image from it by executing the ``buildspec.yml`` and ``Dockerfile`` files from the repository, and pushes the ``Docker`` image to an ``ECR`` repository. This is used when we want to build a ``Docker`` image from the code and push it to ``ECR`` without deploying to ``Elastic Beanstalk``. To activate this mode, don't specify the ``app`` and ``env`` attributes for the module. - http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html ## Usage Include this repository as a module in your existing terraform code: ```hcl module "build" { source = "cloudposse/cicd/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "staging" name = "app" # Enable the pipeline creation enabled = true # Elastic Beanstalk elastic_beanstalk_application_name = "<(Optional) Elastic Beanstalk application name>" elastic_beanstalk_environment_name = "<(Optional) Elastic Beanstalk environment name>" # Application repository on GitHub github_oauth_token = "(Required) " repo_owner = "" repo_name = "" branch = "" # http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html # http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html build_image = "aws/codebuild/standard:2.0" build_compute_type = "BUILD_GENERAL1_SMALL" # These attributes are optional, used as ENV variables when building Docker images and pushing them to ECR # For more info: # http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html # https://www.terraform.io/docs/providers/aws/r/codebuild_project.html privileged_mode = true region = "us-east-1" aws_account_id = "xxxxxxxxxx" image_repo_name = "ecr-repo-name" image_tag = "latest" # Optional extra environment variables environment_variables = [{ name = "JENKINS_URL" value = "https://jenkins.example.com" }, { name = "COMPANY_NAME" value = "Amazon" }, { name = "TIME_ZONE" value = "Pacific/Auckland" }] } ``` ## Examples ### Example: GitHub, NodeJS, S3 and EB This is an example to build a Node app, store the build artifact to an S3 bucket, and then deploy it to Elastic Beanstalk running ``Node`` stack `buildspec.yml` file ```yaml version: 0.2 phases: install: commands: - echo Starting installation ... pre_build: commands: - echo Installing NPM dependencies... - npm install build: commands: - echo Build started on `date` post_build: commands: - echo Build completed on `date` artifacts: files: - node_modules/**/* - public/**/* - routes/**/* - views/**/* - app.js ``` ### Example: GitHub, NodeJS, Docker, ECR and EB This is an example to build a ``Docker`` image for a Node app, push the ``Docker`` image to an ECR repository, and then deploy it to Elastic Beanstalk running ``Docker`` stack `buildspec.yml` file ```yaml version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - $(aws ecr get-login --region $AWS_REGION) build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $IMAGE_REPO_NAME . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker image to ECR... - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG artifacts: files: - '**/*' ``` `Dockerfile` ```dockerfile FROM node:latest WORKDIR /usr/src/app COPY package.json package-lock.json ./ RUN npm install COPY . . EXPOSE 8081 CMD [ "npm", "start" ] ``` ## Variables ### Required Variables
`branch` (`string`) required
Branch of the GitHub repository, _e.g._ `master`
`github_oauth_token` (`string`) required
GitHub Oauth Token
`repo_name` (`string`) required
GitHub repository name of the application to be built (and deployed to Elastic Beanstalk if configured)
`repo_owner` (`string`) required
GitHub Organization or Person name
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`aws_account_id` (`string`) optional
AWS Account ID. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `""`
`build_compute_type` (`string`) optional
`CodeBuild` instance size. Possible values are: ```BUILD_GENERAL1_SMALL``` ```BUILD_GENERAL1_MEDIUM``` ```BUILD_GENERAL1_LARGE``` **Default value:** `"BUILD_GENERAL1_SMALL"`
`build_image` (`string`) optional
Docker image for build environment, _e.g._ `aws/codebuild/standard:2.0` or `aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0` **Default value:** `"aws/codebuild/standard:2.0"`
`buildspec` (`string`) optional
Declaration to use for building the project. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `""`
`cache_type` (`string`) optional
The type of storage that will be used for the AWS CodeBuild project cache. Valid values: NO_CACHE, LOCAL, and S3. Defaults to S3 to keep same behavior as before upgrading `codebuild` module to 0.18+ version. If cache_type is S3, it will create an S3 bucket for storing codebuild cache inside **Default value:** `"S3"`
`codebuild_cache_bucket_suffix_enabled` (`bool`) optional
The cache bucket generates a random 13 character string to generate a unique bucket name. If set to false it uses terraform-null-label's id value **Default value:** `true`
`elastic_beanstalk_application_name` (`string`) optional
Elastic Beanstalk application name. If not provided or set to empty string, the ``Deploy`` stage of the pipeline will not be created **Default value:** `""`
`elastic_beanstalk_environment_name` (`string`) optional
Elastic Beanstalk environment name. If not provided or set to empty string, the ``Deploy`` stage of the pipeline will not be created **Default value:** `""`
`environment_variables` optional
A list of maps, that contain the keys 'name', 'value', and 'type' to be used as additional environment variables for the build. Valid types are 'PLAINTEXT', 'PARAMETER_STORE', or 'SECRETS_MANAGER' **Type:** ```hcl list(object( { name = string value = string type = string })) ``` **Default value:** ```hcl [ { "name": "NO_ADDITIONAL_BUILD_VARS", "type": "PLAINTEXT", "value": "TRUE" } ] ```
`force_destroy` (`bool`) optional
Force destroy the CI/CD S3 bucket even if it's not empty **Default value:** `false`
`github_webhook_events` (`list(string)`) optional
A list of events which should trigger the webhook. See a list of [available events](https://developer.github.com/v3/activity/events/types/) **Default value:** ```hcl [ "push" ] ```
`github_webhooks_token` (`string`) optional
GitHub OAuth Token with permissions to create webhooks. If not provided, can be sourced from the `GITHUB_TOKEN` environment variable **Default value:** `""`
`image_repo_name` (`string`) optional
ECR repository name to store the Docker image built by this module. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `"UNSET"`
`image_tag` (`string`) optional
Docker image tag in the ECR repository, e.g. 'latest'. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `"latest"`
`poll_source_changes` (`bool`) optional
Periodically check the location of your source content and run the pipeline if changes are detected **Default value:** `true`
`privileged_mode` (`bool`) optional
If set to true, enables running the Docker daemon inside a Docker container on the CodeBuild instance. Used when building Docker images **Default value:** `false`
`region` (`string`) optional
AWS Region, e.g. `us-east-1`. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `""`
`s3_bucket_encryption_enabled` (`bool`) optional
When set to 'true' the 'aws_s3_bucket' resource will have AES256 encryption enabled by default **Default value:** `true`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
`webhook_authentication` (`string`) optional
The type of authentication to use. One of IP, GITHUB_HMAC, or UNAUTHENTICATED **Default value:** `"GITHUB_HMAC"`
`webhook_enabled` (`bool`) optional
Set to false to prevent the module from creating any webhook resources **Default value:** `false`
`webhook_filter_json_path` (`string`) optional
The JSON path to filter on **Default value:** `"$.ref"`
`webhook_filter_match_equals` (`string`) optional
The value to match on (e.g. refs/heads/\{Branch\}) **Default value:** `"refs/heads/{Branch}"`
`webhook_target_action` (`string`) optional
The name of the action in a pipeline you want to connect to the webhook. The action must be from the source (first) stage of the pipeline **Default value:** `"Source"`
`website_bucket_acl` (`string`) optional
Canned ACL of the S3 bucket objects that get served as a website, can be private if using CloudFront with OAI **Default value:** `"public-read"`
`website_bucket_name` (`string`) optional
Name of the S3 bucket where the website will be deployed **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`codebuild_badge_url`
The URL of the build badge when badge_enabled is enabled
`codebuild_cache_bucket_arn`
CodeBuild cache S3 bucket ARN
`codebuild_cache_bucket_name`
CodeBuild cache S3 bucket name
`codebuild_project_id`
CodeBuild project ID
`codebuild_project_name`
CodeBuild project name
`codebuild_role_arn`
CodeBuild IAM Role ARN
`codebuild_role_id`
CodeBuild IAM Role ID
`codepipeline_arn`
CodePipeline ARN
`codepipeline_id`
CodePipeline ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Providers - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `codebuild` | 2.0.1 | [`cloudposse/codebuild/aws`](https://registry.terraform.io/modules/cloudposse/codebuild/aws/2.0.1) | n/a `github_webhook` | 0.12.1 | [`cloudposse/repository-webhooks/github`](https://registry.terraform.io/modules/cloudposse/repository-webhooks/github/0.12.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_codepipeline.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline) (resource) - [`aws_codepipeline_webhook.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline_webhook) (resource) - [`aws_iam_policy.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.codebuild_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`random_password.webhook_secret`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_s3_bucket.website`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## cloudformation-stack # Module: `cloudformation-stack` Terraform module to provision CloudFormation Stack. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-cloudformation-stack/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-cloudformation-stack/tree/main/test). This example creates stack from CIS cloudformation template. ```hcl module "cloudformation_stack" { source = "cloudposse/cloudformation-stack/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" enabled = true namespace = "eg" stage = "prod" name = "app" template_url = "https://aws-quickstart.s3.amazonaws.com/quickstart-compliance-cis-benchmark/templates/main.template" parameters = { NotificationEmailAddressForCloudWatchAlarms = "notify-me@example.com" ConfigureCloudtrail = "Yes" ConfigureConfig = "Yes" ProfileLevel = "Level 2" QSS3BucketName = "aws-quickstart" QSS3KeyPrefix = "quickstart-compliance-cis-benchmark/" } capabilities = ["CAPABILITY_IAM"] } ``` ## Variables ### Required Variables
### Optional Variables
`capabilities` (`list(string)`) optional
A list of capabilities. Valid values: CAPABILITY_IAM, CAPABILITY_NAMED_IAM, CAPABILITY_AUTO_EXPAND **Default value:** `[ ]`
`disable_rollback` (`bool`) optional
Set to true to disable rollback of the stack if stack creation failed. You can specify either on_failure or disable_rollback, but not both. **Default value:** `false`
`on_failure` (`string`) optional
Action to be taken if stack creation fails. This must be one of: `DO_NOTHING`, `ROLLBACK`, or `DELETE` **Default value:** `"ROLLBACK"`
`parameters` (`map(string)`) optional
Key-value map of input parameters for the Stack Set template. (_e.g._ map("BusinessUnit","ABC") **Default value:** `{ }`
`policy_body` (`string`) optional
Structure containing the stack policy body **Default value:** `""`
`template_body` (`string`) optional
Structure containing the CloudFormation template body. Maximum size: 51,200 bytes **Default value:** `null`
`template_url` (`string`) optional
Amazon S3 bucket URL location of a file containing the CloudFormation template body. Maximum file size: 460,800 bytes **Default value:** `null`
`timeout_in_minutes` (`number`) optional
The amount of time that can pass before the stack status becomes `CREATE_FAILED` **Default value:** `30`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
ID of the CloudFormation Stack
`name`
Name of the CloudFormation Stack
`outputs`
Outputs of the CloudFormation Stack
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudformation_stack.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack) (resource) ## Data Sources The following data sources are used by this module: --- ## cloudformation-stack-set # Module: `cloudformation-stack-set` Terraform module to provision Cloudformation Stack Set and Administrator IAM role. ## Usage This example creates a role with the name `cp-prod-app` with permission to assume `` Chamber with S3 bucket as parameter store, and gives permission to the entities specified in `assume_role_arns` to assume the role. ```hcl module "default" { source = "git::https://github.com/cloudposse/terraform-aws-cloudformation-stack-set.git?ref=init" enabled = "true" namespace = "cp" stage = "prod" name = "app" template_url = "https://aws-quickstart.s3.amazonaws.com/quickstart-compliance-cis-benchmark/templates/main.template" executor_role_name = "cis-executor" parameters = { NotificationEmailAddressForCloudWatchAlarms = "notify-me@example.com" ConfigureCloudtrail = "Yes" ConfigureConfig = "Yes" ProfileLevel = "Level 2" QSS3BucketName = "aws-quickstart" QSS3KeyPrefix = "quickstart-compliance-cis-benchmark/" } capabilities = ["CAPABILITY_IAM"] } ``` ## Variables ### Required Variables
`template_url` (`any`) required
Amazon S3 bucket URL location of a file containing the CloudFormation template body. Maximum file size: 460,800 bytes
### Optional Variables
`capabilities` (`list(string)`) optional
A list of capabilities. Valid values: CAPABILITY_IAM, CAPABILITY_NAMED_IAM, CAPABILITY_AUTO_EXPAND **Default value:** `[ ]`
`executor_role_name` (`string`) optional
Name of the IAM Role in all target accounts for Stack Set operations **Default value:** `"AWSCloudFormationStackSetExecutionRole"`
`parameters` (`map(string)`) optional
Key-value map of input parameters for the Stack Set template. (_e.g._ map("BusinessUnit","ABC") **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`namespace` (`string`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (_e.g._ "1") **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter between `name`, `namespace`, `stage` and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `"true"`
`name` (`string`) optional
Name **Required:** No **Default value:** `"cis"`
`tags` (`map(string)`) optional
Additional tags (_e.g._ map("BusinessUnit","ABC") **Required:** No **Default value:** `{ }`
## Outputs
`administrator_role_arn`
Amazon Resource Number (ARN) of the IAM Role in the administrator account
`executor_role_name`
Name of the IAM Role in all target accounts for Stack Set operations
`name`
Name of the Stack Set
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `admin_role` | tags/0.4.0 | [`git::https://github.com/cloudposse/terraform-aws-iam-role.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-aws-iam-role.git/tags/0.4.0) | n/a `label` | tags/0.5.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.5.3) | Define composite variables for resources ## Resources The following resources are used by this module: - [`aws_cloudformation_stack_set.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack_set) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## cloudfront-cdn # Module: `cloudfront-cdn` Terraform Module that implements a CloudFront Distribution (CDN) for a custom origin (e.g. website) and [ships logs to a bucket](https://github.com/cloudposse/terraform-aws-log-storage). If you need to accelerate an S3 bucket, we suggest using [`terraform-aws-cloudfront-s3-cdn`](https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn) instead. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-cloudfront-cdn/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-cloudfront-cdn/tree/main/test). ```hcl module "cdn" { source = "cloudposse/cloudfront-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" aliases = ["www.example.net"] origin_domain_name = "origin.example.com" parent_zone_name = "example.net" } ``` Complete example of setting up CloudFront Distribution with Cache Behaviors for a WordPress site: [`examples/wordpress`](https://github.com/cloudposse/terraform-aws-cloudfront-cdn/tree/main/examples/wordpress) ### Generating ACM Certificate Use the AWS cli to [request new ACM certifiates](http://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request.html) (requires email validation) ``` aws acm request-certificate --domain-name example.com --subject-alternative-names a.example.com b.example.com *.c.example.com ``` ## Variables ### Required Variables
### Optional Variables
`acm_certificate_arn` (`string`) optional
Existing ACM Certificate ARN **Default value:** `""`
`aliases` (`list(string)`) optional
List of aliases. CAUTION! Names MUSTN'T contain trailing `.` **Default value:** `[ ]`
`allowed_methods` (`list(string)`) optional
List of allowed methods (e.g. ` GET, PUT, POST, DELETE, HEAD`) for AWS CloudFront **Default value:** ```hcl [ "DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT" ] ```
`cache_policy_id` (`string`) optional
ID of the cache policy attached to the cache behavior **Default value:** `null`
`cached_methods` (`list(string)`) optional
List of cached methods (e.g. ` GET, PUT, POST, DELETE, HEAD`) **Default value:** ```hcl [ "GET", "HEAD" ] ```
`comment` (`string`) optional
Comment for the origin access identity **Default value:** `"Managed by Terraform"`
`compress` (`bool`) optional
Whether you want CloudFront to automatically compress content for web requests that include Accept-Encoding: gzip in the request header (default: false) **Default value:** `false`
`custom_error_response` optional
List of one or more custom error response element maps **Type:** ```hcl list(object({ error_caching_min_ttl = optional(string, null) error_code = string response_code = optional(string, null) response_page_path = optional(string, null) })) ``` **Default value:** `[ ]`
`custom_header` optional
List of one or more custom headers passed to the origin **Type:** ```hcl list(object({ name = string value = string })) ``` **Default value:** `[ ]`
`custom_origins` optional
One or more custom origins for this distribution (multiples allowed). See documentation for configuration options description https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#origin-arguments **Type:** ```hcl list(object({ domain_name = string origin_id = string origin_path = optional(string, "") origin_access_control_id = optional(string, null) custom_headers = optional(list(object({ name = string value = string })), []) custom_origin_config = optional(object({ http_port = optional(number, 80) https_port = optional(number, 443) origin_protocol_policy = optional(string, "match-viewer") origin_ssl_protocols = optional(list(string), ["TLSv1", "TLSv1.1", "TLSv1.2"]) origin_keepalive_timeout = optional(number, 5) origin_read_timeout = optional(number, 30) }), null) s3_origin_config = optional(object({ origin_access_identity = string }), null) origin_shield = optional(object({ enabled = optional(bool, false) region = optional(string, null) }), null) })) ``` **Default value:** `[ ]`
`default_root_object` (`string`) optional
Object that CloudFront return when requests the root URL **Default value:** `"index.html"`
`default_ttl` (`number`) optional
Default amount of time (in seconds) that an object is in a CloudFront cache **Default value:** `60`
`distribution_enabled` (`bool`) optional
Set to `true` if you want CloudFront to begin processing requests as soon as the distribution is created, or to false if you do not want CloudFront to begin processing requests after the distribution is created. **Default value:** `true`
`dns_aliases_enabled` (`bool`) optional
Set to false to prevent dns records for aliases from being created **Default value:** `true`
`forward_cookies` (`string`) optional
Specifies whether you want CloudFront to forward cookies to the origin. Valid options are all, none or whitelist **Default value:** `"none"`
`forward_cookies_whitelisted_names` (`list(string)`) optional
List of forwarded cookie names **Default value:** `[ ]`
`forward_headers` (`list(string)`) optional
Specifies the Headers, if any, that you want CloudFront to vary upon for this cache behavior. Specify `*` to include all headers. **Default value:** `[ ]`
`forward_query_string` (`bool`) optional
Forward query strings to the origin that is associated with this cache behavior **Default value:** `false`
`function_association` optional
A config block that triggers a CloudFront function with specific actions. See the [aws_cloudfront_distribution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#function-association) documentation for more information. **Type:** ```hcl list(object({ event_type = string function_arn = string })) ``` **Default value:** `[ ]`
`geo_restriction_locations` (`list(string)`) optional
List of country codes for which CloudFront either to distribute content (whitelist) or not distribute your content (blacklist) **Default value:** `[ ]`
`geo_restriction_type` (`string`) optional
Method that use to restrict distribution of your content by country: `none`, `whitelist`, or `blacklist` **Default value:** `"none"`
`grpc_config` optional
The gRPC configuration for the default CloudFront distribution cache behavior **Type:** ```hcl object({ enabled = bool }) ``` **Default value:** ```hcl { "enabled": false } ```
`http_version` (`string`) optional
The maximum HTTP version to support on the distribution. Allowed values are http1.1, http2, http2and3 and http3. **Default value:** `"http2"`
`is_ipv6_enabled` (`bool`) optional
State of CloudFront IPv6 **Default value:** `true`
`lambda_function_association` optional
A config block that triggers a Lambda@Edge function with specific actions **Type:** ```hcl list(object({ event_type = string include_body = optional(bool, false) lambda_arn = string })) ``` **Default value:** `[ ]`
`log_bucket_fqdn` (`string`) optional
Optional fqdn of logging bucket, if not supplied a bucket will be generated. **Default value:** `""`
`log_expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`log_force_destroy` (`bool`) optional
Applies to log bucket created by this module only. If true, all objects will be deleted from the bucket on destroy, so that the bucket can be destroyed without error. These objects are not recoverable. **Default value:** `false`
`log_glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`log_include_cookies` (`bool`) optional
Include cookies in access logs **Default value:** `false`
`log_prefix` (`string`) optional
Path of logs in S3 bucket **Default value:** `""`
`log_standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the glacier tier **Default value:** `30`
`logging_enabled` (`bool`) optional
When true, access logs will be sent to a newly created s3 bucket **Default value:** `true`
`max_ttl` (`number`) optional
Maximum amount of time (in seconds) that an object is in a CloudFront cache **Default value:** `31536000`
`min_ttl` (`number`) optional
Minimum amount of time that you want objects to stay in CloudFront caches **Default value:** `0`
`ordered_cache` optional
An ordered list of cache behaviors resource for this distribution. List from top to bottom in order of precedence. The topmost cache behavior will have precedence 0. The fields can be described by the other variables in this file. For example, the field 'lambda_function_association' in this object has a description in var.lambda_function_association variable earlier in this file. The only difference is that fields on this object are in ordered caches, whereas the rest of the vars in this file apply only to the default cache. Put value `""` on field `target_origin_id` to specify default s3 bucket origin. **Type:** ```hcl list(object({ target_origin_id = string path_pattern = string allowed_methods = optional(list(string), ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]) cached_methods = optional(list(string), ["GET", "HEAD"]) cache_policy_id = optional(string, null) origin_request_policy_id = optional(string, null) compress = optional(bool, false) viewer_protocol_policy = optional(string, "redirect-to-https") min_ttl = optional(number, 0) default_ttl = optional(number, 60) max_ttl = optional(number, 31536000) forward_query_string = optional(bool, false) forward_header_values = optional(list(string), []) forward_cookies = optional(string, "none") forward_cookies_whitelisted_names = optional(list(string), []) response_headers_policy_id = optional(string, "") grpc_config = optional(object({ enabled = bool }), { enabled = false }) lambda_function_association = optional(list(object({ event_type = string include_body = optional(bool, false) lambda_arn = string })), []) function_association = optional(list(object({ event_type = string function_arn = string })), []) })) ``` **Default value:** `[ ]`
`origin_access_control_id` (`string`) optional
CloudFront provides two ways to send authenticated requests to an Amazon S3 origin: origin access control (OAC) and origin access identity (OAI). OAC helps you secure your origins, such as for Amazon S3. **Default value:** `null`
`origin_access_identity_enabled` (`bool`) optional
When true, creates origin access identity resource **Default value:** `true`
`origin_domain_name` (`string`) optional
The DNS domain name of your custom origin (e.g. website) **Default value:** `""`
`origin_http_port` (`number`) optional
The HTTP port the custom origin listens on **Default value:** `"80"`
`origin_https_port` (`number`) optional
The HTTPS port the custom origin listens on **Default value:** `443`
`origin_keepalive_timeout` (`number`) optional
The Custom KeepAlive timeout, in seconds. By default, AWS enforces a limit of 60. But you can request an increase. **Default value:** `5`
`origin_path` (`string`) optional
An optional element that causes CloudFront to request your content from a directory in your Amazon S3 bucket or your custom origin. It must begin with a /. Do not add a / at the end of the path. **Default value:** `""`
`origin_protocol_policy` (`string`) optional
The origin protocol policy to apply to your origin. One of http-only, https-only, or match-viewer **Default value:** `"match-viewer"`
`origin_read_timeout` (`number`) optional
The Custom Read timeout, in seconds. By default, AWS enforces a limit of 60. But you can request an increase. **Default value:** `30`
`origin_request_policy_id` (`string`) optional
ID of the origin request policy attached to the cache behavior **Default value:** `null`
`origin_shield` optional
The CloudFront Origin Shield settings **Type:** ```hcl object({ enabled = optional(bool, false) region = optional(string, null) }) ``` **Default value:** `null`
`origin_ssl_protocols` (`list(string)`) optional
The SSL/TLS protocols that you want CloudFront to use when communicating with your origin over HTTPS **Default value:** ```hcl [ "TLSv1", "TLSv1.1", "TLSv1.2" ] ```
`origin_type` (`string`) optional
The type of origin configuration to use. Valid values are 'custom' or 's3'. **Default value:** `"custom"`
`parent_zone_id` (`string`) optional
ID of the hosted zone to contain this record (or specify `parent_zone_name`) **Default value:** `""`
`parent_zone_name` (`string`) optional
Name of the hosted zone to contain this record (or specify `parent_zone_id`) **Default value:** `""`
`price_class` (`string`) optional
Price class for this distribution: `PriceClass_All`, `PriceClass_200`, `PriceClass_100` **Default value:** `"PriceClass_100"`
`realtime_log_config_arn` (`string`) optional
The ARN of the real-time log configuration that is attached to this cache behavior **Default value:** `null`
`response_headers_policy_id` (`string`) optional
The identifier for a response headers policy **Default value:** `""`
`s3_origin_config` optional
Optional configuration for an S3 origin. **Type:** ```hcl object({ origin_access_identity = string }) ``` **Default value:** `null`
`trusted_signers` (`list(string)`) optional
List of AWS account IDs (or self) that you want to allow to create signed URLs for private content **Default value:** `[ ]`
`viewer_minimum_protocol_version` (`string`) optional
The minimum version of the SSL protocol that you want CloudFront to use for HTTPS connections. This is ignored if the default CloudFront certificate is used. **Default value:** `"TLSv1.2_2021"`
`viewer_protocol_policy` (`string`) optional
allow-all, redirect-to-https **Default value:** `"redirect-to-https"`
`web_acl_id` (`string`) optional
ID of the AWS WAF web ACL that is associated with the distribution **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cf_aliases`
Extra CNAMEs of AWS CloudFront
`cf_arn`
ARN of CloudFront distribution
`cf_domain_name`
Domain name corresponding to the distribution
`cf_etag`
Current version of the distribution's information
`cf_hosted_zone_id`
CloudFront Route 53 Zone ID
`cf_id`
ID of CloudFront distribution
`cf_origin_access_identity`
A shortcut to the full path for the origin access identity to use in CloudFront
`cf_status`
Current status of the distribution
`logs`
Logs resource
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.9.0` - `local`, version: `>= 1.2` ### Providers - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | 0.13.0 | [`cloudposse/route53-alias/aws`](https://registry.terraform.io/modules/cloudposse/route53-alias/aws/0.13.0) | n/a `logs` | 1.4.4 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.4) | n/a `origin_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudfront_distribution.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution) (resource) - [`aws_cloudfront_origin_access_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_origin_access_identity) (resource) ## Data Sources The following data sources are used by this module: --- ## cloudfront-s3-cdn # Module: `cloudfront-s3-cdn` Terraform module to provision an AWS CloudFront CDN with an S3 origin. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-cloudfront-s3-cdn/tree/main/test). The following will create a new s3 bucket `eg-prod-app` for a cloudfront cdn, and allow `principal1` to upload to `prefix1` and `prefix2`, while allowing `principal2` to manage the whole bucket. ```hcl module "cdn" { source = "cloudposse/cloudfront-s3-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" aliases = ["assets.cloudposse.com"] dns_alias_enabled = true parent_zone_name = "cloudposse.com" deployment_principal_arns = { "arn:aws:iam::123456789012:role/principal1" = ["prefix1/", "prefix2/"] "arn:aws:iam::123456789012:role/principal2" = [""] } } ``` The following will reuse an existing s3 bucket `eg-prod-app` for a cloudfront cdn. ```hcl module "cdn" { source = "cloudposse/cloudfront-s3-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" origin_bucket = "eg-prod-app" aliases = ["assets.cloudposse.com"] dns_alias_enabled = true parent_zone_name = "cloudposse.com" name = "eg-prod-app" } ``` The following will create an Origin Group with the origin created by this module as a primary origin and an additional S3 bucket as a failover origin. ```hcl module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = ["failover-assets"] } module "cdn" { source = "cloudposse/cloudfront-s3-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" aliases = ["assets.cloudposse.com"] dns_alias_enabled = true parent_zone_name = "cloudposse.com" s3_origins = [{ domain_name = module.s3_bucket.bucket_regional_domain_name origin_id = module.s3_bucket.bucket_id origin_path = null s3_origin_config = { origin_access_identity = null # will get translated to the origin_access_identity used by the origin created by this module. } }] origin_groups = [{ primary_origin_id = null # will get translated to the origin id of the origin created by this module. failover_origin_id = module.s3_bucket.bucket_id failover_criteria = [ 403, 404, 500, 502 ] }] } ``` ### Background on CDNs, "Origins", S3 Buckets, and Web Servers #### CDNs and Origin Servers There are some settings you need to be aware of when using this module. In order to understand the settings, you need to understand some of the basics of CDNs and web servers, so we are providing this _highly simplified_ explanation of how they work in order for you to understand the implications of the settings you are providing. A "**CDN**" ([Content Distribution Network](https://www.cloudflare.com/learning/cdn/what-is-a-cdn/)) is a collection of servers scattered around the internet with the aim of making it faster for people to retrieve content from a website. The details of why that is wanted/needed are beyond the scope of this document, as are most of the details of how a CDN is implemented. For this discussion, we will simply treat a CDN as a set of web servers all serving the same content to different users. In a normal web server (again, greatly simplified), you place files on the server and the web server software receives requests from browsers and responds with the contents of the files. For a variety of reasons, the web servers in a CDN do not work the way normal web servers work. Instead of getting their content from files on the local server, the CDN web servers get their content by acting like web browsers (proxies). When they get a request from a browser, they make the same request to what is called an "**Origin Server**". It is called an origin server because it _serves_ the original content of the website, and thus is the _origin_ of the content. As a website publisher, you put content on an Origin Server (which users usually should be prevented from accessing) and configure your CDN to use your Origin Server. Then you direct users to a URL hosted by your CDN provider, the users' browsers connect to the CDN, the CDN gets the content from your Origin Server, your Origin Server gets the content from a file on the server, and the data gets sent back hop by hop to the user. (The reason this ends up being a good idea is that the CDN can cache the content for a while, serving multiple users the same content while only contacting the origin server once.) #### S3 Buckets: file storage and web server S3 buckets were originally designed just to store files, and they are still most often used for that. The have a lot of access controls to make it possible to strictly limit who can read what files in the bucket, so that companies can store sensitive information there. You may have heard of a number of "data breaches" being caused by misconfigured permissions on S3 buckets, making them publicly accessible. As a result of that, Amazon has some extra settings on top of everything else to keep S3 buckets from being publicly accessible, which is usually a good thing. However, at some point someone realized that since these files were in the cloud, and Amazon already had these web servers running to provide access to the files in the cloud, it was only a tiny leap to turn an S3 bucket into a web server. So now S3 buckets [can be published as websites](https://docs.aws.amazon.com/AmazonS3/latest/userguide/EnableWebsiteHosting.html) with a few configuration settings, including making the contents publicly accessible. #### Web servers, files, and the different modes of S3 buckets In the simplest websites, the URL "path" (the part after the site name) corresponds directly to the path (under a special directory we will call `/webroot`) and name of a file on the web server. So if the web server gets a request for "http://example.com/foo/bar/baz.html" it will look for a file `/webroot/foo/bar/baz.html`. If it exists, the server will return its contents, and if it does not exist, the server will return a `Not Found` error. An S3 bucket, whether configured as a file store or a website, will always do both of these things. Web servers, however, do some helpful extra things. To name a few: - If the URL ends with a `/`, as in `http://example.com/foo/bar/`, the web server (depending on how it is configured) will either return a list of files in the directory or it will return the contents of a file in the directory with a special name (by default, `index.html`) if it exists. - If the URL does not end with a `/` but the last part, instead of being a file name, is a directory name, the web server will redirect the user to the URL with the `/` at the end instead of saying the file was `Not Found`. This redirect will get you to the `index.html` file we just talked about. Given the way people pass URLs around, this turns out to be quite helpful. - If the URL does not point to a directory or a file, instead of just sending back a cryptic `Not Found` error code, it can return the contents of a special file called an "error document". #### Your Critical Decision: S3 bucket or website? All of this background is to help you decide how to set `website_enabled` and `s3_website_password_enabled`. The default for `website_enabled` is `false` which is the easiest to configure and the most secure, and with this setting, `s3_website_password_enabled` is ignored. S3 buckets, in file storage mode (`website_enabled = false`), do none of these extra things that web servers do. If the URL points to a file, it will return the file, and if it does not _exactly_ match a file, it will return `Not Found`. One big advantage, though, is that the S3 bucket can remain private (not publicly accessible). A second, related advantage is that you can limit the website to a portion of the S3 bucket (everything under a certain prefix) and keep the contents under the the other prefixes private. S3 buckets configured as static websites (`website_enabled = true`), however, have these extra web server features like redirects, `index.html`, and error documents. The disadvantage is that you have to make the entire bucket public (although you can still restrict access to some portions of the bucket). Another feature or drawback (depending on your point of view) of S3 buckets configured as static websites is that they are directly accessible via their [website endpoint](https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteEndpoints.html) as well as through Cloudfront. This module has a feature, `s3_website_password_enabled`, that requires a password be passed in the HTTP request header and configures the CDN to do that, which will make it much harder to access the S3 website directly. So set `s3_website_password_enabled = true` to limit direct access to the S3 website or set it to false if you want to be able to bypass Cloudfront when you want to. In addition to setting `website_enabled=true`, you must also: * Specify at least one `aliases`, like `["example.com"]` or `["example.com", "www.example.com"]` * Specify an ACM certificate ### Custom Domain Names and Generating a TLS Certificate with ACM When you set up Cloudfront, Amazon will generate a domain name for your website. You amost certainly will not want to publish that. Instead, you will want to use a custom domain name. This module refers to them as "aliases". To use the custom domain names, you need to - Pass them in as `aliases` so that Cloudfront will respond to them with your content - Create CNAMEs for the aliases to point to the Cloudfront domain name. If your alias domains are hosted by Route53 and you have IAM permissions to modify them, this module will set that up for you if you set `dns_alias_enabled = true`. - Generate a TLS Certificate via ACM that includes the all the aliases and pass the ARN for the certificate in `acm_certificate_arn`. Note that for Cloudfront, the certificate has to be provisioned in the `us-east-1` region regardless of where any other resources are. ```hcl # For cloudfront, the acm has to be created in us-east-1 or it will not work provider "aws" { region = "us-east-1" alias = "aws.us-east-1" } # create acm and explicitly set it to us-east-1 provider module "acm_request_certificate" { source = "cloudposse/acm-request-certificate/aws" providers = { aws = aws.us-east-1 } # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" domain_name = "example.com" subject_alternative_names = ["a.example.com", "b.example.com", "*.c.example.com"] process_domain_validation_options = true ttl = "300" } module "cdn" { source = "cloudposse/cloudfront-s3-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" aliases = ["assets.cloudposse.com"] dns_alias_enabled = true parent_zone_name = "cloudposse.com" acm_certificate_arn = module.acm_request_certificate.arn depends_on = [module.acm_request_certificate] } ``` Or use the AWS cli to [request new ACM certifiates](http://docs.aws.amazon.com/acm/latest/userguide/gs-acm-request.html) (requires email validation) ``` aws acm request-certificate --domain-name example.com --subject-alternative-names a.example.com b.example.com *.c.example.com ``` __NOTE__: Although AWS Certificate Manager is supported in many AWS regions, to use an SSL certificate with CloudFront, it should be requested only in US East (N. Virginia) region. https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cnames-and-https-requirements.html > If you want to require HTTPS between viewers and CloudFront, you must change the AWS region to US East (N. Virginia) in the AWS Certificate Manager console before you request or import a certificate. https://docs.aws.amazon.com/acm/latest/userguide/acm-regions.html > To use an ACM Certificate with Amazon CloudFront, you must request or import the certificate in the US East (N. Virginia) region. ACM Certificates in this region that are associated with a CloudFront distribution are distributed to all the geographic locations configured for that distribution. This is a fundamental requirement of CloudFront, and you will need to request the certificate in `us-east-1` region. If there are warnings around the outputs when destroying using this module. Then you can use this method for supressing the superfluous errors. `TF_WARN_OUTPUT_ERRORS=1 terraform destroy` #### Lambda@Edge This module also features a Lambda@Edge submodule. Its `lambda_function_association` output is meant to feed directly into the variable of the same name in the parent module. ```hcl provider "aws" { region = var.region } provider "aws" { region = "us-east-1" alias = "us-east-1" } module "lambda_at_edge" { source = "cloudposse/cloudfront-s3-cdn/aws//modules/lambda@edge" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" functions = { origin_request = { source = [{ content = <<-EOT 'use strict'; exports.handler = (event, context, callback) => { //Get contents of response const response = event.Records[0].cf.response; const headers = response.headers; //Set new headers headers['strict-transport-security'] = [{key: 'Strict-Transport-Security', value: 'max-age=63072000; includeSubdomains; preload'}]; headers['content-security-policy'] = [{key: 'Content-Security-Policy', value: "default-src 'none'; img-src 'self'; script-src 'self'; style-src 'self'; object-src 'none'"}]; headers['x-content-type-options'] = [{key: 'X-Content-Type-Options', value: 'nosniff'}]; headers['x-frame-options'] = [{key: 'X-Frame-Options', value: 'DENY'}]; headers['x-xss-protection'] = [{key: 'X-XSS-Protection', value: '1; mode=block'}]; headers['referrer-policy'] = [{key: 'Referrer-Policy', value: 'same-origin'}]; //Return modified response callback(null, response); }; EOT filename = "index.js" }] runtime = "nodejs16.x" handler = "index.handler" memory_size = 128 timeout = 3 event_type = "origin-response" include_body = false } } # An AWS Provider configured for us-east-1 must be passed to the module, as Lambda@Edge functions must exist in us-east-1 providers = { aws = aws.us-east-1 } context = module.this.context } module "cdn" { source = "cloudposse/cloudfront-s3-cdn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ... lambda_function_association = module.lambda_at_edge.lambda_function_association } ``` ## Variables ### Required Variables
### Optional Variables
`access_log_bucket_name` (`string`) optional
DEPRECATED. Use `s3_access_log_bucket_name` instead. **Default value:** `null`
`acm_certificate_arn` (`string`) optional
Existing ACM Certificate ARN **Default value:** `""`
`additional_bucket_policy` (`string`) optional
Additional policies for the bucket. If included in the policies, the variables `${bucket_name}`, `${origin_path}` and `${cloudfront_origin_access_identity_iam_arn}` will be substituted. It is also possible to override the default policy statements by providing statements with `S3GetObjectForCloudFront` and `S3ListBucketForCloudFront` sid. **Default value:** `"{ }"`
`aliases` (`list(string)`) optional
List of FQDN's - Used to set the Alternate Domain Names (CNAMEs) setting on Cloudfront **Default value:** `[ ]`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `true`
`allowed_methods` (`list(string)`) optional
List of allowed methods (e.g. GET, PUT, POST, DELETE, HEAD) for AWS CloudFront **Default value:** ```hcl [ "DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT" ] ```
`block_origin_public_access_enabled` (`bool`) optional
When set to 'true' the s3 origin bucket will have public access block enabled **Default value:** `false`
`bucket_versioning` (`string`) optional
State of bucket versioning option **Default value:** `"Disabled"`
`cache_policy_id` (`string`) optional
The unique identifier of the existing cache policy to attach to the default cache behavior. If not provided, this module will add a default cache policy using other provided inputs. **Default value:** `null`
`cached_methods` (`list(string)`) optional
List of cached methods (e.g. GET, PUT, POST, DELETE, HEAD) **Default value:** ```hcl [ "GET", "HEAD" ] ```
`cloudfront_access_log_bucket_name` (`string`) optional
When `cloudfront_access_log_create_bucket` is `false`, this is the name of the existing S3 Bucket where Cloudfront Access Logs are to be delivered and is required. IGNORED when `cloudfront_access_log_create_bucket` is `true`. **Default value:** `""`
`cloudfront_access_log_create_bucket` (`bool`) optional
When `true` and `cloudfront_access_logging_enabled` is also true, this module will create a new, separate S3 bucket to receive Cloudfront Access Logs. **Default value:** `true`
`cloudfront_access_log_include_cookies` (`bool`) optional
Set true to include cookies in Cloudfront Access Logs **Default value:** `false`
`cloudfront_access_log_prefix` (`string`) optional
Prefix to use for Cloudfront Access Log object keys. Defaults to no prefix. **Default value:** `""`
`cloudfront_access_logging_enabled` (`bool`) optional
Set true to enable delivery of Cloudfront Access Logs to an S3 bucket **Default value:** `true`
`cloudfront_origin_access_control_id` (`string`) optional
CloudFront provides two ways to send authenticated requests to an Amazon S3 origin: origin access control (OAC) and origin access identity (OAI). OAC helps you secure your origins, such as for Amazon S3. **Default value:** `""`
`cloudfront_origin_access_identity_iam_arn` (`string`) optional
Existing cloudfront origin access identity iam arn that is supplied in the s3 bucket policy **Default value:** `""`
`cloudfront_origin_access_identity_path` (`string`) optional
Existing cloudfront origin access identity path used in the cloudfront distribution's s3_origin_config content **Default value:** `""`
`comment` (`string`) optional
Comment for the CloudFront distribution **Default value:** `"Managed by Terraform"`
`compress` (`bool`) optional
Compress content for web requests that include Accept-Encoding: gzip in the request header **Default value:** `true`
`cors_allowed_headers` (`list(string)`) optional
List of allowed headers for S3 bucket **Default value:** ```hcl [ "*" ] ```
`cors_allowed_methods` (`list(string)`) optional
List of allowed methods (e.g. GET, PUT, POST, DELETE, HEAD) for S3 bucket **Default value:** ```hcl [ "GET" ] ```
`cors_allowed_origins` (`list(string)`) optional
List of allowed origins (e.g. example.com, test.com) for S3 bucket **Default value:** `[ ]`
`cors_expose_headers` (`list(string)`) optional
List of expose header in the response for S3 bucket **Default value:** ```hcl [ "ETag" ] ```
`cors_max_age_seconds` (`number`) optional
Time in seconds that browser can cache the response for S3 bucket **Default value:** `3600`
`custom_error_response` optional
List of one or more custom error response element maps **Type:** ```hcl list(object({ error_caching_min_ttl = optional(number, null) error_code = string response_code = optional(number, null) response_page_path = optional(string, null) })) ``` **Default value:** `[ ]`
`custom_origin_headers` (`list(object({ name = string, value = string }))`) optional
A list of origin header parameters that will be sent to origin **Default value:** `[ ]`
`custom_origins` optional
A list of additional custom website [origins](https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#origin-arguments) for this distribution. The `origin_access_control_id` field specifies the Origin Access Control configuration to use for this origin. This is used to configure secure access between CloudFront and the origin. **Type:** ```hcl list(object({ domain_name = string origin_id = string origin_path = optional(string, "") origin_access_control_id = optional(string, null) custom_headers = optional(list(object({ name = string value = string })), []) custom_origin_config = object({ http_port = optional(number, 80) https_port = optional(number, 443) origin_protocol_policy = optional(string, "https-only") origin_ssl_protocols = optional(list(string), ["TLSv1.2"]) origin_keepalive_timeout = optional(number, 5) origin_read_timeout = optional(number, 30) }) origin_shield = optional(object({ enabled = optional(bool, false) region = optional(string, null) }), null) })) ``` **Default value:** `[ ]`
`default_root_object` (`string`) optional
Object that CloudFront return when requests the root URL **Default value:** `"index.html"`
`default_ttl` (`number`) optional
Default amount of time (in seconds) that an object is in a CloudFront cache **Default value:** `60`
`deployment_actions` (`list(string)`) optional
List of actions to permit `deployment_principal_arns` to perform on bucket and bucket prefixes (see `deployment_principal_arns`) **Default value:** ```hcl [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] ```
`deployment_principal_arns` (`map(list(string))`) optional
(Optional) Map of IAM Principal ARNs to lists of S3 path prefixes to grant `deployment_actions` permissions. Resource list will include the bucket itself along with all the prefixes. Prefixes should not begin with '/'. **Default value:** `{ }`
`distribution_enabled` (`bool`) optional
Set to `false` to create the distribution but still prevent CloudFront from serving requests. **Default value:** `true`
`dns_alias_enabled` (`bool`) optional
Create a DNS alias for the CDN. Requires `parent_zone_id` or `parent_zone_name` **Default value:** `false`
`dns_allow_overwrite` (`bool`) optional
Allow creation of DNS records in Terraform to overwrite an existing record, if any. This does not affect the ability to update the record in Terraform and does not prevent other resources within Terraform or manual Route 53 changes outside Terraform from overwriting this record. false by default. This configuration is not recommended for most environments **Default value:** `false`
`encryption_enabled` (`bool`) optional
When set to 'true' the resource will have aes256 encryption enabled by default **Default value:** `true`
`error_document` (`string`) optional
An absolute path to the document to return in case of a 4XX error **Default value:** `""`
`external_aliases` (`list(string)`) optional
List of FQDN's - Used to set the Alternate Domain Names (CNAMEs) setting on Cloudfront. No new route53 records will be created for these **Default value:** `[ ]`
`extra_logs_attributes` (`list(string)`) optional
Additional attributes to add to the end of the generated Cloudfront Access Log S3 Bucket name. Only effective if `cloudfront_access_log_create_bucket` is `true`. **Default value:** ```hcl [ "logs" ] ```
`extra_origin_attributes` (`list(string)`) optional
Additional attributes to put onto the origin label **Default value:** ```hcl [ "origin" ] ```
`forward_cookies` (`string`) optional
Specifies whether you want CloudFront to forward all or no cookies to the origin. Can be 'all' or 'none' **Default value:** `"none"`
`forward_cookies_whitelisted_names` (`list(string)`) optional
List of forwarded cookie names **Default value:** `[ ]`
`forward_header_values` (`list(string)`) optional
A list of whitelisted header values to forward to the origin (incompatible with `cache_policy_id`) **Default value:** ```hcl [ "Access-Control-Request-Headers", "Access-Control-Request-Method", "Origin" ] ```
`forward_query_string` (`bool`) optional
Forward query strings to the origin that is associated with this cache behavior (incompatible with `cache_policy_id`) **Default value:** `false`
`function_association` optional
A config block that triggers a CloudFront function with specific actions. See the [aws_cloudfront_distribution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#function-association) documentation for more information. **Type:** ```hcl list(object({ event_type = string function_arn = string })) ``` **Default value:** `[ ]`
`geo_restriction_locations` (`list(string)`) optional
List of country codes for which CloudFront either to distribute content (whitelist) or not distribute your content (blacklist) **Default value:** `[ ]`
`geo_restriction_type` (`string`) optional
Method that use to restrict distribution of your content by country: `none`, `whitelist`, or `blacklist` **Default value:** `"none"`
`http_version` (`string`) optional
The maximum HTTP version to support on the distribution. Allowed values are http1.1, http2, http2and3 and http3 **Default value:** `"http2"`
`index_document` (`string`) optional
Amazon S3 returns this index document when requests are made to the root domain or any of the subfolders **Default value:** `"index.html"`
`ipv6_enabled` (`bool`) optional
Set to true to enable an AAAA DNS record to be set as well as the A record **Default value:** `true`
`lambda_function_association` optional
A config block that triggers a lambda@edge function with specific actions **Type:** ```hcl list(object({ event_type = string include_body = optional(bool, false) lambda_arn = string })) ``` **Default value:** `[ ]`
`log_expiration_days` (`number`) optional
Number of days after object creation to expire Cloudfront Access Log objects. Only effective if `cloudfront_access_log_create_bucket` is `true`. **Default value:** `90`
`log_glacier_transition_days` (`number`) optional
Number of days after object creation to move Cloudfront Access Log objects to the glacier tier. Only effective if `cloudfront_access_log_create_bucket` is `true`. **Default value:** `60`
`log_include_cookies` (`bool`) optional
DEPRECATED. Use `cloudfront_access_log_include_cookies` instead. **Default value:** `null`
`log_prefix` (`string`) optional
DEPRECATED. Use `cloudfront_access_log_prefix` instead. **Default value:** `null`
`log_standard_transition_days` (`number`) optional
Number of days after object creation to move Cloudfront Access Log objects to the infrequent access tier. Only effective if `cloudfront_access_log_create_bucket` is `true`. **Default value:** `30`
`log_versioning_enabled` (`bool`) optional
Set `true` to enable object versioning in the created Cloudfront Access Log S3 Bucket. Only effective if `cloudfront_access_log_create_bucket` is `true`. **Default value:** `false`
`logging_enabled` (`bool`) optional
DEPRECATED. Use `cloudfront_access_logging_enabled` instead. **Default value:** `null`
`max_ttl` (`number`) optional
Maximum amount of time (in seconds) that an object is in a CloudFront cache **Default value:** `31536000`
`min_ttl` (`number`) optional
Minimum amount of time that you want objects to stay in CloudFront caches **Default value:** `0`
`minimum_protocol_version` (`string`) optional
Cloudfront TLS minimum protocol version. If `var.acm_certificate_arn` is unset, only "TLSv1" can be specified. See: [AWS Cloudfront create-distribution documentation](https://docs.aws.amazon.com/cli/latest/reference/cloudfront/create-distribution.html) and [Supported protocols and ciphers between viewers and CloudFront](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/secure-connections-supported-viewer-protocols-ciphers.html#secure-connections-supported-ciphers) for more information. Defaults to "TLSv1.2_2021" unless `var.acm_certificate_arn` is unset, in which case it defaults to `TLSv1` **Default value:** `""`
`ordered_cache` optional
An ordered list of [cache behaviors](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#cache-behavior-arguments) resource for this distribution. List in order of precedence (first match wins). This is in addition to the default cache policy. Set `target_origin_id` to `""` to specify the S3 bucket origin created by this module. **Type:** ```hcl list(object({ target_origin_id = string path_pattern = string allowed_methods = optional(list(string), ["DELETE", "GET", "HEAD", "OPTIONS", "PATCH", "POST", "PUT"]) cached_methods = optional(list(string), ["GET", "HEAD"]) compress = optional(bool, false) trusted_signers = optional(list(string), []) trusted_key_groups = optional(list(string), []) cache_policy_id = optional(string, null) origin_request_policy_id = optional(string, null) realtime_log_config_arn = optional(string, null) viewer_protocol_policy = optional(string, "redirect-to-https") min_ttl = optional(number, 0) default_ttl = optional(number, 60) max_ttl = optional(number, 31536000) response_headers_policy_id = optional(string, "") grpc_config = optional(object({ enabled = bool }), { enabled = false }) forward_query_string = optional(bool, false) forward_header_values = optional(list(string), []) forward_cookies = optional(string, "none") forward_cookies_whitelisted_names = optional(list(string), []) lambda_function_association = optional(list(object({ event_type = string include_body = optional(bool, false) lambda_arn = string })), []) function_association = optional(list(object({ event_type = string function_arn = string })), []) })) ``` **Default value:** `[ ]`
`origin_access_control_signing_behavior` (`string`) optional
Specifies which requests CloudFront signs. Specify always for the most common use case. Allowed values: always, never, and no-override. **Default value:** `"always"`
`origin_access_type` (`string`) optional
Choose to use `origin_access_control` or `orgin_access_identity` **Default value:** `"origin_access_identity"`
`origin_bucket` (`string`) optional
Name of an existing S3 bucket to use as the origin. If this is not provided, it will create a new s3 bucket using `var.name` and other context related inputs **Default value:** `null`
`origin_force_destroy` (`bool`) optional
Delete all objects from the bucket so that the bucket can be destroyed without error (e.g. `true` or `false`) **Default value:** `false`
`origin_groups` optional
List of [Origin Groups](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#origin-group-arguments) to create in the distribution. The values of `primary_origin_id` and `failover_origin_id` must correspond to origin IDs existing in `var.s3_origins` or `var.custom_origins`. If `primary_origin_id` is set to `null` or `""`, then the origin id of the origin created by this module will be used in its place. This is to allow for the use case of making the origin created by this module the primary origin in an origin group. **Type:** ```hcl list(object({ primary_origin_id = string failover_origin_id = string failover_criteria = list(string) })) ``` **Default value:** `[ ]`
`origin_keepalive_timeout` (`number`) optional
The Custom KeepAlive timeout, in seconds. By default, AWS enforces a limit of 60. But you can request an increase. **Default value:** `5`
`origin_path` (`string`) optional
An optional element that causes CloudFront to request your content from a directory in your Amazon S3 bucket or your custom origin. It must begin with a /. Do not add a / at the end of the path. **Default value:** `""`
`origin_read_timeout` (`number`) optional
The Custom Read timeout, in seconds. By default, AWS enforces a limit of 60. But you can request an increase. **Default value:** `30`
`origin_request_policy_id` (`string`) optional
The unique identifier of the origin request policy that is attached to the behavior. Should be used in conjunction with `cache_policy_id`. **Default value:** `null`
`origin_shield_enabled` (`bool`) optional
If enabled, origin shield will be enabled for the default origin **Default value:** `false`
`origin_ssl_protocols` (`list(string)`) optional
The SSL/TLS protocols that you want CloudFront to use when communicating with your origin over HTTPS. **Default value:** ```hcl [ "TLSv1", "TLSv1.1", "TLSv1.2" ] ```
`override_origin_bucket_policy` (`bool`) optional
When using an existing origin bucket (through var.origin_bucket), setting this to 'false' will make it so the existing bucket policy will not be overriden **Default value:** `true`
`parent_zone_id` (`string`) optional
ID of the hosted zone to contain this record (or specify `parent_zone_name`). Requires `dns_alias_enabled` set to true **Default value:** `null`
`parent_zone_name` (`string`) optional
Name of the hosted zone to contain this record (or specify `parent_zone_id`). Requires `dns_alias_enabled` set to true **Default value:** `""`
`price_class` (`string`) optional
Price class for this distribution: `PriceClass_All`, `PriceClass_200`, `PriceClass_100` **Default value:** `"PriceClass_100"`
`query_string_cache_keys` (`list(string)`) optional
When `forward_query_string` is enabled, only the query string keys listed in this argument are cached (incompatible with `cache_policy_id`) **Default value:** `[ ]`
`realtime_log_config_arn` (`string`) optional
The ARN of the real-time log configuration that is attached to this cache behavior **Default value:** `null`
`redirect_all_requests_to` (`string`) optional
A hostname to redirect all website requests for this distribution to. If this is set, it overrides other website settings **Default value:** `""`
`response_headers_policy_id` (`string`) optional
The identifier for a response headers policy **Default value:** `""`
`routing_rules` (`string`) optional
A json array containing routing rules describing redirect behavior and when redirects are applied **Default value:** `""`
`s3_access_log_bucket_name` (`string`) optional
Name of the existing S3 bucket where S3 Access Logs will be delivered. Default is not to enable S3 Access Logging. **Default value:** `""`
`s3_access_log_prefix` (`string`) optional
Prefix to use for S3 Access Log object keys. Defaults to `logs/${module.this.id}` **Default value:** `""`
`s3_access_logging_enabled` (`bool`) optional
Set `true` to deliver S3 Access Logs to the `s3_access_log_bucket_name` bucket. Defaults to `false` if `s3_access_log_bucket_name` is empty (the default), `true` otherwise. Must be set explicitly if the access log bucket is being created at the same time as this module is being invoked. **Default value:** `null`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control on the origin bucket. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. **Default value:** `"ObjectWriter"`
`s3_origins` optional
A list of S3 [origins](https://www.terraform.io/docs/providers/aws/r/cloudfront_distribution.html#origin-arguments) (in addition to the one created by this module) for this distribution. S3 buckets configured as websites are `custom_origins`, not `s3_origins`. Specifying `s3_origin_config.origin_access_identity` as `null` or `""` will have it translated to the `origin_access_identity` used by the origin created by the module. **Type:** ```hcl list(object({ domain_name = string origin_id = string origin_path = optional(string, "") origin_access_control_id = optional(string, null) s3_origin_config = optional(object({ origin_access_identity = string }), null) origin_shield_enabled = optional(bool, false) })) ``` **Default value:** `[ ]`
`s3_website_password_enabled` (`bool`) optional
If set to true, and `website_enabled` is also true, a password will be required in the `Referrer` field of the HTTP request in order to access the website, and Cloudfront will be configured to pass this password in its requests. This will make it much harder for people to bypass Cloudfront and access the S3 website directly via its website endpoint. **Default value:** `false`
`trusted_key_groups` (`list(string)`) optional
A list of key group IDs that CloudFront can use to validate signed URLs or signed cookies. **Default value:** `[ ]`
`trusted_signers` (`list(string)`) optional
The AWS accounts, if any, that you want to allow to create signed URLs for private content. 'self' is acceptable. **Default value:** `[ ]`
`viewer_protocol_policy` (`string`) optional
Limit the protocol users can use to access content. One of `allow-all`, `https-only`, or `redirect-to-https` **Default value:** `"redirect-to-https"`
`wait_for_deployment` (`bool`) optional
When set to 'true' the resource will wait for the distribution status to change from InProgress to Deployed **Default value:** `true`
`web_acl_id` (`string`) optional
ID of the AWS WAF web ACL that is associated with the distribution **Default value:** `""`
`website_enabled` (`bool`) optional
Set to true to enable the created S3 bucket to serve as a website independently of Cloudfront, and to use that website as the origin. See the README for details and caveats. See also `s3_website_password_enabled`. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aliases`
Aliases of the CloudFront distribution.
`cf_access_control_id`
CloudFront Origin Access Control ID
`cf_arn`
ARN of AWS CloudFront distribution
`cf_domain_name`
Domain name corresponding to the distribution
`cf_etag`
Current version of the distribution's information
`cf_hosted_zone_id`
CloudFront Route 53 zone ID
`cf_id`
ID of AWS CloudFront distribution
`cf_identity_iam_arn`
CloudFront Origin Access Identity IAM ARN
`cf_origin_groups`
List of Origin Groups in the CloudFront distribution.
`cf_origin_ids`
List of Origin IDs in the CloudFront distribution.
`cf_primary_origin_id`
The ID of the origin created by this module.
`cf_s3_canonical_user_id`
Canonical user ID for CloudFront Origin Access Identity
`cf_status`
Current status of the distribution
`logs`
Log bucket resource
`s3_bucket`
Name of origin S3 bucket
`s3_bucket_arn`
ARN of origin S3 bucket
`s3_bucket_domain_name`
Domain of origin S3 bucket
`s3_bucket_policy`
Final computed S3 bucket policy
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 4.9` - `random`, version: `>= 2.2` - `time`, version: `>= 0.7` ### Providers - `aws`, version: `>= 4.9` - `random`, version: `>= 2.2` - `time`, version: `>= 0.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | 0.13.0 | [`cloudposse/route53-alias/aws`](https://registry.terraform.io/modules/cloudposse/route53-alias/aws/0.13.0) | n/a `logs` | 1.4.2 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.2) | n/a `origin_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudfront_distribution.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution) (resource) - [`aws_cloudfront_origin_access_control.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_origin_access_control) (resource) - [`aws_cloudfront_origin_access_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_origin_access_identity) (resource) - [`aws_s3_bucket.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_acl.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) (resource) - [`aws_s3_bucket_cors_configuration.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_cors_configuration) (resource) - [`aws_s3_bucket_ownership_controls.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_ownership_controls) (resource) - [`aws_s3_bucket_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) (resource) - [`aws_s3_bucket_public_access_block.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) (resource) - [`aws_s3_bucket_server_side_encryption_configuration.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) (resource) - [`aws_s3_bucket_versioning.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) (resource) - [`random_password.referer`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`time_sleep.wait_for_aws_s3_bucket_settings`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.combined`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.deployment`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_origin_access_control`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_origin_access_identity`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_ssl_only`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_website_origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_s3_bucket.cf_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) - [`aws_s3_bucket.origin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## Lambda@Edge # Module: `Lambda@Edge` Submodule which creates Lambda@Edge functions to associate with the CloudFront distribution in the parent module. ## Usage The `lambda_function_association` output feeds in directly to the variable of the same name in the parent module. > **Important**: An AWS Provider configured for us-east-1 must be passed to the module, as Lambda@Edge functions must exist > in us-east-1. This is the primary reason why these resources exist in a submodule and not in the parent module. ```hcl module "lambda_at_edge" { source = "cloudposse/cloudfront-s3-cdn/aws//modules/lambda@edge" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" functions = { ... } providers = { aws = aws.us-east-1 } context = module.this.context } ``` ## Variables ### Required Variables
`functions` required
Lambda@Edge functions to create. The key of this map is the name label of the Lambda@Edge function. One of `source`, `source_dir` or `source_zip` should be specified. These variables are mutually exclusive. `source.filename` and `source.content` dictate the name and content of the files that will make up the Lambda function source, respectively. `source_dir` contains path to whole directory that has to be archived. `source_zip` contains path to zip file with lambda source. `runtime`, `handler`, `memory_size` and `timeout` correspond to the attributes of the same name in the [lambda_function](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) resource. See [here](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-at-edge-function-restrictions.html) for Lambda@Edge function restrictions. `additional_policy` contains additional IAM policies for Lambda@Edge function. It's possible to override default policy statement by providing your own statement with `LambdaWriteCloudWatchLogs` sid. `event_type` and `include_body` correspond to the attributes of the same name in the [Lambda Function association block of the cloudfront_distribution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudfront_distribution#lambda-function-association) resource. **Type:** ```hcl map(object({ source = optional(list(object({ filename = string content = string }))) source_dir = optional(string) source_zip = optional(string) runtime = string handler = string memory_size = optional(number, 128) timeout = optional(number, 3) additional_policy = optional(string, "{}") event_type = string include_body = bool })) ```
### Optional Variables
`destruction_delay` (`string`) optional
The delay, in [Golang ParseDuration](https://pkg.go.dev/time#ParseDuration) format, to wait before destroying the Lambda@Edge functions. This delay is meant to circumvent Lambda@Edge functions not being immediately deletable following their dissociation from a CloudFront distribution, since they are replicated to CloudFront Edge servers around the world. If set to `null`, no delay will be introduced. By default, the delay is 20 minutes. This is because it takes about 3 minutes to destroy a CloudFront distribution, and around 15 minutes until the Lambda@Edge function is available for deletion, in most cases. For more information, see: https://github.com/hashicorp/terraform-provider-aws/issues/1721. **Default value:** `"20m"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lambda_function_association`
The Lambda@Edge function association configuration to pass to `var.lambda_function_association` in the parent module.
`lambda_functions`
The Lambda@Edge functions
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `archive`, version: `>= 2.2.0` - `aws`, version: `>= 5.0` - `local`, version: `>= 1.2` - `time`, version: `>= 0.7.0` ### Providers - `archive`, version: `>= 2.2.0` - `aws`, version: `>= 5.0` - `local`, version: `>= 1.2` - `time`, version: `>= 0.7.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `function_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `role` | 0.19.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.19.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lambda_function.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.allow_cloudfront`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`time_sleep.lambda_at_edge_destruction_delay`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) (resource) ## Data Sources The following data sources are used by this module: - [`archive_file.lambda_zip`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`aws_iam_policy_document.lambda_write_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`local_file.lambda_zip`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/data-sources/file) (data source) --- ## cloudtrail(Cloudtrail) # Module: `cloudtrail` Terraform module to provision an AWS [CloudTrail](https://aws.amazon.com/cloudtrail/). The module accepts an encrypted S3 bucket with versioning to store CloudTrail logs. The bucket could be from the same AWS account or from a different account. This is useful if an organization uses a number of separate AWS accounts to isolate the Audit environment from other environments (production, staging, development). In this case, you create CloudTrail in the production environment (production AWS account), while the S3 bucket to store the CloudTrail logs is created in the Audit AWS account, restricting access to the logs only to the users/groups from the Audit account. ## Usage ```hcl module "cloudtrail" { source = "cloudposse/cloudtrail/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" enable_log_file_validation = true include_global_service_events = true is_multi_region_trail = false enable_logging = true s3_bucket_name = "my-cloudtrail-logs-bucket" } ``` __NOTE:__ To create an S3 bucket for CloudTrail logs, use [terraform-aws-cloudtrail-s3-bucket](https://github.com/cloudposse/terraform-aws-cloudtrail-s3-bucket) module. It creates an S3 bucket and an IAM policy to allow CloudTrail logs. ```hcl module "cloudtrail" { source = "cloudposse/cloudtrail/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" enable_log_file_validation = true include_global_service_events = true is_multi_region_trail = false enable_logging = true s3_bucket_name = module.cloudtrail_s3_bucket.bucket_id } module "cloudtrail_s3_bucket" { source = "cloudposse/cloudtrail-s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" } ``` For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-cloudtrail/tree/main/examples/complete). ## Variables ### Required Variables
`s3_bucket_name` (`string`) required
S3 bucket name for CloudTrail logs
### Optional Variables
`advanced_event_selector` optional
Specifies an advanced event selector for enabling data event logging. See: https://www.terraform.io/docs/providers/aws/r/cloudtrail.html for details on this variable **Type:** ```hcl list(object({ name = optional(string) field_selector = list(object({ field = string ends_with = optional(list(string)) not_ends_with = optional(list(string)) equals = optional(list(string)) not_equals = optional(list(string)) starts_with = optional(list(string)) not_starts_with = optional(list(string)) })) })) ``` **Default value:** `[ ]`
`cloud_watch_logs_group_arn` (`string`) optional
Specifies a log group name using an Amazon Resource Name (ARN), that represents the log group to which CloudTrail logs will be delivered **Default value:** `""`
`cloud_watch_logs_role_arn` (`string`) optional
Specifies the role for the CloudWatch Logs endpoint to assume to write to a user’s log group **Default value:** `""`
`enable_log_file_validation` (`bool`) optional
Specifies whether log file integrity validation is enabled. Creates signed digest for validated contents of logs **Default value:** `true`
`enable_logging` (`bool`) optional
Enable logging for the trail **Default value:** `true`
`event_selector` optional
Specifies an event selector for enabling data event logging. See: https://www.terraform.io/docs/providers/aws/r/cloudtrail.html for details on this variable **Type:** ```hcl list(object({ include_management_events = bool read_write_type = string exclude_management_event_sources = optional(set(string)) data_resource = list(object({ type = string values = list(string) })) })) ``` **Default value:** `[ ]`
`include_global_service_events` (`bool`) optional
Specifies whether the trail is publishing events from global services such as IAM to the log files **Default value:** `false`
`insight_selector` optional
Specifies an insight selector for type of insights to log on a trail **Type:** ```hcl list(object({ insight_type = string })) ``` **Default value:** `[ ]`
`is_multi_region_trail` (`bool`) optional
Specifies whether the trail is created in the current region or in all regions **Default value:** `true`
`is_organization_trail` (`bool`) optional
The trail is an AWS Organizations trail **Default value:** `false`
`kms_key_arn` (`string`) optional
Specifies the KMS key ARN to use to encrypt the logs delivered by CloudTrail **Default value:** `""`
`s3_key_prefix` (`string`) optional
Prefix for S3 bucket used by Cloudtrail to store logs **Default value:** `null`
`sns_topic_name` (`string`) optional
Specifies the name of the Amazon SNS topic defined for notification of log file delivery **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudtrail_arn`
The Amazon Resource Name of the trail
`cloudtrail_home_region`
The region in which the trail was created
`cloudtrail_id`
The ID of the trail. (Name for provider < v5, ARN for provider >= v5).
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudtrail.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudtrail) (resource) ## Data Sources The following data sources are used by this module: --- ## cloudtrail-cloudwatch-alarms # Module: `cloudtrail-cloudwatch-alarms` Terraform module for creating alarms for tracking important changes and occurances from cloudtrail. This module creates a set of filter metrics and alarms based on the security best practices covered in the [AWS CIS Foundations Benchmark](https://d0.awsstatic.com/whitepapers/compliance/AWS_CIS_Foundations_Benchmark.pdf) guide. ## Usage ```hcl module "metric_configs" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" map_config_local_base_path = "./catalog" map_config_paths = "*.yaml" context = module.this.context } module "cloudtrail_api_alarms" { source = "cloudposse/cloudtrail-cloudwatch-alarms/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" log_group_name = "${aws_cloudwatch_log_group.default.name}" metrics = module.metric_configs.map_configs } ``` For detailed usage which includes setting up cloudtrail, cloudwatch logs, roles, policies, and the s3 bucket - as well as using this module see the [example directory](https://github.com/cloudposse/terraform-aws-cloudtrail-cloudwatch-alarms/tree/main/examples/complete) For aditional CIS rules and controls https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-cis-controls.html#securityhub-cis-controls-3.8 ## Examples Here's a complete [example](https://github.com/cloudposse/terraform-aws-cloudtrail-cloudwatch-alarms/tree/main/examples/complete/main.tf) of using this `terraform-aws-cloudtrail-cloudwatch-alarms` module. ## Variables ### Required Variables
`log_group_name` (`string`) required
The cloudtrail cloudwatch log group name
### Optional Variables
`additional_endpoint_arns` (`list(string)`) optional
Any alert endpoints, such as autoscaling, or app scaling endpoint arns that will respond to an alert **Default value:** `[ ]`
`dashboard_enabled` (`bool`) optional
When true a dashboard that displays the statistics as a line graph will be created in CloudWatch **Default value:** `true`
`kms_master_key_id` (`string`) optional
The ID or alias of the customer master key (CMK) to use for encrypting the Amazon SNS topic. The CMK must have its resource-based policy allow the service `cloudwatch.amazonaws.com` to perform `kms:Decrypt` and `kms:GenerateDataKey` on it. If this variable is not supplied, a CMK with the sufficient resource-based policy will be created and used when configuring encryption for the SNS topic. **Default value:** `null`
`log_group_region` (`string`) optional
The log group region that should be monitored for unauthorised AWS API Access. Current region used if none provided. **Default value:** `""`
`metric_namespace` (`string`) optional
A namespace for grouping all of the metrics together **Default value:** `"CISBenchmark"`
`metrics` optional
The cloudwatch metrics and corresponding alarm definitions **Type:** ```hcl map(object({ metric_name = string filter_pattern = string metric_namespace = string metric_value = string alarm_name = string alarm_comparison_operator = string alarm_evaluation_periods = string alarm_period = string alarm_statistic = string alarm_treat_missing_data = string alarm_threshold = string alarm_description = string })) ``` **Default value:** `{ }`
`sns_policy_enabled` (`bool`) optional
Attach a policy that allows the notifications through to the SNS topic endpoint **Default value:** `false`
`sns_topic_arn` (`string`) optional
An SNS topic ARN that has already been created. Its policy must already allow access from CloudWatch Alarms, or set `add_sns_policy` to `true` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dashboard_combined`
URL to CloudWatch Combined Metric Dashboard
`dashboard_individual`
URL to CloudWatch Individual Metric Dashboard
`sns_topic_arn`
The ARN of the SNS topic used
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_sns_topic_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sns_kms_key` | 0.10.0 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.10.0) | n/a `sns_kms_key_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_dashboard.combined`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_dashboard) (resource) - [`aws_cloudwatch_dashboard.individual`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_dashboard) (resource) - [`aws_cloudwatch_log_metric_filter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_metric_filter) (resource) - [`aws_cloudwatch_metric_alarm.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_sns_topic.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) (resource) - [`aws_sns_topic_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sns_kms_key_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.sns_topic_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) ## Metrics Tracked | Alarm's Name | Description | |:--------------|:-------------------| | `AuthorizationFailureCount` | Alarms when an unauthorized API call is made. | | `S3BucketActivityEventCount` | Alarms when an API call is made to S3 to put or delete a Bucket, Bucket Policy or Bucket ACL. | | `SecurityGroupEventCount` | Alarms when an API call is made to create, update or delete a Security Group. | | `NetworkAclEventCount` | Alarms when an API call is made to create, update or delete a Network ACL. | | `GatewayEventCount` | Alarms when an API call is made to create, update or delete a Customer or Internet Gateway. | | `VpcEventCount` | Alarms when an API call is made to create, update or delete a VPC, VPC peering connection or VPC connection to classic. | | `EC2InstanceEventCount` | Alarms when an API call is made to create, terminate, start, stop or reboot an EC2 instance. | | `EC2LargeInstanceEventCount` | Alarms when an API call is made to create, terminate, start, stop or reboot a 4x-large or greater EC2 instance. | | `CloudTrailEventCount` | Alarms when an API call is made to create, update or delete a .cloudtrail. trail, or to start or stop logging to a trail. | | `ConsoleSignInFailureCount` | Alarms when an unauthenticated API call is made to sign into the console. | | `IAMPolicyEventCount` | Alarms when an API call is made to change an IAM policy. | | `ConsoleSignInWithoutMfaCount` | Alarms when a user logs into the console without MFA. | | `RootAccountUsageCount` | Alarms when a root account usage is detected. | | `KMSKeyPendingDeletionErrorCount` | Alarms when a customer created KMS key is pending deletion. | | `AWSConfigChangeCount` | Alarms when AWS Config changes. | | `RouteTableChangesCount` | Alarms when route table changes are detected. | ## Dashboard Created Two CloudWatch Dashboards can be created as well, and will be automatically created by default. ![CloudWatch Dashboard](https://github.com/cloudposse/terraform-aws-cloudtrail-cloudwatch-alarms/tree/main/docs/screen1.png) ## Credits The alarm metric names, descriptions, and filters [from this repository were used](https://github.com/TeliaSoneraNorge/telia-terraform-modules/tree/master/cloudtrail-forwarder). With many thanks to [Anton Babenko](https://github.com/antonbabenko) for pointing it out and saving us a lot of time scouring reference documents and describing alarms! --- ## cloudtrail-s3-bucket # Module: `cloudtrail-s3-bucket` Terraform module to provision an S3 bucket with built in policy to allow [CloudTrail](https://aws.amazon.com/cloudtrail/) [logs](https://docs.aws.amazon.com/awscloudtrail/latest/userguide/logging-management-and-data-events-with-cloudtrail.html). This is useful if an organization uses a number of separate AWS accounts to isolate the Audit environment from other environments (production, staging, development). In this case, you create CloudTrail in the production environment (Production AWS account), while the S3 bucket to store the CloudTrail logs is created in the Audit AWS account, restricting access to the logs only to the users/groups from the Audit account. The module supports the following: 1. Forced [server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) at rest for the S3 bucket 2. S3 bucket [versioning](https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html) to easily recover from both unintended user actions and application failures 3. S3 bucket is protected from deletion if it's not empty ([force_destroy](https://www.terraform.io/docs/providers/aws/r/s3_bucket.html#force_destroy) set to `false`) ## Usage ```hcl module "s3_bucket" { source = "cloudposse/cloudtrail-s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "cluster" } ``` ## Variables ### Required Variables
### Optional Variables
`abort_incomplete_multipart_upload_days` (`number`) optional
Maximum time (in days) that you want to allow multipart uploads to remain in progress **Default value:** `5`
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`acl` (`string`) optional
The canned ACL to apply. We recommend log-delivery-write for compatibility with AWS services **Default value:** `"log-delivery-write"`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `true`
`block_public_acls` (`bool`) optional
Set to `false` to disable the blocking of new public access lists on the bucket **Default value:** `true`
`block_public_policy` (`bool`) optional
Set to `false` to disable the blocking of new public policies on the bucket **Default value:** `true`
`bucket_notifications_enabled` (`bool`) optional
Send notifications for the object created events. Used for 3rd-party log collection from a bucket. This does not affect access log bucket created by this module. To enable bucket notifications on the access log bucket, create it separately using the cloudposse/s3-log-storage/aws **Default value:** `false`
`bucket_notifications_prefix` (`string`) optional
Prefix filter. Used to manage object notifications **Default value:** `""`
`bucket_notifications_type` (`string`) optional
Type of the notification configuration. Only SQS is supported. **Default value:** `"SQS"`
`create_access_log_bucket` (`bool`) optional
A flag to indicate if a bucket for s3 access logs should be created **Default value:** `false`
`enable_glacier_transition` (`bool`) optional
Glacier transition might just increase your bill. Set to false to disable lifecycle transitions to AWS Glacier. **Default value:** `false`
`expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`force_destroy` (`bool`) optional
(Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`ignore_public_acls` (`bool`) optional
Set to `false` to disable the ignoring of public access lists on the bucket **Default value:** `true`
`kms_master_key_arn` (`string`) optional
The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms **Default value:** `""`
`lifecycle_prefix` (`string`) optional
Prefix filter. Used to manage object lifecycle events **Default value:** `""`
`lifecycle_rule_enabled` (`bool`) optional
Enable lifecycle events on this bucket **Default value:** `true`
`lifecycle_tags` (`map(string)`) optional
Tags filter. Used to manage object lifecycle events **Default value:** `{ }`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Specifies when noncurrent object versions transitions **Default value:** `30`
`policy` (`string`) optional
A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy **Default value:** `""`
`restrict_public_buckets` (`bool`) optional
Set to `false` to disable the restricting of making the bucket public **Default value:** `true`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are AES256 and aws:kms **Default value:** `"AES256"`
`standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
Bucket ARN
`bucket_domain_name`
FQDN of bucket
`bucket_id`
Bucket ID
`bucket_notifications_sqs_queue_arn`
Notifications SQS queue ARN
`prefix`
Prefix configured for lifecycle rules
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `access_log_bucket_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `s3_access_log_bucket` | 1.4.5 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.5) | n/a `s3_bucket` | 1.4.5 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.5) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## cloudwatch-events # Module: `cloudwatch-events` This is `terraform-aws-cloudwatch-events` module that creates CloudWatch Events rules and according targets. > Amazon CloudWatch Events delivers a near real-time stream of system events that describe changes in Amazon Web Services (AWS) resources. Using simple rules that you can quickly set up, you can match events and route them to one or more target functions or streams. CloudWatch Events becomes aware of operational changes as they occur. CloudWatch Events responds to these operational changes and takes corrective action as necessary, by sending messages to respond to the environment, activating functions, making changes, and capturing state information. ## Usage Here's how to invoke this example module in your projects ```hcl module "cloudwatch_event" { source = "cloudposse/cloudwatch-events/aws" version = "0.7.0" name = var.name namespace = var.namespace tenant = var.tenant environment = var.environment stage = var.stage cloudwatch_event_rule_description = var.cloudwatch_event_rule_description cloudwatch_event_rule_pattern = var.cloudwatch_event_rule_pattern_json cloudwatch_event_target_arn = module.sns.sns_topic.arn } ``` ## Variables ### Required Variables
`cloudwatch_event_rule_pattern` (`any`) required
Event pattern described a HCL map which will be encoded as JSON with jsonencode function. See full documentation of CloudWatch Events and Event Patterns for details. http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/CloudWatchEventsandEventPatterns.html
`cloudwatch_event_target_arn` (`string`) required
The Amazon Resource Name (ARN) associated of the target.
### Optional Variables
`cloudwatch_event_rule_description` (`string`) optional
The description of the rule. **Default value:** `""`
`cloudwatch_event_rule_is_enabled` (`bool`) optional
DEPRECATED (use `cloudwatch_event_rule_state` instead): Whether the rule should be enabled. Conflicts with `cloudwatch_event_rule_is_enabled` **Default value:** `null`
`cloudwatch_event_rule_state` (`string`) optional
State of the rule. Valid values are DISABLED, ENABLED, and ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS. When state is ENABLED, the rule is enabled for all events except those delivered by CloudTrail. To also enable the rule for events delivered by CloudTrail, set state to ENABLED_WITH_ALL_CLOUDTRAIL_MANAGEMENT_EVENTS. **Default value:** `"ENABLED"`
`cloudwatch_event_target_id` (`string`) optional
The unique target assignment ID. If missing, will generate a random, unique id. **Default value:** `null`
`cloudwatch_event_target_role_arn` (`string`) optional
IAM role to be used for this target when the rule is triggered. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_cloudwatch_event_rule_arn`
The Amazon Resource Name (ARN) of the rule.
`aws_cloudwatch_event_rule_id`
The name of the rule
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `rule_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) ## Data Sources The following data sources are used by this module: --- ## cloudwatch-flow-logs # Module: `cloudwatch-flow-logs` Terraform module for enabling [`flow logs`](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/flow-logs.html) for `vpc` and `subnets`. ## Usage ```terraform module "flow_logs" { source = "git::https://github.com/cloudposse/terraform-aws-cloudwatch-flow-logs.git?ref=master" vpc_id = "${var.vpc_id}" namespace = "${var.namespace}" stage = "${var.stage}" } ``` ## Variables ### Required Variables
`vpc_id` (`any`) required
ID of VPC
### Optional Variables
`encryption_type` (`string`) optional
GUID for the customer-managed KMS key to use for encryption. The only acceptable values are NONE or KMS **Default value:** `"NONE"`
`eni_ids` (`list(string)`) optional
IDs of ENIs **Default value:** `[ ]`
`filter_pattern` (`string`) optional
Valid CloudWatch Logs filter pattern for subscribing to a filtered stream of log events **Default value:** `"[version, account, eni, source, destination, srcport, destport, protocol, packets, bytes, windowstart, windowend, action, flowlogstatus]"`
`kms_key_id` (`string`) optional
ID of KMS key **Default value:** `""`
`region` (`string`) optional
AWS region **Default value:** `""`
`retention_in_days` (`string`) optional
Number of days you want to retain log events in the log group **Default value:** `"30"`
`retention_period` (`string`) optional
Length of time data records are accessible after they are added to the stream **Default value:** `"48"`
`shard_count` (`string`) optional
Number of shards that the stream will use **Default value:** `"1"`
`shard_level_metrics` (`list`) optional
List of shard-level CloudWatch metrics which can be enabled for the stream **Default value:** ```hcl [ "IncomingBytes", "OutgoingBytes" ] ```
`subnet_ids` (`list(string)`) optional
IDs of subnets **Default value:** `[ ]`
`traffic_type` (`string`) optional
Type of traffic to capture. Valid values: ACCEPT,REJECT, ALL **Default value:** `"ALL"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`namespace` (`string`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `policy` or `role`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage`, etc. **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Set to false to prevent the module from creating anything **Required:** No **Default value:** `"true"`
`name` (`string`) optional
Name (e.g. `bastion` or `db`) **Required:** No **Default value:** `""`
`tags` (`map(string)`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`eni_flow_ids`
Flow Log IDs of ENIs
`kinesis_arn`
Kinesis Stream ARN
`kinesis_id`
Kinesis Stream ID
`kinesis_name`
Kinesis Stream name
`kinesis_shard_count`
Kinesis Stream Shard count
`log_group_arn`
ARN of the log group
`subnet_flow_ids`
Flow Log IDs of subnets
`vpc_flow_id`
VPC Flow Log ID
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `kinesis_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `log_group_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `subnet_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `subscription_filter_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `vpc_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_cloudwatch_log_subscription_filter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) (resource) - [`aws_flow_log.eni`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) (resource) - [`aws_flow_log.subnets`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) (resource) - [`aws_flow_log.vpc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) (resource) - [`aws_iam_role.kinesis`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.kinesis`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy.log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_kinesis_stream.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.kinesis`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.kinesis_assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.log_assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## cloudwatch-logs(Cloudwatch-logs) # Module: `cloudwatch-logs` Terraform module for creation of CloudWatch Log Streams and Log Groups. Useful in combination with Fluentd/Fluent-bit for shipping logs. ## Usage ```terraform module "cloudwatch_logs" { source = "cloudposse/cloudwatch-logs/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" stream_names = ["kafka-instance-1", "kafka-instance-2"] } ``` ## Examples Here's a complete [example](https://github.com/cloudposse/terraform-aws-cloudwatch-logs/tree/main/examples/complete/main.tf) of using this `terraform-aws-cloudwatch-logs` module. ## Variables ### Required Variables
### Optional Variables
`additional_permissions` (`list(string)`) optional
Additional permissions granted to the IAM role **Default value:** ```hcl [ "logs:CreateLogStream", "logs:DeleteLogStream" ] ```
`iam_role_enabled` (`bool`) optional
Whether to create an IAM role which is able to write logs to the CloudWatch Logs log group **Default value:** `true`
`iam_tags_enabled` (`string`) optional
Enable/disable tags on IAM roles and policies **Default value:** `true`
`kms_key_arn` (`string`) optional
The ARN of the KMS Key to use when encrypting log data. Please note, after the AWS KMS CMK is disassociated from the log group, AWS CloudWatch Logs stops encrypting newly ingested data for the log group. All previously ingested data remains encrypted, and AWS CloudWatch Logs requires permissions for the CMK whenever the encrypted data is requested. **Default value:** `""`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the IAM role **Default value:** `""`
`principals` (`map(any)`) optional
Map of service name as key and a list of ARNs to allow assuming the role as value. (e.g. map(`AWS`, list(`arn:aws:iam:::role/admin`))) **Default value:** ```hcl { "Service": [ "ec2.amazonaws.com" ] } ```
`retention_in_days` (`string`) optional
Number of days you want to retain log events in the log group **Default value:** `"30"`
`stream_names` (`list(string)`) optional
Names of streams **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`log_group_arn`
ARN of the log group
`log_group_name`
Name of log group
`role_arn`
ARN of the IAM role
`role_name`
Name of the IAM role
`stream_arns`
ARNs of the log streams
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `log_group_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `role` | 0.19.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.19.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_cloudwatch_log_stream.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_stream) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.log_agent`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## code-deploy # Module: `code-deploy` Terraform module to provision AWS Code Deploy app and group. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-code-deploy/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-code-deploy/tree/main/test). ```hcl module "example" { source = "cloudposse/code-deploy/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-code-deploy/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`alarm_configuration` optional
Configuration of deployment to stop when a CloudWatch alarm detects that a metric has fallen below or exceeded a defined threshold. alarms: A list of alarms configured for the deployment group. ignore_poll_alarm_failure: Indicates whether a deployment should continue if information about the current state of alarms cannot be retrieved from CloudWatch. **Type:** ```hcl object({ alarms = list(string) ignore_poll_alarm_failure = bool }) ``` **Default value:** `null`
`auto_rollback_configuration_events` (`string`) optional
The event type or types that trigger a rollback. Supported types are `DEPLOYMENT_FAILURE` and `DEPLOYMENT_STOP_ON_ALARM`. **Default value:** `"DEPLOYMENT_FAILURE"`
`autoscaling_groups` (`list(string)`) optional
A list of Autoscaling Groups associated with the deployment group. **Default value:** `[ ]`
`blue_green_deployment_config` (`any`) optional
Configuration block of the blue/green deployment options for a deployment group, see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_deployment_group#blue_green_deployment_config **Default value:** `null`
`compute_platform` (`string`) optional
The compute platform can either be `ECS`, `Lambda`, or `Server` **Default value:** `"ECS"`
`create_default_service_role` (`bool`) optional
Whether to create default IAM role ARN that allows deployments. **Default value:** `true`
`create_default_sns_topic` (`bool`) optional
Whether to create default SNS topic through which notifications are sent. **Default value:** `true`
`deployment_style` optional
Configuration of the type of deployment, either in-place or blue/green, you want to run and whether to route deployment traffic behind a load balancer. deployment_option: Indicates whether to route deployment traffic behind a load balancer. Possible values: `WITH_TRAFFIC_CONTROL`, `WITHOUT_TRAFFIC_CONTROL`. deployment_type: Indicates whether to run an in-place deployment or a blue/green deployment. Possible values: `IN_PLACE`, `BLUE_GREEN`. **Type:** ```hcl object({ deployment_option = string deployment_type = string }) ``` **Default value:** `null`
`ec2_tag_filter` optional
The Amazon EC2 tags on which to filter. The deployment group includes EC2 instances with any of the specified tags. Cannot be used in the same call as ec2TagSet. **Type:** ```hcl set(object({ key = string type = string value = string })) ``` **Default value:** `[ ]`
`ec2_tag_set` optional
A list of sets of tag filters. If multiple tag groups are specified, any instance that matches to at least one tag filter of every tag group is selected. key: The key of the tag filter. type: The type of the tag filter, either `KEY_ONLY`, `VALUE_ONLY`, or `KEY_AND_VALUE`. value: The value of the tag filter. **Type:** ```hcl set(object( { ec2_tag_filter = set(object( { key = string type = string value = string } )) } )) ``` **Default value:** `[ ]`
`ecs_service` optional
Configuration block(s) of the ECS services for a deployment group. cluster_name: The name of the ECS cluster. service_name: The name of the ECS service. **Type:** ```hcl list(object({ cluster_name = string service_name = string })) ``` **Default value:** `null`
`load_balancer_info` (`map(any)`) optional
Single configuration block of the load balancer to use in a blue/green deployment, see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_deployment_group#load_balancer_info **Default value:** `null`
`minimum_healthy_hosts` optional
type: The type can either be `FLEET_PERCENT` or `HOST_COUNT`. value: The value when the type is `FLEET_PERCENT` represents the minimum number of healthy instances as a percentage of the total number of instances in the deployment. When the type is `HOST_COUNT`, the value represents the minimum number of healthy instances as an absolute value. **Type:** ```hcl object({ type = string value = number }) ``` **Default value:** `null`
`service_role_arn` (`string`) optional
The service IAM role ARN that allows deployments. **Default value:** `null`
`sns_topic_arn` (`string`) optional
The ARN of the SNS topic through which notifications are sent. **Default value:** `null`
`traffic_routing_config` optional
type: Type of traffic routing config. One of `TimeBasedCanary`, `TimeBasedLinear`, `AllAtOnce`. interval: The number of minutes between the first and second traffic shifts of a deployment. percentage: The percentage of traffic to shift in the first increment of a deployment. **Type:** ```hcl object({ type = string interval = number percentage = number }) ``` **Default value:** `null`
`trigger_events` (`list(string)`) optional
The event type or types for which notifications are triggered. Some values that are supported: `DeploymentStart`, `DeploymentSuccess`, `DeploymentFailure`, `DeploymentStop`, `DeploymentRollback`, `InstanceStart`, `InstanceSuccess`, `InstanceFailure`. See the CodeDeploy documentation for all possible values. http://docs.aws.amazon.com/codedeploy/latest/userguide/monitoring-sns-event-notifications-create-trigger.html **Default value:** ```hcl [ "DeploymentFailure" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`deployment_config_id`
The deployment config ID.
`deployment_config_name`
The deployment group's config name.
`group_id`
The application group ID.
`id`
The application ID.
`name`
The application's name.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2.0` - `local`, version: `>= 1.2` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `sns_topic` | 0.21.0 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.21.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_codedeploy_app.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_app) (resource) - [`aws_codedeploy_deployment_config.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_deployment_config) (resource) - [`aws_codedeploy_deployment_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_deployment_group) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## codebuild # Module: `codebuild` Terraform module to create AWS CodeBuild project for AWS CodePipeline. ## Usage Include this module in your existing terraform code: ```hcl module "build" { source = "cloudposse/codebuild/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "staging" name = "app" # https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html build_image = "aws/codebuild/standard:2.0" build_compute_type = "BUILD_GENERAL1_SMALL" build_timeout = 60 # These attributes are optional, used as ENV variables when building Docker images and pushing them to ECR # For more info: # http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html # https://www.terraform.io/docs/providers/aws/r/codebuild_project.html privileged_mode = true aws_region = "us-east-1" aws_account_id = "xxxxxxxxxx" image_repo_name = "ecr-repo-name" image_tag = "latest" # Optional extra environment variables environment_variables = [ { name = "JENKINS_URL" value = "https://jenkins.example.com" type = "PLAINTEXT" }, { name = "COMPANY_NAME" value = "Amazon" type = "PLAINTEXT" }, { name = "TIME_ZONE" value = "Pacific/Auckland" type = "PLAINTEXT" } ] } ``` ## Variables ### Required Variables
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`artifact_location` (`string`) optional
Location of artifact. Applies only for artifact of type S3 **Default value:** `""`
`artifact_type` (`string`) optional
The build output artifact's type. Valid values for this parameter are: CODEPIPELINE, NO_ARTIFACTS or S3 **Default value:** `"CODEPIPELINE"`
`aws_account_id` (`string`) optional
(Optional) AWS Account ID. Used as CodeBuild ENV variable when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html **Default value:** `""`
`aws_region` (`string`) optional
(Optional) AWS Region, e.g. us-east-1. Used as CodeBuild ENV variable when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html **Default value:** `""`
`badge_enabled` (`bool`) optional
Generates a publicly-accessible URL for the projects build badge. Available as badge_url attribute when enabled **Default value:** `false`
`build_compute_type` (`string`) optional
Instance type of the build instance **Default value:** `"BUILD_GENERAL1_SMALL"`
`build_image` (`string`) optional
Docker image for build environment, e.g. 'aws/codebuild/standard:2.0' or 'aws/codebuild/eb-nodejs-6.10.0-amazonlinux-64:4.0.0'. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref.html **Default value:** `"aws/codebuild/standard:2.0"`
`build_image_pull_credentials_type` (`string`) optional
Type of credentials AWS CodeBuild uses to pull images in your build.Valid values: CODEBUILD, SERVICE_ROLE. When you use a cross-account or private registry image, you must use SERVICE_ROLE credentials. **Default value:** `"CODEBUILD"`
`build_timeout` (`number`) optional
How long in minutes, from 5 to 480 (8 hours), for AWS CodeBuild to wait until timing out any related build that does not get marked as completed **Default value:** `60`
`build_type` (`string`) optional
The type of build environment, e.g. 'LINUX_CONTAINER' or 'WINDOWS_CONTAINER' **Default value:** `"LINUX_CONTAINER"`
`buildspec` (`string`) optional
Optional buildspec declaration to use for building the project **Default value:** `""`
`cache_bucket_suffix_enabled` (`bool`) optional
The cache bucket generates a random 13 character string to generate a unique bucket name. If set to false it uses terraform-null-label's id value. It only works when cache_type is 'S3 **Default value:** `true`
`cache_expiration_days` (`number`) optional
How many days should the build cache be kept. It only works when cache_type is 'S3' **Default value:** `7`
`cache_type` (`string`) optional
The type of storage that will be used for the AWS CodeBuild project cache. Valid values: NO_CACHE, LOCAL, and S3. Defaults to NO_CACHE. If cache_type is S3, it will create an S3 bucket for storing codebuild cache inside **Default value:** `"NO_CACHE"`
`concurrent_build_limit` (`number`) optional
Specify a maximum number of concurrent builds for the project. The value specified must be greater than 0 and less than the account concurrent running builds limit. **Default value:** `null`
`custom_policy` (`list(string)`) optional
JSON encoded IAM policy to add to the IAM service account permissions. **Default value:** `[ ]`
`default_permissions_enabled` (`bool`) optional
When 'true' default base IAM permissions to get up and running with CodeBuild are attached. Those who want a least privileged policy can instead set to `false` and use the `custom_policy` variable. **Default value:** `true`
`description` (`string`) optional
Short description of the CodeBuild project **Default value:** `"Managed by Terraform"`
`encryption_enabled` (`bool`) optional
When set to 'true' the resource will have AES256 encryption enabled by default **Default value:** `false`
`encryption_key` (`string`) optional
AWS Key Management Service (AWS KMS) customer master key (CMK) to be used for encrypting the build project's build output artifacts. **Default value:** `null`
`environment_variables` optional
A list of maps, that contain the keys 'name', 'value', and 'type' to be used as additional environment variables for the build. Valid types are 'PLAINTEXT', 'PARAMETER_STORE', or 'SECRETS_MANAGER' **Type:** ```hcl list(object( { name = string value = string type = string } )) ``` **Default value:** ```hcl [ { "name": "NO_ADDITIONAL_BUILD_VARS", "type": "PLAINTEXT", "value": "TRUE" } ] ```
`extra_permissions` (`list(string)`) optional
List of action strings which will be added to IAM service account permissions. Only used if `default_permissions_enabled` is set to true. **Default value:** `[ ]`
`fetch_git_submodules` (`bool`) optional
If set to true, fetches Git submodules for the AWS CodeBuild build project. **Default value:** `false`
`file_system_locations` (`any`) optional
A set of file system locations to to mount inside the build. File system locations are documented below. **Default value:** `{ }`
`git_clone_depth` (`number`) optional
Truncate git history to this many commits. **Default value:** `null`
`github_token` (`string`) optional
(Optional) GitHub auth token environment variable (`GITHUB_TOKEN`) **Default value:** `""`
`github_token_type` (`string`) optional
Storage type of GITHUB_TOKEN environment variable (`PARAMETER_STORE`, `PLAINTEXT`, `SECRETS_MANAGER`) **Default value:** `"PARAMETER_STORE"`
`iam_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role. **Default value:** `null`
`iam_policy_path` (`string`) optional
Path to the policy. **Default value:** `"/service-role/"`
`iam_role_path` (`string`) optional
Path to the role. **Default value:** `null`
`image_repo_name` (`string`) optional
(Optional) ECR repository name to store the Docker image built by this module. Used as CodeBuild ENV variable when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html **Default value:** `"UNSET"`
`image_tag` (`string`) optional
(Optional) Docker image tag in the ECR repository, e.g. 'latest'. Used as CodeBuild ENV variable when building Docker images. For more info: http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html **Default value:** `"latest"`
`local_cache_modes` (`list(string)`) optional
Specifies settings that AWS CodeBuild uses to store and reuse build dependencies. Valid values: LOCAL_SOURCE_CACHE, LOCAL_DOCKER_LAYER_CACHE, and LOCAL_CUSTOM_CACHE **Default value:** `[ ]`
`logs_config` (`any`) optional
Configuration for the builds to store log data to CloudWatch or S3. **Default value:** `{ }`
`private_repository` (`bool`) optional
Set to true to login into private repository with credentials supplied in source_credential variable. **Default value:** `false`
`privileged_mode` (`bool`) optional
(Optional) If set to true, enables running the Docker daemon inside a Docker container on the CodeBuild instance. Used when building Docker images **Default value:** `false`
`report_build_status` (`bool`) optional
Set to true to report the status of a build's start and finish to your source provider. This option is only valid when the source_type is BITBUCKET or GITHUB **Default value:** `false`
`s3_cache_bucket_name` (`string`) optional
Use an existing s3 bucket name for cache. Relevant if `cache_type` is set to `S3`. **Default value:** `null`
`secondary_artifact_encryption_enabled` (`bool`) optional
Set to true to enable encryption on the secondary artifact bucket **Default value:** `false`
`secondary_artifact_identifier` (`string`) optional
Secondary artifact identifier. Must match the identifier in the build spec **Default value:** `null`
`secondary_artifact_location` (`string`) optional
Location of secondary artifact. Must be an S3 reference **Default value:** `null`
`secondary_sources` optional
(Optional) secondary source for the codebuild project in addition to the primary location **Type:** ```hcl list(object( { git_clone_depth = number location = string source_identifier = string type = string fetch_submodules = bool insecure_ssl = bool report_build_status = bool })) ``` **Default value:** `[ ]`
`source_credential_auth_type` (`string`) optional
The type of authentication used to connect to a GitHub, GitHub Enterprise, or Bitbucket repository. **Default value:** `"PERSONAL_ACCESS_TOKEN"`
`source_credential_server_type` (`string`) optional
The source provider used for this project. **Default value:** `"GITHUB"`
`source_credential_token` (`string`) optional
For GitHub or GitHub Enterprise, this is the personal access token. For Bitbucket, this is the app password. **Default value:** `""`
`source_credential_user_name` (`string`) optional
The Bitbucket username when the authType is BASIC_AUTH. This parameter is not valid for other types of source providers or connections. **Default value:** `""`
`source_location` (`string`) optional
The location of the source code from git or s3 **Default value:** `""`
`source_type` (`string`) optional
The type of repository that contains the source code to be built. Valid values for this parameter are: CODECOMMIT, CODEPIPELINE, GITHUB, GITHUB_ENTERPRISE, BITBUCKET or S3 **Default value:** `"CODEPIPELINE"`
`source_version` (`string`) optional
A version of the build input to be built for this project. If not specified, the latest version is used. **Default value:** `""`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
`vpc_config` (`any`) optional
Configuration for the builds to run inside a VPC. **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`badge_url`
The URL of the build badge when badge_enabled is enabled
`cache_bucket_arn`
Cache S3 bucket ARN
`cache_bucket_name`
Cache S3 bucket name
`project_arn`
Project ARN
`project_id`
Project ID
`project_name`
Project name
`role_arn`
IAM Role ARN
`role_id`
IAM Role ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Providers - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_codebuild_project.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) (resource) - [`aws_codebuild_source_credential.authorization`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_source_credential) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.default_cache_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.default_cache_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.cache_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) (resource) - [`aws_s3_bucket_lifecycle_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration) (resource) - [`aws_s3_bucket_logging.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) (resource) - [`aws_s3_bucket_ownership_controls.s3_bucket_acl_ownership`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_ownership_controls) (resource) - [`aws_s3_bucket_public_access_block.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) (resource) - [`aws_s3_bucket_server_side_encryption_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) (resource) - [`aws_s3_bucket_versioning.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) (resource) - [`random_string.bucket_prefix`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.combined_permissions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.permissions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.permissions_cache_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.vpc_permissions`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_s3_bucket.secondary_artifact`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## config # Module: `config` This module enables [AWS Config](https://aws.amazon.com/config/) and optionally sets up an SNS topic to receive notifications of its findings. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-config/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-config/tree/main/test). ```hcl module "example" { source = "cloudposse/config/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" create_sns_topic = true create_iam_role = true managed_rules = { account-part-of-organizations = { description = "Checks whether AWS account is part of AWS Organizations. The rule is NON_COMPLIANT if an AWS account is not part of AWS Organizations or AWS Organizations master account ID does not match rule parameter MasterAccountId.", identifier = "ACCOUNT_PART_OF_ORGANIZATIONS", trigger_type = "PERIODIC" enabled = true } } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-config/) - complete example of using this module ## Variables ### Required Variables
`global_resource_collector_region` (`string`) required
The region that collects AWS Config data for global resources such as IAM
`s3_bucket_arn` (`string`) required
The ARN of the S3 bucket used to store the configuration history
`s3_bucket_id` (`string`) required
The id (name) of the S3 bucket used to store the configuration history
### Optional Variables
`allowed_aws_services_for_sns_published` (`list(string)`) optional
AWS services that will have permission to publish to SNS topic. Used when no external JSON policy is used **Default value:** `[ ]`
`allowed_iam_arns_for_sns_publish` (`list(string)`) optional
IAM role/user ARNs that will have permission to publish to SNS topic. Used when no external json policy is used. **Default value:** `[ ]`
`central_resource_collector_account` (`string`) optional
The account ID of a central account that will aggregate AWS Config from other accounts **Default value:** `null`
`child_resource_collector_accounts` (`set(string)`) optional
The account IDs of other accounts that will send their AWS Configuration to this account **Default value:** `null`
`create_iam_role` (`bool`) optional
Flag to indicate whether an IAM Role should be created to grant the proper permissions for AWS Config **Default value:** `false`
`create_organization_aggregator_iam_role` (`bool`) optional
Flag to indicate whether an IAM Role should be created to grant the proper permissions for AWS Config to send logs from organization accounts **Default value:** `false`
`create_sns_topic` (`bool`) optional
Flag to indicate whether an SNS topic should be created for notifications If you want to send findings to a new SNS topic, set this to true and provide a valid configuration for subscribers **Default value:** `false`
`disabled_aggregation_regions` (`list(string)`) optional
A list of regions where config aggregation is disabled **Default value:** ```hcl [ "ap-northeast-3" ] ```
`findings_notification_arn` (`string`) optional
The ARN for an SNS topic to send findings notifications to. This is only used if create_sns_topic is false. If you want to send findings to an existing SNS topic, set the value of this to the ARN of the existing topic and set create_sns_topic to false. **Default value:** `null`
`force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`iam_role_arn` (`string`) optional
The ARN for an IAM Role AWS Config uses to make read or write requests to the delivery channel and to describe the AWS resources associated with the account. This is only used if create_iam_role is false. If you want to use an existing IAM Role, set the value of this to the ARN of the existing topic and set create_iam_role to false. See the AWS Docs for further information: http://docs.aws.amazon.com/config/latest/developerguide/iamrole-permissions.html **Default value:** `null`
`iam_role_organization_aggregator_arn` (`string`) optional
The ARN for an IAM Role that AWS Config uses for the organization aggregator that fetches AWS config data from AWS accounts. This is only used if create_organization_aggregator_iam_role is false. If you want to use an existing IAM Role, set the value of this to the ARN of the existing role and set create_organization_aggregator_iam_role to false. See the AWS docs for further information: http://docs.aws.amazon.com/config/latest/developerguide/iamrole-permissions.html **Default value:** `null`
`is_organization_aggregator` (`bool`) optional
The aggregator is an AWS Organizations aggregator **Default value:** `false`
`managed_rules` optional
A list of AWS Managed Rules that should be enabled on the account. See the following for a list of possible rules to enable: https://docs.aws.amazon.com/config/latest/developerguide/managed-rules-by-aws-config.html **Type:** ```hcl map(object({ description = string identifier = string input_parameters = any tags = map(string) enabled = bool })) ``` **Default value:** `{ }`
`recording_mode` optional
The mode for AWS Config to record configuration changes. recording_frequency: The frequency with which AWS Config records configuration changes (service defaults to CONTINUOUS). - CONTINUOUS - DAILY You can also override the recording frequency for specific resource types. recording_mode_override: description: A description for the override. recording_frequency: The frequency with which AWS Config records configuration changes for the specified resource types. - CONTINUOUS - DAILY resource_types: A list of resource types for which AWS Config records configuration changes. For example, AWS::EC2::Instance. See the following for more information: https://docs.aws.amazon.com/config/latest/developerguide/stop-start-recorder.html /* recording_mode = \{ recording_frequency = "DAILY" recording_mode_override = \{ description = "Override for specific resource types" recording_frequency = "CONTINUOUS" resource_types = ["AWS::EC2::Instance"] \} \} */ **Type:** ```hcl object({ recording_frequency = string recording_mode_override = optional(object({ description = string recording_frequency = string resource_types = list(string) })) }) ``` **Default value:** `null`
`s3_key_prefix` (`string`) optional
The prefix for AWS Config objects stored in the the S3 bucket. If this variable is set to null, the default, no prefix will be used. Examples: with prefix: \{S3_BUCKET NAME\}:/\{S3_KEY_PREFIX\}/AWSLogs/\{ACCOUNT_ID\}/Config/*. without prefix: \{S3_BUCKET NAME\}:/AWSLogs/\{ACCOUNT_ID\}/Config/*. **Default value:** `null`
`sns_encryption_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SNS or a custom CMK. **Default value:** `""`
`sqs_queue_kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SQS Queue or a custom CMK **Default value:** `""`
`subscribers` (`map(any)`) optional
A map of subscription configurations for SNS topics For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription#argument-reference protocol: The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see link) (email is an option but is unsupported in terraform, see link). endpoint: The endpoint to send data to, the contents will vary with the protocol. (see link for more information) endpoint_auto_confirms (Optional): Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty. Default is false raw_message_delivery (Optional): Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property). Default is false. **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_config_configuration_recorder_id`
The ID of the AWS Config Recorder
`iam_role`
IAM Role used to make read or write requests to the delivery channel and to describe the AWS resources associated with the account.
`iam_role_organization_aggregator`
IAM Role used to make read or write requests to the delivery channel and to describe the AWS resources associated with the account.
`sns_topic`
SNS topic
`sns_topic_subscriptions`
SNS topic subscriptions
`storage_bucket_arn`
Bucket ARN
`storage_bucket_id`
Bucket Name (aka ID)
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.38.0` - `http`, version: `>= 3.4.1` ### Providers - `aws`, version: `>= 5.38.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_config_aggregator_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | ----------------------------------------------------------------------------------------------------------------------- CONFIG AGGREGATION ----------------------------------------------------------------------------------------------------------------------- `aws_config_findings_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `aws_config_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | ---------------------------------------------------------------------------------------------------------------------- Enable and configure AWS Config ---------------------------------------------------------------------------------------------------------------------- `iam_role` | 0.19.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.19.0) | ----------------------------------------------------------------------------------------------------------------------- Optionally create IAM Roles ----------------------------------------------------------------------------------------------------------------------- Create Optional IAM ROLE for S3 bucket and SNS `iam_role_organization_aggregator` | 0.19.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.19.0) | Create Optional IAM ROLE for organization wide aggregator `sns_topic` | 0.20.1 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.20.1) | ----------------------------------------------------------------------------------------------------------------------- Optionally create an SNS topic and subscriptions ----------------------------------------------------------------------------------------------------------------------- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_config_aggregate_authorization.central`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_aggregate_authorization) (resource) - [`aws_config_aggregate_authorization.child`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_aggregate_authorization) (resource) - [`aws_config_config_rule.rules`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_config_rule) (resource) - [`aws_config_configuration_aggregator.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_configuration_aggregator) (resource) - [`aws_config_configuration_recorder.recorder`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_configuration_recorder) (resource) - [`aws_config_configuration_recorder_status.recorder_status`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_configuration_recorder_status) (resource) - [`aws_config_delivery_channel.channel`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/config_delivery_channel) (resource) - [`aws_iam_role_policy_attachment.config_policy_attachment`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.organization_config_policy_attachment`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy.aws_config_built_in_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) (data source) - [`aws_iam_policy.aws_config_organization_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) (data source) - [`aws_iam_policy_document.config_s3_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.config_sns_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## cis-1-2-rules # AWS Config Rules for CIS AWS Foundations Benchmark Compliance This module outputs a map of [AWS Config](https://aws.amazon.com/config) Rules that should be in place as part of acheiving compliance with the [CIS AWS Foundation Benchmak 1.2](https://www.cisecurity.org/cis-benchmarks/#amazon_web_services) standard. These rules are meant to be used as an input to the [Cloud Posse AWS Config Module](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/cis-1-2-rules/../../) and are defined in the rules [catalog](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/cis-1-2-rules/../../catalog). ## Usage ### Which Account(s) Should Rules Be Applied In In general, these rules are meant to be enabled in every region of each of your accounts, with some exceptions noted below. ### Controls You May Want to Disable There are some controls that are part of the standard that should be disabled in certain scenarios. #### CIS AWS Foundations Benchmark Control 2.7: Ensure CloudTrail logs are encrypted at rest using AWS KMS CMKs When you are using a centralized CloudTrail account, you should only run this rule in the centralized account. The rule can be enabled in the centralized account by setting the `is_logging_account` variable to true and disabled in all other accounts by setting `is_logging_account` to false or omitting it as false is the default value. #### CIS AWS Foundations Benchmark Controls 1.2-1.14, 1.16, 1.20, 1.22, and 2.5: Global Resources These controls deal with ensuring various global resources, such as IAM Users, are configured in a way that aligns with the Benchmark. Since these resources are global, there is no reason to have AWS Config check them in each region. One region should be designated as the _Global Region_ for AWS Config and checks for these controls should only be run in that region. This set of checks can be enabled in the _Global Region_ by setting the `is_global_resource_region` to true and disabled in all other regions by setting `is_global_resource_region` to false or omitting it as false is the default value. ### Parameter Overrides You may also override the values any of the AWS Config Parameters set by the rules from our [catalog](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/cis-1-2-rules/../../catalog) by providing a map of maps to the `parameter_overrides` variable. The example below shows overriding the `MaxPasswordAge` of the `iam-password-policy` rule. The rule defaults to 90 days, while in this example we want to set it to 45 days. **IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-config/releases). For a complete example, see [examples/cis](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/cis-1-2-rules/../../examples/cis). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/cis-1-2-rules/test). ```hcl module "cis_1_2_rules" { source = "cloudposse/config/aws//modules/cis-1-2-rules" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" is_global_resource_region = true is_logging_account = true parameter_overrides = { "iam-password-policy": { "MaxPasswordAge": "45" } } } module "config" { source = "cloudposse/config/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" create_sns_topic = true create_iam_role = true managed_rules = module.cis_1_2_rules.rules } } ``` --- ## conformance-pack # AWS Config Conformance Pack This module deploys a [Conformance Pack](https://docs.aws.amazon.com/config/latest/developerguide/conformance-packs.html). A conformance pack is a collection of AWS Config rules and remediation actions that can be easily deployed as a single entity in an account and a Region or across an organization in AWS Organizations.Conformance packs are created by authoring a YAML template that contains the list of AWS Config managed or custom rules and remediation actions. The Conformance Pack cannot be deployed until AWS Config is deployed, which can be deployed using the [root module](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/conformance-pack/../../) of this repository. ## Usage **IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-config/releases). For a complete example, see [examples/hipaa](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/conformance-pack/../../examples/hipaa). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest)(which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-config/tree/main/modules/conformance-pack/test). ```hcl module "hipaa_conformance_pack" { source = "cloudposse/config/aws//modules/conformance-pack" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "Operational-Best-Practices-for-HIPAA-Security" conformance_pack="https://raw.githubusercontent.com/awslabs/aws-config-rules/master/aws-config-conformance-packs/Operational-Best-Practices-for-HIPAA-Security.yaml" parameter_overrides = { AccessKeysRotatedParamMaxAccessKeyAge = "45" } depends_on = [ module.config ] } module "config" { source = "cloudposse/config/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" create_sns_topic = true create_iam_role = true } ``` --- ## config-storage # Module: `config-storage` This module creates an S3 bucket suitable for storing `AWS Config` data. It implements a configurable log retention policy, which allows you to efficiently manage logs across different storage classes (_e.g._ `Glacier`) and ultimately expire the data altogether. It enables server-side default encryption. https://docs.aws.amazon.com/AmazonS3/latest/dev/bucket-encryption.html It blocks public access to the bucket by default. https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html ## Usage Here's how to invoke this example module in your projects ```hcl module "aws_config_storage" { source = "cloudposse/config-storage/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "aws-config" stage = "test" namespace = "eg" standard_transition_days = 30 glacier_transition_days = 60 } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-config-storage/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`abort_incomplete_multipart_upload_days` (`number`) optional
Maximum time (in days) that you want to allow multipart uploads to remain in progress **Default value:** `5`
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `false`
`bucket_notifications_enabled` (`bool`) optional
Send notifications for the object created events. Used for 3rd-party log collection from a bucket **Default value:** `false`
`bucket_notifications_prefix` (`string`) optional
Prefix filter. Used to manage object notifications **Default value:** `""`
`bucket_notifications_type` (`string`) optional
Type of the notification configuration. Only SQS is supported. **Default value:** `"SQS"`
`enable_glacier_transition` (`bool`) optional
Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files **Default value:** `true`
`expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`force_destroy` (`bool`) optional
(Optional, Default:false ) A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`kms_master_key_arn` (`string`) optional
The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms **Default value:** `""`
`lifecycle_prefix` (`string`) optional
Prefix filter. Used to manage object lifecycle events **Default value:** `""`
`lifecycle_rule_enabled` (`bool`) optional
Enable lifecycle events on this bucket **Default value:** `true`
`lifecycle_tags` (`map(string)`) optional
Tags filter. Used to manage object lifecycle events **Default value:** `{ }`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Specifies when noncurrent object versions transitions **Default value:** `30`
`policy` (`string`) optional
A valid bucket policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy **Default value:** `""`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are AES256 and aws:kms **Default value:** `"AES256"`
`standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
Bucket ARN
`bucket_domain_name`
FQDN of bucket
`bucket_id`
Bucket Name (aka ID)
`bucket_notifications_sqs_queue_arn`
Notifications SQS queue ARN
`enabled`
Is module enabled
`prefix`
Prefix configured for lifecycle rules
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_config_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `storage` | 1.4.1 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.aws_config_bucket_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## datadog-integration(Datadog-integration) # Module: `datadog-integration` Terraform module to configure [Datadog AWS integration](https://docs.datadoghq.com/api/v1/aws-integration/). ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-datadog-integration/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-datadog-integration/tree/main/test). **Note:** At the moment this module supports a limited set of IAM policies to support Datadog integrations. More can be added as needed. ### Structure This module aligns with [Datadog's documentation](https://docs.datadoghq.com/integrations/amazon_web_services/) by providing a `core-integration` policy for minimal permissions and additional policies for specific services. It also includes a `full-integration` policy (formerly `all`), encompassing all permissions listed under "All Permissions" for comprehensive coverage. The variable `var.integrations` is deprecated and replaced by `var.policies`, which supports Datadog-defined IAM policy names such as `core-integration`, `full-integration`, `resource-collection`, `CSPM`, `SecurityAudit`, and `everything`. Policy files have been updated for clarity and functionality. The `full-integration` policy reflects Datadog’s latest permissions and replaces the former `all` policy. A new `resource-collection` policy has been added for resource-specific permissions, while the `SecurityAudit` policy attaches the AWS-managed role for compliance. Backward compatibility is maintained by mapping old `var.integrations` values to new `var.policies`, ensuring a seamless transition while supporting legacy configurations.``` ### Migration Guide To migrate from the `v1.3.0` configuration, replace `var.integrations` with `var.policies` in your module usage. The values `"core"` and `"all"` previously used in `var.integrations` should be updated to `"core-integration"` and `"full-integration"`, respectively. If you were using `"CSPM"`, it now serves as an alias for `"SecurityAudit"`. Existing configurations will remain functional due to backward compatibility mappings, but updating to the new `var.policies` variable ensures alignment with the latest module structure and Datadog's documentation. ### Installation Include this module in your existing terraform code: ```hcl module "datadog_integration" { source = "cloudposse/datadog-integration/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "datadog" policies = ["full-integration"] } ``` The DataDog integration will be linked with your configured datadog account via the provider's `api_key`. ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-datadog-integration/tree/main/examples/complete) to see how to use this module. ## Variables ### Required Variables
### Optional Variables
`account_specific_namespace_rules` (`map(string)`) optional
An object, (in the form \{"namespace1":true/false, "namespace2":true/false\} ), that enables or disables metric collection for specific AWS namespaces for this AWS account only **Default value:** `null`
`cspm_resource_collection_enabled` (`bool`) optional
Whether Datadog collects cloud security posture management resources from your AWS account. **Default value:** `null`
`datadog_aws_account_id` (`string`) optional
The AWS account ID Datadog's integration servers use for all integrations **Default value:** `"464622532012"`
`excluded_regions` (`list(string)`) optional
An array of AWS regions to exclude from metrics collection **Default value:** `null`
`extended_resource_collection_enabled` (`bool`) optional
Whether Datadog collects additional attributes and configuration information about the resources in your AWS account. Required for `cspm_resource_collection_enabled`. **Default value:** `null`
`filter_tags` (`list(string)`) optional
An array of EC2 tags (in the form `key:value`) that defines a filter that Datadog use when collecting metrics from EC2. Wildcards, such as ? (for single characters) and * (for multiple characters) can also be used **Default value:** `null`
`host_tags` (`list(string)`) optional
An array of tags (in the form `key:value`) to add to all hosts and metrics reporting through this integration **Default value:** `null`
`integrations` (`list(string)`) optional
DEPRECATED: Use the `policies` variable instead. List of AWS permission names to apply for different integrations (e.g. 'all', 'core') **Default value:** `null`
`metrics_collection_enabled` (`bool`) optional
Whether Datadog collects metrics for this AWS account. **Default value:** `null`
`policies` (`list(string)`) optional
List of Datadog's names for AWS IAM policies names to apply to the role. Valid options are "core-integration", "full-integration", "resource-collection", "CSPM", "SecurityAudit", "everything". "CSPM" is for Cloud Security Posture Management, which also requires "full-integration". "SecurityAudit" is for the AWS-managed `SecurityAudit` Policy. "everything" means all permissions for offerings. **Default value:** `[ ]`
`resource_collection_enabled` (`bool`) optional
DEPRECATED: Use the `extended_resource_collection_enabled` variables instead. Whether Datadog collects a standard set of resources from your AWS account. **Default value:** `null`
`security_audit_policy_enabled` (`bool`) optional
DEPRECATED: Include `SecurityAudit` in the `policies` variable instead. Enable/disable attaching the AWS managed `SecurityAudit` policy to the Datadog IAM role to collect information about how AWS resources are configured (used in Datadog Cloud Security Posture Management to read security configuration metadata). If var.cspm_resource_collection_enabled, this is enabled automatically." **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_account_id`
AWS Account ID of the IAM Role for Datadog to use for this integration
`aws_role_arn`
ARN of the AWS IAM Role for Datadog to use for this integration
`aws_role_name`
Name of the AWS IAM Role for Datadog to use for this integration
`datadog_external_id`
Datadog integration external ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 3.0` - `datadog`, version: `>= 3.9` ### Providers - `aws`, version: `>= 3.0` - `datadog`, version: `>= 3.9` ### Modules Name | Version | Source | Description --- | --- | --- | --- `core_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `full_integration_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `resource_collection_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.full_integration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.resource_collection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.full_integration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.resource_collection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.security_audit`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`datadog_integration_aws.integration`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/integration_aws) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.full_integration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_collection`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## datadog-lambda-forwarder(Datadog-lambda-forwarder) # Module: `datadog-lambda-forwarder` Terraform module to provision all the necessary infrastructure to deploy [Datadog Lambda forwarders](https://github.com/DataDog/datadog-serverless-functions/tree/master/aws/logs_monitoring) ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder/tree/main/test). To enable Datadog forwarder for RDS Enhanced monitoring: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_rds_enabled = true } ``` To enable Datadog forwarder for a CloudTrail S3 bucket: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_log_enabled = true s3_buckets = ["cloudtrail-audit-bucket"] s3_bucket_kms_arns = ["arn:aws:kms:us-west-2:1234567890:key/b204f3d2-1111-2222-94333332-4444ccc222"] } ``` To enable Datadog forwarder for a S3 bucket with prefix: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_log_enabled = true s3_buckets_with_prefixes = { MyBucketWithPrefix = {bucket_name = "my-bucket-with-prefix", bucket_prefix = "events/"} AnotherWithPrefix = {bucket_name = "another-with-prefix", bucket_prefix = "records/"} } s3_bucket_kms_arns = ["arn:aws:kms:us-west-2:1234567890:key/b204f3d2-1111-2222-94333332-4444ccc222"] } ``` To enable Datadog forwarder for RDS authentication CloudWatch logs: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_log_enabled = true cloudwatch_forwarder_log_groups = { postgres = { name = "/aws/rds/cluster/pg-main/postgresql" filter_pattern = "" } } } ``` To enable Datadog forwarder for VPC Flow Logs CloudWatch logs: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_vpc_logs_enabled = true vpclogs_cloudwatch_log_group = "/aws/vpc/flowlogs/vpc1" } ``` To use a local copy of the lambda code you can specify the artifact url: ```hcl module "datadog_lambda_forwarder" { source = "cloudposse/datadog-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" forwarder_rds_enabled = true forwarder_rds_artifact_url = file("${path.module}/function.zip") } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-datadog-lambda-forwarder/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`api_key_ssm_arn` (`string`) optional
ARN of the SSM parameter for the Datadog API key. Passing this removes the need to fetch the key from the SSM parameter store. This could be the case if the SSM Key is in a different region than the lambda. **Default value:** `null`
`cloudwatch_forwarder_event_patterns` optional
Map of title => CloudWatch Event patterns to forward to Datadog. Event structure from here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html#CloudWatchEventsPatterns Example: ```hcl cloudwatch_forwarder_event_rules = { "guardduty" = { source = ["aws.guardduty"] detail-type = ["GuardDuty Finding"] } "ec2-terminated" = { source = ["aws.ec2"] detail-type = ["EC2 Instance State-change Notification"] detail = { state = ["terminated"] } } } ``` **Type:** ```hcl map(object({ version = optional(list(string)) id = optional(list(string)) detail-type = optional(list(string)) source = optional(list(string)) account = optional(list(string)) time = optional(list(string)) region = optional(list(string)) resources = optional(list(string)) detail = optional(map(list(string))) })) ``` **Default value:** `{ }`
`cloudwatch_forwarder_log_groups` optional
Map of CloudWatch Log Groups with a filter pattern that the Lambda forwarder will send logs from. For example: \{ mysql1 = \{ name = "/aws/rds/maincluster", filter_pattern = "" \} **Type:** ```hcl map(object({ name = string filter_pattern = string })) ``` **Default value:** `{ }`
`datadog_forwarder_lambda_environment_variables` (`map(string)`) optional
Map of environment variables to pass to the Lambda Function **Default value:** `{ }`
`dd_api_key_kms_ciphertext_blob` (`string`) optional
CiphertextBlob stored in environment variable DD_KMS_API_KEY used by the lambda function, along with the KMS key, to decrypt Datadog API key **Default value:** `""`
`dd_api_key_source` optional
One of: ARN for AWS Secrets Manager (asm) to retrieve the Datadog (DD) api key, ARN for the KMS (kms) key used to decrypt the ciphertext_blob of the api key, or the name of the SSM (ssm) parameter used to retrieve the Datadog API key **Type:** ```hcl object({ resource = string identifier = string }) ``` **Default value:** ```hcl { "identifier": "", "resource": "" } ```
`dd_artifact_filename` (`string`) optional
The Datadog artifact filename minus extension **Default value:** `"aws-dd-forwarder"`
`dd_forwarder_version` (`string`) optional
Version tag of Datadog lambdas to use. https://github.com/DataDog/datadog-serverless-functions/releases **Default value:** `"3.116.0"`
`dd_module_name` (`string`) optional
The Datadog GitHub repository name **Default value:** `"datadog-serverless-functions"`
`dd_tags` (`list(string)`) optional
A list of Datadog tags to apply to all logs forwarded to Datadog **Default value:** `[ ]`
`dd_tags_map` (`map(string)`) optional
A map of Datadog tags to apply to all logs forwarded to Datadog. This will override dd_tags. **Default value:** `{ }`
`forwarder_iam_path` (`string`) optional
Path to the IAM roles and policies created **Default value:** `"/"`
`forwarder_lambda_datadog_host` (`string`) optional
Datadog Site to send data to. Possible values are `datadoghq.com`, `datadoghq.eu`, `us3.datadoghq.com`, `us5.datadoghq.com` and `ddog-gov.com` **Default value:** `"datadoghq.com"`
`forwarder_lambda_debug_enabled` (`bool`) optional
Whether to enable or disable debug for the Lambda forwarder **Default value:** `false`
`forwarder_log_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for Logs. It can be a local file, URL or git repo **Default value:** `null`
`forwarder_log_enabled` (`bool`) optional
Flag to enable or disable Datadog log forwarder **Default value:** `false`
`forwarder_log_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog log forwarder lambda function **Default value:** `[ ]`
`forwarder_log_retention_days` (`number`) optional
Number of days to retain Datadog forwarder lambda execution logs. One of [0 1 3 5 7 14 30 60 90 120 150 180 365 400 545 731 1827 3653] **Default value:** `14`
`forwarder_rds_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for RDS. It can be a local file, url or git repo **Default value:** `null`
`forwarder_rds_enabled` (`bool`) optional
Flag to enable or disable Datadog RDS enhanced monitoring forwarder **Default value:** `false`
`forwarder_rds_filter_pattern` (`string`) optional
Filter pattern for Lambda forwarder RDS **Default value:** `""`
`forwarder_rds_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog RDS enhanced monitoring lambda function **Default value:** `[ ]`
`forwarder_use_cache_bucket` (`bool`) optional
Flag to enable or disable the cache bucket for lambda tags and failed events. See https://docs.datadoghq.com/logs/guide/forwarder/?tab=cloudformation#upgrade-an-older-version-to-31060. Recommended for forwarder versions 3.106 and higher. **Default value:** `true`
`forwarder_vpc_logs_artifact_url` (`string`) optional
The URL for the code of the Datadog forwarder for VPC Logs. It can be a local file, url or git repo **Default value:** `null`
`forwarder_vpc_logs_enabled` (`bool`) optional
Flag to enable or disable Datadog VPC flow log forwarder **Default value:** `false`
`forwarder_vpc_logs_layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to Datadog VPC flow log forwarder lambda function **Default value:** `[ ]`
`forwarder_vpclogs_filter_pattern` (`string`) optional
Filter pattern for Lambda forwarder VPC Logs **Default value:** `""`
`kms_key_id` (`string`) optional
Optional KMS key ID to encrypt Datadog Lambda function logs **Default value:** `null`
`lambda_architectures` (`list(string)`) optional
Instruction set architecture for your Lambda function. Valid values are ["x86_64"] and ["arm64"]. **Default value:** `null`
`lambda_custom_policy_name` (`string`) optional
Additional IAM policy document that can optionally be passed and merged with the created policy document **Default value:** `"DatadogForwarderCustomPolicy"`
`lambda_memory_size` (`number`) optional
Amount of memory in MB your Lambda Function can use at runtime **Default value:** `128`
`lambda_policy_source_json` (`string`) optional
Additional IAM policy document that can optionally be passed and merged with the created policy document **Default value:** `""`
`lambda_reserved_concurrent_executions` (`number`) optional
Amount of reserved concurrent executions for the lambda function. A value of 0 disables Lambda from being triggered and -1 removes any concurrency limitations. Defaults to Unreserved Concurrency Limits -1 **Default value:** `-1`
`lambda_runtime` (`string`) optional
Runtime environment for Datadog Lambda **Default value:** `"python3.11"`
`lambda_timeout` (`number`) optional
Amount of time your Datadog Lambda Function has to run in seconds **Default value:** `120`
`log_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the lambda-log role managed by this module. **Default value:** `null`
`rds_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the lambda-rds role managed by this module. **Default value:** `null`
`s3_bucket_kms_arns` (`list(string)`) optional
List of KMS key ARNs for s3 bucket encryption **Default value:** `[ ]`
`s3_buckets` (`list(string)`) optional
The names of S3 buckets to forward logs to Datadog **Default value:** `[ ]`
`s3_buckets_with_prefixes` (`map(object({ bucket_name : string, bucket_prefix : string }))`) optional
The names S3 buckets and prefix to forward logs to Datadog **Default value:** `{ }`
`s3_notification_events` (`list(string)`) optional
List of S3 events to trigger the Lambda notification **Default value:** ```hcl [ "s3:ObjectCreated:*" ] ```
`security_group_ids` (`list(string)`) optional
List of security group IDs to use when the Lambda Function runs in a VPC **Default value:** `null`
`subnet_ids` (`list(string)`) optional
List of subnet IDs to use when deploying the Lambda Function in a VPC **Default value:** `null`
`tracing_config_mode` (`string`) optional
Can be either PassThrough or Active. If PassThrough, Lambda will only trace the request from an upstream service if it contains a tracing header with 'sampled=1'. If Active, Lambda will respect any tracing header it receives from an upstream service **Default value:** `"PassThrough"`
`vpc_logs_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the lambda-vpc-logs role managed by this module. **Default value:** `null`
`vpclogs_cloudwatch_log_group` (`string`) optional
The name of the CloudWatch Log Group for VPC flow logs **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lambda_forwarder_log_function_arn`
Datadog Lambda forwarder CloudWatch/S3 function ARN
`lambda_forwarder_log_function_name`
Datadog Lambda forwarder CloudWatch/S3 function name
`lambda_forwarder_rds_enhanced_monitoring_function_name`
Datadog Lambda forwarder RDS Enhanced Monitoring function name
`lambda_forwarder_rds_function_arn`
Datadog Lambda forwarder RDS Enhanced Monitoring function ARN
`lambda_forwarder_vpc_log_function_arn`
Datadog Lambda forwarder VPC Flow Logs function ARN
`lambda_forwarder_vpc_log_function_name`
Datadog Lambda forwarder VPC Flow Logs function name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `archive`, version: `>= 2.2.0` - `aws`, version: `>= 3.0` ### Providers - `archive`, version: `>= 2.2.0` - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_event` | 0.6.1 | [`cloudposse/cloudwatch-events/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-events/aws/0.6.1) | n/a `forwarder_log_artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `forwarder_log_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `forwarder_log_s3_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `forwarder_rds_artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `forwarder_rds_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `forwarder_vpclogs_artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `forwarder_vpclogs_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `tags_cache_s3_bucket` | 4.2.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.forwarder_log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_cloudwatch_log_group.forwarder_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_cloudwatch_log_group.forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_cloudwatch_log_subscription_filter.cloudwatch_log_subscription_filter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) (resource) - [`aws_cloudwatch_log_subscription_filter.datadog_log_subscription_filter_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) (resource) - [`aws_cloudwatch_log_subscription_filter.datadog_log_subscription_filter_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_subscription_filter) (resource) - [`aws_iam_policy.datadog_custom_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.lambda_forwarder_log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.lambda_forwarder_log_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.lambda_forwarder_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.lambda_forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.lambda_forwarder_log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.lambda_forwarder_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.lambda_forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.datadog_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.lambda_forwarder_log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.lambda_forwarder_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.lambda_forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_function.forwarder_log`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_function.forwarder_rds`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_function.forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.allow_eventbridge`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.allow_s3_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.cloudwatch_enhanced_rds_monitoring`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.cloudwatch_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.cloudwatch_vpclogs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_s3_bucket_notification.s3_bucket_notification`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_notification) (resource) - [`aws_s3_bucket_notification.s3_bucket_notification_with_prefixes`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_notification) (resource) ## Data Sources The following data sources are used by this module: - [`archive_file.forwarder_rds`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`archive_file.forwarder_vpclogs`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.lambda_default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_log_bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_ssm_parameter.api_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## dms # Module: `dms` Terraform modules for provisioning and managing AWS [DMS](https://aws.amazon.com/dms/) resources. The following DMS resources are supported: - [IAM Roles for DMS](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules/dms-iam) - [DMS Endpoints](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules/dms-endpoint) - [DMS Replication Instances](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules/dms-replication-instance) - [DMS Replication Tasks](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules/dms-replication-task) - [DMS Event Subscriptions](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules/dms-event-subscription) Refer to [modules](https://github.com/cloudposse/terraform-aws-dms/tree/main/modules) for more details. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-dms/tree/main/examples/complete). For automated tests of the example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-dms/tree/main/test). ## Examples ```hcl module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "dms_replication_instance" { source = "cloudposse/dms/aws//modules/dms-replication-instance" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine_version = "3.5" replication_instance_class = "dms.t2.small" allocated_storage = 50 apply_immediately = true auto_minor_version_upgrade = true allow_major_version_upgrade = false multi_az = false publicly_accessible = false preferred_maintenance_window = "sun:10:30-sun:14:30" vpc_security_group_ids = [module.vpc.vpc_default_security_group_id, module.aurora_postgres_cluster.security_group_id] subnet_ids = module.subnets.private_subnet_ids context = module.this.context depends_on = [ # The required DMS roles must be present before replication instances can be provisioned module.dms_iam, aws_vpc_endpoint.s3 ] } module "aurora_postgres_cluster" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora-postgresql" engine_mode = "provisioned" engine_version = "13.4" cluster_family = "aurora-postgresql13" cluster_size = 1 admin_user = "admin_user" admin_password = "admin_password" db_name = "postgres" db_port = 5432 instance_type = "db.t3.medium" vpc_id = module.vpc.vpc_id subnets = module.subnets.private_subnet_ids security_groups = [module.vpc.vpc_default_security_group_id] deletion_protection = false autoscaling_enabled = false storage_encrypted = false intra_security_group_traffic_enabled = false skip_final_snapshot = true enhanced_monitoring_role_enabled = false iam_database_authentication_enabled = false cluster_parameters = [ { name = "rds.logical_replication" value = "1" apply_method = "pending-reboot" }, { name = "max_replication_slots" value = "10" apply_method = "pending-reboot" }, { name = "wal_sender_timeout" value = "0" apply_method = "pending-reboot" }, { name = "max_worker_processes" value = "8" apply_method = "pending-reboot" }, { name = "max_logical_replication_workers" value = "10" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" } ] context = module.this.context } module "dms_endpoint_aurora_postgres" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "source" engine_name = "aurora-postgresql" server_name = module.aurora_postgres_cluster.reader_endpoint database_name = "postgres" port = 5432 username = "admin_user" password = "admin_password" extra_connection_attributes = "" secrets_manager_access_role_arn = null secrets_manager_arn = null ssl_mode = "none" attributes = ["source"] context = module.this.context } resource "aws_vpc_endpoint" "s3" { vpc_endpoint_type = "Gateway" vpc_id = module.vpc.vpc_id service_name = "com.amazonaws.${var.region}.s3" route_table_ids = module.subnets.private_route_table_ids tags = module.this.tags } module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false allow_encrypted_uploads_only = false allow_ssl_requests_only = false force_destroy = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "dms_endpoint_s3_bucket" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "target" engine_name = "s3" s3_settings = { bucket_name = module.s3_bucket.bucket_id bucket_folder = null cdc_inserts_only = false csv_row_delimiter = " " csv_delimiter = "," data_format = "parquet" compression_type = "GZIP" date_partition_delimiter = "NONE" date_partition_enabled = true date_partition_sequence = "YYYYMMDD" include_op_for_full_load = true parquet_timestamp_in_millisecond = true timestamp_column_name = "timestamp" service_access_role_arn = aws_iam_role.s3.arn } extra_connection_attributes = "" attributes = ["target"] context = module.this.context } module "dms_replication_task" { source = "cloudposse/dms/aws//modules/dms-replication-task" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" replication_instance_arn = module.dms_replication_instance.replication_instance_arn start_replication_task = true migration_type = "full-load-and-cdc" source_endpoint_arn = module.dms_endpoint_aurora_postgres.endpoint_arn target_endpoint_arn = module.dms_endpoint_s3_bucket.endpoint_arn replication_task_settings = file("${path.module}/config/replication-task-settings.json") table_mappings = file("${path.module}/config/replication-task-table-mappings.json") context = module.this.context } module "sns_topic" { source = "cloudposse/sns-topic/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" sqs_dlq_enabled = false fifo_topic = false fifo_queue_enabled = false encryption_enabled = false allowed_aws_services_for_sns_published = [ "cloudwatch.amazonaws.com", "dms.amazonaws.com" ] context = module.this.context } module "dms_replication_instance_event_subscription" { source = "cloudposse/dms/aws//modules/dms-event-subscription" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" event_subscription_enabled = true source_type = "replication-instance" source_ids = [module.dms_replication_instance.replication_instance_id] sns_topic_arn = module.sns_topic.sns_topic_arn # https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dms/describe-event-categories.html event_categories = [ "low storage", "configuration change", "maintenance", "deletion", "creation", "failover", "failure" ] attributes = ["instance"] context = module.this.context } module "dms_replication_task_event_subscription" { source = "cloudposse/dms/aws//modules/dms-event-subscription" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" event_subscription_enabled = true source_type = "replication-task" source_ids = [module.dms_replication_task.replication_task_id] sns_topic_arn = module.sns_topic.sns_topic_arn # https://awscli.amazonaws.com/v2/documentation/api/latest/reference/dms/describe-event-categories.html event_categories = [ "configuration change", "state change", "deletion", "creation", "failure" ] attributes = ["task"] context = module.this.context } ``` __NOTE:__ If a replication task is in "Failed" state (for any reason, e.g. network connectivity issues, database table issues, configuration issues), it can't be destroyed with Terraform (but can be updated). The task needs to be updated/fixed and moved to any other state like "Running", "Stopped", "Starting", "Ready", etc. You can monitor the progress of your task by checking the task status and by monitoring the task's control table. The task status indicates the condition of an AWS DMS task and its associated resources. It includes such indications as if the task is being created, starting, running, stopped, or failed. It also includes the current state of the tables that the task is migrating, such as if a full load of a table has begun or is in progress and details such as the number of inserts, deletes, and updates have occurred for the table. Refer to [Monitoring DMS Task Status](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html#CHAP_Tasks.Status) for more information. ## Variables ### Required Variables
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## dms-endpoint Terraform module to provision DMS Endpoints. ## Usage ```hcl module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "aurora_postgres_cluster" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora-postgresql" engine_mode = "provisioned" engine_version = "13.4" cluster_family = "aurora-postgresql13" cluster_size = 1 admin_user = "admin_user" admin_password = "admin_password" db_name = "postgres" db_port = 5432 instance_type = "db.t3.medium" vpc_id = module.vpc.vpc_id subnets = module.subnets.private_subnet_ids security_groups = [module.vpc.vpc_default_security_group_id] deletion_protection = false autoscaling_enabled = false storage_encrypted = false intra_security_group_traffic_enabled = false skip_final_snapshot = true enhanced_monitoring_role_enabled = false iam_database_authentication_enabled = false cluster_parameters = [ { name = "rds.logical_replication" value = "1" apply_method = "pending-reboot" }, { name = "max_replication_slots" value = "10" apply_method = "pending-reboot" }, { name = "wal_sender_timeout" value = "0" apply_method = "pending-reboot" }, { name = "max_worker_processes" value = "8" apply_method = "pending-reboot" }, { name = "max_logical_replication_workers" value = "10" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" } ] context = module.this.context } module "dms_endpoint_aurora_postgres" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "source" engine_name = "aurora-postgresql" server_name = module.aurora_postgres_cluster.reader_endpoint database_name = "postgres" port = 5432 username = "admin_user" password = "admin_password" extra_connection_attributes = "" secrets_manager_access_role_arn = null secrets_manager_arn = null ssl_mode = "none" attributes = ["source"] context = module.this.context } resource "aws_vpc_endpoint" "s3" { vpc_id = module.vpc.vpc_id service_name = "com.amazonaws.${var.region}.s3" tags = module.this.tags } module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false versioning_enabled = false allow_encrypted_uploads_only = false allow_ssl_requests_only = false force_destroy = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "dms_endpoint_s3_bucket" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "target" engine_name = "s3" s3_settings = { bucket_name = module.s3_bucket.bucket_id bucket_folder = null cdc_inserts_only = false csv_row_delimiter = " " csv_delimiter = "," data_format = "parquet" compression_type = "GZIP" date_partition_delimiter = "NONE" date_partition_enabled = true date_partition_sequence = "YYYYMMDD" include_op_for_full_load = true parquet_timestamp_in_millisecond = true timestamp_column_name = "timestamp" service_access_role_arn = aws_iam_role.s3.arn } extra_connection_attributes = "" attributes = ["target"] context = module.this.context } ``` --- ## dms-event-subscription Terraform module to provision DMS Event Subscriptions. ## Usage ```hcl module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "dms_replication_instance" { source = "cloudposse/dms/aws//modules/dms-replication-instance" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine_version = "3.5" replication_instance_class = "dms.t2.small" allocated_storage = 50 apply_immediately = true auto_minor_version_upgrade = true allow_major_version_upgrade = false multi_az = false publicly_accessible = false preferred_maintenance_window = "sun:10:30-sun:14:30" vpc_security_group_ids = [module.vpc.vpc_default_security_group_id, module.aurora_postgres_cluster.security_group_id] subnet_ids = module.subnets.private_subnet_ids context = module.this.context depends_on = [ # The required DMS roles must be present before replication instances can be provisioned module.dms_iam, aws_vpc_endpoint.s3 ] } module "aurora_postgres_cluster" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora-postgresql" engine_mode = "provisioned" engine_version = "13.4" cluster_family = "aurora-postgresql13" cluster_size = 1 admin_user = "admin_user" admin_password = "admin_password" db_name = "postgres" db_port = 5432 instance_type = "db.t3.medium" vpc_id = module.vpc.vpc_id subnets = module.subnets.private_subnet_ids security_groups = [module.vpc.vpc_default_security_group_id] deletion_protection = false autoscaling_enabled = false storage_encrypted = false intra_security_group_traffic_enabled = false skip_final_snapshot = true enhanced_monitoring_role_enabled = false iam_database_authentication_enabled = false cluster_parameters = [ { name = "rds.logical_replication" value = "1" apply_method = "pending-reboot" }, { name = "max_replication_slots" value = "10" apply_method = "pending-reboot" }, { name = "wal_sender_timeout" value = "0" apply_method = "pending-reboot" }, { name = "max_worker_processes" value = "8" apply_method = "pending-reboot" }, { name = "max_logical_replication_workers" value = "10" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" } ] context = module.this.context } module "dms_endpoint_aurora_postgres" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "source" engine_name = "aurora-postgresql" server_name = module.aurora_postgres_cluster.reader_endpoint database_name = "postgres" port = 5432 username = "admin_user" password = "admin_password" extra_connection_attributes = "" secrets_manager_access_role_arn = null secrets_manager_arn = null ssl_mode = "none" attributes = ["source"] context = module.this.context } resource "aws_vpc_endpoint" "s3" { vpc_id = module.vpc.vpc_id service_name = "com.amazonaws.${var.region}.s3" tags = module.this.tags } module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false allow_encrypted_uploads_only = false allow_ssl_requests_only = false force_destroy = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "dms_endpoint_s3_bucket" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "target" engine_name = "s3" s3_settings = { bucket_name = module.s3_bucket.bucket_id bucket_folder = null cdc_inserts_only = false csv_row_delimiter = " " csv_delimiter = "," data_format = "parquet" compression_type = "GZIP" date_partition_delimiter = "NONE" date_partition_enabled = true date_partition_sequence = "YYYYMMDD" include_op_for_full_load = true parquet_timestamp_in_millisecond = true timestamp_column_name = "timestamp" service_access_role_arn = aws_iam_role.s3.arn } extra_connection_attributes = "" attributes = ["target"] context = module.this.context } module "dms_replication_task" { source = "cloudposse/dms/aws//modules/dms-replication-task" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" replication_instance_arn = module.dms_replication_instance.replication_instance_arn start_replication_task = true migration_type = "full-load-and-cdc" source_endpoint_arn = module.dms_endpoint_aurora_postgres.endpoint_arn target_endpoint_arn = module.dms_endpoint_s3_bucket.endpoint_arn replication_task_settings = file("${path.module}/config/replication-task-settings.json") table_mappings = file("${path.module}/config/replication-task-table-mappings.json") context = module.this.context } module "sns_topic" { source = "cloudposse/sns-topic/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" sqs_dlq_enabled = false fifo_topic = false fifo_queue_enabled = false encryption_enabled = false allowed_aws_services_for_sns_published = [ "cloudwatch.amazonaws.com", "dms.amazonaws.com" ] context = module.this.context } module "dms_replication_instance_event_subscription" { source = "cloudposse/dms/aws//modules/dms-event-subscription" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" event_subscription_enabled = true event_categories = ["creation", "failure"] source_type = "replication-instance" source_ids = [module.dms_replication_instance.replication_instance_id] sns_topic_arn = module.sns_topic.sns_topic_arn attributes = ["instance"] context = module.this.context } module "dms_replication_task_event_subscription" { source = "cloudposse/dms/aws//modules/dms-event-subscription" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" event_subscription_enabled = true event_categories = ["creation", "failure"] source_type = "replication-task" source_ids = [module.dms_replication_task.replication_task_id] sns_topic_arn = module.sns_topic.sns_topic_arn attributes = ["task"] context = module.this.context } ``` --- ## dms-iam Terraform module to provision IAM roles required for DMS. ## Usage ```hcl # Database Migration Service requires # the below IAM Roles to be created before # replication instances can be created. # The roles should be provisioned only once per account. # https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html # https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html#CHAP_Security.APIRole # https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dms_replication_instance # * dms-vpc-role # * dms-cloudwatch-logs-role # * dms-access-for-endpoint module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "dms_replication_instance" { source = "cloudposse/dms/aws//modules/dms-replication-instance" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine_version = "3.5" replication_instance_class = "dms.t2.small" allocated_storage = 50 apply_immediately = true auto_minor_version_upgrade = true allow_major_version_upgrade = false multi_az = false publicly_accessible = false preferred_maintenance_window = "sun:10:30-sun:14:30" vpc_security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.private_subnet_ids context = module.this.context depends_on = [ # The required DMS roles must be present before replication instances can be provisioned module.dms_iam ] } ``` ## DMS IAM Roles Two IAM roles that you need to create are `dms-vpc-role` and `dms-cloudwatch-logs-role`. If you use Amazon Redshift as a target database, you must also create and add the IAM role `dms-access-for-endpoint` to your AWS account. See [Creating the IAM roles to use with the AWS CLI and AWS DMS API](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Security.html) for more information. --- ## dms-replication-instance Terraform module to provision DMS Replication Instances. ## Usage ```hcl module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "dms_replication_instance" { source = "cloudposse/dms/aws//modules/dms-replication-instance" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" # If `auto_minor_version_upgrade` is enabled, # then we should omit the patch part of the version or Terraform will try to revert the version upon detected drift engine_version = "3.5" replication_instance_class = "dms.t2.small" allocated_storage = 50 apply_immediately = true auto_minor_version_upgrade = true allow_major_version_upgrade = false multi_az = false publicly_accessible = false preferred_maintenance_window = "sun:10:30-sun:14:30" vpc_security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.private_subnet_ids context = module.this.context depends_on = [ # The required DMS roles must be present before replication instances can be provisioned module.dms_iam ] } ``` --- ## dms-replication-task Terraform module to provision DMS Replication Tasks. ## Usage ```hcl module "dms_iam" { source = "cloudposse/dms/aws//modules/dms-iam" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = ["us-east-2a", "us-east-2b"] vpc_id = local.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "dms_replication_instance" { source = "cloudposse/dms/aws//modules/dms-replication-instance" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine_version = "3.5" replication_instance_class = "dms.t2.small" allocated_storage = 50 apply_immediately = true auto_minor_version_upgrade = true allow_major_version_upgrade = false multi_az = false publicly_accessible = false preferred_maintenance_window = "sun:10:30-sun:14:30" vpc_security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.private_subnet_ids context = module.this.context depends_on = [ # The required DMS roles must be present before replication instances can be provisioned module.dms_iam, aws_vpc_endpoint.s3 ] } module "aurora_postgres_cluster" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora-postgresql" engine_mode = "provisioned" engine_version = "13.4" cluster_family = "aurora-postgresql13" cluster_size = 1 admin_user = "admin_user" admin_password = "admin_password" db_name = "postgres" db_port = 5432 instance_type = "db.t3.medium" vpc_id = module.vpc.vpc_id subnets = module.subnets.private_subnet_ids security_groups = [module.vpc.vpc_default_security_group_id] deletion_protection = false autoscaling_enabled = false storage_encrypted = false intra_security_group_traffic_enabled = false skip_final_snapshot = true enhanced_monitoring_role_enabled = false iam_database_authentication_enabled = false cluster_parameters = [ { name = "rds.logical_replication" value = "1" apply_method = "pending-reboot" }, { name = "max_replication_slots" value = "10" apply_method = "pending-reboot" }, { name = "wal_sender_timeout" value = "0" apply_method = "pending-reboot" }, { name = "max_worker_processes" value = "8" apply_method = "pending-reboot" }, { name = "max_logical_replication_workers" value = "10" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" }, { name = "max_parallel_workers" value = "8" apply_method = "pending-reboot" } ] context = module.this.context } module "dms_endpoint_aurora_postgres" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "source" engine_name = "aurora-postgresql" server_name = module.aurora_postgres_cluster.reader_endpoint database_name = "postgres" port = 5432 username = "admin_user" password = "admin_password" extra_connection_attributes = "" secrets_manager_access_role_arn = null secrets_manager_arn = null ssl_mode = "none" attributes = ["source"] context = module.this.context } # Upgrades to AWS DMS versions 3.4.7 and higher require that you configure AWS DMS to use VPC endpoints or use public routes. # This requirement applies to source and target endpoints for these data stores: S3, Kinesis, Secrets Manager, DynamoDB, Amazon Redshift, and OpenSearch Service. resource "aws_vpc_endpoint" "s3" { vpc_endpoint_type = "Gateway" vpc_id = module.vpc.vpc_id service_name = "com.amazonaws.${var.region}.s3" route_table_ids = module.subnets.private_route_table_ids tags = module.this.tags } module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false allow_encrypted_uploads_only = false allow_ssl_requests_only = false force_destroy = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "dms_endpoint_s3_bucket" { source = "cloudposse/dms/aws//modules/dms-endpoint" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" endpoint_type = "target" engine_name = "s3" s3_settings = { bucket_name = module.s3_bucket.bucket_id bucket_folder = null cdc_inserts_only = false csv_row_delimiter = " " csv_delimiter = "," data_format = "parquet" compression_type = "GZIP" date_partition_delimiter = "NONE" date_partition_enabled = true date_partition_sequence = "YYYYMMDD" include_op_for_full_load = true parquet_timestamp_in_millisecond = true timestamp_column_name = "timestamp" service_access_role_arn = aws_iam_role.s3.arn } extra_connection_attributes = "" attributes = ["target"] context = module.this.context } module "dms_replication_task" { source = "cloudposse/dms/aws//modules/dms-replication-task" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" replication_instance_arn = module.dms_replication_instance.replication_instance_arn start_replication_task = true migration_type = "full-load-and-cdc" source_endpoint_arn = module.dms_endpoint_aurora_postgres.endpoint_arn target_endpoint_arn = module.dms_endpoint_s3_bucket.endpoint_arn # https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.html replication_task_settings = file("${path.module}/config/replication-task-settings.json") # https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.html table_mappings = file("${path.module}/config/replication-task-table-mappings.json") context = module.this.context } ``` ## Notes If a replication task is in "Failed" state (for any reason, e.g. network connectivity issues, database table issues, configuration issues), it can't be destroyed with Terraform (but can be updated). The task needs to be updated/fixed and moved to any other state like "Running", "Stopped", "Starting", "Ready", etc. You can monitor the progress of your task by checking the task status and by monitoring the task's control table. The task status indicates the condition of an AWS DMS task and its associated resources. It includes such indications as if the task is being created, starting, running, stopped, or failed. It also includes the current state of the tables that the task is migrating, such as if a full load of a table has begun or is in progress and details such as the number of inserts, deletes, and updates have occurred for the table. Refer to [Monitoring DMS Task Status](https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Monitoring.html#CHAP_Tasks.Status) for more information. --- ## documentdb-cluster # Module: `documentdb-cluster` Terraform module to provision an [`Amazon DocumentDB`](https://aws.amazon.com/documentdb/) cluster. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-documentdb-cluster/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-documentdb-cluster/tree/main/test). ```hcl module "documentdb_cluster" { source = "cloudposse/documentdb-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "testing" name = "docdb" cluster_size = 3 master_username = "admin1" master_password = "Test123456789" instance_class = "db.r4.large" vpc_id = "vpc-xxxxxxxx" subnet_ids = ["subnet-xxxxxxxx", "subnet-yyyyyyyy"] allowed_security_groups = ["sg-xxxxxxxx"] zone_id = "Zxxxxxxxx" } ``` ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
List of VPC subnet IDs to place DocumentDB instances in
`vpc_id` (`string`) required
VPC ID to create the cluster in (e.g. `vpc-a22222ee`)
### Optional Variables
`allow_ingress_from_self` (`bool`) optional
Adds the Document DB security group itself as a source for ingress rules. Useful when this security group will be shared with applications. **Default value:** `false`
`allow_major_version_upgrade` (`bool`) optional
Specifies whether major version upgrades are allowed. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster#allow_major_version_upgrade **Default value:** `false`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the DocumentDB cluster **Default value:** `[ ]`
`allowed_egress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to send traffic outside of the DocumentDB cluster **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`allowed_security_groups` (`list(string)`) optional
List of existing Security Groups to be allowed to connect to the DocumentDB cluster **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Specifies whether any minor engine upgrades will be applied automatically to the DB instance during the maintenance window or not **Default value:** `true`
`ca_cert_identifier` (`string`) optional
The identifier of the CA certificate for the DB instance **Default value:** `null`
`cluster_dns_name` (`string`) optional
Name of the cluster CNAME record to create in the parent DNS zone specified by `zone_id`. If left empty, the name will be auto-asigned using the format `master.var.name` **Default value:** `""`
`cluster_family` (`string`) optional
The family of the DocumentDB cluster parameter group. For more details, see https://docs.aws.amazon.com/documentdb/latest/developerguide/db-cluster-parameter-group-create.html **Default value:** `"docdb3.6"`
`cluster_parameters` optional
List of DB parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`cluster_size` (`number`) optional
Number of DB instances to create in the cluster **Default value:** `3`
`db_port` (`number`) optional
DocumentDB port **Default value:** `27017`
`deletion_protection` (`bool`) optional
A value that indicates whether the DB cluster has deletion protection enabled **Default value:** `false`
`egress_from_port` (`number`) optional
[from_port]DocumentDB initial port range for egress (e.g. `0`) **Default value:** `0`
`egress_protocol` (`string`) optional
DocumentDB protocol for egress (e.g. `-1`, `tcp`) **Default value:** `"-1"`
`egress_to_port` (`number`) optional
[to_port]DocumentDB initial port range for egress (e.g. `65535`) **Default value:** `0`
`enable_performance_insights` (`bool`) optional
Specifies whether to enable Performance Insights for the DB Instance. **Default value:** `false`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to export to cloudwatch. The following log types are supported: `audit`, `profiler` **Default value:** `[ ]`
`engine` (`string`) optional
The name of the database engine to be used for this DB cluster. Defaults to `docdb`. Valid values: `docdb` **Default value:** `"docdb"`
`engine_version` (`string`) optional
The version number of the database engine to use **Default value:** `"3.6.0"`
`external_security_group_id_list` (`list(string)`) optional
List of external security group IDs to attach to the Document DB **Default value:** `[ ]`
`instance_class` (`string`) optional
The instance class to use. For more details, see https://docs.aws.amazon.com/documentdb/latest/developerguide/db-instance-classes.html#db-instance-class-specs **Default value:** `"db.r4.large"`
`kms_key_id` (`string`) optional
The ARN for the KMS encryption key. When specifying `kms_key_id`, `storage_encrypted` needs to be set to `true` **Default value:** `""`
`manage_master_user_password` (`bool`) optional
Whether to manage the master user password using AWS Secrets Manager. **Default value:** `null`
`master_password` (`string`) optional
(Required unless a snapshot_identifier is provided) Password for the master DB user. Note that this may show up in logs, and it will be stored in the state file. Please refer to the DocumentDB Naming Constraints **Default value:** `null`
`master_username` (`string`) optional
(Required unless a snapshot_identifier is provided) Username for the master DB user **Default value:** `"admin1"`
`preferred_backup_window` (`string`) optional
Daily time range during which the backups happen **Default value:** `"07:00-09:00"`
`preferred_maintenance_window` (`string`) optional
The window to perform maintenance in. Syntax: `ddd:hh24:mi-ddd:hh24:mi`. **Default value:** `"Mon:22:00-Mon:23:00"`
`reader_dns_name` (`string`) optional
Name of the reader endpoint CNAME record to create in the parent DNS zone specified by `zone_id`. If left empty, the name will be auto-asigned using the format `replicas.var.name` **Default value:** `""`
`retention_period` (`number`) optional
Number of days to retain backups for **Default value:** `5`
`serverless_v2_scaling_configuration` optional
Configuration for serverless v2 scaling. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster#serverless_v2_scaling_configuration **Type:** ```hcl object({ min_capacity = optional(number, 0.5) max_capacity = optional(number, 256) }) ``` **Default value:** `null`
`skip_final_snapshot` (`bool`) optional
Determines whether a final DB snapshot is created before the DB cluster is deleted **Default value:** `true`
`snapshot_identifier` (`string`) optional
Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot **Default value:** `""`
`ssm_parameter_enabled` (`bool`) optional
Whether an SSM parameter store value is created to store the database password. **Default value:** `false`
`ssm_parameter_path_prefix` (`string`) optional
The path prefix for the created SSM parameter e.g. '/docdb/master-password/dev'. `ssm_parameter_enabled` must be set to `true` for this to take affect. **Default value:** `"/docdb/master-password/"`
`storage_encrypted` (`bool`) optional
Specifies whether the DB cluster is encrypted **Default value:** `true`
`storage_type` (`string`) optional
The storage type to associate with the DB cluster. Valid values: standard, iopt1 **Default value:** `"standard"`
`zone_id` (`string`) optional
Route53 parent zone ID. If provided (not empty), the module will create sub-domain DNS records for the DocumentDB master and replicas **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amazon Resource Name (ARN) of the cluster
`cluster_members`
List of DocumentDB Instances that are a part of this cluster
`cluster_name`
Cluster Identifier
`endpoint`
Endpoint of the DocumentDB cluster
`master_host`
DB master hostname
`master_password`
Password for the master DB user. If `manage_master_user_password` is set to true, this will be set to null and the password is managed by AWS in Secrets Manager.
`master_username`
Username for the master DB user
`reader_endpoint`
A read-only endpoint of the DocumentDB cluster, automatically load-balanced across replicas
`replicas_host`
DB replicas hostname
`security_group_arn`
ARN of the DocumentDB cluster Security Group
`security_group_id`
ID of the DocumentDB cluster Security Group
`security_group_name`
Name of the DocumentDB cluster Security Group
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 6.8.0` - `local`, version: `>= 1.3` - `random`, version: `>= 1.0` ### Providers - `aws`, version: `>= 6.8.0` - `random`, version: `>= 1.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_master` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `dns_replicas` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `ssm_write_db_password` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_docdb_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster) (resource) - [`aws_docdb_cluster_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster_instance) (resource) - [`aws_docdb_cluster_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_cluster_parameter_group) (resource) - [`aws_docdb_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/docdb_subnet_group) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.allow_ingress_from_self`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`random_password.password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: --- ## dynamic-subnets # Module: `dynamic-subnets` Terraform module to provision public and private [`subnets`](https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html) in an existing [`VPC`](https://aws.amazon.com/vpc) __Note:__ This module is intended for use with an existing VPC and existing Internet Gateway. To create a new VPC, use [terraform-aws-vpc](https://github.com/cloudposse/terraform-aws-vpc) module. __Note:__ Due to Terraform [limitations](https://github.com/hashicorp/terraform/issues/26755#issuecomment-719103775), many optional inputs to this module are specified as a `list(string)` that can have zero or one element, rather than as a `string` that could be empty or `null`. The designation of an input as a `list` type does not necessarily mean that you can supply more than one value in the list, so check the input's description before supplying more than one value. The core function of this module is to create 2 sets of subnets, a "public" set with bidirectional access to the public internet, and a "private" set behind a firewall with egress-only access to the public internet. This includes dividing up a given CIDR range so that a each subnet gets its own distinct CIDR range within that range, and then creating those subnets in the appropriate availability zones. The intention is to keep this module relatively simple and easy to use for the most popular use cases. In its default configuration, this module creates 1 public subnet and 1 private subnet in each of the specified availability zones. The public subnets are configured for bi-directional traffic to the public internet, while the private subnets are configured for egress-only traffic to the public internet. Rather than provide a wealth of configuration options allowing for numerous special cases, this module provides some common options and further provides the ability to suppress the creation of resources, allowing you to create and configure them as you like from outside this module. For example, rather than give you the option to customize the Network ACL, the module gives you the option to create a completely open one (and control access via Security Groups and other means) or not create one at all, allowing you to create and configure one yourself. ### Public subnets This module defines a public subnet as one that has direct access to an internet gateway and can accept incoming connection requests. In the simplest configuration, the module creates a single route table with a default route targeted to the VPC's internet gateway, and associates all the public subnets with that single route table. Likewise it creates a single Network ACL with associated rules allowing all ingress and all egress, and associates that ACL with all the public subnets. ### Private subnets A private subnet may be able to initiate traffic to the public internet through a NAT gateway, a NAT instance, or an egress-only internet gateway, or it might only have direct access to other private subnets. In the simple configuration, for IPv4 and/or IPv6 with NAT64 enabled via `public_dns64_enabled` or `private_dns64_enabled`, the module creates 1 NAT Gateway or NAT Instance for each private subnet (in the public subnet in the same availability zone), creates 1 route table for each private subnet, and adds to that route table a default route from the subnet to its NAT Gateway or Instance. For IPv6, the module adds a route to the Egress-Only Internet Gateway configured via input. As with the Public subnets, the module creates a single Network ACL with associated rules allowing all ingress and all egress, and associates that ACL with all the private subnets. ### Customization for special use cases Various features are controlled by `bool` inputs with names ending in `_enabled`. By changing the default values, you can enable or disable creation of public subnets, private subnets, route tables, NAT gateways, NAT instances, or Network ACLs. So for example, you could use this module to create only private subnets and the open Network ACL, and then add your own route table associations to the subnets and route all non-local traffic to a Transit Gateway or VPN. ### CIDR allocation For IPv4, you provide a CIDR and the module divides the address space into the largest CIDRs possible that are still small enough to accommodate `max_subnet_count` subnets of each enabled type (public or private). When `max_subnet_count` is left at the default `0`, it is set to the total number of availability zones in the region. Private subnets are allocated out of the first half of the reserved range, and public subnets are allocated out of the second half. For IPv6, you provide a `/56` CIDR and the module assigns `/64` subnets of that CIDR in consecutive order starting at zero. (You have the option of specifying a list of CIDRs instead.) As with IPv4, enough CIDRs are allocated to cover `max_subnet_count` private and public subnets (when both are enabled, which is the default), with the private subnets being allocated out of the lower half of the reservation and the public subnets allocated out of the upper half. ## Usage ```hcl module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" vpc_id = "vpc-XXXXXXXX" igw_id = ["igw-XXXXXXXX"] ipv4_cidr_block = ["10.0.0.0/16"] availability_zones = ["us-east-1a", "us-east-1b"] } ``` Create only private subnets, route to transit gateway: ```hcl module "private_tgw_subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" vpc_id = "vpc-XXXXXXXX" igw_id = ["igw-XXXXXXXX"] ipv4_cidr_block = ["10.0.0.0/16"] availability_zones = ["us-east-1a", "us-east-1b"] nat_gateway_enabled = false public_subnets_enabled = false } resource "aws_route" "private" { count = length(module.private_tgw_subnets.private_route_table_ids) route_table_id = module.private_tgw_subnets.private_route_table_ids[count.index] destination_cidr_block = "0.0.0.0/0" transit_gateway_id = "tgw-XXXXXXXXX" } ``` See [examples](https://github.com/cloudposse/terraform-aws-dynamic-subnets/tree/main/examples) for working examples. In particular, see [examples/nacls](https://github.com/cloudposse/terraform-aws-dynamic-subnets/tree/main/examples/nacls) for an example of how to create custom Network Access Control Lists (NACLs) outside of but in conjunction with this module. ## Variables ### Required Variables
`vpc_id` (`string`) required
VPC ID where subnets will be created (e.g. `vpc-aceb2723`)
### Optional Variables
`availability_zone_attribute_style` (`string`) optional
The style of Availability Zone code to use in tags and names. One of `full`, `short`, or `fixed`. When using `availability_zone_ids`, IDs will first be translated into AZ names. **Default value:** `"short"`
`availability_zone_ids` (`list(string)`) optional
List of Availability Zones IDs where subnets will be created. Overrides `availability_zones`. Useful in some regions when using only some AZs and you want to use the same ones across multiple accounts. **Default value:** `[ ]`
`availability_zones` (`list(string)`) optional
List of Availability Zones (AZs) where subnets will be created. Ignored when `availability_zone_ids` is set. The order of zones in the list ***must be stable*** or else Terraform will continually make changes. If no AZs are specified, then `max_subnet_count` AZs will be selected in alphabetical order. If `max_subnet_count > 0` and `length(var.availability_zones) > max_subnet_count`, the list will be truncated. We recommend setting `availability_zones` and `max_subnet_count` explicitly as constant (not computed) values for predictability, consistency, and stability. **Default value:** `[ ]`
`aws_route_create_timeout` (`string`) optional
DEPRECATED: Use `route_create_timeout` instead. Time to wait for AWS route creation, specified as a Go Duration, e.g. `2m` **Default value:** `null`
`aws_route_delete_timeout` (`string`) optional
DEPRECATED: Use `route_delete_timeout` instead. Time to wait for AWS route deletion, specified as a Go Duration, e.g. `2m` **Default value:** `null`
`igw_id` (`list(string)`) optional
The Internet Gateway ID that the public subnets will route traffic to. Used if `public_route_table_enabled` is `true`, ignored otherwise. **Default value:** `[ ]`
`ipv4_cidr_block` (`list(string)`) optional
Base IPv4 CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`). Ignored if `ipv4_cidrs` is set. If no CIDR block is provided, the VPC's default IPv4 CIDR block will be used. **Default value:** `[ ]`
`ipv4_cidrs` optional
Lists of CIDRs to assign to subnets. Order of CIDRs in the lists must not change over time. Lists may contain more CIDRs than needed. **Type:** ```hcl list(object({ private = list(string) public = list(string) })) ``` **Default value:** `[ ]`
`ipv4_enabled` (`bool`) optional
Set `true` to enable IPv4 addresses in the subnets **Default value:** `true`
`ipv4_private_instance_hostname_type` (`string`) optional
How to generate the DNS name for the instances in the private subnets. Either `ip-name` to generate it from the IPv4 address, or `resource-name` to generate it from the instance ID. **Default value:** `"ip-name"`
`ipv4_private_instance_hostnames_enabled` (`bool`) optional
If `true`, DNS queries for instance hostnames in the private subnets will be answered with A (IPv4) records. **Default value:** `false`
`ipv4_public_instance_hostname_type` (`string`) optional
How to generate the DNS name for the instances in the public subnets. Either `ip-name` to generate it from the IPv4 address, or `resource-name` to generate it from the instance ID. **Default value:** `"ip-name"`
`ipv4_public_instance_hostnames_enabled` (`bool`) optional
If `true`, DNS queries for instance hostnames in the public subnets will be answered with A (IPv4) records. **Default value:** `false`
`ipv6_cidr_block` (`list(string)`) optional
Base IPv6 CIDR block from which `/64` subnet CIDRs will be assigned. Must be `/56`. (e.g. `2600:1f16:c52:ab00::/56`). Ignored if `ipv6_cidrs` is set. If no CIDR block is provided, the VPC's default IPv6 CIDR block will be used. **Default value:** `[ ]`
`ipv6_cidrs` optional
Lists of CIDRs to assign to subnets. Order of CIDRs in the lists must not change over time. Lists may contain more CIDRs than needed. **Type:** ```hcl list(object({ private = list(string) public = list(string) })) ``` **Default value:** `[ ]`
`ipv6_egress_only_igw_id` (`list(string)`) optional
The Egress Only Internet Gateway ID the private IPv6 subnets will route traffic to. Used if `private_route_table_enabled` is `true` and `ipv6_enabled` is `true`, ignored otherwise. **Default value:** `[ ]`
`ipv6_enabled` (`bool`) optional
Set `true` to enable IPv6 addresses in the subnets **Default value:** `false`
`ipv6_private_instance_hostnames_enabled` (`bool`) optional
If `true` (or if `ipv4_enabled` is `false`), DNS queries for instance hostnames in the private subnets will be answered with AAAA (IPv6) records. **Default value:** `false`
`ipv6_public_instance_hostnames_enabled` (`bool`) optional
If `true` (or if `ipv4_enabled` is false), DNS queries for instance hostnames in the public subnets will be answered with AAAA (IPv6) records. **Default value:** `false`
`map_public_ip_on_launch` (`bool`) optional
If `true`, instances launched into a public subnet will be assigned a public IPv4 address **Default value:** `true`
`max_nats` (`number`) optional
Upper limit on number of NAT Gateways/Instances to create. Set to 1 or 2 for cost savings at the expense of availability. **Default value:** `999`
`max_subnet_count` (`number`) optional
Sets the maximum number of each type (public or private) of subnet to deploy. `0` will reserve a CIDR for every Availability Zone (excluding Local Zones) in the region, and deploy a subnet in each availability zone specified in `availability_zones` or `availability_zone_ids`, or every zone if none are specified. We recommend setting this equal to the maximum number of AZs you anticipate using, to avoid causing subnets to be destroyed and recreated with smaller IPv4 CIDRs when AWS adds an availability zone. Due to Terraform limitations, you can not set `max_subnet_count` from a computed value, you have to set it from an explicit constant. For most cases, `3` is a good choice. **Default value:** `0`
`metadata_http_endpoint_enabled` (`bool`) optional
Whether the metadata service is available on the created NAT instances **Default value:** `true`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for instance metadata requests on the created NAT instances **Default value:** `1`
`metadata_http_tokens_required` (`bool`) optional
Whether or not the metadata service requires session tokens, also referred to as Instance Metadata Service Version 2, on the created NAT instances **Default value:** `true`
`nat_elastic_ips` (`list(string)`) optional
Existing Elastic IPs (not EIP IDs) to attach to the NAT Gateway(s) or Instance(s) instead of creating new ones. **Default value:** `[ ]`
`nat_gateway_enabled` (`bool`) optional
Set `true` to create NAT Gateways to perform IPv4 NAT and NAT64 as needed. Defaults to `true` unless `nat_instance_enabled` is `true`. **Default value:** `null`
`nat_instance_ami_id` (`list(string)`) optional
A list optionally containing the ID of the AMI to use for the NAT instance. If the list is empty (the default), the latest official AWS NAT instance AMI will be used. NOTE: The Official NAT instance AMI is being phased out and does not support NAT64. Use of a NAT gateway is recommended instead. **Default value:** `[ ]`
`nat_instance_cpu_credits_override` (`string`) optional
NAT Instance credit option for CPU usage. Valid values are "standard" or "unlimited". T3 and later instances are launched as unlimited by default. T2 instances are launched as standard by default. **Default value:** `""`
`nat_instance_enabled` (`bool`) optional
Set `true` to create NAT Instances to perform IPv4 NAT. Defaults to `false`. **Default value:** `null`
`nat_instance_root_block_device_encrypted` (`bool`) optional
Whether to encrypt the root block device on the created NAT instances **Default value:** `true`
`nat_instance_type` (`string`) optional
NAT Instance type **Default value:** `"t3.micro"`
`open_network_acl_ipv4_rule_number` (`number`) optional
The `rule_no` assigned to the network ACL rules for IPv4 traffic generated by this module **Default value:** `100`
`open_network_acl_ipv6_rule_number` (`number`) optional
The `rule_no` assigned to the network ACL rules for IPv6 traffic generated by this module **Default value:** `111`
`private_assign_ipv6_address_on_creation` (`bool`) optional
If `true`, network interfaces created in a private subnet will be assigned an IPv6 address **Default value:** `true`
`private_dns64_nat64_enabled` (`bool`) optional
If `true` and IPv6 is enabled, DNS queries made to the Amazon-provided DNS Resolver in private subnets will return synthetic IPv6 addresses for IPv4-only destinations, and these addresses will be routed to the NAT Gateway. Requires `public_subnets_enabled`, `nat_gateway_enabled`, and `private_route_table_enabled` to be `true` to be fully operational. Defaults to `true` unless there is no public IPv4 subnet for egress, in which case it defaults to `false`. **Default value:** `null`
`private_label` (`string`) optional
The string to use in IDs and elsewhere to identify resources for the private subnets and distinguish them from resources for the public subnets **Default value:** `"private"`
`private_open_network_acl_enabled` (`bool`) optional
If `true`, a single network ACL be created and it will be associated with every private subnet, and a rule (number 100) will be created allowing all ingress and all egress. You can add additional rules to this network ACL using the `aws_network_acl_rule` resource. If `false`, you will need to manage the network ACL outside of this module. **Default value:** `true`
`private_route_table_enabled` (`bool`) optional
If `true`, a network route table and default route to the NAT gateway, NAT instance, or egress-only gateway will be created for each private subnet (1:1). If false, you will need to create your own route table(s) and route(s). **Default value:** `true`
`private_subnets_additional_tags` (`map(string)`) optional
Additional tags to be added to private subnets **Default value:** `{ }`
`private_subnets_enabled` (`bool`) optional
If false, do not create private subnets (or NAT gateways or instances) **Default value:** `true`
`public_assign_ipv6_address_on_creation` (`bool`) optional
If `true`, network interfaces created in a public subnet will be assigned an IPv6 address **Default value:** `true`
`public_dns64_nat64_enabled` (`bool`) optional
If `true` and IPv6 is enabled, DNS queries made to the Amazon-provided DNS Resolver in public subnets will return synthetic IPv6 addresses for IPv4-only destinations, and these addresses will be routed to the NAT Gateway. Requires `nat_gateway_enabled` and `public_route_table_enabled` to be `true` to be fully operational. **Default value:** `false`
`public_label` (`string`) optional
The string to use in IDs and elsewhere to identify resources for the public subnets and distinguish them from resources for the private subnets **Default value:** `"public"`
`public_open_network_acl_enabled` (`bool`) optional
If `true`, a single network ACL be created and it will be associated with every public subnet, and a rule will be created allowing all ingress and all egress. You can add additional rules to this network ACL using the `aws_network_acl_rule` resource. If `false`, you will need to manage the network ACL outside of this module. **Default value:** `true`
`public_route_table_enabled` (`bool`) optional
If `true`, network route table(s) will be created as determined by `public_route_table_per_subnet_enabled` and appropriate routes will be added to destinations this module knows about. If `false`, you will need to create your own route table(s) and route(s). Ignored if `public_route_table_ids` is non-empty. **Default value:** `true`
`public_route_table_ids` (`list(string)`) optional
List optionally containing the ID of a single route table shared by all public subnets or exactly one route table ID for each public subnet. If provided, it overrides `public_route_table_per_subnet_enabled`. If omitted and `public_route_table_enabled` is `true`, one or more network route tables will be created for the public subnets, according to the setting of `public_route_table_per_subnet_enabled`. **Default value:** `[ ]`
`public_route_table_per_subnet_enabled` (`bool`) optional
If `true` (and `public_route_table_enabled` is `true`), a separate network route table will be created for and associated with each public subnet. If `false` (and `public_route_table_enabled` is `true`), a single network route table will be created and it will be associated with every public subnet. If not set, it will be set to the value of `public_dns64_nat64_enabled`. **Default value:** `null`
`public_subnets_additional_tags` (`map(string)`) optional
Additional tags to be added to public subnets **Default value:** `{ }`
`public_subnets_enabled` (`bool`) optional
If false, do not create public subnets. Since NAT gateways and instances must be created in public subnets, these will also not be created when `false`. **Default value:** `true`
`root_block_device_encrypted` (`bool`) optional
DEPRECATED: use `nat_instance_root_block_device_encrypted` instead. Whether to encrypt the root block device on the created NAT instances **Default value:** `null`
`route_create_timeout` (`string`) optional
Time to wait for a network routing table entry to be created, specified as a Go Duration, e.g. `2m`. Use `null` for proivder default. **Default value:** `null`
`route_delete_timeout` (`string`) optional
Time to wait for a network routing table entry to be deleted, specified as a Go Duration, e.g. `2m`. Use `null` for proivder default. **Default value:** `null`
`subnet_create_timeout` (`string`) optional
Time to wait for a subnet to be created, specified as a Go Duration, e.g. `2m`. Use `null` for proivder default. **Default value:** `null`
`subnet_delete_timeout` (`string`) optional
Time to wait for a subnet to be deleted, specified as a Go Duration, e.g. `5m`. Use `null` for proivder default. **Default value:** `null`
`subnet_type_tag_key` (`string`) optional
DEPRECATED: Use `public_subnets_additional_tags` and `private_subnets_additional_tags` instead Key for subnet type tag to provide information about the type of subnets, e.g. `cpco.io/subnet/type: private` or `cpco.io/subnet/type: public` **Default value:** `null`
`subnet_type_tag_value_format` (`string`) optional
DEPRECATED: Use `public_subnets_additional_tags` and `private_subnets_additional_tags` instead. The value of the `subnet_type_tag_key` will be set to `format(var.subnet_type_tag_value_format, )` where `` is either `public` or `private`. **Default value:** `"%s"`
`subnets_per_az_count` (`number`) optional
The number of subnet of each type (public or private) to provision per Availability Zone. **Default value:** `1`
`subnets_per_az_names` (`list(string)`) optional
The subnet names of each type (public or private) to provision per Availability Zone. This variable is optional. If a list of names is provided, the list items will be used as keys in the outputs `named_private_subnets_map`, `named_public_subnets_map`, `named_private_route_table_ids_map` and `named_public_route_table_ids_map` **Default value:** ```hcl [ "common" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`availability_zone_ids`
List of Availability Zones IDs where subnets were created, when available
`availability_zones`
List of Availability Zones where subnets were created
`az_private_route_table_ids_map`
Map of AZ names to list of private route table IDs in the AZs
`az_private_subnets_map`
Map of AZ names to list of private subnet IDs in the AZs
`az_public_route_table_ids_map`
Map of AZ names to list of public route table IDs in the AZs
`az_public_subnets_map`
Map of AZ names to list of public subnet IDs in the AZs
`named_private_route_table_ids_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of private route table IDs
`named_private_subnets_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of private subnet IDs
`named_private_subnets_stats_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of objects with each object having three items: AZ, private subnet ID, private route table ID
`named_public_route_table_ids_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of public route table IDs
`named_public_subnets_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of public subnet IDs
`named_public_subnets_stats_map`
Map of subnet names (specified in `subnets_per_az_names` variable) to lists of objects with each object having three items: AZ, public subnet ID, public route table ID
`nat_eip_allocation_ids`
Elastic IP allocations in use by NAT
`nat_gateway_ids`
IDs of the NAT Gateways created
`nat_gateway_public_ips`
DEPRECATED: use `nat_ips` instead. Public IPv4 IP addresses in use by NAT.
`nat_instance_ami_id`
ID of AMI used by NAT instance
`nat_instance_ids`
IDs of the NAT Instances created
`nat_ips`
Elastic IP Addresses in use by NAT
`private_network_acl_id`
ID of the Network ACL created for private subnets
`private_route_table_ids`
IDs of the created private route tables
`private_subnet_arns`
ARNs of the created private subnets
`private_subnet_cidrs`
IPv4 CIDR blocks of the created private subnets
`private_subnet_ids`
IDs of the created private subnets
`private_subnet_ipv6_cidrs`
IPv6 CIDR blocks of the created private subnets
`public_network_acl_id`
ID of the Network ACL created for public subnets
`public_route_table_ids`
IDs of the created public route tables
`public_subnet_arns`
ARNs of the created public subnets
`public_subnet_cidrs`
IPv4 CIDR blocks of the created public subnets
`public_subnet_ids`
IDs of the created public subnets
`public_subnet_ipv6_cidrs`
IPv6 CIDR blocks of the created public subnets
## Dependencies ### Requirements - `terraform`, version: `>= 1.1.0` - `aws`, version: `>= 3.71.0` ### Providers - `aws`, version: `>= 3.71.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `nat_instance_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `nat_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `private_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `public_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | 1.4.0 | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/1.4.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_eip_association.nat_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip_association) (resource) - [`aws_instance.nat_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) (resource) - [`aws_nat_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway) (resource) - [`aws_network_acl.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_network_acl.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_network_acl_rule.private4_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.private4_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.private6_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.private6_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.public4_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.public4_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.public6_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_network_acl_rule.public6_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl_rule) (resource) - [`aws_route.nat4`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.nat_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.private6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.private_nat64`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public_nat64`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route_table.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table_association.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_route_table_association.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_security_group.nat_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.nat_instance_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.nat_instance_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_subnet.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) - [`aws_subnet.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.nat_instance`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_availability_zones.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/availability_zones) (data source) - [`aws_eip.nat`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eip) (data source) - [`aws_vpc.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) ## Subnet calculation logic `terraform-aws-dynamic-subnets` creates a set of subnets based on various CIDR inputs and the maximum possible number of subnets, which is `max_subnet_count` when specified or the number of Availability Zones in the region when `max_subnet_count` is left at its default value of zero. You can explicitly provide CIDRs for subnets via `ipv4_cidrs` and `ipv6_cidrs` inputs if you want, but the usual use case is to provide a single CIDR which this module will subdivide into a set of CIDRs as follows: 1. Get number of available AZ in the region: ``` existing_az_count = length(data.aws_availability_zones.available.names) ``` 2. Determine how many sets of subnets are being created. (Usually it is `2`: `public` and `private`): `subnet_type_count`. 3. Multiply the results of (1) and (2) to determine how many CIDRs to reserve: ``` cidr_count = existing_az_count * subnet_type_count ``` 4. Calculate the number of bits needed to enumerate all the CIDRs: ``` subnet_bits = ceil(log(cidr_count, 2)) ``` 5. Reserve CIDRs for private subnets using [`cidrsubnet`](https://www.terraform.io/language/functions/cidrsubnet): ``` private_subnet_cidrs = [ for netnumber in range(0, existing_az_count): cidrsubnet(cidr_block, subnet_bits, netnumber) ] ``` 6. Reserve CIDRs for public subnets in the second half of the CIDR block: ``` public_subnet_cidrs = [ for netnumber in range(existing_az_count, existing_az_count * 2): cidrsubnet(cidr_block, subnet_bits, netnumber) ] ``` Note that this means that, for example, in a region with 4 availability zones, if you specify only 3 availability zones in `var.availability_zones`, this module will still reserve CIDRs for the 4th zone. This is so that if you later want to expand into that zone, the existing subnet CIDR assignments will not be disturbed. If you do not want to reserve these CIDRs, set `max_subnet_count` to the number of zones you are actually using. --- ## dynamodb(Dynamodb) # Module: `dynamodb` Terraform module to provision a DynamoDB table with autoscaling. Autoscaler scales up/down the provisioned OPS for the DynamoDB table based on the load. ## Requirements This module requires [AWS Provider](https://github.com/terraform-providers/terraform-provider-aws) `>= 4.22.0` ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-dynamodb/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-dynamodb/tree/main/test). ```hcl module "dynamodb_table" { source = "cloudposse/dynamodb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" hash_key = "HashKey" range_key = "RangeKey" autoscale_write_target = 50 autoscale_read_target = 50 autoscale_min_read_capacity = 5 autoscale_max_read_capacity = 20 autoscale_min_write_capacity = 5 autoscale_max_write_capacity = 20 enable_autoscaler = true } ``` ## Advanced Usage With additional attributes, global secondary indexes and `non_key_attributes` (see [examples/complete](https://github.com/cloudposse/terraform-aws-dynamodb/tree/main/examples/complete)). ```hcl module "dynamodb_table" { source = "cloudposse/dynamodb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" hash_key = "HashKey" range_key = "RangeKey" autoscale_write_target = 50 autoscale_read_target = 50 autoscale_min_read_capacity = 5 autoscale_max_read_capacity = 20 autoscale_min_write_capacity = 5 autoscale_max_write_capacity = 20 enable_autoscaler = true dynamodb_attributes = [ { name = "DailyAverage" type = "N" }, { name = "HighWater" type = "N" }, { name = "Timestamp" type = "S" } ] local_secondary_index_map = [ { name = "TimestampSortIndex" range_key = "Timestamp" projection_type = "INCLUDE" non_key_attributes = ["HashKey", "RangeKey"] }, { name = "HighWaterIndex" range_key = "Timestamp" projection_type = "INCLUDE" non_key_attributes = ["HashKey", "RangeKey"] } ] global_secondary_index_map = [ { name = "DailyAverageIndex" hash_key = "DailyAverage" range_key = "HighWater" write_capacity = 5 read_capacity = 5 projection_type = "INCLUDE" non_key_attributes = ["HashKey", "RangeKey"] } ] replicas = ["us-east-1"] } ``` __NOTE:__ Variables "global_secondary_index_map" and "local_secondary_index_map" have a predefined schema, but in some cases not all fields are required or needed. For example: * `non_key_attributes` can't be specified for Global Secondary Indexes (GSIs) when `projection_type` is `ALL` * `read_capacity` and `write_capacity` are not required for GSIs In these cases, set the fields to `null` and Terraform will treat them as if they were not provided at all, but will not complain about missing values: ```hcl global_secondary_index_map = [ { write_capacity = null read_capacity = null projection_type = "ALL" non_key_attributes = null } ] ``` See [Terraform types and values](https://www.terraform.io/docs/configuration/expressions.html#types-and-values) for more details. ## Variables ### Required Variables
`hash_key` (`string`) required
DynamoDB table Hash Key
### Optional Variables
`autoscale_max_read_capacity` (`number`) optional
DynamoDB autoscaling max read capacity **Default value:** `20`
`autoscale_max_write_capacity` (`number`) optional
DynamoDB autoscaling max write capacity **Default value:** `20`
`autoscale_min_read_capacity` (`number`) optional
DynamoDB autoscaling min read capacity **Default value:** `5`
`autoscale_min_write_capacity` (`number`) optional
DynamoDB autoscaling min write capacity **Default value:** `5`
`autoscale_read_target` (`number`) optional
The target value (in %) for DynamoDB read autoscaling **Default value:** `50`
`autoscale_write_target` (`number`) optional
The target value (in %) for DynamoDB write autoscaling **Default value:** `50`
`autoscaler_attributes` (`list(string)`) optional
Additional attributes for the autoscaler module **Default value:** `[ ]`
`autoscaler_tags` (`map(string)`) optional
Additional resource tags for the autoscaler module **Default value:** `{ }`
`billing_mode` (`string`) optional
DynamoDB Billing mode. Can be PROVISIONED or PAY_PER_REQUEST **Default value:** `"PROVISIONED"`
`deletion_protection_enabled` (`bool`) optional
Enable/disable DynamoDB table deletion protection **Default value:** `false`
`dynamodb_attributes` optional
Additional DynamoDB attributes in the form of a list of mapped values **Type:** ```hcl list(object({ name = string type = string })) ``` **Default value:** `[ ]`
`enable_autoscaler` (`bool`) optional
Flag to enable/disable DynamoDB autoscaling **Default value:** `false`
`enable_encryption` (`bool`) optional
Enable DynamoDB server-side encryption **Default value:** `true`
`enable_point_in_time_recovery` (`bool`) optional
Enable DynamoDB point in time recovery **Default value:** `true`
`enable_streams` (`bool`) optional
Enable DynamoDB streams **Default value:** `false`
`global_secondary_index_map` optional
Additional global secondary indexes in the form of a list of mapped values **Type:** ```hcl list(object({ hash_key = string name = string non_key_attributes = list(string) projection_type = string range_key = string read_capacity = number write_capacity = number })) ``` **Default value:** `[ ]`
`hash_key_type` (`string`) optional
Hash Key type, which must be a scalar type: `S`, `N`, or `B` for (S)tring, (N)umber or (B)inary data **Default value:** `"S"`
`import_table` optional
Import Amazon S3 data into a new table. **Type:** ```hcl object({ # Valid values are GZIP, ZSTD and NONE input_compression_type = optional(string, null) # Valid values are CSV, DYNAMODB_JSON, and ION. input_format = string input_format_options = optional(object({ csv = object({ delimiter = string header_list = list(string) }) }), null) s3_bucket_source = object({ bucket = string bucket_owner = optional(string) key_prefix = optional(string) }) }) ``` **Default value:** `null`
`local_secondary_index_map` optional
Additional local secondary indexes in the form of a list of mapped values **Type:** ```hcl list(object({ name = string non_key_attributes = list(string) projection_type = string range_key = string })) ``` **Default value:** `[ ]`
`range_key` (`string`) optional
DynamoDB table Range Key **Default value:** `""`
`range_key_type` (`string`) optional
Range Key type, which must be a scalar type: `S`, `N`, or `B` for (S)tring, (N)umber or (B)inary data **Default value:** `"S"`
`replicas` (`list(string)`) optional
List of regions to create replica **Default value:** `[ ]`
`server_side_encryption_kms_key_arn` (`string`) optional
The ARN of the CMK that should be used for the AWS KMS encryption. This attribute should only be specified if the key is different from the default DynamoDB CMK, alias/aws/dynamodb. **Default value:** `null`
`stream_view_type` (`string`) optional
When an item in the table is modified, what information is written to the stream **Default value:** `""`
`table_class` (`string`) optional
DynamoDB storage class of the table. Can be STANDARD or STANDARD_INFREQUENT_ACCESS **Default value:** `"STANDARD"`
`table_name` (`string`) optional
Table name. If provided, the bucket will be created with this name instead of generating the name from the context **Default value:** `null`
`tags_enabled` (`bool`) optional
Set to `false` to disable tagging. This can be helpful if you're managing tables on dynamodb-local with terraform as it doesn't support tagging. **Default value:** `true`
`ttl_attribute` (`string`) optional
DynamoDB table TTL attribute **Default value:** `"Expires"`
`ttl_enabled` (`bool`) optional
Set to false to disable DynamoDB table TTL **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`global_secondary_index_names`
DynamoDB secondary index names
`local_secondary_index_names`
DynamoDB local index names
`table_arn`
DynamoDB table ARN
`table_id`
DynamoDB table ID
`table_name`
DynamoDB table name
`table_stream_arn`
DynamoDB table stream ARN
`table_stream_label`
DynamoDB table stream label
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.59` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 4.59` - `null`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dynamodb_autoscaler` | 0.16.0 | [`cloudposse/dynamodb-autoscaler/aws`](https://registry.terraform.io/modules/cloudposse/dynamodb-autoscaler/aws/0.16.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_dynamodb_table.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dynamodb_table) (resource) - [`null_resource.global_secondary_index_names`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`null_resource.local_secondary_index_names`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) ## Data Sources The following data sources are used by this module: --- ## dynamodb-autoscaler # Module: `dynamodb-autoscaler` Terraform module to provision DynamoDB autoscaler. Autoscaler scales up/down the provisioned OPS for a DynamoDB table based on the load. ## Usage ```hcl module "dynamodb_autoscaler" { source = "cloudposse/dynamodb-autoscaler/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" dynamodb_table_name = "eg-dev-cluster-terraform-state-lock" dynamodb_indexes = ["first-index", "second-index"] dynamodb_table_arn = "arn:aws:dynamodb:us-east-1:123456789012:table/eg-dev-cluster-terraform-state-lock" autoscale_write_target = 50 autoscale_read_target = 50 autoscale_min_read_capacity = 5 autoscale_max_read_capacity = 20 autoscale_min_write_capacity = 5 autoscale_max_write_capacity = 20 autoscale_scale_in_cooldown = null # defaults to AWS's default autoscale_scale_out_cooldown = null # defaults to AWS's default } ``` ## Variables ### Required Variables
`dynamodb_table_arn` (`string`) required
DynamoDB table ARN
`dynamodb_table_name` (`string`) required
DynamoDB table name
### Optional Variables
`autoscale_max_read_capacity` (`number`) optional
DynamoDB autoscaling max read capacity **Default value:** `20`
`autoscale_max_read_capacity_index` (`number`) optional
DynamoDB autoscaling max read capacity of the index **Default value:** `null`
`autoscale_max_write_capacity` (`number`) optional
DynamoDB autoscaling max write capacity **Default value:** `20`
`autoscale_max_write_capacity_index` (`number`) optional
DynamoDB autoscaling max write capacity of the index **Default value:** `null`
`autoscale_min_read_capacity` (`number`) optional
DynamoDB autoscaling min read capacity **Default value:** `5`
`autoscale_min_read_capacity_index` (`number`) optional
DynamoDB autoscaling min read capacity of the index **Default value:** `null`
`autoscale_min_write_capacity` (`number`) optional
DynamoDB autoscaling min write capacity **Default value:** `5`
`autoscale_min_write_capacity_index` (`number`) optional
DynamoDB autoscaling min write capacity of the index **Default value:** `null`
`autoscale_read_target` (`number`) optional
The target value for DynamoDB read autoscaling **Default value:** `50`
`autoscale_read_target_index` (`number`) optional
The target value for DynamoDB read autoscaling of the index **Default value:** `null`
`autoscale_scale_in_cooldown` (`number`) optional
DynamoDB autoscaling scale in cooldown **Default value:** `null`
`autoscale_scale_out_cooldown` (`number`) optional
DynamoDB autoscaling scale out cooldown **Default value:** `null`
`autoscale_write_target` (`number`) optional
The target value for DynamoDB write autoscaling **Default value:** `50`
`autoscale_write_target_index` (`number`) optional
The target value for DynamoDB write autoscaling of the index **Default value:** `null`
`dynamodb_indexes` (`list(string)`) optional
List of DynamoDB indexes **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`appautoscaling_read_policy_arn`
Appautoscaling read policy ARN
`appautoscaling_write_policy_arn`
Appautoscaling write policy ARN
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_appautoscaling_policy.read_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_policy.read_policy_index`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_policy.write_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_policy.write_policy_index`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_target.read_target`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) - [`aws_appautoscaling_target.read_target_index`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) - [`aws_appautoscaling_target.write_target`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) - [`aws_appautoscaling_target.write_target_index`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) ## Data Sources The following data sources are used by this module: --- ## ec2-admin-server # Module: `ec2-admin-server` Terraform Module for providing a server capable of running admin tasks. Use `terraform-aws-ec2-admin-server` to create and manage an admin instance. ## Usage Note: add `${var.ssh_key_pair}` private key to the `ssh agent`. Include this repository as a module in your existing terraform code: ```terraform module "admin_tier" { source = "git::https://github.com/cloudposse/terraform-aws-ec2-admin-server.git?ref=master" ssh_key_pair = "${var.ssh_key_pair}" github_api_token = "${var.github_api_token}" github_organization = "${var.github_organization}" github_team = "${var.github_team}" instance_type = "${var.instance_type}" vpc_id = "${var.vpc_id}" name = "admin" namespace = "${var.namespace}" stage = "${var.stage}" subnets = ["${var.subnets}"] zone_id = "${module.terraform-aws-route53-cluster-zone.zone_id}" security_groups = ["${var.security_groups}"] allow_cidr_blocks = ["${var.allow_cidr_blocks}"] } ``` ### Module `terraform-aws-route53-cluster-zone` Module `terraform-aws-ec2-admin-server` requires another module to be used additionally - `terraform-aws-route53-cluster-zone`. `terraform-aws-ec2-admin-server` uses `terraform-aws-route53-cluster-hostname` to create a DNS record for created host. `terraform-aws-route53-cluster-hostname` module needs `zone_id` parameter as an input, and this parameter actually is an output from `terraform-aws-route53-cluster-zone`. That is why `terraform-aws-route53-cluster-zone` should be implemented in `root` TF manifest when we need `terraform-aws-ec2-admin-server`. ### This module depends on the next modules: * [terraform-null-label](https://github.com/cloudposse/terraform-null-label) * [terraform-aws-ubuntu-github-authorized-keys-user-data](https://github.com/cloudposse/terraform-aws-ubuntu-github-authorized-keys-user-data) * [terraform-aws-route53-cluster-hostname](https://github.com/cloudposse/terraform-aws-route53-cluster-hostname) * [terraform-aws-route53-cluster-zone](https://github.com/cloudposse/terraform-aws-route53-cluster-zone) (not directly, but `terraform-aws-route53-cluster-hostname` need child `zone_id`) It is necessary to run `terraform get` to download those modules. Now reference the label when creating an instance (for example): ```terraform resource "aws_ami_from_instance" "example" { name = "terraform-example" source_instance_id = "${module.admin_tier.id}" } ``` ## Variables ### Required Variables
`github_api_token` (`any`) required
GitHub API token
`github_organization` (`any`) required
GitHub organization name
`github_team` (`any`) required
GitHub team
`ssh_key_pair` (`any`) required
SSH key pair to be provisioned on instance
`subnets` (`list(string)`) required
List of VPC Subnet IDs where the instance may be launched
`vpc_id` (`any`) required
The ID of the VPC where the instance will be created
### Optional Variables
`allow_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to permit SSH access **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`dns_ttl` (`string`) optional
The time for which a DNS resolver caches a response **Default value:** `"60"`
`ec2_ami` (`string`) optional
By default it is an AMI provided by Amazon with Ubuntu 16.04 **Default value:** `"ami-cd0f5cb6"`
`instance_type` (`string`) optional
The type of instance that will be created (e.g. `t2.micro`) **Default value:** `"t2.micro"`
`security_groups` (`list(string)`) optional
List of Security Group IDs permitted to connect to this instance **Default value:** `[ ]`
`zone_id` (`string`) optional
Route53 DNS Zone id **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`any`) required
The Name of the application or solution (e.g. `bastion` or `portal`) **Required:** Yes **Default value:** ``
`namespace` (`any`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`any`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `policy` or `role`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage`, etc. **Required:** No **Default value:** `"-"`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit','XYZ')`) **Required:** No **Default value:** `{ }`
## Outputs
`fqhn`
DNS name (Fully Qualified Host Name) of creating instance
`id`
Disambiguated ID
`public_ip`
IPv4 Public IP
`role`
Name of AWS IAM Role associated with creating instance
`security_group_ids`
List of IDs of AWS Security Groups associated with creating instance
`ssh_key_pair`
Name of used AWS SSH key
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `instance` | 1.2.1 | [`cloudposse/ec2-instance/aws`](https://registry.terraform.io/modules/cloudposse/ec2-instance/aws/1.2.1) | Use terraform-aws-ec2-instance module `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ssh`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: --- ## ec2-ami-backup # Module: `ec2-ami-backup` This repo contains a terraform module that creates two lambda functions that will create AMI automatically at regular intervals. It is based on the code at https://serverlesscode.com/post/lambda-schedule-ebs-snapshot-backups/ and https://serverlesscode.com/post/lambda-schedule-ebs-snapshot-backups-2/. ## Usage Include this repository as a module in your existing terraform code: ```hcl module "lambda_ami_backup" { source = "git::https://github.com/cloudposse/terraform-aws-ec2-ami-backup.git?ref=tags/0.3.2" name = "${var.name}" stage = "${var.stage}" namespace = "${var.namespace}" region = "${var.region}" ami_owner = "${var.ami_owner}" instance_id = "${var.instance_id}" retention_days = "14" } ``` ## Examples Example on excluding some of attached EBS volumes: ```hcl module "lambda_ami_backup" { source = "git::https://github.com/cloudposse/terraform-aws-ec2-ami-backup.git?ref=tags/0.3.2" name = "${var.name}" stage = "${var.stage}" namespace = "${var.namespace}" region = "${var.region}" ami_owner = "${var.ami_owner}" instance_id = "${var.instance_id}" retention_days = "14" block_device_mappings = [ { "DeviceName" = "/dev/xvdf", "NoDevice" = "" }, { "DeviceName" = "/dev/xvdg", "NoDevice" = "" }, ] } ``` ## Variables ### Required Variables
`instance_id` (`any`) required
AWS Instance ID which is used for creating the AMI image (e.g. `id-123456789012`)
### Optional Variables
`ami_owner` (`string`) optional
AWS Account ID which is used as a filter for AMI list (e.g. `123456789012`) **Default value:** `""`
`backup_schedule` (`string`) optional
The scheduling expression. (e.g. cron(0 20 * * ? *) or rate(5 minutes) **Default value:** `"cron(00 19 * * ? *)"`
`block_device_mappings` (`list(string)`) optional
List of block device mappings to be included/excluded from created AMIs. With default value of [], AMIs will include all attached EBS volumes **Default value:** `[ ]`
`cleanup_schedule` (`string`) optional
The scheduling expression. (e.g. cron(0 20 * * ? *) or rate(5 minutes) **Default value:** `"cron(05 19 * * ? *)"`
`reboot` (`string`) optional
Reboot the machine as part of the snapshot process **Default value:** `"false"`
`region` (`string`) optional
AWS Region where module should operate (e.g. `us-east-1`) **Default value:** `""`
`retention_days` (`string`) optional
Is the number of days you want to keep the backups for (e.g. `14`) **Default value:** `"14"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`string`) optional
Name (e.g. `bastion` or `db`) **Required:** No **Default value:** `""`
`namespace` (`string`) optional
Namespace (e.g. `cp` or `cloudposse`) **Required:** No **Default value:** `""`
`stage` (`string`) optional
Stage (e.g. `prod`, `dev`, `staging`) **Required:** No **Default value:** `""`
## Dependencies ### Providers - `archive` - `aws` - `null` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | tags/0.3.7 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.7) | n/a `label_backup` | tags/0.3.7 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.7) | n/a `label_cleanup` | tags/0.3.7 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.7) | n/a `label_role` | tags/0.3.7 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.7) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_rule.ami_cleanup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_cloudwatch_event_target.ami_cleanup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_role.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_lambda_function.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_function.ami_cleanup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_lambda_permission.ami_cleanup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`null_resource.schedule`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) ## Data Sources The following data sources are used by this module: - [`archive_file.ami_backup`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`archive_file.ami_cleanup`](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) (data source) - [`aws_iam_policy_document.ami_backup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## ec2-ami-snapshot # Module: `ec2-ami-snapshot` Terraform module to easily generate AMI snapshots to create replica instances ## Usage ### Create an AWS instance ``` resource "aws_instance" "default" { ami = "ami-408c7f28" instance_type = "t1.micro" tags = { Name = "test1" } } ``` ### Create AMI from an AWS instance. Instance ID is required" ``` module "tf_ami_from_instance" { source = "git::https://github.com/cloudposse/tf_ami_from_instance.git?ref=master" source_instance_id = "${aws_instance.web.id}" stage = "${var.stage}" namespace = "${var.namespace}" name = "${var.name}" attributes = "${var.attributes}" tags = "${var.tags}" } ``` ## Caveats * Terraform will only keep the latest AMI snapshot (terraform will delete the previously generated AMI) See our Lamda based solution which avoids this pitfall: https://github.com/cloudposse/tf_lambda_ami_backup * This is is not compatible with autoscaling groups ## Variables ### Required Variables
`source_instance_id` (`any`) required
### Optional Variables
`snapshot_without_reboot` (`string`) optional
**Default value:** `"true"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`attributes` (`list(string)`) optional
Additional attributes (e.g. `policy` or `role`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage`, etc. **Required:** No **Default value:** `"-"`
`name` (`string`) optional
The Name of the application or solution (e.g. `bastion` or `portal`) **Required:** No **Default value:** `""`
`namespace` (`string`) optional
Namespace (e.g. `cp` or `cloudposse`) **Required:** No **Default value:** `""`
`stage` (`string`) optional
Stage (e.g. `prod`, `dev`, `staging`) **Required:** No **Default value:** `""`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit','XYZ')`) **Required:** No **Default value:** `{ }`
## Outputs
`ami_id`
AMI ID depends on the instance type and region in which you're launching your stack. And IDs can change regularly, such as when an AMI is updated with software updates.
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.2.0 | [`git::https://github.com/cloudposse/tf_label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/tf_label.git/0.2.0) | n/a ## Resources The following resources are used by this module: - [`aws_ami_from_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ami_from_instance) (resource) ## Data Sources The following data sources are used by this module: --- ## ec2-autoscale-group # Module: `ec2-autoscale-group` Terraform module to provision [Auto Scaling Group](https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html) and [Launch Template](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-launch-templates.html) on AWS. The module also creates AutoScaling Policies and CloudWatch Metric Alarms to monitor CPU utilization on the EC2 instances and scale the number of instance in the AutoScaling Group up or down. If you don't want to use the provided functionality, or want to provide your own policies, disable it by setting the variable `autoscaling_policies_enabled` to `false`. At present, although you can set the created AutoScaling Policy type to any legal value, in practice [only `SimpleScaling` is supported](https://github.com/cloudposse/terraform-aws-ec2-autoscale-group/issues/55). To use a `StepScaling` or `TargetTrackingScaling` policy, create it yourself and then pass it in the `alarm_actions` field of `custom_alarms`. ## Usage ```hcl locals { userdata = <<-USERDATA #!/bin/bash cat <<"__EOF__" > /home/ec2-user/.ssh/config Host * StrictHostKeyChecking no __EOF__ chmod 600 /home/ec2-user/.ssh/config chown ec2-user:ec2-user /home/ec2-user/.ssh/config USERDATA } module "autoscale_group" { source = "cloudposse/ec2-autoscale-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage environment = var.environment name = var.name image_id = "ami-08cab282f9979fc7a" instance_type = "t2.small" security_group_ids = ["sg-xxxxxxxx"] subnet_ids = ["subnet-xxxxxxxx", "subnet-yyyyyyyy", "subnet-zzzzzzzz"] health_check_type = "EC2" min_size = 2 max_size = 3 wait_for_capacity_timeout = "5m" associate_public_ip_address = true user_data_base64 = base64encode(local.userdata) # All inputs to `block_device_mappings` have to be defined block_device_mappings = [ { device_name = "/dev/sda1" no_device = "false" virtual_name = "root" ebs = { encrypted = true volume_size = 200 delete_on_termination = true iops = null kms_key_id = null snapshot_id = null volume_type = "standard" } } ] tags = { Tier = "1" KubernetesCluster = "us-west-2.testing.cloudposse.co" } # Auto-scaling policies and CloudWatch metric alarms autoscaling_policies_enabled = true cpu_utilization_high_threshold_percent = "70" cpu_utilization_low_threshold_percent = "20" } ``` To enable custom_alerts the map needs to be defined like so : ```hlc alarms = { alb_scale_up = { alarm_name = "${module.default_label.id}-alb-target-response-time-for-scale-up" comparison_operator = "GreaterThanThreshold" evaluation_periods = var.alb_target_group_alarms_evaluation_periods metric_name = "TargetResponseTime" namespace = "AWS/ApplicationELB" period = var.alb_target_group_alarms_period statistic = "Average" threshold = var.alb_target_group_alarms_response_time_max_threshold dimensions_name = "LoadBalancer" dimensions_target = data.alb.arn_suffix treat_missing_data = "missing" ok_actions = var.alb_target_group_alarms_ok_actions insufficient_data_actions = var.alb_target_group_alarms_insufficient_data_actions alarm_description = "ALB Target response time is over ${var.alb_target_group_alarms_response_time_max_threshold} for more than ${var.alb_target_group_alarms_period}" alarm_actions = [module.autoscaling.scale_up_policy_arn] } } ``` All those keys are required to be there so if the alarm you are setting does not requiere one or more keys you can just set to empty but do not remove the keys otherwise you could get a weird merge error due to the maps being of different sizes. ## Variables ### Required Variables
`instance_type` (`string`) required
Instance type to launch
`max_size` (`number`) required
The maximum size of the autoscale group
`min_size` (`number`) required
The minimum size of the autoscale group
`subnet_ids` (`list(string)`) required
A list of subnet IDs to launch resources in
### Optional Variables
`associate_public_ip_address` (`bool`) optional
Associate a public IP address with an instance in a VPC. If `network_interface_id` is specified, this can only be `false` (see here for more info: https://stackoverflow.com/a/76808361). **Default value:** `false`
`autoscaling_policies_enabled` (`bool`) optional
Whether to create `aws_autoscaling_policy` and `aws_cloudwatch_metric_alarm` resources to control Auto Scaling **Default value:** `true`
`block_device_mappings` optional
Specify volumes to attach to the instance besides the volumes specified by the AMI **Type:** ```hcl list(object({ device_name = optional(string) no_device = optional(bool) virtual_name = optional(string) ebs = object({ delete_on_termination = optional(bool) encrypted = optional(bool) iops = optional(number) throughput = optional(number) kms_key_id = optional(string) snapshot_id = optional(string) volume_size = optional(number) volume_type = optional(string) }) })) ``` **Default value:** `[ ]`
`capacity_rebalance` (`bool`) optional
Indicates whether capacity rebalance is enabled. Otherwise, capacity rebalance is disabled. **Default value:** `false`
`cpu_utilization_high_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_high_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_high_statistic` (`string`) optional
The statistic to apply to the alarm's associated metric. Either of the following is supported: `SampleCount`, `Average`, `Sum`, `Minimum`, `Maximum` **Default value:** `"Average"`
`cpu_utilization_high_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `90`
`cpu_utilization_low_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_low_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_low_statistic` (`string`) optional
The statistic to apply to the alarm's associated metric. Either of the following is supported: `SampleCount`, `Average`, `Sum`, `Minimum`, `Maximum` **Default value:** `"Average"`
`cpu_utilization_low_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `10`
`credit_specification` optional
Customize the credit specification of the instances **Type:** ```hcl object({ cpu_credits = string }) ``` **Default value:** `null`
`custom_alarms` optional
Map of custom CloudWatch alarms configurations **Type:** ```hcl map(object({ alarm_name = string comparison_operator = string evaluation_periods = string metric_name = string namespace = string period = string statistic = string extended_statistic = string threshold = string treat_missing_data = string ok_actions = list(string) insufficient_data_actions = list(string) dimensions_name = string dimensions_target = string alarm_description = string alarm_actions = list(string) })) ``` **Default value:** `{ }`
`default_alarms_enabled` (`bool`) optional
Enable or disable cpu and memory Cloudwatch alarms **Default value:** `true`
`default_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes before another scaling activity can start **Default value:** `300`
`desired_capacity` (`number`) optional
The number of Amazon EC2 instances that should be running in the group, if not set will use `min_size` as value **Default value:** `null`
`disable_api_termination` (`bool`) optional
If `true`, enables EC2 Instance Termination Protection **Default value:** `false`
`ebs_optimized` (`bool`) optional
If true, the launched EC2 instance will be EBS-optimized **Default value:** `false`
`enable_monitoring` (`bool`) optional
Enable/disable detailed monitoring **Default value:** `true`
`enabled_metrics` (`list(string)`) optional
A list of metrics to collect. The allowed values are `GroupMinSize`, `GroupMaxSize`, `GroupDesiredCapacity`, `GroupInServiceInstances`, `GroupPendingInstances`, `GroupStandbyInstances`, `GroupTerminatingInstances`, `GroupTotalInstances` **Default value:** ```hcl [ "GroupMinSize", "GroupMaxSize", "GroupDesiredCapacity", "GroupInServiceInstances", "GroupPendingInstances", "GroupStandbyInstances", "GroupTerminatingInstances", "GroupTotalInstances" ] ```
`force_delete` (`bool`) optional
Allows deleting the autoscaling group without waiting for all instances in the pool to terminate. You can force an autoscaling group to delete even if it's in the process of scaling a resource. Normally, Terraform drains all the instances before deleting the group. This bypasses that behavior and potentially leaves resources dangling **Default value:** `false`
`health_check_grace_period` (`number`) optional
Time (in seconds) after instance comes into service before checking health **Default value:** `300`
`health_check_type` (`string`) optional
Controls how health checking is done. Valid values are `EC2` or `ELB` **Default value:** `"EC2"`
`iam_instance_profile_name` (`string`) optional
The IAM instance profile name to associate with launched instances **Default value:** `""`
`image_id` (`string`) optional
The EC2 image ID to launch **Default value:** `""`
`instance_initiated_shutdown_behavior` (`string`) optional
Shutdown behavior for the instances. Can be `stop` or `terminate` **Default value:** `"terminate"`
`instance_market_options` optional
The market (purchasing) option for the instances **Type:** ```hcl object({ market_type = string spot_options = optional(object({ block_duration_minutes = optional(number) instance_interruption_behavior = optional(string) max_price = optional(number) spot_instance_type = optional(string) valid_until = optional(string) })) }) ``` **Default value:** `null`
`instance_refresh` optional
The instance refresh definition **Type:** ```hcl object({ strategy = string preferences = optional(object({ instance_warmup = optional(number, null) min_healthy_percentage = optional(number, null) max_healthy_percentage = optional(number, null) skip_matching = optional(bool, null) auto_rollback = optional(bool, null) scale_in_protected_instances = optional(string, null) standby_instances = optional(string, null) }), null) triggers = optional(list(string), []) }) ``` **Default value:** `null`
`instance_reuse_policy` optional
If warm pool and this block are configured, instances in the Auto Scaling group can be returned to the warm pool on scale in. The default is to terminate instances in the Auto Scaling group when the group scales in. **Type:** ```hcl object({ reuse_on_scale_in = bool }) ``` **Default value:** `null`
`key_name` (`string`) optional
The SSH key name that should be used for the instance **Default value:** `""`
`launch_template_version` (`string`) optional
Launch template version. Can be version number, `$Latest` or `$Default` **Default value:** `"$Latest"`
`load_balancers` (`list(string)`) optional
A list of elastic load balancer names to add to the autoscaling group names. Only valid for classic load balancers. For ALBs, use `target_group_arns` instead **Default value:** `[ ]`
`max_instance_lifetime` (`number`) optional
The maximum amount of time, in seconds, that an instance can be in service, values must be either equal to 0 or between 604800 and 31536000 seconds **Default value:** `null`
`metadata_http_endpoint_enabled` (`bool`) optional
Set false to disable the Instance Metadata Service. **Default value:** `true`
`metadata_http_protocol_ipv6_enabled` (`bool`) optional
Set true to enable IPv6 in the launch template. **Default value:** `false`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for Instance Metadata Service requests. The default is `2` to support containerized workloads. **Default value:** `2`
`metadata_http_tokens_required` (`bool`) optional
Set true to require IMDS session tokens, disabling Instance Metadata Service Version 1. **Default value:** `true`
`metadata_instance_metadata_tags_enabled` (`bool`) optional
Set true to enable metadata tags in the launch template. **Default value:** `false`
`metrics_granularity` (`string`) optional
The granularity to associate with the metrics to collect. The only valid value is 1Minute **Default value:** `"1Minute"`
`min_elb_capacity` (`number`) optional
Setting this causes Terraform to wait for this number of instances to show up healthy in the ELB only on creation. Updates will not wait on ELB instance number changes **Default value:** `0`
`mixed_instances_policy` optional
policy to used mixed group of on demand/spot of differing types. Launch template is automatically generated. https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#mixed_instances_policy-1 **Type:** ```hcl object({ instances_distribution = optional(object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string })) override = optional(list(object({ instance_type = string weighted_capacity = number }))) }) ``` **Default value:** `null`
`network_interface_id` (`string`) optional
The ID of the network interface to attach. If specified, all the other network_interface block arguments are ignored. **Default value:** `null`
`placement` optional
The placement specifications of the instances **Type:** ```hcl object({ affinity = string availability_zone = string group_name = string host_id = string tenancy = string }) ``` **Default value:** `null`
`placement_group` (`string`) optional
The name of the placement group into which you'll launch your instances, if any **Default value:** `""`
`protect_from_scale_in` (`bool`) optional
Allows setting instance protection. The autoscaling group will not select instances with this setting for terminination during scale in events **Default value:** `false`
`scale_down_adjustment_type` (`string`) optional
Specifies whether the adjustment is an absolute number or a percentage of the current capacity. Valid values are `ChangeInCapacity`, `ExactCapacity` and `PercentChangeInCapacity` **Default value:** `"ChangeInCapacity"`
`scale_down_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`scale_down_policy_type` (`string`) optional
The scaling policy type. Currently only `SimpleScaling` is supported **Default value:** `"SimpleScaling"`
`scale_down_scaling_adjustment` (`number`) optional
The number of instances by which to scale. `scale_down_scaling_adjustment` determines the interpretation of this number (e.g. as an absolute number or as a percentage of the existing Auto Scaling group size). A positive increment adds to the current capacity and a negative value removes from the current capacity **Default value:** `-1`
`scale_up_adjustment_type` (`string`) optional
Specifies whether the adjustment is an absolute number or a percentage of the current capacity. Valid values are `ChangeInCapacity`, `ExactCapacity` and `PercentChangeInCapacity` **Default value:** `"ChangeInCapacity"`
`scale_up_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`scale_up_policy_type` (`string`) optional
The scaling policy type. Currently only `SimpleScaling` is supported **Default value:** `"SimpleScaling"`
`scale_up_scaling_adjustment` (`number`) optional
The number of instances by which to scale. `scale_up_adjustment_type` determines the interpretation of this number (e.g. as an absolute number or as a percentage of the existing Auto Scaling group size). A positive increment adds to the current capacity and a negative value removes from the current capacity **Default value:** `1`
`security_group_ids` (`list(string)`) optional
A list of associated security group IDs **Default value:** `[ ]`
`service_linked_role_arn` (`string`) optional
The ARN of the service-linked role that the ASG will use to call other AWS services **Default value:** `""`
`suspended_processes` (`list(string)`) optional
A list of processes to suspend for the AutoScaling Group. The allowed values are `Launch`, `Terminate`, `HealthCheck`, `ReplaceUnhealthy`, `AZRebalance`, `AlarmNotification`, `ScheduledActions`, `AddToLoadBalancer`. Note that if you suspend either the `Launch` or `Terminate` process types, it can prevent your autoscaling group from functioning properly. **Default value:** `[ ]`
`tag_specifications_resource_types` (`set(string)`) optional
List of tag specification resource types to tag. Valid values are instance, volume, elastic-gpu and spot-instances-request. **Default value:** ```hcl [ "instance", "volume" ] ```
`target_group_arns` (`list(string)`) optional
A list of aws_alb_target_group ARNs, for use with Application Load Balancing **Default value:** `[ ]`
`termination_policies` (`list(string)`) optional
A list of policies to decide how the instances in the auto scale group should be terminated. The allowed values are `OldestInstance`, `NewestInstance`, `OldestLaunchConfiguration`, `ClosestToNextInstanceHour`, `Default` **Default value:** ```hcl [ "Default" ] ```
`update_default_version` (`bool`) optional
Whether to update Default version of Launch template each update **Default value:** `false`
`user_data` (`string`) optional
The cleartext user data to be Base64-encoded to provide when launching the instances **Default value:** `""`
`user_data_base64` (`string`) optional
The Base64-encoded user data to provide when launching the instances **Default value:** `""`
`wait_for_capacity_timeout` (`string`) optional
A maximum duration that Terraform should wait for ASG instances to be healthy before timing out. Setting this to '0' causes Terraform to skip all Capacity Waiting behavior **Default value:** `"10m"`
`wait_for_elb_capacity` (`number`) optional
Setting this will cause Terraform to wait for exactly this number of healthy instances in all attached load balancers on both create and update operations. Takes precedence over `min_elb_capacity` behavior **Default value:** `0`
`warm_pool` optional
If this block is configured, add a Warm Pool to the specified Auto Scaling group. See [warm_pool](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group#warm_pool). **Type:** ```hcl object({ pool_state = string min_size = number max_group_prepared_capacity = number }) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`autoscaling_group_arn`
ARN of the AutoScaling Group
`autoscaling_group_default_cooldown`
Time between a scaling activity and the succeeding scaling activity
`autoscaling_group_desired_capacity`
The number of Amazon EC2 instances that should be running in the group
`autoscaling_group_health_check_grace_period`
Time after instance comes into service before checking health
`autoscaling_group_health_check_type`
`EC2` or `ELB`. Controls how health checking is done
`autoscaling_group_id`
The AutoScaling Group id
`autoscaling_group_max_size`
The maximum size of the autoscale group
`autoscaling_group_min_size`
The minimum size of the autoscale group
`autoscaling_group_name`
The AutoScaling Group name
`autoscaling_group_tags`
A list of tag settings associated with the AutoScaling Group
`autoscaling_policy_scale_down_arn`
ARN of the AutoScaling policy scale down
`autoscaling_policy_scale_up_arn`
ARN of the AutoScaling policy scale up
`launch_template_arn`
The ARN of the launch template
`launch_template_id`
The ID of the launch template
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.16` ### Providers - `aws`, version: `>= 5.16` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_autoscaling_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_group) (resource) - [`aws_autoscaling_policy.scale_down`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_policy) (resource) - [`aws_autoscaling_policy.scale_up`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/autoscaling_policy) (resource) - [`aws_cloudwatch_metric_alarm.all_alarms`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_launch_template.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) (resource) ## Data Sources The following data sources are used by this module: - [`aws_subnet.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) --- ## ec2-bastion-server # Module: `ec2-bastion-server` Terraform module to define a generic Bastion host with parameterized `user_data` and support for AWS SSM Session Manager for remote access with IAM authentication. ## Variables ### Required Variables
`subnets` (`list(string)`) required
AWS subnet IDs
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`ami` (`string`) optional
AMI to use for the instance. Setting this will ignore `ami_filter` and `ami_owners`. **Default value:** `null`
`ami_filter` (`map(list(string))`) optional
List of maps used to create the AMI filter for the action runner AMI. **Default value:** ```hcl { "name": [ "amzn2-ami-hvm-2.*-x86_64-ebs" ] } ```
`ami_owners` (`list(string)`) optional
The list of owners used to select the AMI of action runner instances. **Default value:** ```hcl [ "amazon" ] ```
`assign_eip_address` (`bool`) optional
Assign an Elastic IP address to the instance **Default value:** `true`
`associate_public_ip_address` (`bool`) optional
Whether to associate a public IP to the instance. **Default value:** `false`
`disable_api_termination` (`bool`) optional
Enable EC2 Instance Termination Protection **Default value:** `false`
`ebs_block_device_encrypted` (`bool`) optional
Whether to encrypt the EBS block device **Default value:** `true`
`ebs_block_device_volume_size` (`number`) optional
The volume size (in GiB) to provision for the EBS block device. Creation skipped if size is 0 **Default value:** `0`
`ebs_delete_on_termination` (`bool`) optional
Whether the EBS volume should be destroyed on instance termination **Default value:** `true`
`ebs_device_name` (`string`) optional
The name of the EBS block device to mount on the instance **Default value:** `"/dev/sdh"`
`ebs_snapshot_id` (`string`) optional
The snapshot id to use for the EBS block device **Default value:** `""`
`host_name` (`string`) optional
The Bastion hostname created in Route53 **Default value:** `"bastion"`
`instance_profile` (`string`) optional
A pre-defined profile to attach to the instance (default is to build our own) **Default value:** `""`
`instance_type` (`string`) optional
Bastion instance type **Default value:** `"t2.micro"`
`key_name` (`string`) optional
Key name **Default value:** `""`
`metadata_http_endpoint_enabled` (`bool`) optional
Whether the metadata service is available **Default value:** `true`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for instance metadata requests. **Default value:** `1`
`metadata_http_tokens_required` (`bool`) optional
Whether or not the metadata service requires session tokens, also referred to as Instance Metadata Service Version 2. **Default value:** `true`
`monitoring` (`bool`) optional
Launched EC2 instance will have detailed monitoring enabled **Default value:** `true`
`root_block_device_encrypted` (`bool`) optional
Whether to encrypt the root block device **Default value:** `true`
`root_block_device_volume_size` (`number`) optional
The volume size (in GiB) to provision for the root block device. It cannot be smaller than the AMI it refers to. **Default value:** `8`
`security_group_description` (`string`) optional
The Security Group description. **Default value:** `"Bastion host security group"`
`security_group_enabled` (`bool`) optional
Whether to create default Security Group for bastion host. **Default value:** `true`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully complated with `aws_security_group_rule` resource. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "description": "Allow all outbound traffic", "from_port": 0, "protocol": -1, "to_port": 0, "type": "egress" }, { "cidr_blocks": [ "0.0.0.0/0" ], "description": "Allow all inbound to SSH", "from_port": 22, "protocol": "tcp", "to_port": 22, "type": "ingress" } ] ```
`security_group_use_name_prefix` (`bool`) optional
Whether to create a default Security Group with unique name beginning with the normalized prefix. **Default value:** `false`
`security_groups` (`list(string)`) optional
A list of Security Group IDs to associate with bastion host. **Default value:** `[ ]`
`ssh_user` (`string`) optional
Default SSH user for this AMI. e.g. `ec2-user` for Amazon Linux and `ubuntu` for Ubuntu systems **Default value:** `"ec2-user"`
`ssm_enabled` (`bool`) optional
Enable SSM Agent on Host. **Default value:** `true`
`user_data` (`list(string)`) optional
User data content. Will be ignored if `user_data_base64` is set **Default value:** `[ ]`
`user_data_base64` (`string`) optional
The Base64-encoded user data to provide when launching the instances. If this is set then `user_data` will not be used. **Default value:** `""`
`user_data_template` (`string`) optional
User Data template to use for provisioning EC2 Bastion Host **Default value:** `"user_data/amazon-linux.sh"`
`zone_id` (`string`) optional
Route53 DNS Zone ID **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the instance
`hostname`
DNS hostname
`id`
Disambiguated ID of the instance
`instance_id`
Instance ID
`name`
Instance name
`private_dns`
Private DNS of instance
`private_ip`
Private IP of the instance
`public_dns`
Public DNS of instance (or DNS of EIP)
`public_ip`
Public IP of the instance (or EIP)
`role`
Name of AWS IAM Role associated with the instance
`security_group_arn`
Bastion host Security Group ARN
`security_group_id`
Bastion host Security Group ID
`security_group_ids`
IDs on the AWS Security Groups associated with the instance
`security_group_name`
Bastion host Security Group name
`ssh_user`
SSH user
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.55` ### Providers - `aws`, version: `>= 2.55` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | 0.12.2 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.12.2) | n/a `security_group` | 0.3.3 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/0.3.3) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_route53_zone.domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## ec2-client-vpn(Ec2-client-vpn) # Module: `ec2-client-vpn` The `terraform-aws-ec2-client-vpn` project provides for ec2 client vpn infrastructure. AWS Client VPN is a managed client-based VPN service based on OpenVPN that enables you to securely access your AWS resources and resources in your on-premises network. With Client VPN, you can access your resources from any location using [any OpenVPN-based VPN client](https://docs.aws.amazon.com/vpn/latest/clientvpn-user/connect-aws-client-vpn-connect.html). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ec2-client-vpn/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ec2-client-vpn/tree/main/test). ```hcl module "vpc_target" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "172.16.0.0/16" context = module.this.context } module "vpc_client" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "172.31.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc_target.vpc_id igw_id = module.vpc_target.igw_id cidr_block = module.vpc_target.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false context = module.this.context } module "ec2_client_vpn" { source = "cloudposse/ec2-client-vpn/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" client_cidr = module.vpc_client.vpc_cidr_block organization_name = var.organization_name logging_enabled = var.logging_enabled retention_in_days = var.retention_in_days associated_subnets = module.subnets.private_subnet_ids authorization_rules = var.authorization_rules additional_routes = [ { destination_cidr_block = "0.0.0.0/0" description = "Internet Route" target_vpc_subnet_id = element(module.subnets.private_subnet_ids, 0) } ] } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-ec2-client-vpn/examples/complete/) - complete example of using this module ## Variables ### Required Variables
`associated_subnets` (`list(string)`) required
List of subnets to associate with the VPN endpoint
`client_cidr` (`string`) required
Network CIDR to use for clients
`logging_stream_name` (`string`) required
Names of stream used for logging
`organization_name` (`string`) required
Name of organization to use in private certificate
`vpc_id` (`string`) required
ID of VPC to attach VPN to
### Optional Variables
`additional_routes` optional
A list of additional routes that should be attached to the Client VPN endpoint **Type:** ```hcl list(object({ destination_cidr_block = string description = string target_vpc_subnet_id = string })) ``` **Default value:** `[ ]`
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`additional_security_groups` (`list(string)`) optional
DEPRECATED: Use `associated_security_group_ids` instead. List of security groups to attach to the client vpn network associations **Default value:** `[ ]`
`allow_self_security_group` (`bool`) optional
Whether the security group itself will be added as a source to this ingress rule. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_cidr_blocks` (`list(string)`) optional
A list of IPv6 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_prefix_list_ids` (`list(string)`) optional
A list of IPv6 Prefix Lists IDs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the VPN endpoints with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`authentication_type` (`string`) optional
One of `certificate-authentication` or `federated-authentication` **Default value:** `"certificate-authentication"`
`authorization_rules` (`list(map(any))`) optional
List of objects describing the authorization rules for the client vpn **Default value:** `[ ]`
`ca_common_name` (`string`) optional
Unique Common Name for CA self-signed certificate **Default value:** `null`
`client_conf_tmpl_path` (`string`) optional
Path to template file of vpn client exported configuration. Path is relative to $\{path.module\} **Default value:** `null`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`dns_servers` (`list(string)`) optional
Information about the DNS servers to be used for DNS resolution. A Client VPN endpoint can have up to two DNS servers. If no DNS server is specified, the DNS address of the connecting device is used. **Default value:** `[ ]`
`export_client_certificate` (`bool`) optional
Flag to determine whether to export the client certificate with the VPN configuration **Default value:** `false`
`logging_enabled` (`bool`) optional
Enables or disables Client VPN Cloudwatch logging. **Default value:** `false`
`logging_permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the IAM role used in the CloudWatch logging configuration. **Default value:** `null`
`retention_in_days` (`number`) optional
Number of days you want to retain log events in the log group **Default value:** `"30"`
`root_common_name` (`string`) optional
Unique Common Name for Root self-signed certificate **Default value:** `null`
`saml_metadata_document` (`string`) optional
Optional SAML metadata document. Must include this or `saml_provider_arn` **Default value:** `null`
`saml_provider_arn` (`string`) optional
Optional SAML provider ARN. Must include this or `saml_metadata_document` **Default value:** `null`
`secret_path_format` (`string`) optional
The path format to use when writing secrets to the certificate backend. The certificate secret path will be computed as `format(var.secret_path_format, var.name, var.secret_extensions.certificate)` and the private key path as `format(var.secret_path_format, var.name, var.secret_extensions.private_key)`. Thus by default, if `var.name`=`example-self-signed-cert`, then the resulting secret paths for the self-signed certificate's PEM file and private key will be `/example-self-signed-cert.pem` and `/example-self-signed-cert.key`, respectively. This variable can be overridden in order to create more specific certificate backend paths. **Default value:** `"/%s.%s"`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `null`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`self_service_portal_enabled` (`bool`) optional
Specify whether to enable the self-service portal for the Client VPN endpoint **Default value:** `false`
`self_service_saml_provider_arn` (`string`) optional
The ARN of the IAM SAML identity provider for the self service portal if type is federated-authentication. **Default value:** `null`
`server_common_name` (`string`) optional
Unique Common Name for Server self-signed certificate **Default value:** `null`
`session_timeout_hours` (`string`) optional
The maximum session duration is a trigger by which end-users are required to re-authenticate prior to establishing a VPN session. Default value is 24. Valid values: 8 | 10 | 12 | 24 **Default value:** `"24"`
`split_tunnel` (`bool`) optional
Indicates whether split-tunnel is enabled on VPN endpoint. Default value is false. **Default value:** `false`
`transport_protocol` (`string`) optional
Transport protocol used by the TLS sessions. **Default value:** `"udp"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`client_configuration`
VPN Client Configuration data.
`full_client_configuration`
Client configuration including client certificate and private key
`log_group_arn`
The ARN of the CloudWatch Log Group used for Client VPN connection logging.
`log_group_name`
The name of the CloudWatch Log Group used for Client VPN connection logging.
`security_group_arn`
The ARN of the security group associated with the Client VPN endpoint.
`security_group_id`
The ID of the security group associated with the Client VPN endpoint.
`security_group_name`
The name of the security group associated with the Client VPN endpoint.
`vpn_endpoint_arn`
The ARN of the Client VPN Endpoint Connection.
`vpn_endpoint_dns_name`
The DNS Name of the Client VPN Endpoint Connection.
`vpn_endpoint_id`
The ID of the Client VPN Endpoint Connection.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` - `awsutils`, version: `>= 0.16.0` ### Providers - `aws`, version: `>= 4.0` - `awsutils`, version: `>= 0.16.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_log` | 0.6.8 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.8) | n/a `self_signed_cert_ca` | 1.3.0 | [`cloudposse/ssm-tls-self-signed-cert/aws`](https://registry.terraform.io/modules/cloudposse/ssm-tls-self-signed-cert/aws/1.3.0) | n/a `self_signed_cert_root` | 1.3.0 | [`cloudposse/ssm-tls-self-signed-cert/aws`](https://registry.terraform.io/modules/cloudposse/ssm-tls-self-signed-cert/aws/1.3.0) | n/a `self_signed_cert_server` | 1.3.0 | [`cloudposse/ssm-tls-self-signed-cert/aws`](https://registry.terraform.io/modules/cloudposse/ssm-tls-self-signed-cert/aws/1.3.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpn_security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a ## Resources The following resources are used by this module: - [`aws_ec2_client_vpn_authorization_rule.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_client_vpn_authorization_rule) (resource) - [`aws_ec2_client_vpn_endpoint.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_client_vpn_endpoint) (resource) - [`aws_ec2_client_vpn_network_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_client_vpn_network_association) (resource) - [`aws_ec2_client_vpn_route.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_client_vpn_route) (resource) - [`aws_iam_saml_provider.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_saml_provider) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.ca_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`aws_ssm_parameter.root_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) - [`awsutils_ec2_client_vpn_export_client_config.default`](https://registry.terraform.io/providers/cloudposse/awsutils/latest/docs/data-sources/ec2_client_vpn_export_client_config) (data source) --- ## ec2-instance(Ec2-instance) # Module: `ec2-instance` Terraform Module for provisioning a general purpose EC2 host. Included features: * Automatically create a Security Group * Option to switch EIP attachment * CloudWatch monitoring and automatic reboot if instance hangs * Assume Role capability ## Usage Note: add `${var.ssh_key_pair}` private key to the `ssh agent`. Include this repository as a module in your existing terraform code. ### Simple example: ```hcl module "instance" { source = "cloudposse/ec2-instance/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ssh_key_pair = var.ssh_key_pair instance_type = var.instance_type vpc_id = var.vpc_id security_groups = var.security_groups subnet = var.subnet name = "ec2" namespace = "eg" stage = "dev" } ``` ### Example with additional volumes and EIP ```hcl module "kafka_instance" { source = "cloudposse/ec2-instance/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ssh_key_pair = var.ssh_key_pair vpc_id = var.vpc_id security_groups = var.security_groups subnet = var.subnet associate_public_ip_address = true name = "kafka" namespace = "eg" stage = "dev" additional_ips_count = 1 ebs_volume_count = 2 security_group_rules = [ { type = "egress" from_port = 0 to_port = 65535 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 53 to_port = 53 protocol = "udp" cidr_blocks = ["0.0.0.0/0"] }, ] } ``` ## Variables ### Required Variables
`subnet` (`string`) required
VPC Subnet ID the instance is launched in
`vpc_id` (`string`) required
The ID of the VPC that the instance security group belongs to
### Optional Variables
`additional_ips_count` (`number`) optional
Count of additional EIPs **Default value:** `0`
`ami` (`string`) optional
The AMI to use for the instance. By default it is the AMI provided by Amazon with Ubuntu 16.04 **Default value:** `""`
`ami_owner` (`string`) optional
Owner of the given AMI (ignored if `ami` unset, required if set) **Default value:** `""`
`applying_period` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `60`
`assign_eip_address` (`bool`) optional
Assign an Elastic IP address to the instance **Default value:** `true`
`associate_public_ip_address` (`bool`) optional
Associate a public IP address with the instance **Default value:** `false`
`availability_zone` (`string`) optional
Availability Zone the instance is launched in. If not set, will be launched in the first AZ of the region **Default value:** `""`
`burstable_mode` (`string`) optional
Enable burstable mode for the instance. Can be standard or unlimited. Applicable only for T2/T3/T4g instance types. **Default value:** `null`
`comparison_operator` (`string`) optional
The arithmetic operation to use when comparing the specified Statistic and Threshold. Possible values are: GreaterThanOrEqualToThreshold, GreaterThanThreshold, LessThanThreshold, LessThanOrEqualToThreshold. **Default value:** `"GreaterThanOrEqualToThreshold"`
`default_alarm_action` (`string`) optional
Default alarm action **Default value:** `"action/actions/AWS_EC2.InstanceId.Reboot/1.0"`
`delete_on_termination` (`bool`) optional
Whether the volume should be destroyed on instance termination **Default value:** `true`
`disable_alarm_action` (`bool`) optional
Disable the creation of Alarm Action **Default value:** `false`
`disable_api_stop` (`bool`) optional
Enable EC2 Instance Stop Protection **Default value:** `false`
`disable_api_termination` (`bool`) optional
Enable EC2 Instance Termination Protection **Default value:** `false`
`ebs_device_name` (`list(string)`) optional
Name of the EBS device to mount **Default value:** ```hcl [ "/dev/xvdb", "/dev/xvdc", "/dev/xvdd", "/dev/xvde", "/dev/xvdf", "/dev/xvdg", "/dev/xvdh", "/dev/xvdi", "/dev/xvdj", "/dev/xvdk", "/dev/xvdl", "/dev/xvdm", "/dev/xvdn", "/dev/xvdo", "/dev/xvdp", "/dev/xvdq", "/dev/xvdr", "/dev/xvds", "/dev/xvdt", "/dev/xvdu", "/dev/xvdv", "/dev/xvdw", "/dev/xvdx", "/dev/xvdy", "/dev/xvdz" ] ```
`ebs_iops` (`number`) optional
Amount of provisioned IOPS. This must be set with a volume_type of `io1`, `io2` or `gp3` **Default value:** `0`
`ebs_optimized` (`bool`) optional
Launched EC2 instance will be EBS-optimized **Default value:** `true`
`ebs_throughput` (`number`) optional
Amount of throughput. This must be set if volume_type is set to `gp3` **Default value:** `0`
`ebs_volume_count` (`number`) optional
Count of EBS volumes that will be attached to the instance **Default value:** `0`
`ebs_volume_encrypted` (`bool`) optional
Whether to encrypt the additional EBS volumes **Default value:** `true`
`ebs_volume_size` (`number`) optional
Size of the additional EBS volumes in gigabytes **Default value:** `10`
`ebs_volume_type` (`string`) optional
The type of the additional EBS volumes. Can be standard, gp2, gp3, io1 or io2 **Default value:** `"gp2"`
`evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold. **Default value:** `5`
`external_network_interface_enabled` (`bool`) optional
Wheter to attach an external ENI as the eth0 interface for the instance. Any change to the interface will force instance recreation. **Default value:** `false`
`external_network_interfaces` optional
The external interface definitions to attach to the instances. This depends on the instance type **Type:** ```hcl list(object({ delete_on_termination = bool device_index = number network_card_index = number network_interface_id = string })) ``` **Default value:** `null`
`force_detach_ebs` (`bool`) optional
force the volume/s to detach from the instance. **Default value:** `false`
`instance_initiated_shutdown_behavior` (`string`) optional
Specifies whether an instance stops or terminates when you initiate shutdown from the instance. Can be one of 'stop' or 'terminate'. **Default value:** `null`
`instance_profile` (`string`) optional
A pre-defined profile to attach to the instance (default is to build our own) **Default value:** `""`
`instance_profile_enabled` (`bool`) optional
Whether an IAM instance profile is created to pass a role to an Amazon EC2 instance when the instance starts **Default value:** `true`
`instance_type` (`string`) optional
The type of the instance **Default value:** `"t2.micro"`
`ipv6_address_count` (`number`) optional
Number of IPv6 addresses to associate with the primary network interface. Amazon EC2 chooses the IPv6 addresses from the range of your subnet (-1 to use subnet default) **Default value:** `0`
`ipv6_addresses` (`list(string)`) optional
List of IPv6 addresses from the range of the subnet to associate with the primary network interface **Default value:** `[ ]`
`kms_key_id` (`string`) optional
KMS key ID used to encrypt EBS volume. When specifying kms_key_id, ebs_volume_encrypted needs to be set to true **Default value:** `null`
`metadata_http_endpoint_enabled` (`bool`) optional
Whether the metadata service is available **Default value:** `true`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for instance metadata requests. **Default value:** `2`
`metadata_http_tokens_required` (`bool`) optional
Whether or not the metadata service requires session tokens, also referred to as Instance Metadata Service Version 2. **Default value:** `true`
`metadata_tags_enabled` (`bool`) optional
Whether the tags are enabled in the metadata service. **Default value:** `false`
`metric_name` (`string`) optional
The name for the alarm's associated metric. Allowed values can be found in https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ec2-metricscollected.html **Default value:** `"StatusCheckFailed_Instance"`
`metric_namespace` (`string`) optional
The namespace for the alarm's associated metric. Allowed values can be found in https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-namespaces.html **Default value:** `"AWS/EC2"`
`metric_threshold` (`number`) optional
The value against which the specified statistic is compared **Default value:** `1`
`metric_treat_missing_data` (`string`) optional
Sets how this alarm is to handle missing data points. The following values are supported: `missing`, `ignore`, `breaching` and `notBreaching`. Defaults to `missing`. **Default value:** `"missing"`
`monitoring` (`bool`) optional
Launched EC2 instance will have detailed monitoring enabled **Default value:** `true`
`permissions_boundary_arn` (`string`) optional
Policy ARN to attach to instance role as a permissions boundary **Default value:** `""`
`private_ip` (`string`) optional
Private IP address to associate with the instance in the VPC **Default value:** `null`
`region` (`string`) optional
AWS Region the instance is launched in **Default value:** `""`
`root_block_device_encrypted` (`bool`) optional
Whether to encrypt the root block device **Default value:** `true`
`root_block_device_kms_key_id` (`string`) optional
KMS key ID used to encrypt EBS volume. When specifying root_block_device_kms_key_id, root_block_device_encrypted needs to be set to true **Default value:** `null`
`root_iops` (`number`) optional
Amount of provisioned IOPS. This must be set if root_volume_type is set of `io1`, `io2` or `gp3` **Default value:** `0`
`root_throughput` (`number`) optional
Amount of throughput. This must be set if root_volume_type is set to `gp3` **Default value:** `0`
`root_volume_size` (`number`) optional
Size of the root volume in gigabytes **Default value:** `10`
`root_volume_type` (`string`) optional
Type of root volume. Can be standard, gp2, gp3, io1 or io2 **Default value:** `"gp2"`
`secondary_private_ips` (`list(string)`) optional
List of secondary private IP addresses to associate with the instance in the VPC **Default value:** `[ ]`
`security_group_description` (`string`) optional
The Security Group description. **Default value:** `"EC2 Security Group"`
`security_group_enabled` (`bool`) optional
Whether to create default Security Group for EC2. **Default value:** `true`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully complated with `aws_security_group_rule` resource. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** ```hcl [ { "cidr_blocks": [ "0.0.0.0/0" ], "description": "Allow all outbound traffic", "from_port": 0, "protocol": "-1", "to_port": 65535, "type": "egress" } ] ```
`security_group_use_name_prefix` (`bool`) optional
Whether to create a default Security Group with unique name beginning with the normalized prefix. **Default value:** `false`
`security_groups` (`list(string)`) optional
A list of Security Group IDs to associate with EC2 instance. **Default value:** `[ ]`
`source_dest_check` (`bool`) optional
Controls if traffic is routed to the instance when the destination address does not match the instance. Used for NAT or VPNs **Default value:** `true`
`ssh_key_pair` (`string`) optional
SSH key pair to be provisioned on the instance **Default value:** `null`
`ssm_patch_manager_enabled` (`bool`) optional
Whether to enable SSM Patch manager **Default value:** `false`
`ssm_patch_manager_iam_policy_arn` (`string`) optional
IAM policy ARN to allow Patch Manager to manage the instance. If not provided, `arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore` will be used **Default value:** `null`
`ssm_patch_manager_s3_log_bucket` (`string`) optional
The name of the s3 bucket to export the patch log to **Default value:** `null`
`statistic_level` (`string`) optional
The statistic to apply to the alarm's associated metric. Allowed values are: SampleCount, Average, Sum, Minimum, Maximum **Default value:** `"Maximum"`
`stop_ec2_before_detaching_vol` (`bool`) optional
Set this to true to ensure that the target instance is stopped before trying to detach the volume/s. **Default value:** `false`
`tenancy` (`string`) optional
Tenancy of the instance (if the instance is running in a VPC). An instance with a tenancy of 'dedicated' runs on single-tenant hardware. The 'host' tenancy is not supported for the import-instance command. Valid values are 'default', 'dedicated', and 'host'. **Default value:** `"default"`
`user_data` (`string`) optional
The user data to provide when launching the instance. Do not pass gzip-compressed data via this argument; use `user_data_base64` instead **Default value:** `null`
`user_data_base64` (`string`) optional
Can be used instead of `user_data` to pass base64-encoded binary data directly. Use this instead of `user_data` whenever the value is not a valid UTF-8 string. For example, gzip-encoded user data must be base64-encoded and passed via this argument to avoid corruption **Default value:** `null`
`user_data_replace_on_change` (`bool`) optional
When used in combination with user_data or user_data_base64 will trigger a destroy and recreate when set to true. **Default value:** `false`
`volume_tags_enabled` (`bool`) optional
Whether or not to copy instance tags to root and EBS volumes **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`additional_eni_ids`
Map of ENI to EIP
`alarm`
CloudWatch Alarm ID
`arn`
ARN of the instance
`ebs_ids`
IDs of EBSs
`id`
Disambiguated ID of the instance
`instance_profile`
Name of the instance's profile (either built or supplied)
`name`
Instance name
`primary_network_interface_id`
ID of the instance's primary network interface
`private_dns`
Private DNS of instance
`private_ip`
Private IP of instance
`public_dns`
Public DNS of instance (or DNS of EIP)
`public_ip`
Public IP of instance (or EIP)
`role`
Name of AWS IAM Role associated with the instance
`role_arn`
ARN of AWS IAM Role associated with the instance
`security_group_arn`
EC2 instance Security Group ARN
`security_group_id`
EC2 instance Security Group ID
`security_group_ids`
IDs on the AWS Security Groups associated with the instance
`security_group_name`
EC2 instance Security Group name
`ssh_key_pair`
Name of the SSH key pair provisioned on the instance
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.7.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 4.7.0` - `null`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label_ssm_patch_s3_log_policy` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_group` | 0.3.3 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/0.3.3) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_ebs_volume.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ebs_volume) (resource) - [`aws_eip.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_eip.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_policy.ssm_patch_s3_log_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.ssm_core`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ssm_s3_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) (resource) - [`aws_network_interface.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_interface) (resource) - [`aws_network_interface_attachment.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_interface_attachment) (resource) - [`aws_volume_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/volume_attachment) (resource) - [`null_resource.check_alarm_action`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_ami.info`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_instance_profile.given`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_instance_profile) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ssm_patch_s3_log_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_subnet.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) --- ## ec2-instance-group # Module: `ec2-instance-group` Terraform Module for providing N general purpose EC2 hosts. If you only need to provision a single EC2 instance, consider using the [terraform-aws-ec2-instance](https://github.com/cloudposse/terraform-aws-ec2-instance) module instead. **IMPORTANT** This module by-design does not provision an AutoScaling group. It was designed to provision a discrete number of instances suitable for running stateful services such as databases (e.g. Kafka, Redis, etc). Included features: * Automatically create a Security Group * Option to switch EIP attachment * CloudWatch monitoring and automatic reboot if instance hangs * Assume Role capability ## Usage Note: add `${var.ssh_key_pair}` private key to the `ssh agent`. Include this repository as a module in your existing terraform code. ### Simple example: ```hcl module "instance" { source = "cloudposse/ec2-instance-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" ami = "ami-a4dc46db" ami_owner = "099720109477" ssh_key_pair = var.ssh_key_pair instance_type = var.instance_type vpc_id = var.vpc_id security_groups = var.security_groups subnet = var.subnet instance_count = 3 } ``` ### Example with additional volumes and EIP ```hcl module "kafka_instance" { source = "cloudposse/ec2-instance-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" ami = "ami-a4dc46db" ami_owner = "099720109477" ssh_key_pair = var.ssh_key_pair vpc_id = var.vpc_id security_groups = var.security_groups subnet = var.subnet associate_public_ip_address = true additional_ips_count = 1 ebs_volume_count = 2 instance_count = 3 security_group_rules = [ { type = "egress" from_port = 0 to_port = 65535 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] }, { type = "ingress" from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } ] } ``` ### Additional complete working example with variations of how to use the module In /examples directory This module depends on these modules: * [terraform-terraform-label](https://github.com/cloudposse/terraform-terraform-label) It is necessary to run `terraform get` or `terraform init` to download this module. Now reference the label when creating an instance (for example): ```hcl resource "aws_ami_from_instance" "example" { count = length(module.instance.*.id) name = "app" source_instance_id = element(module.instance.*.id, count.index) } ``` ## Variables ### Required Variables
`ami` (`string`) required
The AMI to use for the instance
`ami_owner` (`string`) required
Owner of the given AMI
`region` (`string`) required
AWS Region the instance is launched in
`subnet` (`string`) required
VPC Subnet ID the instance is launched in
`vpc_id` (`string`) required
The ID of the VPC that the instance security group belongs to
### Optional Variables
`additional_ips_count` (`number`) optional
Count of additional EIPs **Default value:** `0`
`applying_period` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `60`
`assign_eip_address` (`bool`) optional
Assign an Elastic IP address to the instance **Default value:** `true`
`associate_public_ip_address` (`bool`) optional
Associate a public IP address with the instance **Default value:** `false`
`availability_zone` (`string`) optional
Availability Zone the instance is launched in. If not set, will be launched in the first AZ of the region **Default value:** `""`
`comparison_operator` (`string`) optional
The arithmetic operation to use when comparing the specified Statistic and Threshold. Possible values are: GreaterThanOrEqualToThreshold, GreaterThanThreshold, LessThanThreshold, LessThanOrEqualToThreshold **Default value:** `"GreaterThanOrEqualToThreshold"`
`default_alarm_action` (`string`) optional
Default alarm action **Default value:** `"action/actions/AWS_EC2.InstanceId.Reboot/1.0"`
`delete_on_termination` (`bool`) optional
Whether the volume should be destroyed on instance termination **Default value:** `true`
`disable_api_termination` (`bool`) optional
Enable EC2 Instance Termination Protection **Default value:** `false`
`ebs_device_names` (`list(string)`) optional
Name of the EBS device to mount **Default value:** ```hcl [ "/dev/xvdb", "/dev/xvdc", "/dev/xvdd", "/dev/xvde", "/dev/xvdf", "/dev/xvdg", "/dev/xvdh", "/dev/xvdi", "/dev/xvdj", "/dev/xvdk", "/dev/xvdl", "/dev/xvdm", "/dev/xvdn", "/dev/xvdo", "/dev/xvdp", "/dev/xvdq", "/dev/xvdr", "/dev/xvds", "/dev/xvdt", "/dev/xvdu", "/dev/xvdv", "/dev/xvdw", "/dev/xvdx", "/dev/xvdy", "/dev/xvdz" ] ```
`ebs_iops` (`number`) optional
Amount of provisioned IOPS. This must be set with a volume_type of io1 **Default value:** `0`
`ebs_optimized` (`bool`) optional
Launched EC2 instance will be EBS-optimized **Default value:** `false`
`ebs_volume_count` (`number`) optional
Count of EBS volumes that will be attached to the instance **Default value:** `0`
`ebs_volume_encrypted` (`bool`) optional
Size of the EBS volume in gigabytes **Default value:** `true`
`ebs_volume_size` (`number`) optional
Size of the EBS volume in gigabytes **Default value:** `10`
`ebs_volume_type` (`string`) optional
The type of EBS volume. Can be standard, gp2 or io1 **Default value:** `"gp2"`
`evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `5`
`generate_ssh_key_pair` (`bool`) optional
If true, create a new key pair and save the pem for it to the current working directory **Default value:** `false`
`instance_count` (`number`) optional
Count of ec2 instances to create **Default value:** `1`
`instance_type` (`string`) optional
The type of the instance **Default value:** `"t2.micro"`
`ipv6_address_count` (`number`) optional
Number of IPv6 addresses to associate with the primary network interface. Amazon EC2 chooses the IPv6 addresses from the range of your subnet **Default value:** `0`
`ipv6_addresses` (`list(string)`) optional
List of IPv6 addresses from the range of the subnet to associate with the primary network interface **Default value:** `[ ]`
`kms_key_id` (`string`) optional
KMS key ID used to encrypt EBS volume. When specifying kms_key_id, ebs_volume_encrypted needs to be set to true **Default value:** `null`
`metadata_http_endpoint_enabled` (`bool`) optional
Whether the metadata service is available **Default value:** `true`
`metadata_http_tokens_required` (`bool`) optional
Whether or not the metadata service requires session tokens, also referred to as Instance Metadata Service Version 2. **Default value:** `true`
`metric_name` (`string`) optional
The name for the alarm's associated metric. Allowed values can be found in https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ec2-metricscollected.html **Default value:** `"StatusCheckFailed_Instance"`
`metric_namespace` (`string`) optional
The namespace for the alarm's associated metric. Allowed values can be found in https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-namespaces.html **Default value:** `"AWS/EC2"`
`metric_threshold` (`number`) optional
The value against which the specified statistic is compared **Default value:** `1`
`monitoring` (`bool`) optional
Launched EC2 instance will have detailed monitoring enabled **Default value:** `true`
`permissions_boundary_arn` (`string`) optional
Policy ARN to attach to instance role as a permissions boundary **Default value:** `""`
`private_ips` (`list(string)`) optional
Private IP address to associate with the instances in the VPC **Default value:** `[ ]`
`root_block_device_encrypted` (`bool`) optional
Whether to encrypt the root block device **Default value:** `true`
`root_iops` (`number`) optional
Amount of provisioned IOPS. This must be set if root_volume_type is set to `io1` **Default value:** `0`
`root_volume_size` (`number`) optional
Size of the root volume in gigabytes **Default value:** `10`
`root_volume_type` (`string`) optional
Type of root volume. Can be standard, gp2 or io1 **Default value:** `"gp2"`
`security_group_description` (`string`) optional
The Security Group description. **Default value:** `"EC2 instances Security Group"`
`security_group_enabled` (`bool`) optional
Whether to create default Security Group for EC2 instances. **Default value:** `true`
`security_group_name` (`list(string)`) optional
The name to assign to the security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`security_group_rules` (`list(any)`) optional
A list of maps of Security Group rules. The values of map is fully complated with `aws_security_group_rule` resource. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`security_groups` (`list(string)`) optional
A list of Security Group IDs to associate with EC2 instances. **Default value:** `[ ]`
`source_dest_check` (`bool`) optional
Controls if traffic is routed to the instance when the destination address does not match the instance. Used for NAT or VPNs **Default value:** `true`
`ssh_key_pair` (`string`) optional
SSH key pair to be provisioned on the instance **Default value:** `""`
`ssh_key_pair_path` (`string`) optional
Path to where the generated key pairs will be created. Defaults to $$\{path.cwd\} **Default value:** `""`
`statistic_level` (`string`) optional
The statistic to apply to the alarm's associated metric. Allowed values are: SampleCount, Average, Sum, Minimum, Maximum **Default value:** `"Maximum"`
`user_data` (`string`) optional
Instance user data. Do not pass gzip-compressed data via this argument **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`alarm_ids`
CloudWatch Alarm IDs
`aws_key_pair_name`
Name of AWS key pair
`ebs_ids`
IDs of EBSs
`eip_per_instance_count`
Number of EIPs per instance.
`eni_to_eip_map`
Map of ENI with EIP
`ids`
Disambiguated IDs list
`instance_count`
Total number of instances created
`name`
Instance(s) name
`new_ssh_keypair_generated`
Was a new ssh_key_pair generated
`primary_network_interface_ids`
IDs of the instance's primary network interface
`private_dns`
Private DNS records of instances
`private_ips`
Private IPs of instances
`public_dns`
All public DNS records for the public interfaces and ENIs
`public_ips`
List of Public IPs of instances (or EIP)
`role_names`
Names of AWS IAM Roles associated with creating instance
`security_group_arn`
EC2 instances Security Group ARN
`security_group_id`
EC2 instances Security Group ID
`security_group_ids`
ID on the new AWS Security Group associated with creating instance
`security_group_name`
EC2 instances Security Group name
`ssh_key_pem_path`
Path where SSH key pair was created (if applicable)
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `ssh_key_pair` | 0.19.0 | [`cloudposse/key-pair/aws`](https://registry.terraform.io/modules/cloudposse/key-pair/aws/0.19.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_ebs_volume.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ebs_volume) (resource) - [`aws_eip.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_eip.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/instance) (resource) - [`aws_network_interface.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_interface) (resource) - [`aws_network_interface_attachment.additional`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_interface_attachment) (resource) - [`aws_volume_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/volume_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.info`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## ecr(Ecr) # Module: `ecr` Terraform module to provision an [`AWS ECR Docker Container registry`](https://aws.amazon.com/ecr/). ## Usage The module creates one or more Elastic Container Registry (ECR) repositories. All repositories created will share the same configuration. Use this module multiple times to create repositories with different configurations. If you provide 1 or more names in `image_names` then one repository will be created for each of the names you provide. Those names can include "namespaces", which are just prefixes ending with a slash (`/`). If you do not provide any names in `image_names`, the module will create a single ECR repo named `namespace-stage-name` or just `name` depending on the value of `use_fullname`. Access to the repositories is granted to via the `principals_full_access` and `principals_readonly_access` lists, which are lists of strings that can designate [any valid AWS Principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#Principal_specifying). This module only creates the Repository Policy allowing those Principals access. The Principals will still separately need IAM policies allowing them permission to execute ECR actions against the repository. For more details, see [How Amazon Elastic Container Registry Works with IAM](https://docs.aws.amazon.com/AmazonECR/latest/userguide/security_iam_service-with-iam.html). Include this repository as a module in your existing terraform code: ```hcl # IAM Role to be granted ECR permissions data "aws_iam_role" "ecr" { name = "ecr" } module "ecr" { source = "cloudposse/ecr/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "ecr" principals_full_access = [data.aws_iam_role.ecr.arn] } ``` ### Advanced Usage with Image Tag Mutability Exclusion Filters The module supports `image_tag_mutability_exclusion_filter` to exclude specific tags from the repository's image tag mutability setting. This is useful when you want most tags to be immutable but allow certain tag patterns (like `latest`, `dev-*`) to be mutable. **Note**: This feature requires AWS provider version >= 6.8.0. ```hcl module "ecr_with_exclusions" { source = "cloudposse/ecr/aws" # version = "x.x.x" namespace = "eg" stage = "prod" name = "api" image_tag_mutability = "IMMUTABLE_WITH_EXCLUSION" principals_full_access = [data.aws_iam_role.ecr.arn] # Allow 'latest' and 'dev-*' tags to be mutable while keeping others immutable image_tag_mutability_exclusion_filter = [ { filter = "latest" filter_type = "WILDCARD" }, { filter = "dev-" filter_type = "WILDCARD" }, { filter = "feature-" filter_type = "WILDCARD" } ] } ``` ## Variables ### Required Variables
### Optional Variables
`custom_lifecycle_rules` optional
Custom lifecycle rules to override or complement the default ones **Type:** ```hcl list(object({ description = optional(string) selection = object({ tagStatus = string countType = string countNumber = number countUnit = optional(string) tagPrefixList = optional(list(string)) tagPatternList = optional(list(string)) }) action = object({ type = string }) })) ``` **Default value:** `[ ]`
`default_lifecycle_rules_settings` optional
Default lifecycle rules settings **Type:** ```hcl object({ untagged_image_rule = optional(object({ enabled = optional(bool, true) }), { enabled = true }) remove_old_image_rule = optional(object({ enabled = optional(bool, true) }), { enabled = true }) }) ``` **Default value:** ```hcl { "remove_old_image_rule": { "enabled": true }, "untagged_image_rule": { "enabled": true } } ```
`enable_lifecycle_policy` (`bool`) optional
Set to false to prevent the module from adding any lifecycle policies to any repositories **Default value:** `true`
`encryption_configuration` optional
ECR encryption configuration **Type:** ```hcl object({ encryption_type = string kms_key = any }) ``` **Default value:** `null`
`force_delete` (`bool`) optional
Whether to delete the repository even if it contains images **Default value:** `false`
`image_names` (`list(string)`) optional
List of Docker local image names, used as repository names for AWS ECR **Default value:** `[ ]`
`image_tag_mutability` (`string`) optional
The tag mutability setting for the repository. Must be one of: `MUTABLE`, `IMMUTABLE`, `IMMUTABLE_WITH_EXCLUSION`, or `MUTABLE_WITH_EXCLUSION`. Defaults to `IMMUTABLE` **Default value:** `"IMMUTABLE"`
`image_tag_mutability_exclusion_filter` optional
List of exclusion filters for image tag mutability. Each filter object must contain 'filter' and 'filter_type' attributes. Requires AWS provider >= 6.8.0 **Type:** ```hcl list(object({ filter = string filter_type = optional(string, "WILDCARD") })) ``` **Default value:** `[ ]`
`max_image_count` (`number`) optional
How many Docker Image versions AWS ECR will store **Default value:** `500`
`organizations_full_access` (`list(string)`) optional
Organization IDs to provide with full access to the ECR. **Default value:** `[ ]`
`organizations_push_access` (`list(string)`) optional
Organization IDs to provide with push access to the ECR **Default value:** `[ ]`
`organizations_readonly_access` (`list(string)`) optional
Organization IDs to provide with readonly access to the ECR. **Default value:** `[ ]`
`prefixes_pull_through_repositories` (`list(string)`) optional
Organization IDs to provide with push access to the ECR **Default value:** `[ ]`
`principals_full_access` (`list(string)`) optional
Principal ARNs to provide with full access to the ECR **Default value:** `[ ]`
`principals_lambda` (`list(string)`) optional
Principal account IDs of Lambdas allowed to consume ECR **Default value:** `[ ]`
`principals_pull_though_access` (`list(string)`) optional
Principal ARNs to provide with pull though access to the ECR **Default value:** `[ ]`
`principals_push_access` (`list(string)`) optional
Principal ARNs to provide with push access to the ECR **Default value:** `[ ]`
`principals_readonly_access` (`list(string)`) optional
Principal ARNs to provide with readonly access to the ECR **Default value:** `[ ]`
`protected_tags` (`set(string)`) optional
List of image tags prefixes and wildcards that should not be destroyed. Useful if you tag images with prefixes like `dev`, `staging`, `prod` or wildcards like `*dev`, `*prod`,`*.*.*` **Default value:** `[ ]`
`protected_tags_keep_count` (`number`) optional
Number of Image versions to keep for protected tags **Default value:** `999999`
`replication_configurations` optional
Replication configuration for a registry. See [Replication Configuration](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_replication_configuration#replication-configuration). **Type:** ```hcl list(object({ rules = list(object({ # Maximum 10 destinations = list(object({ # Maximum 25 region = string registry_id = string })) repository_filters = list(object({ filter = string filter_type = string })) })) })) ``` **Default value:** `[ ]`
`scan_images_on_push` (`bool`) optional
Indicates whether images are scanned after being pushed to the repository (true) or not (false) **Default value:** `true`
`time_based_rotation` (`bool`) optional
Set to true to filter image based on the `sinceImagePushed` count type. **Default value:** `false`
`use_fullname` (`bool`) optional
Set 'true' to use `namespace-stage-name` for ecr repository name, else `name` **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`registry_id`
Registry ID
`repository_arn`
ARN of first repository created
`repository_arn_map`
Map of repository names to repository ARNs
`repository_name`
Name of first repository created
`repository_url`
URL of first repository created
`repository_url_map`
Map of repository names to repository URLs
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 6.8.0` ### Providers - `aws`, version: `>= 6.8.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ecr_lifecycle_policy.name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_lifecycle_policy) (resource) - [`aws_ecr_replication_configuration.replication_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_replication_configuration) (resource) - [`aws_ecr_repository.name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_repository) (resource) - [`aws_ecr_repository_policy.name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecr_repository_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.empty`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.lambda_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.organization_full_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.organization_push_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.organizations_readonly_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_full_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_pull_through_cache`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_push_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_readonly_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## ecr-public # Module: `ecr-public` Terraform module to provision a Public [`AWS ECR Docker Container registry`](https://docs.aws.amazon.com/AmazonECR/latest/public/public-repositories.html/). ## Usage The module creates one or more Elastic Container Registry (ECR) Public repositories. Please note that this module can only be provisioned in the US-EAST-1 region. Write access to the repositories is granted to via the `principals_full_access` list, which is a list of strings that can designate [any valid AWS Principal](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html#Principal_specifying). This module only creates the Repository Policy allowing those Principals access. The Principals will still separately need IAM policies allowing them permission to execute ECR actions against the repository. For more details, see [How Amazon Elastic Container Registry Works with IAM](https://docs.aws.amazon.com/AmazonECR/latest/userguide/security_iam_service-with-iam.html). Include this repository as a module in your existing terraform code: ```hcl # IAM Role to be granted ECR permissions data "aws_iam_role" "ecrpublic" { name = "ecr" } module "ecrpublic" { source = "cloudposse/ecr-public/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "ecr" principals_full_access = [data.aws_iam_role.ecr.arn] repository_configs = [ { name = "foo" description = "The repository for the foo image" about_text = null usage_text = null architectures = ["AMD64"] operating_systems = ["Linux"] }, ] } ``` ## Variables ### Required Variables
`repository_configs` required
`name`: List of Docker local image names, used as repository names for AWS ECR `description`: A short description of the contents of the repository. This text appears in both the image details and also when searching for repositories on the Amazon ECR Public Gallery. `about_text`: A detailed description of the contents of the repository. It is publicly visible in the Amazon ECR Public Gallery. The text must be in markdown format. `usage_text`: Detailed information on how to use the contents of the repository. It is publicly visible in the Amazon ECR Public Gallery. The usage text provides context, support information, and additional usage details for users of the repository. The text must be in markdown format. `architectures`: The system architecture that the images in the repository are compatible with. On the Amazon ECR Public Gallery, the following supported architectures will appear as badges on the repository and are used as search filters: ARM, ARM 64, x86, x86-64 `operating_systems`: The operating systems that the images in the repository are compatible with. On the Amazon ECR Public Gallery, the following supported operating systems will appear as badges on the repository and are used as search filters: Linux, Windows `logo_image_blob`: The base64-encoded repository logo payload. (Only visible for verified accounts) Note that drift detection is disabled for this attribute. **Type:** ```hcl list(object({ name = string description = string about_text = string usage_text = string architectures = list(string) operating_systems = list(string) logo_image_blob = string })) ```
### Optional Variables
`principals_full_access` (`list(string)`) optional
Principal ARNs to provide with full access to the ECR **Default value:** `[ ]`
`principals_readonly_access` (`list(string)`) optional
Principal ARNs to provide with readonly access to the ECR **Default value:** `[ ]`
`use_fullname` (`bool`) optional
Set 'true' to use `namespace-stage-name` for ecr repository name, else `name` **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`registry_id`
Registry ID
`repository_arn`
ARN of first repository created
`repository_arn_map`
Map of repository names to repository ARNs
`repository_name`
Name of first repository created
`repository_url`
URL of first repository created
`repository_url_map`
Map of repository names to repository URLs
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.1` ### Providers - `aws`, version: `>= 3.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ecrpublic_repository.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecrpublic_repository) (resource) - [`aws_ecrpublic_repository_policy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecrpublic_repository_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.empty`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.resource_full_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## ecs-alb-service-task # Module: `ecs-alb-service-task` Terraform module to create an ECS Service for a web app (task), and an ALB target group to route requests. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/tree/main/examples/complete). For automated test of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-ecs-alb-service-task/tree/main/test). ```hcl provider "aws" { region = var.region } module "label" { source = "git::https://github.com/cloudposse/terraform-null-label.git?ref=tags/0.15.0" namespace = var.namespace name = var.name stage = var.stage delimiter = var.delimiter attributes = var.attributes tags = var.tags } module "vpc" { source = "git::https://github.com/cloudposse/terraform-aws-vpc.git?ref=tags/0.8.1" namespace = var.namespace stage = var.stage name = var.name delimiter = var.delimiter attributes = var.attributes cidr_block = var.vpc_cidr_block tags = var.tags } module "subnets" { source = "git::https://github.com/cloudposse/terraform-aws-dynamic-subnets.git?ref=tags/0.16.1" availability_zones = var.availability_zones namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false tags = var.tags } resource "aws_ecs_cluster" "default" { name = module.label.id tags = module.label.tags } module "container_definition" { source = "git::https://github.com/cloudposse/terraform-aws-ecs-container-definition.git?ref=tags/0.21.0" container_name = var.container_name container_image = var.container_image container_memory = var.container_memory container_memory_reservation = var.container_memory_reservation container_cpu = var.container_cpu essential = var.container_essential readonly_root_filesystem = var.container_readonly_root_filesystem environment = var.container_environment port_mappings = var.container_port_mappings log_configuration = var.container_log_configuration } module "ecs_alb_service_task" { source = "cloudposse/ecs-alb-service-task/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = var.attributes delimiter = var.delimiter alb_security_group = module.vpc.vpc_default_security_group_id container_definition_json = module.container_definition.json ecs_cluster_arn = aws_ecs_cluster.default.arn launch_type = var.ecs_launch_type vpc_id = module.vpc.vpc_id security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.public_subnet_ids tags = var.tags ignore_changes_task_definition = var.ignore_changes_task_definition network_mode = var.network_mode assign_public_ip = var.assign_public_ip propagate_tags = var.propagate_tags health_check_grace_period_seconds = var.health_check_grace_period_seconds deployment_minimum_healthy_percent = var.deployment_minimum_healthy_percent deployment_maximum_percent = var.deployment_maximum_percent deployment_controller_type = var.deployment_controller_type desired_count = var.desired_count task_memory = var.task_memory task_cpu = var.task_cpu } ``` The `container_image` in the `container_definition` module is the Docker image used to start a container. The `container_definition` is a string of JSON-encoded container definitions. Normally, you would place only one container definition here as the example above demonstrates. However, there might be situations where more than one container per task is more appropriate such as optionally in [Fargate](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/application_architecture.html#application_architecture_fargate) or in other cases where sidecars may be required. With [cloudposse/terraform-aws-ecs-container-definition](https://github.com/cloudposse/terraform-aws-ecs-container-definition) multi-container task definitions can be created using: ```hcl module "ecs_alb_service_task" { ... container_definition_json = jsonencode([ module.first_container.json_map_object, module.second_container.json_map_object, ]) ... } ``` Refer to the [multiple definitions](https://github.com/cloudposse/terraform-aws-ecs-container-definition/blob/master/examples/multiple_definitions/main.tf) example in cloudposse/terraform-aws-ecs-container-definition for details on defining multiple definitions. This string is passed directly to the Docker daemon. Images in the Docker Hub registry are available by default. Other repositories are specified with either `repository-url/image:tag` or `repository-url/image@digest`. Up to 255 letters (uppercase and lowercase), numbers, hyphens, underscores, colons, periods, forward slashes, and number signs are allowed. This parameter maps to Image in the Create a container section of the Docker Remote API and the IMAGE parameter of `docker run`. When a new task starts, the Amazon ECS container agent pulls the latest version of the specified image and tag for the container to use. However, subsequent updates to a repository image are not propagated to already running tasks. Images in Amazon ECR repositories can be specified by either using the full `registry/repository:tag` or `registry/repository@digest`. For example, `012345678910.dkr.ecr..amazonaws.com/:latest` or `012345678910.dkr.ecr..amazonaws.com/@sha256:94afd1f2e64d908bc90dbca0035a5b567EXAMPLE`. Images in official repositories on Docker Hub use a single name (for example, `ubuntu` or `mongo`). Images in other repositories on Docker Hub are qualified with an organization name (for example, `amazon/amazon-ecs-agent`). Images in other online repositories are qualified further by a domain name (for example, `quay.io/assemblyline/ubuntu`). For more info, see [Container Definition](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerDefinition.html). ## Variables ### Required Variables
`container_definition_json` (`string`) required
A string containing a JSON-encoded array of container definitions (`"[{ "name": "container1", ... }, { "name": "container2", ... }]"`). See [API_ContainerDefinition](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerDefinition.html), [cloudposse/terraform-aws-ecs-container-definition](https://github.com/cloudposse/terraform-aws-ecs-container-definition), or [ecs_task_definition#container_definitions](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition#container_definitions)
`ecs_cluster_arn` (`string`) required
The ARN of the ECS cluster where service will be provisioned
`vpc_id` (`string`) required
The VPC ID where resources are created
### Optional Variables
`alb_security_group` (`string`) optional
Security group of the ALB **Default value:** `""`
`assign_public_ip` (`bool`) optional
Assign a public IP address to the ENI (Fargate launch type only). Valid values are `true` or `false`. Default `false` **Default value:** `false`
`availability_zone_rebalancing` (`string`) optional
ECS automatically redistributes tasks within a service across Availability Zones (AZs) to mitigate the risk of impaired application availability due to underlying infrastructure failures and task lifecycle activities. The valid values are `ENABLED` and `DISABLED`. **Default value:** `"DISABLED"`
`bind_mount_volumes` (`list(any)`) optional
Task bind mount volume definitions as list of configuration objects. You can define multiple bind mount volumes on the same task definition. Requires `name` and optionally `host_path` **Default value:** `[ ]`
`capacity_provider_strategies` optional
The capacity provider strategies to use for the service. See `capacity_provider_strategy` configuration block: https://www.terraform.io/docs/providers/aws/r/ecs_service.html#capacity_provider_strategy **Type:** ```hcl list(object({ capacity_provider = string weight = number base = number })) ``` **Default value:** `[ ]`
`circuit_breaker_deployment_enabled` (`bool`) optional
If `true`, enable the deployment circuit breaker logic for the service. If using `CODE_DEPLOY` for `deployment_controller_type`, this value will be ignored **Default value:** `false`
`circuit_breaker_rollback_enabled` (`bool`) optional
If `true`, Amazon ECS will roll back the service if a service deployment fails. If using `CODE_DEPLOY` for `deployment_controller_type`, this value will be ignored **Default value:** `false`
`container_port` (`number`) optional
The port on the container to allow traffic from the ALB security group **Default value:** `80`
`deployment_controller_type` (`string`) optional
Type of deployment controller. Valid values are `CODE_DEPLOY` and `ECS` **Default value:** `"ECS"`
`deployment_maximum_percent` (`number`) optional
The upper limit of the number of tasks (as a percentage of `desired_count`) that can be running in a service during a deployment **Default value:** `200`
`deployment_minimum_healthy_percent` (`number`) optional
The lower limit (as a percentage of `desired_count`) of the number of tasks that must remain running and healthy in a service during a deployment **Default value:** `100`
`desired_count` (`number`) optional
The number of instances of the task definition to place and keep running **Default value:** `1`
`docker_volumes` optional
Task docker volume definitions as list of configuration objects. You can define multiple Docker volumes on the same task definition, but a single volume can only have one `docker_volume_configuration`. **Type:** ```hcl list(object({ host_path = string name = string docker_volume_configuration = list(object({ autoprovision = bool driver = string driver_opts = map(string) labels = map(string) scope = string })) })) ``` **Default value:** `[ ]`
`ecs_load_balancers` optional
A list of load balancer config objects for the ECS service; see [ecs_service#load_balancer](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#load_balancer) docs **Type:** ```hcl list(object({ container_name = string container_port = number elb_name = optional(string) target_group_arn = string })) ``` **Default value:** `[ ]`
`ecs_service_enabled` (`bool`) optional
Whether or not to create the aws_ecs_service resource **Default value:** `true`
`efs_volumes` optional
Task EFS volume definitions as list of configuration objects. You can define multiple EFS volumes on the same task definition, but a single volume can only have one `efs_volume_configuration`. **Type:** ```hcl list(object({ host_path = string name = string efs_volume_configuration = list(object({ file_system_id = string root_directory = string transit_encryption = string transit_encryption_port = string authorization_config = list(object({ access_point_id = string iam = string })) })) })) ``` **Default value:** `[ ]`
`enable_all_egress_rule` (`bool`) optional
A flag to enable/disable adding the all ports egress rule to the service security group **Default value:** `true`
`enable_ecs_managed_tags` (`bool`) optional
Specifies whether to enable Amazon ECS managed tags for the tasks within the service **Default value:** `false`
`enable_fault_injection` (`bool`) optional
Enables fault injection and allows for fault injection requests to be accepted from the task's containers **Default value:** `false`
`enable_icmp_rule` (`bool`) optional
Specifies whether to enable ICMP on the service security group **Default value:** `false`
`ephemeral_storage_size` (`number`) optional
The number of GBs to provision for ephemeral storage on Fargate tasks. Must be greater than or equal to 21 and less than or equal to 200 **Default value:** `0`
`exec_enabled` (`bool`) optional
Specifies whether to enable Amazon ECS Exec for the tasks within the service **Default value:** `false`
`force_new_deployment` (`bool`) optional
Enable to force a new task deployment of the service. **Default value:** `false`
`fsx_volumes` optional
Task FSx volume definitions as list of configuration objects. You can define multiple FSx volumes on the same task definition, but a single volume can only have one `fsx_windows_file_server_volume_configuration`. **Type:** ```hcl list(object({ host_path = string name = string fsx_windows_file_server_volume_configuration = list(object({ file_system_id = string root_directory = string authorization_config = list(object({ credentials_parameter = string domain = string })) })) })) ``` **Default value:** `[ ]`
`health_check_grace_period_seconds` (`number`) optional
Seconds to ignore failing load balancer health checks on newly instantiated tasks to prevent premature shutdown, up to 7200. Only valid for services configured to use load balancers **Default value:** `0`
`ignore_changes_desired_count` (`bool`) optional
Whether to ignore changes for desired count in the ECS service **Default value:** `false`
`ignore_changes_task_definition` (`bool`) optional
Whether to ignore changes in container definition and task definition in the ECS service **Default value:** `true`
`ipc_mode` (`string`) optional
The IPC resource namespace to be used for the containers in the task. The valid values are `host`, `task`, and `none`. If `host` is specified, then all containers within the tasks that specified the `host` IPC mode on the same container instance share the same IPC resources with the host Amazon EC2 instance. If `task` is specified, all containers within the specified task share the same IPC resources. If `none` is specified, then IPC resources within the containers of a task are private and not shared with other containers in a task or on the container instance. If no value is specified, then the IPC resource namespace sharing depends on the Docker daemon setting on the container instance. For more information, see IPC settings in the Docker documentation." **Default value:** `null`
`launch_type` (`string`) optional
The launch type on which to run your service. Valid values are `EC2` and `FARGATE` **Default value:** `"FARGATE"`
`network_mode` (`string`) optional
The network mode to use for the task. This is required to be `awsvpc` for `FARGATE` `launch_type` or `null` for `EC2` `launch_type` **Default value:** `"awsvpc"`
`nlb_cidr_blocks` (`list(string)`) optional
A list of CIDR blocks to add to the ingress rule for the NLB container port **Default value:** `[ ]`
`nlb_container_port` (`number`) optional
The port on the container to allow traffic from the NLB **Default value:** `80`
`ordered_placement_strategy` optional
Service level strategy rules that are taken into consideration during task placement. List from top to bottom in order of precedence. The maximum number of ordered_placement_strategy blocks is 5. See [`ordered_placement_strategy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#ordered_placement_strategy) **Type:** ```hcl list(object({ type = string field = string })) ``` **Default value:** `[ ]`
`permissions_boundary` (`string`) optional
A permissions boundary ARN to apply to the 3 roles that are created. **Default value:** `""`
`pid_mode` (`string`) optional
The process namespace to use for the containers in the task. The valid values are `host` and `task`. If `host` is specified, then all containers within the tasks that specified the `host` PID mode on the same container instance share the same process namespace with the host Amazon EC2 instanc . If `task` is specified, all containers within the specified task share the same process namespace. If no value is specified, then the process namespace sharing depends on the Docker daemon setting on the container instance. For more information, see PID settings in the Docker documentation. **Default value:** `null`
`platform_version` (`string`) optional
The platform version on which to run your service. Only applicable for `launch_type` set to `FARGATE`. More information about Fargate platform versions can be found in the AWS ECS User Guide. **Default value:** `"LATEST"`
`propagate_tags` (`string`) optional
Specifies whether to propagate the tags from the task definition or the service to the tasks. The valid values are SERVICE and TASK_DEFINITION **Default value:** `null`
`proxy_configuration` optional
The proxy configuration details for the App Mesh proxy. See `proxy_configuration` docs https://www.terraform.io/docs/providers/aws/r/ecs_task_definition.html#proxy-configuration-arguments **Type:** ```hcl object({ type = string container_name = string properties = map(string) }) ``` **Default value:** `null`
`redeploy_on_apply` (`bool`) optional
Updates the service to the latest task definition on each apply **Default value:** `false`
`role_tags_enabled` (`bool`) optional
Whether or not to create tags on ECS roles **Default value:** `true`
`runtime_platform` (`list(map(string))`) optional
Zero or one runtime platform configurations that containers in your task may use. Map of strings with optional keys `operating_system_family` and `cpu_architecture`. See `runtime_platform` docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition#runtime_platform **Default value:** `[ ]`
`scheduling_strategy` (`string`) optional
The scheduling strategy to use for the service. The valid values are `REPLICA` and `DAEMON`. Note that Fargate tasks do not support the DAEMON scheduling strategy. **Default value:** `"REPLICA"`
`security_group_description` (`string`) optional
The description to assign to the service security group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Allow ALL egress from ECS service"`
`security_group_enabled` (`bool`) optional
Whether to create a security group for the service. **Default value:** `true`
`security_group_ids` (`list(string)`) optional
Security group IDs to allow in Service `network_configuration` if `var.network_mode = "awsvpc"` **Default value:** `[ ]`
`service_connect_configurations` optional
The list of Service Connect configurations. See `service_connect_configuration` docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#service_connect_configuration **Type:** ```hcl list(object({ enabled = bool namespace = optional(string, null) log_configuration = optional(object({ log_driver = string options = optional(map(string), null) secret_option = optional(list(object({ name = string value_from = string })), []) }), null) service = optional(list(object({ client_alias = list(object({ dns_name = string port = number })) timeout = optional(list(object({ idle_timeout_seconds = optional(number, null) per_request_timeout_seconds = optional(number, null) })), []) tls = optional(list(object({ kms_key = optional(string, null) role_arn = optional(string, null) issuer_cert_authority = object({ aws_pca_authority_arn = string }) })), []) discovery_name = optional(string, null) ingress_port_override = optional(number, null) port_name = string })), []) })) ``` **Default value:** `[ ]`
`service_placement_constraints` optional
The rules that are taken into consideration during task placement. Maximum number of placement_constraints is 10. See [`placement_constraints`](https://www.terraform.io/docs/providers/aws/r/ecs_service.html#placement_constraints-1) docs **Type:** ```hcl list(object({ type = string expression = string })) ``` **Default value:** `[ ]`
`service_registries` (`list(any)`) optional
Zero or one service discovery registries for the service. The currently supported service registry is Amazon Route 53 Auto Naming Service - `aws_service_discovery_service`; see `service_registries` docs https://www.terraform.io/docs/providers/aws/r/ecs_service.html#service_registries-1" Service registry is object with required key `registry_arn = string` and optional keys `port = number` `container_name = string` `container_port = number` **Default value:** `[ ]`
`service_role_arn` (`string`) optional
ARN of the IAM role that allows Amazon ECS to make calls to your load balancer on your behalf. This parameter is required if you are using a load balancer with your service, but only if your task definition does not use the awsvpc network mode. If using awsvpc network mode, do not specify this role. If your account has already created the Amazon ECS service-linked role, that role is used by default for your service unless you specify a role here. **Default value:** `null`
`subnet_ids` (`list(string)`) optional
Subnet IDs used in Service `network_configuration` if `var.network_mode = "awsvpc"` **Default value:** `null`
`task_cpu` (`number`) optional
The number of CPU units used by the task. If using `FARGATE` launch type `task_cpu` must match [supported memory values](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size) **Default value:** `256`
`task_definition` (`any`) optional
A `list(string)` of zero or one ARNs of task definitions, to reuse reuse an existing task definition family and revision for the ecs service instead of creating one DEPRECATED: you can also pass a `string` with the ARN, but that string must be known a "plan" time. **Default value:** `[ ]`
`task_exec_policy_arns` (`list(string)`) optional
A list of IAM Policy ARNs to attach to the generated task execution role. Changes to the list will have ripple effects, so use `task_exec_policy_arns_map` if possible. **Default value:** `[ ]`
`task_exec_policy_arns_map` (`map(string)`) optional
A map of name to IAM Policy ARNs to attach to the generated task execution role. The names are arbitrary, but must be known at plan time. The purpose of the name is so that changes to one ARN do not cause a ripple effect on the other ARNs. If you cannot provide unique names known at plan time, use `task_exec_policy_arns` instead. **Default value:** `{ }`
`task_exec_role_arn` (`any`) optional
A `list(string)` of zero or one ARNs of IAM roles that allows the ECS/Fargate agent to make calls to the ECS API on your behalf. If the list is empty, a role will be created for you. DEPRECATED: you can also pass a `string` with the ARN, but that string must be known a "plan" time. **Default value:** `[ ]`
`task_memory` (`number`) optional
The amount of memory (in MiB) used by the task. If using Fargate launch type `task_memory` must match [supported cpu value](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size) **Default value:** `512`
`task_placement_constraints` optional
A set of placement constraints rules that are taken into consideration during task placement. Maximum number of placement_constraints is 10. See [`placement_constraints`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition#placement-constraints-arguments) **Type:** ```hcl list(object({ type = string expression = string })) ``` **Default value:** `[ ]`
`task_policy_arns` (`list(string)`) optional
A list of IAM Policy ARNs to attach to the generated task role. Changes to the list will have ripple effects, so use `task_policy_arns_map` if possible. **Default value:** `[ ]`
`task_policy_arns_map` (`map(string)`) optional
A map of name to IAM Policy ARNs to attach to the generated task role. The names are arbitrary, but must be known at plan time. The purpose of the name is so that changes to one ARN do not cause a ripple effect on the other ARNs. If you cannot provide unique names known at plan time, use `task_policy_arns` instead. **Default value:** `{ }`
`task_role_arn` (`any`) optional
A `list(string)` of zero or one ARNs of IAM roles that allows your Amazon ECS container task to make calls to other AWS services. If the list is empty, a role will be created for you. DEPRECATED: you can also pass a `string` with the ARN, but that string must be known a "plan" time. **Default value:** `[ ]`
`track_latest` (`bool`) optional
Whether should track latest task definition or the one created with the resource. **Default value:** `false`
`use_alb_security_group` (`bool`) optional
A flag to enable/disable allowing traffic from the ALB security group to the service security group **Default value:** `false`
`use_nlb_cidr_blocks` (`bool`) optional
A flag to enable/disable adding the NLB ingress rule to the service security group **Default value:** `false`
`use_old_arn` (`bool`) optional
A flag to enable/disable tagging the ecs resources that require the new arn format **Default value:** `false`
`wait_for_steady_state` (`bool`) optional
If true, it will wait for the service to reach a steady state (like aws ecs wait services-stable) before continuing **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ecs_exec_role_policy_id`
The ECS service role policy ID, in the form of `role_name:role_policy_name`
`ecs_exec_role_policy_name`
ECS service role name
`service_arn`
ECS Service ARN
`service_name`
ECS Service name
`service_role_arn`
ECS Service role ARN
`service_security_group_id`
Security Group ID of the ECS task
`task_definition_arn`
ECS task definition ARN
`task_definition_arn_without_revision`
ECS task definition ARN without revision
`task_definition_family`
ECS task definition family
`task_definition_revision`
ECS task definition revision
`task_exec_role_arn`
ECS Task exec role ARN
`task_exec_role_id`
ECS Task exec role id
`task_exec_role_name`
ECS Task exec role name
`task_role_arn`
ECS Task role ARN
`task_role_id`
ECS Task role id
`task_role_name`
ECS Task role name
## Dependencies ### Requirements - `terraform`, version: `>= 0.14.0` - `aws`, version: `>= 5.85` ### Providers - `aws`, version: `>= 5.85` ### Modules Name | Version | Source | Description --- | --- | --- | --- `exec_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `service_connect_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `service_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `task_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ecs_service.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) (resource) - [`aws_ecs_service.ignore_changes_desired_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) (resource) - [`aws_ecs_service.ignore_changes_task_definition`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) (resource) - [`aws_ecs_service.ignore_changes_task_definition_and_desired_count`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) (resource) - [`aws_ecs_task_definition.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition) (resource) - [`aws_iam_role.ecs_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.ecs_service_connect_tls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.ecs_task`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.ecs_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy.ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy.ecs_ssm_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.ecs_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ecs_service_connect_tls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ecs_task`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_security_group.ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.alb`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.allow_all_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.allow_icmp_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.nlb`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.ecs_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_service_connect_tls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_service_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_ssm_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_task`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ecs_task_exec`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## ecs-cloudwatch-autoscaling # Module: `ecs-cloudwatch-autoscaling` Terraform module for creating alarms for tracking important changes and occurrences from ECS Services. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling/tree/main/examples/complete). For automated tests of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-autoscaling/tree/main/test). ```hcl module "ecs_cloudwatch_autoscaling" { source = "cloudposse/ecs-cloudwatch-autoscaling/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "staging" name = "app" service_name = "eg-staging-example-service" cluster_name = "eg-staging-example-cluster" min_capacity = 1 max_capacity = 2 scale_up_adjustment = 1 scale_up_cooldown = 60 scale_down_adjustment = -1 scale_down_cooldown = 300 } ``` ## Examples For a complete usage example, see [terraform-aws-ecs-web-app module](https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/main.tf). ## Variables ### Required Variables
`cluster_name` (`string`) required
The name of the ECS cluster where service is to be autoscaled
`service_name` (`string`) required
The name of the ECS Service to autoscale
### Optional Variables
`max_capacity` (`number`) optional
Maximum number of running instances of a Service **Default value:** `2`
`min_capacity` (`number`) optional
Minimum number of running instances of a Service **Default value:** `1`
`scale_down_cooldown` (`number`) optional
Period (in seconds) to wait between scale down events **Default value:** `300`
`scale_down_step_adjustments` optional
List of step adjustments for scale down policy **Type:** ```hcl list(object({ metric_interval_lower_bound = optional(number) metric_interval_upper_bound = optional(number) scaling_adjustment = number })) ``` **Default value:** ```hcl [ { "metric_interval_lower_bound": null, "metric_interval_upper_bound": 0, "scaling_adjustment": -1 } ] ```
`scale_up_cooldown` (`number`) optional
Period (in seconds) to wait between scale up events **Default value:** `60`
`scale_up_step_adjustments` optional
List of step adjustments for scale up policy **Type:** ```hcl list(object({ metric_interval_lower_bound = optional(number) metric_interval_upper_bound = optional(number) scaling_adjustment = number })) ``` **Default value:** ```hcl [ { "metric_interval_lower_bound": 0, "metric_interval_upper_bound": null, "scaling_adjustment": 1 } ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`scale_down_policy_arn`
ARN of the scale down policy
`scale_up_policy_arn`
ARN of the scale up policy
## Dependencies ### Requirements - `terraform`, version: `>= 1` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `scale_down_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `scale_up_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_appautoscaling_policy.down`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_policy.up`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_target.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) ## Data Sources The following data sources are used by this module: --- ## ecs-cloudwatch-sns-alarms # Module: `ecs-cloudwatch-sns-alarms` Terraform module for creating alarms for tracking important changes and occurrences from ECS Services. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms/tree/main/examples/complete). For automated tests of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-ecs-cloudwatch-sns-alarms/tree/main/test). ```hcl module "ecs_service_alarms" { source = "cloudposse/ecs-cloudwatch-sns-alarms/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "staging" name = "app" cluster_name = "example" service_name = "app" } ``` ## Variables ### Required Variables
`cluster_name` (`string`) required
The name of the ECS cluster to monitor
### Optional Variables
`alarm_description` (`string`) optional
The string to format and use as the alarm description. **Default value:** `"Average service %v utilization %v last %d minute(s) over %v period(s)"`
`cpu_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High Alarm action **Default value:** `[ ]`
`cpu_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`cpu_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High OK action **Default value:** `[ ]`
`cpu_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`cpu_utilization_high_threshold` (`number`) optional
The maximum percentage of CPU utilization average **Default value:** `80`
`cpu_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low Alarm action **Default value:** `[ ]`
`cpu_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`cpu_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low OK action **Default value:** `[ ]`
`cpu_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`cpu_utilization_low_threshold` (`number`) optional
The minimum percentage of CPU utilization average **Default value:** `20`
`memory_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High Alarm action **Default value:** `[ ]`
`memory_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`memory_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High OK action **Default value:** `[ ]`
`memory_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`memory_utilization_high_threshold` (`number`) optional
The maximum percentage of Memory utilization average **Default value:** `80`
`memory_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low Alarm action **Default value:** `[ ]`
`memory_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`memory_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low OK action **Default value:** `[ ]`
`memory_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`memory_utilization_low_threshold` (`number`) optional
The minimum percentage of Memory utilization average **Default value:** `20`
`service_name` (`string`) optional
The name of the ECS Service in the ECS cluster to monitor **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cpu_utilization_high_cloudwatch_metric_alarm_arn`
CPU utilization high CloudWatch metric alarm ARN
`cpu_utilization_high_cloudwatch_metric_alarm_id`
CPU utilization high CloudWatch metric alarm ID
`cpu_utilization_low_cloudwatch_metric_alarm_arn`
CPU utilization low CloudWatch metric alarm ARN
`cpu_utilization_low_cloudwatch_metric_alarm_id`
CPU utilization low CloudWatch metric alarm ID
`memory_utilization_high_cloudwatch_metric_alarm_arn`
Memory utilization high CloudWatch metric alarm ARN
`memory_utilization_high_cloudwatch_metric_alarm_id`
Memory utilization high CloudWatch metric alarm ID
`memory_utilization_low_cloudwatch_metric_alarm_arn`
Memory utilization low CloudWatch metric alarm ARN
`memory_utilization_low_cloudwatch_metric_alarm_id`
Memory utilization low CloudWatch metric alarm ID
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cpu_utilization_high_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `cpu_utilization_low_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `memory_utilization_high_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `memory_utilization_low_alarm_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.cpu_utilization_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.cpu_utilization_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.memory_utilization_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.memory_utilization_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) ## Data Sources The following data sources are used by this module: --- ## ecs-cluster # Module: `ecs-cluster` Terraform module to provision an [`ECS Cluster`](https://aws.amazon.com/ru/ecs/) with list of [`capacity providers`](https://docs.aws.amazon.com/AmazonECS/latest/userguide/cluster-capacity-providers.html). Supports [Amazon ECS Fargate](https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-capacity-providers.html) and [EC2 Autoscaling](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html) capacity providers. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-cluster/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ecs-cluster/tree/main/test). ### ECS cluster with Fargate capacity provider ```hcl module "ecs_cluster" { source = "cloudposse/ecs-cluster/aws" namespace = "eg" name = "example" container_insights_enabled = true capacity_providers_fargate = true } ``` ### ECS cluster with Fargate and EC2 autoscale capacity provider ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "vpc" { source = "cloudposse/vpc/aws" version = "1.2.0" context = module.label.context ipv4_primary_cidr_block = "172.16.0.0/16" } module "subnets" { source = "cloudposse/dynamic-subnets/aws" version = "2.0.4" context = module.label.context availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false } module "ecs_cluster" { source = "cloudposse/ecs-cluster/aws" context = module.label.context container_insights_enabled = true capacity_providers_fargate = true capacity_providers_fargate_spot = true capacity_providers_ec2 = { default = { instance_type = "t3.medium" security_group_ids = [module.vpc.vpc_default_security_group_id] subnet_ids = module.subnets.private_subnet_ids associate_public_ip_address = false min_size = 0 max_size = 2 } } } ``` ## Variables ### Required Variables
### Optional Variables
`capacity_providers_ec2` optional
EC2 autoscale groups capacity providers **Type:** ```hcl map(object({ name = optional(string, null) instance_type = string max_size = number min_size = number subnet_ids = list(string) security_group_ids = list(string) image_id = optional(string, null) instance_initiated_shutdown_behavior = optional(string, "terminate") key_name = optional(string, "") user_data = optional(string, "") enable_monitoring = optional(bool, true) instance_warmup_period = optional(number, 300) maximum_scaling_step_size = optional(number, 1) minimum_scaling_step_size = optional(number, 1) target_capacity_utilization = optional(number, 100) ebs_optimized = optional(bool, false) associate_public_ip_address = optional(bool, false) block_device_mappings = optional(list(object({ device_name = string no_device = bool virtual_name = string ebs = object({ delete_on_termination = bool encrypted = bool iops = number throughput = number kms_key_id = string snapshot_id = string volume_size = number volume_type = string }) })), []) instance_market_options = optional(object({ market_type = string spot_options = object({ block_duration_minutes = number instance_interruption_behavior = string max_price = number spot_instance_type = string valid_until = string }) })) instance_refresh = optional(object({ strategy = string preferences = object({ instance_warmup = number min_healthy_percentage = number skip_matching = bool auto_rollback = bool scale_in_protected_instances = string standby_instances = string }) triggers = list(string) })) mixed_instances_policy = optional(object({ instances_distribution = object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string }) })) placement = optional(object({ affinity = string availability_zone = string group_name = string host_id = string tenancy = string })) credit_specification = optional(object({ cpu_credits = string })) disable_api_termination = optional(bool, false) default_cooldown = optional(number, 300) health_check_grace_period = optional(number, 300) force_delete = optional(bool, false) termination_policies = optional(list(string), ["Default"]) suspended_processes = optional(list(string), []) placement_group = optional(string, "") metrics_granularity = optional(string, "1Minute") enabled_metrics = optional(list(string), [ "GroupMinSize", "GroupMaxSize", "GroupDesiredCapacity", "GroupInServiceInstances", "GroupPendingInstances", "GroupStandbyInstances", "GroupTerminatingInstances", "GroupTotalInstances", "GroupInServiceCapacity", "GroupPendingCapacity", "GroupStandbyCapacity", "GroupTerminatingCapacity", "GroupTotalCapacity", "WarmPoolDesiredCapacity", "WarmPoolWarmedCapacity", "WarmPoolPendingCapacity", "WarmPoolTerminatingCapacity", "WarmPoolTotalCapacity", "GroupAndWarmPoolDesiredCapacity", "GroupAndWarmPoolTotalCapacity", ]) wait_for_capacity_timeout = optional(string, "10m") service_linked_role_arn = optional(string, "") metadata_http_endpoint_enabled = optional(bool, true) metadata_http_put_response_hop_limit = optional(number, 2) metadata_http_tokens_required = optional(bool, true) metadata_http_protocol_ipv6_enabled = optional(bool, false) tag_specifications_resource_types = optional(set(string), ["instance", "volume"]) max_instance_lifetime = optional(number, null) capacity_rebalance = optional(bool, false) launch_template_version = optional(string, "$Latest") update_default_version = optional(bool, false) warm_pool = optional(object({ pool_state = string min_size = number max_group_prepared_capacity = number })) instance_reuse_policy = optional(object({ reuse_on_scale_in = optional(bool, false) }), null) })) ``` **Default value:** `{ }`
`capacity_providers_fargate` (`bool`) optional
Use FARGATE capacity provider **Default value:** `true`
`capacity_providers_fargate_spot` (`bool`) optional
Use FARGATE_SPOT capacity provider **Default value:** `false`
`container_insights_enabled` (`bool`) optional
Whether or not to enable container insights **Default value:** `true`
`default_capacity_strategy` optional
The capacity provider strategy to use by default for the cluster **Type:** ```hcl object({ base = object({ provider = string value = number }) weights = map(number) }) ``` **Default value:** ```hcl { "base": { "provider": "FARGATE", "value": 1 }, "weights": {} } ```
`external_ec2_capacity_providers` optional
External EC2 autoscale groups capacity providers **Type:** ```hcl map(object({ autoscaling_group_arn = string managed_termination_protection = bool managed_scaling_status = bool instance_warmup_period = optional(number, 300) maximum_scaling_step_size = optional(number, 1) minimum_scaling_step_size = optional(number, 1) target_capacity_utilization = optional(number, 100) })) ``` **Default value:** `{ }`
`kms_key_id` (`string`) optional
The AWS Key Management Service key ID to encrypt the data between the local client and the container. **Default value:** `null`
`log_configuration` optional
The log configuration for the results of the execute command actions Required when logging is OVERRIDE **Type:** ```hcl object({ cloud_watch_encryption_enabled = string cloud_watch_log_group_name = string s3_bucket_name = string s3_key_prefix = string }) ``` **Default value:** `null`
`logging` (`string`) optional
The AWS Key Management Service key ID to encrypt the data between the local client and the container. (Valid values: 'NONE', 'DEFAULT', 'OVERRIDE') **Default value:** `"DEFAULT"`
`service_discovery_namespace_arn` (`string`) optional
The Amazon Resource Name (ARN) of the service discovery namespace that you want to use for default service discovery. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ECS cluster arn
`id`
ECS cluster id
`name`
ECS cluster name
`role_name`
IAM role name
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 6.0.0` ### Providers - `aws`, version: `>= 6.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `autoscale_group` | 0.41.1 | [`cloudposse/ec2-autoscale-group/aws`](https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.41.1) | n/a `ecs_labels` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ecs_capacity_provider.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_capacity_provider) (resource) - [`aws_ecs_capacity_provider.external_ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_capacity_provider) (resource) - [`aws_ecs_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster) (resource) - [`aws_ecs_cluster_capacity_providers.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_cluster_capacity_providers) (resource) - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.ami`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## ecs-codepipeline # Module: `ecs-codepipeline` Terraform Module for CI/CD with AWS Code Pipeline using GitHub webhook triggers and Code Build for ECS. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-codepipeline/tree/main/examples/complete). For automated tests of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-ecs-codepipeline/tree/main/test). ### Trigger on GitHub Push In this example, we'll trigger the pipeline anytime the `master` branch is updated. ```hcl module "ecs_push_pipeline" { source = "cloudposse/ecs-codepipeline/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "app" namespace = "eg" stage = "staging" github_oauth_token = "xxxxxxxxxxxxxx" github_webhooks_token = "xxxxxxxxxxxxxx" repo_owner = "cloudposse" repo_name = "example" branch = "master" service_name = "example" ecs_cluster_name = "eg-staging-example-cluster" privileged_mode = "true" } ``` ### Trigger on GitHub Releases In this example, we'll trigger anytime a new GitHub release is cut by setting the even type to `release` and using the `json_path` to *exactly* match an `action` of `published`. ```hcl module "ecs_release_pipeline" { source = "cloudposse/ecs-codepipeline/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "app" namespace = "eg" stage = "staging" github_oauth_token = "xxxxxxxxxxxxxx" github_webhooks_token = "xxxxxxxxxxxxxx" repo_owner = "cloudposse" repo_name = "example" branch = "master" service_name = "example" ecs_cluster_name = "eg-staging-example-cluster" privileged_mode = "true" github_webhook_events = ["release"] webhook_filter_json_path = "$.action" webhook_filter_match_equals = "published" } ``` (Thanks to [Stack Overflow](https://stackoverflow.com/questions/52516087/trigger-aws-codepipeline-by-github-release-webhook#comment91997146_52524711)) ## Examples Complete usage can be seen in the [terraform-aws-ecs-web-app](https://github.com/cloudposse/terraform-aws-ecs-web-app/blob/master/main.tf) module. ## Example Buildspec Here's an example `buildspec.yaml`. Stick this in the root of your project repository. ```yaml version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - aws --version - eval $(aws ecr get-login --region $AWS_DEFAULT_REGION --no-include-email) - REPOSITORY_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_REPO_NAME - IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7) build: commands: - echo Build started on `date` - echo Building the Docker image... - REPO_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_REPO_NAME - docker pull $REPO_URI:latest || true - docker build --cache-from $REPO_URI:latest --tag $REPO_URI:latest --tag $REPO_URI:$IMAGE_TAG . post_build: commands: - echo Build completed on `date` - echo Pushing the Docker images... - REPO_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_REPO_NAME - docker push $REPO_URI:latest - docker push $REPO_URI:$IMAGE_TAG - echo Writing image definitions file... - printf '[{"name":"%s","imageUri":"%s"}]' "$CONTAINER_NAME" "$REPO_URI:$IMAGE_TAG" | tee imagedefinitions.json artifacts: files: imagedefinitions.json ``` ## Variables ### Required Variables
`branch` (`string`) required
Branch of the GitHub repository, _e.g._ `master`
`ecs_cluster_name` (`string`) required
ECS Cluster Name
`image_repo_name` (`string`) required
ECR repository name to store the Docker image built by this module. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html)
`region` (`string`) required
AWS Region, e.g. us-east-1. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html)
`repo_name` (`string`) required
GitHub repository name of the application to be built and deployed to ECS
`repo_owner` (`string`) required
GitHub Organization or Username
`service_name` (`string`) required
ECS Service Name
### Optional Variables
`aws_account_id` (`string`) optional
AWS Account ID. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `""`
`badge_enabled` (`bool`) optional
Generates a publicly-accessible URL for the projects build badge. Available as badge_url attribute when enabled **Default value:** `false`
`build_compute_type` (`string`) optional
`CodeBuild` instance size. Possible values are: `BUILD_GENERAL1_SMALL` `BUILD_GENERAL1_MEDIUM` `BUILD_GENERAL1_LARGE` **Default value:** `"BUILD_GENERAL1_SMALL"`
`build_image` (`string`) optional
Docker image for build environment, https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html **Default value:** `"aws/codebuild/amazonlinux2-x86_64-standard:5.0"`
`build_timeout` (`number`) optional
How long in minutes, from 5 to 480 (8 hours), for AWS CodeBuild to wait until timing out any related build that does not get marked as completed **Default value:** `60`
`build_type` (`string`) optional
The type of build environment, e.g. 'LINUX_CONTAINER' or 'WINDOWS_CONTAINER' or 'ARM_CONTAINER' **Default value:** `"LINUX_CONTAINER"`
`buildspec` (`string`) optional
Declaration to use for building the project. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `""`
`cache_bucket_suffix_enabled` (`bool`) optional
The cache bucket generates a random 13 character string to generate a unique bucket name. If set to false it uses terraform-null-label's id value. It only works when cache_type is 'S3' **Default value:** `true`
`cache_type` (`string`) optional
The type of storage that will be used for the AWS CodeBuild project cache. Valid values: NO_CACHE, LOCAL, and S3. Defaults to S3. If cache_type is S3, it will create an S3 bucket for storing codebuild cache inside **Default value:** `"S3"`
`codebuild_extra_policy_arns` (`list(string)`) optional
List of ARNs of extra policies to attach to the CodeBuild role **Default value:** `[ ]`
`codebuild_vpc_config` (`any`) optional
Configuration for the builds to run inside a VPC. **Default value:** `{ }`
`codestar_connection_arn` (`string`) optional
CodeStar connection ARN required for Bitbucket integration with CodePipeline **Default value:** `""`
`codestar_output_artifact_format` (`string`) optional
Output artifact type for Source stage in pipeline. Valid values are "CODE_ZIP" (default) and "CODEBUILD_CLONE_REF". See https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodestarConnectionSource.html **Default value:** `"CODE_ZIP"`
`environment_variables` optional
A list of maps, that contain the keys 'name', 'value', and 'type' to be used as additional environment variables for the build. Valid types are 'PLAINTEXT', 'PARAMETER_STORE', or 'SECRETS_MANAGER' **Type:** ```hcl list(object( { name = string value = string type = string })) ``` **Default value:** `[ ]`
`github_oauth_token` (`string`) optional
GitHub OAuth Token with permissions to access private repositories **Default value:** `""`
`github_webhook_events` (`list(string)`) optional
A list of events which should trigger the webhook. See a list of [available events](https://developer.github.com/v3/activity/events/types/) **Default value:** ```hcl [ "push" ] ```
`image_tag` (`string`) optional
Docker image tag in the ECR repository, e.g. 'latest'. Used as CodeBuild ENV variable when building Docker images. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/sample-docker.html) **Default value:** `"latest"`
`local_cache_modes` (`list(string)`) optional
Specifies settings that AWS CodeBuild uses to store and reuse build dependencies. Valid values: LOCAL_SOURCE_CACHE, LOCAL_DOCKER_LAYER_CACHE, and LOCAL_CUSTOM_CACHE **Default value:** `[ ]`
`poll_source_changes` (`bool`) optional
Periodically check the location of your source content and run the pipeline if changes are detected **Default value:** `false`
`privileged_mode` (`bool`) optional
If set to true, enables running the Docker daemon inside a Docker container on the CodeBuild instance. Used when building Docker images **Default value:** `false`
`s3_bucket_force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the CodePipeline artifact store S3 bucket so that the bucket can be destroyed without error **Default value:** `false`
`secondary_artifact_bucket_id` (`string`) optional
Optional bucket for secondary artifact deployment. If specified, the buildspec must include a secondary artifacts section which controls the artifacts deployed to the bucket [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `null`
`secondary_artifact_encryption_enabled` (`bool`) optional
If set to true, enable encryption on the secondary artifact bucket **Default value:** `false`
`secondary_artifact_identifier` (`string`) optional
Identifier for optional secondary artifact deployment. If specified, the identifier must appear in the buildspec as the name of the section which controls the artifacts deployed to the secondary artifact bucket [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `null`
`webhook_authentication` (`string`) optional
The type of authentication to use. One of IP, GITHUB_HMAC, or UNAUTHENTICATED **Default value:** `"GITHUB_HMAC"`
`webhook_enabled` (`bool`) optional
Set to false to prevent the module from creating any webhook resources **Default value:** `true`
`webhook_filter_json_path` (`string`) optional
The JSON path to filter on **Default value:** `"$.ref"`
`webhook_filter_match_equals` (`string`) optional
The value to match on (e.g. refs/heads/\{Branch\}) **Default value:** `"refs/heads/{Branch}"`
`webhook_target_action` (`string`) optional
The name of the action in a pipeline you want to connect to the webhook. The action must be from the source (first) stage of the pipeline **Default value:** `"Source"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`badge_url`
The URL of the build badge when badge_enabled is enabled
`codebuild_badge_url`
The URL of the build badge when badge_enabled is enabled
`codebuild_cache_bucket_arn`
CodeBuild cache S3 bucket ARN
`codebuild_cache_bucket_name`
CodeBuild cache S3 bucket name
`codebuild_project_id`
CodeBuild project ID
`codebuild_project_name`
CodeBuild project name
`codebuild_role_arn`
CodeBuild IAM Role ARN
`codebuild_role_id`
CodeBuild IAM Role ID
`codepipeline_arn`
CodePipeline ARN
`codepipeline_id`
CodePipeline ID
`codepipeline_resource`
CodePipeline resource
`webhook_id`
The CodePipeline webhook's ID
`webhook_url`
The CodePipeline webhook's URL. POST events to this endpoint to trigger the target
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Providers - `aws`, version: `>= 5.0` - `random`, version: `>= 2.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `codebuild` | 2.0.2 | [`cloudposse/codebuild/aws`](https://registry.terraform.io/modules/cloudposse/codebuild/aws/2.0.2) | n/a `codebuild_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `codepipeline_assume_role_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `codepipeline_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `codepipeline_s3_policy_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `codestar_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `github_webhooks` | 0.14.0 | [`cloudposse/repository-webhooks/github`](https://registry.terraform.io/modules/cloudposse/repository-webhooks/github/0.14.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_codepipeline.bitbucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline) (resource) - [`aws_codepipeline.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline) (resource) - [`aws_codepipeline_webhook.webhook`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codepipeline_webhook) (resource) - [`aws_iam_policy.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.codestar`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.codebuild_codestar`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.codebuild_extras`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.codebuild_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.codestar`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`random_string.webhook_secret`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.codebuild`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.codestar`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## ecs-container-definition # Module: `ecs-container-definition` Terraform module to generate well-formed JSON documents that are passed to the `aws_ecs_task_definition` Terraform resource as [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions). ## Usage This module is meant to be used as output only, meaning it will be used to create outputs which are consumed as a parameter by Terraform resources or other modules. Caution: This module, unlike nearly all other Cloud Posse Terraform modules, does not use [terraform-null-label](https://github.com/cloudposse/terraform-null-label/). Furthermore, it has an input named `environment` which has a completely different meaning than the one in `terraform-null-label`. Do not call this module with the conventional `context = module.this.context`. See the documentation below for the usage of `environment`. For complete examples, see - [multi-port mappings](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/examples/multi_port_mappings) - [multi-type env vars](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/examples/multi_type_env_vars) - [multiple definitions](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/examples/multiple_definitions) - [string env vars](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/examples/string_env_vars) For a complete example with automated tests, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/examples/complete) with `bats` and `Terratest` for the example [test](https://github.com/cloudposse/terraform-aws-ecs-container-definition/tree/main/test). ```hcl module "container_definition" { source = "cloudposse/ecs-container-definition/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" container_name = "geodesic" container_image = "cloudposse/geodesic" } ``` The output of this module can then be used with one of our other modules. ```hcl module "ecs_alb_service_task" { source = "cloudposse/ecs-alb-service-task/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" # ... container_definition_json = module.container_definition.json_map_encoded_list # ... } ``` ## Variables ### Required Variables
`container_image` (`string`) required
The image used to start the container. Images in the Docker Hub registry available by default
`container_name` (`string`) required
The name of the container. Up to 255 characters ([a-z], [A-Z], [0-9], -, _ allowed)
### Optional Variables
`command` (`list(string)`) optional
The command that is passed to the container **Default value:** `null`
`container_cpu` (`number`) optional
The number of cpu units to reserve for the container. This is optional for tasks using Fargate launch type and the total amount of container_cpu of all containers in a task will need to be lower than the task-level cpu value **Default value:** `0`
`container_definition` optional
Container definition overrides which allows for extra keys or overriding existing keys. **Type:** ```hcl object({ command = optional(list(string)) cpu = optional(number) dependsOn = optional(list(object({ condition = string containerName = string }))) disableNetworking = optional(bool) dnsSearchDomains = optional(list(string)) dnsServers = optional(list(string)) dockerLabels = optional(map(string)) dockerSecurityOptions = optional(list(string)) entryPoint = optional(list(string)) environment = optional(list(object({ name = string value = string }))) environmentFiles = optional(list(object({ type = string value = string }))) essential = optional(bool) extraHosts = optional(list(object({ hostname = string ipAddress = string }))) firelensConfiguration = optional(object({ options = optional(map(string)) type = string })) healthCheck = optional(object({ command = list(string) interval = optional(number) retries = optional(number) startPeriod = optional(number) timeout = optional(number) })) hostname = optional(string) image = optional(string) interactive = optional(bool) links = optional(list(string)) linuxParameters = optional(object({ capabilities = optional(object({ add = optional(list(string)) drop = optional(list(string)) })) devices = optional(list(object({ containerPath = string hostPath = string permissions = optional(list(string)) }))) initProcessEnabled = optional(bool) maxSwap = optional(number) sharedMemorySize = optional(number) swappiness = optional(number) tmpfs = optional(list(object({ containerPath = string mountOptions = optional(list(string)) size = number }))) })) logConfiguration = optional(object({ logDriver = string options = optional(map(string)) secretOptions = optional(list(object({ name = string valueFrom = string }))) })) memory = optional(number) memoryReservation = optional(number) mountPoints = optional(list(object({ containerPath = optional(string) readOnly = optional(bool) sourceVolume = optional(string) }))) name = optional(string) portMappings = optional(list(object({ containerPort = number hostPort = optional(number) protocol = optional(string) name = optional(string) appProtocol = optional(string) }))) privileged = optional(bool) pseudoTerminal = optional(bool) readonlyRootFilesystem = optional(bool) repositoryCredentials = optional(object({ credentialsParameter = string })) resourceRequirements = optional(list(object({ type = string value = string }))) restartPolicy = optional(object({ enabled = bool ignoredExitCodes = optional(list(number)) restartAttemptPeriod = optional(number) })) secrets = optional(list(object({ name = string valueFrom = string }))) startTimeout = optional(number) stopTimeout = optional(number) systemControls = optional(list(object({ namespace = string value = string }))) ulimits = optional(list(object({ hardLimit = number name = string softLimit = number }))) user = optional(string) versionConsistency = optional(string) volumesFrom = optional(list(object({ readOnly = optional(bool) sourceContainer = string }))) workingDirectory = optional(string) }) ``` **Default value:** `{ }`
`container_depends_on` optional
The dependencies defined for container startup and shutdown. A container can contain multiple dependencies. When a dependency is defined for container startup, for container shutdown it is reversed. The condition can be one of START, COMPLETE, SUCCESS or HEALTHY **Type:** ```hcl list(object({ condition = string containerName = string })) ``` **Default value:** `null`
`container_memory` (`number`) optional
The amount of memory (in MiB) to allow the container to use. This is a hard limit, if the container attempts to exceed the container_memory, the container is killed. This field is optional for Fargate launch type and the total amount of container_memory of all containers in a task will need to be lower than the task memory value **Default value:** `null`
`container_memory_reservation` (`number`) optional
The amount of memory (in MiB) to reserve for the container. If container needs to exceed this threshold, it can do so up to the set container_memory hard limit **Default value:** `null`
`disable_networking` (`bool`) optional
When this parameter is true, networking is disabled within the container. **Default value:** `null`
`dns_search_domains` (`list(string)`) optional
Container DNS search domains. A list of DNS search domains that are presented to the container **Default value:** `null`
`dns_servers` (`list(string)`) optional
Container DNS servers. This is a list of strings specifying the IP addresses of the DNS servers **Default value:** `null`
`docker_labels` (`map(string)`) optional
The configuration options to send to the `docker_labels` **Default value:** `null`
`docker_security_options` (`list(string)`) optional
A list of strings to provide custom labels for SELinux and AppArmor multi-level security systems. **Default value:** `null`
`entrypoint` (`list(string)`) optional
The entry point that is passed to the container **Default value:** `null`
`environment_files` optional
One or more files containing the environment variables to pass to the container. This maps to the --env-file option to docker run. The file must be hosted in Amazon S3. This option is only available to tasks using the EC2 launch type. This is a list of maps **Type:** ```hcl list(object({ type = string value = string })) ``` **Default value:** `null`
`essential` (`bool`) optional
Determines whether all other containers in a task are stopped, if this container fails or stops for any reason. Due to how Terraform type casts booleans in json it is required to double quote this value **Default value:** `true`
`extra_hosts` optional
A list of hostnames and IP address mappings to append to the /etc/hosts file on the container. This is a list of maps **Type:** ```hcl list(object({ hostname = string ipAddress = string })) ``` **Default value:** `null`
`firelens_configuration` optional
The FireLens configuration for the container. This is used to specify and configure a log router for container logs. For more details, see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_FirelensConfiguration.html **Type:** ```hcl object({ options = optional(map(string)) type = string }) ``` **Default value:** `null`
`healthcheck` optional
A map containing command (string), timeout, interval (duration in seconds), retries (1-10, number of times to retry before marking container unhealthy), and startPeriod (0-300, optional grace period to wait, in seconds, before failed healthchecks count toward retries) **Type:** ```hcl object({ command = list(string) interval = optional(number) retries = optional(number) startPeriod = optional(number) timeout = optional(number) }) ``` **Default value:** `null`
`hostname` (`string`) optional
The hostname to use for your container. **Default value:** `null`
`interactive` (`bool`) optional
When this parameter is true, this allows you to deploy containerized applications that require stdin or a tty to be allocated. **Default value:** `null`
`links` (`list(string)`) optional
List of container names this container can communicate with without port mappings **Default value:** `null`
`linux_parameters` optional
Linux-specific modifications that are applied to the container, such as Linux kernel capabilities. For more details, see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LinuxParameters.html **Type:** ```hcl object({ capabilities = optional(object({ add = optional(list(string)) drop = optional(list(string)) })) devices = optional(list(object({ containerPath = string hostPath = string permissions = optional(list(string)) }))) initProcessEnabled = optional(bool) maxSwap = optional(number) sharedMemorySize = optional(number) swappiness = optional(number) tmpfs = optional(list(object({ containerPath = string mountOptions = optional(list(string)) size = number }))) }) ``` **Default value:** `null`
`log_configuration` optional
Log configuration options to send to a custom log driver for the container. For more details, see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LogConfiguration.html **Type:** ```hcl object({ logDriver = string options = optional(map(string)) secretOptions = optional(list(object({ name = string valueFrom = string }))) }) ``` **Default value:** `null`
`map_environment` (`map(string)`) optional
The environment variables to pass to the container. This is a map of string: \{key: value\}. map_environment overrides environment **Default value:** `null`
`map_secrets` (`map(string)`) optional
The secrets variables to pass to the container. This is a map of string: \{key: value\}. map_secrets overrides secrets **Default value:** `null`
`mount_points` optional
Container mount points. This is a list of maps, where each map should contain `containerPath`, `sourceVolume` and `readOnly` **Type:** ```hcl list(object({ containerPath = optional(string) readOnly = optional(bool) sourceVolume = optional(string) })) ``` **Default value:** `null`
`port_mappings` optional
The port mappings to configure for the container. This is a list of maps. Each map should contain "containerPort", "hostPort", and "protocol", where "protocol" is one of "tcp" or "udp". If using containers in a task with the awsvpc or host network mode, the hostPort can either be left blank or set to the same value as the containerPort **Type:** ```hcl list(object({ containerPort = number hostPort = optional(number) protocol = optional(string) name = optional(string) appProtocol = optional(string) })) ``` **Default value:** `null`
`privileged` (`bool`) optional
When this variable is `true`, the container is given elevated privileges on the host container instance (similar to the root user). This parameter is not supported for Windows containers or tasks using the Fargate launch type. **Default value:** `null`
`pseudo_terminal` (`bool`) optional
When this parameter is true, a TTY is allocated. **Default value:** `null`
`readonly_root_filesystem` (`bool`) optional
Determines whether a container is given read-only access to its root filesystem. Due to how Terraform type casts booleans in json it is required to double quote this value **Default value:** `false`
`repository_credentials` optional
Container repository credentials; required when using a private repo. This map currently supports a single key; "credentialsParameter", which should be the ARN of a Secrets Manager's secret holding the credentials **Type:** ```hcl object({ credentialsParameter = string }) ``` **Default value:** `null`
`resource_requirements` optional
The type and amount of a resource to assign to a container. The only supported resource is a GPU. **Type:** ```hcl list(object({ type = string value = string })) ``` **Default value:** `null`
`restart_policy` optional
The restart policy for a container. When you set up a restart policy, Amazon ECS can restart the container without needing to replace the task. **Type:** ```hcl object({ enabled = bool ignoredExitCodes = optional(list(number)) restartAttemptPeriod = optional(number) }) ``` **Default value:** `null`
`secrets` optional
The secrets to pass to the container. This is a list of maps **Type:** ```hcl list(object({ name = string valueFrom = string })) ``` **Default value:** `null`
`start_timeout` (`number`) optional
Time duration (in seconds) to wait before giving up on resolving dependencies for a container **Default value:** `null`
`stop_timeout` (`number`) optional
Time duration (in seconds) to wait before the container is forcefully killed if it doesn't exit normally on its own **Default value:** `null`
`system_controls` optional
A list of namespaced kernel parameters to set in the container, mapping to the --sysctl option to docker run. This is a list of maps: \{ namespace = "", value = ""\} **Type:** ```hcl list(object({ namespace = string value = string })) ``` **Default value:** `null`
`ulimits` optional
Container ulimit settings. This is a list of maps, where each map should contain "name", "hardLimit" and "softLimit" **Type:** ```hcl list(object({ hardLimit = number name = string softLimit = number })) ``` **Default value:** `null`
`user` (`string`) optional
The user to run as inside the container. Can be any of these formats: user, user:group, uid, uid:gid, user:gid, uid:group. The default (null) will use the container's configured `USER` directive or root if not set. **Default value:** `null`
`version_consistency` (`string`) optional
Specifies whether Amazon ECS will resolve the container image tag provided in the container definition to an image digest. **Default value:** `null`
`volumes_from` optional
A list of VolumesFrom maps which contain "sourceContainer" (name of the container that has the volumes to mount) and "readOnly" (whether the container can write to the volume) **Type:** ```hcl list(object({ readOnly = optional(bool) sourceContainer = string })) ``` **Default value:** `null`
`working_directory` (`string`) optional
The working directory to run commands inside the container **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`environment` optional
The environment variables to pass to the container. This is a list of maps. map_environment overrides environment **Required:** No **Type:** ```hcl list(object({ name = string value = string })) ``` **Default value:** `null`
## Outputs
`json_map_encoded`
JSON string encoded container definitions for use with other terraform resources such as aws_ecs_task_definition
`json_map_encoded_list`
JSON string encoded list of container definitions for use with other terraform resources such as aws_ecs_task_definition
`json_map_object`
JSON map encoded container definition
`sensitive_json_map_encoded`
JSON string encoded container definitions for use with other terraform resources such as aws_ecs_task_definition (sensitive)
`sensitive_json_map_encoded_list`
JSON string encoded list of container definitions for use with other terraform resources such as aws_ecs_task_definition (sensitive)
`sensitive_json_map_object`
JSON map encoded container definition (sensitive)
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `local`, version: `>= 1.2` --- ## ecs-web-app # Module: `ecs-web-app` A Terraform module which implements a web app on ECS and supporting AWS resources. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which test and deploy the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/main/test). Other examples: - [without authentication](https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/main/examples/without_authentication) - without authentication - [with Google OIDC authentication](https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/main/examples/with_google_oidc_authentication) - with Google OIDC authentication - [with Cognito authentication](https://github.com/cloudposse/terraform-aws-ecs-web-app/tree/main/examples/with_cognito_authentication) - with Cognito authentication ``` module "default_backend_web_app" { source = "cloudposse/ecs-web-app/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "testing" name = "appname" vpc_id = module.vpc.vpc_id alb_ingress_unauthenticated_listener_arns = module.alb.listener_arns alb_ingress_unauthenticated_listener_arns_count = 1 aws_logs_region = "us-east-2" ecs_cluster_arn = aws_ecs_cluster.default.arn ecs_cluster_name = aws_ecs_cluster.default.name ecs_security_group_ids = [module.vpc.vpc_default_security_group_id] ecs_private_subnet_ids = module.subnets.private_subnet_ids alb_ingress_healthcheck_path = "/healthz" alb_ingress_unauthenticated_paths = ["/*"] codepipeline_enabled = false container_environment = [ { name = "COOKIE" value = "cookiemonster" }, { name = "PORT" value = "80" } ] } ``` ## Variables ### Required Variables
`alb_security_group` (`string`) required
Security group of the ALB
`ecs_cluster_arn` (`string`) required
The ECS Cluster ARN where ECS Service will be provisioned
`ecs_private_subnet_ids` (`list(string)`) required
List of Private Subnet IDs to provision ECS Service onto if `var.network_mode = "awsvpc"`
`vpc_id` (`string`) required
The VPC ID where resources are created
### Optional Variables
`additional_lbs` optional
List of additional load balancer configurations. Each config should specify container_name (optional), container_port (optional, defaults to main container_port), and target_group_arn **Type:** ```hcl list(object({ container_name = optional(string) container_port = optional(number) target_group_arn = string })) ``` **Default value:** `[ ]`
`alb_arn_suffix` (`string`) optional
ARN suffix of the ALB for the Target Group **Default value:** `""`
`alb_container_name` (`string`) optional
The name of the container to associate with the ALB. If not provided, the generated container will be used **Default value:** `null`
`alb_ingress_authenticated_hosts` (`list(string)`) optional
Authenticated hosts to match in Hosts header **Default value:** `[ ]`
`alb_ingress_authenticated_listener_arns` (`list(string)`) optional
A list of authenticated ALB listener ARNs to attach ALB listener rules to **Default value:** `[ ]`
`alb_ingress_authenticated_listener_arns_count` (`number`) optional
The number of authenticated ARNs in `alb_ingress_authenticated_listener_arns`. This is necessary to work around a limitation in Terraform where counts cannot be computed **Default value:** `0`
`alb_ingress_authenticated_paths` (`list(string)`) optional
Authenticated path pattern to match (a maximum of 1 can be defined) **Default value:** `[ ]`
`alb_ingress_enable_default_target_group` (`bool`) optional
If true, create a default target group for the ALB ingress **Default value:** `true`
`alb_ingress_health_check_healthy_threshold` (`number`) optional
The number of consecutive health checks successes required before healthy **Default value:** `2`
`alb_ingress_health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `15`
`alb_ingress_health_check_matcher` (`string`) optional
The HTTP response codes to indicate a healthy check **Default value:** `"200-399"`
`alb_ingress_health_check_timeout` (`number`) optional
The amount of time to wait in seconds before failing a health check request **Default value:** `10`
`alb_ingress_health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before unhealthy **Default value:** `2`
`alb_ingress_healthcheck_path` (`string`) optional
The path of the healthcheck which the ALB checks **Default value:** `"/"`
`alb_ingress_healthcheck_protocol` (`string`) optional
The protocol to use to connect with the target. Defaults to `HTTP`. Not applicable when `target_type` is `lambda` **Default value:** `"HTTP"`
`alb_ingress_listener_authenticated_priority` (`number`) optional
The priority for the rules with authentication, between 1 and 50000 (1 being highest priority). Must be different from `alb_ingress_listener_unauthenticated_priority` since a listener can't have multiple rules with the same priority **Default value:** `300`
`alb_ingress_listener_unauthenticated_priority` (`number`) optional
The priority for the rules without authentication, between 1 and 50000 (1 being highest priority). Must be different from `alb_ingress_listener_authenticated_priority` since a listener can't have multiple rules with the same priority **Default value:** `1000`
`alb_ingress_load_balancing_algorithm_type` (`string`) optional
Determines how the load balancer selects targets when routing requests. Only applicable for Application Load Balancer Target Groups. The value is round_robin or least_outstanding_requests. The default is round_robin. **Default value:** `"round_robin"`
`alb_ingress_protocol` (`string`) optional
The protocol for the created ALB target group (if target_group_arn is not set). One of `HTTP`, `HTTPS`. Defaults to `HTTP`. **Default value:** `"HTTP"`
`alb_ingress_protocol_version` (`string`) optional
The protocol version. One of `HTTP1`, `HTTP2`, `GRPC`. Only applicable when protocol is HTTP or HTTPS. Specify GRPC to send requests to targets using gRPC. Specify HTTP2 to send requests to targets using HTTP/2. The default is `HTTP1`, which sends requests to targets using HTTP/1.1 **Default value:** `"HTTP1"`
`alb_ingress_target_group_arn` (`string`) optional
Existing ALB target group ARN. If provided, set `alb_ingress_enable_default_target_group` to `false` to disable creation of the default target group **Default value:** `""`
`alb_ingress_target_type` (`string`) optional
Target type for the ALB ingress. One of `ip`, `instance`, `lambda` or `container`. Defaults to `ip`, for bridge networking mode should be `instance` **Default value:** `"ip"`
`alb_ingress_unauthenticated_hosts` (`list(string)`) optional
Unauthenticated hosts to match in Hosts header **Default value:** `[ ]`
`alb_ingress_unauthenticated_listener_arns` (`list(string)`) optional
A list of unauthenticated ALB listener ARNs to attach ALB listener rules to **Default value:** `[ ]`
`alb_ingress_unauthenticated_listener_arns_count` (`number`) optional
The number of unauthenticated ARNs in `alb_ingress_unauthenticated_listener_arns`. This is necessary to work around a limitation in Terraform where counts cannot be computed **Default value:** `0`
`alb_ingress_unauthenticated_paths` (`list(string)`) optional
Unauthenticated path pattern to match (a maximum of 1 can be defined) **Default value:** `[ ]`
`alb_stickiness_cookie_duration` (`number`) optional
The time period, in seconds, during which requests from a client should be routed to the same target. After this time period expires, the load balancer-generated cookie is considered stale. The range is 1 second to 1 week (604800 seconds). The default value is 1 day (86400 seconds) **Default value:** `86400`
`alb_stickiness_enabled` (`bool`) optional
Boolean to enable / disable `stickiness`. Default is `true` **Default value:** `true`
`alb_stickiness_type` (`string`) optional
The type of sticky sessions. The only current possible value is `lb_cookie` **Default value:** `"lb_cookie"`
`alb_target_group_alarms_3xx_threshold` (`number`) optional
The maximum number of 3XX HTTPCodes in a given period for ECS Service **Default value:** `25`
`alb_target_group_alarms_4xx_threshold` (`number`) optional
The maximum number of 4XX HTTPCodes in a given period for ECS Service **Default value:** `25`
`alb_target_group_alarms_5xx_threshold` (`number`) optional
The maximum number of 5XX HTTPCodes in a given period for ECS Service **Default value:** `25`
`alb_target_group_alarms_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when ALB Target Group alarms transition into an ALARM state from any other state **Default value:** `[ ]`
`alb_target_group_alarms_enabled` (`bool`) optional
A boolean to enable/disable CloudWatch Alarms for ALB Target metrics **Default value:** `false`
`alb_target_group_alarms_evaluation_periods` (`number`) optional
The number of periods to analyze for ALB CloudWatch Alarms **Default value:** `1`
`alb_target_group_alarms_insufficient_data_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when ALB Target Group alarms transition into an INSUFFICIENT_DATA state from any other state **Default value:** `[ ]`
`alb_target_group_alarms_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to execute when ALB Target Group alarms transition into an OK state from any other state **Default value:** `[ ]`
`alb_target_group_alarms_period` (`number`) optional
The period (in seconds) to analyze for ALB CloudWatch Alarms **Default value:** `300`
`alb_target_group_alarms_response_time_threshold` (`number`) optional
The maximum ALB Target Group response time **Default value:** `0.5`
`assign_public_ip` (`bool`) optional
Assign a public IP address to the ENI (Fargate launch type only). Valid values are `true` or `false`. Default `false` **Default value:** `false`
`authentication_cognito_scope` (`string`) optional
Cognito scope, which should be a space separated string of requested scopes (see https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims) **Default value:** `null`
`authentication_cognito_user_pool_arn` (`string`) optional
Cognito User Pool ARN **Default value:** `""`
`authentication_cognito_user_pool_client_id` (`string`) optional
Cognito User Pool Client ID **Default value:** `""`
`authentication_cognito_user_pool_domain` (`string`) optional
Cognito User Pool Domain. The User Pool Domain should be set to the domain prefix (`xxx`) instead of full domain (https://xxx.auth.us-west-2.amazoncognito.com) **Default value:** `""`
`authentication_oidc_authorization_endpoint` (`string`) optional
OIDC Authorization Endpoint **Default value:** `""`
`authentication_oidc_client_id` (`string`) optional
OIDC Client ID **Default value:** `""`
`authentication_oidc_client_secret` (`string`) optional
OIDC Client Secret **Default value:** `""`
`authentication_oidc_issuer` (`string`) optional
OIDC Issuer **Default value:** `""`
`authentication_oidc_scope` (`string`) optional
OIDC scope, which should be a space separated string of requested scopes (see https://openid.net/specs/openid-connect-core-1_0.html#ScopeClaims, and https://developers.google.com/identity/protocols/oauth2/openid-connect#scope-param for an example set of scopes when using Google as the IdP) **Default value:** `null`
`authentication_oidc_token_endpoint` (`string`) optional
OIDC Token Endpoint **Default value:** `""`
`authentication_oidc_user_info_endpoint` (`string`) optional
OIDC User Info Endpoint **Default value:** `""`
`authentication_type` (`string`) optional
Authentication type. Supported values are `COGNITO` and `OIDC` **Default value:** `""`
`autoscaling_dimension` (`string`) optional
Dimension to autoscale on (valid options: cpu, memory) **Default value:** `"memory"`
`autoscaling_enabled` (`bool`) optional
A boolean to enable/disable Autoscaling policy for ECS Service **Default value:** `false`
`autoscaling_max_capacity` (`number`) optional
Maximum number of running instances of a Service **Default value:** `2`
`autoscaling_min_capacity` (`number`) optional
Minimum number of running instances of a Service **Default value:** `1`
`autoscaling_scale_down_adjustment` (`number`) optional
Scaling adjustment to make during scale down event **Default value:** `-1`
`autoscaling_scale_down_cooldown` (`number`) optional
Period (in seconds) to wait between scale down events **Default value:** `300`
`autoscaling_scale_up_adjustment` (`number`) optional
Scaling adjustment to make during scale up event **Default value:** `1`
`autoscaling_scale_up_cooldown` (`number`) optional
Period (in seconds) to wait between scale up events **Default value:** `60`
`aws_logs_prefix` (`string`) optional
Custom AWS Logs prefix. If empty name from label module will be used **Default value:** `""`
`aws_logs_region` (`string`) optional
The region for the AWS Cloudwatch Logs group **Default value:** `null`
`badge_enabled` (`bool`) optional
Generates a publicly-accessible URL for the projects build badge. Available as badge_url attribute when enabled **Default value:** `false`
`branch` (`string`) optional
Branch of the GitHub repository, e.g. `master` **Default value:** `""`
`build_environment_variables` optional
A list of maps, that contain the keys 'name', 'value', and 'type' to be used as additional environment variables for the build. Valid types are 'PLAINTEXT', 'PARAMETER_STORE', or 'SECRETS_MANAGER' **Type:** ```hcl list(object( { name = string value = string type = string })) ``` **Default value:** `[ ]`
`build_image` (`string`) optional
Docker image for build environment, _e.g._ `aws/codebuild/docker:docker:17.09.0` **Default value:** `"aws/codebuild/docker:17.09.0"`
`build_timeout` (`number`) optional
How long in minutes, from 5 to 480 (8 hours), for AWS CodeBuild to wait until timing out any related build that does not get marked as completed **Default value:** `60`
`buildspec` (`string`) optional
Declaration to use for building the project. [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `""`
`capacity_provider_strategies` optional
The capacity provider strategies to use for the service. See `capacity_provider_strategy` configuration block: https://www.terraform.io/docs/providers/aws/r/ecs_service.html#capacity_provider_strategy **Type:** ```hcl list(object({ capacity_provider = string weight = number base = number })) ``` **Default value:** `[ ]`
`circuit_breaker_deployment_enabled` (`bool`) optional
If `true`, enable the deployment circuit breaker logic for the service **Default value:** `false`
`circuit_breaker_rollback_enabled` (`bool`) optional
If `true`, Amazon ECS will roll back the service if a service deployment fails **Default value:** `false`
`cloudwatch_log_group_enabled` (`bool`) optional
A boolean to disable cloudwatch log group creation **Default value:** `true`
`codebuild_cache_type` (`string`) optional
The type of storage that will be used for the AWS CodeBuild project cache. Valid values: NO_CACHE, LOCAL, and S3. Defaults to NO_CACHE. If cache_type is S3, it will create an S3 bucket for storing codebuild cache inside **Default value:** `"S3"`
`codepipeline_build_cache_bucket_suffix_enabled` (`bool`) optional
The codebuild cache bucket generates a random 13 character string to generate a unique bucket name. If set to false it uses terraform-null-label's id value. It only works when cache_type is 'S3' **Default value:** `true`
`codepipeline_build_compute_type` (`string`) optional
`CodeBuild` instance size. Possible values are: `BUILD_GENERAL1_SMALL` `BUILD_GENERAL1_MEDIUM` `BUILD_GENERAL1_LARGE` **Default value:** `"BUILD_GENERAL1_SMALL"`
`codepipeline_cdn_bucket_buildspec_identifier` (`string`) optional
Identifier for buildspec section controlling the optional CDN asset deployment. **Default value:** `null`
`codepipeline_cdn_bucket_encryption_enabled` (`bool`) optional
If set to true, enable encryption on the optional CDN asset deployment bucket **Default value:** `false`
`codepipeline_cdn_bucket_id` (`string`) optional
Optional bucket for static asset deployment. If specified, the buildspec must include a secondary artifacts section which controls the files deployed to the bucket [For more info](http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html) **Default value:** `null`
`codepipeline_enabled` (`bool`) optional
A boolean to enable/disable AWS Codepipeline. If `false`, use `ecr_enabled` to control if AWS ECR stays enabled. **Default value:** `true`
`codepipeline_s3_bucket_force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the CodePipeline artifact store S3 bucket so that the bucket can be destroyed without error **Default value:** `false`
`command` (`list(string)`) optional
The command that is passed to the container **Default value:** `null`
`container_cpu` (`number`) optional
The vCPU setting to control cpu limits of container. (If FARGATE launch type is used below, this must be a supported vCPU size from the table here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html) **Default value:** `256`
`container_definition` (`string`) optional
Override the main container_definition **Default value:** `""`
`container_environment` optional
The environment variables to pass to the container. This is a list of maps **Type:** ```hcl list(object({ name = string value = string })) ``` **Default value:** `null`
`container_image` (`string`) optional
The default container image to use in container definition **Default value:** `"cloudposse/default-backend"`
`container_memory` (`number`) optional
The amount of RAM to allow container to use in MB. (If FARGATE launch type is used below, this must be a supported Memory size from the table here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-cpu-memory-error.html) **Default value:** `512`
`container_memory_reservation` (`number`) optional
The amount of RAM (Soft Limit) to allow container to use in MB. This value must be less than `container_memory` if set **Default value:** `128`
`container_port` (`number`) optional
The port number on the container bound to assigned host_port **Default value:** `80`
`container_repo_credentials` (`map(string)`) optional
Container repository credentials; required when using a private repo. This map currently supports a single key; "credentialsParameter", which should be the ARN of a Secrets Manager's secret holding the credentials **Default value:** `null`
`container_start_timeout` (`number`) optional
Time duration (in seconds) to wait before giving up on resolving dependencies for a container **Default value:** `30`
`container_stop_timeout` (`number`) optional
Time duration (in seconds) to wait before the container is forcefully killed if it doesn't exit normally on its own **Default value:** `30`
`deployment_controller_type` (`string`) optional
Type of deployment controller. Valid values are CODE_DEPLOY and ECS **Default value:** `"ECS"`
`deployment_maximum_percent` (`number`) optional
The upper limit of the number of tasks (as a percentage of `desired_count`) that can be running in a service during a deployment **Default value:** `200`
`deployment_minimum_healthy_percent` (`number`) optional
The lower limit (as a percentage of `desired_count`) of the number of tasks that must remain running and healthy in a service during a deployment **Default value:** `100`
`desired_count` (`number`) optional
The desired number of tasks to start with. Set this to 0 if using DAEMON Service type. (FARGATE does not suppoert DAEMON Service type) **Default value:** `1`
`docker_labels` (`map(string)`) optional
Map of Docker labels to assign to the container **Default value:** `null`
`ecr_enabled` (`bool`) optional
A boolean to enable/disable AWS ECR **Default value:** `true`
`ecr_image_tag_mutability` (`string`) optional
The tag mutability setting for the ecr repository. Must be one of: `MUTABLE` or `IMMUTABLE` **Default value:** `"IMMUTABLE"`
`ecr_scan_images_on_push` (`bool`) optional
Indicates whether images are scanned after being pushed to the repository (true) or not (false) **Default value:** `false`
`ecs_alarms_cpu_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High Alarm action **Default value:** `[ ]`
`ecs_alarms_cpu_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`ecs_alarms_cpu_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization High OK action **Default value:** `[ ]`
`ecs_alarms_cpu_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`ecs_alarms_cpu_utilization_high_threshold` (`number`) optional
The maximum percentage of CPU utilization average **Default value:** `80`
`ecs_alarms_cpu_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low Alarm action **Default value:** `[ ]`
`ecs_alarms_cpu_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`ecs_alarms_cpu_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on CPU Utilization Low OK action **Default value:** `[ ]`
`ecs_alarms_cpu_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`ecs_alarms_cpu_utilization_low_threshold` (`number`) optional
The minimum percentage of CPU utilization average **Default value:** `20`
`ecs_alarms_enabled` (`bool`) optional
A boolean to enable/disable CloudWatch Alarms for ECS Service metrics **Default value:** `false`
`ecs_alarms_memory_utilization_high_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High Alarm action **Default value:** `[ ]`
`ecs_alarms_memory_utilization_high_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`ecs_alarms_memory_utilization_high_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization High OK action **Default value:** `[ ]`
`ecs_alarms_memory_utilization_high_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`ecs_alarms_memory_utilization_high_threshold` (`number`) optional
The maximum percentage of Memory utilization average **Default value:** `80`
`ecs_alarms_memory_utilization_low_alarm_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low Alarm action **Default value:** `[ ]`
`ecs_alarms_memory_utilization_low_evaluation_periods` (`number`) optional
Number of periods to evaluate for the alarm **Default value:** `1`
`ecs_alarms_memory_utilization_low_ok_actions` (`list(string)`) optional
A list of ARNs (i.e. SNS Topic ARN) to notify on Memory Utilization Low OK action **Default value:** `[ ]`
`ecs_alarms_memory_utilization_low_period` (`number`) optional
Duration in seconds to evaluate for the alarm **Default value:** `300`
`ecs_alarms_memory_utilization_low_threshold` (`number`) optional
The minimum percentage of Memory utilization average **Default value:** `20`
`ecs_cluster_name` (`string`) optional
The ECS Cluster Name to use in ECS Code Pipeline Deployment step **Default value:** `null`
`ecs_security_group_enabled` (`bool`) optional
Whether to create a security group for the service. **Default value:** `true`
`ecs_security_group_ids` (`list(string)`) optional
Additional Security Group IDs to allow into ECS Service if `var.network_mode = "awsvpc"` **Default value:** `[ ]`
`enable_all_egress_rule` (`bool`) optional
A flag to enable/disable adding the all ports egress rule to the ECS security group **Default value:** `true`
`enable_ecs_managed_tags` (`bool`) optional
Specifies whether to enable Amazon ECS managed tags for the tasks within the service **Default value:** `false`
`entrypoint` (`list(string)`) optional
The entry point that is passed to the container **Default value:** `null`
`exec_enabled` (`bool`) optional
Specifies whether to enable Amazon ECS Exec for the tasks within the service **Default value:** `false`
`force_new_deployment` (`bool`) optional
Enable to force a new task deployment of the service. **Default value:** `false`
`github_oauth_token` (`string`) optional
GitHub Oauth Token with permissions to access private repositories **Default value:** `""`
`github_webhook_events` (`list(string)`) optional
A list of events which should trigger the webhook. See a list of [available events](https://developer.github.com/v3/activity/events/types/) **Default value:** ```hcl [ "push" ] ```
`health_check_grace_period_seconds` (`number`) optional
Seconds to ignore failing load balancer health checks on newly instantiated tasks to prevent premature shutdown, up to 7200. Only valid for services configured to use load balancers **Default value:** `0`
`healthcheck` optional
A map containing command (string), timeout, interval (duration in seconds), retries (1-10, number of times to retry before marking container unhealthy), and startPeriod (0-300, optional grace period to wait, in seconds, before failed healthchecks count toward retries) **Type:** ```hcl object({ command = list(string) retries = number timeout = number interval = number startPeriod = number }) ``` **Default value:** `null`
`ignore_changes_desired_count` (`bool`) optional
Whether to ignore changes for desired count in the ECS service **Default value:** `false`
`ignore_changes_task_definition` (`bool`) optional
Ignore changes (like environment variables) to the ECS task definition **Default value:** `true`
`init_containers` optional
A list of additional init containers to start. The map contains the container_definition (JSON) and the main container's dependency condition (string) on the init container. The latter can be one of START, COMPLETE, SUCCESS, HEALTHY, or null. If null, the init container will not be added to the depends_on list of the main container. **Type:** ```hcl list(object({ container_definition = any condition = string })) ``` **Default value:** `[ ]`
`launch_type` (`string`) optional
The ECS launch type (valid options: FARGATE or EC2) **Default value:** `"FARGATE"`
`linux_parameters` optional
Linux-specific modifications that are applied to the container, such as Linux kernel capabilities. For more details, see https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LinuxParameters.html **Type:** ```hcl object({ capabilities = optional(object({ add = optional(list(string)) drop = optional(list(string)) })) devices = optional(list(object({ containerPath = optional(string) hostPath = optional(string) permissions = optional(list(string)) }))) initProcessEnabled = optional(bool) maxSwap = optional(number) sharedMemorySize = optional(number) swappiness = optional(number) tmpfs = optional(list(object({ containerPath = optional(string) mountOptions = optional(list(string)) size = number }))) }) ``` **Default value:** `{ }`
`log_driver` (`string`) optional
The log driver to use for the container. If using Fargate launch type, only supported value is awslogs **Default value:** `"awslogs"`
`log_retention_in_days` (`number`) optional
The number of days to retain logs for the log group **Default value:** `90`
`map_container_environment` (`map(string)`) optional
The environment variables to pass to the container. This is a map of string: \{key: value\}. `environment` overrides `map_environment` **Default value:** `null`
`mount_points` optional
Container mount points. This is a list of maps, where each map should contain a `containerPath` and `sourceVolume` **Type:** ```hcl list(object({ containerPath = string sourceVolume = string readOnly = bool })) ``` **Default value:** `[ ]`
`network_mode` (`string`) optional
The network mode to use for the task. This is required to be `awsvpc` for `FARGATE` `launch_type` or `null` for `EC2` `launch_type` **Default value:** `"awsvpc"`
`nlb_cidr_blocks` (`list(string)`) optional
A list of CIDR blocks to add to the ingress rule for the NLB container port **Default value:** `[ ]`
`nlb_container_name` (`string`) optional
The name of the container to associate with the NLB. If not provided, the generated container will be used **Default value:** `null`
`nlb_container_port` (`number`) optional
The port number on the container bound to assigned NLB host_port **Default value:** `80`
`nlb_ingress_target_group_arn` (`string`) optional
Target group ARN of the NLB ingress **Default value:** `""`
`permissions_boundary` (`string`) optional
A permissions boundary ARN to apply to the 3 roles that are created. **Default value:** `""`
`platform_version` (`string`) optional
The platform version on which to run your service. Only applicable for launch_type set to FARGATE. More information about Fargate platform versions can be found in the AWS ECS User Guide. **Default value:** `"LATEST"`
`poll_source_changes` (`bool`) optional
Periodically check the location of your source content and run the pipeline if changes are detected **Default value:** `false`
`port_mappings` optional
The port mappings to configure for the container. This is a list of maps. Each map should contain "containerPort", "hostPort", and "protocol", where "protocol" is one of "tcp" or "udp". If using containers in a task with the awsvpc or host network mode, the hostPort can either be left blank or set to the same value as the containerPort **Type:** ```hcl list(object({ containerPort = number hostPort = number protocol = string })) ``` **Default value:** ```hcl [ { "containerPort": 80, "hostPort": 80, "protocol": "tcp" } ] ```
`privileged` (`string`) optional
When this variable is `true`, the container is given elevated privileges on the host container instance (similar to the root user). This parameter is not supported for Windows containers or tasks using the Fargate launch type. Due to how Terraform type casts booleans in json it is required to double quote this value **Default value:** `null`
`propagate_tags` (`string`) optional
Specifies whether to propagate the tags from the task definition or the service to the tasks. The valid values are SERVICE and TASK_DEFINITION **Default value:** `null`
`region` (`string`) optional
AWS Region for S3 bucket **Default value:** `null`
`repo_name` (`string`) optional
GitHub repository name of the application to be built and deployed to ECS **Default value:** `""`
`repo_owner` (`string`) optional
GitHub Organization or Username **Default value:** `""`
`runtime_platform` (`list(map(string))`) optional
Zero or one runtime platform configurations that containers in your task may use. Map of strings with optional keys `operating_system_family` and `cpu_architecture`. See `runtime_platform` docs https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_task_definition#runtime_platform **Default value:** `[ ]`
`secrets` optional
The secrets to pass to the container. This is a list of maps **Type:** ```hcl list(object({ name = string valueFrom = string })) ``` **Default value:** `null`
`service_registries` optional
The service discovery registries for the service. The maximum number of service_registries blocks is 1. The currently supported service registry is Amazon Route 53 Auto Naming Service - `aws_service_discovery_service`; see `service_registries` docs https://www.terraform.io/docs/providers/aws/r/ecs_service.html#service_registries-1 **Type:** ```hcl list(object({ registry_arn = string port = optional(number) container_name = optional(string) container_port = optional(number) })) ``` **Default value:** `[ ]`
`system_controls` (`list(map(string))`) optional
A list of namespaced kernel parameters to set in the container, mapping to the --sysctl option to docker run. This is a list of maps: \{ namespace = "", value = ""\} **Default value:** `null`
`task_cpu` (`number`) optional
The number of CPU units used by the task. If unspecified, it will default to `container_cpu`. If using `FARGATE` launch type `task_cpu` must match supported memory values (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size) **Default value:** `null`
`task_memory` (`number`) optional
The amount of memory (in MiB) used by the task. If unspecified, it will default to `container_memory`. If using Fargate launch type `task_memory` must match supported cpu value (https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#task_size) **Default value:** `null`
`task_policy_arns` (`list(string)`) optional
A list of IAM Policy ARNs to attach to the generated task role. **Default value:** `[ ]`
`task_role_arn` (`string`) optional
The ARN of IAM role that allows your Amazon ECS container task to make calls to other AWS services **Default value:** `""`
`ulimits` optional
The ulimits to configure for the container. This is a list of maps. Each map should contain "name", "softLimit" and "hardLimit" **Type:** ```hcl list(object({ name = string softLimit = number hardLimit = number })) ``` **Default value:** `[ ]`
`use_alb_security_group` (`bool`) optional
A boolean to enable adding an ALB security group rule for the service task **Default value:** `false`
`use_ecr_image` (`bool`) optional
If true, use ECR repo URL for image, otherwise use value in container_image **Default value:** `false`
`use_nlb_cidr_blocks` (`bool`) optional
A flag to enable/disable adding the NLB ingress rule to the security group **Default value:** `false`
`volumes` optional
Task volume definitions as list of configuration objects **Type:** ```hcl list(object({ host_path = string name = string docker_volume_configuration = list(object({ autoprovision = bool driver = string driver_opts = map(string) labels = map(string) scope = string })) efs_volume_configuration = list(object({ file_system_id = string root_directory = string transit_encryption = string transit_encryption_port = string authorization_config = list(object({ access_point_id = string iam = string })) })) })) ``` **Default value:** `[ ]`
`webhook_authentication` (`string`) optional
The type of authentication to use. One of IP, GITHUB_HMAC, or UNAUTHENTICATED **Default value:** `"GITHUB_HMAC"`
`webhook_enabled` (`bool`) optional
Set to false to prevent the module from creating any webhook resources **Default value:** `true`
`webhook_filter_json_path` (`string`) optional
The JSON path to filter on **Default value:** `"$.ref"`
`webhook_filter_match_equals` (`string`) optional
The value to match on (e.g. refs/heads/\{Branch\}) **Default value:** `"refs/heads/{Branch}"`
`webhook_target_action` (`string`) optional
The name of the action in a pipeline you want to connect to the webhook. The action must be from the source (first) stage of the pipeline **Default value:** `"Source"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`alb_ingress`
All outputs from `module.alb_ingress`
`alb_ingress_target_group_arn`
ALB Target Group ARN
`alb_ingress_target_group_arn_suffix`
ALB Target Group ARN suffix
`alb_ingress_target_group_name`
ALB Target Group name
`alb_target_group_cloudwatch_sns_alarms`
All outputs from `module.alb_target_group_cloudwatch_sns_alarms`
`cloudwatch_log_group`
All outputs from `aws_cloudwatch_log_group.app`
`cloudwatch_log_group_arn`
Cloudwatch log group ARN
`cloudwatch_log_group_name`
Cloudwatch log group name
`codebuild`
All outputs from `module.ecs_codepipeline`
`codebuild_badge_url`
The URL of the build badge when badge_enabled is enabled
`codebuild_cache_bucket_arn`
CodeBuild cache S3 bucket ARN
`codebuild_cache_bucket_name`
CodeBuild cache S3 bucket name
`codebuild_project_id`
CodeBuild project ID
`codebuild_project_name`
CodeBuild project name
`codebuild_role_arn`
CodeBuild IAM Role ARN
`codebuild_role_id`
CodeBuild IAM Role ID
`codepipeline_arn`
CodePipeline ARN
`codepipeline_id`
CodePipeline ID
`codepipeline_webhook_id`
The CodePipeline webhook's ID
`codepipeline_webhook_url`
The CodePipeline webhook's URL. POST events to this endpoint to trigger the target
`container_definition`
All outputs from `module.container_definition`
`container_definition_json`
JSON encoded list of container definitions for use with other terraform resources such as aws_ecs_task_definition
`container_definition_json_map`
JSON encoded container definitions for use with other terraform resources such as aws_ecs_task_definition
`ecr`
All outputs from `module.ecr`
`ecr_registry_id`
Registry ID
`ecr_registry_url`
Repository URL
`ecr_repository_arn`
ARN of ECR repository
`ecr_repository_name`
Registry name
`ecr_repository_url`
Repository URL
`ecs_alarms`
All outputs from `module.ecs_cloudwatch_sns_alarms`
`ecs_alarms_cpu_utilization_high_cloudwatch_metric_alarm_arn`
ECS CPU utilization high CloudWatch metric alarm ARN
`ecs_alarms_cpu_utilization_high_cloudwatch_metric_alarm_id`
ECS CPU utilization high CloudWatch metric alarm ID
`ecs_alarms_cpu_utilization_low_cloudwatch_metric_alarm_arn`
ECS CPU utilization low CloudWatch metric alarm ARN
`ecs_alarms_cpu_utilization_low_cloudwatch_metric_alarm_id`
ECS CPU utilization low CloudWatch metric alarm ID
`ecs_alarms_memory_utilization_high_cloudwatch_metric_alarm_arn`
ECS Memory utilization high CloudWatch metric alarm ARN
`ecs_alarms_memory_utilization_high_cloudwatch_metric_alarm_id`
ECS Memory utilization high CloudWatch metric alarm ID
`ecs_alarms_memory_utilization_low_cloudwatch_metric_alarm_arn`
ECS Memory utilization low CloudWatch metric alarm ARN
`ecs_alarms_memory_utilization_low_cloudwatch_metric_alarm_id`
ECS Memory utilization low CloudWatch metric alarm ID
`ecs_alb_service_task`
All outputs from `module.ecs_alb_service_task`
`ecs_cloudwatch_autoscaling`
All outputs from `module.ecs_cloudwatch_autoscaling`
`ecs_cloudwatch_autoscaling_scale_down_policy_arn`
ARN of the scale down policy
`ecs_cloudwatch_autoscaling_scale_up_policy_arn`
ARN of the scale up policy
`ecs_exec_role_policy_id`
The ECS service role policy ID, in the form of `role_name:role_policy_name`
`ecs_exec_role_policy_name`
ECS service role name
`ecs_service_arn`
ECS Service ARN
`ecs_service_name`
ECS Service name
`ecs_service_role_arn`
ECS Service role ARN
`ecs_service_security_group_id`
Security Group ID of the ECS task
`ecs_task_definition_family`
ECS task definition family
`ecs_task_definition_revision`
ECS task definition revision
`ecs_task_exec_role_arn`
ECS Task exec role ARN
`ecs_task_exec_role_name`
ECS Task role name
`ecs_task_role_arn`
ECS Task role ARN
`ecs_task_role_id`
ECS Task role id
`ecs_task_role_name`
ECS Task role name
`httpcode_elb_5xx_count_cloudwatch_metric_alarm_arn`
ALB 5xx count CloudWatch metric alarm ARN
`httpcode_elb_5xx_count_cloudwatch_metric_alarm_id`
ALB 5xx count CloudWatch metric alarm ID
`httpcode_target_3xx_count_cloudwatch_metric_alarm_arn`
ALB Target Group 3xx count CloudWatch metric alarm ARN
`httpcode_target_3xx_count_cloudwatch_metric_alarm_id`
ALB Target Group 3xx count CloudWatch metric alarm ID
`httpcode_target_4xx_count_cloudwatch_metric_alarm_arn`
ALB Target Group 4xx count CloudWatch metric alarm ARN
`httpcode_target_4xx_count_cloudwatch_metric_alarm_id`
ALB Target Group 4xx count CloudWatch metric alarm ID
`httpcode_target_5xx_count_cloudwatch_metric_alarm_arn`
ALB Target Group 5xx count CloudWatch metric alarm ARN
`httpcode_target_5xx_count_cloudwatch_metric_alarm_id`
ALB Target Group 5xx count CloudWatch metric alarm ID
`target_response_time_average_cloudwatch_metric_alarm_arn`
ALB Target Group response time average CloudWatch metric alarm ARN
`target_response_time_average_cloudwatch_metric_alarm_id`
ALB Target Group response time average CloudWatch metric alarm ID
## Dependencies ### Requirements - `terraform`, version: `>= 1` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alb_ingress` | 0.28.0 | [`cloudposse/alb-ingress/aws`](https://registry.terraform.io/modules/cloudposse/alb-ingress/aws/0.28.0) | n/a `alb_target_group_cloudwatch_sns_alarms` | 0.17.0 | [`cloudposse/alb-target-group-cloudwatch-sns-alarms/aws`](https://registry.terraform.io/modules/cloudposse/alb-target-group-cloudwatch-sns-alarms/aws/0.17.0) | n/a `container_definition` | 0.58.1 | [`cloudposse/ecs-container-definition/aws`](https://registry.terraform.io/modules/cloudposse/ecs-container-definition/aws/0.58.1) | n/a `ecr` | 0.41.0 | [`cloudposse/ecr/aws`](https://registry.terraform.io/modules/cloudposse/ecr/aws/0.41.0) | n/a `ecs_alb_service_task` | 0.78.0 | [`cloudposse/ecs-alb-service-task/aws`](https://registry.terraform.io/modules/cloudposse/ecs-alb-service-task/aws/0.78.0) | n/a `ecs_cloudwatch_autoscaling` | 0.7.5 | [`cloudposse/ecs-cloudwatch-autoscaling/aws`](https://registry.terraform.io/modules/cloudposse/ecs-cloudwatch-autoscaling/aws/0.7.5) | n/a `ecs_cloudwatch_sns_alarms` | 0.12.2 | [`cloudposse/ecs-cloudwatch-sns-alarms/aws`](https://registry.terraform.io/modules/cloudposse/ecs-cloudwatch-sns-alarms/aws/0.12.2) | n/a `ecs_codepipeline` | 0.34.1 | [`cloudposse/ecs-codepipeline/aws`](https://registry.terraform.io/modules/cloudposse/ecs-codepipeline/aws/0.34.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.app`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) ## Data Sources The following data sources are used by this module: - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## efs(Efs) # Module: `efs` Terraform module to provision an AWS [`EFS`](https://aws.amazon.com/efs/) Network File System. **NOTE**: Release `0.32.0` contains breaking changes. To preserve the SG, follow the instructions in the [0.30.1 to 0.32.x+ migration path](https://github.com/cloudposse/terraform-aws-efs/tree/main/docs/migration-0.30.1-0.32.x+.md). ## Usage Include this repository as a module in your existing terraform code: ```hcl module "efs" { source = "cloudposse/efs/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" region = "us-west-1" vpc_id = var.vpc_id subnets = var.private_subnets zone_id = [var.aws_route53_dns_zone_id] allowed_security_group_ids = [var.security_group_id] } ``` ## Variables ### Required Variables
`region` (`string`) required
AWS Region
`subnets` (`list(string)`) required
Subnet IDs
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`access_points` (`map(map(map(any)))`) optional
A map of the access points you would like in your EFS volume See [examples/complete] for an example on how to set this up. All keys are strings. The primary keys are the names of access points. The secondary keys are `posix_user` and `creation_info`. The secondary_gids key should be a comma separated value. More information can be found in the terraform resource [efs_access_point](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_access_point). **Default value:** `{ }`
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`allow_all_egress` (`bool`) optional
Passed to the security group module (if one is created). **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
The CIDR blocks from which to allow `ingress` traffic to the EFS **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the EFS Mount Targets with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`availability_zone_name` (`string`) optional
AWS Availability Zone in which to create the file system. Used to create a file system that uses One Zone storage classes. If set, a single subnet in the same availability zone should be provided to `subnets` **Default value:** `null`
`bypass_policy_lockout_safety_check` (`bool`) optional
A flag to indicate whether to bypass the `aws_efs_file_system_policy` lockout safety check. **Default value:** `false`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`dns_name` (`string`) optional
Name of the CNAME record to create **Default value:** `""`
`efs_backup_policy_enabled` (`bool`) optional
If `true`, it will turn on automatic backups. **Default value:** `false`
`efs_file_system_policy` (`string`) optional
EFS policy to attach. **Default value:** `""`
`encrypted` (`bool`) optional
If true, the file system will be encrypted **Default value:** `true`
`kms_key_id` (`string`) optional
If set, use a specific KMS key **Default value:** `null`
`mount_target_ip_address` (`string`) optional
The address (within the address range of the specified subnet) at which the file system may be mounted via the mount target **Default value:** `null`
`performance_mode` (`string`) optional
The file system performance mode. Can be either `generalPurpose` or `maxIO` **Default value:** `"generalPurpose"`
`provisioned_throughput_in_mibps` (`number`) optional
The throughput, measured in MiB/s, that you want to provision for the file system. Only applicable with `throughput_mode` set to provisioned **Default value:** `0`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"EFS Security Group"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`security_groups` (`list(string)`) optional
DEPRECATED: Use `allowed_security_group_ids` instead. A list of Security Group IDs to associate with EFS. **Default value:** `[ ]`
`throughput_mode` (`string`) optional
Throughput mode for the file system. Defaults to bursting. Valid values: `bursting`, `provisioned`, `elastic`. When using `provisioned`, also set `provisioned_throughput_in_mibps` **Default value:** `"bursting"`
`transition_to_archive` (`list(string)`) optional
Indicates how long it takes to transition files to the Glacier storage class. Valid values: AFTER_1_DAY, AFTER_7_DAYS, AFTER_14_DAYS, AFTER_30_DAYS, AFTER_60_DAYS, AFTER_90_DAYS, AFTER_180_DAYS, AFTER_270_DAYS and AFTER_365_DAYS. Default (no value) means "never". **Default value:** `[ ]`
`transition_to_ia` (`list(string)`) optional
Indicates how long it takes to transition files to the Infrequent Access (IA) storage class. Valid values: AFTER_1_DAY, AFTER_7_DAYS, AFTER_14_DAYS, AFTER_30_DAYS, AFTER_60_DAYS, AFTER_90_DAYS, AFTER_180_DAYS, AFTER_270_DAYS and AFTER_365_DAYS. Default (no value) means "never". **Default value:** `[ ]`
`transition_to_primary_storage_class` (`list(string)`) optional
Describes the policy used to transition a file from Infrequent Access (IA) storage to primary storage. Valid values: AFTER_1_ACCESS. **Default value:** `[ ]`
`zone_id` (`list(string)`) optional
Route53 DNS Zone ID as list of string (0 or 1 items). If empty, no custom DNS name will be published. If the list contains a single Zone ID, a custom DNS name will be pulished in that zone. Can also be a plain string, but that use is DEPRECATED because of Terraform issues. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_point_arns`
EFS AP ARNs
`access_point_ids`
EFS AP ids
`arn`
EFS ARN
`dns_name`
EFS DNS name
`host`
Route53 DNS hostname for the EFS
`id`
EFS ID
`mount_target_dns_names`
List of EFS mount target DNS names
`mount_target_ids`
List of EFS mount target IDs (one per Availability Zone)
`mount_target_ips`
List of EFS mount target IPs (one per Availability Zone)
`network_interface_ids`
List of mount target network interface IDs
`security_group_arn`
EFS Security Group ARN
`security_group_id`
EFS Security Group ID
`security_group_name`
EFS Security Group name
## Dependencies ### Requirements - `terraform`, version: `>= 1.1.0` - `aws`, version: `>= 5.32.0` ### Providers - `aws`, version: `>= 5.32.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_efs_access_point.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_access_point) (resource) - [`aws_efs_backup_policy.policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_backup_policy) (resource) - [`aws_efs_file_system.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system) (resource) - [`aws_efs_file_system_policy.policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system_policy) (resource) - [`aws_efs_mount_target.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_mount_target) (resource) ## Data Sources The following data sources are used by this module: --- ## efs-backup # Module: `efs-backup` Terraform module designed to easily backup EFS filesystems to S3 using DataPipeline. The workflow is simple: * Periodically launch resource (EC2 instance) based on schedule * Execute the shell command defined in the activity on the instance * Sync data from Production EFS to S3 Bucket by using `aws-cli` * The execution log of the activity is stored in `S3` * Publish the success or failure of the activity to an `SNS` topic * Automatically rotate the backups using `S3 lifecycle rule` ## Usage Include this module in your existing terraform code: ```hcl module "efs_backup" { source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=master" name = "${var.name}" stage = "${var.stage}" namespace = "${var.namespace}" vpc_id = "${var.vpc_id}" efs_mount_target_id = "${var.efs_mount_target_id}" use_ip_address = "false" noncurrent_version_expiration_days = "${var.noncurrent_version_expiration_days}" ssh_key_pair = "${var.ssh_key_pair}" datapipeline_config = "${var.datapipeline_config}" modify_security_group = "true" } output "efs_backup_security_group" { value = "${module.efs_backup.security_group_id}" } ``` ## Integration with `EFS` To enable connectivity between the `DataPipeline` instances and the `EFS`, use one of the following methods to configure Security Groups: 1. Explicitly add the `DataPipeline` SG (the output of this module `security_group_id`) to the list of the `ingress` rules of the `EFS` SG. For example: ```hcl module "elastic_beanstalk_environment" { source = "git::https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment.git?ref=master" namespace = "${var.namespace}" name = "${var.name}" stage = "${var.stage}" delimiter = "${var.delimiter}" attributes = ["${compact(concat(var.attributes, list("eb-env")))}"] tags = "${var.tags}" # .............................. } module "efs" { source = "git::https://github.com/cloudposse/terraform-aws-efs.git?ref=tmaster" namespace = "${var.namespace}" name = "${var.name}" stage = "${var.stage}" delimiter = "${var.delimiter}" attributes = ["${compact(concat(var.attributes, list("efs")))}"] tags = "${var.tags}" # Allow EB/EC2 instances and DataPipeline instances to connect to the EFS security_groups = ["${module.elastic_beanstalk_environment.security_group_id}", "${module.efs_backup.security_group_id}"] } module "efs_backup" { source = "git::https://github.com/cloudposse/terraform-aws-efs-backup.git?ref=master" name = "${var.name}" stage = "${var.stage}" namespace = "${var.namespace}" delimiter = "${var.delimiter}" attributes = ["${compact(concat(var.attributes, list("efs-backup")))}"] tags = "${var.tags}" # Important to set it to `false` since we added the `DataPipeline` SG (output of the `efs_backup` module) to the `security_groups` of the `efs` module # See NOTE below for more information modify_security_group = "false" # .............................. } ``` 2. Set `modify_security_group` attribute to `true` so the module will modify the `EFS` SG to allow the `DataPipeline` to connect to the `EFS` **NOTE:** Do not mix these two methods together. `Terraform` does not support using a Security Group with in-line rules in conjunction with any Security Group Rule resources. https://www.terraform.io/docs/providers/aws/r/security_group_rule.html > NOTE on Security Groups and Security Group Rules: Terraform currently provides both a standalone Security Group Rule resource (a single ingress or egress rule), and a Security Group resource with ingress and egress rules defined in-line. At this time you cannot use a Security Group with in-line rules in conjunction with any Security Group Rule resources. Doing so will cause a conflict of rule settings and will overwrite rules. ## Variables ### Required Variables
`efs_mount_target_id` (`string`) required
EFS Mount Target ID (e.g. `fsmt-279bfc62`)
`ssh_key_pair` (`string`) required
`SSH` key that will be deployed on DataPipeline's instance
### Optional Variables
`datapipeline_config` (`map(string)`) optional
DataPipeline configuration options **Default value:** ```hcl { "email": "", "instance_type": "t2.micro", "period": "24 hours", "timeout": "60 Minutes" } ```
`datapipeline_security_group` (`string`) optional
Optionally specify a security group to use for the datapipeline instances **Default value:** `""`
`modify_security_group` (`string`) optional
Should the module modify the `EFS` security group **Default value:** `"false"`
`noncurrent_version_expiration_days` (`string`) optional
S3 object versions expiration period (days) **Default value:** `"35"`
`region` (`string`) optional
(Optional) AWS Region. If not specified, will be derived from 'aws_region' data source **Default value:** `""`
`subnet_id` (`string`) optional
Optionally specify the subnet to use **Default value:** `""`
`use_ip_address` (`string`) optional
If set to `true`, will use IP address instead of DNS name to connect to the `EFS` **Default value:** `"false"`
`vpc_id` (`string`) optional
VPC ID **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`any`) required
The Name of the application or solution (e.g. `bastion` or `portal`) **Required:** Yes **Default value:** ``
`namespace` (`any`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`any`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `efs-backup`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage`, etc. **Required:** No **Default value:** `"-"`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`backups_bucket_name`
Backups bucket name
`datapipeline_ids`
Datapipeline ids
`logs_bucket_name`
Logs bucket name
`security_group_id`
Security group id
`sns_topic_arn`
Backup notification SNS topic ARN
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `backups_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `datapipeline_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `logs_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `resource_role_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `role_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a `sns_label` | tags/0.3.1 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.1) | n/a ## Resources The following resources are used by this module: - [`aws_cloudformation_stack.datapipeline`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack) (resource) - [`aws_cloudformation_stack.sns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudformation_stack) (resource) - [`aws_iam_instance_profile.resource_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.resource_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.resource_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.backups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket.logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_security_group.datapipeline`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.datapipeline_efs_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.amazon_linux`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_efs_mount_target.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/efs_mount_target) (data source) - [`aws_iam_policy_document.resource_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_subnet_ids.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet_ids) (data source) - [`aws_vpc.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## efs-cloudwatch-sns-alarms # Module: `efs-cloudwatch-sns-alarms` Create a set of sane EFS CloudWatch alerts for monitoring the health of an EFS resource. | area | metric | comparison operator | threshold | rationale | |---------|--------------------|----------------------|-------------------|--------------------------------------------------------------------| | Storage | BurstCreditBalance | `<` | 192000000000 | 192 GB in Bytes (last hour where you can burst at 100 MB/sec) | | Storage | PercentIOLimit | `>` | 95 | When the IO limit has been exceeded, the system performance drops. | ## Examples ```hcl resource "aws_efs_file_system" "default" { creation_token = "app" } module "efs_alarms" { source = "git::https://github.com/cloudposse/terraform-aws-efs-cloudwatch-sns-alarms.git?ref=master" filesystem_id = "${aws_efs_file_system.default.id}" } ``` ## Variables ### Required Variables
`filesystem_id` (`string`) required
The EFS file system ID that you want to monitor
### Optional Variables
`add_sns_policy` (`string`) optional
Attach a policy that allows the notifications through to the SNS topic endpoint **Default value:** `"false"`
`additional_endpoint_arns` (`list(string)`) optional
Any alert endpoints, such as autoscaling, or app escaling endpoint arns that will respond to an alert **Default value:** `[ ]`
`burst_credit_balance_threshold` (`string`) optional
The minimum number of burst credits that a file system should have. **Default value:** `"192000000000"`
`percent_io_limit_threshold` (`string`) optional
IO Limit threshold **Default value:** `"95"`
`sns_topic_arn` (`string`) optional
An SNS topic ARN that has already been created. Its policy must already allow access from CloudWatch Alarms, or set `add_sns_policy` to `true` **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Outputs
`sns_topic_arn`
An SNS topic ARN that has already been created. Its policy must already allow access from CloudWatch Alarms, or set `add_sns_policy` to `true`
## Dependencies ### Providers - `aws` ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.burst_credit_balance_too_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.percent_io_limit_too_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_sns_topic.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) (resource) - [`aws_sns_topic_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sns_topic_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## eks-cluster # Module: `eks-cluster` Terraform module to provision an [EKS](https://aws.amazon.com/eks/) cluster on AWS. This Terraform module provisions a fully configured AWS [EKS](https://aws.amazon.com/eks/) (Elastic Kubernetes Service) cluster. It's engineered to integrate smoothly with [Karpenter](https://karpenter.sh/) and [EKS addons](https://docs.aws.amazon.com/eks/latest/userguide/eks-add-ons.html), forming a critical part of [Cloud Posse's reference architecture](https://cloudposse.com/reference-architecture). Ideal for teams looking to deploy scalable and manageable Kubernetes clusters on AWS with minimal fuss. ## Introduction The module provisions the following resources: - EKS cluster of master nodes that can be used together with the [terraform-aws-eks-node-group](https://github.com/cloudposse/terraform-aws-eks-node-group) and [terraform-aws-eks-fargate-profile](https://github.com/cloudposse/terraform-aws-eks-fargate-profile) modules to create a full-blown EKS/Kubernetes cluster. You can also use the [terraform-aws-eks-workers](https://github.com/cloudposse/terraform-aws-eks-workers) module to provision worker nodes for the cluster, but it is now rare for that to be a better choice than to use `terraform-aws-eks-node-group`. - IAM Role to allow the cluster to access other AWS services - EKS access entries to allow IAM users to access and administer the cluster ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-eks-cluster/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test/src](https://github.com/cloudposse/terraform-aws-eks-cluster/tree/main/test/src). Other examples: - [terraform-aws-components/eks/cluster](https://github.com/cloudposse/terraform-aws-components/tree/master/modules/eks/cluster) - Cloud Posse's service catalog of "root module" invocations for provisioning reference architectures ```hcl provider "aws" { region = var.region } # Note: This example creates an explicit access entry for the current user, # but in practice, you should use a static map of IAM users or roles that should have access to the cluster. # Granting access to the current user in this way is not recommended for production use. data "aws_caller_identity" "current" {} # IAM session context converts an assumed role ARN into an IAM Role ARN. # Again, this is primarily to simplify the example, and in practice, you should use a static map of IAM users or roles. data "aws_iam_session_context" "current" { arn = data.aws_caller_identity.current.arn } locals { # The usage of the specific kubernetes.io/cluster/* resource tags below are required # for EKS and Kubernetes to discover and manage networking resources # https://aws.amazon.com/premiumsupport/knowledge-center/eks-vpc-subnet-discovery/ # https://github.com/kubernetes-sigs/aws-load-balancer-controller/blob/main/docs/deploy/subnet_discovery.md tags = { "kubernetes.io/cluster/${module.label.id}" = "shared" } # required tags to make ALB ingress work https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html public_subnets_additional_tags = { "kubernetes.io/role/elb" : 1 } private_subnets_additional_tags = { "kubernetes.io/role/internal-elb" : 1 } # Enable the IAM user creating the cluster to administer it, # without using the bootstrap_cluster_creator_admin_permissions option, # as an example of how to use the access_entry_map feature. # In practice, this should be replaced with a static map of IAM users or roles # that should have access to the cluster, but we use the current user # to simplify the example. access_entry_map = { (data.aws_iam_session_context.current.issuer_arn) = { access_policy_associations = { ClusterAdmin = {} } } } } module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace name = var.name stage = var.stage delimiter = var.delimiter tags = var.tags } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.16.0.0/16" tags = local.tags context = module.label.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = true nat_instance_enabled = false public_subnets_additional_tags = local.public_subnets_additional_tags private_subnets_additional_tags = local.private_subnets_additional_tags tags = local.tags context = module.label.context } module "eks_node_group" { source = "cloudposse/eks-node-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" instance_types = [var.instance_type] subnet_ids = module.subnets.private_subnet_ids health_check_type = var.health_check_type min_size = var.min_size max_size = var.max_size cluster_name = module.eks_cluster.eks_cluster_id # Enable the Kubernetes cluster auto-scaler to find the auto-scaling group cluster_autoscaler_enabled = var.autoscaling_policies_enabled context = module.label.context } module "eks_cluster" { source = "cloudposse/eks-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" subnet_ids = concat(module.subnets.private_subnet_ids, module.subnets.public_subnet_ids) kubernetes_version = var.kubernetes_version oidc_provider_enabled = true addons = [ # https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html#vpc-cni-latest-available-version { addon_name = "vpc-cni" addon_version = var.vpc_cni_version resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "OVERWRITE" service_account_role_arn = var.vpc_cni_service_account_role_arn # Creating this role is outside the scope of this example }, # https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html { addon_name = "kube-proxy" addon_version = var.kube_proxy_version resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "OVERWRITE" service_account_role_arn = null }, # https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html { addon_name = "coredns" addon_version = var.coredns_version resolve_conflicts_on_create = "OVERWRITE" resolve_conflicts_on_update = "OVERWRITE" service_account_role_arn = null }, ] addons_depends_on = [module.eks_node_group] context = module.label.context cluster_depends_on = [module.subnets] } ``` Module usage with two unmanaged worker groups: ```hcl locals { # Unfortunately, the `aws_ami` data source attribute `most_recent` (https://github.com/cloudposse/terraform-aws-eks-workers/blob/34a43c25624a6efb3ba5d2770a601d7cb3c0d391/main.tf#L141) # does not work as you might expect. If you are not going to use a custom AMI you should # use the `eks_worker_ami_name_filter` variable to set the right kubernetes version for EKS workers, # otherwise the first version of Kubernetes supported by AWS (v1.11) for EKS workers will be selected, but the # EKS control plane will ignore it to use one that matches the version specified by the `kubernetes_version` variable. eks_worker_ami_name_filter = "amazon-eks-node-${var.kubernetes_version}*" } module "eks_workers" { source = "cloudposse/eks-workers/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = ["small"] instance_type = "t3.small" eks_worker_ami_name_filter = local.eks_worker_ami_name_filter vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids health_check_type = var.health_check_type min_size = var.min_size max_size = var.max_size wait_for_capacity_timeout = var.wait_for_capacity_timeout cluster_name = module.label.id cluster_endpoint = module.eks_cluster.eks_cluster_endpoint cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data cluster_security_group_id = module.eks_cluster.eks_cluster_managed_security_group_id # Auto-scaling policies and CloudWatch metric alarms autoscaling_policies_enabled = var.autoscaling_policies_enabled cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent context = module.label.context } module "eks_workers_2" { source = "cloudposse/eks-workers/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = ["medium"] instance_type = "t3.medium" eks_worker_ami_name_filter = local.eks_worker_ami_name_filter vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids health_check_type = var.health_check_type min_size = var.min_size max_size = var.max_size wait_for_capacity_timeout = var.wait_for_capacity_timeout cluster_name = module.label.id cluster_endpoint = module.eks_cluster.eks_cluster_endpoint cluster_certificate_authority_data = module.eks_cluster.eks_cluster_certificate_authority_data cluster_security_group_id = module.eks_cluster.eks_cluster_managed_security_group_id # Auto-scaling policies and CloudWatch metric alarms autoscaling_policies_enabled = var.autoscaling_policies_enabled cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent context = module.label.context } module "eks_cluster" { source = "cloudposse/eks-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" subnet_ids = concat(module.subnets.private_subnet_ids, module.subnets.public_subnet_ids) kubernetes_version = var.kubernetes_version oidc_provider_enabled = true # needed for VPC CNI access_entries_for_nodes = { EC2_LINUX = [module.eks_workers.workers_role_arn, module.eks_workers_2.workers_role_arn] } context = module.label.context } ``` :::warning Release `4.0.0` contains major breaking changes that will require you to update your existing EKS cluster and configuration to use this module. Please see the [v3 to v4 migration path](https://github.com/cloudposse/terraform-aws-eks-cluster/tree/main/docs/migration-v3-v4.md) for more information. Release `2.0.0` (previously released as version `0.45.0`) contains some changes that, if applied to a cluster created with an earlier version of this module, could result in your existing EKS cluster being replaced (destroyed and recreated). To prevent this, follow the instructions in the [v1 to v2 migration path](https://github.com/cloudposse/terraform-aws-eks-cluster/tree/main/docs/migration-v1-v2.md). ::: :::note Prior to v4 of this module, AWS did not provide an API to manage access to the EKS cluster, causing numerous challenges. With v4 of this module, it exclusively uses the AWS API, resolving many issues you may read about that had affected prior versions. See the version 2 README and release notes for more information on the challenges and workarounds that were required prior to v3. ::: ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
A list of subnet IDs to launch the cluster in
### Optional Variables
`access_config` optional
Access configuration for the EKS cluster. **Type:** ```hcl object({ authentication_mode = optional(string, "API") bootstrap_cluster_creator_admin_permissions = optional(bool, false) }) ``` **Default value:** `{ }`
`access_entries` optional
List of IAM principles to allow to access the EKS cluster. It is recommended to use the default `user_name` because the default includes the IAM role or user name and the session name for assumed roles. Use when Principal ARN is not known at plan time. **Type:** ```hcl list(object({ principal_arn = string user_name = optional(string, null) kubernetes_groups = optional(list(string), null) })) ``` **Default value:** `[ ]`
`access_entries_for_nodes` (`map(list(string))`) optional
Map of list of IAM roles for the EKS non-managed worker nodes. The map key is the node type, either `EC2_LINUX` or `EC2_WINDOWS`, and the list contains the IAM roles of the nodes of that type. There is no need for or utility in creating Fargate access entries, as those are always created automatically by AWS, just as with managed nodes. Use when Principal ARN is not known at plan time. **Default value:** `{ }`
`access_entry_map` optional
Map of IAM Principal ARNs to access configuration. Preferred over other inputs as this configuration remains stable when elements are added or removed, but it requires that the Principal ARNs and Policy ARNs are known at plan time. Can be used along with other `access_*` inputs, but do not duplicate entries. Map `access_policy_associations` keys are policy ARNs, policy full name (AmazonEKSViewPolicy), or short name (View). It is recommended to use the default `user_name` because the default includes IAM role or user name and the session name for assumed roles. As a special case in support of backwards compatibility, membership in the `system:masters` group is is translated to an association with the ClusterAdmin policy. In all other cases, including any `system:*` group in `kubernetes_groups` is prohibited. **Type:** ```hcl map(object({ # key is principal_arn user_name = optional(string) # Cannot assign "system:*" groups to IAM users, use ClusterAdmin and Admin instead kubernetes_groups = optional(list(string), []) type = optional(string, "STANDARD") access_policy_associations = optional(map(object({ # key is policy_arn or policy_name access_scope = optional(object({ type = optional(string, "cluster") namespaces = optional(list(string)) }), {}) # access_scope })), {}) # access_policy_associations })) ``` **Default value:** `{ }`
`access_policy_associations` optional
List of AWS managed EKS access policies to associate with IAM principles. Use when Principal ARN or Policy ARN is not known at plan time. `policy_arn` can be the full ARN, the full name (AmazonEKSViewPolicy) or short name (View). **Type:** ```hcl list(object({ principal_arn = string policy_arn = string access_scope = optional(object({ type = optional(string, "cluster") namespaces = optional(list(string)) }), {}) })) ``` **Default value:** `[ ]`
`addons` optional
Manages [`aws_eks_addon`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) resources. Note: `resolve_conflicts` is deprecated. If `resolve_conflicts` is set and `resolve_conflicts_on_create` or `resolve_conflicts_on_update` is not set, `resolve_conflicts` will be used instead. If `resolve_conflicts_on_create` is not set and `resolve_conflicts` is `PRESERVE`, `resolve_conflicts_on_create` will be set to `NONE`. If `additional_tags` are specified, they are added to the standard resource tags. **Type:** ```hcl list(object({ addon_name = string addon_version = optional(string, null) configuration_values = optional(string, null) # resolve_conflicts is deprecated, but we keep it for backwards compatibility # and because if not declared, Terraform will silently ignore it. resolve_conflicts = optional(string, null) resolve_conflicts_on_create = optional(string, null) resolve_conflicts_on_update = optional(string, null) service_account_role_arn = optional(string, null) pod_identity_association = optional(map(string), {}) create_timeout = optional(string, null) update_timeout = optional(string, null) delete_timeout = optional(string, null) additional_tags = optional(map(string), {}) })) ``` **Default value:** `[ ]`
`addons_depends_on` (`any`) optional
If provided, all addons will depend on this object, and therefore not be installed until this object is finalized. This is useful if you want to ensure that addons are not applied before some other condition is met, e.g. node groups are created. See [issue #170](https://github.com/cloudposse/terraform-aws-eks-cluster/issues/170) for more details. **Default value:** `null`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the cluster. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the cluster. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the cluster with. These security groups will not be modified. **Default value:** `[ ]`
`bootstrap_self_managed_addons_enabled` (`bool`) optional
Manages bootstrap of default networking addons after cluster has been created **Default value:** `null`
`cloudwatch_log_group_class` (`string`) optional
Specified the log class of the log group. Possible values are: `STANDARD` or `INFREQUENT_ACCESS` **Default value:** `null`
`cloudwatch_log_group_kms_key_id` (`string`) optional
If provided, the KMS Key ID to use to encrypt AWS CloudWatch logs **Default value:** `null`
`cluster_attributes` (`list(string)`) optional
Override label module default cluster attributes **Default value:** ```hcl [ "cluster" ] ```
`cluster_depends_on` (`any`) optional
If provided, the EKS will depend on this object, and therefore not be created until this object is finalized. This is useful if you want to ensure that the cluster is not created before some other condition is met, e.g. VPNs into the subnet are created. **Default value:** `null`
`cluster_encryption_config_enabled` (`bool`) optional
Set to `true` to enable Cluster Encryption Configuration **Default value:** `true`
`cluster_encryption_config_kms_key_deletion_window_in_days` (`number`) optional
Cluster Encryption Config KMS Key Resource argument - key deletion windows in days post destruction **Default value:** `10`
`cluster_encryption_config_kms_key_enable_key_rotation` (`bool`) optional
Cluster Encryption Config KMS Key Resource argument - enable kms key rotation **Default value:** `true`
`cluster_encryption_config_kms_key_id` (`string`) optional
KMS Key ID to use for cluster encryption config **Default value:** `""`
`cluster_encryption_config_kms_key_policy` (`string`) optional
Cluster Encryption Config KMS Key Resource argument - key policy **Default value:** `null`
`cluster_encryption_config_resources` (`list(any)`) optional
Cluster Encryption Config Resources to encrypt, e.g. ['secrets'] **Default value:** ```hcl [ "secrets" ] ```
`cluster_log_retention_period` (`number`) optional
Number of days to retain cluster logs. Requires `enabled_cluster_log_types` to be set. See https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. **Default value:** `0`
`create_eks_service_role` (`bool`) optional
Set `false` to use existing `eks_cluster_service_role_arn` instead of creating one **Default value:** `true`
`custom_ingress_rules` optional
A List of Objects, which are custom security group rules that **Type:** ```hcl list(object({ description = string from_port = number to_port = number protocol = string source_security_group_id = string })) ``` **Default value:** `[ ]`
`eks_cluster_service_role_arn` (`string`) optional
The ARN of an IAM role for the EKS cluster to use that provides permissions for the Kubernetes control plane to perform needed AWS API operations. Required if `create_eks_service_role` is `false`, ignored otherwise. **Default value:** `null`
`enabled_cluster_log_types` (`list(string)`) optional
A list of the desired control plane logging to enable. For more information, see https://docs.aws.amazon.com/en_us/eks/latest/userguide/control-plane-logs.html. Possible values [`api`, `audit`, `authenticator`, `controllerManager`, `scheduler`] **Default value:** `[ ]`
`endpoint_private_access` (`bool`) optional
Indicates whether or not the Amazon EKS private API server endpoint is enabled. Default to AWS EKS resource and it is false **Default value:** `false`
`endpoint_public_access` (`bool`) optional
Indicates whether or not the Amazon EKS public API server endpoint is enabled. Default to AWS EKS resource and it is true **Default value:** `true`
`kubernetes_network_ipv6_enabled` (`bool`) optional
Set true to use IPv6 addresses for Kubernetes pods and services **Default value:** `false`
`kubernetes_version` (`string`) optional
Desired Kubernetes master version. If you do not specify a value, the latest available version is used **Default value:** `"1.21"`
`managed_security_group_rules_enabled` (`bool`) optional
Flag to enable/disable the ingress and egress rules for the EKS managed Security Group **Default value:** `true`
`oidc_provider_enabled` (`bool`) optional
Create an IAM OIDC identity provider for the cluster, then you can create IAM roles to associate with a service account in the cluster, instead of using kiam or kube2iam. For more information, see [EKS User Guide](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html). **Default value:** `false`
`permissions_boundary` (`string`) optional
If provided, all IAM roles will be created with this permissions boundary attached **Default value:** `null`
`public_access_cidrs` (`list(string)`) optional
Indicates which CIDR blocks can access the Amazon EKS public API server endpoint when enabled. EKS defaults this to a list with 0.0.0.0/0. **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`region` (`string`) optional
OBSOLETE (not needed): AWS Region **Default value:** `null`
`remote_network_config` optional
Configuration block for the cluster remote network configuration **Type:** ```hcl object({ remote_node_networks_cidrs = list(string) remote_pod_networks_cidrs = optional(list(string)) }) ``` **Default value:** `null`
`service_ipv4_cidr` (`string`) optional
The CIDR block to assign Kubernetes service IP addresses from. You can only specify a custom CIDR block when you create a cluster, changing this value will force a new cluster to be created. **Default value:** `null`
`upgrade_policy` optional
Configuration block for the support policy to use for the cluster **Type:** ```hcl object({ support_type = optional(string, null) }) ``` **Default value:** `null`
`zonal_shift_config` optional
Configuration block with zonal shift configuration for the cluster **Type:** ```hcl object({ enabled = optional(bool, null) }) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cloudwatch_log_group_kms_key_id`
KMS Key ID to encrypt AWS CloudWatch logs
`cloudwatch_log_group_name`
The name of the log group created in cloudwatch where cluster logs are forwarded to if enabled
`cluster_encryption_config_enabled`
If true, Cluster Encryption Configuration is enabled
`cluster_encryption_config_provider_key_alias`
Cluster Encryption Config KMS Key Alias ARN
`cluster_encryption_config_provider_key_arn`
Cluster Encryption Config KMS Key ARN
`cluster_encryption_config_resources`
Cluster Encryption Config Resources
`eks_addons_versions`
Map of enabled EKS Addons names and versions
`eks_cluster_arn`
The Amazon Resource Name (ARN) of the cluster
`eks_cluster_certificate_authority_data`
The Kubernetes cluster certificate authority data
`eks_cluster_endpoint`
The endpoint for the Kubernetes API server
`eks_cluster_id`
The name of the cluster
`eks_cluster_identity_oidc_issuer`
The OIDC Identity issuer for the cluster
`eks_cluster_identity_oidc_issuer_arn`
The OIDC Identity issuer ARN for the cluster that can be used to associate IAM roles with a service account
`eks_cluster_ipv4_service_cidr`
The IPv4 CIDR block that Kubernetes pod and service IP addresses are assigned from if `kubernetes_network_ipv6_enabled` is set to false. If set to true this output will be null.
`eks_cluster_ipv6_service_cidr`
The IPv6 CIDR block that Kubernetes pod and service IP addresses are assigned from if `kubernetes_network_ipv6_enabled` is set to true. If set to false this output will be null.
`eks_cluster_managed_security_group_id`
Security Group ID that was created by EKS for the cluster. EKS creates a Security Group and applies it to the ENI that are attached to EKS Control Plane master nodes and to any managed workloads.
`eks_cluster_role_arn`
ARN of the EKS cluster IAM role
`eks_cluster_version`
The Kubernetes server version of the cluster
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 5.75.1` - `tls`, version: `>= 3.1.0, != 4.0.0` ### Providers - `aws`, version: `>= 5.75.1` - `tls`, version: `>= 3.1.0, != 4.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_eks_access_entry.linux`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) (resource) - [`aws_eks_access_entry.map`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) (resource) - [`aws_eks_access_entry.standard`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) (resource) - [`aws_eks_access_entry.windows`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_entry) (resource) - [`aws_eks_access_policy_association.list`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_policy_association) (resource) - [`aws_eks_access_policy_association.map`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_access_policy_association) (resource) - [`aws_eks_addon.cluster`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) (resource) - [`aws_eks_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_cluster) (resource) - [`aws_iam_openid_connect_provider.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_openid_connect_provider) (resource) - [`aws_iam_policy.cluster_elb_service_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_cluster_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_service_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.cluster_elb_service_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_kms_alias.cluster`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_alias) (resource) - [`aws_kms_key.cluster`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) (resource) - [`aws_vpc_security_group_ingress_rule.custom_ingress_rules`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) (resource) - [`aws_vpc_security_group_ingress_rule.managed_ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) (resource) - [`aws_vpc_security_group_ingress_rule.managed_ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_security_group_ingress_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.cluster_elb_service_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`tls_certificate.cluster`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/data-sources/certificate) (data source) --- ## eks-fargate-profile # Module: `eks-fargate-profile` Terraform module to provision an [AWS Fargate Profile](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html) and Fargate Pod Execution Role for [EKS](https://aws.amazon.com/eks/). ## Introduction By default, this module will provision an [AWS Fargate Profile](https://docs.aws.amazon.com/eks/latest/userguide/fargate-profile.html) and Fargate Pod Execution Role for [EKS](https://aws.amazon.com/eks/). Note that in general, you only need one Fargate Pod Execution Role per AWS account, and it can be shared across regions. So if you are creating multiple Faragte Profiles, you can reuse the role created by the first one, or instantiate this module with `fargate_profile_enabled = false` to create the role separate from the profile. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-eks-fargate-profile/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-eks-fargate-profile/tree/main/test). ```hcl module "label" { source = "cloudposse/label/null" version = "0.25.0" # This is the preferred way to add attributes. It will put "cluster" last # after any attributes set in `var.attributes` or `context.attributes`. # In this case, we do not care, because we are only using this instance # of this module to create tags. attributes = ["cluster"] context = module.this.context } locals { tags = try(merge(module.label.tags, tomap("kubernetes.io/cluster/${module.label.id}", "shared")), null) eks_worker_ami_name_filter = "amazon-eks-node-${var.kubernetes_version}*" allow_all_ingress_rule = { key = "allow_all_ingress" type = "ingress" from_port = 0 to_port = 0 # [sic] from and to port ignored when protocol is "-1", warning if not zero protocol = "-1" description = "Allow all ingress" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } allow_http_ingress_rule = { key = "http" type = "ingress" from_port = 80 to_port = 80 protocol = "tcp" description = "Allow HTTP ingress" cidr_blocks = ["0.0.0.0/0"] ipv6_cidr_blocks = ["::/0"] } extra_policy_arn = "arn:aws:iam::aws:policy/job-function/ViewOnlyAccess" } module "vpc" { source = "cloudposse/vpc/aws" version = "1.1.0" cidr_block = var.vpc_cidr_block tags = local.tags context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" version = "1.0.0" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false tags = local.tags context = module.this.context } module "ssh_source_access" { source = "cloudposse/security-group/aws" version = "1.0.1" attributes = ["ssh", "source"] security_group_description = "Test source security group ssh access only" create_before_destroy = true allow_all_egress = true rules = [local.allow_all_ingress_rule] vpc_id = module.vpc.vpc_id context = module.label.context } module "https_sg" { source = "cloudposse/security-group/aws" version = "1.0.1" attributes = ["http"] security_group_description = "Allow http access" create_before_destroy = true allow_all_egress = true rules = [local.allow_http_ingress_rule] vpc_id = module.vpc.vpc_id context = module.label.context } module "eks_cluster" { source = "cloudposse/eks-cluster/aws" version = "2.2.0" region = var.region vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids kubernetes_version = var.kubernetes_version local_exec_interpreter = var.local_exec_interpreter oidc_provider_enabled = var.oidc_provider_enabled enabled_cluster_log_types = var.enabled_cluster_log_types cluster_log_retention_period = var.cluster_log_retention_period # data auth has problems destroying the auth-map kube_data_auth_enabled = false kube_exec_auth_enabled = true context = module.this.context } module "eks_node_group" { source = "cloudposse/eks-node-group/aws" version = "2.4.0" subnet_ids = module.subnets.public_subnet_ids cluster_name = module.eks_cluster.eks_cluster_id instance_types = var.instance_types desired_size = var.desired_size min_size = var.min_size max_size = var.max_size kubernetes_version = [var.kubernetes_version] kubernetes_labels = merge(var.kubernetes_labels, { attributes = coalesce(join(module.this.delimiter, module.this.attributes), "none") }) kubernetes_taints = var.kubernetes_taints ec2_ssh_key_name = var.ec2_ssh_key_name ssh_access_security_group_ids = [module.ssh_source_access.id] associated_security_group_ids = [module.ssh_source_access.id, module.https_sg.id] node_role_policy_arns = [local.extra_policy_arn] update_config = var.update_config after_cluster_joining_userdata = var.after_cluster_joining_userdata ami_type = var.ami_type ami_release_version = var.ami_release_version before_cluster_joining_userdata = [var.before_cluster_joining_userdata] context = module.this.context # Ensure ordering of resource creation to eliminate the race conditions when applying the Kubernetes Auth ConfigMap. # Do not create Node Group before the EKS cluster is created and the `aws-auth` Kubernetes ConfigMap is applied. depends_on = [module.eks_cluster, module.eks_cluster.kubernetes_config_map_id] create_before_destroy = true node_group_terraform_timeouts = [{ create = "40m" update = null delete = "20m" }] } module "eks_fargate_profile" { source = "cloudposse/eks-fargate-profile/aws" version = "x.x.x" subnet_ids = module.subnets.public_subnet_ids cluster_name = module.eks_cluster.eks_cluster_id kubernetes_namespace = var.kubernetes_namespace kubernetes_labels = var.kubernetes_labels iam_role_kubernetes_namespace_delimiter = var.iam_role_kubernetes_namespace_delimiter context = module.this.context } ``` ## Variables ### Required Variables
### Optional Variables
`cluster_name` (`string`) optional
The name of the EKS cluster **Default value:** `""`
`fargate_pod_execution_role_arn` (`string`) optional
ARN of the Fargate Pod Execution Role. Required if `fargate_pod_execution_role_enabled` is `false`, otherwise ignored. **Default value:** `null`
`fargate_pod_execution_role_enabled` (`bool`) optional
Set false to disable the Fargate Pod Execution Role creation **Default value:** `true`
`fargate_pod_execution_role_name` (`string`) optional
Fargate Pod Execution Role name. If not provided, will be derived from the context **Default value:** `null`
`fargate_profile_enabled` (`bool`) optional
Set false to disable the Fargate Profile creation **Default value:** `true`
`fargate_profile_iam_role_name` (`string`) optional
DEPRECATED (use `fargate_pod_execution_role_name` instead): Fargate profile IAM role name. If not provided, will be derived from the context **Default value:** `null`
`fargate_profile_name` (`string`) optional
Fargate profile name. If not provided, will be derived from the context **Default value:** `null`
`iam_role_kubernetes_namespace_delimiter` (`string`) optional
Delimiter for the Kubernetes namespace in the IAM Role name **Default value:** `"-"`
`kubernetes_labels` (`map(string)`) optional
Key-value mapping of Kubernetes labels for selection **Default value:** `{ }`
`kubernetes_namespace` (`string`) optional
Kubernetes namespace for selection **Default value:** `""`
`permissions_boundary` (`string`) optional
If provided, all IAM roles will be created with this permissions boundary attached **Default value:** `null`
`subnet_ids` (`list(string)`) optional
Identifiers of private EC2 Subnets to associate with the EKS Fargate Profile. These subnets must have the following resource tag: kubernetes.io/cluster/CLUSTER_NAME (where CLUSTER_NAME is replaced with the name of the EKS Cluster) **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`eks_fargate_pod_execution_role_arn`
ARN of the EKS Fargate Pod Execution role
`eks_fargate_pod_execution_role_name`
Name of the EKS Fargate Pod Execution role
`eks_fargate_profile_arn`
Amazon Resource Name (ARN) of the EKS Fargate Profile
`eks_fargate_profile_id`
EKS Cluster name and EKS Fargate Profile name separated by a colon
`eks_fargate_profile_role_arn`
DEPRECATED (use `eks_fargate_pod_execution_role_arn` instead): ARN of the EKS Fargate Profile IAM role
`eks_fargate_profile_role_name`
DEPRECATED (use `eks_fargate_pod_execution_role_name` instead): Name of the EKS Fargate Profile IAM role
`eks_fargate_profile_status`
Status of the EKS Fargate Profile
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.71.0` ### Providers - `aws`, version: `>= 3.71.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `fargate_profile_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `role_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eks_fargate_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_fargate_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_fargate_pod_execution_role_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## eks-iam-role # Module: `eks-iam-role` This `terraform-aws-eks-iam-role` project provides a simplified mechanism for provisioning [AWS EKS Service Account IAM roles](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html). ## Usage Here's how to invoke this module in your projects ```hcl module "eks_iam_role" { source = "cloudposse/eks-iam-role/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace environment = var.environment stage = var.stage name = var.name delimiter = var.delimiter attributes = var.attributes tags = var.tags aws_account_number = local.account_id eks_cluster_oidc_issuer_url = module.eks_cluster.eks_cluster_identity_oidc_issuer # Create a role for the service account named `autoscaler` in the Kubernetes namespace `kube-system` service_account_name = "autoscaler" service_account_namespace = "kube-system" # JSON IAM policy document to assign to the service account role aws_iam_policy_document = [data.aws_iam_policy_document.autoscaler.json] } data "aws_iam_policy_document" "autoscaler" { statement { sid = "AllowToScaleEKSNodeGroupAutoScalingGroup" actions = [ "ec2:DescribeLaunchTemplateVersions", "autoscaling:TerminateInstanceInAutoScalingGroup", "autoscaling:SetDesiredCapacity", "autoscaling:DescribeTags", "autoscaling:DescribeLaunchConfigurations", "autoscaling:DescribeAutoScalingInstances", "autoscaling:DescribeAutoScalingGroups" ] effect = "Allow" resources = ["*"] } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-eks-iam-role/tree/master/examples/complete) - complete example of using this module ## Variables ### Required Variables
`eks_cluster_oidc_issuer_url` (`string`) required
OIDC issuer URL for the EKS cluster (initial "https://" may be omitted)
### Optional Variables
`aws_account_number` (`string`) optional
AWS account number of EKS cluster owner. If an AWS account number is not provided, the current aws provider account number will be used. **Default value:** `null`
`aws_iam_policy_document` (`any`) optional
JSON string representation of the IAM policy for this service account as list of string (0 or 1 items). If empty, no custom IAM policy document will be used. If the list contains a single document, a custom IAM policy will be created and attached to the IAM role. Can also be a plain string, but that use is DEPRECATED because of Terraform issues. **Default value:** `[ ]`
`aws_partition` (`string`) optional
AWS partition: 'aws', 'aws-cn', or 'aws-us-gov' **Default value:** `"aws"`
`managed_policy_arns` (`set(string)`) optional
List of managed policies to attach to created role **Default value:** `[ ]`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role. **Default value:** `null`
`service_account_name` (`string`) optional
Kubernetes ServiceAccount name. Leave empty or set to "*" to indicate all Service Accounts, or if using `service_account_namespace_name_list`. **Default value:** `null`
`service_account_namespace` (`string`) optional
Kubernetes Namespace where service account is deployed. Leave empty or set to "*" to indicate all Namespaces, or if using `service_account_namespace_name_list`. **Default value:** `null`
`service_account_namespace_name_list` (`list(string)`) optional
List of `namespace:name` for service account assume role IAM policy if you need more than one. May include wildcards. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`service_account_name`
Kubernetes Service Account name
`service_account_namespace`
Kubernetes Service Account namespace
`service_account_policy_arn`
IAM policy ARN
`service_account_policy_id`
IAM policy ID
`service_account_policy_name`
IAM policy name
`service_account_role_arn`
IAM role ARN
`service_account_role_name`
IAM role name
`service_account_role_unique_id`
IAM role unique ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.2.0` - `aws`, version: `>= 3.0` - `local`, version: `>= 1.2` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `service_account_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.service_account`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.service_account`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.managed`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.service_account`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.service_account_assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## eks-node-group # Module: `eks-node-group` Terraform module to provision an EKS Managed Node Group for [Elastic Kubernetes Service](https://aws.amazon.com/eks/). Instantiate it multiple times to create EKS Managed Node Groups with specific settings such as GPUs, EC2 instance types, or autoscale parameters. **IMPORTANT:** When SSH access is enabled without specifying a source security group, this module provisions `EKS Node Group` nodes that are globally accessible by SSH (22) port. Normally, AWS recommends that no security group allows unrestricted ingress access to port 22 . ## Introduction This module creates an [EKS Managed Node Group](https://docs.aws.amazon.com/eks/latest/userguide/managed-node-groups.html) for an [EKS](https://aws.amazon.com/eks/) cluster. It assumes you have already created an EKS cluster, but you can create the cluster and the node group in the same Terraform configuration. See our [full-featured root module (a.k.a. component) `eks/cluster`](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/eks/cluster) for an example of how to do that. ### Launch Templates This module always uses a [launch template](https://docs.aws.amazon.com/autoscaling/ec2/userguide/launch-templates.html) to create the node group. You can create your own launch template and pass in its ID, or else this module will create one for you. The AWS default for EKS is that if the launch template is updated, the existing nodes will not be affected. Only new instances added to the node group would get the changes specified in the new launch template. In contrast, when the launch template changes, this module can immediately create a new node group from the new launch template to replace the old one. See the inputs `create_before_destroy` and `immediately_apply_lt_changes` for details about how to control this behavior. ### Operating system differences Currently, EKS supports 4 Operating Systems: Amazon Linux 2, Amazon Linux 2023, Bottlerocket, and Windows Server. This module supports all 4 OSes, but support for detailed configuration of the nodes varies by OS. The 4 inputs: 1. `before_cluster_joining_userdata` 2. `kubelet_additional_options` 3. `bootstrap_additional_options` 4. `after_cluster_joining_userdata` are fully supported for Amazon Linux 2 and Windows, and take advantage of the [bootstrap.sh](https://github.com/awslabs/amazon-eks-ami/blob/main/templates/al2/runtime/bootstrap.sh) supplied on those AMIs. **NONE** of these inputs are supported on Bottlerocket. On AL2023, only the first 2 are supported. Note that for all OSes, you can supply the complete `userdata` contents, which will be untouched by this module, via `userdata_override_base64`. ## Usage ### Major Changes (breaking and otherwise) With the v3.0.0 release of this module, support for Amazon Linux 2023 (AL2023) has been added, and some breaking changes have been made. Please see the [release notes](https://github.com/cloudposse/terraform-aws-eks-node-group/releases/tag/3.0.0) for details. With the v2.0.0 (a.k.a. v0.25.0) release of this module, it has undergone major breaking changes and added new features. Please see the [migration](https://github.com/cloudposse/terraform-aws-eks-node-group/tree/main/docs/migration-v1-v2.md) document for details. For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-eks-node-group/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-eks-node-group/tree/main/test). ### Sources of Information - The code examples below are manually updated and have a tendency to fall out of sync with actual code, particularly with respect to usage of other modules. Do not rely on them. - The documentation on this page about this module's inputs, outputs, and compliance is all automatically generated and is up-to-date as of the release date. After the code itself, this is your best source of information. - The code in [examples/complete](https://github.com/cloudposse/terraform-aws-eks-node-group/tree/main/examples/complete) is automatically tested before every release, so that is a good place to look for verified example code. Keep in mind, however, it is code for testing, so it may not represent average use cases or best practices. - Of course, the READMEs and `examples/complete` directories in the other modules' GitHub repos are more authoritative with respect to how to use those modules than this README is. #### Example Code ```hcl provider "aws" { region = var.region } module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace name = var.name stage = var.stage delimiter = var.delimiter attributes = ["cluster"] tags = var.tags } locals { # Prior to Kubernetes 1.19, the usage of the specific kubernetes.io/cluster/* resource tags below are required # for EKS and Kubernetes to discover and manage networking resources # https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking tags = { "kubernetes.io/cluster/${module.label.id}" = "shared" } } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "1.x.x" cidr_block = "172.16.0.0/16" tags = local.tags context = module.label.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "2.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = true nat_instance_enabled = false tags = local.tags context = module.label.context } module "eks_cluster" { source = "cloudposse/eks-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "4.x.x" vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids kubernetes_version = var.kubernetes_version oidc_provider_enabled = true context = module.label.context } module "eks_node_group" { source = "cloudposse/eks-node-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "3.x.x" instance_types = [var.instance_type] subnet_ids = module.subnets.public_subnet_ids min_size = var.min_size max_size = var.max_size cluster_name = module.eks_cluster.eks_cluster_id create_before_destroy = true kubernetes_version = var.kubernetes_version == null || var.kubernetes_version == "" ? [] : [var.kubernetes_version] # Enable the Kubernetes cluster auto-scaler to find the auto-scaling group cluster_autoscaler_enabled = var.autoscaling_policies_enabled context = module.label.context # Ensure the cluster is fully created before trying to add the node group module_depends_on = [module.eks_cluster.kubernetes_config_map_id] } ``` ## Variables ### Required Variables
`cluster_name` (`string`) required
The name of the EKS cluster
`desired_size` (`number`) required
Initial desired number of worker nodes (external changes ignored)
`max_size` (`number`) required
Maximum number of worker nodes
`min_size` (`number`) required
Minimum number of worker nodes
`subnet_ids` (`list(string)`) required
A list of subnet IDs to launch resources in
### Optional Variables
`after_cluster_joining_userdata` (`list(string)`) optional
Additional `bash` commands to execute on each worker node after joining the EKS cluster (after executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production **Default value:** `[ ]`
`ami_image_id` (`list(string)`) optional
AMI to use, overriding other AMI specifications, but must match `ami_type`. Ignored if `launch_template_id` is supplied. **Default value:** `[ ]`
`ami_release_version` (`list(string)`) optional
The EKS AMI "release version" to use. Defaults to the latest recommended version. For Amazon Linux, it is the "Release version" from [Amazon AMI Releases](https://github.com/awslabs/amazon-eks-ami/releases) For Bottlerocket, it is the release tag from [Bottlerocket Releases](https://github.com/bottlerocket-os/bottlerocket/releases) without the "v" prefix. For Windows, it is "AMI version" from [AWS docs](https://docs.aws.amazon.com/eks/latest/userguide/eks-ami-versions-windows.html). Note that unlike AMI names, release versions never include the "v" prefix. Examples: AL2: 1.29.3-20240531 Bottlerocket: 1.2.0 or 1.2.0-ccf1b754 Windows: 1.29-2024.04.09 **Default value:** `[ ]`
`ami_type` (`string`) optional
Type of Amazon Machine Image (AMI) associated with the EKS Node Group. Defaults to `AL2_x86_64`. Valid values: `AL2_x86_64, AL2_x86_64_GPU, AL2_ARM_64, CUSTOM, BOTTLEROCKET_ARM_64, BOTTLEROCKET_x86_64, BOTTLEROCKET_ARM_64_FIPS, BOTTLEROCKET_x86_64_FIPS, BOTTLEROCKET_ARM_64_NVIDIA, BOTTLEROCKET_x86_64_NVIDIA, WINDOWS_CORE_2019_x86_64, WINDOWS_FULL_2019_x86_64, WINDOWS_CORE_2022_x86_64, WINDOWS_FULL_2022_x86_64, AL2023_x86_64_STANDARD, AL2023_ARM_64_STANDARD, AL2023_x86_64_NEURON, AL2023_x86_64_NVIDIA, AL2023_ARM_64_NVIDIA`. **Default value:** `"AL2_x86_64"`
`associate_cluster_security_group` (`bool`) optional
When true, associate the default cluster security group to the nodes. If disabled the EKS managed security group will not be associated to the nodes and you will need to provide another security group that allows the nodes to communicate with the EKS control plane. Be aware that if no `associated_security_group_ids` or `ssh_access_security_group_ids` are provided, then the nodes will have no inbound or outbound rules. **Default value:** `true`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the node group with, in addition to the EKS' created security group. These security groups will not be modified. **Default value:** `[ ]`
`before_cluster_joining_userdata` (`list(string)`) optional
Additional `bash` commands to execute on each worker node before joining the EKS cluster (before executing the `bootstrap.sh` script). For more info, see https://kubedex.com/90-days-of-aws-eks-in-production **Default value:** `[ ]`
`block_device_map` optional
Map of block device name specification, see [launch_template.block-devices](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#block-devices). **Type:** ```hcl map(object({ no_device = optional(bool, null) virtual_name = optional(string, null) ebs = optional(object({ delete_on_termination = optional(bool, true) encrypted = optional(bool, true) iops = optional(number, null) kms_key_id = optional(string, null) snapshot_id = optional(string, null) throughput = optional(number, null) volume_size = optional(number, 20) volume_type = optional(string, "gp3") })) })) ``` **Default value:** ```hcl { "/dev/xvda": { "ebs": {} } } ```
`block_device_mappings` (`list(any)`) optional
DEPRECATED: Use `block_device_map` instead. List of block device mappings for the launch template. Each list element is an object with a `device_name` key and any keys supported by the `ebs` block of `launch_template`. **Default value:** `null`
`bootstrap_additional_options` (`list(string)`) optional
Additional options to bootstrap.sh. DO NOT include `--kubelet-additional-args`, use `kubelet_additional_options` var instead. Not used with AL2023 AMI types. **Default value:** `[ ]`
`capacity_type` (`string`) optional
Type of capacity associated with the EKS Node Group. Valid values: "ON_DEMAND", "SPOT", or `null`. Terraform will only perform drift detection if a configuration value is provided. **Default value:** `null`
`cluster_autoscaler_enabled` (`bool`) optional
OBSOLETE. Used to add support for the Kubernetes Cluster Autoscaler, but additional support is no longer needed. **Default value:** `null`
`cpu_options` (`list(any)`) optional
Configuration for the [`cpu_options` Configuration Block](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#cpu_options) of the launch template. Leave list empty for defaults. Pass list with single object with attributes matching the `cpu_options` block to configure it. Note that this configures the launch template only. Some elements will be ignored by the Auto Scaling Group that actually launches instances. Consult AWS documentation for details. **Default value:** `[ ]`
`create_before_destroy` (`bool`) optional
If `true` (default), a new node group will be created before destroying the old one. If `false`, the old node group will be destroyed first, causing downtime. Changing this setting will always cause node group to be replaced. **Default value:** `true`
`detailed_monitoring_enabled` (`bool`) optional
The launched EC2 instance will have detailed monitoring enabled. Defaults to false **Default value:** `false`
`ebs_optimized` (`bool`) optional
Set `false` to disable EBS optimization **Default value:** `true`
`ec2_ssh_key_name` (`list(string)`) optional
SSH key pair name to use to access the worker nodes **Default value:** `[ ]`
`enclave_enabled` (`bool`) optional
Set to `true` to enable Nitro Enclaves on the instance. **Default value:** `false`
`force_update_version` (`bool`) optional
When updating the Kubernetes version, force Pods to be removed even if PodDisruptionBudget or taint/toleration issues would otherwise prevent them from being removed (and cause the update to fail) **Default value:** `false`
`immediately_apply_lt_changes` (`bool`) optional
When `true`, any change to the launch template will be applied immediately. When `false`, the changes will only affect new nodes when they are launched. When `null` (default) this input takes the value of `create_before_destroy`. **NOTE:** Setting this to `false` does not guarantee that other changes, such as `ami_type`, will not cause changes to be applied immediately. **Default value:** `null`
`instance_types` (`list(string)`) optional
Instance types to use for this node group (up to 20). Defaults to ["t3.medium"]. Must be empty if the launch template configured by `launch_template_id` specifies an instance type. **Default value:** ```hcl [ "t3.medium" ] ```
`kubelet_additional_options` (`list(string)`) optional
Additional flags to pass to kubelet. DO NOT include `--node-labels` or `--node-taints`, use `kubernetes_labels` and `kubernetes_taints` to specify those." **Default value:** `[ ]`
`kubernetes_labels` (`map(string)`) optional
Key-value mapping of Kubernetes labels. Only labels that are applied with the EKS API are managed by this argument. Other Kubernetes labels applied to the EKS Node Group will not be managed. **Default value:** `{ }`
`kubernetes_taints` optional
List of `key`, `value`, `effect` objects representing Kubernetes taints. `effect` must be one of `NO_SCHEDULE`, `NO_EXECUTE`, or `PREFER_NO_SCHEDULE`. `key` and `effect` are required, `value` may be null. **Type:** ```hcl list(object({ key = string value = optional(string) effect = string })) ``` **Default value:** `[ ]`
`kubernetes_version` (`list(string)`) optional
Kubernetes version. Defaults to EKS Cluster Kubernetes version. Terraform will only perform drift detection if a configuration value is provided **Default value:** `[ ]`
`launch_template_id` (`list(string)`) optional
The ID (not name) of a custom launch template to use for the EKS node group. If provided, it must specify the AMI image ID. **Default value:** `[ ]`
`launch_template_version` (`list(string)`) optional
The version of the specified launch template to use. Defaults to latest version. **Default value:** `[ ]`
`metadata_http_endpoint_enabled` (`bool`) optional
Set false to disable the Instance Metadata Service. **Default value:** `true`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for Instance Metadata Service requests. The default is `2` to allows containerized workloads assuming the instance profile, but it's not really recomended. You should use OIDC service accounts instead. **Default value:** `2`
`metadata_http_tokens_required` (`bool`) optional
Set true to require IMDS session tokens, disabling Instance Metadata Service Version 1. **Default value:** `true`
`module_depends_on` (`any`) optional
Can be any value desired. Module will wait for this value to be computed before creating node group. **Default value:** `null`
`node_group_terraform_timeouts` optional
Configuration for the Terraform [`timeouts` Configuration Block](https://www.terraform.io/docs/language/resources/syntax.html#operation-timeouts) of the node group resource. Leave list empty for defaults. Pass list with single object with attributes matching the `timeouts` block to configure it. Leave attribute values `null` to preserve individual defaults while setting others. **Type:** ```hcl list(object({ create = optional(string) update = optional(string) delete = optional(string) })) ``` **Default value:** `[ ]`
`node_repair_enabled` (`bool`) optional
The node auto-repair configuration for the node group will be enabled. Defaults to false **Default value:** `false`
`node_role_arn` (`list(string)`) optional
If provided, assign workers the given role, which this module will not modify **Default value:** `[ ]`
`node_role_cni_policy_enabled` (`bool`) optional
When true, the `AmazonEKS_CNI_Policy` will be attached to the node IAM role. This used to be required, but it is [now recommended](https://docs.aws.amazon.com/eks/latest/userguide/create-node-role.html) that this policy be attached only to the `aws-node` Kubernetes service account. However, that is difficult to do with Terraform, so this module defaults to the old pattern. **Default value:** `true`
`node_role_permissions_boundary` (`string`) optional
If provided, all IAM roles will be created with this permissions boundary attached. **Default value:** `null`
`node_role_policy_arns` (`list(string)`) optional
List of policy ARNs to attach to the worker role this module creates in addition to the default ones **Default value:** `[ ]`
`placement` (`list(any)`) optional
Configuration for the [`placement` Configuration Block](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template#placement) of the launch template. Leave list empty for defaults. Pass list with single object with attributes matching the `placement` block to configure it. Note that this configures the launch template only. Some elements will be ignored by the Auto Scaling Group that actually launches instances. Consult AWS documentation for details. **Default value:** `[ ]`
`random_pet_length` (`number`) optional
In order to support "create before destroy" behavior, this module uses the `random_pet` resource to generate a unique pet name for the node group, since the node group name must be unique, meaning the new node group must have a different name than the old one. This variable controls the length of the pet name, meaning the number of pet names concatenated together. This module defaults to 1, but there are only 452 names available, so users with large numbers of node groups may want to increase this value. **Default value:** `1`
`replace_node_group_on_version_update` (`bool`) optional
Force Node Group replacement when updating to a new Kubernetes version. If set to `false` (the default), the Node Groups will be updated in-place **Default value:** `false`
`resources_to_tag` (`list(string)`) optional
List of auto-launched resource types to tag. Valid types are "instance", "volume", "elastic-gpu", "spot-instances-request", "network-interface". **Default value:** ```hcl [ "instance", "volume", "network-interface" ] ```
`ssh_access_security_group_ids` (`list(string)`) optional
Set of EC2 Security Group IDs to allow SSH access (port 22) to the worker nodes. If you specify `ec2_ssh_key`, but do not specify this configuration when you create an EKS Node Group, port 22 on the worker nodes is opened to the Internet (0.0.0.0/0) **Default value:** `[ ]`
`update_config` (`list(map(number))`) optional
Configuration for the `eks_node_group` [`update_config` Configuration Block](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group#update_config-configuration-block). Specify exactly one of `max_unavailable` (node count) or `max_unavailable_percentage` (percentage of nodes). **Default value:** `[ ]`
`userdata_override_base64` (`list(string)`) optional
Many features of this module rely on the `bootstrap.sh` provided with Amazon Linux, and this module may generate "user data" that expects to find that script. If you want to use an AMI that is not compatible with the userdata generated by this module, then use `userdata_override_base64` to provide your own (Base64 encoded) user data. Use "" to prevent any user data from being set. Setting `userdata_override_base64` disables `kubernetes_taints`, `kubelet_additional_options`, `before_cluster_joining_userdata`, `after_cluster_joining_userdata`, and `bootstrap_additional_options`. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`WARNING_cluster_autoscaler_enabled`
WARNING
`eks_node_group_ami_id`
The ID of the AMI used for the worker nodes, if specified
`eks_node_group_arn`
Amazon Resource Name (ARN) of the EKS Node Group
`eks_node_group_cbd_pet_name`
The pet name of this node group, if this module generated one
`eks_node_group_id`
EKS Cluster name and EKS Node Group name separated by a colon
`eks_node_group_launch_template_id`
The ID of the launch template used for this node group
`eks_node_group_launch_template_name`
The name of the launch template used for this node group
`eks_node_group_remote_access_security_group_id`
The ID of the security group generated to allow SSH access to the nodes, if this module generated one
`eks_node_group_resources`
List of objects containing information about underlying resources of the EKS Node Group
`eks_node_group_role_arn`
ARN of the worker nodes IAM role
`eks_node_group_role_name`
Name of the worker nodes IAM role
`eks_node_group_status`
Status of the EKS Node Group
`eks_node_group_tags_all`
A map of tags assigned to the resource, including those inherited from the provider default_tags configuration block.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 5.8` - `null`, version: `>= 3.0` - `random`, version: `>= 2.0` ### Providers - `aws`, version: `>= 5.8` - `null`, version: `>= 3.0` - `random`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `ssh_access` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eks_node_group.cbd`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group) (resource) - [`aws_eks_node_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_node_group) (resource) - [`aws_iam_policy.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.amazon_ec2_container_registry_read_only`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_worker_node_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.existing_policies_for_eks_workers_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_launch_template.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/launch_template) (resource) - [`null_resource.seq`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`random_pet.cbd`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.windows_ami`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_eks_cluster.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ipv6_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_launch_template.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/launch_template) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_ssm_parameter.ami_id`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) ## Windows Managed Node groups Windows managed node-groups have a few pre-requisites. * Your cluster must contain at least one linux based worker node * Your EKS Cluster must have the `AmazonEKSVPCResourceController` and `AmazonEKSClusterPolicy` policies attached * Your cluster must have a config-map called amazon-vpc-cni with the following content ```yaml apiVersion: v1 kind: ConfigMap metadata: name: amazon-vpc-cni namespace: kube-system data: enable-windows-ipam: "true" ``` * Windows nodes will automatically be tainted ```yaml kubernetes_taints = [{ key = "WINDOWS" value = "true" effect = "NO_SCHEDULE" }] ``` * Any pods that target Windows will need to have the following attributes set in their manifest ```yaml nodeSelector: kubernetes.io/os: windows kubernetes.io/arch: amd64 ``` https://docs.aws.amazon.com/eks/latest/userguide/windows-support.html --- ## eks-workers # Module: `eks-workers` Terraform module to provision AWS resources to run EC2 worker nodes for [Elastic Kubernetes Service](https://aws.amazon.com/eks/). Instantiate it multiple times to create many EKS worker node pools with specific settings such as GPUs, EC2 instance types, or autoscale parameters. ## Introduction The module provisions the following resources: - IAM Role and Instance Profile to allow Kubernetes nodes to access other AWS services - Security Group with rules for EKS workers to allow networking traffic - AutoScaling Group with Launch Template to configure and launch worker instances - AutoScaling Policies and CloudWatch Metric Alarms to monitor CPU utilization on the EC2 instances and scale the number of instance in the AutoScaling Group up or down. If you don't want to use the provided functionality, or want to provide your own policies, disable it by setting the variable `autoscaling_policies_enabled` to `"false"`. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-eks-workers/tree/main/examples/complete) ```hcl provider "aws" { region = var.region } locals { # The usage of the specific kubernetes.io/cluster/* resource tags below are required # for EKS and Kubernetes to discover and manage networking resources # https://www.terraform.io/docs/providers/aws/guides/eks-getting-started.html#base-vpc-networking tags = merge(var.tags, map("kubernetes.io/cluster/${var.cluster_name}", "shared")) } module "vpc" { source = "cloudposse/vpc/aws" version = "2.1.1" namespace = var.namespace stage = var.stage name = var.name ipv4_primary_cidr_block = "172.16.0.0/16" tags = local.tags } module "subnets" { source = "cloudposse/dynamic-subnets/aws" version = "2.4.1" namespace = var.namespace stage = var.stage name = var.name availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false tags = local.tags } module "eks_workers" { source = "cloudposse/eks-workers/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name instance_type = var.instance_type vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids health_check_type = var.health_check_type min_size = var.min_size max_size = var.max_size wait_for_capacity_timeout = var.wait_for_capacity_timeout cluster_name = var.cluster_name cluster_endpoint = var.cluster_endpoint cluster_certificate_authority_data = var.cluster_certificate_authority_data cluster_security_group_id = var.cluster_security_group_id # Auto-scaling policies and CloudWatch metric alarms autoscaling_policies_enabled = var.autoscaling_policies_enabled cpu_utilization_high_threshold_percent = var.cpu_utilization_high_threshold_percent cpu_utilization_low_threshold_percent = var.cpu_utilization_low_threshold_percent } ``` ## Variables ### Required Variables
`cluster_certificate_authority_data` (`string`) required
The base64 encoded certificate data required to communicate with the cluster
`cluster_endpoint` (`string`) required
EKS cluster endpoint
`cluster_name` (`string`) required
The name of the EKS cluster
`cluster_security_group_id` (`string`) required
Security Group ID of the EKS cluster
`instance_type` (`string`) required
Instance type to launch
`max_size` (`number`) required
The maximum size of the autoscale group
`min_size` (`number`) required
The minimum size of the autoscale group
`subnet_ids` (`list(string)`) required
A list of subnet IDs to launch resources in
`vpc_id` (`string`) required
VPC ID for the EKS cluster
### Optional Variables
`additional_security_group_ids` (`list(string)`) optional
Additional list of security groups that will be attached to the autoscaling group **Default value:** `[ ]`
`after_cluster_joining_userdata` (`string`) optional
Additional commands to execute on each worker node after joining the EKS cluster (after executing the `bootstrap.sh` script). For mot info, see https://kubedex.com/90-days-of-aws-eks-in-production **Default value:** `""`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the worker nodes **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
List of Security Group IDs to be allowed to connect to the worker nodes **Default value:** `[ ]`
`associate_public_ip_address` (`bool`) optional
Associate a public IP address with an instance in a VPC **Default value:** `false`
`autoscaling_group_tags` (`map(string)`) optional
Additional tags only for the autoscaling group, e.g. "k8s.io/cluster-autoscaler/node-template/taint/dedicated" = "ci-cd:NoSchedule". **Default value:** `{ }`
`autoscaling_policies_enabled` (`bool`) optional
Whether to create `aws_autoscaling_policy` and `aws_cloudwatch_metric_alarm` resources to control Auto Scaling **Default value:** `true`
`aws_iam_instance_profile_name` (`string`) optional
The name of the existing instance profile that will be used in autoscaling group for EKS workers. If empty will create a new instance profile. **Default value:** `""`
`before_cluster_joining_userdata` (`string`) optional
Additional commands to execute on each worker node before joining the EKS cluster (before executing the `bootstrap.sh` script). For mot info, see https://kubedex.com/90-days-of-aws-eks-in-production **Default value:** `""`
`block_device_mappings` optional
Specify volumes to attach to the instance besides the volumes specified by the AMI **Type:** ```hcl list(object({ device_name = string no_device = bool virtual_name = string ebs = object({ delete_on_termination = bool encrypted = bool iops = number kms_key_id = string snapshot_id = string volume_size = number volume_type = string }) })) ``` **Default value:** `[ ]`
`bootstrap_extra_args` (`string`) optional
Extra arguments to the `bootstrap.sh` script to enable `--enable-docker-bridge` or `--use-max-pods` **Default value:** `""`
`cluster_security_group_ingress_enabled` (`bool`) optional
Whether to enable the EKS cluster Security Group as ingress to workers Security Group **Default value:** `true`
`cpu_utilization_high_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_high_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_high_statistic` (`string`) optional
The statistic to apply to the alarm's associated metric. Either of the following is supported: `SampleCount`, `Average`, `Sum`, `Minimum`, `Maximum` **Default value:** `"Average"`
`cpu_utilization_high_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `90`
`cpu_utilization_low_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold **Default value:** `2`
`cpu_utilization_low_period_seconds` (`number`) optional
The period in seconds over which the specified statistic is applied **Default value:** `300`
`cpu_utilization_low_statistic` (`string`) optional
The statistic to apply to the alarm's associated metric. Either of the following is supported: `SampleCount`, `Average`, `Sum`, `Minimum`, `Maximum` **Default value:** `"Average"`
`cpu_utilization_low_threshold_percent` (`number`) optional
The value against which the specified statistic is compared **Default value:** `10`
`credit_specification` optional
Customize the credit specification of the instances **Type:** ```hcl object({ cpu_credits = string }) ``` **Default value:** `null`
`default_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes before another scaling activity can start **Default value:** `300`
`disable_api_termination` (`bool`) optional
If `true`, enables EC2 Instance Termination Protection **Default value:** `false`
`ebs_optimized` (`bool`) optional
If true, the launched EC2 instance will be EBS-optimized **Default value:** `false`
`eks_worker_ami_name_filter` (`string`) optional
AMI name filter to lookup the most recent EKS AMI if `image_id` is not provided **Default value:** `"amazon-eks-node-*"`
`eks_worker_ami_name_regex` (`string`) optional
A regex string to apply to the AMI list returned by AWS **Default value:** `"^amazon-eks-node-[0-9,.]+-v[0-9]{8}$"`
`elastic_gpu_specifications` optional
Specifications of Elastic GPU to attach to the instances **Type:** ```hcl object({ type = string }) ``` **Default value:** `null`
`enable_monitoring` (`bool`) optional
Enable/disable detailed monitoring **Default value:** `true`
`enabled_metrics` (`list(string)`) optional
A list of metrics to collect. The allowed values are `GroupMinSize`, `GroupMaxSize`, `GroupDesiredCapacity`, `GroupInServiceInstances`, `GroupPendingInstances`, `GroupStandbyInstances`, `GroupTerminatingInstances`, `GroupTotalInstances` **Default value:** ```hcl [ "GroupMinSize", "GroupMaxSize", "GroupDesiredCapacity", "GroupInServiceInstances", "GroupPendingInstances", "GroupStandbyInstances", "GroupTerminatingInstances", "GroupTotalInstances" ] ```
`force_delete` (`bool`) optional
Allows deleting the autoscaling group without waiting for all instances in the pool to terminate. You can force an autoscaling group to delete even if it's in the process of scaling a resource. Normally, Terraform drains all the instances before deleting the group. This bypasses that behavior and potentially leaves resources dangling **Default value:** `false`
`health_check_grace_period` (`number`) optional
Time (in seconds) after instance comes into service before checking health **Default value:** `300`
`health_check_type` (`string`) optional
Controls how health checking is done. Valid values are `EC2` or `ELB` **Default value:** `"EC2"`
`image_id` (`string`) optional
EC2 image ID to launch. If not provided, the module will lookup the most recent EKS AMI. See https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html for more details on EKS-optimized images **Default value:** `""`
`instance_initiated_shutdown_behavior` (`string`) optional
Shutdown behavior for the instances. Can be `stop` or `terminate` **Default value:** `"terminate"`
`instance_market_options` optional
The market (purchasing) option for the instances **Type:** ```hcl object({ market_type = string spot_options = object({ block_duration_minutes = number instance_interruption_behavior = string max_price = number spot_instance_type = string valid_until = string }) }) ``` **Default value:** `null`
`key_name` (`string`) optional
SSH key name that should be used for the instance **Default value:** `""`
`kubelet_extra_args` (`string`) optional
Extra arguments to pass to kubelet, like "--register-with-taints=dedicated=ci-cd:NoSchedule --node-labels=purpose=ci-worker" **Default value:** `""`
`load_balancers` (`list(string)`) optional
A list of elastic load balancer names to add to the autoscaling group. Only valid for classic load balancers. For ALBs, use `target_group_arns` instead **Default value:** `[ ]`
`max_instance_lifetime` (`number`) optional
The maximum amount of time, in seconds, that an instance can be in service, values must be either equal to 0 or between 604800 and 31536000 seconds **Default value:** `null`
`metadata_http_endpoint_enabled` (`bool`) optional
Set false to disable the Instance Metadata Service. **Default value:** `true`
`metadata_http_put_response_hop_limit` (`number`) optional
The desired HTTP PUT response hop limit (between 1 and 64) for Instance Metadata Service requests. The default is `2` to support containerized workloads. **Default value:** `2`
`metadata_http_tokens_required` (`bool`) optional
Set true to require IMDS session tokens, disabling Instance Metadata Service Version 1. **Default value:** `true`
`metrics_granularity` (`string`) optional
The granularity to associate with the metrics to collect. The only valid value is 1Minute **Default value:** `"1Minute"`
`min_elb_capacity` (`number`) optional
Setting this causes Terraform to wait for this number of instances to show up healthy in the ELB only on creation. Updates will not wait on ELB instance number changes **Default value:** `0`
`mixed_instances_policy` optional
policy to used mixed group of on demand/spot of differing types. Launch template is automatically generated. https://www.terraform.io/docs/providers/aws/r/autoscaling_group.html#mixed_instances_policy-1 **Type:** ```hcl object({ instances_distribution = object({ on_demand_allocation_strategy = string on_demand_base_capacity = number on_demand_percentage_above_base_capacity = number spot_allocation_strategy = string spot_instance_pools = number spot_max_price = string }) override = list(object({ instance_type = string weighted_capacity = number })) }) ``` **Default value:** `null`
`placement` optional
The placement specifications of the instances **Type:** ```hcl object({ affinity = string availability_zone = string group_name = string host_id = string tenancy = string }) ``` **Default value:** `null`
`placement_group` (`string`) optional
The name of the placement group into which you'll launch your instances, if any **Default value:** `""`
`protect_from_scale_in` (`bool`) optional
Allows setting instance protection. The autoscaling group will not select instances with this setting for terminination during scale in events **Default value:** `false`
`scale_down_adjustment_type` (`string`) optional
Specifies whether the adjustment is an absolute number or a percentage of the current capacity. Valid values are `ChangeInCapacity`, `ExactCapacity` and `PercentChangeInCapacity` **Default value:** `"ChangeInCapacity"`
`scale_down_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`scale_down_policy_type` (`string`) optional
The scalling policy type, either `SimpleScaling`, `StepScaling` or `TargetTrackingScaling` **Default value:** `"SimpleScaling"`
`scale_down_scaling_adjustment` (`number`) optional
The number of instances by which to scale. `scale_down_scaling_adjustment` determines the interpretation of this number (e.g. as an absolute number or as a percentage of the existing Auto Scaling group size). A positive increment adds to the current capacity and a negative value removes from the current capacity **Default value:** `-1`
`scale_up_adjustment_type` (`string`) optional
Specifies whether the adjustment is an absolute number or a percentage of the current capacity. Valid values are `ChangeInCapacity`, `ExactCapacity` and `PercentChangeInCapacity` **Default value:** `"ChangeInCapacity"`
`scale_up_cooldown_seconds` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling activity can start **Default value:** `300`
`scale_up_policy_type` (`string`) optional
The scalling policy type, either `SimpleScaling`, `StepScaling` or `TargetTrackingScaling` **Default value:** `"SimpleScaling"`
`scale_up_scaling_adjustment` (`number`) optional
The number of instances by which to scale. `scale_up_adjustment_type` determines the interpretation of this number (e.g. as an absolute number or as a percentage of the existing Auto Scaling group size). A positive increment adds to the current capacity and a negative value removes from the current capacity **Default value:** `1`
`service_linked_role_arn` (`string`) optional
The ARN of the service-linked role that the ASG will use to call other AWS services **Default value:** `""`
`suspended_processes` (`list(string)`) optional
A list of processes to suspend for the AutoScaling Group. The allowed values are `Launch`, `Terminate`, `HealthCheck`, `ReplaceUnhealthy`, `AZRebalance`, `AlarmNotification`, `ScheduledActions`, `AddToLoadBalancer`. Note that if you suspend either the `Launch` or `Terminate` process types, it can prevent your autoscaling group from functioning properly. **Default value:** `[ ]`
`target_group_arns` (`list(string)`) optional
A list of aws_alb_target_group ARNs, for use with Application Load Balancing **Default value:** `[ ]`
`termination_policies` (`list(string)`) optional
A list of policies to decide how the instances in the auto scale group should be terminated. The allowed values are `OldestInstance`, `NewestInstance`, `OldestLaunchConfiguration`, `ClosestToNextInstanceHour`, `Default` **Default value:** ```hcl [ "Default" ] ```
`use_custom_image_id` (`bool`) optional
If set to `true`, will use variable `image_id` for the EKS workers inside autoscaling group **Default value:** `false`
`use_existing_aws_iam_instance_profile` (`bool`) optional
If set to `true`, will use variable `aws_iam_instance_profile_name` to run EKS workers using an existing AWS instance profile that was created outside of this module, workaround for error like `count cannot be computed` **Default value:** `false`
`use_existing_security_group` (`bool`) optional
If set to `true`, will use variable `workers_security_group_id` to run EKS workers using an existing security group that was created outside of this module, workaround for errors like `count cannot be computed` **Default value:** `false`
`wait_for_capacity_timeout` (`string`) optional
A maximum duration that Terraform should wait for ASG instances to be healthy before timing out. Setting this to '0' causes Terraform to skip all Capacity Waiting behavior **Default value:** `"10m"`
`wait_for_elb_capacity` (`number`) optional
Setting this will cause Terraform to wait for exactly this number of healthy instances in all attached load balancers on both create and update operations. Takes precedence over `min_elb_capacity` behavior **Default value:** `0`
`workers_role_policy_arns` (`list(string)`) optional
List of policy ARNs that will be attached to the workers default role on creation **Default value:** `[ ]`
`workers_role_policy_arns_count` (`number`) optional
Count of policy ARNs that will be attached to the workers default role on creation. Needed to prevent Terraform error `count can't be computed` **Default value:** `0`
`workers_security_group_id` (`string`) optional
The name of the existing security group that will be used in autoscaling group for EKS workers. If empty, a new security group will be created **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`autoscaling_group_arn`
ARN of the AutoScaling Group
`autoscaling_group_default_cooldown`
Time between a scaling activity and the succeeding scaling activity
`autoscaling_group_desired_capacity`
The number of Amazon EC2 instances that should be running in the group
`autoscaling_group_health_check_grace_period`
Time after instance comes into service before checking health
`autoscaling_group_health_check_type`
`EC2` or `ELB`. Controls how health checking is done
`autoscaling_group_id`
The AutoScaling Group ID
`autoscaling_group_max_size`
The maximum size of the AutoScaling Group
`autoscaling_group_min_size`
The minimum size of the AutoScaling Group
`autoscaling_group_name`
The AutoScaling Group name
`autoscaling_group_tags`
A list of tag settings associated with the AutoScaling Group
`launch_template_arn`
ARN of the launch template
`launch_template_id`
The ID of the launch template
`security_group_arn`
ARN of the worker nodes Security Group
`security_group_id`
ID of the worker nodes Security Group
`security_group_name`
Name of the worker nodes Security Group
`workers_role_arn`
ARN of the worker nodes IAM role
`workers_role_name`
Name of the worker nodes IAM role
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.16.0` ### Providers - `aws`, version: `>= 5.16.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `autoscale_group` | 0.40.0 | [`cloudposse/ec2-autoscale-group/aws`](https://registry.terraform.io/modules/cloudposse/ec2-autoscale-group/aws/0.40.0) | n/a `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.amazon_ec2_container_registry_read_only`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_cni_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.amazon_eks_worker_node_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.existing_policies_attach_to_eks_workers_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cluster`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_self`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ami.eks_worker`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami) (data source) - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_instance_profile) (data source) - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## elastic-beanstalk-application # Module: `elastic-beanstalk-application` Terraform module to provision AWS Elastic Beanstalk application ## Variables ### Required Variables
### Optional Variables
`appversion_lifecycle_delete_source_from_s3` (`bool`) optional
Whether to delete application versions from S3 source **Default value:** `false`
`appversion_lifecycle_max_age_in_days` (`number`) optional
The number of days to retain an application version **Default value:** `null`
`appversion_lifecycle_max_count` (`number`) optional
The max number of application versions to keep. if left as default and `appversion_lifecycle_max_age_in_days` is set, `appversion_lifecycle_max_age_in_days` will be used. Otherwise `appversion_lifecycle_max_count` will take place **Default value:** `200`
`appversion_lifecycle_service_role_arn` (`string`) optional
The service role ARN to use for application version cleanup. If left empty, the `appversion_lifecycle` block will not be created **Default value:** `""`
`create_lifecycle_service_role` (`bool`) optional
Whether to create service role for application version cleanup. If set, `appversion_lifecycle_service_role_arn` will be ignored **Default value:** `false`
`description` (`string`) optional
Elastic Beanstalk Application description **Default value:** `""`
`prefer_legacy_service_policy` (`bool`) optional
Whether to use AWSElasticBeanstalkService (deprecated) or AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy policy **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`appversion_lifecycle_service_role_arn`
Elastic Beanstalk Application Lifecycle Service Role Arn
`elastic_beanstalk_application_arn`
Elastic Beanstalk Application Arn
`elastic_beanstalk_application_lifecycle_rules`
Elastic Beanstalk Application Lifecycle Rules
`elastic_beanstalk_application_name`
Elastic Beanstalk Application name
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_elastic_beanstalk_application.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_application) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.enhanced_health`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## elastic-beanstalk-environment # Module: `elastic-beanstalk-environment` Terraform module to provision AWS Elastic Beanstalk environment ## Searching for Maintainer! The Cloud Posse team no longer utilizes Beanstalk all that much, but this module is still fairly popular. In an effort to give it the attention it deserves, we're searching for a volunteer maintainer to manage this specific repository's issues and pull requests (of which a number are already stacked up). This is a great opportunity for anyone who is looking to solidify and strengthen their Terraform skillset while also giving back to the SweetOps open source community! [You can learn more about being a SweetOps contributor on our docs site here](https://docs.cloudposse.com/community/contributors/). If you're interested, reach out to us via the `#terraform` channel in [the SweetOps Slack](https://slack.sweetops.com/) or directly [via email @ hello@cloudposse.com](mailto:hello@cloudposse.com) ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-elastic-beanstalk-environment/tree/main/examples/complete) ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version version = "x.x.x" cidr_block = "172.16.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false context = module.this.context } module "elastic_beanstalk_application" { source = "cloudposse/elastic-beanstalk-application/aws" # Cloud Posse recommends pinning every module to a specific version version = "x.x.x" description = "Test Elastic Beanstalk application" context = module.this.context } module "elastic_beanstalk_environment" { source = "../../" description = var.description region = var.region availability_zone_selector = var.availability_zone_selector dns_zone_id = var.dns_zone_id wait_for_ready_timeout = var.wait_for_ready_timeout elastic_beanstalk_application_name = module.elastic_beanstalk_application.elastic_beanstalk_application_name environment_type = var.environment_type loadbalancer_type = var.loadbalancer_type elb_scheme = var.elb_scheme tier = var.tier version_label = var.version_label force_destroy = var.force_destroy instance_type = var.instance_type root_volume_size = var.root_volume_size root_volume_type = var.root_volume_type autoscale_min = var.autoscale_min autoscale_max = var.autoscale_max autoscale_measure_name = var.autoscale_measure_name autoscale_statistic = var.autoscale_statistic autoscale_unit = var.autoscale_unit autoscale_lower_bound = var.autoscale_lower_bound autoscale_lower_increment = var.autoscale_lower_increment autoscale_upper_bound = var.autoscale_upper_bound autoscale_upper_increment = var.autoscale_upper_increment vpc_id = module.vpc.vpc_id loadbalancer_subnets = module.subnets.public_subnet_ids application_subnets = module.subnets.private_subnet_ids allow_all_egress = true additional_security_group_rules = [ { type = "ingress" from_port = 0 to_port = 65535 protocol = "-1" source_security_group_id = module.vpc.vpc_default_security_group_id description = "Allow all inbound traffic from trusted Security Groups" } ] rolling_update_enabled = var.rolling_update_enabled rolling_update_type = var.rolling_update_type updating_min_in_service = var.updating_min_in_service updating_max_batch = var.updating_max_batch healthcheck_url = var.healthcheck_url application_port = var.application_port solution_stack_name = var.solution_stack_name additional_settings = var.additional_settings env_vars = var.env_vars extended_ec2_policy_document = data.aws_iam_policy_document.minimal_s3_permissions.json prefer_legacy_ssm_policy = false prefer_legacy_service_policy = false scheduled_actions = var.scheduled_actions context = module.this.context } data "aws_iam_policy_document" "minimal_s3_permissions" { statement { sid = "AllowS3OperationsOnElasticBeanstalkBuckets" actions = [ "s3:ListAllMyBuckets", "s3:GetBucketLocation" ] resources = ["*"] } } ``` ## Variables ### Required Variables
`application_subnets` (`list(string)`) required
List of subnets to place EC2 instances
`elastic_beanstalk_application_name` (`string`) required
Elastic Beanstalk application name
`region` (`string`) required
AWS region
`solution_stack_name` (`string`) required
Elastic Beanstalk stack, e.g. Docker, Go, Node, Java, IIS. For more info, see https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platforms-supported.html
`vpc_id` (`string`) required
ID of the VPC in which to provision the AWS resources
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`additional_settings` optional
Additional Elastic Beanstalk setttings. For full list of options, see https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/command-options-general.html **Type:** ```hcl list(object({ namespace = string name = string value = string })) ``` **Default value:** `[ ]`
`alb_zone_id` (`map(string)`) optional
ALB zone id **Default value:** ```hcl { "af-south-1": "Z1EI3BVKMKK4AM", "ap-east-1": "ZPWYUBWRU171A", "ap-northeast-1": "Z1R25G3KIG2GBW", "ap-northeast-2": "Z3JE5OI70TWKCP", "ap-south-1": "Z18NTBI3Y7N9TZ", "ap-southeast-1": "Z16FZ9L249IFLT", "ap-southeast-2": "Z2PCDNR3VC2G1N", "ca-central-1": "ZJFCZL7SSZB5I", "eu-central-1": "Z1FRNW7UH4DEZJ", "eu-north-1": "Z23GO28BZ5AETM", "eu-south-1": "Z10VDYYOA2JFKM", "eu-west-1": "Z2NYPWQ7DFZAZH", "eu-west-2": "Z1GKAAAUGATPF1", "eu-west-3": "Z3Q77PNBQS71R4", "me-south-1": "Z2BBTEKR2I36N2", "sa-east-1": "Z10X7K2B4QSOFV", "us-east-1": "Z117KPS5GTRQ2G", "us-east-2": "Z14LCN19Q5QHIC", "us-gov-east-1": "Z2NIFVYYW2VKV1", "us-gov-west-1": "Z31GFT0UA1I2HV", "us-west-1": "Z1LQECGX5PH1X", "us-west-2": "Z38NKT9BP95V3O" } ```
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP addresses. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`ami_id` (`string`) optional
The id of the AMI to associate with the Amazon EC2 instances **Default value:** `null`
`application_port` (`number`) optional
Port application is listening on **Default value:** `80`
`associate_public_ip_address` (`bool`) optional
Whether to associate public IP addresses to the instances **Default value:** `false`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`autoscale_lower_bound` (`number`) optional
Minimum level of autoscale metric to remove an instance **Default value:** `20`
`autoscale_lower_increment` (`number`) optional
How many Amazon EC2 instances to remove when performing a scaling activity. **Default value:** `-1`
`autoscale_max` (`number`) optional
Maximum instances to launch **Default value:** `3`
`autoscale_measure_name` (`string`) optional
Metric used for your Auto Scaling trigger **Default value:** `"CPUUtilization"`
`autoscale_min` (`number`) optional
Minumum instances to launch **Default value:** `2`
`autoscale_statistic` (`string`) optional
Statistic the trigger should use, such as Average **Default value:** `"Average"`
`autoscale_unit` (`string`) optional
Unit for the trigger measurement, such as Bytes **Default value:** `"Percent"`
`autoscale_upper_bound` (`number`) optional
Maximum level of autoscale metric to add an instance **Default value:** `80`
`autoscale_upper_increment` (`number`) optional
How many Amazon EC2 instances to add when performing a scaling activity **Default value:** `1`
`availability_zone_selector` (`string`) optional
Availability Zone selector **Default value:** `"Any 2"`
`create_security_group` (`bool`) optional
Set `true` to create and configure a Security Group for the cluster. **Default value:** `true`
`deployment_batch_size` (`number`) optional
Percentage or fixed number of Amazon EC2 instances in the Auto Scaling group on which to simultaneously perform deployments. Valid values vary per deployment_batch_size_type setting **Default value:** `1`
`deployment_batch_size_type` (`string`) optional
The type of number that is specified in deployment_batch_size_type **Default value:** `"Fixed"`
`deployment_ignore_health_check` (`bool`) optional
Do not cancel a deployment due to failed health checks **Default value:** `false`
`deployment_policy` (`string`) optional
Use the DeploymentPolicy option to set the deployment type. The following values are supported: `AllAtOnce`, `Rolling`, `RollingWithAdditionalBatch`, `Immutable`, `TrafficSplitting` **Default value:** `"Rolling"`
`deployment_timeout` (`number`) optional
Number of seconds to wait for an instance to complete executing commands **Default value:** `600`
`description` (`string`) optional
Short description of the Environment **Default value:** `""`
`dns_subdomain` (`string`) optional
The subdomain to create on Route53 for the EB environment. For the subdomain to be created, the `dns_zone_id` variable must be set as well **Default value:** `""`
`dns_zone_id` (`string`) optional
Route53 parent zone ID. The module will create sub-domain DNS record in the parent zone for the EB environment **Default value:** `""`
`elb_scheme` (`string`) optional
Specify `internal` if you want to create an internal load balancer in your Amazon VPC so that your Elastic Beanstalk application cannot be accessed from outside your Amazon VPC **Default value:** `"public"`
`enable_capacity_rebalancing` (`bool`) optional
Specifies whether to enable the Capacity Rebalancing feature for Spot Instances in your Auto Scaling Group **Default value:** `false`
`enable_loadbalancer_logs` (`bool`) optional
Whether to enable Load Balancer Logging to the S3 bucket. **Default value:** `true`
`enable_log_publication_control` (`bool`) optional
Copy the log files for your application's Amazon EC2 instances to the Amazon S3 bucket associated with your application **Default value:** `false`
`enable_spot_instances` (`bool`) optional
Enable Spot Instance requests for your environment **Default value:** `false`
`enable_stream_logs` (`bool`) optional
Whether to create groups in CloudWatch Logs for proxy and deployment logs, and stream logs from each instance in your environment **Default value:** `false`
`enhanced_reporting_enabled` (`bool`) optional
Whether to enable "enhanced" health reporting for this environment. If false, "basic" reporting is used. When you set this to false, you must also set `enable_managed_actions` to false **Default value:** `true`
`env_vars` (`map(string)`) optional
Map of custom ENV variables to be provided to the application running on Elastic Beanstalk, e.g. env_vars = \{ DB_USER = 'admin' DB_PASS = 'xxxxxx' \} **Default value:** `{ }`
`environment_type` (`string`) optional
Environment type, e.g. 'LoadBalanced' or 'SingleInstance'. If setting to 'SingleInstance', `rolling_update_type` must be set to 'Time', `updating_min_in_service` must be set to 0, and `loadbalancer_subnets` will be unused (it applies to the ELB, which does not exist in SingleInstance environments) **Default value:** `"LoadBalanced"`
`extended_ec2_policy_document` (`string`) optional
Extensions or overrides for the IAM role assigned to EC2 instances **Default value:** `""`
`force_destroy` (`bool`) optional
Force destroy the S3 bucket for load balancer logs **Default value:** `false`
`health_streaming_delete_on_terminate` (`bool`) optional
Whether to delete the log group when the environment is terminated. If false, the health data is kept RetentionInDays days. **Default value:** `false`
`health_streaming_enabled` (`bool`) optional
For environments with enhanced health reporting enabled, whether to create a group in CloudWatch Logs for environment health and archive Elastic Beanstalk environment health data. For information about enabling enhanced health, see aws:elasticbeanstalk:healthreporting:system. **Default value:** `false`
`health_streaming_retention_in_days` (`number`) optional
The number of days to keep the archived health data before it expires. **Default value:** `7`
`healthcheck_healthy_threshold_count` (`number`) optional
The number of consecutive successful requests before Elastic Load Balancing changes the instance health status **Default value:** `3`
`healthcheck_httpcodes_to_match` (`list(string)`) optional
List of HTTP codes that indicate that an instance is healthy. Note that this option is only applicable to environments with a network or application load balancer **Default value:** ```hcl [ "200" ] ```
`healthcheck_interval` (`number`) optional
The interval of time, in seconds, that Elastic Load Balancing checks the health of the Amazon EC2 instances of your application **Default value:** `10`
`healthcheck_timeout` (`number`) optional
The amount of time, in seconds, to wait for a response during a health check. Note that this option is only applicable to environments with an application load balancer **Default value:** `5`
`healthcheck_unhealthy_threshold_count` (`number`) optional
The number of consecutive unsuccessful requests before Elastic Load Balancing changes the instance health status **Default value:** `3`
`healthcheck_url` (`string`) optional
Application Health Check URL. Elastic Beanstalk will call this URL to check the health of the application running on EC2 instances **Default value:** `"/"`
`http_listener_enabled` (`bool`) optional
Enable port 80 (http) **Default value:** `true`
`instance_refresh_enabled` (`bool`) optional
Enable weekly instance replacement. **Default value:** `true`
`instance_type` (`string`) optional
Instances type **Default value:** `"t2.micro"`
`keypair` (`string`) optional
Name of SSH key that will be deployed on Elastic Beanstalk and DataPipeline instance. The key should be present in AWS **Default value:** `""`
`loadbalancer_certificate_arn` (`string`) optional
Load Balancer SSL certificate ARN. The certificate must be present in AWS Certificate Manager **Default value:** `""`
`loadbalancer_connection_idle_timeout` (`number`) optional
Classic load balancer only: Number of seconds that the load balancer waits for any data to be sent or received over the connection. If no data has been sent or received after this time period elapses, the load balancer closes the connection. **Default value:** `60`
`loadbalancer_crosszone` (`bool`) optional
Configure the classic load balancer to route traffic evenly across all instances in all Availability Zones rather than only within each zone. **Default value:** `true`
`loadbalancer_is_shared` (`bool`) optional
Flag to create a shared application loadbalancer. Only when loadbalancer_type = "application" https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-alb-shared.html **Default value:** `false`
`loadbalancer_managed_security_group` (`string`) optional
Load balancer managed security group **Default value:** `""`
`loadbalancer_redirect_http_to_https` (`bool`) optional
Redirect HTTP traffic to HTTPS listener **Default value:** `false`
`loadbalancer_redirect_http_to_https_host` (`string`) optional
Defines the host for the HTTP to HTTPS redirection rule **Default value:** `"#{host}"`
`loadbalancer_redirect_http_to_https_path_pattern` (`list(string)`) optional
Defines the path pattern for the HTTP to HTTPS redirection rule **Default value:** ```hcl [ "*" ] ```
`loadbalancer_redirect_http_to_https_port` (`string`) optional
Defines the port for the HTTP to HTTPS redirection rule **Default value:** `"443"`
`loadbalancer_redirect_http_to_https_priority` (`number`) optional
Defines the priority for the HTTP to HTTPS redirection rule **Default value:** `1`
`loadbalancer_redirect_http_to_https_status_code` (`string`) optional
The redirect status code **Default value:** `"HTTP_301"`
`loadbalancer_security_groups` (`list(string)`) optional
Load balancer security groups **Default value:** `[ ]`
`loadbalancer_ssl_policy` (`string`) optional
Specify a security policy to apply to the listener. This option is only applicable to environments with an application load balancer **Default value:** `""`
`loadbalancer_subnets` (`list(string)`) optional
List of subnets to place Elastic Load Balancer **Default value:** `[ ]`
`loadbalancer_type` (`string`) optional
Load Balancer type, e.g. 'application' or 'classic' **Default value:** `"classic"`
`logs_delete_on_terminate` (`bool`) optional
Whether to delete the log groups when the environment is terminated. If false, the logs are kept RetentionInDays days **Default value:** `false`
`logs_retention_in_days` (`number`) optional
The number of days to keep log events before they expire. **Default value:** `7`
`managed_actions_enabled` (`bool`) optional
Enable managed platform updates. When you set this to true, you must also specify a `PreferredStartTime` and `UpdateLevel` **Default value:** `true`
`prefer_legacy_service_policy` (`bool`) optional
Whether to use AWSElasticBeanstalkService (deprecated) or AWSElasticBeanstalkManagedUpdatesCustomerRolePolicy policy **Default value:** `true`
`prefer_legacy_ssm_policy` (`bool`) optional
Whether to use AmazonEC2RoleforSSM (will soon be deprecated) or AmazonSSMManagedInstanceCore policy **Default value:** `true`
`preferred_start_time` (`string`) optional
Configure a maintenance window for managed actions in UTC **Default value:** `"Sun:10:00"`
`rolling_update_enabled` (`bool`) optional
Whether to enable rolling update **Default value:** `true`
`rolling_update_type` (`string`) optional
`Health` or `Immutable`. Set it to `Immutable` to apply the configuration change to a fresh group of instances **Default value:** `"Health"`
`root_volume_iops` (`number`) optional
The IOPS of the EBS root volume (only applies for gp3 and io1 types) **Default value:** `null`
`root_volume_size` (`number`) optional
The size of the EBS root volume **Default value:** `8`
`root_volume_throughput` (`number`) optional
The type of the EBS root volume (only applies for gp3 type) **Default value:** `null`
`root_volume_type` (`string`) optional
The type of the EBS root volume **Default value:** `"gp2"`
`s3_bucket_access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`s3_bucket_versioning_enabled` (`bool`) optional
When set to 'true' the s3 origin bucket will have versioning enabled **Default value:** `true`
`scheduled_actions` optional
Define a list of scheduled actions **Type:** ```hcl list(object({ name = string minsize = string maxsize = string desiredcapacity = string starttime = string endtime = string recurrence = string suspend = bool })) ``` **Default value:** `[ ]`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. We recommend setting this `true` on new security groups, but default it to `false` because `true` will cause existing security groups to be replaced, possibly requiring the resource to be deleted and recreated. Note that changing this value will always cause the security group to be replaced. **Default value:** `false`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Security Group for the EB environment"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`shared_loadbalancer_arn` (`string`) optional
ARN of the shared application load balancer. Only when loadbalancer_type = "application". **Default value:** `""`
`spot_fleet_on_demand_above_base_percentage` (`number`) optional
The percentage of On-Demand Instances as part of additional capacity that your Auto Scaling group provisions beyond the SpotOnDemandBase instances. This option is relevant only when enable_spot_instances is true. **Default value:** `-1`
`spot_fleet_on_demand_base` (`number`) optional
The minimum number of On-Demand Instances that your Auto Scaling group provisions before considering Spot Instances as your environment scales up. This option is relevant only when enable_spot_instances is true. **Default value:** `0`
`spot_max_price` (`number`) optional
The maximum price per unit hour, in US$, that you're willing to pay for a Spot Instance. This option is relevant only when enable_spot_instances is true. Valid values are between 0.001 and 20.0 **Default value:** `-1`
`ssh_listener_enabled` (`bool`) optional
Enable SSH port **Default value:** `false`
`ssh_listener_port` (`number`) optional
SSH port **Default value:** `22`
`tier` (`string`) optional
Elastic Beanstalk Environment tier, 'WebServer' or 'Worker' **Default value:** `"WebServer"`
`update_level` (`string`) optional
The highest level of update to apply with managed platform updates **Default value:** `"minor"`
`updating_max_batch` (`number`) optional
Maximum number of instances to update at once **Default value:** `1`
`updating_min_in_service` (`number`) optional
Minimum number of instances in service during update **Default value:** `1`
`version_label` (`string`) optional
Elastic Beanstalk Application version to deploy **Default value:** `""`
`wait_for_ready_timeout` (`string`) optional
The maximum duration to wait for the Elastic Beanstalk Environment to be in a ready state before timing out **Default value:** `"20m"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`all_settings`
List of all option settings configured in the environment. These are a combination of default settings and their overrides from setting in the configuration
`application`
The Elastic Beanstalk Application for this environment
`autoscaling_groups`
The autoscaling groups used by this environment
`ec2_instance_profile_role_name`
Instance IAM role name
`ec2_service_role_arn`
EC2 service IAM role ARN
`elb_zone_id`
ELB zone ID
`endpoint`
Fully qualified DNS name for the environment
`hostname`
DNS hostname
`id`
ID of the Elastic Beanstalk environment
`instances`
Instances used by this environment
`launch_configurations`
Launch configurations in use by this environment
`load_balancer_log_bucket`
Name of bucket where Load Balancer logs are stored (if enabled)
`load_balancers`
Elastic Load Balancers in use by this environment
`name`
Name of the Elastic Beanstalk environment
`queues`
SQS queues in use by this environment
`security_group_arn`
Elastic Beanstalk environment Security Group ARN
`security_group_id`
Elastic Beanstalk environment Security Group ID
`security_group_name`
Elastic Beanstalk environment Security Group name
`setting`
Settings specifically set for this environment
`tier`
The environment tier
`triggers`
Autoscaling triggers in use by this environment
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0` - `random`, version: `>= 3.5.1` ### Providers - `aws`, version: `>= 4.0` - `random`, version: `>= 3.5.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_security_group` | 1.0.1 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/1.0.1) | n/a `dns_hostname` | 0.12.2 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.12.2) | n/a `elb_logs` | 0.20.0 | [`cloudposse/lb-s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/lb-s3-bucket/aws/0.20.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_elastic_beanstalk_environment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elastic_beanstalk_environment) (resource) - [`aws_iam_instance_profile.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.ecr_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.elastic_beanstalk_multi_container_docker`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.enhanced_health`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ssm_automation`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ssm_ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.web_tier`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.worker_tier`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lb_listener_rule.redirect_http_to_https`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_rule) (resource) - [`aws_ssm_activation.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_activation) (resource) - [`random_string.elb_logs_suffix`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/string) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.extended`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_lb_listener.http`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lb_listener) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## elasticache-memcached # Module: `elasticache-memcached` Terraform module to provision an [`ElastiCache`](https://aws.amazon.com/elasticache/) Memcached Cluster ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-elasticache-memcached/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-elasticache-memcached/tree/main/test). ```hcl provider "aws" { region = var.region } module "this" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "172.16.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false context = module.this.context } module "memcached" { source = "cloudposse/elasticache-memcached/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id allowed_security_groups = [module.vpc.vpc_default_security_group_id] subnets = module.subnets.private_subnet_ids cluster_size = var.cluster_size instance_type = var.instance_type engine_version = var.engine_version apply_immediately = true zone_id = var.zone_id elasticache_parameter_group_family = var.elasticache_parameter_group_family context = module.this.context } ``` ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-elasticache-memcached/tree/main/examples/complete) to see how to use this module. ## Variables ### Required Variables
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`alarm_actions` (`list(string)`) optional
Alarm actions **Default value:** `[ ]`
`alarm_cpu_threshold_percent` (`number`) optional
CPU threshold alarm level **Default value:** `75`
`alarm_memory_threshold_bytes` (`number`) optional
Alarm memory threshold bytes **Default value:** `10000000`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP address. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_cidr_blocks` (`list(string)`) optional
A list of IPv6 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_prefix_list_ids` (`list(string)`) optional
A list of IPv6 Prefix Lists IDs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
DEPRECATED: Use `allowed_security_group_ids` instead. **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any database modifications are applied immediately, or during the next maintenance window **Default value:** `true`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`availability_zone` (`string`) optional
The Availability Zone of the cluster. az_mode must be set to single-az when used. **Default value:** `""`
`availability_zones` (`list(string)`) optional
List of Availability Zones for the cluster. az_mode must be set to cross-az when used. **Default value:** `[ ]`
`az_mode` (`string`) optional
Enable or disable multiple AZs, eg: single-az or cross-az **Default value:** `"single-az"`
`cloudwatch_metric_alarms_enabled` (`bool`) optional
Boolean flag to enable/disable CloudWatch metrics alarms **Default value:** `false`
`cluster_size` (`number`) optional
Cluster size **Default value:** `1`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`dns_subdomain` (`string`) optional
The subdomain to use for the CNAME record. If not provided then the CNAME record will use var.name. **Default value:** `""`
`elasticache_parameter_group_family` (`string`) optional
ElastiCache parameter group family **Default value:** `"memcached1.5"`
`elasticache_subnet_group_name` (`string`) optional
Subnet group name for the ElastiCache instance **Default value:** `""`
`engine_version` (`string`) optional
Memcached engine version. For more info, see https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/supported-engine-versions.html **Default value:** `"1.5.16"`
`existing_security_groups` (`list(string)`) optional
DEPRECATED: Use `associated_security_group_ids` instead. Historical description: List of existing Security Group IDs to place the cluster into. Set `use_existing_security_groups` to `true` to enable using `existing_security_groups` as Security Groups for the cluster. **Default value:** `[ ]`
`instance_type` (`string`) optional
Elastic cache instance type **Default value:** `"cache.t2.micro"`
`maintenance_window` (`string`) optional
Maintenance window **Default value:** `"wed:03:00-wed:04:00"`
`max_item_size` (`number`) optional
Max item size **Default value:** `10485760`
`network_type` (`string`) optional
The network type of the cluster. Valid values: ipv4, ipv6, dual_stack. **Default value:** `"ipv4"`
`notification_topic_arn` (`string`) optional
Notification topic arn **Default value:** `""`
`port` (`number`) optional
Memcached port **Default value:** `11211`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are upgrading this module and need to keep the existing security group from being replaced. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Security group for Elasticache Memcached"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`subnets` (`list(string)`) optional
AWS subnet ids **Default value:** `[ ]`
`transit_encryption_enabled` (`bool`) optional
Boolean flag to enable transit encryption (requires Memcached version 1.6.12+) **Default value:** `false`
`use_existing_security_groups` (`bool`) optional
DEPRECATED: Use `create_security_group` instead. Historical description: Flag to enable/disable creation of Security Group in the module. Set to `true` to disable Security Group creation and provide a list of existing security Group IDs in `existing_security_groups` to place the cluster into. Historical default: `false` **Default value:** `null`
`vpc_id` (`string`) optional
VPC ID **Default value:** `""`
`zone_id` (`string`) optional
Route53 DNS Zone ID **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cluster_address`
Cluster address
`cluster_configuration_endpoint`
Cluster configuration endpoint
`cluster_id`
Cluster ID
`cluster_urls`
Cluster URLs
`hostname`
Cluster hostname
`security_group_arn`
The ARN of the created security group
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.16` - `null`, version: `>= 3.0` ### Providers - `aws`, version: `>= 5.16` - `null`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `dns` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.cache_cpu`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.cache_memory`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_elasticache_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_cluster) (resource) - [`aws_elasticache_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_parameter_group) (resource) - [`aws_elasticache_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_subnet_group) (resource) - [`null_resource.cluster_urls`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) ## Data Sources The following data sources are used by this module: --- ## elasticache-redis(Elasticache-redis) # Module: `elasticache-redis` Terraform module to provision an [`ElastiCache`](https://aws.amazon.com/elasticache/) Redis Cluster or Serverless instance. ## Usage _**Disruptive changes introduced at version 0.41.0**. If upgrading from an earlier version, see [migration notes](https://github.com/cloudposse/terraform-aws-elasticache-redis/blob/master/docs/migration-notes-0.41.0.md) for details._ Note that this uses secure defaults. One of the ways this module can trip users up is with `transit_encryption_enabled` which is `true` by default. With this enabled, one does not simply `redis-cli` in without setting up an `stunnel`. Amazon provides [good documentation on how to connect with it enabled](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/in-transit-encryption.html#connect-tls). If this is not desired behavior, set `transit_encryption_enabled=false`. This module creates, by default, a new security group for the Elasticache Redis Cluster / Serverless Instance. When a configuration change (for example, a different security group name) cannot be applied to the security group, Terraform will replace that security group with a new one with the new configuration. In order to allow Terraform to fully manage the security group, you should not place any other resources in (or associate any other resources with) the security group this module creates. Also, in order to keep things from breaking when this module replaces the security group, you should not reference the created security group anywhere else (such as in rules in other security groups). If you want to associate the cluster with a more stable security group that you can reference elsewhere, create that security group outside this module (perhaps with [terraform-aws-security-group](https://github.com/cloudposse/terraform-aws-security-group)) and pass the security group ID in via `associated_security_group_ids`. **Note about `zone_id`**: Previously, `zone_id` was a string. This caused problems (see [#82](https://github.com/cloudposse/terraform-aws-elasticache-redis/issues/82)). Now `zone_id` should be supplied as a `list(string)`, either empty or with exactly 1 zone ID in order to avoid the problem. For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-elasticache-redis/tree/main/examples/complete) or [examples/serverless](https://github.com/cloudposse/terraform-aws-elasticache-redis/tree/main/examples/serverless). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-elasticache-redis/tree/main/test). ```hcl provider "aws" { region = var.region } module "this" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "172.16.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = true nat_instance_enabled = false context = module.this.context } module "redis" { source = "cloudposse/elasticache-redis/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones zone_id = var.zone_id vpc_id = module.vpc.vpc_id allowed_security_group_ids = [module.vpc.vpc_default_security_group_id] subnets = module.subnets.private_subnet_ids cluster_size = var.cluster_size instance_type = var.instance_type apply_immediately = true automatic_failover_enabled = false engine_version = var.engine_version family = var.family at_rest_encryption_enabled = var.at_rest_encryption_enabled transit_encryption_enabled = var.transit_encryption_enabled parameter = [ { name = "notify-keyspace-events" value = "lK" } ] context = module.this.context } ``` ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-elasticache-redis/tree/main/examples/complete) or [serverless example](https://github.com/cloudposse/terraform-aws-elasticache-redis/tree/main/examples/serverless) to see how to use this module. ## Variables ### Required Variables
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule . **Default value:** `[ ]`
`alarm_actions` (`list(string)`) optional
Alarm action list **Default value:** `[ ]`
`alarm_cpu_threshold_percent` (`number`) optional
CPU threshold alarm level **Default value:** `75`
`alarm_memory_threshold_bytes` (`number`) optional
Ram threshold alarm level **Default value:** `10000000`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP address. If this is false and no egress rules are otherwise specified, then no egress will be allowed. Defaults to `true` unless the deprecated `egress_cidr_blocks` is provided and is not `["0.0.0.0/0"]`, in which case defaults to `false`. **Default value:** `null`
`allowed_cidr_blocks` (`list(string)`) optional
DEPRECATED: Use `additional_security_group_rules` instead. Historical description: List of CIDR blocks that are allowed ingress to the cluster's Security Group created in the module **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
DEPRECATED: Use `allowed_security_group_ids` instead. **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Apply changes immediately **Default value:** `true`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must provide all the required access. **Default value:** `[ ]`
`at_rest_encryption_enabled` (`bool`) optional
Enable encryption at rest **Default value:** `false`
`auth_token` (`string`) optional
Auth token for password protecting redis, `transit_encryption_enabled` must be set to `true`. Password must be longer than 16 chars **Default value:** `null`
`auth_token_update_strategy` (`string`) optional
Strategy to use when updating the auth_token. Valid values are `SET`, `ROTATE`, and `DELETE`. Defaults to `ROTATE`. **Default value:** `"ROTATE"`
`auto_minor_version_upgrade` (`bool`) optional
Specifies whether minor version engine upgrades will be applied automatically to the underlying Cache Cluster instances during the maintenance window. Only supported if the engine version is 6 or higher. **Default value:** `null`
`automatic_failover_enabled` (`bool`) optional
Automatic failover (Not available for T1/T2 instances) **Default value:** `false`
`availability_zones` (`list(string)`) optional
Availability zone IDs **Default value:** `[ ]`
`cloudwatch_metric_alarms_enabled` (`bool`) optional
Boolean flag to enable/disable CloudWatch metrics alarms **Default value:** `false`
`cluster_mode_enabled` (`bool`) optional
Flag to enable/disable creation of a native redis cluster. `automatic_failover_enabled` must be set to `true`. Only 1 `cluster_mode` block is allowed **Default value:** `false`
`cluster_mode_num_node_groups` (`number`) optional
Number of node groups (shards) for this Redis replication group. Changing this number will trigger an online resizing operation before other settings modifications **Default value:** `0`
`cluster_mode_replicas_per_node_group` (`number`) optional
Number of replica nodes in each node group. Valid values are 0 to 5. Changing this number will force a new resource **Default value:** `0`
`cluster_size` (`number`) optional
Number of nodes in cluster. *Ignored when `cluster_mode_enabled` == `true`* **Default value:** `1`
`create_parameter_group` (`bool`) optional
Whether new parameter group should be created. Set to false if you want to use existing parameter group **Default value:** `true`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`data_tiering_enabled` (`bool`) optional
Enables data tiering. Data tiering is only supported for replication groups using the r6gd node type. **Default value:** `false`
`description` (`string`) optional
Description of elasticache replication group **Default value:** `null`
`dns_subdomain` (`string`) optional
The subdomain to use for the CNAME record. If not provided then the CNAME record will use var.name. **Default value:** `""`
`egress_cidr_blocks` (`list(any)`) optional
DEPRECATED: Use `allow_all_egress` and `additional_security_group_rules` instead. Historical description: Outbound traffic address. Historical default: ["0.0.0.0/0"] **Default value:** `null`
`elasticache_subnet_group_name` (`string`) optional
Subnet group name for the ElastiCache instance **Default value:** `""`
`engine` (`string`) optional
Name of the cache engine **Default value:** `"redis"`
`engine_version` (`string`) optional
Version number of the cache engine **Default value:** `"4.0.10"`
`existing_security_groups` (`list(string)`) optional
DEPRECATED: Use `associated_security_group_ids` instead. Historical description: List of existing Security Group IDs to place the cluster into. Set `use_existing_security_groups` to `true` to enable using `existing_security_groups` as Security Groups for the cluster. **Default value:** `[ ]`
`family` (`string`) optional
The family of the ElastiCache parameter group **Default value:** `"redis4.0"`
`final_snapshot_identifier` (`string`) optional
The name of your final node group (shard) snapshot. ElastiCache creates the snapshot from the primary node in the cluster. If omitted, no final snapshot will be made. **Default value:** `null`
`global_replication_group_id` (`string`) optional
The ID of the global replication group to which this replication group should belong. If this parameter is specified, the replication group is added to the specified global replication group as a secondary replication group; otherwise, the replication group is not part of any global replication group. If global_replication_group_id is set, the num_node_groups parameter cannot be set. **Default value:** `null`
`inline_rules_enabled` (`bool`) optional
NOT RECOMMENDED. Create rules "inline" instead of as separate `aws_security_group_rule` resources. See [#20046](https://github.com/hashicorp/terraform-provider-aws/issues/20046) for one of several issues with inline rules. See [this post](https://github.com/hashicorp/terraform-provider-aws/pull/9032#issuecomment-639545250) for details on the difference between inline rules and rule resources. **Default value:** `false`
`instance_type` (`string`) optional
Elastic cache instance type **Default value:** `"cache.t2.micro"`
`kms_key_id` (`string`) optional
The ARN of the key that you wish to use if encrypting at rest. If not supplied, uses service managed encryption. `at_rest_encryption_enabled` must be set to `true` **Default value:** `null`
`log_delivery_configuration` (`list(map(any))`) optional
The log_delivery_configuration block allows the streaming of Redis SLOWLOG or Redis Engine Log to CloudWatch Logs or Kinesis Data Firehose. Max of 2 blocks. **Default value:** `[ ]`
`maintenance_window` (`string`) optional
Maintenance window **Default value:** `"wed:03:00-wed:04:00"`
`multi_az_enabled` (`bool`) optional
Multi AZ (Automatic Failover must also be enabled. If Cluster Mode is enabled, Multi AZ is on by default, and this setting is ignored) **Default value:** `false`
`network_type` (`string`) optional
The network type of the cluster. Valid values: ipv4, ipv6, dual_stack. **Default value:** `"ipv4"`
`notification_topic_arn` (`string`) optional
Notification topic arn **Default value:** `""`
`ok_actions` (`list(string)`) optional
The list of actions to execute when this alarm transitions into an OK state from any other state. Each action is specified as an Amazon Resource Number (ARN) **Default value:** `[ ]`
`parameter` optional
A list of Redis parameters to apply. Note that parameters may differ from one Redis family to another **Type:** ```hcl list(object({ name = string value = string })) ``` **Default value:** `[ ]`
`parameter_group_description` (`string`) optional
Managed by Terraform **Default value:** `null`
`parameter_group_name` (`string`) optional
Override the default parameter group name **Default value:** `null`
`port` (`number`) optional
Port number on which the cache nodes will accept connections **Default value:** `6379`
`preserve_security_group_id` (`bool`) optional
When `false` and `create_before_destroy` is `true`, changes to security group rules cause a new security group to be created with the new rules, and the existing security group is then replaced with the new one, eliminating any service interruption. When `true` or when changing the value (from `false` to `true` or from `true` to `false`), existing security group rules will be deleted before new ones are created, resulting in a service interruption, but preserving the security group itself. **NOTE:** Setting this to `true` does not guarantee the security group will never be replaced, it only keeps changes to the security group rules from triggering a replacement. See the README for further discussion. **Default value:** `false`
`replication_group_id` (`string`) optional
Replication group ID with the following constraints: A name must contain from 1 to 20 alphanumeric characters or hyphens. The first character must be a letter. A name cannot end with a hyphen or contain two consecutive hyphens. **Default value:** `""`
`revoke_rules_on_delete` (`bool`) optional
Instruct Terraform to revoke all of the Security Group's attached ingress and egress rules before deleting the security group itself. This is normally not needed. **Default value:** `false`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are upgrading this module and need to keep the existing security group from being replaced. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. Set this to `null` to maintain parity with releases \<= `0.34.0`. **Default value:** `"Security group for Elasticache Redis"`
`security_group_name` (`list(string)`) optional
The name to assign to the security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`serverless_cache_usage_limits` (`map(any)`) optional
The usage limits for the serverless cache **Default value:** `{ }`
`serverless_enabled` (`bool`) optional
Flag to enable/disable creation of a serverless redis cluster **Default value:** `false`
`serverless_major_engine_version` (`string`) optional
The major version of the engine to use for the serverless cluster **Default value:** `"7"`
`serverless_snapshot_arns_to_restore` (`list(string)`) optional
The list of ARN(s) of the snapshot that the new serverless cache will be created from. Available for Redis only. **Default value:** `[ ]`
`serverless_snapshot_time` (`string`) optional
The daily time that snapshots will be created from the serverless cache. **Default value:** `"06:00"`
`serverless_user_group_id` (`string`) optional
User Group ID to associate with the replication group **Default value:** `null`
`snapshot_arns` (`list(string)`) optional
A single-element string list containing an Amazon Resource Name (ARN) of a Redis RDB snapshot file stored in Amazon S3. Example: arn:aws:s3:::my_bucket/snapshot1.rdb **Default value:** `[ ]`
`snapshot_name` (`string`) optional
The name of a snapshot from which to restore data into the new node group. Changing the snapshot_name forces a new resource. **Default value:** `null`
`snapshot_retention_limit` (`number`) optional
The number of days for which ElastiCache will retain automatic cache cluster snapshots before deleting them. Set a value to enable automated backups on the cache. If the value of snapshot_retention_limit is set to zero (0), backups are turned off. **Default value:** `0`
`snapshot_window` (`string`) optional
The daily time range (in UTC) during which ElastiCache will begin taking a daily snapshot of your cache cluster. **Default value:** `"06:30-07:30"`
`subnets` (`list(string)`) optional
Subnet IDs **Default value:** `[ ]`
`target_security_group_id` (`list(string)`) optional
The ID of an existing Security Group to which Security Group rules will be assigned. The Security Group's name and description will not be changed. Not compatible with `inline_rules_enabled` or `revoke_rules_on_delete`. If not provided (the default), this module will create a security group. **Default value:** `[ ]`
`transit_encryption_enabled` (`bool`) optional
Set `true` to enable encryption in transit. Forced `true` if `var.auth_token` is set. If this is enabled, use the [following guide](https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/in-transit-encryption.html#connect-tls) to access redis. **Default value:** `true`
`transit_encryption_mode` (`string`) optional
A setting that enables clients to migrate to in-transit encryption with no downtime. Valid values are `preferred` and `required`. When enabling encryption on an existing replication group, this must first be set to `preferred` before setting it to `required` in a subsequent apply. See the TransitEncryptionMode field in the [CreateReplicationGroup](https://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_CreateReplicationGroup.html) API documentation for additional details." **Default value:** `null`
`use_existing_security_groups` (`bool`) optional
DEPRECATED: Use `create_security_group` instead. Historical description: Flag to enable/disable creation of Security Group in the module. Set to `true` to disable Security Group creation and provide a list of existing security Group IDs in `existing_security_groups` to place the cluster into. Historical default: `false` **Default value:** `null`
`user_group_ids` (`list(string)`) optional
User Group ID to associate with the replication group **Default value:** `null`
`zone_id` (`any`) optional
Route53 DNS Zone ID as list of string (0 or 1 items). If empty, no custom DNS name will be published. If the list contains a single Zone ID, a custom DNS name will be pulished in that zone. Can also be a plain string, but that use is DEPRECATED because of Terraform issues. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Elasticache Replication Group ARN
`cluster_enabled`
Indicates if cluster mode is enabled
`endpoint`
Redis primary, configuration or serverless endpoint , whichever is appropriate for the given configuration
`engine_version_actual`
The running version of the cache engine
`host`
Redis hostname
`id`
Redis cluster ID
`member_clusters`
Redis cluster members
`port`
Redis port
`reader_endpoint_address`
The address of the endpoint for the reader node in the replication group, if the cluster mode is disabled or serverless is being used.
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
`serverless_enabled`
Indicates if serverless mode is enabled
`transit_encryption_mode`
The transit encryption mode of the replication group
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.73.0` ### Providers - `aws`, version: `>= 5.73.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `dns` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.cache_cpu`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.cache_memory`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_elasticache_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_parameter_group) (resource) - [`aws_elasticache_replication_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_replication_group) (resource) - [`aws_elasticache_serverless_cache.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_serverless_cache) (resource) - [`aws_elasticache_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticache_subnet_group) (resource) ## Data Sources The following data sources are used by this module: --- ## elasticsearch(Elasticsearch) # Module: `elasticsearch` Terraform module to provision an [`Elasticsearch`](https://aws.amazon.com/elasticsearch-service/) cluster with built-in integrations with [Kibana](https://aws.amazon.com/elasticsearch-service/kibana/) and [Logstash](https://aws.amazon.com/elasticsearch-service/logstash/). ## Introduction This module will create: - Elasticsearch cluster with the specified node count in the provided subnets in a VPC - Elasticsearch domain policy that accepts a list of IAM role ARNs from which to permit management traffic to the cluster - Security Group to control access to the Elasticsearch domain (inputs to the Security Group are other Security Groups or CIDRs blocks to be allowed to connect to the cluster) - DNS hostname record for Elasticsearch cluster (if DNS Zone ID is provided) - DNS hostname record for Kibana (if DNS Zone ID is provided) __NOTE:__ To enable [zone awareness](https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-managedomains.html#es-managedomains-zoneawareness) to deploy Elasticsearch nodes into two different Availability Zones, you need to set `zone_awareness_enabled` to `true` and provide two different subnets in `subnet_ids`. If you enable zone awareness for your domain, Amazon ES places an endpoint into two subnets. The subnets must be in different Availability Zones in the same region. If you don't enable zone awareness, Amazon ES places an endpoint into only one subnet. You also need to set `availability_zone_count` to `1`. ## Usage ```hcl module "elasticsearch" { source = "cloudposse/elasticsearch/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "es" dns_zone_id = "Z14EN2YD427LRQ" security_groups = ["sg-XXXXXXXXX", "sg-YYYYYYYY"] vpc_id = "vpc-XXXXXXXXX" subnet_ids = ["subnet-XXXXXXXXX", "subnet-YYYYYYYY"] zone_awareness_enabled = true elasticsearch_version = "6.5" instance_type = "t2.small.elasticsearch" instance_count = 4 ebs_volume_size = 10 iam_role_arns = ["arn:aws:iam::XXXXXXXXX:role/ops", "arn:aws:iam::XXXXXXXXX:role/dev"] iam_actions = ["es:ESHttpGet", "es:ESHttpPut", "es:ESHttpPost"] encrypt_at_rest_enabled = true kibana_subdomain_name = "kibana-es" advanced_options = { "rest.action.multi.allow_explicit_index" = "true" } } ``` ## Examples Here is a working example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-elasticsearch/tree/main/examples/complete) Here are automated tests for the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS): - [`test`](https://github.com/cloudposse/terraform-aws-elasticsearch/tree/main/test) ## Variables ### Required Variables
### Optional Variables
`access_policies` (`string`) optional
JSON string for the IAM policy document specifying the access policies for the domain. **Default value:** `""`
`advanced_options` (`map(string)`) optional
Key-value string pairs to specify advanced configuration options **Default value:** `{ }`
`advanced_security_options_anonymous_auth_enabled` (`bool`) optional
Whether Anonymous auth is enabled. Enables fine-grained access control on an existing domain **Default value:** `false`
`advanced_security_options_enabled` (`bool`) optional
AWS Elasticsearch Kibana enchanced security plugin enabling (forces new resource) **Default value:** `false`
`advanced_security_options_internal_user_database_enabled` (`bool`) optional
Whether to enable or not internal Kibana user database for ELK OpenDistro security plugin **Default value:** `false`
`advanced_security_options_master_user_arn` (`string`) optional
ARN of IAM user who is to be mapped to be Kibana master user (applicable if advanced_security_options_internal_user_database_enabled set to false) **Default value:** `""`
`advanced_security_options_master_user_name` (`string`) optional
Master user username (applicable if advanced_security_options_internal_user_database_enabled set to true) **Default value:** `""`
`advanced_security_options_master_user_password` (`string`) optional
Master user password (applicable if advanced_security_options_internal_user_database_enabled set to true) **Default value:** `""`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to connect to the cluster **Default value:** `[ ]`
`anonymous_iam_actions` (`list(string)`) optional
List of actions to allow for the anonymous (`*`) IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` **Default value:** `[ ]`
`auto_tune` optional
This object represents the auto_tune configuration. It contains the following filed: - enabled - Whether to enable autotune. - rollback_on_disable - Whether to roll back to default Auto-Tune settings when disabling Auto-Tune. - starting_time - Date and time at which to start the Auto-Tune maintenance schedule in RFC3339 format. Time should be in the future. - cron_schedule - A cron expression specifying the recurrence pattern for an Auto-Tune maintenance schedule. - duration - Autotune maintanance window duration time in hours. **Type:** ```hcl object({ enabled = bool rollback_on_disable = string starting_time = string cron_schedule = string duration = number }) ``` **Default value:** ```hcl { "cron_schedule": null, "duration": null, "enabled": false, "rollback_on_disable": "NO_ROLLBACK", "starting_time": null } ```
`automated_snapshot_start_hour` (`number`) optional
Hour at which automated snapshots are taken, in UTC **Default value:** `0`
`availability_zone_count` (`number`) optional
Number of Availability Zones for the domain to use. **Default value:** `2`
`aws_ec2_service_name` (`list(string)`) optional
AWS EC2 Service Name **Default value:** ```hcl [ "ec2.amazonaws.com" ] ```
`aws_service_type` (`string`) optional
The type of AWS service to deploy (`elasticsearch` or `opensearch`). **Default value:** `"elasticsearch"`
`cognito_authentication_enabled` (`bool`) optional
Whether to enable Amazon Cognito authentication with Kibana **Default value:** `false`
`cognito_iam_role_arn` (`string`) optional
ARN of the IAM role that has the AmazonESCognitoAccess policy attached **Default value:** `""`
`cognito_identity_pool_id` (`string`) optional
The ID of the Cognito Identity Pool to use **Default value:** `""`
`cognito_user_pool_id` (`string`) optional
The ID of the Cognito User Pool to use **Default value:** `""`
`cold_storage_enabled` (`bool`) optional
Enables cold storage support. **Default value:** `false`
`create_elasticsearch_user_role` (`bool`) optional
Whether to create an IAM role for Users/EC2 to assume to access the Elasticsearch domain. Set it to `false` if you already manage access through other means. **Default value:** `true`
`create_iam_service_linked_role` (`bool`) optional
Whether to create `AWSServiceRoleForAmazonElasticsearchService` service-linked role. Set it to `false` if you already have an ElasticSearch cluster created in the AWS account and AWSServiceRoleForAmazonElasticsearchService already exists. See https://github.com/terraform-providers/terraform-provider-aws/issues/5218 for more info **Default value:** `true`
`create_security_group` (`bool`) optional
Whether to create a dedicated security group for the Elasticsearch domain. Set it to `false` if you already have security groups that you want to attach to the domain and specify them in the `security_groups` variable. **Default value:** `true`
`custom_endpoint` (`string`) optional
Fully qualified domain for custom endpoint. **Default value:** `""`
`custom_endpoint_certificate_arn` (`string`) optional
ACM certificate ARN for custom endpoint. **Default value:** `""`
`custom_endpoint_enabled` (`bool`) optional
Whether to enable custom endpoint for the Elasticsearch domain. **Default value:** `false`
`dedicated_master_count` (`number`) optional
Number of dedicated master nodes in the cluster **Default value:** `0`
`dedicated_master_enabled` (`bool`) optional
Indicates whether dedicated master nodes are enabled for the cluster **Default value:** `false`
`dedicated_master_type` (`string`) optional
Instance type of the dedicated master nodes in the cluster **Default value:** `"t2.small.elasticsearch"`
`dns_zone_id` (`string`) optional
Route53 DNS Zone ID to add hostname records for Elasticsearch domain and Kibana **Default value:** `""`
`domain_endpoint_options_enforce_https` (`bool`) optional
Whether or not to require HTTPS **Default value:** `true`
`domain_endpoint_options_tls_security_policy` (`string`) optional
The name of the TLS security policy that needs to be applied to the HTTPS endpoint **Default value:** `"Policy-Min-TLS-1-0-2019-07"`
`domain_hostname_enabled` (`bool`) optional
Explicit flag to enable creating a DNS hostname for ES. If `true`, then `var.dns_zone_id` is required. **Default value:** `false`
`ebs_iops` (`number`) optional
The baseline input/output (I/O) performance of EBS volumes attached to data nodes. Applicable only for the Provisioned IOPS EBS volume type **Default value:** `0`
`ebs_throughput` (`number`) optional
Specifies the throughput (in MiB/s) of the EBS volumes attached to data nodes. Applicable only for the gp3 volume type. Valid values are between 125 and 1000. **Default value:** `null`
`ebs_volume_size` (`number`) optional
EBS volumes for data storage in GB **Default value:** `0`
`ebs_volume_type` (`string`) optional
Storage type of EBS volumes **Default value:** `"gp2"`
`elasticsearch_domain_name` (`string`) optional
The name of the Elasticsearch domain. Must be at least 3 and no more than 28 characters long. Valid characters are a-z (lowercase letters), 0-9, and - (hyphen). **Default value:** `""`
`elasticsearch_subdomain_name` (`string`) optional
The name of the subdomain for Elasticsearch in the DNS zone (_e.g._ `elasticsearch`, `ui`, `ui-es`, `search-ui`) **Default value:** `""`
`elasticsearch_version` (`string`) optional
Version of Elasticsearch to deploy (_e.g._ `7.4`, `7.1`, `6.8`, `6.7`, `6.5`, `6.4`, `6.3`, `6.2`, `6.0`, `5.6`, `5.5`, `5.3`, `5.1`, `2.3`, `1.5` **Default value:** `"7.4"`
`encrypt_at_rest_enabled` (`bool`) optional
Whether to enable encryption at rest **Default value:** `true`
`encrypt_at_rest_kms_key_id` (`string`) optional
The KMS key ID to encrypt the Elasticsearch domain with. If not specified, then it defaults to using the AWS/Elasticsearch service KMS key **Default value:** `""`
`iam_actions` (`list(string)`) optional
List of actions to allow for the user IAM roles, _e.g._ `es:ESHttpGet`, `es:ESHttpPut`, `es:ESHttpPost` **Default value:** `[ ]`
`iam_authorizing_role_arns` (`list(string)`) optional
List of IAM role ARNs to permit to assume the Elasticsearch user role **Default value:** `[ ]`
`iam_irsa_openid_connect_provider_arn` (`string`) optional
ARN of the OpenID connect provider to allow usage of IRSA **Default value:** `""`
`iam_irsa_openid_connect_provider_url` (`string`) optional
URL of the OpenID connect provider to allow usage of IRSA **Default value:** `""`
`iam_irsa_service_account` (`string`) optional
Kubernetes ServiceAccount to allow to access the Elastic Domain via IRSA **Default value:** `"system:serviceaccount:default:*"`
`iam_role_arns` (`list(string)`) optional
List of IAM role ARNs to permit access to the Elasticsearch domain **Default value:** `[ ]`
`iam_role_max_session_duration` (`number`) optional
The maximum session duration (in seconds) for the user role. Can have a value from 1 hour to 12 hours **Default value:** `3600`
`iam_role_permissions_boundary` (`string`) optional
The ARN of the permissions boundary policy which will be attached to the Elasticsearch user role **Default value:** `null`
`ingress_port_range_end` (`number`) optional
End number for allowed port range. (e.g. `443`) **Default value:** `65535`
`ingress_port_range_start` (`number`) optional
Start number for allowed port range. (e.g. `443`) **Default value:** `0`
`instance_count` (`number`) optional
Number of data nodes in the cluster **Default value:** `4`
`instance_type` (`string`) optional
Elasticsearch instance type for data nodes in the cluster **Default value:** `"t2.small.elasticsearch"`
`kibana_hostname_enabled` (`bool`) optional
Explicit flag to enable creating a DNS hostname for Kibana. If `true`, then `var.dns_zone_id` is required. **Default value:** `false`
`kibana_subdomain_name` (`string`) optional
The name of the subdomain for Kibana in the DNS zone (_e.g._ `kibana`, `ui`, `ui-es`, `search-ui`, `kibana.elasticsearch`) **Default value:** `""`
`log_publishing_application_cloudwatch_log_group_arn` (`string`) optional
ARN of the CloudWatch log group to which log for ES_APPLICATION_LOGS needs to be published **Default value:** `""`
`log_publishing_application_enabled` (`bool`) optional
Specifies whether log publishing option for ES_APPLICATION_LOGS is enabled or not **Default value:** `false`
`log_publishing_audit_cloudwatch_log_group_arn` (`string`) optional
ARN of the CloudWatch log group to which log for AUDIT_LOGS needs to be published **Default value:** `""`
`log_publishing_audit_enabled` (`bool`) optional
Specifies whether log publishing option for AUDIT_LOGS is enabled or not **Default value:** `false`
`log_publishing_index_cloudwatch_log_group_arn` (`string`) optional
ARN of the CloudWatch log group to which log for INDEX_SLOW_LOGS needs to be published **Default value:** `""`
`log_publishing_index_enabled` (`bool`) optional
Specifies whether log publishing option for INDEX_SLOW_LOGS is enabled or not **Default value:** `false`
`log_publishing_search_cloudwatch_log_group_arn` (`string`) optional
ARN of the CloudWatch log group to which log for SEARCH_SLOW_LOGS needs to be published **Default value:** `""`
`log_publishing_search_enabled` (`bool`) optional
Specifies whether log publishing option for SEARCH_SLOW_LOGS is enabled or not **Default value:** `false`
`multi_az_with_standby_enabled` (`bool`) optional
Enable domain with standby for OpenSearch cluster **Default value:** `false`
`node_to_node_encryption_enabled` (`bool`) optional
Whether to enable node-to-node encryption **Default value:** `false`
`security_groups` (`list(string)`) optional
List of security group IDs to be allowed to connect to the cluster or the security group IDs to apply to the cluster when the `create_security_group` variable is set to false. **Default value:** `[ ]`
`subnet_ids` (`list(string)`) optional
VPC Subnet IDs **Default value:** `[ ]`
`vpc_enabled` (`bool`) optional
Set to false if ES should be deployed outside of VPC. **Default value:** `true`
`vpc_id` (`string`) optional
VPC ID **Default value:** `null`
`warm_count` (`number`) optional
Number of UltraWarm nodes **Default value:** `2`
`warm_enabled` (`bool`) optional
Whether AWS UltraWarm is enabled **Default value:** `false`
`warm_type` (`string`) optional
Type of UltraWarm nodes **Default value:** `"ultrawarm1.medium.elasticsearch"`
`zone_awareness_enabled` (`bool`) optional
Enable zone awareness for Elasticsearch cluster **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`domain_arn`
ARN of the Elasticsearch domain
`domain_endpoint`
Domain-specific endpoint used to submit index, search, and data upload requests
`domain_hostname`
Elasticsearch domain hostname to submit index, search, and data upload requests
`domain_id`
Unique identifier for the Elasticsearch domain
`domain_name`
Name of the Elasticsearch domain
`elasticsearch_user_iam_role_arn`
The ARN of the IAM role to allow access to Elasticsearch cluster
`elasticsearch_user_iam_role_name`
The name of the IAM role to allow access to Elasticsearch cluster
`kibana_endpoint`
Domain-specific endpoint for Kibana without https scheme
`kibana_hostname`
Kibana hostname
`security_group_id`
Security Group ID to control access to the Elasticsearch domain
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.15.0` ### Providers - `aws`, version: `>= 5.15.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `domain_hostname` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `kibana_hostname` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `kibana_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `user_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_elasticsearch_domain.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain) (resource) - [`aws_elasticsearch_domain_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_policy) (resource) - [`aws_iam_role.elasticsearch_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_service_linked_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) (resource) - [`aws_opensearch_domain.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain) (resource) - [`aws_opensearch_domain_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/opensearch_domain_policy) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## emr-cluster # Module: `emr-cluster` Terraform module to provision an Elastic MapReduce (EMR) cluster on AWS. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-emr-cluster/tree/main/examples/complete) For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-emr-cluster/tree/main/test). ```hcl provider "aws" { region = "us-east-2" } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "s3_log_storage" { source = "cloudposse/s3-log-storage/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" region = var.region namespace = var.namespace stage = var.stage name = var.name attributes = ["logs"] force_destroy = true } module "aws_key_pair" { source = "cloudposse/key-pair/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name attributes = ["ssh", "key"] ssh_public_key_path = var.ssh_public_key_path generate_ssh_key = var.generate_ssh_key } module "emr_cluster" { source = "cloudposse/emr-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name master_allowed_security_groups = [module.vpc.vpc_default_security_group_id] slave_allowed_security_groups = [module.vpc.vpc_default_security_group_id] region = var.region vpc_id = module.vpc.vpc_id subnet_id = module.subnets.private_subnet_ids[0] route_table_id = module.subnets.private_route_table_ids[0] subnet_type = "private" ebs_root_volume_size = var.ebs_root_volume_size visible_to_all_users = var.visible_to_all_users release_label = var.release_label applications = var.applications configurations_json = var.configurations_json core_instance_group_instance_type = var.core_instance_group_instance_type core_instance_group_instance_count = var.core_instance_group_instance_count core_instance_group_ebs_size = var.core_instance_group_ebs_size core_instance_group_ebs_type = var.core_instance_group_ebs_type core_instance_group_ebs_volumes_per_instance = var.core_instance_group_ebs_volumes_per_instance master_instance_group_instance_type = var.master_instance_group_instance_type master_instance_group_instance_count = var.master_instance_group_instance_count master_instance_group_ebs_size = var.master_instance_group_ebs_size master_instance_group_ebs_type = var.master_instance_group_ebs_type master_instance_group_ebs_volumes_per_instance = var.master_instance_group_ebs_volumes_per_instance create_task_instance_group = var.create_task_instance_group log_uri = format("s3n://%s/", module.s3_log_storage.bucket_id) key_name = module.aws_key_pair.key_name } ``` ## Variables ### Required Variables
`applications` (`list(string)`) required
A list of applications for the cluster. Valid values are: Flink, Ganglia, Hadoop, HBase, HCatalog, Hive, Hue, JupyterHub, Livy, Mahout, MXNet, Oozie, Phoenix, Pig, Presto, Spark, Sqoop, TensorFlow, Tez, Zeppelin, and ZooKeeper (as of EMR 5.25.0). Case insensitive
`core_instance_group_ebs_size` (`number`) required
Core instances volume size, in gibibytes (GiB)
`core_instance_group_instance_type` (`string`) required
EC2 instance type for all instances in the Core instance group
`master_instance_group_ebs_size` (`number`) required
Master instances volume size, in gibibytes (GiB)
`master_instance_group_instance_type` (`string`) required
EC2 instance type for all instances in the Master instance group
`region` (`string`) required
AWS region
`subnet_id` (`string`) required
VPC subnet ID where you want the job flow to launch. Cannot specify the `cc1.4xlarge` instance type for nodes of a job flow launched in a Amazon VPC
`vpc_id` (`string`) required
VPC ID to create the cluster in (e.g. `vpc-a22222ee`)
### Optional Variables
`additional_info` (`string`) optional
A JSON string for selecting additional features such as adding proxy information. Note: Currently there is no API to retrieve the value of this argument after EMR cluster creation from provider, therefore Terraform cannot detect drift from the actual EMR cluster if its value is changed outside Terraform **Default value:** `null`
`additional_master_security_group` (`string`) optional
The id of the existing additional security group that will be used for EMR master node. If empty, a new security group will be created **Default value:** `""`
`additional_slave_security_group` (`string`) optional
The id of the existing additional security group that will be used for EMR core & task nodes. If empty, a new security group will be created **Default value:** `""`
`auto_termination_idle_timeout` (`string`) optional
Auto termination policy idle timeout in seconds (60 - 604800 supported) **Default value:** `null`
`bootstrap_action` optional
List of bootstrap actions that will be run before Hadoop is started on the cluster nodes **Type:** ```hcl list(object({ path = string name = string args = list(string) })) ``` **Default value:** `[ ]`
`configurations_json` (`string`) optional
A JSON string for supplying list of configurations for the EMR cluster. See https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-configure-apps.html for more details **Default value:** `""`
`core_instance_group_autoscaling_policy` (`string`) optional
String containing the EMR Auto Scaling Policy JSON for the Core instance group **Default value:** `null`
`core_instance_group_bid_price` (`string`) optional
Bid price for each EC2 instance in the Core instance group, expressed in USD. By setting this attribute, the instance group is being declared as a Spot Instance, and will implicitly create a Spot request. Leave this blank to use On-Demand Instances **Default value:** `null`
`core_instance_group_ebs_iops` (`number`) optional
The number of I/O operations per second (IOPS) that the Core volume supports **Default value:** `null`
`core_instance_group_ebs_type` (`string`) optional
Core instances volume type. Valid options are `gp2`, `io1`, `standard` and `st1` **Default value:** `"gp2"`
`core_instance_group_ebs_volumes_per_instance` (`number`) optional
The number of EBS volumes with this configuration to attach to each EC2 instance in the Core instance group **Default value:** `1`
`core_instance_group_instance_count` (`number`) optional
Target number of instances for the Core instance group. Must be at least 1 **Default value:** `1`
`create_task_instance_group` (`bool`) optional
Whether to create an instance group for Task nodes. For more info: https://www.terraform.io/docs/providers/aws/r/emr_instance_group.html, https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-master-core-task-nodes.html **Default value:** `false`
`create_vpc_endpoint_s3` (`bool`) optional
Set to false to prevent the module from creating VPC S3 Endpoint **Default value:** `true`
`custom_ami_id` (`string`) optional
A custom Amazon Linux AMI for the cluster (instead of an EMR-owned AMI). Available in Amazon EMR version 5.7.0 and later **Default value:** `null`
`ebs_root_volume_size` (`number`) optional
Size in GiB of the EBS root device volume of the Linux AMI that is used for each EC2 instance. Available in Amazon EMR version 4.x and later **Default value:** `10`
`ec2_autoscaling_role_enabled` (`bool`) optional
If set to `false`, will use `existing_ec2_autoscaling_role_arn` for an existing EC2 autoscaling IAM role that was created outside of this module **Default value:** `true`
`ec2_autoscaling_role_permissions_boundary` (`string`) optional
The Permissions Boundary ARN to apply to the EC2 Autoscaling Role. **Default value:** `""`
`ec2_role_enabled` (`bool`) optional
If set to `false`, will use `existing_ec2_instance_profile_arn` for an existing EC2 IAM role that was created outside of this module **Default value:** `true`
`ec2_role_permissions_boundary` (`string`) optional
The Permissions Boundary ARN to apply to the EC2 Role. **Default value:** `""`
`emr_role_permissions_boundary` (`string`) optional
The Permissions Boundary ARN to apply to the EMR Role. **Default value:** `""`
`enable_ssm_access` (`bool`) optional
If set to `true`, attach the existing `AmazonSSMManagedInstanceCore` IAM policy to the EMR EC2 instance profile role **Default value:** `false`
`existing_ec2_autoscaling_role_arn` (`string`) optional
ARN of an existing EC2 autoscaling role to attach to the cluster **Default value:** `""`
`existing_ec2_instance_profile_arn` (`string`) optional
ARN of an existing EC2 instance profile **Default value:** `""`
`existing_service_role_arn` (`string`) optional
ARN of an existing EMR service role to attach to the cluster **Default value:** `""`
`keep_job_flow_alive_when_no_steps` (`bool`) optional
Switch on/off run cluster with no steps or when all steps are complete **Default value:** `true`
`kerberos_ad_domain_join_password` (`string`) optional
The Active Directory password for ad_domain_join_user. Terraform cannot perform drift detection of this configuration. **Default value:** `null`
`kerberos_ad_domain_join_user` (`string`) optional
Required only when establishing a cross-realm trust with an Active Directory domain. A user with sufficient privileges to join resources to the domain. Terraform cannot perform drift detection of this configuration. **Default value:** `null`
`kerberos_cross_realm_trust_principal_password` (`string`) optional
Required only when establishing a cross-realm trust with a KDC in a different realm. The cross-realm principal password, which must be identical across realms. Terraform cannot perform drift detection of this configuration. **Default value:** `null`
`kerberos_enabled` (`bool`) optional
Set to true if EMR cluster will use kerberos_attributes **Default value:** `false`
`kerberos_kdc_admin_password` (`string`) optional
The password used within the cluster for the kadmin service on the cluster-dedicated KDC, which maintains Kerberos principals, password policies, and keytabs for the cluster. Terraform cannot perform drift detection of this configuration. **Default value:** `null`
`kerberos_realm` (`string`) optional
The name of the Kerberos realm to which all nodes in a cluster belong. For example, EC2.INTERNAL **Default value:** `"EC2.INTERNAL"`
`key_name` (`string`) optional
Amazon EC2 key pair that can be used to ssh to the master node as the user called `hadoop` **Default value:** `null`
`log_uri` (`string`) optional
The path to the Amazon S3 location where logs for this cluster are stored **Default value:** `null`
`managed_master_security_group` (`string`) optional
The id of the existing managed security group that will be used for EMR master node. If empty, a new security group will be created **Default value:** `""`
`managed_slave_security_group` (`string`) optional
The id of the existing managed security group that will be used for EMR core & task nodes. If empty, a new security group will be created **Default value:** `""`
`master_allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to access the master instances **Default value:** `[ ]`
`master_allowed_security_groups` (`list(string)`) optional
List of security group ids to be allowed to connect to the master instances **Default value:** `[ ]`
`master_dns_name` (`string`) optional
Name of the cluster CNAME record to create in the parent DNS zone specified by `zone_id`. If left empty, the name will be auto-asigned using the format `emr-master-var.name` **Default value:** `null`
`master_instance_group_bid_price` (`string`) optional
Bid price for each EC2 instance in the Master instance group, expressed in USD. By setting this attribute, the instance group is being declared as a Spot Instance, and will implicitly create a Spot request. Leave this blank to use On-Demand Instances **Default value:** `null`
`master_instance_group_ebs_iops` (`number`) optional
The number of I/O operations per second (IOPS) that the Master volume supports **Default value:** `null`
`master_instance_group_ebs_type` (`string`) optional
Master instances volume type. Valid options are `gp2`, `io1`, `standard` and `st1` **Default value:** `"gp2"`
`master_instance_group_ebs_volumes_per_instance` (`number`) optional
The number of EBS volumes with this configuration to attach to each EC2 instance in the Master instance group **Default value:** `1`
`master_instance_group_instance_count` (`number`) optional
Target number of instances for the Master instance group. Must be at least 1 **Default value:** `1`
`release_label` (`string`) optional
The release label for the Amazon EMR release. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-release-5x.html **Default value:** `"emr-5.25.0"`
`route_table_id` (`string`) optional
Route table ID for the VPC S3 Endpoint when launching the EMR cluster in a private subnet. Required when `subnet_type` is `private` **Default value:** `""`
`scale_down_behavior` (`string`) optional
The way that individual Amazon EC2 instances terminate when an automatic scale-in activity occurs or an instance group is resized **Default value:** `null`
`security_configuration` (`string`) optional
The security configuration name to attach to the EMR cluster. Only valid for EMR clusters with `release_label` 4.8.0 or greater. See https://www.terraform.io/docs/providers/aws/r/emr_security_configuration.html for more info **Default value:** `null`
`service_access_security_group` (`string`) optional
The id of the existing additional security group that will be used for EMR core & task nodes. If empty, a new security group will be created **Default value:** `""`
`service_role_enabled` (`bool`) optional
If set to `false`, will use `existing_service_role_arn` for an existing IAM role that was created outside of this module **Default value:** `true`
`slave_allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to be allowed to access the slave instances **Default value:** `[ ]`
`slave_allowed_security_groups` (`list(string)`) optional
List of security group ids to be allowed to connect to the slave instances **Default value:** `[ ]`
`step_concurrency_level` (`number`) optional
The number of steps that can be executed concurrently. You can specify a maximum of 256 steps. Only valid for EMR clusters with release_label 5.28.0 or greater. **Default value:** `null`
`steps` optional
List of steps to run when creating the cluster. **Type:** ```hcl list(object({ name = string action_on_failure = string hadoop_jar_step = object({ args = list(string) jar = string main_class = string properties = map(string) }) })) ``` **Default value:** `[ ]`
`subnet_type` (`string`) optional
Type of VPC subnet ID where you want the job flow to launch. Supported values are `private` or `public` **Default value:** `"private"`
`task_instance_group_autoscaling_policy` (`string`) optional
String containing the EMR Auto Scaling Policy JSON for the Task instance group **Default value:** `null`
`task_instance_group_bid_price` (`string`) optional
Bid price for each EC2 instance in the Task instance group, expressed in USD. By setting this attribute, the instance group is being declared as a Spot Instance, and will implicitly create a Spot request. Leave this blank to use On-Demand Instances **Default value:** `null`
`task_instance_group_ebs_iops` (`number`) optional
The number of I/O operations per second (IOPS) that the Task volume supports **Default value:** `null`
`task_instance_group_ebs_optimized` (`bool`) optional
Indicates whether an Amazon EBS volume in the Task instance group is EBS-optimized. Changing this forces a new resource to be created **Default value:** `false`
`task_instance_group_ebs_size` (`number`) optional
Task instances volume size, in gibibytes (GiB) **Default value:** `10`
`task_instance_group_ebs_type` (`string`) optional
Task instances volume type. Valid options are `gp2`, `io1`, `standard` and `st1` **Default value:** `"gp2"`
`task_instance_group_ebs_volumes_per_instance` (`number`) optional
The number of EBS volumes with this configuration to attach to each EC2 instance in the Task instance group **Default value:** `1`
`task_instance_group_instance_count` (`number`) optional
Target number of instances for the Task instance group. Must be at least 1 **Default value:** `1`
`task_instance_group_instance_type` (`string`) optional
EC2 instance type for all instances in the Task instance group **Default value:** `null`
`termination_protection` (`bool`) optional
Switch on/off termination protection (default is false, except when using multiple master nodes). Before attempting to destroy the resource when termination protection is enabled, this configuration must be applied with its value set to false **Default value:** `false`
`use_existing_additional_master_security_group` (`bool`) optional
If set to `true`, will use variable `additional_master_security_group` using an existing security group that was created outside of this module **Default value:** `false`
`use_existing_additional_slave_security_group` (`bool`) optional
If set to `true`, will use variable `additional_slave_security_group` using an existing security group that was created outside of this module **Default value:** `false`
`use_existing_managed_master_security_group` (`bool`) optional
If set to `true`, will use variable `managed_master_security_group` using an existing security group that was created outside of this module **Default value:** `false`
`use_existing_managed_slave_security_group` (`bool`) optional
If set to `true`, will use variable `managed_slave_security_group` using an existing security group that was created outside of this module **Default value:** `false`
`use_existing_service_access_security_group` (`bool`) optional
If set to `true`, will use variable `service_access_security_group` using an existing security group that was created outside of this module **Default value:** `false`
`visible_to_all_users` (`bool`) optional
Whether the job flow is visible to all IAM users of the AWS account associated with the job flow **Default value:** `true`
`zone_id` (`string`) optional
Route53 parent zone ID. If provided (not empty), the module will create sub-domain DNS records for the masters and slaves **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`cluster_id`
EMR cluster ID
`cluster_name`
EMR cluster name
`ec2_role`
Role name of EMR EC2 instances so users can attach more policies
`master_host`
Name of the cluster CNAME record for the master nodes in the parent DNS zone
`master_public_dns`
Master public DNS
`master_security_group_id`
Master security group ID
`slave_security_group_id`
Slave security group ID
## Dependencies ### Requirements - `terraform`, version: `>= 0.14.0` - `aws`, version: `>= 3.5.0` ### Providers - `aws`, version: `>= 3.5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_master` | 0.12.2 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.12.2) | n/a `label_core` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_ec2` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_ec2_autoscaling` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_emr` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_master` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_master_managed` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_service_managed` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_slave` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_slave_managed` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label_task` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_emr_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/emr_cluster) (resource) - [`aws_emr_instance_group.task`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/emr_instance_group) (resource) - [`aws_iam_instance_profile.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_role.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.ec2_autoscaling`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.emr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ec2_autoscaling`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.emr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.emr_ssm_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_security_group.managed_master`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group.managed_service_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group.managed_slave`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group.master`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group.slave`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.managed_master_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.managed_master_service_access_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.managed_service_access_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.managed_slave_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.master_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.master_ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.master_ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.slave_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.slave_ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.slave_ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_vpc_endpoint.vpc_endpoint_s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role_ec2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.assume_role_emr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## firewall-manager # Module: `firewall-manager` Terraform module to create and manage AWS Firewall Manager policies. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-firewall-manager/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-firewall-manager/tree/main/test). ```hcl module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "fms" delimiter = "-" tags = { "BusinessUnit" = "XYZ", } } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "10.0.0.0/16" context = module.label.context } provider "aws" { region = "us-east-2" } provider "aws" { region = "us-east-2" alias = "admin" assume_role { role_arn = "arn:aws:xyz" } } module "fms" { source = "cloudposse/firewall-manager/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" providers = { aws.admin = aws.admin aws = aws } security_groups_usage_audit_policies = [ { name = "unused-sg" resource_type_list = ["AWS::EC2::SecurityGroup"] policy_data = { delete_unused_security_groups = false coalesce_redundant_security_groups = false } } ] security_groups_content_audit_policies = [ { name = "maxmimum-allowed" resource_type_list = ["AWS::EC2::SecurityGroup"] policy_data = { security_group_action = "allow" security_groups = [module.vpc.security_group_id] } } ] security_groups_common_policies = [ { name = "disabled-all" resource_type_list = ["AWS::EC2::SecurityGroup"] policy_data = { revert_manual_security_group_changes = false exclusive_resource_security_group_management = false apply_to_all_ec2_instance_enis = false security_groups = [module.vpc.security_group_id] } } ] waf_v2_policies = [ { name = "linux-policy" resource_type_list = ["AWS::ElasticLoadBalancingV2::LoadBalancer", "AWS::ApiGateway::Stage"] policy_data = { default_action = "allow" override_customer_web_acl_association = false pre_process_rule_groups = [ { "managedRuleGroupIdentifier" : { "vendorName" : "AWS", "managedRuleGroupName" : "AWSManagedRulesLinuxRuleSet", "version" : null }, "overrideAction" : { "type" : "NONE" }, "ruleGroupArn" : null, "excludeRules" : [], "ruleGroupType" : "ManagedRuleGroup" } ] } } ] context = module.label.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-firewall-manager/examples/complete) - complete example of using this module ## Variables ### Required Variables
`security_groups_usage_audit_policies` (`list(any)`) required
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: delete_unused_security_groups: Whether to delete unused Security Groups. Defaults to `false`. coalesce_redundant_security_groups: Whether to coalesce redundant Security Groups. Defaults to `false`.
### Optional Variables
`admin_account_enabled` (`bool`) optional
Resource for aws_fms_admin_account is enabled and will be created or destroyed **Default value:** `true`
`admin_account_id` (`string`) optional
The AWS account ID to associate to associate with AWS Firewall Manager as the AWS Firewall Manager administrator account. This can be an AWS Organizations master account or a member account. Defaults to the current account. **Default value:** `null`
`dns_firewall_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: pre_process_rule_groups: A list of maps of pre-proccess rule groups in the format `{ "ruleGroupId": "rslvr-frg-1", "priority": 10 }`. post_process_rule_groups: A list of maps post-proccess rule groups in the format `{ "ruleGroupId": "rslvr-frg-1", "priority": 10 }`. **Default value:** `[ ]`
`firehose_arn` (`string`) optional
Kinesis Firehose ARN used to create a Kinesis Firehose destination for WAF_V2 Rules. Conflicts with `firehose_enabled` **Default value:** `null`
`firehose_enabled` (`bool`) optional
Create a Kinesis Firehose destination for WAF_V2 Rules. Conflicts with `firehose_arn` **Default value:** `false`
`network_firewall_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: stateless_rule_group_references: A list of maps of configuration blocks containing references to the stateful rule groups that are used in the policy. Format: `{ "resourceARN": "arn:aws:network-firewall:us-west-1:1234567891011:stateless-rulegroup/rulegroup2", "priority": 10 }` stateless_default_actions: A list of actions to take on a packet if it does not match any of the stateless rules in the policy. You must specify one of the standard actions including: `aws:drop`, `aws:pass`, or `aws:forward_to_sfe`. In addition, you can specify custom actions that are compatible with your standard action choice. If you want non-matching packets to be forwarded for stateful inspection, specify aws:forward_to_sfe. stateless_fragment_default_actions: A list of actions to take on a fragmented packet if it does not match any of the stateless rules in the policy. You must specify one of the standard actions including: `aws:drop`, `aws:pass`, or `aws:forward_to_sfe`. In addition, you can specify custom actions that are compatible with your standard action choice. If you want non-matching packets to be forwarded for stateful inspection, specify aws:forward_to_sfe. stateless_custom_actions: A list of maps describing the custom action definitions that are available for use in the firewall policy's `stateless_default_actions`. Format: `{ "actionName": "custom1", "actionDefinition": { "publishMetricAction": { "dimensions": [ { "value": "dimension1" } ] } } }` stateful_rule_group_references_arns: A list of ARNs of the stateful rule groups. orchestration_config: single_firewall_endpoint_per_vpc: Whether to use single Firewall Endpoint per VPC. Defaults to `false`. allowed_ipv4_cidrs: A list of allowed ipv4 cidrs. **Default value:** `[ ]`
`security_groups_common_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: revert_manual_security_group_changes: Whether to revert manual Security Group changes. Defaults to `false`. exclusive_resource_security_group_management: Wheter to exclusive resource Security Group management. Defaults to `false`. apply_to_all_ec2_instance_enis: Whether to apply to all EC2 instance ENIs. Defaults to `false`. security_groups: A list of Security Group IDs. **Default value:** `[ ]`
`security_groups_content_audit_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: security_group_action: For `ALLOW`, all in-scope security group rules must be within the allowed range of the policy's security group rules. For `DENY`, all in-scope security group rules must not contain a value or a range that matches a rule value or range in the policy security group. Possible values: `ALLOW`, `DENY`. security_groups: A list of Security Group IDs. **Default value:** `[ ]`
`shield_advanced_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. delete_unused_fm_managed_resources: If true, Firewall Manager will automatically remove protections from resources that leave the policy scope. Defaults to `false`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: automatic_response_status: Status of shield automatic response. Possible values: ENABLED|IGNORED|DISABLED. Default is IGNORED. automatic_response_action: The automatic response action. Possible values: BLOCK|COUNT. Default is null. override_customer_webacl_classic: Whether to replace AWS WAF Classic web ACLs with this policy's AWS WAF v2 web ACLs where possible. Possible values: true|false Default is false. **Default value:** `[ ]`
`waf_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: default_action: The action that you want AWS WAF to take. Possible values: `ALLOW`, `BLOCK` or `COUNT`. rule_groups: A list of rule groups. **Default value:** `[ ]`
`waf_v2_policies` (`list(any)`) optional
name: The friendly name of the AWS Firewall Manager Policy. delete_all_policy_resources: Whether to perform a clean-up process. Defaults to `true`. exclude_resource_tags: A boolean value, if `true` the tags that are specified in the `resource_tags` are not protected by this policy. If set to `false` and `resource_tags` are populated, resources that contain tags will be protected by this policy. Defaults to `false`. remediation_enabled: A boolean value, indicates if the policy should automatically applied to resources that already exist in the account. Defaults to `false`. resource_type_list: A list of resource types to protect. Conflicts with `resource_type`. resource_type: A resource type to protect. Conflicts with `resource_type_list`. resource_tags: A map of resource tags, that if present will filter protections on resources based on the `exclude_resource_tags`. exclude_account_ids: A list of AWS Organization member Accounts that you want to exclude from this AWS FMS Policy. include_account_ids: A list of AWS Organization member Accounts that you want to include for this AWS FMS Policy. policy_data: default_action: The action that you want AWS WAF to take. Possible values: `ALLOW`, `BLOCK` or `COUNT`. override_customer_web_acl_association: Wheter to override customer Web ACL association logging_configuration: The WAFv2 Web ACL logging configuration. pre_process_rule_groups: A list of pre-proccess rule groups. post_process_rule_groups: A list of post-proccess rule groups. custom_request_handling: A custom header for custom request and response handling. Defaults to null. custom_response: A custom response for the web request. Defaults to null. sampled_requests_enabled_for_default_actions: Whether WAF should store a sampling of the web requests that match the rules. Possible values: `true` or `false`. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_account`
AWS Account ID of the designated admin account.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.9.0` ### Providers - `aws`, version: `>= 4.9.0` - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_firewall_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `firehose_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `firehose_s3_bucket` | 4.3.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.3.0) | n/a `network_firewall_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_groups_common_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_groups_content_audit_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `security_groups_usage_audit_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `shield_advanced_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `waf_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `waf_v2_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_fms_admin_account.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_admin_account) (resource) - [`aws_fms_policy.dns_firewall`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.network_firewall`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.security_groups_common`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.security_groups_content_audit`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.security_groups_usage_audit`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.shield_advanced`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.waf`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_fms_policy.waf_v2`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/fms_policy) (resource) - [`aws_iam_role.firehose_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_kinesis_firehose_delivery_stream.firehose_stream`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## github-action-token-rotator(Github-action-token-rotator) # Module: `github-action-token-rotator` This module deploys a [lambda function](https://github.com/cloudposse/lambda-github-action-token-rotator) that runs as a GitHub Application and periodically gets a new GitHub Runner Registration Token from the GitHub API. This token is then stored in AWS Systems Manager Parameter Store. ## Usage ```hcl module "github_action_token_rotator" { source = "cloudposse/github-action-token-rotator/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" parameter_store_token_path = "/github/runners/cloudposse/registrationToken" parameter_store_private_key_path = "/github/runners/cloudposse/privateKey" github_app_id = "111111" github_app_installation_id = "22222222" github_org = "cloudposse" } ``` ## Quick Start 1. Browse to https://github.com/organizations/\{YOUR_ORG\}/settings/apps and click the New GitHub App button 1. Set the name to "GitHub Action Token Rotator" 1. Set the Homepage URL to `https://github.com/cloudposse/lambda-github-action-token-rotator` 1. Uncheck the Active checkbox under the Webhook heading 1. Select `Read and write` under Organization permissions -> Self-hosted runners 1. Click the Create GitHub App button at the bottom of the page 1. Under the `Client secrets` section, click the `Generate a new client secret` button 1. Copy the Client secret to a safe place, it will be needed to install the app 1. Under the `Private key` section, click the `Generate a private key` button 1. Download the private key to a safe place, it will be needed to install the app 1. Convert the private key to a PEM file using the following command: `openssl pkcs8 -topk8 -inform PEM -outform PEM -nocrypt -in {DOWNLOADED_FILE_NAME}.pem -out private-key-pkcs8.key` 1. Base64 encode the private key using the following command: `cat private-key-pkcs8.key | base64` 1. Copy the Base64 value to AWS SSM Parameter store at `/github/runners/${YOUR_GITHUB_ORG}/privateKey` ## Variables ### Required Variables
`github_app_id` (`string`) required
GitHub App ID
`github_app_installation_id` (`string`) required
GitHub App Installation ID
`github_org` (`string`) required
GitHub Organization
`parameter_store_private_key_path` (`string`) required
Path to read the GitHub App private key from parameter store
`parameter_store_token_path` (`string`) required
Path to store the token in parameter store
### Optional Variables
`function_description` (`string`) optional
**Default value:** `"An app that automatically rotates the GitHub Action Runner token and stores it in SSM Parameter Store"`
`function_name` (`string`) optional
**Default value:** `"GitHubRunnerTokenRotator"`
`lambda_zip_version` (`string`) optional
The version of the Lambda function to deploy **Default value:** `"0.1.0"`
`memory_size` (`number`) optional
Amount of memory in MB the Lambda Function can use at runtime. **Default value:** `512`
`schedule_expression` (`string`) optional
AWS Schedule Expression: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html **Default value:** `"rate(30 minutes)"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the lambda function
`function_name`
Lambda function name
`invoke_arn`
Invoke ARN of the lambda function
`qualified_arn`
ARN identifying your Lambda Function Version (if versioning is enabled via publish = true)
`role_arn`
Lambda IAM role ARN
`role_name`
Lambda IAM role name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `random`, version: `>= 2.2` ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `lambda` | 0.3.2 | [`cloudposse/lambda-function/aws`](https://registry.terraform.io/modules/cloudposse/lambda-function/aws/0.3.2) | n/a `parameter_store_private_key` | 0.10.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.10.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `utils` | latest | [`cloudposse/utils/aws`](https://registry.terraform.io/modules/cloudposse/utils/aws/) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_policy.allow_ssm_parameter_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role_policy_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_permission.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.allow_ssm_parameter_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## global-accelerator(Global-accelerator) # Module: `global-accelerator` This module provisions AWS Global Accelerator. Multiple listeners can be specified when instantiating this module. The `endpoint-group` submodule provisions a Global Accelerator Endpoint Group for a listener created by this module and can be instantiated multiple times in order to provision multiple Endpoint Groups. The reason why `endpoint-group` is its own submodule is because an AWS Provider needs to be instantiated for the region the Endpoint Group's endpoints reside in. For more information, see the [endpoint-group documentation](https://github.com/cloudposse/terraform-aws-global-accelerator/tree/main/modules/endpoint-group/README.md). ## Usage For a complete examples, see [examples/](https://github.com/cloudposse/terraform-aws-global-accelerator/tree/main/examples/). The following deploys Global Accelerator with endpoints in a single region: ```hcl module "global_accelerator" { source = "cloudposse/global-accelerator/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ip_address_type = "IPV4" flow_logs_enabled = true flow_logs_s3_prefix = "logs/" flow_logs_s3_bucket = module.s3_bucket.bucket_id listeners = [ { client_affinity = "NONE" protocol = "TCP" port_ranges = [ { from_port = 80 to_port = 80 } ] } ] context = module.this.context } module "endpoint_group" { source = "cloudposse/global-accelerator/aws//modules/endpoint-group" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" listener_arn = module.global_accelerator.listener_ids[0] config = { endpoint_region = "us-east-2" endpoint_configuration = [ { endpoint_lb_name = "my-load-balancer" } ] } context = module.this.context } ``` The following deploys Global Accelerator with endpoints in multiple regions: ```hcl module "global_accelerator" { source = "cloudposse/global-accelerator/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ip_address_type = "IPV4" flow_logs_enabled = true flow_logs_s3_prefix = "logs/" flow_logs_s3_bucket = module.s3_bucket.bucket_id listeners = [ { client_affinity = "NONE" protocol = "TCP" port_ranges = [ { from_port = 80 to_port = 80 } ] } ] context = module.this.context } module "endpoint_group" { source = "cloudposse/global-accelerator/aws//modules/endpoint-group" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" listener_arn = module.global_accelerator.listener_ids[0] config = { endpoint_region = "us-east-2" endpoint_configuration = [ { endpoint_lb_name = "my-load-balancer" } ] } context = module.this.context } module "endpoint_group_failover" { source = "cloudposse/global-accelerator/aws//modules/endpoint-group" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" listener_arn = module.global_accelerator.listener_ids[0] config = { endpoint_region = "us-west-2" endpoint_configuration = [ { endpoint_lb_name = "my-failover-load-balancer" } ] } providers = { aws = aws.failover } context = module.failover_label.context } ``` ## Examples Here are some examples of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-global-accelerator/tree/main/examples/complete/) - complete example of using this module - [`examples/multiple_endpoints`](https://github.com/cloudposse/terraform-aws-global-accelerator/tree/main/examples/multiple_endpoints/) - multi-region example ## Variables ### Required Variables
### Optional Variables
`flow_logs_enabled` (`bool`) optional
Enable or disable flow logs for the Global Accelerator. **Default value:** `false`
`flow_logs_s3_bucket` (`string`) optional
The name of the S3 Bucket for the Accelerator Flow Logs. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`flow_logs_s3_prefix` (`string`) optional
The Object Prefix within the S3 Bucket for the Accelerator Flow Logs. Required if `var.flow_logs_enabled` is set to `true`. **Default value:** `null`
`ip_address_type` (`string`) optional
The address type to use for the Global Accelerator. At this moment, [only IPV4 is supported](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_accelerator#ip_address_type). **Default value:** `"IPV4"`
`listeners` optional
list of listeners to configure for the global accelerator. For more information, see: [aws_globalaccelerator_listener](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_listener). **Type:** ```hcl list(object({ client_affinity = string port_ranges = list(object({ from_port = number to_port = number })) protocol = string })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dns_name`
DNS name of the Global Accelerator.
`hosted_zone_id`
Route 53 zone ID that can be used to route an Alias Resource Record Set to the Global Accelerator.
`listener_ids`
Global Accelerator Listener IDs.
`name`
Name of the Global Accelerator.
`static_ips`
Global Static IPs owned by the Global Accelerator.
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_globalaccelerator_accelerator.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_accelerator) (resource) - [`aws_globalaccelerator_listener.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_listener) (resource) ## Data Sources The following data sources are used by this module: --- ## endpoint-group # Endpoint Group Submodule for provisioning Global Accelerator Endpoint Groups. This submodule takes in a variable `config` that is essentially fully compliant with the `aws_globalaccelerator_endpoint_group` resource, except that `listener_arn` is specified separately and that the `config` object supports a dynamic lookup of `endpoint_id` inside the `endpoint_configuration` list: ```hcl module "endpoint_group" { source = "cloudposse/global-accelerator/aws//modules/endpoint-group" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" context = module.this.context listener_arn = "arn:aws:globalaccelerator::111111111111:accelerator/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/listener/xxxxxxxx" config = { endpoint_region = "us-east-1" endpoint_configuration = [ { endpoint_lb_name = "my-load-balancer" } ] } } ``` For more information, see _Inputs_. ## Requirements No requirements. ## Providers | Name | Version | |------|---------| | [aws](#provider\_aws) | n/a | ## Modules | Name | Source | Version | |------|--------|---------| | [this](#module\_this) | cloudposse/label/null | 0.24.1 | ## Resources | Name | Type | |------|------| | [aws_globalaccelerator_endpoint_group.default](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_endpoint_group) | resource | | [aws_lb.lb](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/lb) | data source | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional tags for appending to tags\_as\_list\_of\_maps. Not added to `tags`. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | Additional attributes (e.g. `1`) | `list(string)` | `[]` | no | | [config](#input\_config) | Endpoint Group configuration.This object needs to be fully compliant with the `aws_globalaccelerator_endpoint_group` resource, except for the following differences:* `listener_arn`, which is specified separately, is omitted.* The values for `endpoint_configuration` and `port_override` within each object in `endpoint_groups` should be lists.* Inside the `endpoint_configuration` block, `endpoint_lb_name` can be supplied in place of `endpoint_id` as long as it is a valid unique name for an existing ALB or NLB.For more information, see: [aws\_globalaccelerator\_endpoint\_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/globalaccelerator_endpoint_group). | `any` | n/a | yes | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\}\}
| no | | [delimiter](#input\_delimiter) | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for default, which is `0`.Does not affect `id_full`. | `number` | `null` | no | | [label\_key\_case](#input\_label\_key\_case) | The letter case of label keys (`tag` names) (i.e. `name`, `namespace`, `environment`, `stage`, `attributes`) to use in `tags`.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The naming order of the id output and Name tag.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 5 elements, but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | The letter case of output label values (also used in `tags` and `id`).Possible values: `lower`, `title`, `upper` and `none` (no transformation).Default value: `lower`. | `string` | `null` | no | | [listener\_arn](#input\_listener\_arn) | The ARN of the Global Accelerator Listener which this Endpoint Group will be associated with. | `string` | n/a | yes | | [name](#input\_name) | Solution name, e.g. 'app' or 'jenkins' | `string` | `null` | no | | [namespace](#input\_namespace) | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | `string` | `null` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [stage](#input\_stage) | Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit','XYZ')` | `map(string)` | `{}` | no | ## Outputs | Name | Description | |------|-------------| | [id](#output\_id) | n/a | --- ## glue # Module: `glue` Terraform modules for provisioning and managing AWS [Glue](https://docs.aws.amazon.com/glue/latest/dg/what-is-glue.html) resources. The following Glue resources are supported: - [Catalog database](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-catalog-database) - [Catalog table](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-catalog-table) - [Connection](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-connection) - [Crawler](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-crawler) - [Job](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-job) - [Registry](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-registry) - [Schema](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-schema) - [Trigger](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-trigger) - [Workflow](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules/glue-workflow) Refer to [modules](https://github.com/cloudposse/terraform-aws-glue/tree/main/modules) for more details. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-glue/tree/main/examples/complete). The example provisions a Glue catalog database and a Glue crawler that crawls a public dataset in an S3 bucket and writes the metadata into the Glue catalog database. It also provisions an S3 bucket with a Glue Job Python script, and a destination S3 bucket for Glue job results. And finally, it provisions a Glue job pointing to the Python script in the S3 bucket, and a Glue trigger that triggers the Glue job on a schedule. The Glue job processes the dataset, cleans up the data, and writes the result into the destination S3 bucket. For an example on how to provision source and destination S3 buckets, Glue Catalog database and table, and a Glue crawler that processes data in the source S3 bucket and writes the result into the destination S3 bucket, see [examples/crawler](https://github.com/cloudposse/terraform-aws-glue/tree/main/examples/crawler). For automated tests of the examples using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the examples on AWS), see [test](https://github.com/cloudposse/terraform-aws-glue/tree/main/test). ## Examples ```hcl locals { enabled = module.this.enabled s3_bucket_source = module.s3_bucket_source.bucket_id role_arn = module.iam_role.arn # The dataset used in this example consists of Medicare-Provider payment data downloaded from two Data.CMS.gov sites: # Inpatient Prospective Payment System Provider Summary for the Top 100 Diagnosis-Related Groups - FY2011, and Inpatient Charge Data FY 2011. # AWS modified the data to introduce a couple of erroneous records at the tail end of the file data_source = "s3://awsglue-datasets/examples/medicare/Medicare_Hospital_Provider.csv" } module "glue_catalog_database" { source = "cloudposse/glue/aws//modules/glue-catalog-database" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_database_description = "Glue Catalog database for the data located in ${local.data_source}" location_uri = local.data_source attributes = ["payments"] context = module.this.context } module "glue_catalog_table" { source = "cloudposse/glue/aws//modules/glue-catalog-table" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_table_name = "medicare" catalog_table_description = "Test Glue Catalog table" database_name = module.glue_catalog_database.name storage_descriptor = { # Physical location of the table location = local.data_source } context = module.this.context } resource "aws_lakeformation_permissions" "default" { principal = local.role_arn permissions = ["ALL"] table { database_name = module.glue_catalog_database.name name = module.glue_catalog_table.name } } # Crawls the data in the S3 bucket and puts the results into a database in the Glue Data Catalog. # The crawler will read the first 2 MB of data from that file, and recognize the schema. # After that, the crawler will sync the table `medicare` in the Glue database. module "glue_crawler" { source = "cloudposse/glue/aws//modules/glue-crawler" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" crawler_description = "Glue crawler that processes data in ${local.data_source} and writes the metadata into a Glue Catalog database" database_name = module.glue_catalog_database.name role = local.role_arn schedule = "cron(0 1 * * ? *)" schema_change_policy = { delete_behavior = "LOG" update_behavior = null } catalog_target = [ { database_name = module.glue_catalog_database.name tables = [module.glue_catalog_table.name] } ] context = module.this.context depends_on = [ aws_lakeformation_permissions.default ] } # Source S3 bucket to store Glue Job scripts module "s3_bucket_source" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["source"] context = module.this.context } resource "aws_s3_object" "job_script" { bucket = local.s3_bucket_source key = "data_cleaning.py" source = "${path.module}/scripts/data_cleaning.py" force_destroy = true etag = filemd5("${path.module}/scripts/data_cleaning.py") tags = module.this.tags } # Destination S3 bucket to store Glue Job results module "s3_bucket_destination" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["destination"] context = module.this.context } module "iam_role" { source = "cloudposse/iam-role/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" principals = { "Service" = ["glue.amazonaws.com"] } managed_policy_arns = [ "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ] policy_document_count = 0 policy_description = "Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs" role_description = "Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs" context = module.this.context } module "glue_workflow" { source = "cloudposse/glue/aws//modules/glue-workflow" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" workflow_description = "Test Glue Workflow" max_concurrent_runs = 2 context = module.this.context } module "glue_job" { source = "cloudposse/glue/aws//modules/glue-job" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" job_description = "Glue Job that runs data_cleaning.py Python script" role_arn = local.role_arn glue_version = var.glue_version worker_type = "Standard" number_of_workers = 2 max_retries = 2 # The job timeout in minutes timeout = 20 command = { # The name of the job command. Defaults to `glueetl`. # Use `pythonshell` for Python Shell Job Type, or `gluestreaming` for Streaming Job Type. name = "glueetl" script_location = format("s3://%s/data_cleaning.py", local.s3_bucket_source) python_version = 3 } context = module.this.context } module "glue_trigger" { source = "cloudposse/glue/aws//modules/glue-trigger" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" workflow_name = module.glue_workflow.name trigger_enabled = true start_on_creation = true trigger_description = "Glue Trigger that triggers a Glue Job on a schedule" schedule = "cron(15 12 * * ? *)" type = "SCHEDULED" actions = [ { job_name = module.glue_job.name # The job run timeout in minutes. It overrides the timeout value of the job timeout = 10 } ] context = module.this.context } ``` ## Variables ### Required Variables
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 3.74.0` - `awsutils`, version: `>= 0.11.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## glue-catalog-database Terraform module to provision AWS Glue Catalog Databases. ## Usage ```hcl module "s3_bucket_source" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["source"] context = module.this.context } module "glue_catalog_database" { source = "cloudposse/glue/aws//modules/glue-catalog-database" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_database_name = "analytics" catalog_database_description = "Glue Catalog database using data located in an S3 bucket" location_uri = format("s3://%s", module.s3_bucket_source.bucket_id) context = module.this.context } ``` --- ## glue-catalog-table Terraform module to provision AWS Glue Catalog Tables. ## Usage ```hcl module "s3_bucket_source" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["source"] context = module.this.context } module "glue_catalog_database" { source = "cloudposse/glue/aws//modules/glue-catalog-database" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_database_name = "analytics" catalog_database_description = "Glue Catalog database using data located in an S3 bucket" location_uri = format("s3://%s", module.s3_bucket_source.bucket_id) context = module.this.context } module "glue_catalog_table" { source = "cloudposse/glue/aws//modules/glue-catalog-table" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_table_name = "geo" catalog_table_description = "region/state/county Glue Catalog table" database_name = module.glue_catalog_database.name parameters = { "lakeformation.aso.status" = true "classification" = "parquet" } storage_descriptor = { # List of reducer grouping columns, clustering columns, and bucketing columns in the table bucket_columns = null # Configuration block for columns in the table columns = [ { name = "county", type = "string" }, { name = "state", type = "string" }, { name = "region", type = "string" } ] # Whether the data in the table is compressed compressed = false # Input format: SequenceFileInputFormat (binary), or TextInputFormat, or a custom format input_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat" # Physical location of the table. By default this takes the form of the warehouse location, followed by the database location in the warehouse, followed by the table name location = format("s3://%s/geo", module.s3_bucket_source.bucket_id) # Must be specified if the table contains any dimension columns number_of_buckets = 0 # Output format: SequenceFileOutputFormat (binary), or IgnoreKeyTextOutputFormat, or a custom format output_format = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat" # Configuration block for serialization and deserialization ("SerDe") information ser_de_info = { # Map of initialization parameters for the SerDe, in key-value form parameters = { "serialization.format" = "1" } # Usually the class that implements the SerDe. An example is org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe serialization_library = "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe" } # Whether the table data is stored in subdirectories stored_as_sub_directories = false } context = module.this.context } ``` --- ## glue-connection # glue-workflow Terraform module to provision AWS Glue Connections. ## Usage ```hcl module "vpc" { source = "cloudposse/vpc/aws" version = "1.1.0" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } data "aws_subnet" "selected" { id = module.vpc.private_subnet_ids[0] } module "security_group" { source = "cloudposse/security-group/aws" version = "1.0.1" vpc_id = module.vpc.vpc_id create_before_destroy = true allow_all_egress = true rules = [ { type = "ingress" from_port = 5432 to_port = 5432 protocol = "all" cidr_blocks = [module.vpc.vpc_cidr_block] } ] context = module.this.context } module "glue_connection" { source = "cloudposse/glue/aws//modules/glue-connection" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" connection_name = "geo" connection_description = "Glue connection to Postgres database" connection_type = "JDBC" connection_properties = {} physical_connection_requirements = { # List of security group IDs used by the connection security_group_id_list = [module.security_group.id] # The availability zone of the connection. This field is redundant and implied by subnet_id, but is currently an API requirement availability_zone = data.aws_subnet.selected.availability_zone # The subnet ID used by the connection subnet_id = module.vpc.private_subnet_ids[0] } context = module.this.context } ``` --- ## glue-crawler Terraform module to provision AWS Glue Crawlers. ## Usage ```hcl module "s3_bucket_source" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["source"] context = module.this.context } module "iam_role" { source = "cloudposse/iam-role/aws" version = "0.16.2" principals = { "Service" = ["glue.amazonaws.com"] } managed_policy_arns = [ "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ] policy_document_count = 0 policy_description = "Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs" role_description = "Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs" context = module.this.context } module "s3_bucket_destination" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true attributes = ["destination"] context = module.this.context } module "glue_catalog_database" { source = "cloudposse/glue/aws//modules/glue-catalog-database" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" catalog_database_name = "analytics" catalog_database_description = "Glue Catalog database using data located in an S3 bucket" location_uri = format("s3://%s", module.s3_bucket_source.bucket_id) context = module.this.context } module "glue_crawler" { source = "cloudposse/glue/aws//modules/glue-crawler" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" crawler_description = "Glue crawler that processes data in the source S3 bucket and writes the result into the destination S3 bucket" database_name = module.glue_catalog_database.name role = module.iam_role.arn schedule = "cron(0 1 * * ? *)" schema_change_policy = { delete_behavior = "LOG" update_behavior = null } s3_target = [ { path = format("s3://%s", module.s3_bucket_destination.bucket_id) } ] configuration = jsonencode( { Grouping = { TableGroupingPolicy = "CombineCompatibleSchemas" } CrawlerOutput = { Partitions = { AddOrUpdateBehavior = "InheritFromTable" } } Version = 1 } ) context = module.this.context } ``` --- ## glue-job Terraform module to provision AWS Glue Jobs. ## Usage ```hcl module "iam_role" { source = "cloudposse/iam-role/aws" version = "0.16.2" principals = { "Service" = ["glue.amazonaws.com"] } managed_policy_arns = [ "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ] policy_document_count = 0 policy_description = "Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs" role_description = "Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs" context = module.this.context } module "s3_bucket_job_source" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "glue_job" { source = "cloudposse/glue/aws//modules/glue-job" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" job_name = "geo_processor" job_description = "Glue Job for processing geo data" role_arn = module.iam_role.arn glue_version = "2.0" default_arguments = {} worker_type = "Standard" number_of_workers = 2 max_retries = 2 # The job timeout in minutes timeout = 20 command = { name = "glueetl" script_location = format("s3://%s/geo.py", module.s3_bucket_job_source.bucket_id) python_version = 3 } context = module.this.context } ``` --- ## glue-registry Terraform module to provision AWS Glue Registries. ## Usage ```hcl module "glue_registry" { source = "cloudposse/glue/aws//modules/glue-registry" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" registry_name = "analytics" registry_description = "Glue Registry for analytics" } ``` --- ## glue-schema Terraform module to provision AWS Glue Schemas. ## Usage ```hcl module "glue_registry" { source = "cloudposse/glue/aws//modules/glue-registry" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" registry_name = "analytics" registry_description = "Glue Registry for analytics" } module "glue_schema" { source = "cloudposse/glue/aws//modules/glue-schema" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" schema_name = "geo" schema_description = "Glue Schema for geo data" registry_arn = module.glue_registry.arn compatibility = "NONE" data_format = "JSON" schema_definition = "{\"type\": \"record\", \"name\": \"geo\", \"fields\": [ {\"name\": \"state\", \"type\": \"string\"}, {\"name\": \"city\", \"type\": \"string\"} ]}" } ``` --- ## glue-trigger Terraform module to provision AWS Glue Triggers. ## Usage ```hcl module "iam_role" { source = "cloudposse/iam-role/aws" version = "0.16.2" principals = { "Service" = ["glue.amazonaws.com"] } managed_policy_arns = [ "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole" ] policy_document_count = 0 policy_description = "Policy for AWS Glue with access to EC2, S3, and Cloudwatch Logs" role_description = "Role for AWS Glue with access to EC2, S3, and Cloudwatch Logs" context = module.this.context } module "s3_bucket_job_source" { source = "cloudposse/s3-bucket/aws" version = "2.0.3" acl = "private" versioning_enabled = false force_destroy = true allow_encrypted_uploads_only = true allow_ssl_requests_only = true block_public_acls = true block_public_policy = true ignore_public_acls = true restrict_public_buckets = true context = module.this.context } module "glue_workflow" { source = "cloudposse/glue/aws//modules/glue-workflow" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" workflow_name = "geo" workflow_description = "Glue workflow to process geo data" max_concurrent_runs = 2 default_run_properties = {} } module "glue_job" { source = "cloudposse/glue/aws//modules/glue-job" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" job_name = "geo_processor" job_description = "Glue Job for processing geo data" role_arn = module.iam_role.arn glue_version = "2.0" default_arguments = {} worker_type = "Standard" number_of_workers = 2 max_retries = 2 # The job timeout in minutes timeout = 20 command = { name = "Run Python script" script_location = format("s3://%s/geo.py", module.s3_bucket_job_source.bucket_id) python_version = 3 } context = module.this.context } module "glue_trigger" { source = "cloudposse/glue/aws//modules/glue-trigger" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" workflow_name = module.glue_workflow.name trigger_enabled = true start_on_creation = true trigger_description = "Glue Trigger that triggers the geo_processor Glue Job on a schedule" schedule = "cron(15 12 * * ? *)" type = "SCHEDULED" actions = [ { job_name = module.glue_job.name # The job run timeout in minutes. It overrides the timeout value of the job timeout = 10 } ] context = module.this.context } ``` --- ## glue-workflow Terraform module to provision AWS Glue Workflows. ## Usage ```hcl module "glue_workflow" { source = "cloudposse/glue/aws//modules/glue-workflow" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" workflow_name = "geo" workflow_description = "Glue workflow to process geo data" max_concurrent_runs = 2 default_run_properties = {} } ``` --- ## guardduty(Guardduty) # Module: `guardduty` This module enables AWS GuardDuty in one region of one account and optionally sets up an SNS topic to receive notifications of its findings. ## Usage Here's how to invoke this example module in your projects ```hcl module "example" { source = "https://github.com/cloudposse/terraform-aws-guardduty.git?ref=master" create_sns_topic = true subscribers = { opsgenie = { protocol = "https" endpoint = "https://api.example.com/v1/" endpoint_auto_confirms = true } } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-guardduty/tree/master/examples/complete) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`cloudwatch_event_rule_pattern_detail_type` (`string`) optional
The detail-type pattern used to match events that will be sent to SNS. For more information, see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html https://docs.aws.amazon.com/eventbridge/latest/userguide/event-types.html https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html **Default value:** `"GuardDuty Finding"`
`create_sns_topic` (`bool`) optional
Flag to indicate whether an SNS topic should be created for notifications. If you want to send findings to a new SNS topic, set this to true and provide a valid configuration for subscribers. **Default value:** `false`
`enable_cloudwatch` (`bool`) optional
Flag to indicate whether an CloudWatch logging should be enabled for GuardDuty **Default value:** `false`
`finding_publishing_frequency` (`string`) optional
The frequency of notifications sent for finding occurrences. If the detector is a GuardDuty member account, the value is determined by the GuardDuty master account and cannot be modified, otherwise it defaults to SIX_HOURS. For standalone and GuardDuty master accounts, it must be configured in Terraform to enable drift detection. Valid values for standalone and master accounts: FIFTEEN_MINUTES, ONE_HOUR, SIX_HOURS." For more information, see: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_findings_cloudwatch.html#guardduty_findings_cloudwatch_notification_frequency **Default value:** `null`
`findings_notification_arn` (`string`) optional
The ARN for an SNS topic to send findings notifications to. This is only used if create_sns_topic is false. If you want to send findings to an existing SNS topic, set the value of this to the ARN of the existing topic and set create_sns_topic to false. **Default value:** `null`
`s3_protection_enabled` (`bool`) optional
Flag to indicate whether S3 protection will be turned on in GuardDuty. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector **Default value:** `false`
`subscribers` optional
A map of subscription configurations for SNS topics For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription#argument-reference protocol: The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see link) (email is an option but is unsupported in terraform, see link). endpoint: The endpoint to send data to, the contents will vary with the protocol. (see link for more information) endpoint_auto_confirms: Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty. Default is false raw_message_delivery: Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property). Default is false **Type:** ```hcl map(object({ protocol = string endpoint = string endpoint_auto_confirms = bool raw_message_delivery = bool })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`guardduty_detector`
GuardDuty detector
`sns_topic`
SNS topic
`sns_topic_subscriptions`
SNS topic subscriptions
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3` ### Providers - `aws`, version: `>= 3` ### Modules Name | Version | Source | Description --- | --- | --- | --- `findings_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sns_topic` | 0.20.1 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.20.1) | ----------------------------------------------------------------------------------------------------------------------- Optionally configure Event Bridge Rules and SNS subscriptions https://docs.aws.amazon.com/securityhub/latest/userguide/securityhub-cwe-integration-types.html https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/resource-based-policies-cwe.html#sns-permissions ----------------------------------------------------------------------------------------------------------------------- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.findings`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.imported_findings`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_guardduty_detector.guardduty`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/guardduty_detector) (resource) - [`aws_sns_topic_policy.sns_topic_publish_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.sns_topic_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## health-events # Module: `health-events` This module creates EventBridge (formerly CloudWatch Events) rules for AWS Personal Health Dashboard Events and an SNS topic. EventBridge will publish messages to this SNS topic, which can be subcribed to using this module as well. Since AWS Personal Health Dashboard is a global service, but since the KMS key and SNS topic are regional, this module is technically regional but only needs to be deployed once per account. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-health-events/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-health-events/tree/main/test). Here's how to invoke this module in your projects: ```hcl module "monitor_yaml_config" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" map_config_local_base_path = path.module map_config_paths = ["catalog/event_rules/*.yaml"] context = module.this.context } module "health_events" { source = "cloudposse/health-events/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" event_rules = module.event_rules_yaml_config.list_configs subscribers = { opsgenie = { protocol = "https" endpoint = "https://api.example.com/v1/" endpoint_auto_confirms = true raw_message_delivery = false } } context = module.this.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-health-events/examples/complete) - complete example of using this module ## Variables ### Required Variables
`event_rules` required
A list of Event Rules to use when creating EventBridge Rules for the AWS Personal Health Service. name: The base name of the Event Rule (final name will be used in tandem with context attributes) description: The description to use for the Event Rule event_rule_pattern: The Event Rule Pattern to use when creating the Event Rule. For more information, see: https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-events.html **Type:** ```hcl list(object({ name = string description = string event_rule_pattern = object({ detail = object({ service = string event_type_category = string event_type_codes = list(string) }) }) })) ```
### Optional Variables
`kms_master_key_id` (`string`) optional
The ID or alias of the customer master key (CMK) to use for encrypting the Amazon SNS topic. The CMK must have its resource-based policy allow the service `cloudwatch.amazonaws.com` to perform `kms:Decrypt` and `kms:GenerateDataKey` on it. If this variable is not supplied, a CMK with the sufficient resource-based policy will be created and used when configuring encryption for the SNS topic. **Default value:** `null`
`subscribers` optional
Required configuration for subscribers to SNS topic. **Type:** ```hcl map(object({ protocol = string # The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see below) (email is an option but is unsupported, see below). endpoint = string # The endpoint to send data to, the contents will vary with the protocol. (see below for more information) endpoint_auto_confirms = bool # Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty (default is false) raw_message_delivery = bool # Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property) (default is false) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`event_rule_arns`
The ARNs of the created EventBridge Rules.
`event_rule_names`
The names of the created EventBridge Rules.
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `health_events_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sns_kms_key` | 0.12.1 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.1) | n/a `sns_topic` | 0.20.1 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.20.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.health_events`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.sns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sns_kms_key_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## helm-release # Module: `helm-release` This `terraform-aws-helm-release` module deploys a [Helm chart](https://helm.sh/docs/topics/charts/) with an option to create an EKS IAM Role for a Service Account ([IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)). ## Usage This module deploys a [Helm chart](https://helm.sh/docs/topics/charts/) with an option to create an EKS IAM Role for a Service Account ([IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html)). It has many of the same features and limitations of Helm, and uses the Terraform [Helm provider](https://github.com/hashicorp/terraform-provider-helm), specifically the [helm_release](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) resource. NOTE: This module is just a convenient wrapper, packaging up 3 concepts: 1. Deploying a Helm Chart to an EKS cluster 1. Creating a Kubernetes namespace in the EKS cluster 1. Creating an IAM role for a Kubernetes Service Account (which, in turn, is presumably created by deploying the Helm Chart) Many issues may arise that are due to limitations of Helm, Kubernetes, EKS, Terraform, or the Terraform providers. Please address issues and complaints to the project that can potentially fix them, which will usually not be this module. ### Provider requirements. This module is unusual in that it requires you to configure 3 separate Terraform providers: 1. AWS 2. Helm 3. Kubernetes Cloud Posse maintains a [provider-helm.tf](https://github.com/cloudposse/terraform-aws-components/blob/master/mixins/provider-helm.tf) file "mixin" for use in Cloud Posse [components](https://github.com/cloudposse/terraform-aws-components) which you can also use as an example of how to configure the Helm and Kubernetes providers in your own component. ### Creating a namespace This module provides 2 options for creating the namespace the chart will be deployed to, for the case where you are deploying the chart into its own namespace that does not already exist. 1. `create_namespace_with_kubernetes` will manage the namespace using a Terraform `kubernetes_namespace` resource. This is the recommended way to create the namespace, because it allows you to annotate (`kubernetes_namespace_annotations`) and label (`kubernetes_namespace_labels`) the namespace, and it provides proper sequencing of creation and destruction of deployments, resources, and IAM roles. When the deployment is destroyed with `terraform destroy`, the namespace will be deleted, too. This will delete everything else in the namespace (but never the Custom Resource Definitions, which themselves are non-namespaced), so if this is not the desired behavior, you should create the namespace in a separate Terraform component. 1. `create_namespace` is the obsolete way to create a namespace, by delegating the responsibility to Helm. This is not recommended because it provides no control over the annotations or labels of the namespace, and when the deployment is destroyed with `terraform destroy`, the namespace will be left behind. This can cause problems with future deployments. Note: You may have trouble deleting a release from within Terraform if the Kubernetes cluster has been modified outside of this module, for example if the namespace or the cluster itself has been deleted. You can delete the Terraform state if the resources are gone, using `terraform state rm` or even `terraform workspace delete`, or you can try using `terraform destroy`. In some cases, it may be helpful to set `var.enabled` to `false` while destroying: ```shell terraform destroy -var enabled=false ``` For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-helm-release/tree/main/examples/complete). ```hcl module "helm_release" { source = "cloudposse/helm-release/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "echo" repository = "https://charts.helm.sh/incubator" chart = "raw" chart_version = "0.2.5" create_namespace = true kubernetes_namespace = "echo" atomic = true cleanup_on_fail = true timeout = 300 wait = true # These values will be deep merged # values = [ # ] # Enable the IAM role iam_role_enabled = true # Add the IAM role using set() service_account_role_arn_annotation_enabled = true # Dictates which ServiceAccounts are allowed to assume the IAM Role. # In this case, only the "echo" ServiceAccount in the "echo" namespace # will be able to assume the IAM Role created by this module. service_account_name = "echo" service_account_namespace = "echo" # IAM policy statements to add to the IAM role iam_policy = [{ statements = [{ sid = "ListMyBucket" effect = "Allow" actions = ["s3:ListBucket"] resources = ["arn:aws:s3:::test"] conditions = [] }, { sid = "WriteMyBucket" effect = "Allow" actions = ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"] resources = ["arn:aws:s3:::test/*"] conditions = [] }] }] } ``` Typically, the prefix for the full name of the created IAM role for the service account ends with the `name` value, supplied either via the `name` or the `context` input. If `service_account_name` is set to something other than `*`, the service account name is then appended to this prefix. In the case where `name` and `service_account_name` are the same, this leads to a repetition, for a name like `eg-echo-echo`. For this reason, we recommend setting `name` to "" when it would otherwise be the same as `service_account_name`: ```hcl module "helm_release" { source = "cloudposse/helm-release/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "" create_namespace = true kubernetes_namespace = "echo" service_account_name = "echo" service_account_namespace = "echo" iam_role_enabled = true service_account_role_arn_annotation_enabled = true # ... } ``` This will result in an IAM role with a name such as: `eg-uw2-dev-echo@echo` instead of `eg-uw2-dev-echo-echo@echo`. Additionally, if `var.name` is empty, the name used for the Helm Release will be that of `var.chart`. ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-helm-release/) - complete example of using this module ## Variables ### Required Variables
`chart` (`string`) required
Chart name to be installed. The chart name can be local path, a URL to a chart, or the name of the chart if `repository` is specified. It is also possible to use the `/` format here if you are running Terraform on a system that the repository has been added to with `helm repo add` but this is not recommended.
`eks_cluster_oidc_issuer_url` (`string`) required
OIDC issuer URL for the EKS cluster (initial "https://" may be omitted).
### Optional Variables
`atomic` (`bool`) optional
If set, installation process purges chart on fail. The wait flag will be set automatically if atomic is used. Defaults to `false`. **Default value:** `null`
`aws_account_number` (`string`) optional
AWS account number of EKS cluster owner. **Default value:** `null`
`aws_partition` (`string`) optional
AWS partition: `aws`, `aws-cn`, or `aws-us-gov`. Applicable when `var.iam_role_enabled` is `true`. **Default value:** `"aws"`
`chart_version` (`string`) optional
Specify the exact chart version to install. If this is not specified, the latest version is installed. **Default value:** `null`
`cleanup_on_fail` (`bool`) optional
Allow deletion of new resources created in this upgrade when upgrade fails. Defaults to `false`. **Default value:** `null`
`create_namespace` (`bool`) optional
(Not recommended, use `create_namespace_with_kubernetes` instead) Create the namespace via Helm if it does not yet exist. Defaults to `false`. Does not support annotations or labels. May have problems when destroying. Ignored when `create_namespace_with_kubernetes` is set. **Default value:** `null`
`create_namespace_with_kubernetes` (`bool`) optional
Create the namespace via Kubernetes if it does not yet exist. Defaults to `false`. Must set `true` if you want to use namespace annotations or labels. **Default value:** `null`
`dependency_update` (`bool`) optional
Runs helm dependency update before installing the chart. Defaults to `false`. **Default value:** `null`
`description` (`string`) optional
Release description attribute (visible in the history). **Default value:** `null`
`devel` (`bool`) optional
Use chart development versions, too. Equivalent to version `>0.0.0-0`. If version is set, this is ignored. **Default value:** `null`
`disable_openapi_validation` (`bool`) optional
If set, the installation process will not validate rendered templates against the Kubernetes OpenAPI Schema. Defaults to `false`. **Default value:** `null`
`disable_webhooks` (`bool`) optional
Prevent hooks from running. Defaults to `false`. **Default value:** `null`
`force_update` (`bool`) optional
Force resource update through delete/recreate if needed. Defaults to `false`. **Default value:** `null`
`iam_override_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document with higher precedence. In merging, statements with non-blank SIDs will override statements with the same SID from earlier documents in the list and from other "source" documents. **Default value:** `null`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `null`
`iam_policy_enabled` (`bool`) optional
Whether to create and attach an IAM policy to the created IAM role **Default value:** `true`
`iam_policy_statements` (`any`) optional
Deprecated: Use `iam_policy` instead. List or Map of IAM policy statements to use in the policy. This can be used with `iam_source_policy_documents` and `iam_override_policy_documents` and with or instead of `iam_source_json_url`. **Default value:** `{ }`
`iam_role_enabled` (`bool`) optional
Whether to create an IAM role. Setting this to `true` will also replace any occurrences of `{service_account_role_arn}` in `var.values_template_path` with the ARN of the IAM role created by this module. **Default value:** `false`
`iam_source_json_url` (`string`) optional
URL of the IAM policy (in JSON format) to download and use as `source_json` argument. This is useful when using a 3rd party service that provides their own policy. Statements in this policy will be overridden by statements with the same SID in `iam_override_policy_documents`. **Default value:** `null`
`iam_source_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document. Statements defined in `iam_source_policy_documents` must have unique SIDs and be distinct from SIDs in `iam_policy` and deprecated `iam_policy_statements`. Statements in these documents will be overridden by statements with the same SID in `iam_override_policy_documents`. **Default value:** `null`
`keyring` (`string`) optional
Location of public keys used for verification. Used only if `verify` is true. Defaults to `/.gnupg/pubring.gpg` in the location set by `home`. **Default value:** `null`
`kubernetes_namespace` (`string`) optional
The namespace to install the release into. Defaults to `default`. **Default value:** `null`
`kubernetes_namespace_annotations` (`map(string)`) optional
Annotations to be added to the created namespace. Ignored unless `create_namespace_with_kubernetes` is `true`. **Default value:** `{ }`
`kubernetes_namespace_labels` (`map(string)`) optional
Labels to be added to the created namespace. Ignored unless `create_namespace_with_kubernetes` is `true`. **Default value:** `{ }`
`lint` (`bool`) optional
Run the helm chart linter during the plan. Defaults to `false`. **Default value:** `null`
`max_history` (`number`) optional
Maximum number of release versions stored per release. Defaults to `0` (no limit). **Default value:** `null`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role. **Default value:** `null`
`postrender_binary_path` (`string`) optional
Relative or full path to command binary. **Default value:** `null`
`recreate_pods` (`bool`) optional
Perform pods restart during upgrade/rollback. Defaults to `false`. **Default value:** `null`
`release_name` (`string`) optional
The name of the release to be installed. If omitted, use the name input, and if that's omitted, use the chart input. **Default value:** `""`
`render_subchart_notes` (`bool`) optional
If set, render subchart notes along with the parent. Defaults to `true`. **Default value:** `null`
`replace` (`bool`) optional
Re-use the given name, even if that name is already used. This is unsafe in production. Defaults to `false`. **Default value:** `null`
`repository` (`string`) optional
Repository URL where to locate the requested chart. **Default value:** `null`
`repository_ca_file` (`string`) optional
The Repositories CA file. **Default value:** `null`
`repository_cert_file` (`string`) optional
The repositories cert file. **Default value:** `null`
`repository_key_file` (`string`) optional
The repositories cert key file. **Default value:** `null`
`repository_password` (`string`) optional
Password for HTTP basic authentication against the repository. **Default value:** `null`
`repository_username` (`string`) optional
Username for HTTP basic authentication against the repository. **Default value:** `null`
`reset_values` (`bool`) optional
When upgrading, reset the values to the ones built into the chart. Defaults to `false`. **Default value:** `null`
`reuse_values` (`bool`) optional
When upgrading, reuse the last release's values and merge in any overrides. If `reset_values` is specified, this is ignored. Defaults to `false`. **Default value:** `null`
`service_account_name` (`string`) optional
Name of the Kubernetes ServiceAccount allowed to assume the IAM role created when `var.iam_role_enabled` is set to `true`. In combination with `var.service_account_namespace`, this variable is used to determine which ServiceAccounts are allowed to assume the IAM role in question. It is *not* recommended to leave this variable as `null` or `""`, as this would mean ServiceAccounts of any name in the namespace specified by `var.service_account_namespace` are allowed to assume the IAM role in question. If both variables are omitted, then a ServiceAccount of any name in any namespace will be able to assume the IAM role in question, which is the least secure scenario. The best practice is to set this variable to the name of the ServiceAccount created by the Helm Chart. **Default value:** `null`
`service_account_namespace` (`string`) optional
Kubernetes Namespace of the Kubernetes ServiceAccount allowed to assume the IAM role created when `var.iam_role_enabled` is set to `true`. In combination with `var.service_account_name`, this variable is used to determine which ServiceAccounts are allowed to assume the IAM role in question. It is *not* recommended to leave this variable as `null` or `""`, as this would mean any ServiceAccounts matching the name specified by `var.service_account_name` in any namespace are allowed to assume the IAM role in question. If both variables are omitted, then a ServiceAccount of any name in any namespace will be able to assume the IAM role in question, which is the least secure scenario. The best practice is to set this variable to the namespace of the ServiceAccount created by the Helm Chart. **Default value:** `null`
`service_account_role_arn_annotation_enabled` (`bool`) optional
Whether or not to dynamically insert an `eks.amazonaws.com/role-arn` annotation into `$var.service_account_set_key_path.annotations` (by default, `serviceAccount.annotations`), with the value being the ARN of the IAM role created when `var.iam_role_enabled`. Assuming the Helm Chart follows the standard convention of rendering ServiceAccount annotations in `serviceAccount.annotations` (or a similar convention, which can be overriden by `var.service_account_set_key_path` as needed), this allows the ServiceAccount created by the Helm Chart to assume the IAM Role in question via the EKS OIDC IdP, without the consumer of this module having to set this annotation via `var.values` or `var.set`, which would involve manually rendering the IAM Role ARN beforehand. Ignored if `var.iam_role_enabled` is `false`. **Default value:** `true`
`service_account_set_key_path` (`string`) optional
The key path used by Helm Chart values for ServiceAccount-related settings (e.g. `serviceAccount...` or `rbac.serviceAccount...`). Ignored if either `var.service_account_role_arn_annotation_enabled` or `var.iam_role_enabled` are set to `false`. **Default value:** `"serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"`
`set` optional
Value block with custom values to be merged with the values yaml. **Type:** ```hcl list(object({ name = string value = string type = string })) ``` **Default value:** `[ ]`
`set_sensitive` optional
Value block with custom sensitive values to be merged with the values yaml that won't be exposed in the plan's diff. **Type:** ```hcl list(object({ name = string value = string type = string })) ``` **Default value:** `[ ]`
`skip_crds` (`bool`) optional
If set, no CRDs will be installed. By default, CRDs are installed if not already present. Defaults to `false`. **Default value:** `null`
`timeout` (`number`) optional
Time in seconds to wait for any individual kubernetes operation (like Jobs for hooks). Defaults to `300` seconds. **Default value:** `null`
`values` (`any`) optional
List of values in raw yaml to pass to helm. Values will be merged, in order, as Helm does with multiple `-f` options. **Default value:** `null`
`verify` (`bool`) optional
Verify the package before installing it. Helm uses a provenance file to verify the integrity of the chart; this must be hosted alongside the chart. For more information see the Helm Documentation. Defaults to `false`. **Default value:** `null`
`wait` (`bool`) optional
Will wait until all resources are in a ready state before marking the release as successful. It will wait for as long as `timeout`. Defaults to `true`. **Default value:** `null`
`wait_for_jobs` (`bool`) optional
If wait is enabled, will wait until all Jobs have been completed before marking the release as successful. It will wait for as long as `timeout`. Defaults to `false`. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`metadata`
Block status of the deployed release.
`service_account_name`
Kubernetes Service Account name
`service_account_namespace`
Kubernetes Service Account namespace
`service_account_policy_arn`
IAM policy ARN
`service_account_policy_id`
IAM policy ID
`service_account_policy_name`
IAM policy name
`service_account_role_arn`
IAM role ARN
`service_account_role_name`
IAM role name
`service_account_role_unique_id`
IAM role unique ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `helm`, version: `>= 2.2` - `kubernetes`, version: `>= 2.7.1` ### Providers - `helm`, version: `>= 2.2` - `kubernetes`, version: `>= 2.7.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eks_iam_policy` | 2.0.1 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.1) | n/a `eks_iam_role` | 2.1.1 | [`cloudposse/eks-iam-role/aws`](https://registry.terraform.io/modules/cloudposse/eks-iam-role/aws/2.1.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`helm_release.this`](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/resources/release) (resource) - [`kubernetes_namespace.default`](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/namespace) (resource) ## Data Sources The following data sources are used by this module: --- ## iam-account-settings # Module: `iam-account-settings` Terraform module to provision general IAM account settings. It will create the IAM account alias for pretty login URLs and set the account password policy." ## Usage ```hcl module "account_settings" { source = "cloudposse/iam-account-settings/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "account" allow_users_to_change_password = true minimum_password_length = 20 password_reuse_prevention = 30 } output "account_alias" { value = "${module.account_settings.account_alias}" } output "signin_url" { value = "${module.account_settings.signin_url}" } ``` ## Security Info For better compliance with modern security best practices the following rules are enabled for the AWS account: Initial password creation (upon user creation) requires password to be reset on first login * Minimum password length: 16 chars * Requires at least one upper case letter * Requires at least one lower case letter * Requires at least one number * Requires at least one alphanumeric character * Passwords expire after 90 days * Passwords may not be repeated more than every 5th time you reset a password * Furthermore, MFA is required to assume a role (access to dev/staging/prod accounts), which is how access is granted to any and all AWS resources for staging/production. These password requirements are a best effort to meet PCI DSS v3.2.1 regulations for passwords and MFA, as published in May 2018 (in this instance, only for AWS) https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-2-1.pdf Source Sections (starting on page 72): * 8.2.3 * 8.2.4 * 8.2.5 * 8.2.6 * 8.4 (documentation only) ## Variables ### Required Variables
### Optional Variables
`allow_users_to_change_password` (`bool`) optional
Whether to allow users to change their own password **Default value:** `true`
`hard_expiry` (`bool`) optional
Whether users are prevented from setting a new password after their password has expired (i.e. require administrator reset) **Default value:** `false`
`max_password_age` (`number`) optional
The number of days that a user's password is valid **Default value:** `90`
`minimum_password_length` (`number`) optional
Minimum length to require for user passwords **Default value:** `14`
`password_policy_enabled` (`bool`) optional
Whether or not to create the IAM account password policy **Default value:** `true`
`password_reuse_prevention` (`number`) optional
The number of previous passwords that users are prevented from reusing **Default value:** `24`
`require_lowercase_characters` (`bool`) optional
Whether to require lowercase characters for user passwords **Default value:** `true`
`require_numbers` (`bool`) optional
Whether to require numbers for user passwords **Default value:** `true`
`require_symbols` (`bool`) optional
Whether to require symbols for user passwords **Default value:** `true`
`require_uppercase_characters` (`bool`) optional
Whether to require uppercase characters for user passwords **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`account_alias`
IAM account alias
`minimum_password_length`
IAM account minimum password length
`signin_url`
IAM users sign-in URL
## Dependencies ### Requirements - `terraform`, version: `>= 0.12.26` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_account_alias.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_account_alias) (resource) - [`aws_iam_account_password_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_account_password_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) --- ## iam-assumed-roles # Module: `iam-assumed-roles` Terraform module to provision two IAM roles and two IAM groups for assuming the roles provided MFA is present, and add IAM users to the groups. - Role and group with Administrator (full) access to AWS resources - Role and group with Readonly access to AWS resources To give a user administrator's access, add the user to the admin group. To give a user readonly access, add the user to the readonly group. ## Usage ```hcl module "assumed_roles" { source = "git::https://github.com/cloudposse/terraform-aws-iam-assumed-roles.git?ref=master" namespace = "cp" stage = "prod" admin_name = "admin" readonly_name = "readonly" admin_user_names = ["User1","User2"] # Add these IAM users to the admin group readonly_user_names = ["User3","User4"] # Add these IAM users to the readonly group } ``` ## Variables ### Required Variables
### Optional Variables
`admin_name` (`string`) optional
Name for the admin group and role (e.g. `admin`) **Default value:** `"admin"`
`admin_user_names` (`list(string)`) optional
Optional list of IAM user names to add to the admin group **Default value:** `[ ]`
`readonly_name` (`string`) optional
Name for the readonly group and role (e.g. `readonly`) **Default value:** `"readonly"`
`readonly_user_names` (`list(string)`) optional
Optional list of IAM user names to add to the readonly group **Default value:** `[ ]`
`switchrole_url` (`string`) optional
URL to the IAM console to switch to a role **Default value:** `"https://signin.aws.amazon.com/switchrole?account=%s&roleName=%s&displayName=%s"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`namespace` (`string`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `policy` or `role`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `stage`, `name`, and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `"true"`
`tags` (`map(string)`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`group_admin_arn`
Admin group ARN
`group_admin_id`
Admin group ID
`group_admin_name`
Admin group name
`group_readonly_arn`
Readonly group ARN
`group_readonly_id`
Readonly group ID
`group_readonly_name`
Readonly group name
`role_admin_arn`
Admin role ARN
`role_admin_name`
Admin role name
`role_readonly_arn`
Readonly role ARN
`role_readonly_name`
Readonly role name
`switchrole_admin_url`
URL to the IAM console to switch to the admin role
`switchrole_readonly_url`
URL to the IAM console to switch to the readonly role
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `admin_label` | tags/0.3.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.3) | n/a `readonly_label` | tags/0.3.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.3) | n/a ## Resources The following resources are used by this module: - [`aws_iam_group.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group) (resource) - [`aws_iam_group.readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group) (resource) - [`aws_iam_group_membership.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_membership) (resource) - [`aws_iam_group_membership.readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_membership) (resource) - [`aws_iam_group_policy_attachment.allow_chage_password_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.allow_change_password_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.assume_role_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.assume_role_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.key_management_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.key_management_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.manage_mfa_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_group_policy_attachment.manage_mfa_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy_attachment) (resource) - [`aws_iam_policy.allow_change_password_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.allow_change_password_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.allow_key_management_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.allow_key_management_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.assume_role_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.assume_role_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.manage_mfa_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.manage_mfa_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.allow_change_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.allow_key_management`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.assume_role_admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.assume_role_readonly`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.manage_mfa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.role_trust`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## iam-policy(Iam-policy) # Module: `iam-policy` This `terraform-aws-iam-policy` module is a wrapper around the Terraform [aws_iam_policy_document](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) data source, enhancing it to provide multiple ways to create an AWS IAM Policy document (as a JSON string). It is primarily intended to simplify creating a policy in Terraform from external inputs. In particular, if you want to specify a policy in a `tfvars` file as a Terraform object, or in YAML as part of an [Atmos](https://atmos.tools/) stack (which is them turned into a Terraform object input), this module provides an object type declaration to use for the input and then it can make the translation to JSON for you. If you can supply the policy as JSON to begin with, or conveniently use the `aws_iam_policy_document` Terraform data source directly, then this module is not helpful in your case. :::note AWS's IAM policy document syntax allows for replacement of policy variables within a statement using `${...}`-style notation, which conflicts with Terraform's interpolation syntax. In order to use AWS policy variables with this module, use `&{...}` notation for interpolations that should be processed by AWS rather than by Terraform. Nevertheless, any `${...}`-style notations that appear in strings passed into this module (somehow escaping Terraform interpolation earlier) will be passed through to the policy document unchanged. ::: ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-iam-policy/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-iam-policy/tree/main/test). ```hcl module "iam_policy" { source = "cloudposse/iam-policy/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" iam_policy = [{ version = "2012-10-17" policy_id = "example" statements = [ { sid = "ListMyBucket" effect = "Allow" actions = ["s3:ListBucket"] resources = ["arn:aws:s3:::test"] conditions = [ { test = "StringLike" variable = "cloudwatch:namespace" values = ["x-*"] }, ] }, { sid = "WriteMyBucket" effect = "Allow" actions = ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"] resources = ["arn:aws:s3:::test/*"] conditions = [ { test = "StringLike" variable = "cloudwatch:namespace" values = ["x-*"] }, ] } ] }] data "aws_iam_policy_document" "assume_role" { statement { actions = ["sts:AssumeRole"] principals { type = "Service" identifiers = ["ec2.amazonaws.com"] } } } resource "aws_iam_role" "example" { name = "hello_role" assume_role_policy = data.aws_iam_policy_document.assume_role.json inline_policy { name = "test_policy" policy = module.iam_policy.json } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-iam-policy/) - overly basic example of using this module - [terraform-aws-helm-release](https://github.com/cloudposse/terraform-aws-helm-release) - realistic use of this module ## Variables ### Required Variables
### Optional Variables
`description` (`string`) optional
Description of created IAM policy **Default value:** `null`
`iam_override_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document with higher precedence. In merging, statements with non-blank SIDs will override statements with the same SID from earlier documents in the list and from other "source" documents. **Default value:** `null`
`iam_policy` optional
IAM policy as list of Terraform objects, compatible with Terraform `aws_iam_policy_document` data source except that `source_policy_documents` and `override_policy_documents` are not included. Use inputs `iam_source_policy_documents` and `iam_override_policy_documents` for that. **Type:** ```hcl list(object({ policy_id = optional(string, null) version = optional(string, null) statements = list(object({ sid = optional(string, null) effect = optional(string, null) actions = optional(list(string), null) not_actions = optional(list(string), null) resources = optional(list(string), null) not_resources = optional(list(string), null) conditions = optional(list(object({ test = string variable = string values = list(string) })), []) principals = optional(list(object({ type = string identifiers = list(string) })), []) not_principals = optional(list(object({ type = string identifiers = list(string) })), []) })) })) ``` **Default value:** `[ ]`
`iam_policy_enabled` (`bool`) optional
Whether to create the IAM managed policy in AWS or not. If false without role_names, it will output the JSON policy, with role_names, it will attach the inline policy. **Default value:** `false`
`iam_policy_id` (`string`) optional
Deprecated: Use `iam_policy` instead: ID for the policy document when using `iam_policy_statements`. **Default value:** `null`
`iam_policy_statements` (`any`) optional
Deprecated: Use `iam_policy` instead. List or Map of IAM policy statements to use in the policy. This can be used with `iam_source_policy_documents` and `iam_override_policy_documents` and with or instead of `iam_source_json_url`. **Default value:** `[ ]`
`iam_source_json_url` (`string`) optional
URL of the IAM policy (in JSON format) to download and use as `source_json` argument. This is useful when using a 3rd party service that provides their own policy. Statements in this policy will be overridden by statements with the same SID in `iam_override_policy_documents`. **Default value:** `null`
`iam_source_policy_documents` (`list(string)`) optional
List of IAM policy documents (as JSON strings) that are merged together into the exported document. Statements defined in `iam_source_policy_documents` must have unique SIDs and be distinct from SIDs in `iam_policy` and deprecated `iam_policy_statements`. Statements in these documents will be overridden by statements with the same SID in `iam_override_policy_documents`. **Default value:** `null`
`role_names` (`list(string)`) optional
IAM role names to attach the policy to. Use iam_policy_enabled to toggle between managed or inline. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`json`
JSON body of the IAM policy document
`policy_arn`
ARN of created IAM policy
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0` - `http`, version: `>= 3.0` ### Providers - `aws`, version: `>= 4.0` - `http`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`http_http.iam_source_json_url`](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) (data source) --- ## iam-role(Iam-role) # Module: `iam-role` A Terraform module that creates IAM role with provided JSON IAM polices documents. #### Warning * If `var.enabled` set `false` the module can be used as [IAM Policy Document Aggregator](https://github.com/cloudposse/terraform-aws-iam-policy-document-aggregator) because [`output.policy`](https://github.com/cloudposse/terraform-aws-iam-role/tree/init#outputs) always aggregates [`var.policy_documents`](https://github.com/cloudposse/terraform-aws-iam-role/tree/init#inputs) * List size [`var.policy_documents`](https://github.com/cloudposse/terraform-aws-iam-role/tree/init#inputs) [limited to 10](https://github.com/cloudposse/terraform-aws-iam-policy-document-aggregator#inputs) ## Usage This example creates a role with the name `eg-prod-app` with permission to grant read-write access to S3 bucket, and gives permission to the entities specified in `principals_arns` to assume the role. ```hcl data "aws_iam_policy_document" "resource_full_access" { statement { sid = "FullAccess" effect = "Allow" resources = ["arn:aws:s3:::bucketname/path/*"] actions = [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] } } data "aws_iam_policy_document" "base" { statement { sid = "BaseAccess" actions = [ "s3:ListBucket", "s3:ListBucketVersions" ] resources = ["arn:aws:s3:::bucketname"] effect = "Allow" } } module "role" { source = "cloudposse/iam-role/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" enabled = true namespace = "eg" stage = "prod" name = "app" policy_description = "Allow S3 FullAccess" role_description = "IAM role with permissions to perform actions on S3 resources" principals = { AWS = ["arn:aws:iam::123456789012:role/workers"] } policy_documents = [ data.aws_iam_policy_document.resource_full_access.json, data.aws_iam_policy_document.base.json ] } ``` ## Examples For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-iam-role/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-iam-role/tree/main/test). ## Variables ### Required Variables
`role_description` (`string`) required
The description of the IAM role that is visible in the IAM role manager
### Optional Variables
`assume_role_actions` (`list(string)`) optional
The IAM action to be granted by the AssumeRole policy **Default value:** ```hcl [ "sts:AssumeRole", "sts:TagSession" ] ```
`assume_role_conditions` optional
List of conditions for the assume role policy **Type:** ```hcl list(object({ test = string variable = string values = list(string) })) ``` **Default value:** `[ ]`
`assume_role_policy` (`string`) optional
A JSON assume role policy document. If set, this will be used as the assume role policy and the principals, assume_role_conditions, and assume_role_actions variables will be ignored. **Default value:** `null`
`inline_policy_enabled` (`bool`) optional
Whether or not to enable an inline policy instead of a reusable managed policy **Default value:** `false`
`instance_profile_enabled` (`bool`) optional
Create EC2 Instance Profile for the role **Default value:** `false`
`managed_policy_arns` (`set(string)`) optional
List of managed policies to attach to created role **Default value:** `[ ]`
`max_session_duration` (`number`) optional
The maximum session duration (in seconds) for the role. Can have a value from 1 hour to 12 hours **Default value:** `3600`
`path` (`string`) optional
Path to the role and policy. See [IAM Identifiers](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_identifiers.html) for more information. **Default value:** `"/"`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role **Default value:** `""`
`policy_description` (`string`) optional
The description of the IAM policy that is visible in the IAM policy manager **Default value:** `""`
`policy_document_count` (`number`) optional
Number of policy documents (length of policy_documents list) **Default value:** `1`
`policy_documents` (`list(string)`) optional
List of JSON IAM policy documents **Default value:** `[ ]`
`policy_name` (`string`) optional
The name of the IAM policy that is visible in the IAM policy manager **Default value:** `null`
`principals` (`map(list(string))`) optional
Map of service name as key and a list of ARNs to allow assuming the role as value (e.g. map(`AWS`, list(`arn:aws:iam:::role/admin`))) **Default value:** `{ }`
`use_fullname` (`bool`) optional
If set to 'true' then the full ID for the IAM role name (e.g. `[var.namespace]-[var.environment]-[var.stage]`) will be used. Otherwise, `var.name` will be used for the IAM role name. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The Amazon Resource Name (ARN) specifying the role
`id`
The stable and unique string identifying the role
`instance_profile`
Name of the ec2 profile (if enabled)
`name`
The name of the IAM role created
`policy`
Role policy document in json format. Outputs always, independent of `enabled` variable
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `role_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_instance_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_instance_profile) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.managed`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.assume_role_aggregated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## iam-s3-user # Module: `iam-s3-user` Terraform module to provision a basic IAM user with permissions to access S3 resources, e.g. to give the user read/write/delete access to the objects in an S3 bucket. Suitable for CI/CD systems (_e.g._ TravisCI, CircleCI) or systems which are *external* to AWS that cannot leverage [AWS IAM Instance Profiles](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) or [AWS OIDC](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html). By default, IAM users, groups, and roles have no access to AWS resources. IAM policies are the means by which privileges are granted to users, groups, or roles. It is recommended that IAM policies be applied directly to groups and roles but not users. **This module intentionally attaches an IAM policy directly to the user and does not use groups** The IAM user name is constructed using [terraform-null-label](https://github.com/cloudposse/terraform-null-label) and some input is required. The simplest input is `name`. By default the name will be converted to lower case and all non-alphanumeric characters except for hyphen will be removed. See the documentation for `terraform-null-label` to learn how to override these defaults if desired. If an AWS Access Key is created, it is stored either in SSM Parameter Store or is provided as a module output, but not both. Using SSM Parameter Store is recommended because module outputs are stored in plaintext in the Terraform state file. ## Usage This example will create an IAM user and allow read access to all objects in the S3 bucket `examplebucket` ```hcl module "s3_user" { source = "cloudposse/iam-s3-user/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" s3_actions = ["s3:GetObject"] s3_resources = "arn:aws:s3:::examplebucket/*" } ``` ## Variables ### Required Variables
`s3_resources` (`list(string)`) required
S3 resources to apply the actions specified in the policy
### Optional Variables
`create_iam_access_key` (`bool`) optional
Set `true` to create an IAM Access Key for the user. To rotate the key, set `false` to delete it and then back to `true` to create a new key. Best practice is to never create a key and instead authenticate with OIDC or some other mechanism that does not require long-lived bearer tokens. **Default value:** `true`
`force_destroy` (`bool`) optional
Destroy even if it has non-Terraform-managed IAM access keys, login profiles or MFA devices **Default value:** `false`
`path` (`string`) optional
Path in which to create the user **Default value:** `"/"`
`permissions_boundary` (`string`) optional
Permissions Boundary ARN to attach to our created user **Default value:** `null`
`s3_actions` (`list(string)`) optional
Actions to allow in the policy **Default value:** ```hcl [ "s3:GetObject" ] ```
`ssm_base_path` (`string`) optional
The base path for SSM parameters where secrets are stored **Default value:** `"/s3_user/"`
`ssm_enabled` (`bool`) optional
Set `true` to store secrets in SSM Parameter Store, `false` to store secrets in Terraform state as outputs. Since Terraform state would contain the secrets in plaintext, use of SSM Parameter Store is recommended. **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_key_id`
Access Key ID
`access_key_id_ssm_path`
The SSM Path under which the S3 User's access key ID is stored
`secret_access_key`
Secret Access Key. This will be written to the state file in plain-text
`secret_access_key_ssm_path`
The SSM Path under which the S3 User's secret access key is stored
`user_arn`
The ARN assigned by AWS for the user
`user_name`
Normalized IAM user name
`user_unique_id`
The user unique ID assigned by AWS
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `s3_user` | 1.2.1 | [`cloudposse/iam-system-user/aws`](https://registry.terraform.io/modules/cloudposse/iam-system-user/aws/1.2.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_user_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## iam-system-user # Module: `iam-system-user` Terraform Module to provision a basic IAM system user suitable for CI/CD Systems (_e.g._ TravisCI, CircleCI) or systems which are *external* to AWS that cannot leverage [AWS IAM Instance Profiles](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html) or [AWS OIDC](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html). We do not recommend creating IAM users this way for any other purpose. By default, IAM users, groups, and roles have no access to AWS resources. IAM policies are the means by which privileges are granted to users, groups, or roles. It is recommended that IAM policies be applied directly to groups and roles but not users. **This module intentionally attaches an IAM policy directly to the user and does not use groups** The IAM user name is constructed using [terraform-null-label](https://github.com/cloudposse/terraform-null-label) and some input is required. The simplest input is `name`. By default the name will be converted to lower case and all non-alphanumeric characters except for hyphen will be removed. See the documentation for `terraform-null-label` to learn how to override these defaults if desired. If an AWS Access Key is created, it is stored either in SSM Parameter Store or is provided as a module output, but not both. Using SSM Parameter Store is recommended because module outputs are stored in plaintext in the Terraform state file. ## Usage ```hcl module "circleci" { source = "cloudposse/iam-system-user/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "circleci" name = "assets" } ``` ## Examples ```hcl module "fluentd_user" { source = "cloudposse/iam-system-user/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "fluentd" policy_arns_map = { logs = "arn:aws:iam::aws:policy/CloudWatchLogsFullAccess" } inline_policies_map = { s3 = data.aws_iam_policy_document.s3_policy.json } } data "aws_iam_policy_document" "s3_policy" { statement { actions = [ "s3:PutObject", "s3:GetObjectAcl", "s3:GetObject", "s3:ListBucket", "s3:PutObjectAcl" ] resources = [ "arn:aws:s3:::bucket_name/*", "arn:aws:s3:::bucket_name/" ] } } ``` ## Variables ### Required Variables
### Optional Variables
`create_iam_access_key` (`bool`) optional
Whether or not to create IAM access keys **Default value:** `true`
`force_destroy` (`bool`) optional
Destroy the user even if it has non-Terraform-managed IAM access keys, login profile or MFA devices **Default value:** `false`
`inline_policies` (`list(string)`) optional
Inline policies to attach to our created user **Default value:** `[ ]`
`inline_policies_map` (`map(string)`) optional
Inline policies to attach (descriptive key => policy) **Default value:** `{ }`
`path` (`string`) optional
Path in which to create the user **Default value:** `"/"`
`permissions_boundary` (`string`) optional
Permissions Boundary ARN to attach to our created user **Default value:** `null`
`policy_arns` (`list(string)`) optional
Policy ARNs to attach to our created user **Default value:** `[ ]`
`policy_arns_map` (`map(string)`) optional
Policy ARNs to attach (descriptive key => arn) **Default value:** `{ }`
`ssm_base_path` (`string`) optional
The base path for SSM parameters where secrets are stored **Default value:** `"/system_user/"`
`ssm_enabled` (`bool`) optional
Set `true` to store secrets in SSM Parameter Store, ` false` to store secrets in Terraform state as outputs. Since Terraform state would contain the secrets in plaintext, use of SSM Parameter Store is recommended. **Default value:** `true`
`ssm_ses_smtp_password_enabled` (`bool`) optional
Whether or not to create an SES SMTP password **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_key_id`
The access key ID
`access_key_id_ssm_path`
The SSM Path under which the IAM User's access key ID is stored
`secret_access_key`
When `ssm_enabled` is `false`, this is the secret access key for the IAM user. This will be written to the state file in plain-text. When `ssm_enabled` is `true`, this output will be empty to keep the value secure.
`secret_access_key_ssm_path`
The SSM Path under which the IAM User's secret access key is stored
`ses_smtp_password_v4`
When `ssm_enabled` is false, this is the secret access key converted into an SES SMTP password by applying AWS's Sigv4 conversion algorithm. It will be written to the Terraform state file in plaintext. When `ssm_enabled` is `true`, this output will be empty to keep the value secure.
`ses_smtp_password_v4_ssm_path`
The SSM Path under which the IAM User's SES SMTP password is stored
`ssm_enabled`
`true` when secrets are stored in SSM Parameter store, `false` when secrets are stored in Terraform state as outputs.
`user_arn`
The ARN assigned by AWS for this user
`user_name`
Normalized IAM user name
`user_unique_id`
The unique ID assigned by AWS
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `store_write` | 0.13.0 | [`cloudposse/ssm-parameter-store/aws`](https://registry.terraform.io/modules/cloudposse/ssm-parameter-store/aws/0.13.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_access_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_access_key) (resource) - [`aws_iam_user.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) (resource) - [`aws_iam_user_policy.inline_policies`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy) (resource) - [`aws_iam_user_policy_attachment.policies`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: --- ## iam-user # Module: `iam-user` Terraform Module to provision a basic IAM user suitable for humans. It will establish a login profile and associate the user with IAM groups. We do not recommend creating IAM users for any other purpose. For external systems (e.g. CI/CD) check out our [`terraform-aws-iam-system-user`](https://github.com/cloudposse/terraform-aws-iam-system-user) module. ## Usage First, make sure all users register with [keybase.io](https://keybase.io) to faciliate public key encryption (PKE). Then use their keybase username to automatically encrypt their AWS secret. It's that easy! ```hcl module "jack" { source = "cloudposse/iam-user/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "jack" user_name = "jack@companyname.com" pgp_key = "keybase:jack" groups = ["admins"] } ``` __NOTE:__ We recommend using email addresses for IAM user accounts. ## Examples ```hcl module "jill" { source = "cloudposse/iam-user/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "jill" user_name = "jill@companyname.com" pgp_key = "keybase:jill" groups = ["engineering"] } ``` ## Variables ### Required Variables
`pgp_key` (`string`) required
Provide a base-64 encoded PGP public key, or a keybase username in the form `keybase:username`. Required to encrypt password.
`user_name` (`string`) required
Desired name for the IAM user. We recommend using email addresses.
### Optional Variables
`force_destroy` (`bool`) optional
When destroying this user, destroy even if it has non-Terraform-managed IAM access keys, login profile or MFA devices. Without force_destroy a user with non-Terraform-managed access keys and login profile will fail to be destroyed. **Default value:** `false`
`groups` (`list(string)`) optional
List of IAM user groups this user should belong to in the account **Default value:** `[ ]`
`login_profile_enabled` (`bool`) optional
Whether to create IAM user login profile **Default value:** `true`
`password_length` (`number`) optional
The length of the generated password **Default value:** `24`
`password_reset_required` (`bool`) optional
Whether the user should be forced to reset the generated password on first login. **Default value:** `true`
`path` (`string`) optional
Desired path for the IAM user **Default value:** `"/"`
`permissions_boundary` (`string`) optional
The ARN of the policy that is used to set the permissions boundary for the user **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`keybase_password_decrypt_command`
Command to decrypt the Keybase encrypted password. Returns empty string if pgp_key is not from keybase
`keybase_password_pgp_message`
PGP encrypted message (e.g. suitable for email exchanges). Returns empty string if pgp_key is not from keybase
`pgp_key`
PGP key used to encrypt sensitive data for this user
`user_arn`
The ARN assigned by AWS for this user
`user_login_profile_encrypted_password`
The encrypted password, base64 encoded
`user_login_profile_key_fingerprint`
The fingerprint of the PGP key used to encrypt the password
`user_name`
IAM user name
`user_unique_id`
The unique ID assigned by AWS
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_user.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user) (resource) - [`aws_iam_user_group_membership.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_group_membership) (resource) - [`aws_iam_user_login_profile.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_login_profile) (resource) ## Data Sources The following data sources are used by this module: --- ## inspector # Module: `inspector` This module enables [AWS Inspector](https://aws.amazon.com/inspector/) in one region of one account and optionally enables [various rules packages provided by AWS](https://docs.aws.amazon.com/inspector/latest/userguide/inspector_rules-arns.html). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-inspector/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-inspector/tree/main/test). ```hcl module "inspector" { source = "cloudposse/inspector/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" create_iam_role = true enabled_rules = ["cis"] } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-inspector/) - complete example of using this module ## Variables ### Required Variables
`enabled_rules` (`list(string)`) required
A list of AWS Inspector rules that should run on a periodic basis. Valid values are `cve`, `cis`, `nr`, `sbp` which map to the appropriate [Inspector rule arns by region](https://docs.aws.amazon.com/inspector/latest/userguide/inspector_rules-arns.html).
### Optional Variables
`assessment_duration` (`string`) optional
The max duration of the Inspector assessment run in seconds **Default value:** `"3600"`
`assessment_event_subscription` optional
Configures sending notifications about a specified assessment template event to a designated SNS topic **Type:** ```hcl map(object({ event = string topic_arn = string })) ``` **Default value:** `{ }`
`create_iam_role` (`bool`) optional
Flag to indicate whether an IAM Role should be created to grant the proper permissions for AWS Config **Default value:** `false`
`event_rule_description` (`string`) optional
A description of the CloudWatch event rule **Default value:** `"Trigger an AWS Inspector Assessment"`
`iam_role_arn` (`string`) optional
The ARN for an IAM Role AWS Config uses to make read or write requests to the delivery channel and to describe the AWS resources associated with the account. This is only used if create_iam_role is false. If you want to use an existing IAM Role, set the value of this to the ARN of the existing topic and set create_iam_role to false. **Default value:** `null`
`schedule_expression` (`string`) optional
An AWS Schedule Expression to indicate how often the scheduled event shoud run. For more information see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html **Default value:** `"rate(7 days)"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_cloudwatch_event_rule`
The AWS Inspector event rule
`aws_cloudwatch_event_target`
The AWS Inspector event target
`aws_inspector_assessment_template`
The AWS Inspector assessment template
`inspector_assessment_target`
The AWS Inspector assessment target
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 4.27` ### Providers - `aws`, version: `>= 4.27` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_role` | 0.16.2 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.16.2) | ----------------------------------------------------------------------------------------------------------------------- Optionally create an IAM Role ----------------------------------------------------------------------------------------------------------------------- `inspector_assessment_target_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | ----------------------------------------------------------------------------------------------------------------------- Setup AWS Inspector ----------------------------------------------------------------------------------------------------------------------- `inspector_assessment_template_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `inspector_schedule_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | ----------------------------------------------------------------------------------------------------------------------- Create a scheduled event to run inspector ----------------------------------------------------------------------------------------------------------------------- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.schedule`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.target`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_inspector_assessment_target.target`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector_assessment_target) (resource) - [`aws_inspector_assessment_template.assessment`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/inspector_assessment_template) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.start_assessment_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## key-pair # Module: `key-pair` Terraform module for generating or importing an SSH public key file into AWS. ## Usage ```hcl module "ssh_key_pair" { source = "cloudposse/key-pair/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" ssh_public_key_path = "/secrets" generate_ssh_key = "true" private_key_extension = ".pem" public_key_extension = ".pub" } ``` ## Variables ### Required Variables
`ssh_public_key_path` (`string`) required
Path to SSH public key directory (e.g. `/secrets`)
### Optional Variables
`generate_ssh_key` (`bool`) optional
If set to `true`, new SSH key pair will be created and `ssh_public_key_file` will be ignored **Default value:** `false`
`private_key_extension` (`string`) optional
Private key extension **Default value:** `""`
`public_key_extension` (`string`) optional
Public key extension **Default value:** `".pub"`
`ssh_key_algorithm` (`string`) optional
SSH key algorithm **Default value:** `"RSA"`
`ssh_public_key_file` (`string`) optional
Name of existing SSH public key file (e.g. `id_rsa.pub`) **Default value:** `null`
`ssm_parameter_enabled` (`bool`) optional
Whether an SSM parameter store value is created to store the key's private key pem. **Default value:** `false`
`ssm_parameter_path_prefix` (`string`) optional
The path prefix for the created SSM parameter e.g. '/ec2/key-pairs/acme-ue1-dev-bastion'. `ssm_parameter_enabled` must be set to `true` for this to take affect. **Default value:** `"/ec2/key-pairs/"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`key_name`
Name of SSH key
`private_key`
Content of the generated private key
`private_key_filename`
Private Key Filename
`public_key`
Content of the generated public key
`public_key_filename`
Public Key Filename
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 3.0` - `local`, version: `>= 2.2.1` - `null`, version: `>= 2.1` - `tls`, version: `>= 2.0` ### Providers - `aws`, version: `>= 3.0` - `local`, version: `>= 2.2.1` - `tls`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_key_pair.generated`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/key_pair) (resource) - [`aws_key_pair.imported`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/key_pair) (resource) - [`aws_ssm_parameter.private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`local_file.public_key_openssh`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) - [`local_sensitive_file.private_key_pem`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/sensitive_file) (resource) - [`tls_private_key.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) ## Data Sources The following data sources are used by this module: --- ## kinesis-stream(Kinesis-stream) # Module: `kinesis-stream` Terraform module to deploy an [Amazon Kinesis Data Stream](https://aws.amazon.com/kinesis/data-streams/) on AWS. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-kinesis-stream/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-kinesis-stream/tree/main/test). ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "kinesis" { source = "cloudposse/kinesis-stream/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" shard_count = 2 context = module.label.this } ``` ## Variables ### Required Variables
### Optional Variables
`consumer_count` (`number`) optional
Number of consumers to register with Kinesis stream **Default value:** `0`
`encryption_type` (`string`) optional
The encryption type to use. Acceptable values are `NONE` and `KMS`. **Default value:** `"KMS"`
`enforce_consumer_deletion` (`bool`) optional
A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. **Default value:** `true`
`kms_key_id` (`string`) optional
The GUID for the customer-managed KMS key to use for encryption. **Default value:** `"alias/aws/kinesis"`
`retention_period` (`number`) optional
Length of time data records are accessible after they are added to the stream. The maximum value of a stream's retention period is 168 hours. Minimum value is 24. Default is 24. **Default value:** `24`
`shard_count` (`number`) optional
The number of shards that the stream will use **Default value:** `1`
`shard_level_metrics` (`list(string)`) optional
A list of shard-level CloudWatch metrics which can be enabled for the stream. **Default value:** ```hcl [ "IncomingBytes", "OutgoingBytes" ] ```
`stream_mode` (`string`) optional
Specifies the capacity mode of the stream. Must be either `PROVISIONED` or `ON_DEMAND`. If `ON_DEMAND` is used, then `shard_count` is ignored. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`consumers`
List of consumers registered with Kinesis stream.
`name`
Name of the Kinesis stream.
`shard_count`
Number of shards provisioned.
`stream_arn`
ARN of the Kinesis stream.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_kinesis_stream.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream) (resource) - [`aws_kinesis_stream_consumer.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream_consumer) (resource) ## Data Sources The following data sources are used by this module: --- ## kms-key # Module: `kms-key` Terraform module to provision a [KMS](https://aws.amazon.com/kms/) key with alias. Can be used with [chamber](https://github.com/segmentio/chamber) for managing secrets by storing them in Amazon EC2 Systems Manager Parameter Store. * https://aws.amazon.com/systems-manager/features * https://aws.amazon.com/blogs/mt/the-right-way-to-store-secrets-using-parameter-store ## Usage ```hcl module "kms_key" { source = "cloudposse/kms-key/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "chamber" description = "KMS key for chamber" deletion_window_in_days = 10 enable_key_rotation = true alias = "alias/parameter_store_key" } ``` ## Variables ### Required Variables
### Optional Variables
`alias` (`string`) optional
The display name of the alias. The name must start with the word `alias` followed by a forward slash. If not specified, the alias name will be auto-generated. **Default value:** `""`
`customer_master_key_spec` (`string`) optional
Specifies whether the key contains a symmetric key or an asymmetric key pair and the encryption algorithms or signing algorithms that the key supports. Valid values: `SYMMETRIC_DEFAULT`, `RSA_2048`, `RSA_3072`, `RSA_4096`, `ECC_NIST_P256`, `ECC_NIST_P384`, `ECC_NIST_P521`, or `ECC_SECG_P256K1`. **Default value:** `"SYMMETRIC_DEFAULT"`
`deletion_window_in_days` (`number`) optional
Duration in days after which the key is deleted after destruction of the resource **Default value:** `10`
`description` (`string`) optional
The description of the key as viewed in AWS console **Default value:** `"Parameter Store KMS master key"`
`enable_key_rotation` (`bool`) optional
Specifies whether key rotation is enabled **Default value:** `true`
`key_usage` (`string`) optional
Specifies the intended use of the key. Valid values: `ENCRYPT_DECRYPT` or `SIGN_VERIFY`. **Default value:** `"ENCRYPT_DECRYPT"`
`multi_region` (`bool`) optional
Indicates whether the KMS key is a multi-Region (true) or regional (false) key. **Default value:** `false`
`policy` (`string`) optional
A valid KMS policy JSON document. Note that if the policy document is not specific enough (but still valid), Terraform may view the policy as constantly changing in a terraform plan. In this case, please make sure you use the verbose/specific version of the policy. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`alias_arn`
Alias ARN
`alias_name`
Alias name
`key_arn`
Key ARN
`key_id`
Key ID
## Dependencies ### Requirements - `terraform`, version: `>= 0.13` - `aws`, version: `>= 3.64.0` ### Providers - `aws`, version: `>= 3.64.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_kms_alias.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_alias) (resource) - [`aws_kms_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) (resource) ## Data Sources The following data sources are used by this module: --- ## lakeformation(Lakeformation) # Module: `lakeformation` Terraform module to deploy an instance of [Amazon Lake Formation](https://aws.amazon.com/lake-formation/) on AWS. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-lakeformation/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-lakeformation/tree/main/test). ```hcl # So we can assign admin permissions to the current user data "aws_caller_identity" "current" {} # Use this if a service-linked role already exists, otherwise it must be created data "aws_iam_role" "lakeformation" { name = "AWSServiceRoleForLakeFormationDataAccess" } # Create a bucket to store Lake Formation data module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" acl = "private" versioning_enabled = false force_destroy = true # Be careful with this! context = module.this.context } # Create an Athena database linked to an S3 bucket resource "aws_athena_database" "default" { count = module.this.enabled ? 1 : 0 name = var.resources.database.name bucket = module.s3_bucket.bucket_id force_destroy = true } # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "lakeformation" { source = "cloudposse/lakeformation/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" s3_bucket_arn = module.s3_bucket.bucket_arn lf_tags = { left = ["test1", "test2"] right = ["test3", "test4"] } resources = { database = { name = "example_db_1" # Athena database created above tags = { left = "test1" } } } context = module.label.this } ``` ## Variables ### Required Variables
`s3_bucket_arn` (`string`) required
Amazon Resource Name (ARN) of the Lake Formation resource, an S3 path.
### Optional Variables
`admin_arn_list` (`list(string)`) optional
Set of ARNs of AWS Lake Formation principals (IAM users or roles). **Default value:** `[ ]`
`catalog_id` (`string`) optional
Identifier for the Data Catalog. If not provided, the account ID will be used. **Default value:** `null`
`database_default_permissions` (`list(any)`) optional
Up to three configuration blocks of principal permissions for default create database permissions. **Default value:** `[ ]`
`lf_tags` (`map(list(string))`) optional
A map of key-value pairs to be used as Lake Formation tags. **Default value:** `{ }`
`resources` (`map(any)`) optional
A map of Lake Formation resources to create, with related attributes. **Default value:** `{ }`
`role_arn` (`string`) optional
Role that has read/write access to the Lake Formation resource. If not provided, the Lake Formation service-linked role must exist and is used. **Default value:** `null`
`table_default_permissions` (`list(map(any))`) optional
Up to three configuration blocks of principal permissions for default create table permissions. **Default value:** `[ ]`
`trusted_resource_owners` (`list(string)`) optional
List of the resource-owning account IDs that the caller's account can use to share their user access details (user ARNs). **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lf_tags`
List of LF tags created.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_lakeformation_data_lake_settings.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_data_lake_settings) (resource) - [`aws_lakeformation_lf_tag.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_lf_tag) (resource) - [`aws_lakeformation_resource.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_resource) (resource) - [`aws_lakeformation_resource_lf_tags.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lakeformation_resource_lf_tags) (resource) ## Data Sources The following data sources are used by this module: --- ## lambda-elasticsearch-cleanup # Module: `lambda-elasticsearch-cleanup` Terraform module to provision a scheduled Lambda function which will delete old Elasticsearch indexes using [SigV4Auth](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) authentication. The lambda function can optionally send output to an SNS topic if the topic ARN is given. This module was largely inspired by [aws-lambda-es-cleanup](https://github.com/cloudreach/aws-lambda-es-cleanup) ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-lambda-elasticsearch-cleanup/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-lambda-elasticsearch-cleanup/tree/main/test). ```hcl module "elasticsearch_cleanup" { source = "https://github.com/cloudposse/terraform-aws-lambda-elasticsearch-cleanup.git?ref=master" es_endpoint = module.elasticsearch.domain_endpoint es_domain_arn = module.elasticsearch.domain_arn es_security_group_id = module.elasticsearch.security_group_id vpc_id = module.vpc.vpc_id namespace = "eg" stage = "dev" schedule = "cron(0 3 * * ? *)" } ``` Indexes are expected to be in the format `name-date` where `date` is in the format specified by `var.index_format`. By default, all indexes except for the ones added by Kibana will be deleted based on the date part of the full index name. The actual creation date of the index is not used. Index matching is done with unanchored regular expresssion, so "bar" matches index "foobarbaz". - If the full index name, including the date part, matches `skip_index_re`, then the index will be skipped (never deleted). Kibana indexes are skipped by the default `skip_index_re` of `^\.kibana*` so if you specify a value for `skip_index_re` you must include the Kibana exception in your regex if you want it excepted. (Since Kibana indexes do not have a date part, this module should not delete them, but will complain about them having malformed dates if they are not excluded.) - If the index name without the trailing `-date` part matches `index_re`, then it will be cleaned up according to the date part. Keep in mind that, fundamentally, this module expects indexes to be in the format of `name-date` so it will not work properly if the regexes end up selecting an index that does not end with `-date`. To avoid edge cases, it is wise not to include dashes in your index name or date format. ## Migration Prior to version 0.10.0, this moudle had inputs `index`, which was a comma-separated list of index names or the special name "all" to indicate all but Kibana indexes, and `index_regex`, which was a regular expression for parsing index name and date parts. There was no mechanism for specifying a list of indexes to exclude. Starting with version 0.10.0 this module drops those inputs and instead takes `index_re` and `skip_index_re`, both of which are regular expressions. (You probably want to anchor your regexes to the beginning of the index name by starting with `^`). | If you previously had | Now use | |----------------------|----------| |`index = "all"`| Default values for `index_re` and `skip_index_re`| |`index = "a,xb,c0"` | `index_re = "^(a\|xb\|c0)"` and `skip_index_re = "^$"`| |`index_regex = "(ipat)-(dpat)"`|`index_re = "ipat"` and be sure `index_format` is correct for your date format| ## Module: cloudposse/terraform-aws-lambda-elasticsearch-cleanup This module creates a scheduled Lambda function which will delete old Elasticsearch indexes using SigV4Auth authentication. The lambda function can optionally send output to an SNS topic if the topic ARN is given ## Variables ### Required Variables
`es_domain_arn` (`string`) required
The Elasticsearch domain ARN
`es_endpoint` (`string`) required
The Elasticsearch endpoint for the Lambda function to connect to
`es_security_group_id` (`string`) required
The Elasticsearch cluster security group ID
`subnet_ids` (`list(string)`) required
Subnet IDs
`vpc_id` (`string`) required
The VPC ID for the Lambda function
### Optional Variables
`artifact_git_ref` (`string`) optional
Git ref of the lambda artifact to use. Use latest version if null. **Default value:** `""`
`artifact_url` (`string`) optional
URL template for the remote artifact **Default value:** `"https://artifacts.cloudposse.com/$${module_name}/$${git_ref}/$${filename}"`
`delete_after` (`number`) optional
Number of days to preserve **Default value:** `15`
`index_format` (`string`) optional
Combined with 'index' variable and is used to evaluate the index age **Default value:** `"%Y.%m.%d"`
`index_re` (`string`) optional
Regular Expression that matches the index names to clean up (not including trailing dash and date) **Default value:** `".*"`
`python_version` (`string`) optional
The Python version to use **Default value:** `"3.12"`
`schedule` (`string`) optional
CloudWatch Events rule schedule using cron or rate expression **Default value:** `"cron(0 3 * * ? *)"`
`skip_index_re` (`string`) optional
Regular Expression that matches the index names to ignore (not clean up). Takes precedence over `index_re`. BY DEFAULT (when value is `null`), a pattern is used to exclude Kibana indexes. Use `"^$"` if you do not want to skip any indexes. Include an exclusion for `kibana` if you want to use a custom value and also exclude the kibana indexes. **Default value:** `null`
`sns_arn` (`string`) optional
SNS ARN to publish alerts **Default value:** `""`
`timeout` (`number`) optional
Timeout for Lambda function in seconds **Default value:** `300`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lambda_function_arn`
ARN of the Lambda Function
`lambda_function_source_code_size`
The size in bytes of the function .zip file
`security_group_id`
Security Group ID of the Lambda Function
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0` - `null`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | Modules -------------------------------------------------------------- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_function.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress_from_lambda_to_es_cluster`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_to_es_cluster_from_lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.tcp_dns_egress_from_lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.udp_dns_egress_from_lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.es_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.sns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## lambda-function # Module: `lambda-function` This module deploys an AWS Lambda function from a Zip file or from a Docker image. Additionally, it creates an IAM role for the Lambda function, which optionally attaches policies to allow for CloudWatch Logs, Cloudwatch Insights, VPC Access and X-Ray tracing. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-lambda-function/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-lambda-function/tree/main/test). ```hcl module "lambda" { source = "cloudposse/lambda-function/aws" version = "xxxx" filename = "lambda.zip" function_name = "my-function" handler = "handler.handler" runtime = "nodejs14.x" } ``` ## Examples - [`examples/complete`](https://github.com/cloudposse/terraform-aws-lambda-function/blob/main/examples/complete) - complete example of using this module - [`examples/docker-image`](https://github.com/cloudposse/terraform-aws-lambda-function/blob/main/examples/docker-image) - example of using Lambda with Docker images ## Variables ### Required Variables
`function_name` (`string`) required
Unique name for the Lambda Function.
### Optional Variables
`architectures` (`list(string)`) optional
Instruction set architecture for your Lambda function. Valid values are ["x86_64"] and ["arm64"]. Default is ["x86_64"]. Removing this attribute, function's architecture stay the same. **Default value:** `null`
`cloudwatch_lambda_insights_enabled` (`bool`) optional
Enable CloudWatch Lambda Insights for the Lambda Function. **Default value:** `false`
`cloudwatch_logs_kms_key_arn` (`string`) optional
The ARN of the KMS Key to use when encrypting log data. **Default value:** `null`
`cloudwatch_logs_retention_in_days` (`number`) optional
Specifies the number of days you want to retain log events in the specified log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653, and 0. If you select 0, the events in the log group are always retained and never expire. **Default value:** `null`
`custom_iam_policy_arns` (`set(string)`) optional
ARNs of custom policies to be attached to the lambda role **Default value:** `[ ]`
`dead_letter_config_target_arn` (`string`) optional
ARN of an SNS topic or SQS queue to notify when an invocation fails. If this option is used, the function's IAM role must be granted suitable access to write to the target object, which means allowing either the sns:Publish or sqs:SendMessage action on this ARN, depending on which service is targeted." **Default value:** `null`
`description` (`string`) optional
Description of what the Lambda Function does. **Default value:** `null`
`ephemeral_storage_size` (`number`) optional
The size of the Lambda function Ephemeral storage (/tmp) represented in MB. The minimum supported ephemeral_storage value defaults to 512MB and the maximum supported value is 10240MB. **Default value:** `null`
`filename` (`string`) optional
The path to the function's deployment package within the local filesystem. If defined, The s3_-prefixed options and image_uri cannot be used. **Default value:** `null`
`handler` (`string`) optional
The function entrypoint in your code. **Default value:** `null`
`iam_policy_description` (`string`) optional
Description of the IAM policy for the Lambda IAM role **Default value:** `"Provides minimum SSM read permissions."`
`image_config` (`any`) optional
The Lambda OCI [image configurations](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function#image_config) block with three (optional) arguments: - *entry_point* - The ENTRYPOINT for the docker image (type `list(string)`). - *command* - The CMD for the docker image (type `list(string)`). - *working_directory* - The working directory for the docker image (type `string`). **Default value:** `{ }`
`image_uri` (`string`) optional
The ECR image URI containing the function's deployment package. Conflicts with filename, s3_bucket, s3_key, and s3_object_version. **Default value:** `null`
`inline_iam_policy` (`string`) optional
Inline policy document (JSON) to attach to the lambda role **Default value:** `null`
`invoke_function_permissions` optional
Defines which external source(s) can invoke this function (action 'lambda:InvokeFunction'). Attributes map to those of https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission. NOTE: to keep things simple, we only expose a subset of said attributes. If a more complex configuration is needed, declare the necessary lambda permissions outside of this module **Type:** ```hcl list(object({ principal = string source_arn = string })) ``` **Default value:** `[ ]`
`kms_key_arn` (`string`) optional
Amazon Resource Name (ARN) of the AWS Key Management Service (KMS) key that is used to encrypt environment variables. If this configuration is not provided when environment variables are in use, AWS Lambda uses a default service key. If this configuration is provided when environment variables are not in use, the AWS Lambda API does not save this configuration and Terraform will show a perpetual difference of adding the key. To fix the perpetual difference, remove this configuration. **Default value:** `""`
`lambda_at_edge_enabled` (`bool`) optional
Enable Lambda@Edge for your Node.js or Python functions. The required trust relationship and publishing of function versions will be configured in this module. **Default value:** `false`
`lambda_environment` optional
Environment (e.g. env variables) configuration for the Lambda function enable you to dynamically pass settings to your function code and libraries. **Type:** ```hcl object({ variables = map(string) }) ``` **Default value:** `null`
`layers` (`list(string)`) optional
List of Lambda Layer Version ARNs (maximum of 5) to attach to the Lambda Function. **Default value:** `[ ]`
`memory_size` (`number`) optional
Amount of memory in MB the Lambda Function can use at runtime. **Default value:** `128`
`package_type` (`string`) optional
The Lambda deployment package type. Valid values are Zip and Image. **Default value:** `"Zip"`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the role **Default value:** `""`
`publish` (`bool`) optional
Whether to publish creation/change as new Lambda Function Version. **Default value:** `false`
`reserved_concurrent_executions` (`number`) optional
The amount of reserved concurrent executions for this lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations. **Default value:** `-1`
`role_name` (`string`) optional
The rolename used for the Lambda Function. If not provided, a default role name will be used. **Default value:** `null`
`runtime` (`string`) optional
The runtime environment for the Lambda function you are uploading. **Default value:** `null`
`s3_bucket` (`string`) optional
The S3 bucket location containing the function's deployment package. Conflicts with filename and image_uri. This bucket must reside in the same AWS region where you are creating the Lambda function. **Default value:** `null`
`s3_key` (`string`) optional
The S3 key of an object containing the function's deployment package. Conflicts with filename and image_uri. **Default value:** `null`
`s3_object_version` (`string`) optional
The object version containing the function's deployment package. Conflicts with filename and image_uri. **Default value:** `null`
`source_code_hash` (`string`) optional
Used to trigger updates. Must be set to a base64-encoded SHA256 hash of the package file specified with either filename or s3_key. The usual way to set this is filebase64sha256('file.zip') where 'file.zip' is the local filename of the lambda function source archive. **Default value:** `""`
`ssm_parameter_names` (`list(string)`) optional
List of AWS Systems Manager Parameter Store parameter names. The IAM role of this Lambda function will be enhanced with read permissions for those parameters. Parameters must start with a forward slash and can be encrypted with the default KMS key. **Default value:** `null`
`timeout` (`number`) optional
The amount of time the Lambda Function has to run in seconds. **Default value:** `3`
`tracing_config_mode` (`string`) optional
Tracing config mode of the Lambda function. Can be either PassThrough or Active. **Default value:** `null`
`vpc_config` optional
Provide this to allow your function to access your VPC (if both 'subnet_ids' and 'security_group_ids' are empty then vpc_config is considered to be empty or unset, see https://docs.aws.amazon.com/lambda/latest/dg/vpc.html for details). **Type:** ```hcl object({ security_group_ids = list(string) subnet_ids = list(string) }) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the lambda function
`cloudwatch_log_group_name`
Name of Cloudwatch log group
`function_name`
Lambda function name
`invoke_arn`
Invoke ARN of the lambda function
`qualified_arn`
ARN identifying your Lambda Function Version (if versioning is enabled via publish = true)
`role_arn`
Lambda IAM role ARN
`role_name`
Lambda IAM role name
## Dependencies ### Requirements - `terraform`, version: `>= 0.14` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `cloudwatch_log_group` | 0.6.6 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.6) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.ssm`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy.inline`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy) (resource) - [`aws_iam_role_policy_attachment.cloudwatch_insights`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.cloudwatch_logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.custom`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.ssm`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.vpc_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.xray`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_function.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.invoke_function`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume_role_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.ssm`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## lb-s3-bucket # Module: `lb-s3-bucket` Terraform module to provision an S3 bucket with built in IAM policy to allow [AWS Load Balancers](https://aws.amazon.com/documentation/elastic-load-balancing/) to ship [access logs](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-lb-s3-bucket/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-lb-s3-bucket/tree/main/test). ```hcl module "s3_bucket" { source = "cloudposse/lb-s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "bucket" } ``` ## Variables ### Required Variables
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where S3 access logs will be sent to **Default value:** `""`
`access_log_bucket_prefix` (`string`) optional
Prefix to prepend to the current S3 bucket name, where S3 access logs will be sent to **Default value:** `null`
`acl` (`string`) optional
Canned ACL to apply to the S3 bucket **Default value:** `"log-delivery-write"`
`allow_ssl_requests_only` (`bool`) optional
Require requests to use Secure Socket Layer (HTTPS/SSL). **Default value:** `true`
`bucket_name` (`string`) optional
Bucket name. If provided, the bucket will be created with this name instead of generating the name from the context **Default value:** `null`
`enable_glacier_transition` (`bool`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files **Default value:** `true`
`expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to expunge the objects **Default value:** `90`
`force_destroy` (`bool`) optional
When `true`, permits a non-empty S3 bucket to be deleted by first deleting all objects in the bucket. THESE OBJECTS ARE NOT RECOVERABLE even if they were versioned and stored in Glacier. Must be set `false` unless `force_destroy_enabled` is also `true`. **Default value:** `false`
`glacier_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to move the data to the Glacier Flexible Retrieval storage tier **Default value:** `60`
`lifecycle_configuration_rules` optional
A list of S3 bucket v2 lifecycle rules, as specified in [terraform-aws-s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket)" These rules are not affected by the deprecated `lifecycle_rule_enabled` flag. **NOTE:** Unless you also set `lifecycle_rule_enabled = false` you will also get the default deprecated rules set on your bucket. **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = any expiration = any transition = list(any) noncurrent_version_expiration = any noncurrent_version_transition = list(any) })) ``` **Default value:** `[ ]`
`lifecycle_prefix` (`string`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Prefix filter. Used to manage object lifecycle events **Default value:** `""`
`lifecycle_rule_enabled` (`bool`) optional
DEPRECATED: Defaults to `false`, use `lifecycle_configuration_rules` instead. When `true`, configures lifecycle events on this bucket using individual (now deprecated) variables." **Default value:** `false`
`noncurrent_version_expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies when non-current object versions expire (in days) **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies (in days) when noncurrent object versions transition to Glacier Flexible Retrieval **Default value:** `30`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. **Default value:** `"BucketOwnerPreferred"`
`standard_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`versioning_enabled` (`bool`) optional
Enable object versioning, keeping multiple variants of an object in the same bucket **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
S3 bucket ARN
`bucket_domain_name`
S3 bucket domain name
`bucket_id`
S3 bucket ID
`bucket_prefix`
S3 bucket prefix
`enabled`
Is module enabled
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `bucket_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `s3_bucket` | 1.4.3 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.3) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_elb_service_account.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/elb_service_account) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## macie(Macie) # Module: `macie` Terraform module to provision [Amazon Macie](https://aws.amazon.com/macie/) - a fully managed data security and data privacy service that uses machine learning and pattern matching to discover and protect your sensitive data in AWS ## Usage ```hcl provider "aws" { assume_role { role_arn = "arn:aws:iam::11111111111:role/my-org-root" } } provider "aws" { alias = "macie_admin" # delegated admin assume_role { role_arn = "arn:aws:iam::22222222222:role/my-org-security" } } module "macie" { source = "cloudposse/macie/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" providers = { aws.admin = aws aws = aws.macie_admin } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-macie/examples/complete) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`account_status` (`bool`) optional
Macie account status. Possible values are `ENABLED` and `PAUSED`. Setting it to `ENABLED` will start all Macie activities for the account. **Default value:** `true`
`admin_account_ids` (`list(string)`) optional
The list of AWS account IDs for the account to designate as the delegated Amazon Macie administrator accounts for the organization. **Default value:** `[ ]`
`classification_jobs` (`list(any)`) optional
A list of maps of classification jobs. name: A custom name for the job. description: A custom description of the job. tags: A map of key-value pairs that specifies the tags to associate with the job. sampling_percentage: The sampling depth, as a percentage, to apply when processing objects. This value determines the percentage of eligible objects that the job analyzes. initial_run: Whether to analyze all existing, eligible objects immediately after the job is created. job_type: The schedule for running the job. If you specify `SCHEDULED` value, use the `schedule_frequency` property to define the recurrence pattern for the job. Possible values: `ONE_TIME`, `SCHEDULED`. job_status: The status for the job. Possible values: `CANCELLED`, `RUNNING` and `USER_PAUSED`. schedule_frequency: daily_schedule: Specifies a daily recurrence pattern for running the job. weekly_schedule: Specifies a weekly recurrence pattern for running the job. monthly_schedule: Specifies a monthly recurrence pattern for running the job. **Default value:** `[ ]`
`custom_data_identifiers` (`list(any)`) optional
A list of maps of custom data identifiers. A custom data identifier is a set of criteria that you defined to detect sensitive data in one or more data sources. regex: The regular expression (regex) that defines the pattern to match. The expression can contain as many as 512 characters. keywords: An array that lists specific character sequences (keywords), one of which must be within proximity (`maximum_match_distance`) of the regular expression to match. The array can contain as many as 50 keywords. Each keyword can contain 3 - 90 characters. Keywords aren't case sensitive. ignore_words: An array that lists specific character sequences (ignore words) to exclude from the results. If the text matched by the regular expression is the same as any string in this array, Amazon Macie ignores it. The array can contain as many as 10 ignore words. Each ignore word can contain 4 - 90 characters. maximum_match_distance: The maximum number of characters that can exist between text that matches the regex pattern and the character sequences specified by the keywords array. Macie includes or excludes a result based on the proximity of a keyword to text that matches the regex pattern. The distance can be 1 - 300 characters. The default value is 50. name: A custom name for the custom data identifier. description: A custom description of the custom data identifier. tags: A map of key-value pairs that specifies the tags to associate with the custom data identifier. **Default value:** `[ ]`
`finding_publishing_frequency` (`string`) optional
Specifies how often to publish updates to policy findings for the account. This includes publishing updates to AWS Security Hub and Amazon EventBridge (formerly called Amazon CloudWatch Events). Valid values are FIFTEEN_MINUTES, ONE_HOUR or SIX_HOURS. **Default value:** `"ONE_HOUR"`
`members` (`list(any)`) optional
A list of maps of Amazon Macie Members. account_id: The AWS account ID for the account. email: The email address for the account. tags: A map of key-value pairs that specifies the tags to associate with the account in Amazon Macie. status: Specifies the status for the account. Possible values: `ENABLED`, `PAUSED`. invite: Whether to send an invitation to a member. invitation_message: A custom message to include in the invitation. Amazon Macie adds this message to the standard content that it sends for an invitation. invitation_disable_email_notification: Whether to send an email notification to the root user of each account that the invitation will be sent to. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`account_id`
The ID of the Macie account.
`account_service_role_arn`
The service role ARN of the Macie account.
`aws_account_to_org_admin_account_ids`
Map of the AWS account IDs to Macie organization admin account IDs
`member_accounts`
List of AWS account IDs the Macie Admin is managing
`org_admin_account_ids`
List of IDs of the Macie organization admin accounts.
## Dependencies ### Requirements - `terraform`, version: `>= 0.15.0` - `aws`, version: `>= 3.38` ### Providers - `aws`, version: `>= 3.38` - `aws`, version: `>= 3.38` ### Modules Name | Version | Source | Description --- | --- | --- | --- `classification_job_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `custom_data_identifier_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `member_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_macie2_account.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_account) (resource) - [`aws_macie2_classification_job.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_classification_job) (resource) - [`aws_macie2_custom_data_identifier.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_custom_data_identifier) (resource) - [`aws_macie2_member.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_member) (resource) - [`aws_macie2_organization_admin_account.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/macie2_organization_admin_account) (resource) ## Data Sources The following data sources are used by this module: --- ## managed-grafana # Module: `managed-grafana` This module is responsible for provisioning an Amazon Managed Grafana workspace. ## Introduction Amazon Managed Grafana is a fully managed service for Grafana, a popular open-source analytics platform that enables you to query, visualize, and alert on your metrics, logs, and traces. Deploy this module alongside [terraform-aws-managed-prometheus](https://github.com/cloudposse/terraform-aws-managed-prometheus) to visual metrics or add a Grafana Loki data source to visualize logs. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-managed-grafana/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-managed-grafana/tree/main/test). ```hcl locals { enabled = module.this.enabled # These are pulled from the output of the cloudposse/terraform-aws-managed-prometheus module additional_allowed_roles = compact([for prometheus in module.prometheus : prometheus.outputs.access_role_arn]) } module "security_group" { source = "cloudposse/security-group/aws" version = "2.2.0" enabled = local.enabled && var.private_network_access_enabled allow_all_egress = true rules = [] vpc_id = module.vpc.outputs.vpc_id context = module.this.context } module "managed_grafana" { source = "cloudposse/managed-grafana/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" enabled = local.enabled prometheus_policy_enabled = var.prometheus_policy_enabled additional_allowed_roles = local.additional_allowed_roles sso_role_associations = [ { "role" = "ADMIN" "group_ids" = ["xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"] } ] vpc_configuration = var.private_network_access_enabled ? { subnet_ids = module.vpc.outputs.private_subnet_ids security_group_ids = [module.security_group.id] } : {} context = module.this.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-managed-grafana/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`account_access_type` (`string`) optional
The type of account access for the workspace. Valid values are CURRENT_ACCOUNT and ORGANIZATION. If ORGANIZATION is specified, then organizational_units must also be present. **Default value:** `"CURRENT_ACCOUNT"`
`additional_allowed_roles` (`list(string)`) optional
A list of IAM Role ARNs that this Grafana IAM role will be allowed to assume. Use this to set up cross-account access. **Default value:** `[ ]`
`authentication_providers` (`list(string)`) optional
The authentication providers for the workspace. Valid values are `AWS_SSO`, `SAML`, or both. **Default value:** ```hcl [ "AWS_SSO" ] ```
`data_sources` (`list(string)`) optional
The data sources for the workspace. Valid values are AMAZON_OPENSEARCH_SERVICE, ATHENA, CLOUDWATCH, PROMETHEUS, REDSHIFT, SITEWISE, TIMESTREAM, XRAY **Default value:** `[ ]`
`organizational_units` (`list(string)`) optional
A list of organizational units (OU) IDs to grant access to the workspace. Only used if `var.account_access_type` is `ORGANIZATION` **Default value:** `[ ]`
`permission_type` (`string`) optional
The permission type of the workspace. If `SERVICE_MANAGED` is specified, the IAM roles and IAM policy attachments are generated automatically. If `CUSTOMER_MANAGED` is specified, the IAM roles and IAM policy attachments will not be created. **Default value:** `"SERVICE_MANAGED"`
`prometheus_policy_enabled` (`bool`) optional
Set this to `true` to allow this Grafana workspace to access Amazon Managed Prometheus (AMP). **Default value:** `false`
`sso_role_associations` optional
A list of role to group ID and user ID list associations for granting Amazon Grafana access. Only used if `var.authentication_providers` includes `AWS_SSO` **Type:** ```hcl list(object({ role = string group_ids = optional(list(string)) user_ids = optional(list(string)) })) ``` **Default value:** `[ ]`
`vpc_configuration` optional
If defined, this map defines the VPC configuration to connect your Grafana workspace to a private network **Type:** ```hcl object({ security_group_ids = list(string) subnet_ids = list(string) }) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`workspace_arn`
The ARN of the Amazon Managed Grafana workspace
`workspace_endpoint`
The URL of the Amazon Managed Grafana workspace
`workspace_id`
The ID of the Amazon Managed Grafana workspace
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `additional_allowed_role_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `prometheus_policy_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_grafana_role_association.sso`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_role_association) (resource) - [`aws_grafana_workspace.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/grafana_workspace) (resource) - [`aws_iam_role.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.aps`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## managed-prometheus(Managed-prometheus) # Module: `managed-prometheus` This module is responsible for provisioning a workspace for Amazon Managed Service for Prometheus, also known as Amazon Managed Prometheus (AMP). ## Introduction Amazon Managed Service for Prometheus provides highly available, secure, and managed monitoring for your containers. It automatically scales as your ingestion and query needs grow, and gives you access to remote write metrics from existing Prometheus servers and to query metrics using PromQL. Deploy this module alongside a managed collector to read metrics from EKS and visualize them with Grafana. For example, see [terraform-aws-managed-grafana](https://github.com/cloudposse/terraform-aws-managed-grafana). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-managed-prometheus/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-managed-prometheus/tree/main/test). ```hcl locals { grafana_account_id = "123456789012" } # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "prometheus" { source = "cloudposse/managed-prometheus/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" allowed_account_id = local.grafana_account_id # The ID of another account allowed to access this Managed Prometheus Workspace scraper_deployed = true vpc_id = var.vpc_id # The ID of some VPC allowed to access this Managed Prometheus Workspace context = module.label.this } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-managed-prometheus/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`alert_manager_definition` (`string`) optional
The alert manager definition that you want to be applied. **Default value:** `""`
`allowed_account_id` (`string`) optional
The Account ID of another AWS account allowed to access Amazon Managed Prometheus in this account. If defined, this module will create a cross-account IAM role for accessing APS. Use this for cross-account Grafana. If not defined, no roles will be created. **Default value:** `""`
`rule_group_namespaces` optional
A list of name, data objects for each Amazon Managed Service for Prometheus (AMP) Rule Group Namespace **Type:** ```hcl list(object({ name = string data = string })) ``` **Default value:** `[ ]`
`scraper_deployed` (`bool`) optional
When connecting an Amazon Managed Collector, also known as a scraper, Amazon will add a tag to this AMP workspace. Set this value to `true` to resolve Terraform drift. **Default value:** `false`
`vpc_id` (`string`) optional
If set, the ID of the VPC in which the endpoint will be used. If not set, no VPC endpoint will be created. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_role_arn`
If enabled with `var.allowed_account_id`, the Role ARN used for accessing Amazon Managed Prometheus in this account
`workspace_arn`
The ARN of this Amazon Managed Prometheus workspace
`workspace_endpoint`
The endpoint URL of this Amazon Managed Prometheus workspace
`workspace_id`
The ID of this Amazon Managed Prometheus workspace
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `access_role` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `account_access_policy_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `vpc_endpoint_policy` | 2.0.1 | [`cloudposse/iam-policy/aws`](https://registry.terraform.io/modules/cloudposse/iam-policy/aws/2.0.1) | n/a ## Resources The following resources are used by this module: - [`aws_iam_role.account_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_prometheus_alert_manager_definition.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_alert_manager_definition) (resource) - [`aws_prometheus_rule_group_namespace.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_rule_group_namespace) (resource) - [`aws_prometheus_workspace.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/prometheus_workspace) (resource) - [`aws_vpc_endpoint.prometheus`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.aps`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## memorydb(Memorydb) # Module: `memorydb` This module allows an engineer to provision MemoryDB clusters along with an admin user, subnet group, and parameter group. MemoryDB is a real-time in-memory database with API compatibility for Redis. ## Introduction AWS MemoryDB is a fully managed, Redis-compatible, in-memory database service that delivers ultra-fast performance and Multi-AZ durability for modern applications built using microservices architectures. MemoryDB also boasts the fastest vector search recall rates in the industry. With single-digit millisecond latency, MemoryDB can work with vector data at real-time speeds for AI models that demand performance. ## Usage For a complete example, see [the examples/complete directory](https://github.com/cloudposse/terraform-aws-memorydb/tree/main/examples/complete). For automated tests of the complete example using [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [the test directory](https://github.com/cloudposse/terraform-aws-memorydb/tree/main/test). ```hcl provider "aws" { region = "us-west-2" } # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" environment = "usw2" stage = "sandbox" name = "memorydb" } module "example" { source = "cloudposse/memorydb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ssm_parameter_name = "${module.label.id}/admin_password" context = module.label.context } ``` ## Quick Start This module relies on two other AWS services: - AWS VPC (Virtual Private Cloud) - AWS SSM (Systems Manager) Make sure you either have a default VPC already made or have subnet IDs for any other VPC. At a minimum specify these variables: - `ssm_parameter_name` - the name of the SSM parameter that stores the admin password for the MemoryDB cluster - `subnet_ids` - required if not using a default VPC, this is a list of subnet IDs where the MemoryDB cluster will be deployed - `security_group_ids` - required if not using default security groups, this is a list of security group IDs to attach to the MemoryDB cluster See our [VPC module](https://github.com/cloudposse/terraform-aws-vpc) for creating a VPC. See our [Security Group module](https://github.com/cloudposse/terraform-aws-security-group) for creating security groups. ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-memorydb/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`additional_users` (`list(string)`) optional
List of additional users to create for the MemoryDB cluster **Default value:** `[ ]`
`admin_password` (`string`) optional
The password for the MemoryDB user. If empty, a random password will be generated. **Default value:** `""`
`admin_username` (`string`) optional
The username for the MemoryDB admin **Default value:** `"admin"`
`auto_minor_version_upgrade` (`bool`) optional
Indicates that minor engine upgrades will be applied automatically to the cluster during the maintenance window **Default value:** `true`
`create_admin_user` (`bool`) optional
Indicates whether to create an admin user for the MemoryDB cluster **Default value:** `true`
`engine_version` (`string`) optional
The version of the Redis engine to use **Default value:** `"6.2"`
`maintenance_window` (`string`) optional
The weekly time range during which system maintenance can occur **Default value:** `null`
`node_type` (`string`) optional
Node type for the MemoryDB cluster **Default value:** `"db.r6g.large"`
`num_replicas_per_shard` (`number`) optional
The number of replicas per shard **Default value:** `1`
`num_shards` (`number`) optional
The number of shards in the cluster **Default value:** `1`
`parameter_group_family` (`string`) optional
The name of the parameter group family **Default value:** `"memorydb_redis6"`
`parameter_group_name` (`string`) optional
The name of the parameter group to associate with this cluster. **Default value:** `null`
`parameters` (`map(string)`) optional
Key-value mapping of parameters to apply to the parameter group **Default value:** `{ }`
`port` (`number`) optional
The port on which the cluster accepts connections **Default value:** `6379`
`security_group_ids` (`list(string)`) optional
List of security group IDs to associate with the MemoryDB cluster **Default value:** `[ ]`
`snapshot_arns` (`list(string)`) optional
List of ARNs for the snapshots to be restored. NOTE: destroys the existing cluster. Use for restoring. **Default value:** `[ ]`
`snapshot_retention_limit` (`number`) optional
The number of days for which MemoryDB retains automatic snapshots before deleting them **Default value:** `null`
`snapshot_window` (`string`) optional
The daily time range during which MemoryDB begins taking daily snapshots **Default value:** `null`
`sns_topic_arn` (`string`) optional
The ARN of the SNS topic to send notifications to **Default value:** `null`
`ssm_kms_key_id` (`string`) optional
The KMS key ID to use for SSM parameter encryption. If not specified, the default key will be used. **Default value:** `null`
`ssm_parameter_name` (`string`) optional
The name of the SSM parameter to store the password in. If not specified, the password will not be stored. **Default value:** `""`
`subnet_ids` (`list(string)`) optional
List of subnet IDs for the MemoryDB cluster. Leave empty to use the default VPC subnets. **Default value:** `[ ]`
`tls_enabled` (`bool`) optional
Indicates whether Transport Layer Security (TLS) encryption is enabled for the cluster **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_acl_arn`
The ARN of the MemoryDB user's ACL
`admin_arn`
The ARN of the MemoryDB user
`admin_password_ssm_parameter_name`
The name of the SSM parameter storing the password for the MemoryDB user
`admin_username`
The username for the MemoryDB user
`arn`
The ARN of the MemoryDB cluster
`cluster_endpoint`
The endpoint of the MemoryDB cluster
`engine_patch_version`
The Redis engine version
`id`
The name of the MemoryDB cluster
`parameter_group_arn`
The ARN of the MemoryDB parameter group
`parameter_group_id`
The name of the MemoryDB parameter group
`shards`
The shard details for the MemoryDB cluster
`subnet_group_arn`
The ARN of the MemoryDB subnet group
`subnet_group_id`
The name of the MemoryDB subnet group
## Dependencies ### Requirements - `terraform`, version: `>= 1.1` - `aws`, version: `>= 5.0` - `random`, version: `>= 2.2` ### Providers - `aws`, version: `>= 5.0` - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_memorydb_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/memorydb_acl) (resource) - [`aws_memorydb_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/memorydb_cluster) (resource) - [`aws_memorydb_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/memorydb_parameter_group) (resource) - [`aws_memorydb_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/memorydb_subnet_group) (resource) - [`aws_memorydb_user.admin`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/memorydb_user) (resource) - [`aws_ssm_parameter.admin_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) ## Data Sources The following data sources are used by this module: --- ## mq-broker(Mq-broker) # Module: `mq-broker` Terraform module to provision AmazonMQ resources on AWS ## Introduction This module provisions the following resources: - ActiveMQ broker - RabbitMQ broker - Security group rules to allow access to the broker Admin and application users are created and credentials written to SSM if not passed in as variables. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-mq-broker/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-mq-broker/tree/main/test). ```hcl module "mq_broker" { source = "cloudposse/mq-broker/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "mq-broker" apply_immediately = true auto_minor_version_upgrade = true deployment_mode = "ACTIVE_STANDBY_MULTI_AZ" engine_type = "ActiveMQ" engine_version = "5.15.14" host_instance_type = "mq.t3.micro" publicly_accessible = false general_log_enabled = true audit_log_enabled = true encryption_enabled = true use_aws_owned_key = true vpc_id = var.vpc_id subnet_ids = var.subnet_ids security_groups = var.security_groups } ``` ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
List of VPC subnet IDs
`vpc_id` (`string`) required
The ID of the VPC to create the broker in
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group(s) via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. For more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule and https://github.com/cloudposse/terraform-aws-security-group. **Default value:** `[ ]`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP addresses. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ingress_ports` (`list(number)`) optional
List of TCP ports to allow access to in the created security group. Default is to allow access to all ports. Set `create_security_group` to `false` to disable. Note: List of ports must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_cidr_blocks` (`list(string)`) optional
A list of IPv6 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_ipv6_prefix_list_ids` (`list(string)`) optional
A list of IPv6 Prefix Lists IDs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_groups` (`list(string)`) optional
DEPRECATED: Use `allowed_security_group_ids` instead. A list of Security Group IDs to to be allowed to connect to the broker instance. **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `false`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`audit_log_enabled` (`bool`) optional
Enables audit logging. User management action made using JMX or the ActiveMQ Web Console is logged **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Enables automatic upgrades to new minor versions for brokers, as Apache releases the versions **Default value:** `false`
`configuration_data` (`string`) optional
data value for configuration **Default value:** `null`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`deployment_mode` (`string`) optional
The deployment mode of the broker. Supported: SINGLE_INSTANCE and ACTIVE_STANDBY_MULTI_AZ, and CLUSTER_MULTI_AZ **Default value:** `"ACTIVE_STANDBY_MULTI_AZ"`
`encryption_enabled` (`bool`) optional
Flag to enable/disable Amazon MQ encryption at rest **Default value:** `true`
`engine_type` (`string`) optional
Type of broker engine, `ActiveMQ` or `RabbitMQ` **Default value:** `"ActiveMQ"`
`engine_version` (`string`) optional
The version of the broker engine. See https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/broker-engine.html for more details **Default value:** `"5.17.6"`
`existing_security_groups` (`list(string)`) optional
DEPRECATED: Use `associated_security_group_ids` instead. List of existing Security Group IDs to place the broker into. **Default value:** `[ ]`
`general_log_enabled` (`bool`) optional
Enables general logging via CloudWatch **Default value:** `true`
`host_instance_type` (`string`) optional
The broker's instance type. e.g. mq.t2.micro or mq.m4.large **Default value:** `"mq.t3.micro"`
`kms_mq_key_arn` (`string`) optional
ARN of the AWS KMS key used for Amazon MQ encryption **Default value:** `null`
`kms_ssm_key_arn` (`string`) optional
ARN of the AWS KMS key used for SSM encryption **Default value:** `"alias/aws/ssm"`
`maintenance_day_of_week` (`string`) optional
The maintenance day of the week. e.g. MONDAY, TUESDAY, or WEDNESDAY **Default value:** `"SUNDAY"`
`maintenance_time_of_day` (`string`) optional
The maintenance time, in 24-hour format. e.g. 02:00 **Default value:** `"03:00"`
`maintenance_time_zone` (`string`) optional
The maintenance time zone, in either the Country/City format, or the UTC offset format. e.g. CET **Default value:** `"UTC"`
`mq_admin_password` (`list(string)`) optional
Admin password **Default value:** `[ ]`
`mq_admin_password_ssm_parameter_name` (`string`) optional
SSM parameter name for Admin password **Default value:** `"mq_admin_password"`
`mq_admin_user` (`list(string)`) optional
Admin username **Default value:** `[ ]`
`mq_admin_user_ssm_parameter_name` (`string`) optional
SSM parameter name for Admin username **Default value:** `"mq_admin_username"`
`mq_application_password` (`list(string)`) optional
Application password **Default value:** `[ ]`
`mq_application_password_ssm_parameter_name` (`string`) optional
SSM parameter name for Application password **Default value:** `"mq_application_password"`
`mq_application_user` (`list(string)`) optional
Application username **Default value:** `[ ]`
`mq_application_user_ssm_parameter_name` (`string`) optional
SSM parameter name for Application username **Default value:** `"mq_application_username"`
`preserve_security_group_id` (`bool`) optional
When `false` and `security_group_create_before_destroy` is `true`, changes to security group rules cause a new security group to be created with the new rules, and the existing security group is then replaced with the new one, eliminating any service interruption. When `true` or when changing the value (from `false` to `true` or from `true` to `false`), existing security group rules will be deleted before new ones are created, resulting in a service interruption, but preserving the security group itself. **NOTE:** Setting this to `true` does not guarantee the security group will never be replaced, it only keeps changes to the security group rules from triggering a replacement. See the [terraform-aws-security-group README](https://github.com/cloudposse/terraform-aws-security-group) for further discussion. **Default value:** `false`
`publicly_accessible` (`bool`) optional
Whether to enable connections from applications outside of the VPC that hosts the broker's subnets **Default value:** `false`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Managed by Terraform"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`ssm_parameter_name_format` (`string`) optional
SSM parameter name format **Default value:** `"/%s/%s"`
`ssm_parameters_enabled` (`bool`) optional
Whether to create SSM parameters for MQ users and passwords **Default value:** `true`
`ssm_path` (`string`) optional
The first parameter to substitute in `ssm_parameter_name_format` **Default value:** `"mq"`
`use_aws_owned_key` (`bool`) optional
Boolean to enable an AWS owned Key Management Service (KMS) Customer Master Key (CMK) for Amazon MQ encryption that is not in your account **Default value:** `true`
`use_existing_security_groups` (`bool`) optional
DEPRECATED: Use `create_security_group` instead. Historical description: Set to `true` to disable Security Group creation **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`admin_username`
AmazonMQ admin username
`application_password`
AmazonMQ application password
`application_username`
AmazonMQ application username
`broker_arn`
AmazonMQ broker ARN
`broker_id`
AmazonMQ broker ID
`primary_amqp_ssl_endpoint`
AmazonMQ primary AMQP+SSL endpoint
`primary_console_url`
AmazonMQ active web console URL
`primary_ip_address`
AmazonMQ primary IP address
`primary_mqtt_ssl_endpoint`
AmazonMQ primary MQTT+SSL endpoint
`primary_ssl_endpoint`
AmazonMQ primary SSL endpoint
`primary_stomp_ssl_endpoint`
AmazonMQ primary STOMP+SSL endpoint
`primary_wss_endpoint`
AmazonMQ primary WSS endpoint
`secondary_amqp_ssl_endpoint`
AmazonMQ secondary AMQP+SSL endpoint
`secondary_console_url`
AmazonMQ secondary web console URL
`secondary_ip_address`
AmazonMQ secondary IP address
`secondary_mqtt_ssl_endpoint`
AmazonMQ secondary MQTT+SSL endpoint
`secondary_ssl_endpoint`
AmazonMQ secondary SSL endpoint
`secondary_stomp_ssl_endpoint`
AmazonMQ secondary STOMP+SSL endpoint
`secondary_wss_endpoint`
AmazonMQ secondary WSS endpoint
`security_group_arn`
The ARN of the created security group
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_mq_broker.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/mq_broker) (resource) - [`aws_mq_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/mq_configuration) (resource) - [`aws_ssm_parameter.mq_application_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.mq_application_username`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.mq_master_password`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.mq_master_username`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_password.mq_admin_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_password.mq_application_password`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/password) (resource) - [`random_pet.mq_admin_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) - [`random_pet.mq_application_user`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: --- ## msk-apache-kafka-cluster # Module: `msk-apache-kafka-cluster` Terraform module to provision [Amazon Managed Streaming](https://aws.amazon.com/msk/) for [Apache Kafka](https://aws.amazon.com/msk/what-is-kafka/) __Note:__ this module is intended for use with an existing VPC. To create a new VPC, use [terraform-aws-vpc](https://github.com/cloudposse/terraform-aws-vpc) module. **NOTE**: Release `0.8.0` contains breaking changes that will result in the destruction of your existing MSK cluster. To preserve the original cluster, follow the instructions in the [0.7.x to 0.8.x+ migration path](https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster/tree/main/docs/migration-0.7.x-0.8.x+.md). ## Usage Here's how to invoke this example module in your projects ```hcl module "kafka" { source = "cloudposse/msk-apache-kafka-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" kafka_version = "3.3.2" namespace = "eg" stage = "prod" name = "app" vpc_id = "vpc-XXXXXXXX" subnet_ids = ["subnet-XXXXXXXXX", "subnet-YYYYYYYY"] broker_per_zone = 2 broker_instance_type = "kafka.m5.large" # A list of IDs of Security Groups to associate the created resource with, in addition to the created security group associated_security_group_ids = ["sg-XXXXXXXXX", "sg-YYYYYYYY"] # A list of IDs of Security Groups to allow access to the cluster allowed_security_group_ids = ["sg-XXXXXXXXX", "sg-YYYYYYYY"] } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-msk-apache-kafka-cluster/) - complete example of using this module ## Variables ### Required Variables
`broker_instance_type` (`string`) required
The instance type to use for the Kafka brokers
`kafka_version` (`string`) required
The desired Kafka software version. Refer to https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html for more details
`subnet_ids` (`list(string)`) required
Subnet IDs for Client Broker
`vpc_id` (`string`) required
The ID of the VPC where the Security Group will be created.
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group(s) via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. For more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule and https://github.com/cloudposse/terraform-aws-security-group. **Default value:** `[ ]`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP addresses. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`autoscaling_enabled` (`bool`) optional
To automatically expand your cluster's storage in response to increased usage, you can enable this. [More info](https://docs.aws.amazon.com/msk/latest/developerguide/msk-autoexpand.html) **Default value:** `true`
`broker_dns_records_count` (`number`) optional
This variable specifies how many DNS records to create for the broker endpoints in the DNS zone provided in the `zone_id` variable. This corresponds to the total number of broker endpoints created by the module. Calculate this number by multiplying the `broker_per_zone` variable by the subnet count. This variable is necessary to prevent the Terraform error: The "count" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. **Default value:** `0`
`broker_per_zone` (`number`) optional
Number of Kafka brokers per zone **Default value:** `1`
`broker_volume_size` (`number`) optional
The size in GiB of the EBS volume for the data drive on each broker node **Default value:** `1000`
`certificate_authority_arns` (`list(string)`) optional
List of ACM Certificate Authority Amazon Resource Names (ARNs) to be used for TLS client authentication **Default value:** `[ ]`
`client_allow_unauthenticated` (`bool`) optional
Enable unauthenticated access **Default value:** `false`
`client_broker` (`string`) optional
Encryption setting for data in transit between clients and brokers. Valid values: `TLS`, `TLS_PLAINTEXT`, and `PLAINTEXT` **Default value:** `"TLS"`
`client_sasl_iam_enabled` (`bool`) optional
Enable client authentication via IAM policies. Cannot be set to `true` at the same time as `client_tls_auth_enabled` **Default value:** `false`
`client_sasl_scram_enabled` (`bool`) optional
Enable SCRAM client authentication via AWS Secrets Manager. Cannot be set to `true` at the same time as `client_tls_auth_enabled` **Default value:** `false`
`client_sasl_scram_secret_association_arns` (`list(string)`) optional
List of AWS Secrets Manager secret ARNs for SCRAM authentication **Default value:** `[ ]`
`client_sasl_scram_secret_association_enabled` (`bool`) optional
Enable the list of AWS Secrets Manager secret ARNs for SCRAM authentication **Default value:** `true`
`client_tls_auth_enabled` (`bool`) optional
Set `true` to enable the Client TLS Authentication **Default value:** `false`
`cloudwatch_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to Cloudwatch Logs **Default value:** `false`
`cloudwatch_logs_log_group` (`string`) optional
Name of the Cloudwatch Log Group to deliver logs to **Default value:** `null`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`custom_broker_dns_name` (`string`) optional
Custom Route53 DNS hostname for MSK brokers. Use `%%ID%%` key to specify brokers index in the hostname. Example: `kafka-broker%%ID%%.example.com` **Default value:** `null`
`encryption_at_rest_kms_key_arn` (`string`) optional
You may specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest **Default value:** `""`
`encryption_in_cluster` (`bool`) optional
Whether data communication among broker nodes is encrypted **Default value:** `true`
`enhanced_monitoring` (`string`) optional
Specify the desired enhanced MSK CloudWatch monitoring level. Valid values: `DEFAULT`, `PER_BROKER`, and `PER_TOPIC_PER_BROKER` **Default value:** `"DEFAULT"`
`firehose_delivery_stream` (`string`) optional
Name of the Kinesis Data Firehose delivery stream to deliver logs to **Default value:** `""`
`firehose_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to Kinesis Data Firehose **Default value:** `false`
`inline_rules_enabled` (`bool`) optional
NOT RECOMMENDED. Create rules "inline" instead of as separate `aws_security_group_rule` resources. See [#20046](https://github.com/hashicorp/terraform-provider-aws/issues/20046) for one of several issues with inline rules. See [this post](https://github.com/hashicorp/terraform-provider-aws/pull/9032#issuecomment-639545250) for details on the difference between inline rules and rule resources. **Default value:** `false`
`jmx_exporter_enabled` (`bool`) optional
Set `true` to enable the JMX Exporter **Default value:** `false`
`node_exporter_enabled` (`bool`) optional
Set `true` to enable the Node Exporter **Default value:** `false`
`preserve_security_group_id` (`bool`) optional
When `false` and `security_group_create_before_destroy` is `true`, changes to security group rules cause a new security group to be created with the new rules, and the existing security group is then replaced with the new one, eliminating any service interruption. When `true` or when changing the value (from `false` to `true` or from `true` to `false`), existing security group rules will be deleted before new ones are created, resulting in a service interruption, but preserving the security group itself. **NOTE:** Setting this to `true` does not guarantee the security group will never be replaced, it only keeps changes to the security group rules from triggering a replacement. See the [terraform-aws-security-group README](https://github.com/cloudposse/terraform-aws-security-group) for further discussion. **Default value:** `false`
`properties` (`map(string)`) optional
Contents of the server.properties file. Supported properties are documented in the [MSK Developer Guide](https://docs.aws.amazon.com/msk/latest/developerguide/msk-configuration-properties.html) **Default value:** `{ }`
`public_access_enabled` (`bool`) optional
Enable public access to MSK cluster (given that all of the requirements are met) **Default value:** `false`
`s3_logs_bucket` (`string`) optional
Name of the S3 bucket to deliver logs to **Default value:** `""`
`s3_logs_enabled` (`bool`) optional
Indicates whether you want to enable or disable streaming broker logs to S3 **Default value:** `false`
`s3_logs_prefix` (`string`) optional
Prefix to append to the S3 folder name logs are delivered to **Default value:** `""`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Managed by Terraform"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`security_group_rule_description` (`string`) optional
The description to place on each security group rule. The %s will be replaced with the protocol name **Default value:** `"Allow inbound %s traffic"`
`storage_autoscaling_disable_scale_in` (`bool`) optional
If the value is true, scale in is disabled and the target tracking policy won't remove capacity from the scalable resource **Default value:** `false`
`storage_autoscaling_max_capacity` (`number`) optional
Maximum size the autoscaling policy can scale storage. Defaults to `broker_volume_size` **Default value:** `null`
`storage_autoscaling_target_value` (`number`) optional
Percentage of storage used to trigger autoscaled storage increase **Default value:** `60`
`vpc_connectivity` optional
Optional VPC connectivity settings. Set to null to omit the entire `vpc_connectivity` block. Provide booleans for SASL IAM and/or SCRAM. Example: vpc_connectivity = \{ sasl_iam_enabled = true sasl_scram_enabled = true \} **Type:** ```hcl object({ sasl_iam_enabled = optional(bool) sasl_scram_enabled = optional(bool) }) ``` **Default value:** `null`
`zone_id` (`string`) optional
Route53 DNS Zone ID for MSK broker hostnames **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bootstrap_brokers`
Comma separated list of one or more hostname:port pairs of Kafka brokers suitable to bootstrap connectivity to the Kafka cluster
`bootstrap_brokers_public_sasl_iam`
Comma separated list of one or more DNS names (or IP addresses) and SASL IAM port pairs for public access to the Kafka cluster using SASL/IAM
`bootstrap_brokers_public_sasl_scram`
Comma separated list of one or more DNS names (or IP addresses) and SASL SCRAM port pairs for public access to the Kafka cluster using SASL/SCRAM
`bootstrap_brokers_public_tls`
Comma separated list of one or more DNS names (or IP addresses) and TLS port pairs for public access to the Kafka cluster using TLS
`bootstrap_brokers_sasl_iam`
Comma separated list of one or more DNS names (or IP addresses) and SASL IAM port pairs for access to the Kafka cluster using SASL/IAM
`bootstrap_brokers_sasl_scram`
Comma separated list of one or more DNS names (or IP addresses) and SASL SCRAM port pairs for access to the Kafka cluster using SASL/SCRAM
`bootstrap_brokers_tls`
Comma separated list of one or more DNS names (or IP addresses) and TLS port pairs for access to the Kafka cluster using TLS
`broker_endpoints`
List of broker endpoints
`cluster_arn`
Amazon Resource Name (ARN) of the MSK cluster
`cluster_name`
MSK Cluster name
`config_arn`
Amazon Resource Name (ARN) of the MSK configuration
`current_version`
Current version of the MSK Cluster
`hostnames`
List of MSK Cluster broker DNS hostnames
`latest_revision`
Latest revision of the MSK configuration
`security_group_arn`
The ARN of the created security group
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
`storage_mode`
Storage mode for supported storage tiers
`zookeeper_connect_string`
Comma separated list of one or more hostname:port pairs to connect to the Apache Zookeeper cluster
`zookeeper_connect_string_tls`
Comma separated list of one or more hostname:port pairs to connect to the Apache Zookeeper cluster via TLS
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `hostname` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `security_group` | 2.2.0 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/2.2.0) | https://github.com/cloudposse/terraform-aws-security-group/blob/master/docs/migration-v1-v2.md `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_appautoscaling_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_target.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) - [`aws_msk_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_cluster) (resource) - [`aws_msk_configuration.config`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_configuration) (resource) - [`aws_msk_scram_secret_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/msk_scram_secret_association) (resource) ## Data Sources The following data sources are used by this module: - [`aws_msk_broker_nodes.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/msk_broker_nodes) (data source) --- ## multi-az-subnets # Module: `multi-az-subnets` Terraform module for multi-AZ [`subnets`](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html) provisioning. The module creates private and public subnets in the provided Availability Zones. The public subnets are routed to the Internet Gateway specified by `var.igw_id`. `nat_gateway_enabled` flag controls the creation of NAT Gateways in the public subnets. The private subnets are routed to the NAT Gateways provided in the `var.az_ngw_ids` map. If you are creating subnets inside a VPC, consider using [cloudposse/terraform-aws-dynamic-subnets](https://github.com/cloudposse/terraform-aws-dynamic-subnets) instead. ## Screenshots ![terraform-aws-multi-az-subnets](images/terraform-aws-multi-az-subnets.png) *Example of `terraform apply` outputs* ## Usage ```hcl locals { public_cidr_block = cidrsubnet(var.cidr_block, 1, 0) private_cidr_block = cidrsubnet(var.cidr_block, 1, 1) } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name cidr_block = var.cidr_block } module "public_subnets" { source = "cloudposse/multi-az-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name availability_zones = ["us-east-2a", "us-east-2b", "us-east-2c"] vpc_id = module.vpc.vpc_id cidr_block = local.public_cidr_block type = "public" igw_id = module.vpc.igw_id nat_gateway_enabled = "true" } module "private_subnets" { source = "cloudposse/multi-az-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name availability_zones = ["us-east-2a", "us-east-2b", "us-east-2c"] vpc_id = module.vpc.vpc_id cidr_block = local.private_cidr_block type = "private" az_ngw_ids = module.public_subnets.az_ngw_ids } ``` ## Examples Given the following configuration ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace name = "vpc" stage = var.stage cidr_block = var.cidr_block } locals { public_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 1, 0) private_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 1, 1) } module "public_subnets" { source = "cloudposse/multi-az-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name availability_zones = ["us-east-2a", "us-east-2b", "us-east-2c"] vpc_id = module.vpc.vpc_id cidr_block = local.public_cidr_block type = "public" igw_id = module.vpc.igw_id nat_gateway_enabled = "true" } module "private_subnets" { source = "cloudposse/multi-az-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name availability_zones = ["us-east-2a", "us-east-2b", "us-east-2c"] vpc_id = module.vpc.vpc_id cidr_block = local.private_cidr_block type = "private" az_ngw_ids = module.public_subnets.az_ngw_ids } output "private_az_subnet_ids" { value = module.private_subnets.az_subnet_ids } output "public_az_subnet_ids" { value = module.public_subnets.az_subnet_ids } ``` the output Maps of AZ names to subnet IDs look like these ```hcl public_az_subnet_ids = { us-east-2a = subnet-ea58d78e us-east-2b = subnet-556ee131 us-east-2c = subnet-6f54db0b } private_az_subnet_ids = { us-east-2a = subnet-376de253 us-east-2b = subnet-9e53dcfa us-east-2c = subnet-a86fe0cc } ``` and the created subnet IDs could be found by the AZ names using `map["key"]` or [`lookup(map, key, [default])`](https://www.terraform.io/docs/configuration/interpolation.html#lookup-map-key-default-), for example: `public_az_subnet_ids["us-east-2a"]` `lookup(private_az_subnet_ids, "us-east-2b")` ## Variables ### Required Variables
`availability_zones` (`list(string)`) required
List of Availability Zones (e.g. `['us-east-1a', 'us-east-1b', 'us-east-1c']`)
`cidr_block` (`string`) required
Base CIDR block which is divided into subnet CIDR blocks (e.g. `10.0.0.0/16`)
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`az_ngw_ids` (`map(string)`) optional
Only for private subnets. Map of AZ names to NAT Gateway IDs that are used as default routes when creating private subnets. You should either supply one NAT Gateway ID for each AZ in `var.availability_zones` or leave the map empty. If empty, no default egress route will be created and you will have to create your own using `aws_route`. **Default value:** `{ }`
`igw_id` (`string`) optional
Internet Gateway ID that is used as a default route when creating public subnets (e.g. `igw-9c26a123`) **Default value:** `""`
`ipv6_cidr_block` (`string`) optional
Base IPv6 CIDR block which is divided into /64 subnet CIDR blocks **Default value:** `null`
`ipv6_enabled` (`bool`) optional
Flag to enable/disable IPv6 creation in public subnets **Default value:** `false`
`max_subnets` (`string`) optional
Maximum number of subnets that can be created. The variable is used for CIDR blocks calculation **Default value:** `"6"`
`nat_gateway_enabled` (`string`) optional
Flag to enable/disable NAT Gateways creation in public subnets **Default value:** `"true"`
`private_network_acl_egress` (`list(map(string))`) optional
Egress network ACL rules **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`private_network_acl_id` (`string`) optional
Network ACL ID that is added to the private subnets. If empty, a new ACL will be created **Default value:** `""`
`private_network_acl_ingress` (`list(map(string))`) optional
Egress network ACL rules **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`public_network_acl_egress` (`list(map(string))`) optional
Egress network ACL rules **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`public_network_acl_id` (`string`) optional
Network ACL ID that is added to the public subnets. If empty, a new ACL will be created **Default value:** `""`
`public_network_acl_ingress` (`list(map(string))`) optional
Egress network ACL rules **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`type` (`string`) optional
Type of subnets to create (`private` or `public`) **Default value:** `"private"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`az_ngw_ids`
Map of AZ names to NAT Gateway IDs (only for public subnets)
`az_route_table_ids`
Map of AZ names to Route Table IDs
`az_subnet_arns`
Map of AZ names to subnet ARNs
`az_subnet_cidr_blocks`
Map of AZ names to subnet CIDR blocks
`az_subnet_ids`
Map of AZ names to subnet IDs
`az_subnet_ipv6_cidr_blocks`
Map of AZ names to subnet IPv6 CIDR blocks
`az_subnet_map`
Map of AZ names to map of information about subnets
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.0` - `local`, version: `>= 1.2` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `private_label` | 0.24.1 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.24.1) | n/a `public_label` | 0.24.1 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.24.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_nat_gateway.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway) (resource) - [`aws_network_acl.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_network_acl.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_route.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public_ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route_table.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table_association.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_route_table_association.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_subnet.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) - [`aws_subnet.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) ## Data Sources The following data sources are used by this module: --- ## mwaa(Mwaa) # Module: `mwaa` Terraform module to provision Amazon Managed Workflows for Apache Airflow ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-mwaa/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-mwaa/tree/main/test). ```hcl module "mwaa" { source = "cloudposse/mwaa/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = var.vpc_id subnet_ids = var.subnet_ids airflow_version = "2.0.2" dag_s3_path = "dags" environment_class = "mw1.small" min_workers = 1 max_workers = 10 webserver_access_mode = "PRIVATE_ONLY" dag_processing_logs_enabled = true dag_processing_logs_level = "INFO" name = "app" stage = "test" namespace = "eg" enabled = true } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-mwaa/) - complete example of using this module ## Variables ### Required Variables
`region` (`string`) required
AWS region
`subnet_ids` (`list(string)`) required
The private subnet IDs in which the environment should be created. MWAA requires two subnets
`vpc_id` (`string`) required
The ID of the VPC where the Security Group will be created.
### Optional Variables
`additional_security_group_rules` (`list(any)`) optional
A list of Security Group rule objects to add to the created security group, in addition to the ones this module normally creates. (To suppress the module's rules, set `create_security_group` to false and supply your own security group(s) via `associated_security_group_ids`.) The keys and values of the objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. For more info see https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule and https://github.com/cloudposse/terraform-aws-security-group. **Default value:** `[ ]`
`additionals_managed_policy_arns` (`list(any)`) optional
List of managed policies to attach to the MWAA IAM role **Default value:** `[ ]`
`additionals_policy_documents` (`list(any)`) optional
List of JSON IAM policy documents to attach to the MWAA IAM role **Default value:** `[ ]`
`airflow_configuration_options` (`any`) optional
The Airflow override options **Default value:** `null`
`airflow_version` (`string`) optional
Airflow version of the MWAA environment, will be set by default to the latest version that MWAA supports. **Default value:** `""`
`allow_all_egress` (`bool`) optional
If `true`, the created security group will allow egress on all ports and protocols to all IP addresses. If this is false and no egress rules are otherwise specified, then no egress will be allowed. **Default value:** `true`
`allowed_cidr_blocks` (`list(string)`) optional
A list of IPv4 CIDRs to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`allowed_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to allow access to the security group created by this module. The length of this list must be known at "plan" time. **Default value:** `[ ]`
`associated_security_group_ids` (`list(string)`) optional
A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if `create_security_group` is `false`, must have rules providing the desired access. **Default value:** `[ ]`
`create_iam_role` (`bool`) optional
Enabling or disabling the creatation of a default IAM Role for AWS MWAA **Default value:** `true`
`create_s3_bucket` (`bool`) optional
Enabling or disabling the creatation of an S3 bucket for AWS MWAA **Default value:** `true`
`create_security_group` (`bool`) optional
Set `true` to create and configure a new security group. If false, `associated_security_group_ids` must be provided. **Default value:** `true`
`dag_processing_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for processing DAGs **Default value:** `false`
`dag_processing_logs_level` (`string`) optional
DAG processing logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`dag_s3_path` (`string`) optional
The relative path to the DAG folder on your Amazon S3 storage bucket. **Default value:** `"dags"`
`environment_class` (`string`) optional
Environment class for the cluster. Possible options are mw1.small, mw1.medium, mw1.large. **Default value:** `"mw1.small"`
`execution_role_arn` (`string`) optional
If `create_iam_role` is `false` then set this to the target MWAA execution role **Default value:** `""`
`kms_key` (`string`) optional
The Amazon Resource Name (ARN) of your KMS key that you want to use for encryption. Will be set to the ARN of the managed KMS key aws/airflow by default. **Default value:** `null`
`max_webservers` (`number`) optional
The maximum number of web servers that you want to run in your environment. **Default value:** `2`
`max_workers` (`number`) optional
The maximum number of workers that can be automatically scaled up. Value need to be between 1 and 25. **Default value:** `10`
`min_webservers` (`number`) optional
The minimum number of web servers that you want to run in your environment. **Default value:** `2`
`min_workers` (`number`) optional
The minimum number of workers that you want to run in your environment. **Default value:** `1`
`plugins_s3_object_version` (`string`) optional
The plugins.zip file version you want to use. **Default value:** `null`
`plugins_s3_path` (`string`) optional
The relative path to the plugins.zip file on your Amazon S3 storage bucket. For example, plugins.zip. If a relative path is provided in the request, then plugins_s3_object_version is required **Default value:** `null`
`requirements_s3_object_version` (`string`) optional
The requirements.txt file version you **Default value:** `null`
`requirements_s3_path` (`string`) optional
The relative path to the requirements.txt file on your Amazon S3 storage bucket. For example, requirements.txt. If a relative path is provided in the request, then requirements_s3_object_version is required **Default value:** `null`
`scheduler_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the schedulers **Default value:** `false`
`scheduler_logs_level` (`string`) optional
Schedulers logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`schedulers` (`number`) optional
The number of schedulers that you want to run in your environment. **Default value:** `2`
`security_group_create_before_destroy` (`bool`) optional
Set `true` to enable Terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are upgrading this module and need to keep the existing security group from being replaced. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
Security Group for AWS MWAA **Default value:** `"Managed by Terraform"`
`security_group_name` (`list(string)`) optional
The name to assign to the created security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`source_bucket_arn` (`string`) optional
If `create_s3_bucket` is `false` then set this to the Amazon Resource Name (ARN) of your Amazon S3 storage bucket. **Default value:** `null`
`startup_script_s3_object_version` (`string`) optional
The version of the startup shell script you want to use. You must specify the version ID that Amazon S3 assigns to the file every time you update the script. **Default value:** `null`
`startup_script_s3_path` (`string`) optional
The relative path to the script hosted in your bucket. The script runs as your environment starts before starting the Apache Airflow process. **Default value:** `null`
`task_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for DAG tasks **Default value:** `false`
`task_logs_level` (`string`) optional
DAG tasks logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`webserver_access_mode` (`string`) optional
Specifies whether the webserver should be accessible over the internet or via your specified VPC. Possible options: PRIVATE_ONLY (default) and PUBLIC_ONLY. **Default value:** `"PRIVATE_ONLY"`
`webserver_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the webservers **Default value:** `false`
`webserver_logs_level` (`string`) optional
Webserver logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
`weekly_maintenance_window_start` (`string`) optional
Specifies the start date for the weekly maintenance window. **Default value:** `null`
`worker_logs_enabled` (`bool`) optional
Enabling or disabling the collection of logs for the workers **Default value:** `false`
`worker_logs_level` (`string`) optional
Workers logging level. Valid values: CRITICAL, ERROR, WARNING, INFO, DEBUG **Default value:** `"INFO"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the Amazon MWAA Environment
`created_at`
The Created At date of the Amazon MWAA Environment
`execution_role_arn`
IAM Role ARN for Amazon MWAA Execution Role
`logging_configuration`
The Logging Configuration of the Amazon MWAA Environment
`s3_bucket_arn`
ARN of the S3 bucket
`security_group_arn`
The ARN of the created security group
`security_group_id`
The ID of the created security group
`security_group_name`
The name of the created security group
`service_role_arn`
The Service Role ARN of the Amazon MWAA Environment
`status`
The status of the Amazon MWAA Environment
`tags_all`
A map of tags assigned to the resource, including those inherited from the provider for the Amazon MWAA Environment
`webserver_url`
The webserver URL of the Amazon MWAA Environment
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.2.0` ### Providers - `aws`, version: `>= 4.2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `mwaa_iam_role` | 0.20.0 | [`cloudposse/iam-role/aws`](https://registry.terraform.io/modules/cloudposse/iam-role/aws/0.20.0) | n/a `mwaa_s3_bucket` | 4.0.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.0.0) | n/a `mwaa_security_group` | 1.0.1 | [`cloudposse/security-group/aws`](https://registry.terraform.io/modules/cloudposse/security-group/aws/1.0.1) | n/a `s3_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sg_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_mwaa_environment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/mwaa_environment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## named-subnets # Module: `named-subnets` Terraform module for named [`subnets`](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html) provisioning. ## Usage Simple example, with private and public subnets in one Availability Zone: ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" name = "vpc" stage = "dev" cidr_block = var.cidr_block } locals { public_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 1, 0) private_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 1, 1) } module "public_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["web1", "web2", "web3"] vpc_id = module.vpc.vpc_id cidr_block = local.public_cidr_block type = "public" igw_id = module.vpc.igw_id availability_zone = "us-east-1a" } module "private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "database" subnet_names = ["kafka", "cassandra", "zookeeper"] vpc_id = module.vpc.vpc_id cidr_block = local.private_cidr_block type = "private" availability_zone = "us-east-1a" ngw_id = module.public_subnets.ngw_id } ``` Simple example, with `ENI` as default route gateway for private subnets ```hcl resource "aws_network_interface" "default" { subnet_id = module.us_east_1b_public_subnets.subnet_ids[0] source_dest_check = false tags = module.network_interface_label.id } module "us_east_1b_private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["charlie", "echo", "bravo"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1b_private_cidr_block type = "private" availability_zone = "us-east-1b" eni_id = aws_network_interface.default.id attributes = ["us-east-1b"] } ``` Full example, with private and public subnets in two Availability Zones for High Availability: ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" name = "vpc" stage = "dev" cidr_block = var.cidr_block } locals { us_east_1a_public_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 2, 0) us_east_1a_private_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 2, 1) us_east_1b_public_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 2, 2) us_east_1b_private_cidr_block = cidrsubnet(module.vpc.vpc_cidr_block, 2, 3) } module "us_east_1a_public_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["apples", "oranges", "grapes"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1a_public_cidr_block type = "public" igw_id = module.vpc.igw_id availability_zone = "us-east-1a" attributes = ["us-east-1a"] } module "us_east_1a_private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["charlie", "echo", "bravo"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1a_private_cidr_block type = "private" availability_zone = "us-east-1a" ngw_id = module.us_east_1a_public_subnets.ngw_id attributes = ["us-east-1a"] } module "us_east_1b_public_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["apples", "oranges", "grapes"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1b_public_cidr_block type = "public" igw_id = module.vpc.igw_id availability_zone = "us-east-1b" attributes = ["us-east-1b"] } module "us_east_1b_private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["charlie", "echo", "bravo"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1b_private_cidr_block type = "private" availability_zone = "us-east-1b" ngw_id = module.us_east_1b_public_subnets.ngw_id attributes = ["us-east-1b"] } resource "aws_network_interface" "default" { subnet_id = module.us_east_1b_public_subnets.subnet_ids[0] source_dest_check = false tags = module.network_interface_label.id } module "us_east_1b_private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["charlie", "echo", "bravo"] vpc_id = module.vpc.vpc_id cidr_block = local.us_east_1b_private_cidr_block type = "private" availability_zone = "us-east-1b" eni_id = aws_network_interface.default.id attributes = ["us-east-1b"] } ``` ## Caveat You must use only one type of device for a default route gateway per route table. `ENI` or `NGW` Given the following configuration (see the Simple example above) ```hcl locals { public_cidr_block = cidrsubnet(var.vpc_cidr, 1, 0) private_cidr_block = cidrsubnet(var.vpc_cidr, 1, 1) } module "public_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "app" subnet_names = ["web1", "web2", "web3"] vpc_id = var.vpc_id cidr_block = local.public_cidr_block type = "public" availability_zone = "us-east-1a" igw_id = var.igw_id } module "private_subnets" { source = "cloudposse/named-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "database" subnet_names = ["kafka", "cassandra", "zookeeper"] vpc_id = var.vpc_id cidr_block = local.private_cidr_block type = "private" availability_zone = "us-east-1a" ngw_id = module.public_subnets.ngw_id } output "private_named_subnet_ids" { value = module.private_subnets.named_subnet_ids } output "public_named_subnet_ids" { value = module.public_subnets.named_subnet_ids } ``` the output Maps of subnet names to subnet IDs look like these ```hcl public_named_subnet_ids = { web1 = subnet-ea58d78e web2 = subnet-556ee131 web3 = subnet-6f54db0b } private_named_subnet_ids = { cassandra = subnet-376de253 kafka = subnet-9e53dcfa zookeeper = subnet-a86fe0cc } ``` and the created subnet IDs could be found by the subnet names using `map["key"]` or [`lookup(map, key, [default])`](https://www.terraform.io/docs/configuration/interpolation.html#lookup-map-key-default-), for example: `public_named_subnet_ids["web1"]` `lookup(private_named_subnet_ids, "kafka")` ## Variables ### Required Variables
`availability_zone` (`string`) required
Availability Zone
`cidr_block` (`string`) required
Base CIDR block which will be divided into subnet CIDR blocks (e.g. `10.0.0.0/16`)
`subnet_names` (`list(string)`) required
List of subnet names (e.g. `['apples', 'oranges', 'grapes']`)
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`eni_id` (`string`) optional
An ID of a network interface which is used as a default route in private route tables (_e.g._ `eni-9c26a123`). Conflicts with `ngw_id`. **Default value:** `""`
`igw_id` (`string`) optional
Internet Gateway ID which will be used as a default route in public route tables (e.g. `igw-9c26a123`). **Default value:** `""`
`map_public_ip_on_launch_enabled` (`bool`) optional
Enable/disable map_public_ip_on_launch subnet attribute. **Default value:** `false`
`max_subnets` (`number`) optional
Maximum number of subnets which can be created. This variable is being used for CIDR blocks calculation. Defaults to length of `subnet_names` argument **Default value:** `16`
`nat_enabled` (`bool`) optional
Enable/disable NAT Gateway **Default value:** `true`
`ngw_id` (`string`) optional
NAT Gateway ID which will be used as a default route in private route tables (e.g. `igw-9c26a123`). Conflicts with `eni_id`. **Default value:** `""`
`private_network_acl_egress` optional
Private network egress ACL rules **Type:** ```hcl list(object( { rule_no = number action = string cidr_block = string from_port = number to_port = number protocol = string })) ``` **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`private_network_acl_id` (`string`) optional
Network ACL ID that will be added to the subnets. If empty, a new ACL will be created **Default value:** `""`
`private_network_acl_ingress` optional
Private network ingress ACL rules **Type:** ```hcl list(object( { rule_no = number action = string cidr_block = string from_port = number to_port = number protocol = string })) ``` **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`public_network_acl_egress` optional
Public network egress ACL rules **Type:** ```hcl list(object( { rule_no = number action = string cidr_block = string from_port = number to_port = number protocol = string })) ``` **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`public_network_acl_id` (`string`) optional
Network ACL ID that will be added to the subnets. If empty, a new ACL will be created **Default value:** `""`
`public_network_acl_ingress` optional
Public network ingress ACL rules **Type:** ```hcl list(object( { rule_no = number action = string cidr_block = string from_port = number to_port = number protocol = string })) ``` **Default value:** ```hcl [ { "action": "allow", "cidr_block": "0.0.0.0/0", "from_port": 0, "protocol": "-1", "rule_no": 100, "to_port": 0 } ] ```
`type` (`string`) optional
Type of subnets (`private` or `public`) **Default value:** `"private"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`named_subnet_ids`
Map of subnet names to subnet IDs
`ngw_id`
NAT Gateway ID
`ngw_private_ip`
Private IP address of the NAT Gateway
`ngw_public_ip`
Public IP address of the NAT Gateway
`route_table_ids`
Route table IDs
`subnet_ids`
Subnet IDs
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `null`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `private_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `public_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_nat_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway) (resource) - [`aws_network_acl.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_network_acl.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/network_acl) (resource) - [`aws_route.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route_table.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table) (resource) - [`aws_route_table_association.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_route_table_association.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route_table_association) (resource) - [`aws_subnet.private`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) - [`aws_subnet.public`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/subnet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_vpc.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## network-firewall(Network-firewall) # Module: `network-firewall` Terraform module to provision AWS Network Firewall resources. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-network-firewall/tree/main/examples/complete) For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-network-firewall/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = [module.vpc.vpc_cidr_block] nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "s3_log_storage" { source = "cloudposse/s3-log-storage/aws" version = "1.0.0" force_destroy = true attributes = ["logs"] context = module.this.context } module "network_firewall" { source = "cloudposse/network-firewall/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.private_subnet_ids network_firewall_name = var.network_firewall_name network_firewall_description = var.network_firewall_description network_firewall_policy_name = var.network_firewall_policy_name policy_stateful_engine_options_rule_order = var.policy_stateful_engine_options_rule_order stateful_default_actions = var.stateful_default_actions stateless_default_actions = var.stateless_default_actions stateless_fragment_default_actions = var.stateless_fragment_default_actions stateless_custom_actions = var.stateless_custom_actions delete_protection = var.delete_protection firewall_policy_change_protection = var.firewall_policy_change_protection subnet_change_protection = var.subnet_change_protection logging_config = { flow = { log_destination_type = "S3" log_type = "FLOW" log_destination = { bucketName = module.s3_log_storage.bucket_id prefix = "/flow" } }, alert = { log_destination_type = "S3" log_type = "ALERT" log_destination = { bucketName = module.s3_log_storage.bucket_id prefix = "/alert" } } } rule_group_config = { stateful-inspection-for-blocking-packets-from-going-to-destination = { capacity = 50 name = "block-packets-from-reaching-destination" description = "Stateful Inspection for blocking packets from going to an intended destination" type = "STATEFUL" rule_group = { stateful_rule_options = { rule_order = "STRICT_ORDER" } rules_source = { stateful_rule = [ { action = "DROP" header = { destination = "124.1.1.24/32" destination_port = 53 direction = "ANY" protocol = "TCP" source = "1.2.3.4/32" source_port = 53 } rule_option = [ { keyword = "sid" settings = ["1"] } ] } ] } } } } context = module.this.context } ``` ## Variables ### Required Variables
`rule_group_config` (`any`) required
Rule group configuration. Refer to [networkfirewall_rule_group](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_rule_group) for configuration details
`subnet_ids` (`list(string)`) required
List of subnet IDs for firewall endpoints
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`delete_protection` (`bool`) optional
A boolean flag indicating whether it is possible to delete the firewall **Default value:** `false`
`firewall_policy_change_protection` (`bool`) optional
A boolean flag indicating whether it is possible to change the associated firewall policy **Default value:** `false`
`logging_config` (`map(any)`) optional
Logging configuration **Default value:** `{ }`
`network_firewall_description` (`string`) optional
AWS Network Firewall description. If not provided, the Network Firewall name will be used **Default value:** `null`
`network_firewall_name` (`string`) optional
AWS Network Firewall name. If not provided, the name will be derived from the context **Default value:** `null`
`network_firewall_policy_name` (`string`) optional
AWS Network Firewall policy name. If not provided, the name will be derived from the context **Default value:** `null`
`policy_stateful_engine_options_rule_order` (`string`) optional
Indicates how to manage the order of stateful rule evaluation for the policy. Valid values: DEFAULT_ACTION_ORDER, STRICT_ORDER **Default value:** `null`
`stateful_default_actions` (`list(string)`) optional
Default stateful actions **Default value:** ```hcl [ "aws:alert_strict" ] ```
`stateless_custom_actions` optional
Set of configuration blocks describing the custom action definitions that are available for use in the firewall policy's `stateless_default_actions` **Type:** ```hcl list(object({ action_name = string dimensions = list(string) })) ``` **Default value:** `[ ]`
`stateless_default_actions` (`list(string)`) optional
Default stateless actions **Default value:** ```hcl [ "aws:forward_to_sfe" ] ```
`stateless_fragment_default_actions` (`list(string)`) optional
Default stateless actions for fragmented packets **Default value:** ```hcl [ "aws:forward_to_sfe" ] ```
`subnet_change_protection` (`bool`) optional
A boolean flag indicating whether it is possible to change the associated subnet(s) **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`az_subnet_endpoint_stats`
List of objects with each object having three items: AZ, subnet ID, firewall VPC endpoint ID
`network_firewall_arn`
Network Firewall ARN
`network_firewall_name`
Network Firewall ID
`network_firewall_policy_arn`
Network Firewall policy ARN
`network_firewall_policy_name`
Network Firewall policy ID
`network_firewall_status`
Nested list of information about the current status of the Network Firewall
`network_firewall_update_token`
A string token used when updating the Network Firewall
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_networkfirewall_firewall.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_firewall) (resource) - [`aws_networkfirewall_firewall_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_firewall_policy) (resource) - [`aws_networkfirewall_logging_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_logging_configuration) (resource) - [`aws_networkfirewall_rule_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/networkfirewall_rule_group) (resource) ## Data Sources The following data sources are used by this module: --- ## nlb(Nlb) # Module: `nlb` Terraform module to create an NLB and a default NLB target and related security groups. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-nlb/tree/main/examples/complete). For automated test of the complete example using `bats` and `Terratest`, see [test](https://github.com/cloudposse/terraform-aws-nlb/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = var.vpc_cidr_block context = module.this.context namespace = "eg" } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context namespace = "eg" } module "nlb" { source = "cloudposse/nlb/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.public_subnet_ids internal = var.internal tcp_enabled = var.tcp_enabled access_logs_enabled = var.access_logs_enabled nlb_access_logs_s3_bucket_force_destroy = var.nlb_access_logs_s3_bucket_force_destroy nlb_access_logs_s3_bucket_force_destroy_enabled = var.nlb_access_logs_s3_bucket_force_destroy_enabled cross_zone_load_balancing_enabled = var.cross_zone_load_balancing_enabled idle_timeout = var.idle_timeout ip_address_type = var.ip_address_type deletion_protection_enabled = var.deletion_protection_enabled deregistration_delay = var.deregistration_delay health_check_path = var.health_check_path health_check_timeout = var.health_check_timeout health_check_threshold = var.health_check_healthy_threshold health_check_unhealthy_threshold = var.health_check_unhealthy_threshold health_check_interval = var.health_check_interval target_group_port = var.target_group_port target_group_target_type = var.target_group_target_type context = module.this.context } ``` ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
A list of subnet IDs to associate with NLB
`vpc_id` (`string`) required
VPC ID to associate with NLB
### Optional Variables
`access_logs_enabled` (`bool`) optional
A boolean flag to enable/disable access_logs **Default value:** `true`
`access_logs_prefix` (`string`) optional
The S3 log bucket prefix **Default value:** `""`
`access_logs_s3_bucket_id` (`string`) optional
An external S3 Bucket name to store access logs in. If specified, no logging bucket will be created. **Default value:** `null`
`additional_certs` (`list(string)`) optional
A list of additonal certs to add to the https listerner **Default value:** `[ ]`
`allow_ssl_requests_only` (`bool`) optional
Set to true to require requests to use Secure Socket Layer (HTTPS/SSL) on the access logs S3 bucket. This will explicitly deny access to HTTP requests **Default value:** `false`
`certificate_arn` (`string`) optional
The ARN of the default SSL certificate for HTTPS listener **Default value:** `""`
`connection_termination_enabled` (`bool`) optional
Whether to terminate connections at the end of the deregistration timeout **Default value:** `false`
`cross_zone_load_balancing_enabled` (`bool`) optional
A boolean flag to enable/disable cross zone load balancing **Default value:** `true`
`default_listener_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in TLS security group **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`default_listener_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to TLS ingress security group **Default value:** `[ ]`
`deletion_protection_enabled` (`bool`) optional
A boolean flag to enable/disable deletion protection for NLB **Default value:** `false`
`deregistration_delay` (`number`) optional
The amount of time to wait in seconds before changing the state of a deregistering target to unused **Default value:** `15`
`eip_additional_tags` (`map(string)`) optional
The additional tags to apply to the generated eip **Default value:** `{ }`
`eip_allocation_ids` (`list(string)`) optional
Allocation ID for EIP for subnets. The length of the list must correspond to the number of defined subnents. If the `subnet_mapping_enabled` variable is not defined and enabled `subnet_mapping_enabled`, EIPs will be created **Default value:** `[ ]`
`enable_glacier_transition` (`bool`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files **Default value:** `true`
`enforce_security_group_inbound_rules_on_private_link_traffic` (`string`) optional
Indicates whether inbound security group rules are enforced for traffic originating from a PrivateLink. Only valid for Load Balancers of type network. The possible values are on and off. **Default value:** `null`
`expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to expunge the objects **Default value:** `90`
`glacier_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to move the data to the Glacier Flexible Retrieval storage tier **Default value:** `60`
`health_check_enabled` (`bool`) optional
A boolean flag to enable/disable the NLB health checks **Default value:** `true`
`health_check_interval` (`number`) optional
The duration in seconds in between health checks **Default value:** `10`
`health_check_matcher` (`string`) optional
The HTTP or gRPC codes to use when checking for a successful response from a target, values can be comma-separated individual values or a range of values **Default value:** `null`
`health_check_path` (`string`) optional
The destination for the health check request **Default value:** `"/"`
`health_check_port` (`number`) optional
The port to send the health check request to (defaults to `traffic-port`) **Default value:** `null`
`health_check_protocol` (`string`) optional
The protocol to use for the health check request **Default value:** `null`
`health_check_threshold` (`number`) optional
The number of consecutive health checks successes required before considering an unhealthy target healthy. **Default value:** `2`
`health_check_timeout` (`number`) optional
The amount of time, in seconds, during which no response means a failed health check **Default value:** `null`
`health_check_unhealthy_threshold` (`number`) optional
The number of consecutive health check failures required before considering the target unhealthy. If not set using value from `health_check_threshold` **Default value:** `null`
`internal` (`bool`) optional
A boolean flag to determine whether the NLB should be internal **Default value:** `false`
`ip_address_type` (`string`) optional
The type of IP addresses used by the subnets for your load balancer. The possible values are `ipv4` and `dualstack`. **Default value:** `"ipv4"`
`lifecycle_configuration_rules` optional
A list of S3 bucket v2 lifecycle rules, as specified in [terraform-aws-s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket)" These rules are not affected by the deprecated `lifecycle_rule_enabled` flag. **NOTE:** Unless you also set `lifecycle_rule_enabled = false` you will also get the default deprecated rules set on your bucket. **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = any expiration = any transition = list(any) noncurrent_version_expiration = any noncurrent_version_transition = list(any) })) ``` **Default value:** `[ ]`
`lifecycle_rule_enabled` (`bool`) optional
DEPRECATED: Defaults to `false`, use `lifecycle_configuration_rules` instead. When `true`, configures lifecycle events on this bucket using individual (now deprecated) variables." **Default value:** `false`
`load_balancer_name` (`string`) optional
The name for the default load balancer, uses a module label name if left empty **Default value:** `""`
`load_balancer_name_max_length` (`number`) optional
The max length of characters for the load balancer name. **Default value:** `32`
`nlb_access_logs_s3_bucket_force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the NLB access logs S3 bucket so that the bucket can be destroyed without error **Default value:** `false`
`noncurrent_version_expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies when non-current object versions expire (in days) **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies (in days) when noncurrent object versions transition to Glacier Flexible Retrieval **Default value:** `30`
`security_group_enabled` (`bool`) optional
Enables the security group **Default value:** `false`
`security_group_ids` (`list(string)`) optional
A list of additional security group IDs to allow access to NLB **Default value:** `[ ]`
`slow_start` (`number`) optional
Amount time for targets to warm up before the load balancer sends them a full share of requests. The range is 30-900 seconds or 0 to disable. **Default value:** `0`
`standard_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `30`
`stickiness_enabled` (`bool`) optional
Whether to enable sticky sessions **Default value:** `false`
`subnet_mapping_enabled` (`bool`) optional
Enable generate EIP for defined subnet ids **Default value:** `false`
`target_group_additional_tags` (`map(string)`) optional
The additional tags to apply to the default target group **Default value:** `{ }`
`target_group_enabled` (`bool`) optional
Whether or not to create the default target group and listener **Default value:** `true`
`target_group_ip_address_type` (`string`) optional
The type of IP addresses used by the target group. The possible values are `ipv4` and `ipv6`. **Default value:** `"ipv4"`
`target_group_name` (`string`) optional
The name for the default target group, uses a module label name if left empty **Default value:** `""`
`target_group_name_max_length` (`number`) optional
The max length of characters for the target group name. **Default value:** `32`
`target_group_port` (`number`) optional
The port for the default target group **Default value:** `80`
`target_group_preserve_client_ip` (`bool`) optional
A boolean flag to enable/disable client IP preservation. **Default value:** `false`
`target_group_proxy_protocol_v2` (`bool`) optional
A boolean flag to enable/disable proxy protocol v2 support **Default value:** `false`
`target_group_target_type` (`string`) optional
The type (`instance`, `ip` or `lambda`) of targets that can be registered with the default target group **Default value:** `"ip"`
`tcp_enabled` (`bool`) optional
A boolean flag to enable/disable TCP listener **Default value:** `true`
`tcp_port` (`number`) optional
The port for the TCP listener **Default value:** `80`
`tls_enabled` (`bool`) optional
A boolean flag to enable/disable TLS listener **Default value:** `false`
`tls_ingress_cidr_blocks` (`list(string)`) optional
List of CIDR blocks to allow in TLS security group **Default value:** ```hcl [ "0.0.0.0/0" ] ```
`tls_ingress_prefix_list_ids` (`list(string)`) optional
List of prefix list IDs for allowing access to TLS ingress security group **Default value:** `[ ]`
`tls_port` (`number`) optional
The port for the TLS listener **Default value:** `443`
`tls_ssl_policy` (`string`) optional
The name of the SSL Policy for the listener **Default value:** `"ELBSecurityPolicy-2016-08"`
`udp_enabled` (`bool`) optional
A boolean flag to enable/disable UDP listener **Default value:** `false`
`udp_port` (`number`) optional
The port for the UDP listener **Default value:** `53`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_logs_bucket_id`
The S3 bucket ID for access logs
`default_listener_arn`
The ARN of the default listener
`default_target_group_arn`
The default target group ARN
`listener_arns`
A list of all the listener ARNs
`nlb_arn`
The ARN of the NLB
`nlb_arn_suffix`
The ARN suffix of the NLB
`nlb_dns_name`
DNS name of NLB
`nlb_name`
The ARN suffix of the NLB
`nlb_zone_id`
The ID of the zone which NLB is provisioned
`security_group_id`
The security group ID of the NLB
`tls_listener_arn`
The ARN of the TLS listener
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2.0` - `local`, version: `>= 1.3` - `null`, version: `>= 2.0` - `template`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `access_logs` | 0.16.4 | [`cloudposse/lb-s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/lb-s3-bucket/aws/0.16.4) | n/a `default_target_group_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `eip_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `lb_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.lb`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_lb.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb) (resource) - [`aws_lb_listener.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) (resource) - [`aws_lb_listener.tls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener) (resource) - [`aws_lb_listener_certificate.https_sni`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_listener_certificate) (resource) - [`aws_lb_target_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lb_target_group) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.default_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.tls_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: --- ## organization-access-group # Module: `organization-access-group` Terraform module to create an IAM Group and Policy to grant permissions to delegated IAM users in the Organization's master account to access a member account https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_access.html ## Introduction By default, when you create a member account as part of your Organization, AWS automatically creates `OrganizationAccountAccessRole` in the member account. The role grants admin permissions to access the member account to delegated IAM users in the master account. In the master account you need to create a Policy to grant permissions to IAM users to assume `OrganizationAccountAccessRole` in the member account. This module does the following: 1. Creates an IAM Group 2. Adds the provided IAM users to the Group 3. Creates a Policy to grant permissions to the IAM users in the master account to assume `OrganizationAccountAccessRole` in the member account 4. Attaches the Policy to the Group Users who are members of the Group will be able to assume the role and administer the member account by going here: (change `XXXXXXXXXXXX` to the ID of the member account) ``` https://signin.aws.amazon.com/switchrole ?account=XXXXXXXXXXXX &roleName=OrganizationAccountAccessRole &displayName=Dev ``` __NOTE__: Member accounts that you invite to join your Organization (that are not part of your Organization) do not automatically get `OrganizationAccountAccessRole` created. You can use [terraform-aws-organization-access-role](https://github.com/cloudposse/terraform-aws-organization-access-role) module to create `OrganizationAccountAccessRole` role in an invited member account. ## Usage ```hcl module "organization_access_group" { source = "git::https://github.com/cloudposse/terraform-aws-organization-access-group.git?ref=master" namespace = "cp" stage = "dev" name = "cluster" user_names = ["User1","User2"] role_arns = { "cp@dev" = "arn:aws:iam::XXXXXXXXX:role/OrganizationAccountAccessRole" } require_mfa = "true" } ``` ## Variables ### Required Variables
`user_names` (`list(string)`) required
A list of IAM User names to associate with the Group
### Optional Variables
`require_mfa` (`string`) optional
Require the users to have MFA enabled **Default value:** `"false"`
`role_arns` (`map(string)`) optional
A map of alias -> IAM Role ARNs the users in the Group can assume **Default value:** `{ }`
`switchrole_url_template` (`string`) optional
URL template for the IAM console to switch to the roles **Default value:** `"https://signin.aws.amazon.com/switchrole?account=%s&roleName=%s&displayName=%s"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`string`) required
Name (e.g. `app` or `cluster`) **Required:** Yes **Default value:** ``
`namespace` (`string`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`, `infra`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `stage`, `name`, and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Whether to create these resources **Required:** No **Default value:** `"true"`
`tags` (`map(string)`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`group_arn`
The ARN assigned by AWS for the Group
`group_id`
The Group's ID
`group_name`
The Group's name
`group_unique_id`
Group's unique ID assigned by AWS
`policy_id`
The policy ID
`policy_name`
The name of the policy
`switchrole_urls`
List of URL to the IAM console to switch to the roles
## Dependencies ### Providers - `aws` - `null` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | tags/0.3.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.3) | n/a ## Resources The following resources are used by this module: - [`aws_iam_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group) (resource) - [`aws_iam_group_policy.with_mfa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy) (resource) - [`aws_iam_group_policy.without_mfa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy) (resource) - [`aws_iam_user_group_membership.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_group_membership) (resource) - [`null_resource.role`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.with_mfa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.without_mfa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## organization-access-role # Module: `organization-access-role` Terraform module to create an IAM Role to grant permissions to delegated IAM users in the master account to access an invited member account https://docs.aws.amazon.com/organizations/latest/userguide/orgs_manage_accounts_access.html ## Screenshots ![OrganizationAccountAccessRole](images/OrganizationAccountAccessRole.png) *Organization Account Access Role as Viewed from the AWS Web Console* ## Introduction By default, when you create a member account as part of your Organization, AWS automatically creates `OrganizationAccountAccessRole` in the member account. The role grants admin permissions to the member account to delegated IAM users in the master account. However, member accounts that you invite to join your Organization do not automatically get the role created. This module creates `OrganizationAccountAccessRole` role in an invited member account. AWS recommends using the same name, `OrganizationAccountAccessRole`, for the created role for consistency and ease of remembering. ## Usage ```hcl module "organization_access_role" { source = "git::https://github.com/cloudposse/terraform-aws-organization-access-role.git?ref=master" master_account_id = "XXXXXXXXXXXX" role_name = "OrganizationAccountAccessRole" policy_arn = "arn:aws:iam::aws:policy/AdministratorAccess" ``` After the role has been created in the invited member account, login to the master account and create the following policy: (change `YYYYYYYYYYYY` to the ID of the invited member account) ```json { "Version": "2012-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Action": [ "sts:AssumeRole" ], "Resource": [ "arn:aws:iam::YYYYYYYYYYYY:role/OrganizationAccountAccessRole" ] } ] } ``` Then attach the policy to a master account Group that you want to delegate administration of the invited member account. After that, users who are members of the Group in the master account will be able to assume the role and administer the invited member account by going here: (change `YYYYYYYYYYYY` to the ID of the invited member account) ``` https://signin.aws.amazon.com/switchrole ?account=YYYYYYYYYYYY &roleName=OrganizationAccountAccessRole &displayName=Dev ``` __NOTE__: You can use [terraform-aws-organization-access-group](https://github.com/cloudposse/terraform-aws-organization-access-group) module to create an IAM Group and Policy to grant permissions to delegated IAM users in the Organization's master account to access a member account. ## Variables ### Required Variables
`master_account_id` (`string`) required
The ID of the master account to grant permissions to access the current account
### Optional Variables
`policy_arn` (`string`) optional
Policy ARN to attach to the role. By default it attaches `AdministratorAccess` managed policy to grant full access to AWS services and resources in the current account **Default value:** `"arn:aws:iam::aws:policy/AdministratorAccess"`
`role_name` (`string`) optional
The name of the role to grant permissions to delegated IAM users in the master account to the current account **Default value:** `"OrganizationAccountAccessRole"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Outputs
`role_arn`
The Amazon Resource Name (ARN) specifying the role
`role_id`
The stable and unique string identifying the role
`role_name`
The name of the crated role
## Dependencies ### Providers - `aws` ## Resources The following resources are used by this module: - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## rds(Rds) # Module: `rds` Terraform module to provision AWS [`RDS`](https://aws.amazon.com/rds/) instances ## Introduction The module will create: * DB instance (MySQL, Postgres, SQL Server, Oracle) * DB Option Group (will create a new one or you may use an existing) * DB Parameter Group * DB Subnet Group * DB Security Group * DNS Record in Route53 for the DB endpoint ## Usage ```hcl module "rds_instance" { source = "cloudposse/rds/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" dns_zone_id = "Z89FN1IW975KPE" host_name = "db" security_group_ids = ["sg-xxxxxxxx"] ca_cert_identifier = "rds-ca-2019" allowed_cidr_blocks = ["XXX.XXX.XXX.XXX/32"] database_name = "wordpress" database_user = "admin" database_password = "xxxxxxxxxxxx" database_port = 3306 multi_az = true storage_type = "gp2" allocated_storage = 100 storage_encrypted = true engine = "mysql" engine_version = "5.7.17" major_engine_version = "5.7" instance_class = "db.t2.medium" db_parameter_group = "mysql5.7" option_group_name = "mysql-options" publicly_accessible = false subnet_ids = ["sb-xxxxxxxxx", "sb-xxxxxxxxx"] vpc_id = "vpc-xxxxxxxx" snapshot_identifier = "rds:production-2015-06-26-06-05" auto_minor_version_upgrade = true allow_major_version_upgrade = false apply_immediately = false maintenance_window = "Mon:03:00-Mon:04:00" skip_final_snapshot = false copy_tags_to_snapshot = true backup_retention_period = 7 backup_window = "22:00-03:00" db_parameter = [ { name = "myisam_sort_buffer_size" value = "1048576" }, { name = "sort_buffer_size" value = "2097152" } ] db_options = [ { option_name = "MARIADB_AUDIT_PLUGIN" option_settings = [ { name = "SERVER_AUDIT_EVENTS" value = "CONNECT" }, { name = "SERVER_AUDIT_FILE_ROTATIONS" value = "37" } ] } ] } ``` ### Character Sets If you wish to create the database in a specific character set you can use one of the following options depending on your database engine of choice. For Oracle and Microsoft SQL you can specify charset name as an input variable to this module. For example, for Microsoft SQL, you could use: ```hcl module "rds_instance" { ... charset_name = "Korean_Wansung_CI_AS" ... } ``` For `mysql` and `mariadb` engines character set of the database can be defined via `db_parameter`. In this example the database is created with `utf8mb4` (character set) and utf8mb4_unicode_ci (collation): ```hcl module "rds_instance" { ... db_parameter = [ { name = "character_set_server" value = "utf8mb4" apply_method = "immediate" }, { name = "collation_server" value = "utf8mb4_unicode_ci" apply_method = "immediate" } ] ... } ``` ## Variables ### Required Variables
`database_port` (`number`) required
Database port (_e.g._ `3306` for `MySQL`). Used in the DB Security Group to allow access to the DB instance from the provided `security_group_ids`
`db_parameter_group` (`string`) required
The DB parameter group family name. The value depends on DB engine used. See [DBParameterGroupFamily](https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_CreateDBParameterGroup.html#API_CreateDBParameterGroup_RequestParameters) for instructions on how to retrieve applicable value.
`engine_version` (`string`) required
Database engine version, depends on engine type.
`instance_class` (`string`) required
Class of RDS instance
`vpc_id` (`string`) required
VPC ID the DB instance will be created in
### Optional Variables
`allocated_storage` (`number`) optional
The allocated storage in GBs. Required unless a `snapshot_identifier` or `replicate_source_db` is provided. **Default value:** `null`
`allow_major_version_upgrade` (`bool`) optional
Allow major version upgrade **Default value:** `false`
`allowed_cidr_blocks` (`list(string)`) optional
The whitelisted CIDRs which to allow `ingress` traffic to the DB instance **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any database modifications are applied immediately, or during the next maintenance window **Default value:** `false`
`associate_security_group_ids` (`list(string)`) optional
The IDs of the existing security groups to associate with the DB instance **Default value:** `[ ]`
`auto_minor_version_upgrade` (`bool`) optional
Allow automated minor version upgrade (e.g. from Postgres 9.5.3 to Postgres 9.5.4) **Default value:** `true`
`availability_zone` (`string`) optional
The AZ for the RDS instance. Specify one of `subnet_ids`, `db_subnet_group_name` or `availability_zone`. If `availability_zone` is provided, the instance will be placed into the default VPC or EC2 Classic **Default value:** `null`
`backup_retention_period` (`number`) optional
Backup retention period in days. Must be > 0 to enable backups **Default value:** `0`
`backup_window` (`string`) optional
When AWS can perform DB snapshots, can't overlap with maintenance window **Default value:** `"22:00-03:00"`
`ca_cert_identifier` (`string`) optional
The identifier of the CA certificate for the DB instance **Default value:** `null`
`charset_name` (`string`) optional
The character set name to use for DB encoding. [Oracle & Microsoft SQL only](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance#character_set_name). For other engines use `db_parameter` **Default value:** `null`
`copy_tags_to_snapshot` (`bool`) optional
Copy tags from DB to a snapshot **Default value:** `true`
`database_manage_master_user_password` (`bool`) optional
Set to true to allow RDS to manage the master user password in Secrets Manager. Ignore if `database_password` is provided. **Default value:** `false`
`database_master_user_secret_kms_key_id` (`string`) optional
The Amazon Web Services KMS key identifier is the key ARN, key ID, alias ARN, or alias name for the KMS key. To use a KMS key in a different Amazon Web Services account, specify the key ARN or alias ARN. If not specified, the default KMS key for your Amazon Web Services account is used. **Default value:** `null`
`database_name` (`string`) optional
The name of the database to create when the DB instance is created **Default value:** `null`
`database_password` (`string`) optional
Password for the primary DB user. Required unless a `snapshot_identifier` or `replicate_source_db` is provided. **Default value:** `null`
`database_user` (`string`) optional
Username for the primary DB user. Required unless a `snapshot_identifier` or `replicate_source_db` is provided. **Default value:** `null`
`db_options` optional
A list of DB options to apply with an option group. Depends on DB engine **Type:** ```hcl list(object({ db_security_group_memberships = list(string) option_name = string port = number version = string vpc_security_group_memberships = list(string) option_settings = list(object({ name = string value = string })) })) ``` **Default value:** `[ ]`
`db_parameter` optional
A list of DB parameters to apply. Note that parameters may differ from a DB family to another **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`db_subnet_group_name` (`string`) optional
Name of DB subnet group. DB instance will be created in the VPC associated with the DB subnet group. Specify one of `subnet_ids`, `db_subnet_group_name` or `availability_zone` **Default value:** `null`
`deletion_protection` (`bool`) optional
Set to true to enable deletion protection on the RDS instance **Default value:** `false`
`dns_zone_id` (`string`) optional
The ID of the DNS Zone in Route53 where a new DNS record will be created for the DB host name **Default value:** `""`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to enable for exporting to CloudWatch logs. If omitted, no logs will be exported. Valid values (depending on engine): alert, audit, error, general, listener, slowquery, trace, postgresql (PostgreSQL), upgrade (PostgreSQL). **Default value:** `[ ]`
`engine` (`string`) optional
Database engine type. Required unless a `snapshot_identifier` or `replicate_source_db` is provided. **Default value:** `null`
`final_snapshot_identifier` (`string`) optional
Final snapshot identifier e.g.: some-db-final-snapshot-2019-06-26-06-05 **Default value:** `""`
`host_name` (`string`) optional
The DB host name created in Route53 **Default value:** `"db"`
`iam_database_authentication_enabled` (`bool`) optional
Specifies whether or mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled **Default value:** `false`
`iops` (`number`) optional
The amount of provisioned IOPS, only valid for certain values of storage_type. **Default value:** `null`
`kms_key_arn` (`string`) optional
The ARN of the existing KMS key to encrypt storage **Default value:** `""`
`license_model` (`string`) optional
License model for this DB. Optional, but required for some DB Engines. Valid values: license-included | bring-your-own-license | general-public-license **Default value:** `""`
`maintenance_window` (`string`) optional
The window to perform maintenance in. Syntax: 'ddd:hh24:mi-ddd:hh24:mi' UTC **Default value:** `"Mon:03:00-Mon:04:00"`
`major_engine_version` (`string`) optional
Database MAJOR engine version, depends on engine type **Default value:** `""`
`max_allocated_storage` (`number`) optional
The upper limit to which RDS can automatically scale the storage in GBs **Default value:** `0`
`monitoring_interval` (`string`) optional
The interval, in seconds, between points when Enhanced Monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. Valid Values are 0, 1, 5, 10, 15, 30, 60. **Default value:** `"0"`
`monitoring_role_arn` (`string`) optional
The ARN for the IAM role that permits RDS to send enhanced monitoring metrics to CloudWatch Logs **Default value:** `null`
`multi_az` (`bool`) optional
Set to true if multi AZ deployment must be supported **Default value:** `false`
`option_group_name` (`string`) optional
Name of the DB option group to associate **Default value:** `""`
`parameter_group_name` (`string`) optional
Name of the DB parameter group to associate **Default value:** `""`
`performance_insights_enabled` (`bool`) optional
Specifies whether Performance Insights are enabled. **Default value:** `false`
`performance_insights_kms_key_id` (`string`) optional
The ARN for the KMS key to encrypt Performance Insights data. Once KMS key is set, it can never be changed. **Default value:** `null`
`performance_insights_retention_period` (`number`) optional
The amount of time in days to retain Performance Insights data. Either 7 (7 days) or 731 (2 years). **Default value:** `7`
`publicly_accessible` (`bool`) optional
Determines if database can be publicly available (NOT recommended) **Default value:** `false`
`replicate_source_db` (`string`) optional
Specifies that this resource is a Replicate database, and to use this value as the source database. This correlates to the `identifier` of another Amazon RDS Database to replicate (if replicating within a single region) or ARN of the Amazon RDS Database to replicate (if replicating cross-region). Note that if you are creating a cross-region replica of an encrypted database you will also need to specify a `kms_key_id`. See [DB Instance Replication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Replication.html) and [Working with PostgreSQL and MySQL Read Replicas](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html) for more information on using Replication. **Default value:** `null`
`restore_to_point_in_time` optional
An object specifying the restore point in time for the DB instance to restore from. Only used when `snapshot_identifier` is not provided. **Type:** ```hcl object({ restore_time = optional(string, null) source_db_instance_identifier = optional(string, null) source_db_instance_automated_backups_arn = optional(string, null) source_dbi_resource_id = optional(string, null) use_latest_restorable_time = optional(bool, null) }) ``` **Default value:** `null`
`security_group_ids` (`list(string)`) optional
The IDs of the security groups from which to allow `ingress` traffic to the DB instance **Default value:** `[ ]`
`skip_final_snapshot` (`bool`) optional
If true (default), no snapshot will be made before deleting DB **Default value:** `true`
`snapshot_identifier` (`string`) optional
Snapshot identifier e.g: `rds:production-2019-06-26-06-05` for automated or `manual-backup-2023-11-16` for manual. If specified, the module create the instance from the snapshot. **Default value:** `null`
`storage_encrypted` (`bool`) optional
(Optional) Specifies whether the DB instance is encrypted. The default is false if not specified **Default value:** `true`
`storage_throughput` (`number`) optional
The storage throughput value for the DB instance. Can only be set when `storage_type` is `gp3`. Cannot be specified if the `allocated_storage` value is below a per-engine threshold. **Default value:** `null`
`storage_type` (`string`) optional
One of 'standard' (magnetic), 'gp2' (general purpose SSD), 'gp3' (general purpose SSD), or 'io1' (provisioned IOPS SSD) **Default value:** `"standard"`
`subnet_ids` (`list(string)`) optional
List of subnet IDs for the DB. DB instance will be created in the VPC associated with the DB subnet group provisioned using the subnet IDs. Specify one of `subnet_ids`, `db_subnet_group_name` or `availability_zone` **Default value:** `[ ]`
`timeouts` optional
A list of DB timeouts to apply to the running code while creating, updating, or deleting the DB instance. **Type:** ```hcl object({ create = string update = string delete = string }) ``` **Default value:** ```hcl { "create": "40m", "delete": "60m", "update": "80m" } ```
`timezone` (`string`) optional
Time zone of the DB instance. timezone is currently only supported by Microsoft SQL Server. The timezone can only be set on creation. See [MSSQL User Guide](http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SQLServer.html#SQLServer.Concepts.General.TimeZone) for more information. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`hostname`
DNS host name of the instance
`instance_address`
Address of the instance
`instance_arn`
ARN of the instance
`instance_endpoint`
DNS Endpoint of the instance
`instance_id`
ID of the instance
`master_user_secret`
Secret object if configured with `var.database_manage_master_user_password = true`.
`option_group_id`
ID of the Option Group
`parameter_group_id`
ID of the Parameter Group
`resource_id`
The RDS Resource ID of this instance.
`security_group_id`
ID of the Security Group
`subnet_group_id`
ID of the created Subnet Group
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_host_name` | 0.12.2 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.12.2) | n/a `final_snapshot_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_db_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance) (resource) - [`aws_db_option_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_option_group) (resource) - [`aws_db_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_parameter_group) (resource) - [`aws_db_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_subnet_group) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: --- ## rds-cloudwatch-sns-alarms # Module: `rds-cloudwatch-sns-alarms` Terraform module that configures important RDS alerts using CloudWatch and sends them to an SNS topic. Create a set of sane RDS CloudWatch alerts for monitoring the health of an RDS instance. ## Usage | area | metric | comparison operator | threshold | rationale | |---------|------------------|----------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | Storage | BurstBalance | `<` | 20 % | 20 % of credits allow you to burst for a few minutes which gives you enough time to a) fix the inefficiency, b) add capacity or c) switch to io1 storage type. | | Storage | DiskQueueDepth | `>` | 64 | This number is calculated from our experience with RDS workloads. | | Storage | FreeStorageSpace | `<` | 2 GB | 2 GB usually provides enough time to a) fix why so much space is consumed or b) add capacity. You can also modify this value to 10% of your database capacity. | | CPU | CPUUtilization | `>` | 80 % | Queuing theory tells us the latency increases exponentially with utilization. In practice, we see higher latency when utilization exceeds 80% and unacceptable high latency with utilization above 90% | | CPU | CPUCreditBalance | `<` | 20 | One credit equals 1 minute of 100% usage of a vCPU. 20 credits should give you enough time to a) fix the inefficiency, b) add capacity or c) don't use t2 type. | | Memory | FreeableMemory | `<` | 64 MB | This number is calculated from our experience with RDS workloads. | | Memory | SwapUsage | `>` | 256 MB | Sometimes you can not entirely avoid swapping. But once the database accesses paged memory, it will slow down. | ## Examples See the [`examples/`](https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms/tree/main/examples/) directory for working examples. ```hcl resource "aws_db_instance" "default" { allocated_storage = 10 storage_type = "gp2" engine = "mysql" engine_version = "5.7" instance_class = "db.t2.micro" identifier_prefix = "rds-server-example" name = "mydb" username = "foo" password = "foobarbaz" parameter_group_name = "default.mysql5.7" apply_immediately = "true" skip_final_snapshot = "true" } module "rds_alarms" { source = "git::https://github.com/cloudposse/terraform-aws-rds-cloudwatch-sns-alarms.git?ref=tags/0.1.5" db_instance_id = "${aws_db_instance.default.id}" } ``` ## Variables ### Required Variables
`db_instance_id` (`string`) required
The instance ID of the RDS database instance that you want to monitor.
### Optional Variables
`burst_balance_threshold` (`number`) optional
The minimum percent of General Purpose SSD (gp2) burst-bucket I/O credits available. **Default value:** `20`
`cpu_credit_balance_threshold` (`number`) optional
The minimum number of CPU credits (t2 instances only) available. **Default value:** `20`
`cpu_utilization_threshold` (`number`) optional
The maximum percentage of CPU utilization. **Default value:** `80`
`disk_queue_depth_threshold` (`number`) optional
The maximum number of outstanding IOs (read/write requests) waiting to access the disk. **Default value:** `64`
`free_storage_space_threshold` (`number`) optional
The minimum amount of available storage space in Byte. **Default value:** `2000000000`
`freeable_memory_threshold` (`number`) optional
The minimum amount of available random access memory in Byte. **Default value:** `64000000`
`swap_usage_threshold` (`number`) optional
The maximum amount of swap space used on the DB instance in Byte. **Default value:** `256000000`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`sns_topic_arn`
The ARN of the SNS topic
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `subscription_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `topic_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.burst_balance_too_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.cpu_credit_balance_too_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.cpu_utilization_too_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.disk_queue_depth_too_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.free_storage_space_too_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.freeable_memory_too_low`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_cloudwatch_metric_alarm.swap_usage_too_high`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) - [`aws_db_event_subscription.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_event_subscription) (resource) - [`aws_sns_topic.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) (resource) - [`aws_sns_topic_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sns_topic_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## rds-cluster # Module: `rds-cluster` Terraform module to provision an [`RDS Aurora`](https://aws.amazon.com/rds/aurora) cluster for MySQL or Postgres. Supports [Amazon Aurora Serverless](https://aws.amazon.com/rds/aurora/serverless/). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/test). [Basic example](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/basic) ```hcl module "rds_cluster_aurora_postgres" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "postgres" engine = "aurora-postgresql" cluster_family = "aurora-postgresql9.6" # 1 writer, 1 reader cluster_size = 2 # 1 writer, 3 reader # cluster_size = 4 # 1 writer, 5 reader # cluster_size = 6 namespace = "eg" stage = "dev" admin_user = "admin1" admin_password = "Test123456789" db_name = "dbname" db_port = 5432 instance_type = "db.r4.large" vpc_id = "vpc-xxxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" } ``` [Serverless Aurora MySQL 5.6](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/serverless_mysql) ```hcl module "rds_cluster_aurora_mysql_serverless" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "db" engine = "aurora" engine_mode = "serverless" cluster_family = "aurora5.6" cluster_size = 0 admin_user = "admin1" admin_password = "Test123456789" db_name = "dbname" db_port = 3306 instance_type = "db.t2.small" vpc_id = "vpc-xxxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" enable_http_endpoint = true scaling_configuration = [ { auto_pause = true max_capacity = 256 min_capacity = 2 seconds_until_auto_pause = 300 } ] } ``` [Serverless Aurora 2.07.1 MySQL 5.7](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/serverless_mysql57) ```hcl module "rds_cluster_aurora_mysql_serverless" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "db" engine = "aurora-mysql" engine_mode = "serverless" engine_version = "5.7.mysql_aurora.2.07.1" cluster_family = "aurora-mysql5.7" cluster_size = 0 admin_user = "admin1" admin_password = "Test123456789" db_name = "dbname" db_port = 3306 vpc_id = "vpc-xxxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" enable_http_endpoint = true scaling_configuration = [ { auto_pause = true max_capacity = 16 min_capacity = 1 seconds_until_auto_pause = 300 timeout_action = "ForceApplyCapacityChange" } ] } ``` [With cluster parameters](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/with_cluster_parameters) ```hcl module "rds_cluster_aurora_mysql" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora" cluster_family = "aurora-mysql5.7" cluster_size = 2 namespace = "eg" stage = "dev" name = "db" admin_user = "admin1" admin_password = "Test123456789" db_name = "dbname" instance_type = "db.t2.small" vpc_id = "vpc-xxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" cluster_parameters = [ { name = "character_set_client" value = "utf8" }, { name = "character_set_connection" value = "utf8" }, { name = "character_set_database" value = "utf8" }, { name = "character_set_results" value = "utf8" }, { name = "character_set_server" value = "utf8" }, { name = "collation_connection" value = "utf8_bin" }, { name = "collation_server" value = "utf8_bin" }, { name = "lower_case_table_names" value = "1" apply_method = "pending-reboot" }, { name = "skip-character-set-client-handshake" value = "1" apply_method = "pending-reboot" } ] } ``` [With enhanced monitoring](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/enhanced_monitoring) ```hcl # create IAM role for monitoring resource "aws_iam_role" "enhanced_monitoring" { name = "rds-cluster-example-1" assume_role_policy = data.aws_iam_policy_document.enhanced_monitoring.json } # Attach Amazon's managed policy for RDS enhanced monitoring resource "aws_iam_role_policy_attachment" "enhanced_monitoring" { role = aws_iam_role.enhanced_monitoring.name policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonRDSEnhancedMonitoringRole" } # allow rds to assume this role data "aws_iam_policy_document" "enhanced_monitoring" { statement { actions = [ "sts:AssumeRole", ] effect = "Allow" principals { type = "Service" identifiers = ["monitoring.rds.amazonaws.com"] } } } module "rds_cluster_aurora_postgres" { source = "cloudposse/rds-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" engine = "aurora-postgresql" cluster_family = "aurora-postgresql9.6" cluster_size = 2 namespace = "eg" stage = "dev" name = "db" admin_user = "admin1" admin_password = "Test123456789" db_name = "dbname" db_port = 5432 instance_type = "db.r4.large" vpc_id = "vpc-xxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" # enable monitoring every 30 seconds rds_monitoring_interval = 30 # reference iam role created above rds_monitoring_role_arn = aws_iam_role.enhanced_monitoring.arn } ``` ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-rds-cluster/tree/main/examples/complete) to see how to use this module. ## Variables ### Required Variables
`subnets` (`list(string)`) required
List of VPC subnet IDs
`vpc_id` (`string`) required
VPC ID to create the cluster in (e.g. `vpc-a22222ee`)
### Optional Variables
`activity_stream_enabled` (`bool`) optional
Whether to enable Activity Streams **Default value:** `false`
`activity_stream_kms_key_id` (`string`) optional
The ARN for the KMS key to encrypt Activity Stream Data data. When specifying `activity_stream_kms_key_id`, `activity_stream_enabled` needs to be set to true **Default value:** `""`
`activity_stream_mode` (`string`) optional
The mode for the Activity Streams. `async` and `sync` are supported. Defaults to `async` **Default value:** `"async"`
`admin_password` (`string`) optional
Password for the master DB user. Ignored if snapshot_identifier or replication_source_identifier is provided **Default value:** `""`
`admin_user` (`string`) optional
Username for the master DB user. Ignored if snapshot_identifier or replication_source_identifier is provided **Default value:** `"admin"`
`admin_user_secret_kms_key_id` (`string`) optional
Amazon Web Services KMS key identifier is the key ARN, key ID, alias ARN, or alias name for the KMS key. To use a KMS key in a different Amazon Web Services account, specify the key ARN or alias ARN. If not specified, the default KMS key for your Amazon Web Services account is used. **Default value:** `null`
`allocated_storage` (`number`) optional
The allocated storage in GBs **Default value:** `null`
`allow_major_version_upgrade` (`bool`) optional
Enable to allow major engine version upgrades when changing engine versions. Defaults to false. **Default value:** `false`
`allowed_cidr_blocks` (`list(string)`) optional
List of CIDR blocks allowed to access the cluster **Default value:** `[ ]`
`allowed_ipv6_cidr_blocks` (`list(string)`) optional
List of IPv6 CIDR blocks allowed to access the cluster **Default value:** `[ ]`
`apply_immediately` (`bool`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `true`
`auto_minor_version_upgrade` (`bool`) optional
Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window **Default value:** `true`
`autoscaling_enabled` (`bool`) optional
Whether to enable cluster autoscaling **Default value:** `false`
`autoscaling_max_capacity` (`number`) optional
Maximum number of instances to be maintained by the autoscaler **Default value:** `5`
`autoscaling_min_capacity` (`number`) optional
Minimum number of instances to be maintained by the autoscaler **Default value:** `1`
`autoscaling_policy_type` (`string`) optional
Autoscaling policy type. `TargetTrackingScaling` and `StepScaling` are supported **Default value:** `"TargetTrackingScaling"`
`autoscaling_scale_in_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling down activity can start. Default is 300s **Default value:** `300`
`autoscaling_scale_out_cooldown` (`number`) optional
The amount of time, in seconds, after a scaling activity completes and before the next scaling up activity can start. Default is 300s **Default value:** `300`
`autoscaling_target_metrics` (`string`) optional
The metrics type to use. If this value isn't provided the default is CPU utilization **Default value:** `"RDSReaderAverageCPUUtilization"`
`autoscaling_target_value` (`number`) optional
The target value to scale with respect to target metrics **Default value:** `75`
`backtrack_window` (`number`) optional
The target backtrack window, in seconds. Only available for aurora engine currently. Must be between 0 and 259200 (72 hours) **Default value:** `0`
`backup_window` (`string`) optional
Daily time range during which the backups happen **Default value:** `"07:00-09:00"`
`ca_cert_identifier` (`string`) optional
The identifier of the CA certificate for the DB instance **Default value:** `null`
`cluster_dns_name` (`string`) optional
Name of the cluster CNAME record to create in the parent DNS zone specified by `zone_id`. If left empty, the name will be auto-asigned using the format `master.var.name` **Default value:** `""`
`cluster_family` (`string`) optional
The family of the DB cluster parameter group **Default value:** `"aurora5.6"`
`cluster_identifier` (`string`) optional
The RDS Cluster Identifier. Will use generated label ID if not supplied **Default value:** `""`
`cluster_parameters` optional
List of DB cluster parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`cluster_size` (`number`) optional
Number of DB instances to create in the cluster **Default value:** `2`
`cluster_type` (`string`) optional
Either `regional` or `global`. If `regional` will be created as a normal, standalone DB. If `global`, will be made part of a Global cluster (requires `global_cluster_identifier`). **Default value:** `"regional"`
`copy_tags_to_snapshot` (`bool`) optional
Copy tags to backup snapshots **Default value:** `false`
`db_cluster_instance_class` (`string`) optional
This setting is required to create a provisioned Multi-AZ DB cluster **Default value:** `null`
`db_name` (`string`) optional
Database name (default is not to create a database) **Default value:** `""`
`db_parameter_group_name` (`string`) optional
The name to give to the created `aws_db_parameter_group` resource. If omitted, the module will generate a name. **Default value:** `""`
`db_port` (`number`) optional
Database port **Default value:** `3306`
`deletion_protection` (`bool`) optional
If the DB instance should have deletion protection enabled **Default value:** `false`
`egress_enabled` (`bool`) optional
Whether or not to apply the egress security group rule to default security group, defaults to `true` **Default value:** `true`
`enable_global_write_forwarding` (`bool`) optional
Set to `true`, to forward writes to an associated global cluster. **Default value:** `null`
`enable_http_endpoint` (`bool`) optional
Enable HTTP endpoint (data API). Only valid when engine_mode is set to serverless **Default value:** `false`
`enable_local_write_forwarding` (`bool`) optional
Set to `true`, to forward writes sent to a reader to the writer instance. **Default value:** `null`
`enabled_cloudwatch_logs_exports` (`list(string)`) optional
List of log types to export to cloudwatch. The following log types are supported: audit, error, general, slowquery **Default value:** `[ ]`
`engine` (`string`) optional
The name of the database engine to be used for this DB cluster. Valid values: `aurora`, `aurora-mysql`, `aurora-postgresql` **Default value:** `"aurora"`
`engine_mode` (`string`) optional
The database engine mode. Valid values: `parallelquery`, `provisioned`, `serverless` **Default value:** `"provisioned"`
`engine_version` (`string`) optional
The version of the database engine to use. See `aws rds describe-db-engine-versions` **Default value:** `""`
`enhanced_monitoring_attributes` (`list(string)`) optional
The attributes for the enhanced monitoring IAM role **Default value:** ```hcl [ "enhanced-monitoring" ] ```
`enhanced_monitoring_role_enabled` (`bool`) optional
A boolean flag to enable/disable the creation of the enhanced monitoring IAM role. If set to `false`, the module will not create a new role and will use `rds_monitoring_role_arn` for enhanced monitoring **Default value:** `false`
`global_cluster_identifier` (`string`) optional
ID of the Aurora global cluster **Default value:** `""`
`iam_database_authentication_enabled` (`bool`) optional
Specifies whether or mappings of AWS Identity and Access Management (IAM) accounts to database accounts is enabled **Default value:** `false`
`iam_roles` (`list(string)`) optional
Iam roles for the Aurora cluster **Default value:** `[ ]`
`instance_availability_zone` (`string`) optional
Optional parameter to place cluster instances in a specific availability zone. If left empty, will place randomly **Default value:** `""`
`instance_parameters` optional
List of DB instance parameters to apply **Type:** ```hcl list(object({ apply_method = string name = string value = string })) ``` **Default value:** `[ ]`
`instance_type` (`string`) optional
Instance type to use **Default value:** `"db.t2.small"`
`intra_security_group_traffic_enabled` (`bool`) optional
Whether to allow traffic between resources inside the database's security group. **Default value:** `false`
`iops` (`number`) optional
The amount of provisioned IOPS. Setting this implies a storage_type of 'io1'. This setting is required to create a Multi-AZ DB cluster. Check TF docs for values based on db engine **Default value:** `null`
`kms_key_arn` (`string`) optional
The ARN for the KMS encryption key. When specifying `kms_key_arn`, `storage_encrypted` needs to be set to `true` **Default value:** `""`
`maintenance_window` (`string`) optional
Weekly time range during which system maintenance can occur, in UTC **Default value:** `"wed:03:00-wed:04:00"`
`manage_admin_user_password` (`bool`) optional
Set to true to allow RDS to manage the master user password in Secrets Manager. Cannot be set if master_password is provided **Default value:** `false`
`network_type` (`string`) optional
The network type of the cluster. Valid values: IPV4, DUAL. **Default value:** `"IPV4"`
`parameter_group_name_prefix_enabled` (`bool`) optional
Set to `true` to use `name_prefix` to name the cluster and database parameter groups. Set to `false` to use `name` instead **Default value:** `true`
`performance_insights_enabled` (`bool`) optional
Whether to enable Performance Insights **Default value:** `false`
`performance_insights_kms_key_id` (`string`) optional
The ARN for the KMS key to encrypt Performance Insights data. When specifying `performance_insights_kms_key_id`, `performance_insights_enabled` needs to be set to true **Default value:** `""`
`performance_insights_retention_period` (`number`) optional
Amount of time in days to retain Performance Insights data. Either 7 (7 days) or 731 (2 years) **Default value:** `null`
`promotion_tier` (`number`) optional
Failover Priority setting on instance level. The reader who has lower tier has higher priority to get promoted to writer. Readers in promotion tiers 0 and 1 scale at the same time as the writer. Readers in promotion tiers 2–15 scale independently from the writer. For more information, see: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/aurora-serverless-v2.how-it-works.html#aurora-serverless-v2.how-it-works.scaling **Default value:** `0`
`publicly_accessible` (`bool`) optional
Set to true if you want your cluster to be publicly accessible (such as via QuickSight) **Default value:** `false`
`rds_cluster_parameter_group_name` (`string`) optional
The name to give to the created `aws_rds_cluster_parameter_group` resource. If omitted, the module will generate a name. **Default value:** `""`
`rds_monitoring_interval` (`number`) optional
The interval, in seconds, between points when enhanced monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. The default is 0. Valid Values: 0, 1, 5, 10, 15, 30, 60 **Default value:** `0`
`rds_monitoring_role_arn` (`string`) optional
The ARN for the IAM role that permits RDS to send enhanced monitoring metrics to CloudWatch Logs **Default value:** `null`
`rds_ri_duration` (`number`) optional
The number of years to reserve the instance. Values can be 1 or 3 (or in seconds, 31536000 or 94608000) **Default value:** `1`
`rds_ri_offering_type` (`string`) optional
Offering type of reserved DB instances. Valid values are 'No Upfront', 'Partial Upfront', 'All Upfront'. **Default value:** `""`
`rds_ri_reservation_id` (`string`) optional
Customer-specified identifier to track the reservation of the reserved DB instance. **Default value:** `null`
`reader_dns_name` (`string`) optional
Name of the reader endpoint CNAME record to create in the parent DNS zone specified by `zone_id`. If left empty, the name will be auto-asigned using the format `replicas.var.name` **Default value:** `""`
`replication_source_identifier` (`string`) optional
ARN of a source DB cluster or DB instance if this DB cluster is to be created as a Read Replica **Default value:** `""`
`restore_to_point_in_time` optional
List of point-in-time recovery options. Valid parameters are: `source_cluster_identifier` Identifier of the source database cluster from which to restore. `restore_type`: Type of restore to be performed. Valid options are "full-copy" and "copy-on-write". `use_latest_restorable_time`: Set to true to restore the database cluster to the latest restorable backup time. Conflicts with `restore_to_time`. `restore_to_time`: Date and time in UTC format to restore the database cluster to. Conflicts with `use_latest_restorable_time`. **Type:** ```hcl list(object({ source_cluster_identifier = string restore_type = optional(string, "copy-on-write") use_latest_restorable_time = optional(bool, true) restore_to_time = optional(string, null) })) ``` **Default value:** `[ ]`
`retention_period` (`number`) optional
Number of days to retain backups for **Default value:** `5`
`s3_import` optional
Restore from a Percona Xtrabackup in S3. The `bucket_name` is required to be in the same region as the resource. **Type:** ```hcl object({ bucket_name = string bucket_prefix = string ingestion_role = string source_engine = string source_engine_version = string }) ``` **Default value:** `null`
`scaling_configuration` optional
List of nested attributes with scaling properties. Only valid when `engine_mode` is set to `serverless` **Type:** ```hcl list(object({ auto_pause = bool max_capacity = number min_capacity = number seconds_until_auto_pause = number timeout_action = string })) ``` **Default value:** `[ ]`
`security_groups` (`list(string)`) optional
List of security groups to be allowed to connect to the DB instance **Default value:** `[ ]`
`serverlessv2_scaling_configuration` optional
serverlessv2 scaling properties **Type:** ```hcl object({ min_capacity = number max_capacity = number seconds_until_auto_pause = optional(number, null) }) ``` **Default value:** `null`
`skip_final_snapshot` (`bool`) optional
Determines whether a final DB snapshot is created before the DB cluster is deleted **Default value:** `true`
`snapshot_identifier` (`string`) optional
Specifies whether or not to create this cluster from a snapshot **Default value:** `null`
`source_region` (`string`) optional
Source Region of primary cluster, needed when using encrypted storage and region replicas **Default value:** `""`
`storage_encrypted` (`bool`) optional
Specifies whether the DB cluster is encrypted. The default is `false` for `provisioned` `engine_mode` and `true` for `serverless` `engine_mode` **Default value:** `false`
`storage_type` (`string`) optional
One of 'standard' (magnetic), 'gp2' (general purpose SSD), 'io1' (provisioned IOPS SSD), 'aurora', or 'aurora-iopt1' **Default value:** `null`
`subnet_group_name` (`string`) optional
Database subnet group name. Will use generated label ID if not supplied. **Default value:** `""`
`timeouts_configuration` optional
List of timeout values per action. Only valid actions are `create`, `update` and `delete` **Type:** ```hcl list(object({ create = string update = string delete = string })) ``` **Default value:** `[ ]`
`use_reserved_instances` (`bool`) optional
WARNING: Observe your plans and applies carefully when using this feature. It has potential to be very expensive if not used correctly. Also, it is not clear what happens when the reservation expires. Whether to use reserved instances. **Default value:** `false`
`vpc_security_group_ids` (`list(string)`) optional
Additional security group IDs to apply to the cluster, in addition to the provisioned default security group with ingress traffic from existing CIDR blocks and existing security groups **Default value:** `[ ]`
`zone_id` (`any`) optional
Route53 DNS Zone ID as list of string (0 or 1 items). If empty, no custom DNS name will be published. If the list contains a single Zone ID, a custom DNS name will be pulished in that zone. Can also be a plain string, but that use is DEPRECATED because of Terraform issues. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`activity_stream_arn`
Activity Stream ARN
`activity_stream_name`
Activity Stream Name
`admin_user_secret`
The secret manager attributes for the managed admin user password (`master_user_secret`).
`arn`
Amazon Resource Name (ARN) of the cluster
`cluster_identifier`
Cluster Identifier
`cluster_resource_id`
The region-unique, immutable identifie of the cluster
`cluster_security_groups`
Default RDS cluster security groups
`database_name`
Database name
`dbi_resource_ids`
List of the region-unique, immutable identifiers for the DB instances in the cluster
`endpoint`
The DNS address of the RDS instance
`instance_endpoints`
List of DNS addresses for the DB instances in the cluster
`master_host`
DB Master hostname
`master_username`
Username for the master DB user
`port`
DB port
`reader_endpoint`
A read-only endpoint for the Aurora cluster, automatically load-balanced across replicas
`replicas_host`
Replicas hostname
`reserved_instance`
All information about the reserved instance(s) if created.
`security_group_arn`
Security Group ARN
`security_group_id`
Security Group ID
`security_group_name`
Security Group name
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 5.81.0` - `null`, version: `>= 2.0` - `random`, version: `>= 2.0` ### Providers - `aws`, version: `>= 5.81.0` - `random`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_master` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `dns_replicas` | 0.13.0 | [`cloudposse/route53-cluster-hostname/aws`](https://registry.terraform.io/modules/cloudposse/route53-cluster-hostname/aws/0.13.0) | n/a `enhanced_monitoring_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `rds_identifier` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_appautoscaling_policy.replicas`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_policy) (resource) - [`aws_appautoscaling_target.replicas`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/appautoscaling_target) (resource) - [`aws_db_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_parameter_group) (resource) - [`aws_db_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_subnet_group) (resource) - [`aws_iam_role.enhanced_monitoring`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.enhanced_monitoring`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_rds_cluster.primary`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster) (resource) - [`aws_rds_cluster.secondary`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster) (resource) - [`aws_rds_cluster_activity_stream.primary`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_activity_stream) (resource) - [`aws_rds_cluster_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_instance) (resource) - [`aws_rds_cluster_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_parameter_group) (resource) - [`aws_rds_reserved_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_reserved_instance) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.egress_ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_ipv6_cidr_blocks`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.ingress_security_groups`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.traffic_inside_security_group`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`random_pet.instance`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/pet) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.enhanced_monitoring`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_rds_reserved_instance_offering.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/rds_reserved_instance_offering) (data source) --- ## rds-cluster-instance-group # Module: `rds-cluster-instance-group` Terraform module to provision an [`RDS Aurora`](https://aws.amazon.com/rds/aurora) instance group for MySQL or Postgres along with a dedicated endpoint. Use this module together with our [`terraform-aws-rds-cluster`](https://github.com/cloudposse/terraform-aws-rds-cluster) to provision pools of RDS instances. This is useful for creating reporting clusters that don't impact the production databases. Supports [Amazon Aurora Serverless](https://aws.amazon.com/rds/aurora/serverless/). ## Usage [Basic example](https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group/tree/main/examples/basic) ```hcl module "rds_cluster_replicas" { source = "git::https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group.git?ref=master" name = "postgres" namespace = "eg" stage = "dev" attributes = ["replicas"] cluster_identifier = "eg-dev-db" cluster_size = "2" db_port = "5432" instance_type = "db.r4.large" vpc_id = "vpc-xxxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" } ``` [With cluster parameters](https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group/tree/main/examples/with_cluster_parameters) ```hcl module "rds_cluster_reporting" { source = "git::https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group.git?ref=master" cluster_size = "2" namespace = "eg" stage = "dev" name = "db" attributes = ["reporting"] cluster_identifier = "eg-dev-db" instance_type = "db.t2.small" vpc_id = "vpc-xxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" cluster_parameters = [ { name = "character_set_client" value = "utf8" }, { name = "character_set_connection" value = "utf8" }, { name = "character_set_database" value = "utf8" }, { name = "character_set_results" value = "utf8" }, { name = "character_set_server" value = "utf8" }, { name = "collation_connection" value = "uft8_bin" }, { name = "collation_server" value = "uft8_bin" }, { name = "lower_case_table_names" value = "1" apply_method = "pending-reboot" }, { name = "skip-character-set-client-handshake" value = "1" apply_method = "pending-reboot" }, ] } ``` [With enhanced monitoring](https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group/tree/main/examples/enhanced_monitoring) ```hcl # create IAM role for monitoring resource "aws_iam_role" "enhanced_monitoring" { name = "rds-cluster-example-1" assume_role_policy = "${data.aws_iam_policy_document.enhanced_monitoring.json}" } # Attach Amazon's managed policy for RDS enhanced monitoring resource "aws_iam_role_policy_attachment" "enhanced_monitoring" { role = "${aws_iam_role.enhanced_monitoring.name}" policy_arn = "arn:aws:iam::aws:policy/service-role/AmazonRDSEnhancedMonitoringRole" } # allow rds to assume this role data "aws_iam_policy_document" "enhanced_monitoring" { statement { actions = [ "sts:AssumeRole", ] effect = "Allow" principals { type = "Service" identifiers = ["monitoring.rds.amazonaws.com"] } } } module "rds_cluster_reporting" { source = "git::https://github.com/cloudposse/terraform-aws-rds-cluster-instance-group.git?ref=master" cluster_size = "2" namespace = "eg" stage = "dev" name = "db" attributes = ["reporting"] cluster_identifier = "eg-dev-db" db_port = "5432" instance_type = "db.r4.large" vpc_id = "vpc-xxxxxxx" security_groups = ["sg-xxxxxxxx"] subnets = ["subnet-xxxxxxxx", "subnet-xxxxxxxx"] zone_id = "Zxxxxxxxx" # enable monitoring every 30 seconds rds_monitoring_interval = "30" # reference iam role created above rds_monitoring_role_arn = "${aws_iam_role.enhanced_monitoring.arn}" } ``` ## Variables ### Required Variables
`cluster_identifier` (`string`) required
The cluster identifier
`subnets` (`list`) required
List of VPC subnet IDs
`vpc_id` (`string`) required
VPC ID to create the cluster in (e.g. `vpc-a22222ee`)
### Optional Variables
`allowed_cidr_blocks` (`list`) optional
List of CIDR blocks allowed to access **Default value:** `[ ]`
`apply_immediately` (`string`) optional
Specifies whether any cluster modifications are applied immediately, or during the next maintenance window **Default value:** `"true"`
`cluster_family` (`string`) optional
The family of the DB cluster parameter group **Default value:** `"aurora5.6"`
`cluster_size` (`string`) optional
Number of DB instances to create in the cluster **Default value:** `"2"`
`custom_endpoint_type` (`string`) optional
The type of the endpoint. One of: READER, ANY **Default value:** `"READER"`
`db_port` (`string`) optional
Database port **Default value:** `"3306"`
`db_subnet_group_name` (`string`) optional
A DB subnet group to associate with this DB instance. NOTE: This must match the db_subnet_group_name of the attached aws_rds_cluster. **Default value:** `""`
`engine` (`string`) optional
The name of the database engine to be used for this DB cluster. Valid values: `aurora`, `aurora-mysql`, `aurora-postgresql` **Default value:** `"aurora"`
`instance_parameters` (`list`) optional
List of DB instance parameters to apply **Default value:** `[ ]`
`instance_type` (`string`) optional
Instance type to use **Default value:** `"db.t2.small"`
`promotion_tier` (`string`) optional
Failover Priority setting on instance level. The reader who has lower tier has higher priority to get promoted to writer (values can range from 0-15). **Default value:** `"15"`
`publicly_accessible` (`string`) optional
Set to true if you want your cluster to be publicly accessible (such as via QuickSight) **Default value:** `"false"`
`rds_monitoring_interval` (`string`) optional
Interval in seconds that metrics are collected, 0 to disable (values can only be 0, 1, 5, 10, 15, 30, 60) **Default value:** `"0"`
`rds_monitoring_role_arn` (`string`) optional
The ARN for the IAM role that can send monitoring metrics to CloudWatch Logs **Default value:** `""`
`security_group_ids` (`list`) optional
The IDs of the security groups from which to allow `ingress` traffic to the DB instances **Default value:** `[ ]`
`security_groups` (`list`) optional
List of security groups to be allowed to connect to the DB instances **Default value:** `[ ]`
`storage_encrypted` (`string`) optional
Specifies whether the DB cluster is encrypted. The default is `false` for `provisioned` `engine_mode` and `true` for `serverless` `engine_mode` **Default value:** `"true"`
`zone_id` (`string`) optional
Route53 parent zone ID. If provided (not empty), the module will create sub-domain DNS record for the cluster endpoint **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`string`) required
Name of the application **Required:** Yes **Default value:** ``
`namespace` (`string`) required
Namespace (e.g. `eg` or `cp`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** ```hcl [ "replicas" ] ```
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage` and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `"true"`
`tags` (`map`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`endpoint`
The endpoint for the Aurora cluster, automatically load-balanced across replicas
`hostname`
The DNS address for the endpoint of the Aurora cluster
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns` | tags/0.2.5 | [`git::https://github.com/cloudposse/terraform-aws-route53-cluster-hostname.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-aws-route53-cluster-hostname.git/tags/0.2.5) | n/a `label` | tags/0.3.5 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.5) | n/a ## Resources The following resources are used by this module: - [`aws_db_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_parameter_group) (resource) - [`aws_rds_cluster_endpoint.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_endpoint) (resource) - [`aws_rds_cluster_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/rds_cluster_instance) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.allow_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.allow_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.allow_ingress_cidr`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: - [`aws_rds_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/rds_cluster) (data source) --- ## rds-db-proxy # Module: `rds-db-proxy` Terraform module to provision an Amazon [RDS Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.html) for MySQL or Postgres. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-rds-db-proxy/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-rds-db-proxy/tree/main/test). ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "172.16.0.0/16" context = module.this.context } module "subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = module.vpc.igw_id cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } resource "random_password" "admin_password" { count = var.database_password == "" || var.database_password == null ? 1 : 0 length = 33 special = false override_special = "!#$%^&*()<>-_" } locals { database_password = var.database_password != "" && var.database_password != null ? var.database_password : join("", random_password.admin_password.*.result) username_password = { username = var.database_user password = local.database_password } auth = [ { auth_scheme = "SECRETS" description = "Access the database instance using username and password from AWS Secrets Manager" iam_auth = "DISABLED" secret_arn = aws_secretsmanager_secret.rds_username_and_password.arn } ] } module "rds_instance" { source = "cloudposse/rds/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" database_name = var.database_name database_user = var.database_user database_password = local.database_password database_port = var.database_port multi_az = var.multi_az storage_type = var.storage_type allocated_storage = var.allocated_storage storage_encrypted = var.storage_encrypted engine = var.engine engine_version = var.engine_version instance_class = var.instance_class db_parameter_group = var.db_parameter_group publicly_accessible = var.publicly_accessible vpc_id = module.vpc.vpc_id subnet_ids = module.subnets.private_subnet_ids security_group_ids = [module.vpc.vpc_default_security_group_id] apply_immediately = var.apply_immediately context = module.this.context } resource "aws_secretsmanager_secret" "rds_username_and_password" { name = module.this.id description = "RDS username and password" recovery_window_in_days = 0 tags = module.this.tags } resource "aws_secretsmanager_secret_version" "rds_username_and_password" { secret_id = aws_secretsmanager_secret.rds_username_and_password.id secret_string = jsonencode(local.username_password) } module "rds_proxy" { source = "cloudposse/rds-db-proxy/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" db_instance_identifier = module.rds_instance.instance_id auth = local.auth vpc_security_group_ids = [module.vpc.vpc_default_security_group_id] vpc_subnet_ids = module.subnets.public_subnet_ids debug_logging = var.debug_logging engine_family = var.engine_family idle_client_timeout = var.idle_client_timeout require_tls = var.require_tls connection_borrow_timeout = var.connection_borrow_timeout init_query = var.init_query max_connections_percent = var.max_connections_percent max_idle_connections_percent = var.max_idle_connections_percent session_pinning_filters = var.session_pinning_filters existing_iam_role_arn = var.existing_iam_role_arn context = module.this.context } ``` ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-rds-db-proxy/tree/main/examples/complete) to see how to use this module. ## Variables ### Required Variables
`auth` required
Configuration blocks with authorization mechanisms to connect to the associated database instances or clusters **Type:** ```hcl list(object({ auth_scheme = string description = string iam_auth = string secret_arn = string })) ```
`vpc_security_group_ids` (`set(string)`) required
One or more VPC security group IDs to associate with the proxy
`vpc_subnet_ids` (`set(string)`) required
One or more VPC subnet IDs to associate with the proxy
### Optional Variables
`connection_borrow_timeout` (`number`) optional
The number of seconds for a proxy to wait for a connection to become available in the connection pool. Only applies when the proxy has opened its maximum number of connections and all connections are busy with client sessions **Default value:** `120`
`db_cluster_identifier` (`string`) optional
DB cluster identifier. Either `db_instance_identifier` or `db_cluster_identifier` should be specified and both should not be specified together **Default value:** `null`
`db_instance_identifier` (`string`) optional
DB instance identifier. Either `db_instance_identifier` or `db_cluster_identifier` should be specified and both should not be specified together **Default value:** `null`
`debug_logging` (`bool`) optional
Whether the proxy includes detailed information about SQL statements in its logs **Default value:** `false`
`engine_family` (`string`) optional
The kinds of databases that the proxy can connect to. This value determines which database network protocol the proxy recognizes when it interprets network traffic to and from the database. The engine family applies to MySQL and PostgreSQL for both RDS and Aurora. Valid values are MYSQL and POSTGRESQL **Default value:** `"MYSQL"`
`existing_iam_role_arn` (`string`) optional
The ARN of an existing IAM role that the proxy can use to access secrets in AWS Secrets Manager. If not provided, the module will create a role to access secrets in Secrets Manager **Default value:** `null`
`iam_role_attributes` (`list(string)`) optional
Additional attributes to add to the ID of the IAM role that the proxy uses to access secrets in AWS Secrets Manager **Default value:** `null`
`idle_client_timeout` (`number`) optional
The number of seconds that a connection to the proxy can be inactive before the proxy disconnects it **Default value:** `1800`
`init_query` (`string`) optional
One or more SQL statements for the proxy to run when opening each new database connection **Default value:** `null`
`kms_key_id` (`string`) optional
The ARN or Id of the AWS KMS customer master key (CMK) to encrypt the secret values in the versions stored in secrets. If you don't specify this value, then Secrets Manager defaults to using the AWS account's default CMK (the one named `aws/secretsmanager`) **Default value:** `null`
`max_connections_percent` (`number`) optional
The maximum size of the connection pool for each target in a target group **Default value:** `100`
`max_idle_connections_percent` (`number`) optional
Controls how actively the proxy closes idle database connections in the connection pool. A high value enables the proxy to leave a high percentage of idle connections open. A low value causes the proxy to close idle client connections and return the underlying database connections to the connection pool **Default value:** `50`
`proxy_create_timeout` (`string`) optional
Proxy creation timeout **Default value:** `"30m"`
`proxy_delete_timeout` (`string`) optional
Proxy delete timeout **Default value:** `"60m"`
`proxy_update_timeout` (`string`) optional
Proxy update timeout **Default value:** `"30m"`
`require_tls` (`bool`) optional
A Boolean parameter that specifies whether Transport Layer Security (TLS) encryption is required for connections to the proxy. By enabling this setting, you can enforce encrypted TLS connections to the proxy **Default value:** `false`
`session_pinning_filters` (`list(string)`) optional
Each item in the list represents a class of SQL operations that normally cause all later statements in a session using a proxy to be pinned to the same underlying database connection **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`proxy_arn`
Proxy ARN
`proxy_default_target_group_arn`
The Amazon Resource Name (ARN) representing the default target group
`proxy_default_target_group_name`
The name of the default target group
`proxy_endpoint`
Proxy endpoint
`proxy_iam_role_arn`
The ARN of the IAM role that the proxy uses to access secrets in AWS Secrets Manager
`proxy_id`
Proxy ID
`proxy_target_endpoint`
Hostname for the target RDS DB Instance. Only returned for `RDS_INSTANCE` type
`proxy_target_id`
Identifier of `db_proxy_name`, `target_group_name`, `target type` (e.g. `RDS_INSTANCE` or `TRACKED_CLUSTER`), and resource identifier separated by forward slashes (`/`)
`proxy_target_port`
Port for the target RDS DB instance or Aurora DB cluster
`proxy_target_rds_resource_id`
Identifier representing the DB instance or DB cluster target
`proxy_target_target_arn`
Amazon Resource Name (ARN) for the DB instance or DB cluster
`proxy_target_tracked_cluster_id`
DB Cluster identifier for the DB instance target. Not returned unless manually importing an `RDS_INSTANCE` target that is part of a DB cluster
`proxy_target_type`
Type of target. e.g. `RDS_INSTANCE` or `TRACKED_CLUSTER`
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.1.15` ### Providers - `aws`, version: `>= 3.1.15` ### Modules Name | Version | Source | Description --- | --- | --- | --- `role_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_db_proxy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_proxy) (resource) - [`aws_db_proxy_default_target_group.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_proxy_default_target_group) (resource) - [`aws_db_proxy_target.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_proxy_target) (resource) - [`aws_iam_policy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_kms_key.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## rds-replica # Module: `rds-replica` Terraform module to provision AWS [`RDS`](https://aws.amazon.com/rds/) replica instances. These are best suited for reporting purposes. **IMPORTANT** It is not possible to create a read replica for a DB Instance that belongs to an Aurora DB Cluster. ## Introduction The module will create an RDS replica instance: * RDS Replica instance (MySQL, Postgres, SQL Server, Oracle) * RDS Subnet Group * RDS DB Security Group * DNS Record in Route53 for the DB endpoint ## Usage ```hcl module "rds_replica" { source = "git::https://github.com/cloudposse/terraform-aws-rds-replica.git?ref=master" namespace = "eg" stage = "prod" name = "reporting" replicate_source_db = "eg-prod-db dns_zone_id = "Z89FN1IW975KPE" host_name = "reporting" security_group_ids = ["sg-xxxxxxxx"] database_port = 3306 multi_az = "true" storage_type = "gp2" storage_encrypted = "true" instance_class = "db.t2.medium" publicly_accessible = "false" subnet_ids = ["subnet-xxxxxxxxx", "subnet-xxxxxxxxx"] vpc_id = "vpc-xxxxxxxx" auto_minor_version_upgrade = "true" allow_major_version_upgrade = "false" apply_immediately = "false" maintenance_window = "Mon:03:00-Mon:04:00" skip_final_snapshot = "false" copy_tags_to_snapshot = "true" backup_retention_period = 7 backup_window = "22:00-03:00" } ``` ## Variables ### Required Variables
`database_port` (`any`) required
Database port (_e.g._ `3306` for `MySQL`). Used in the DB Security Group to allow access to the DB instance from the provided `security_group_ids`
`instance_class` (`string`) required
Class of RDS instance
`replicate_source_db` (`any`) required
Specifies that this resource is a Replicate database, and to use this value as the source database. This correlates to the identifier of another Amazon RDS Database to replicate. Note that if you are creating a cross-region replica of an encrypted database you will also need to specify a kms_key_id. See [DB Instance Replication](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.Replication.html) and [Working with PostgreSQL and MySQL Read Replicas](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ReadRepl.html) for more information on using Replication.
`subnet_ids` (`list(string)`) required
List of subnets for the DB
`vpc_id` (`string`) required
VPC ID the DB instance will be created in
### Optional Variables
`allow_major_version_upgrade` (`string`) optional
Allow major version upgrade **Default value:** `"false"`
`apply_immediately` (`string`) optional
Specifies whether any database modifications are applied immediately, or during the next maintenance window **Default value:** `"false"`
`auto_minor_version_upgrade` (`string`) optional
Allow automated minor version upgrade (e.g. from Postgres 9.5.3 to Postgres 9.5.4) **Default value:** `"true"`
`backup_retention_period` (`number`) optional
Backup retention period in days. Must be > 0 to enable backups **Default value:** `0`
`backup_window` (`string`) optional
When AWS can perform DB snapshots, can't overlap with maintenance window **Default value:** `"22:00-03:00"`
`copy_tags_to_snapshot` (`string`) optional
Copy tags from DB to a snapshot **Default value:** `"true"`
`db_parameter` (`list(string)`) optional
A list of DB parameters to apply. Note that parameters may differ from a DB family to another **Default value:** `[ ]`
`dns_zone_id` (`string`) optional
The ID of the DNS Zone in Route53 where a new DNS record will be created for the DB host name **Default value:** `""`
`final_snapshot_identifier` (`string`) optional
Final snapshot identifier e.g.: some-db-final-snapshot-2015-06-26-06-05 **Default value:** `""`
`host_name` (`string`) optional
The DB host name created in Route53 **Default value:** `"db"`
`iops` (`string`) optional
The amount of provisioned IOPS. Setting this implies a storage_type of 'io1'. Default is 0 if rds storage type is not 'io1' **Default value:** `"0"`
`kms_key_id` (`string`) optional
The ARN for the KMS encryption key. If creating an encrypted replica, set this to the destination KMS ARN **Default value:** `""`
`maintenance_window` (`string`) optional
The window to perform maintenance in. Syntax: 'ddd:hh24:mi-ddd:hh24:mi' UTC **Default value:** `"Mon:03:00-Mon:04:00"`
`monitoring_interval` (`string`) optional
The interval, in seconds, between points when Enhanced Monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. Valid Values are 0, 1, 5, 10, 15, 30, 60. **Default value:** `"0"`
`multi_az` (`string`) optional
Set to true if multi AZ deployment must be supported **Default value:** `"false"`
`parameter_group_name` (`string`) optional
Name of the DB parameter group to associate **Default value:** `""`
`publicly_accessible` (`string`) optional
Determines if database can be publicly available (NOT recommended) **Default value:** `"false"`
`same_region` (`string`) optional
Whether this replica is in the same region as the master. **Default value:** `"false"`
`security_group_ids` (`list(string)`) optional
The IDs of the security groups from which to allow `ingress` traffic to the DB instance **Default value:** `[ ]`
`skip_final_snapshot` (`string`) optional
If true (default), no snapshot will be made before deleting DB **Default value:** `"true"`
`snapshot_identifier` (`string`) optional
Snapshot identifier e.g: rds:production-2015-06-26-06-05. If specified, the module create cluster from the snapshot **Default value:** `""`
`storage_encrypted` (`string`) optional
Specifies whether the DB instance is encrypted. The default is false if not specified. **Default value:** `"false"`
`storage_type` (`string`) optional
One of 'standard' (magnetic), 'gp2' (general purpose SSD), or 'io1' (provisioned IOPS SSD). **Default value:** `"standard"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`string`) required
The Name of the application or solution (e.g. `bastion` or `portal`) **Required:** Yes **Default value:** ``
`namespace` (`string`) required
Namespace (e.g. `eg` or `cp`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `name`, `namespace`, `stage` and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`string`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `"true"`
`tags` (`map(string)`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`hostname`
DNS host name of the instance
`instance_address`
Address of the instance
`instance_endpoint`
DNS Endpoint of the instance
`instance_id`
ID of the instance
`security_group_id`
ID of the Security Group
`subnet_group_id`
ID of the Subnet Group
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `dns_host_name` | tags/0.2.5 | [`git::https://github.com/cloudposse/terraform-aws-route53-cluster-hostname.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-aws-route53-cluster-hostname.git/tags/0.2.5) | n/a `final_snapshot_label` | tags/0.3.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.3) | n/a `label` | tags/0.3.3 | [`git::https://github.com/cloudposse/terraform-null-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-null-label.git/tags/0.3.3) | n/a ## Resources The following resources are used by this module: - [`aws_db_instance.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_instance) (resource) - [`aws_db_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/db_subnet_group) (resource) - [`aws_kms_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kms_key) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.allow_egress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.allow_ingress`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) ## Data Sources The following data sources are used by this module: --- ## redshift-cluster # Module: `redshift-cluster` This is `terraform-example-module` project provides all the scaffolding for a typical well-built Cloud Posse module. It's a template repository you can use when creating new repositories. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-redshift-cluster/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-redshift-cluster/tree/main/test). ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" context = module.this.context } module "subnet" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" availability_zones = var.availability_zones vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] ipv4_cidr_block = module.vpc.vpc_cidr_block nat_gateway_enabled = false nat_instance_enabled = false context = module.this.context } module "security_group" { source = "cloudposse/security-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = module.vpc.vpc_id rules = [ { type = "ingress" from_port = var.port to_port = var.port protocol = "all" cidr_blocks = ["0.0.0.0/0"] }, { type = "egress" from_port = 0 to_port = 0 protocol = "all" cidr_blocks = ["0.0.0.0/0"] } ] context = module.this.context } module "redshift_cluster" { source = "cloudposse/redshift-cluster/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" subnet_ids = module.subnet.private_subnet_ids vpc_security_groups = [module.vpc.vpc_default_security_group_id, module.security_group.id] admin_user = var.admin_user admin_password = var.admin_password database_name = var.database_name node_type = var.node_type cluster_type = var.cluster_type engine_version = var.engine_version publicly_accessible = var.publicly_accessible allow_version_upgrade = var.allow_version_upgrade context = module.this.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-redshift-cluster/) - complete example of using this module ## Variables ### Required Variables
`subnet_ids` (`list(string)`) required
List of VPC subnet IDs
### Optional Variables
`admin_password` (`string`) optional
Password for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `""`
`admin_user` (`string`) optional
Username for the master DB user. Required unless a snapshot_identifier is provided **Default value:** `"admin"`
`allow_version_upgrade` (`bool`) optional
Whether or not to enable major version upgrades which are applied during the maintenance window to the Amazon Redshift engine that is running on the cluster **Default value:** `false`
`automated_snapshot_retention_period` (`number`) optional
The number of days that automated snapshots are retained. If the value is 0, automated snapshots are disabled **Default value:** `1`
`availability_zone` (`string`) optional
Optional parameter to place Amazon Redshift cluster instances in a specific availability zone. If left empty, will place randomly **Default value:** `null`
`availability_zone_relocation_enabled` (`bool`) optional
Whether or not the cluster can be relocated to another availability zone, either automatically by AWS or when requested. Available for use on clusters from the RA3 instance family **Default value:** `false`
`cluster_identifier` (`string`) optional
The Redshift Cluster Identifier. Must be a lower case string. Will use generated label ID if not supplied **Default value:** `""`
`cluster_parameters` optional
List of Redshift parameters to apply **Type:** ```hcl list(object({ name = string value = string })) ``` **Default value:** `[ ]`
`cluster_type` (`string`) optional
The cluster type to use. Either `single-node` or `multi-node` **Default value:** `"single-node"`
`database_name` (`string`) optional
The name of the first database to be created when the cluster is created **Default value:** `"dev"`
`elastic_ip` (`string`) optional
The Elastic IP (EIP) address for the cluster **Default value:** `null`
`encrypted` (`bool`) optional
Specifies whether the cluster is encrypted at rest **Default value:** `false`
`engine_version` (`string`) optional
The version of the Amazon Redshift engine to use. See https://docs.aws.amazon.com/redshift/latest/mgmt/cluster-versions.html **Default value:** `"1.0"`
`enhanced_vpc_routing` (`bool`) optional
If true , enhanced VPC routing is enabled **Default value:** `false`
`final_snapshot_identifier` (`string`) optional
The identifier of the final snapshot that is to be created immediately before deleting the cluster. If this parameter is provided, `skip_final_snapshot` must be `false` **Default value:** `null`
`iam_roles` (`list(string)`) optional
A list of IAM Role ARNs to associate with the cluster. A maximum of 10 can be associated to the cluster at any time **Default value:** `[ ]`
`kms_key_arn` (`string`) optional
The ARN for the KMS encryption key. When specifying `kms_key_arn`, `encrypted` needs to be set to `true` **Default value:** `null`
`logging_bucket_name` (`string`) optional
The name of an existing S3 bucket where the log files are to be stored. Must be in the same region as the cluster and the cluster must have read bucket and put object permissions **Default value:** `null`
`logging_enabled` (`bool`) optional
If true, enables logging information such as queries and connection attempts, for the specified Amazon Redshift cluster **Default value:** `false`
`logging_s3_key_prefix` (`string`) optional
The prefix applied to the log file names **Default value:** `null`
`node_type` (`string`) optional
The node type to be provisioned for the cluster. See https://docs.aws.amazon.com/redshift/latest/mgmt/working-with-clusters.html#working-with-clusters-overview **Default value:** `"dc2.large"`
`number_of_nodes` (`number`) optional
The number of compute nodes in the cluster. This parameter is required when the ClusterType parameter is specified as multi-node **Default value:** `1`
`owner_account` (`string`) optional
The AWS customer account used to create or copy the snapshot. Required if you are restoring a snapshot you do not own, optional if you own the snapshot **Default value:** `null`
`port` (`number`) optional
The port number on which the cluster accepts incoming connections **Default value:** `5439`
`preferred_maintenance_window` (`string`) optional
Weekly time range during which system maintenance can occur, in UTC. Format: ddd:hh24:mi-ddd:hh24:mi **Default value:** `null`
`publicly_accessible` (`bool`) optional
If true, the cluster can be accessed from a public network **Default value:** `false`
`skip_final_snapshot` (`bool`) optional
Determines whether a final snapshot of the cluster is created before Amazon Redshift deletes the cluster **Default value:** `true`
`snapshot_cluster_identifier` (`string`) optional
The name of the cluster the source snapshot was created from **Default value:** `null`
`snapshot_identifier` (`string`) optional
The name of the snapshot from which to create the new cluster **Default value:** `null`
`vpc_security_group_ids` (`list(string)`) optional
A list of Virtual Private Cloud (VPC) security groups to be associated with the cluster. Used for EC2-VPC platform **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
Amazon Resource Name (ARN) of cluster
`cluster_identifier`
The Cluster Identifier
`cluster_parameter_group_name`
The name of the parameter group to be associated with this cluster
`cluster_revision_number`
The specific revision number of the database in the cluster
`cluster_subnet_group_name`
The name of a cluster subnet group to be associated with this cluster
`cluster_type`
The cluster type
`database_name`
The name of the default database in the Cluster
`dns_name`
The DNS name of the cluster
`endpoint`
The connection endpoint
`id`
The Redshift Cluster ID
`node_type`
The type of nodes in the cluster
`port`
The Port the cluster responds on
`redshift_parameter_group_arn`
Amazon Resource Name (ARN) of the Redshift parameter group
`redshift_parameter_group_id`
The Redshift parameter group ID
`redshift_subnet_group_arn`
Amazon Resource Name (ARN) of the Redshift Subnet group name
`redshift_subnet_group_id`
The Redshift Subnet group ID
`vpc_security_group_ids`
The VPC security group IDs associated with the cluster
## Dependencies ### Requirements - `terraform`, version: `>= 0.14` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_redshift_cluster.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_cluster) (resource) - [`aws_redshift_parameter_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_parameter_group) (resource) - [`aws_redshift_subnet_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_subnet_group) (resource) ## Data Sources The following data sources are used by this module: --- ## module # Module: `module` This is an example project to provide all the scaffolding for a typical well-built Cloud Posse Terraform module for AWS resources. It's a template repository you can use when creating new repositories. This is not a useful module by itself. ## Quick Start 1. Use this repo as a template for a new repo. 2. Check out the new repo and create a `git` branch to work on. 3. Replace the Terraform code at the root of the repo with the code you want to publish. 4. Replace the code in `examples/complete` with Terraform code that will make a good automated test. Please keep `context.tf` and `fixtures.us-east-2.tfvars` in place and change only `name`, leaving `region`, `namespace`, `environment`, and `stage` as is. Provide outputs that will be useful for testing. 5. Update `test/src/examples_complete_test.go` to verify the outputs of running `terraform apply` on `examples/complete`. 6. Run `make github/init` to update the repo with the current Cloud Posse framework files (e.g. `CODEOWNERS`). 7. Run `make pr/auto-format` to format the Terraform code and generate documentation. 8. Commit everything to `git` and open your first PR on the new repo. ## Variables ### Required Variables
### Optional Variables
`example` (`string`) optional
Example variable **Default value:** `"hello world"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`example`
Example output
`id`
ID of the created example
`random`
Stable random number for this example
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `random`, version: `>= 2.2` ### Providers - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`random_integer.example`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/integer) (resource) ## Data Sources The following data sources are used by this module: --- ## route53-alias # Module: `route53-alias` Terraform module that implements "vanity" host names (e.g. `brand.com`) as `ALIAS` records to another Route53 DNS resource record (e.g. ELB/ALB, S3 Bucket Endpoint or CloudFront Distribution). Unlike `CNAME` records, the synthetic `ALIAS` record works with zone apexes. ## Usage This will define a `A` resource record for `www.example.com` as an alias of the `aws_elb.example.dns_name`. ```terraform module "production_www" { source = "cloudposse/route53-alias/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" aliases = ["www.example.com.", "static1.cdn.example.com.", "static2.cdn.example.com"] parent_zone_id = var.parent_zone_id target_dns_name = aws_elb.example.dns_name target_zone_id = aws_elb.example.zone_id } ``` ## Variables ### Required Variables
`aliases` (`list(string)`) required
List of aliases
`target_dns_name` (`string`) required
DNS name of target resource (e.g. ALB, ELB)
`target_zone_id` (`string`) required
ID of target resource (e.g. ALB, ELB)
### Optional Variables
`allow_overwrite` (`bool`) optional
Allow creation of this record in Terraform to overwrite an existing record, if any. This does not affect the ability to update the record in Terraform and does not prevent other resources within Terraform or manual Route 53 changes outside Terraform from overwriting this record. false by default. This configuration is not recommended for most environments **Default value:** `false`
`evaluate_target_health` (`bool`) optional
Set to true if you want Route 53 to determine whether to respond to DNS queries **Default value:** `false`
`ipv6_enabled` (`bool`) optional
Set to true to enable an AAAA DNS record to be set as well as the A record **Default value:** `false`
`parent_zone_id` (`string`) optional
ID of the hosted zone to contain this record (or specify `parent_zone_name`) **Default value:** `""`
`parent_zone_name` (`string`) optional
Name of the hosted zone to contain this record (or specify `parent_zone_id`) **Default value:** `""`
`private_zone` (`bool`) optional
Is this a private hosted zone? **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`hostnames`
List of DNS records
`parent_zone_id`
ID of the hosted zone to contain the records
`parent_zone_name`
Name of the hosted zone to contain the records
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `local`, version: `>= 1.2` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_record.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) ## Data Sources The following data sources are used by this module: - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## route53-cluster-hostname # Module: `route53-cluster-hostname` Terraform module to define a consistent AWS Route53 hostname ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-route53-cluster-hostname/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-route53-cluster-hostname/tree/main/test). ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "route53_hostname" { source = "cloudposse/route53-cluster-hostname/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" zone_id = "Z3SO0TKDDQ0RGG" type = "CNAME" records = [ "test-hostname", ] context = module.label.context } ``` ## Variables ### Required Variables
`records` (`list(string)`) required
DNS records to create
### Optional Variables
`dns_name` (`string`) optional
The name of the DNS record **Default value:** `""`
`healthcheck_enabled` (`bool`) optional
Whether to create a Route53 health check **Default value:** `false`
`healthcheck_settings` optional
Route 53 health check configuration settings domain: The fully qualified domain name of the endpoint to be checked ip_address: The IP address of the endpoint to be checked regions: AWS regions to run the health checks from type: The protocol to use for the health check such as HTTP HTTPS TCP etc port: Port on the endpoint to be checked reference_name: Used in caller reference and helpful for identifying individual health check sets resource_path: The URL path Route 53 requests during the health check failure_threshold: Number of consecutive health checks that an endpoint must pass or fail search_string: String searched in response body for match checks measure_latency: Whether to measure and report latency from multiple regions invert_healthcheck: If true a healthy check is considered unhealthy and vice versa child_healthchecks: List of health check IDs for associated child checks routing_control_arn: ARN of the Application Recovery Controller routing control request_interval: Interval between health check requests in seconds child_health_threshold: Minimum number of child checks that must be healthy cloudwatch_alarm_name: Name of the CloudWatch alarm to evaluate cloudwatch_alarm_region: Region where the CloudWatch alarm is configured insufficient_data_health_status: Status to assign when CloudWatch has insufficient data **Type:** ```hcl object({ domain = optional(string) ip_address = optional(string) regions = optional(list(string)) type = optional(string, "HTTPS") request_interval = optional(string, "30") port = optional(number, 443) reference_name = optional(string) resource_path = optional(string) failure_threshold = optional(number) search_string = optional(string) measure_latency = optional(bool) invert_healthcheck = optional(bool) child_healthchecks = optional(list(string)) routing_control_arn = optional(string) child_health_threshold = optional(number) cloudwatch_alarm_name = optional(string) cloudwatch_alarm_region = optional(string) insufficient_data_health_status = optional(string) }) ``` **Default value:** `{ }`
`private_zone` (`bool`) optional
Used with `zone_name` input to get a private Hosted Zone. **Default value:** `null`
`ttl` (`number`) optional
The TTL of the record to add to the DNS zone to complete certificate validation **Default value:** `300`
`type` (`string`) optional
Type of DNS records to create **Default value:** `"CNAME"`
`zone_id` (`string`) optional
Route53 DNS Zone ID **Default value:** `null`
`zone_name` (`string`) optional
The Hosted Zone name of the desired Hosted Zone. **Default value:** `null`
`zone_tags` (`map(string)`) optional
Used with `zone_name` input. A map of tags, each pair of which must exactly match a pair on the desired Hosted Zone. **Default value:** `null`
`zone_vpc_id` (`string`) optional
Used with `zone_name` input to get a private Hosted Zone associated with the `vpc_id` (in this case, private_zone is not mandatory). **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`hostname`
DNS hostname
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.9` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_health_check.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_health_check) (resource) - [`aws_route53_record.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) ## Data Sources The following data sources are used by this module: - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## route53-cluster-zone # Module: `route53-cluster-zone` Terraform module to easily define consistent cluster domains on `Route53`. ## Usage Define a cluster domain of `foobar.example.com` using a custom naming convention for `zone_name`. The `zone_name` variable is optional. It defaults to `$${stage}.$${parent_zone_name}`. ```hcl module "domain" { source = "cloudposse/route53-cluster-zone/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "cluster" parent_zone_name = "example.com" zone_name = "$${name}.$${stage}.$${parent_zone_name}" } ``` ## Variables ### Required Variables
### Optional Variables
`ns_record_ttl` (`number`) optional
The time to live (TTL) of the Name Server Route53 record, in seconds. The default value is short for responsiveness to changes during development and is not recommended for production. Typical production values are 86400 or 172800. **Default value:** `60`
`parent_zone_id` (`string`) optional
ID of the hosted zone to contain this record (or specify `parent_zone_name`) **Default value:** `""`
`parent_zone_name` (`string`) optional
Name of the hosted zone to contain this record (or specify `parent_zone_id`) **Default value:** `""`
`parent_zone_record_enabled` (`bool`) optional
Whether to create the NS record on the parent zone. Useful for creating a cluster zone across accounts. `var.parent_zone_name` required if set to false. **Default value:** `true`
`soa_record_ttl` (`number`) optional
The time to live (TTL) of the Start of Authority Route53 record, in seconds. This sets the maximum time a negative (no data) query can be cached. The default value is short for responsiveness to changes during development and is not recommended for production. Typical production values are in the range of 300 to 3600. **Default value:** `30`
`zone_name` (`string`) optional
Zone name **Default value:** `"${name}.${stage}.${parent_zone_name}"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`fqdn`
Fully-qualified domain name
`parent_zone_id`
ID of the hosted zone to contain this record
`parent_zone_name`
Name of the hosted zone to contain this record
`zone_id`
Route53 DNS Zone ID
`zone_name`
Route53 DNS Zone name
`zone_name_servers`
Route53 DNS Zone Name Servers
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_record.ns`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.soa`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_zone) (resource) ## Data Sources The following data sources are used by this module: - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_route53_zone.parent_zone`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## route53-resolver-dns-firewall(Route53-resolver-dns-firewall) # Module: `route53-resolver-dns-firewall` Terraform module to provision Route 53 Resolver DNS Firewall, domain lists, firewall rules, rule groups, and logging configurations. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-route53-resolver-dns-firewall/tree/main/examples/complete) For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-route53-resolver-dns-firewall/tree/main/test). ```hcl provider "aws" { region = var.region } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ipv4_primary_cidr_block = "172.19.0.0/16" dns_hostnames_enabled = true dns_support_enabled = true internet_gateway_enabled = false ipv6_egress_only_internet_gateway_enabled = false assign_generated_ipv6_cidr_block = false context = module.this.context } module "s3_log_storage" { source = "cloudposse/s3-log-storage/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" force_destroy = true attributes = ["logs"] context = module.this.context } module "route53_resolver_firewall" { # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = module.vpc.vpc_id firewall_fail_open = "ENABLED" query_log_enabled = true query_log_destination_arn = module.s3_log_storage.bucket_arn domains_config = { "not-secure-domains" = { # The dot at the end of domain names is required by Route53 DNS Firewall # If not added, AWS adds it automatically and terraform shows permanent drift domains = [ "not-secure-domain-1.com.", "not-secure-domain-2.com.", "not-secure-domain-3.com." ] }, "alert-domains" = { domains = [ "alert-domain-1.com.", "alert-domain-2.com.", "alert-domain-3.com." ] }, "blacklisted-domains" = { # Concat the lists of domains passed in the `domains` field and loaded from the file `domains_file` domains = [ "blacklisted-domain-1.com.", "blacklisted-domain-2.com.", "blacklisted-domain-3.com." ] domains_file = "config/blacklisted_domains.txt" } } rule_groups_config = { "not-secure-domains-rule-group" = { # 'priority' must be between 100 and 9900 exclusive priority = 101 rules = { "block-not-secure-domains" = { # 'priority' must be between 100 and 9900 exclusive priority = 101 firewall_domain_list_name = "not-secure-domains" action = "BLOCK" block_response = "NXDOMAIN" } } }, "alert-and-blacklisted-domains-rule-group" = { # 'priority' must be between 100 and 9900 exclusive priority = 200 rules = { "alert-domains" = { # 'priority' must be between 100 and 9900 exclusive priority = 101 firewall_domain_list_name = "alert-domains" action = "ALERT" }, "block-and-override-blacklisted-domains" = { # 'priority' must be between 100 and 9900 exclusive priority = 200 firewall_domain_list_name = "blacklisted-domains" action = "BLOCK" block_response = "OVERRIDE" block_override_dns_type = "CNAME" block_override_domain = "go-here.com" block_override_ttl = 1 } } } } context = module.this.context } ``` ## Variables ### Required Variables
`domains_config` required
Map of Route 53 Resolver DNS Firewall domain configurations **Type:** ```hcl map(object({ domains = optional(list(string)) domains_file = optional(string) })) ```
`rule_groups_config` required
Rule groups and rules configuration **Type:** ```hcl map(object({ priority = number mutation_protection = optional(string) # https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_rule rules = map(object({ action = string priority = number block_override_dns_type = optional(string) block_override_domain = optional(string) block_override_ttl = optional(number) block_response = optional(string) firewall_domain_list_name = string })) })) ```
`vpc_id` (`string`) required
VPC ID
### Optional Variables
`firewall_fail_open` (`string`) optional
Determines how Route 53 Resolver handles queries during failures, for example when all traffic that is sent to DNS Firewall fails to receive a reply. By default, fail open is disabled, which means the failure mode is closed. This approach favors security over availability. DNS Firewall blocks queries that it is unable to evaluate properly. If you enable this option, the failure mode is open. This approach favors availability over security. In this case, DNS Firewall allows queries to proceed if it is unable to properly evaluate them. Valid values: ENABLED, DISABLED. **Default value:** `"ENABLED"`
`query_log_config_name` (`string`) optional
Route 53 Resolver query log config name. If omitted, the name will be generated by concatenating the ID from the context with the VPC ID **Default value:** `null`
`query_log_destination_arn` (`string`) optional
The ARN of the resource that you want Route 53 Resolver to send query logs. You can send query logs to an S3 bucket, a CloudWatch Logs log group, or a Kinesis Data Firehose delivery stream. **Default value:** `null`
`query_log_enabled` (`bool`) optional
Flag to enable/disable Route 53 Resolver query logging **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`domains`
Route 53 Resolver DNS Firewall domain configurations
`query_log_config`
Route 53 Resolver query logging configuration
`rule_group_associations`
Route 53 Resolver DNS Firewall rule group associations
`rule_groups`
Route 53 Resolver DNS Firewall rule groups
`rules`
Route 53 Resolver DNS Firewall rules
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route53_resolver_firewall_config.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_config) (resource) - [`aws_route53_resolver_firewall_domain_list.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_domain_list) (resource) - [`aws_route53_resolver_firewall_rule.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_rule) (resource) - [`aws_route53_resolver_firewall_rule_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_rule_group) (resource) - [`aws_route53_resolver_firewall_rule_group_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_firewall_rule_group_association) (resource) - [`aws_route53_resolver_query_log_config.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_query_log_config) (resource) - [`aws_route53_resolver_query_log_config_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_resolver_query_log_config_association) (resource) ## Data Sources The following data sources are used by this module: --- ## s3-bucket(S3-bucket) # Module: `s3-bucket` This module creates an S3 bucket with support for versioning, lifecycles, object locks, replication, encryption, ACL, bucket object policies, and static website hosting. For backward compatibility, it sets the S3 bucket ACL to `private` and the `s3_object_ownership` to `ObjectWriter`. Moving forward, setting `s3_object_ownership` to `BucketOwnerEnforced` is recommended, and doing so automatically disables the ACL. This module blocks public access to the bucket by default. See `block_public_acls`, `block_public_policy`, `ignore_public_acls`, and `restrict_public_buckets` to change the settings. See [AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/dev/access-control-block-public-access.html) for more details. This module can optionally create an IAM User with access to the S3 bucket. This is inherently insecure in that to enable anyone to become the User, access keys must be generated, and anything generated by Terraform is stored unencrypted in the Terraform state. See the [Terraform documentation](https://www.terraform.io/docs/state/sensitive-data.html) for more details The best way to grant access to the bucket is to grant one or more IAM Roles access to the bucket via `privileged_principal_arns`. This IAM Role can be assumed by EC2 instances via their Instance Profile, or Kubernetes (EKS) services using [IRSA](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html). Entities outside of AWS can assume the Role via [OIDC](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html). (See [this example of connecting GitHub](https://aws.amazon.com/blogs/security/use-iam-roles-to-connect-github-actions-to-actions-in-aws/) to enable GitHub actions to assume AWS IAM roles, or use [this Cloud Posse component](https://github.com/cloudposse/terraform-aws-components/tree/main/modules/github-oidc-provider) if you are already using the Cloud Posse reference architecture.) If neither of those approaches work, then as a last resort you can set `user_enabled = true` and this module will provision a basic IAM user with permissions to access the bucket. We do not recommend creating IAM users this way for any other purpose. If an IAM user is created, the IAM user name is constructed using [terraform-null-label](https://github.com/cloudposse/terraform-null-label) and some input is required. The simplest input is `name`. By default the name will be converted to lower case and all non-alphanumeric characters except for hyphen will be removed. See the documentation for `terraform-null-label` to learn how to override these defaults if desired. If an AWS Access Key is created, it is stored either in SSM Parameter Store or is provided as a module output, but not both. Using SSM Parameter Store is recommended because that will keep the secret from being easily accessible via Terraform remote state lookup, but the key will still be stored unencrypted in the Terraform state in any case. ## Usage Using [BucketOwnerEnforced](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html#object-ownership-overview) ```hcl module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "app" stage = "test" namespace = "eg" s3_object_ownership = "BucketOwnerEnforced" enabled = true user_enabled = false versioning_enabled = false privileged_principal_actions = ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"] privileged_principal_arns = [ { (local.deployment_iam_role_arn) = [""] }, { (local.additional_deployment_iam_role_arn) = ["prefix1/", "prefix2/"] } ] } ``` Configuring S3 [storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html): ```hcl locals { lifecycle_configuration_rules = [{ enabled = true # bool id = "v2rule" abort_incomplete_multipart_upload_days = 1 # number filter_and = null expiration = { days = 120 # integer > 0 } noncurrent_version_expiration = { newer_noncurrent_versions = 3 # integer > 0 noncurrent_days = 60 # integer >= 0 } transition = [{ days = 30 # integer >= 0 storage_class = "STANDARD_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }, { days = 60 # integer >= 0 storage_class = "ONEZONE_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }] noncurrent_version_transition = [{ newer_noncurrent_versions = 3 # integer >= 0 noncurrent_days = 30 # integer >= 0 storage_class = "ONEZONE_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }] }] } ``` Allowing specific principal ARNs to perform actions on the bucket: ```hcl module "s3_bucket" { source = "cloudposse/s3-bucket/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" s3_object_ownership = "BucketOwnerEnforced" enabled = true user_enabled = true versioning_enabled = false allowed_bucket_actions = ["s3:GetObject", "s3:ListBucket", "s3:GetBucketLocation"] name = "app" stage = "test" namespace = "eg" privileged_principal_arns = [ { "arn:aws:iam::123456789012:role/principal1" = ["prefix1/", "prefix2/"] }, { "arn:aws:iam::123456789012:role/principal2" = [""] }] privileged_principal_actions = [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] } ``` ## Variables ### Required Variables
### Optional Variables
`access_key_enabled` (`bool`) optional
Set to `true` to create an IAM Access Key for the created IAM user **Default value:** `true`
`acl` (`string`) optional
The [canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to apply. Deprecated by AWS in favor of bucket policies. Automatically disabled if `s3_object_ownership` is set to "BucketOwnerEnforced". Defaults to "private" for backwards compatibility, but we recommend setting `s3_object_ownership` to "BucketOwnerEnforced" instead. **Default value:** `"private"`
`allow_encrypted_uploads_only` (`bool`) optional
Set to `true` to prevent uploads of unencrypted objects to S3 bucket **Default value:** `false`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `false`
`allowed_bucket_actions` (`list(string)`) optional
List of actions the user is permitted to perform on the S3 bucket **Default value:** ```hcl [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] ```
`availability_zone_id` (`string`) optional
The ID of the availability zone. **Default value:** `""`
`block_public_acls` (`bool`) optional
Set to `false` to disable the blocking of new public access lists on the bucket **Default value:** `true`
`block_public_policy` (`bool`) optional
Set to `false` to disable the blocking of new public policies on the bucket **Default value:** `true`
`bucket_key_enabled` (`bool`) optional
Set this to true to use Amazon S3 Bucket Keys for SSE-KMS, which may or may not reduce the number of AWS KMS requests. For more information, see: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html **Default value:** `false`
`bucket_name` (`string`) optional
Bucket name. If provided, the bucket will be created with this name instead of generating the name from the context **Default value:** `null`
`cors_configuration` optional
Specifies the allowed headers, methods, origins and exposed headers when using CORS on this bucket **Type:** ```hcl list(object({ id = optional(string) allowed_headers = optional(list(string)) allowed_methods = optional(list(string)) allowed_origins = optional(list(string)) expose_headers = optional(list(string)) max_age_seconds = optional(number) })) ``` **Default value:** `[ ]`
`create_s3_directory_bucket` (`bool`) optional
Control the creation of the S3 directory bucket. Set to true to create the bucket, false to skip. **Default value:** `false`
`event_notification_details` optional
S3 event notification details **Type:** ```hcl object({ enabled = bool eventbridge = optional(bool, false) lambda_list = optional(list(object({ lambda_function_arn = string events = optional(list(string), ["s3:ObjectCreated:*"]) filter_prefix = optional(string) filter_suffix = optional(string) })), []) queue_list = optional(list(object({ queue_arn = string events = optional(list(string), ["s3:ObjectCreated:*"]) filter_prefix = optional(string) filter_suffix = optional(string) })), []) topic_list = optional(list(object({ topic_arn = string events = optional(list(string), ["s3:ObjectCreated:*"]) filter_prefix = optional(string) filter_suffix = optional(string) })), []) }) ``` **Default value:** ```hcl { "enabled": false } ```
`expected_bucket_owner` (`string`) optional
Account ID of the expected bucket owner. More information: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-owner-condition.html **Default value:** `null`
`force_destroy` (`bool`) optional
When `true`, permits a non-empty S3 bucket to be deleted by first deleting all objects in the bucket. THESE OBJECTS ARE NOT RECOVERABLE even if they were versioned and stored in Glacier. **Default value:** `false`
`grants` optional
A list of policy grants for the bucket, taking a list of permissions. Conflicts with `acl`. Set `acl` to `null` to use this. Deprecated by AWS in favor of bucket policies. Automatically disabled if `s3_object_ownership` is set to "BucketOwnerEnforced". **Type:** ```hcl list(object({ id = string type = string permissions = list(string) uri = string })) ``` **Default value:** `[ ]`
`ignore_public_acls` (`bool`) optional
Set to `false` to disable the ignoring of public access lists on the bucket **Default value:** `true`
`kms_master_key_arn` (`string`) optional
The AWS KMS master key ARN used for the `SSE-KMS` encryption. This can only be used when you set the value of `sse_algorithm` as `aws:kms`. The default aws/s3 AWS KMS master key is used if this element is absent while the `sse_algorithm` is `aws:kms` **Default value:** `""`
`lifecycle_configuration_rules` optional
A list of lifecycle V2 rules **Type:** ```hcl list(object({ enabled = optional(bool, true) id = string abort_incomplete_multipart_upload_days = optional(number) # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = optional(object({ object_size_greater_than = optional(number) # integer >= 0 object_size_less_than = optional(number) # integer >= 1 prefix = optional(string) tags = optional(map(string), {}) })) expiration = optional(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer > 0 expired_object_delete_marker = optional(bool) })) noncurrent_version_expiration = optional(object({ newer_noncurrent_versions = optional(number) # integer > 0 noncurrent_days = optional(number) # integer >= 0 })) transition = optional(list(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer > 0 storage_class = optional(string) # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) noncurrent_version_transition = optional(list(object({ newer_noncurrent_versions = optional(number) # integer >= 0 noncurrent_days = optional(number) # integer >= 0 storage_class = optional(string) # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) })) ``` **Default value:** `[ ]`
`lifecycle_rule_ids` (`list(string)`) optional
DEPRECATED (use `lifecycle_configuration_rules`): A list of IDs to assign to corresponding `lifecycle_rules` **Default value:** `[ ]`
`lifecycle_rules` optional
DEPRECATED (`use lifecycle_configuration_rules`): A list of lifecycle rules **Type:** ```hcl list(object({ prefix = string enabled = bool tags = map(string) enable_glacier_transition = bool enable_deeparchive_transition = bool enable_standard_ia_transition = bool enable_current_object_expiration = bool enable_noncurrent_version_expiration = bool abort_incomplete_multipart_upload_days = number noncurrent_version_glacier_transition_days = number noncurrent_version_deeparchive_transition_days = number noncurrent_version_expiration_days = number standard_transition_days = number glacier_transition_days = number deeparchive_transition_days = number expiration_days = number })) ``` **Default value:** `null`
`logging` optional
Bucket access logging configuration. Empty list for no logging, list of 1 to enable logging. **Type:** ```hcl list(object({ bucket_name = string prefix = string })) ``` **Default value:** `[ ]`
`minimum_tls_version` (`string`) optional
Set the minimum TLS version for in-transit traffic **Default value:** `null`
`object_lock_configuration` optional
A configuration for S3 object locking. With S3 Object Lock, you can store objects using a `write once, read many` (WORM) model. Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. **Type:** ```hcl object({ mode = string # Valid values are GOVERNANCE and COMPLIANCE. days = number years = number }) ``` **Default value:** `null`
`privileged_principal_actions` (`list(string)`) optional
List of actions to permit `privileged_principal_arns` to perform on bucket and bucket prefixes (see `privileged_principal_arns`) **Default value:** `[ ]`
`privileged_principal_arns` (`list(map(list(string)))`) optional
List of maps. Each map has a key, an IAM Principal ARN, whose associated value is a list of S3 path prefixes to grant `privileged_principal_actions` permissions for that principal, in addition to the bucket itself, which is automatically included. Prefixes should not begin with '/'. **Default value:** `[ ]`
`replication_rules` (`list(any)`) optional
DEPRECATED (use `s3_replication_rules`): Specifies the replication rules for S3 bucket replication if enabled. You must also set s3_replication_enabled to true. **Default value:** `null`
`restrict_public_buckets` (`bool`) optional
Set to `false` to disable the restricting of making the bucket public **Default value:** `true`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. Defaults to "ObjectWriter" for backwards compatibility, but we recommend setting "BucketOwnerEnforced" instead. **Default value:** `"ObjectWriter"`
`s3_replica_bucket_arn` (`string`) optional
A single S3 bucket ARN to use for all replication rules. Note: The destination bucket can be specified in the replication rule itself (which allows for multiple destinations), in which case it will take precedence over this variable. **Default value:** `""`
`s3_replication_enabled` (`bool`) optional
Set this to true and specify `s3_replication_rules` to enable replication. `versioning_enabled` must also be `true`. **Default value:** `false`
`s3_replication_permissions_boundary_arn` (`string`) optional
Permissions boundary ARN for the created IAM replication role. **Default value:** `null`
`s3_replication_rules` optional
Specifies the replication rules for S3 bucket replication if enabled. You must also set s3_replication_enabled to true. **Type:** ```hcl list(object({ id = optional(string) priority = optional(number) prefix = optional(string) status = optional(string, "Enabled") # delete_marker_replication { status } had been flattened for convenience delete_marker_replication_status = optional(string, "Disabled") # Add the configuration as it appears in the resource, for consistency # this nested version takes precedence if both are provided. delete_marker_replication = optional(object({ status = string })) # destination_bucket is specified here rather than inside the destination object because before optional # attributes, it made it easier to work with the Terraform type system and create a list of consistent type. # It is preserved for backward compatibility, but the nested version takes priority if both are provided. destination_bucket = optional(string) # destination bucket ARN, overrides s3_replica_bucket_arn destination = object({ bucket = optional(string) # destination bucket ARN, overrides s3_replica_bucket_arn storage_class = optional(string, "STANDARD") # replica_kms_key_id at this level is for backward compatibility, and is overridden by the one in `encryption_configuration` replica_kms_key_id = optional(string, "") encryption_configuration = optional(object({ replica_kms_key_id = string })) access_control_translation = optional(object({ owner = string })) # account_id is for backward compatibility, overridden by account account_id = optional(string) account = optional(string) # For convenience, specifying either metrics or replication_time enables both metrics = optional(object({ event_threshold = optional(object({ minutes = optional(number, 15) # Currently 15 is the only valid number }), { minutes = 15 }) status = optional(string, "Enabled") }), { status = "Disabled" }) # To preserve backward compatibility, Replication Time Control (RTC) is automatically enabled # when metrics are enabled. To enable metrics without RTC, you must explicitly configure # replication_time.status = "Disabled". replication_time = optional(object({ time = optional(object({ minutes = optional(number, 15) # Currently 15 is the only valid number }), { minutes = 15 }) status = optional(string) })) }) source_selection_criteria = optional(object({ replica_modifications = optional(object({ status = string # Either Enabled or Disabled })) sse_kms_encrypted_objects = optional(object({ status = optional(string) })) })) # filter.prefix overrides top level prefix filter = optional(object({ prefix = optional(string) tags = optional(map(string), {}) })) })) ``` **Default value:** `null`
`s3_replication_source_roles` (`list(string)`) optional
Cross-account IAM Role ARNs that will be allowed to perform S3 replication to this bucket (for replication within the same AWS account, it's not necessary to adjust the bucket policy). **Default value:** `[ ]`
`s3_request_payment_configuration` optional
S3 request payment configuration **Type:** ```hcl object({ enabled = bool expected_bucket_owner = optional(string) payer = string }) ``` **Default value:** ```hcl { "enabled": false, "payer": "BucketOwner" } ```
`source_ip_allow_list` (`list(string)`) optional
List of IP addresses to allow to perform all actions to the bucket **Default value:** `[ ]`
`source_policy_documents` (`list(string)`) optional
List of IAM policy documents (in JSON) that are merged together into the exported document. Statements defined in source_policy_documents must have unique SIDs. Statement having SIDs that match policy SIDs generated by this module will override them. **Default value:** `[ ]`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are `AES256` and `aws:kms` **Default value:** `"AES256"`
`ssm_base_path` (`string`) optional
The base path for SSM parameters where created IAM user's access key is stored **Default value:** `"/s3_user/"`
`store_access_key_in_ssm` (`bool`) optional
Set to `true` to store the created IAM user's access key in SSM Parameter Store, `false` to store them in Terraform state as outputs. Since Terraform state would contain the secrets in plaintext, use of SSM Parameter Store is recommended. **Default value:** `false`
`transfer_acceleration_enabled` (`bool`) optional
Set this to `true` to enable S3 Transfer Acceleration for the bucket. Note: When this is set to `false` Terraform does not perform drift detection and will not disable Transfer Acceleration if it was enabled outside of Terraform. To disable it via Terraform, you must set this to `true` and then to `false`. Note: not all regions support Transfer Acceleration. **Default value:** `false`
`user_enabled` (`bool`) optional
Set to `true` to create an IAM user with permission to access the bucket **Default value:** `false`
`user_permissions_boundary_arn` (`string`) optional
Permission boundary ARN for the IAM user created to access the bucket. **Default value:** `null`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
`website_configuration` optional
Specifies the static website hosting configuration object **Type:** ```hcl list(object({ index_document = string error_document = string routing_rules = list(object({ condition = object({ http_error_code_returned_equals = string key_prefix_equals = string }) redirect = object({ host_name = string http_redirect_code = string protocol = string replace_key_prefix_with = string replace_key_with = string }) })) })) ``` **Default value:** `[ ]`
`website_redirect_all_requests_to` optional
If provided, all website requests will be redirected to the specified host name and protocol **Type:** ```hcl list(object({ host_name = string protocol = string })) ``` **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_key_id`
The access key ID, if `var.user_enabled && var.access_key_enabled`. While sensitive, it does not need to be kept secret, so this is output regardless of `var.store_access_key_in_ssm`.
`access_key_id_ssm_path`
The SSM Path under which the S3 User's access key ID is stored
`bucket_arn`
Bucket ARN
`bucket_domain_name`
FQDN of bucket
`bucket_id`
Bucket Name (aka ID)
`bucket_region`
Bucket region
`bucket_regional_domain_name`
The bucket region-specific domain name
`bucket_website_domain`
The bucket website domain, if website is enabled
`bucket_website_endpoint`
The bucket website endpoint, if website is enabled
`enabled`
Is module enabled
`replication_role_arn`
The ARN of the replication IAM Role
`secret_access_key`
The secret access key will be output if created and not stored in SSM. However, the secret access key, if created, will be written to the Terraform state file unencrypted, regardless of any other settings. See the [Terraform documentation](https://www.terraform.io/docs/state/sensitive-data.html) for more details.
`secret_access_key_ssm_path`
The SSM Path under which the S3 User's secret access key is stored
`user_arn`
The ARN assigned by AWS for the user
`user_enabled`
Is user creation enabled
`user_name`
Normalized IAM user name
`user_unique_id`
The user unique ID assigned by AWS
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0` - `time`, version: `>= 0.7` ### Providers - `aws`, version: `>= 4.9.0` - `time`, version: `>= 0.7` ### Modules Name | Version | Source | Description --- | --- | --- | --- `s3_user` | 1.2.0 | [`cloudposse/iam-s3-user/aws`](https://registry.terraform.io/modules/cloudposse/iam-s3-user/aws/1.2.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_accelerate_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_accelerate_configuration) (resource) - [`aws_s3_bucket_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) (resource) - [`aws_s3_bucket_cors_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_cors_configuration) (resource) - [`aws_s3_bucket_lifecycle_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_lifecycle_configuration) (resource) - [`aws_s3_bucket_logging.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) (resource) - [`aws_s3_bucket_notification.bucket_notification`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_notification) (resource) - [`aws_s3_bucket_object_lock_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object_lock_configuration) (resource) - [`aws_s3_bucket_ownership_controls.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_ownership_controls) (resource) - [`aws_s3_bucket_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) (resource) - [`aws_s3_bucket_public_access_block.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) (resource) - [`aws_s3_bucket_replication_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_replication_configuration) (resource) - [`aws_s3_bucket_request_payment_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_request_payment_configuration) (resource) - [`aws_s3_bucket_server_side_encryption_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) (resource) - [`aws_s3_bucket_versioning.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) (resource) - [`aws_s3_bucket_website_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_website_configuration) (resource) - [`aws_s3_bucket_website_configuration.redirect`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_website_configuration) (resource) - [`aws_s3_directory_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_directory_bucket) (resource) - [`time_sleep.wait_for_aws_s3_bucket_settings`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) (resource) ## Data Sources The following data sources are used by this module: - [`aws_canonical_user_id.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/canonical_user_id) (data source) - [`aws_iam_policy_document.aggregated_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.bucket_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.replication_sts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## s3-log-storage # Module: `s3-log-storage` This module creates an S3 bucket suitable for receiving logs from other `AWS` services such as `S3`, `CloudFront`, and `CloudTrails`. This module implements a configurable log retention policy, which allows you to efficiently manage logs across different storage classes (_e.g._ `Glacier`) and ultimately expire the data altogether. It enables [default server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-encryption.html). It [blocks public access to the bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/access-control-block-public-access.html) by default. As of March, 2022, this module is primarily a wrapper around our [s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket) module, with some options preconfigured and SQS notifications added. If it does not exactly suit your needs, you may want to use the `s3-bucket` module directly. As of version 1.0 of this module, most of the inputs are marked `nullable = false`, meaning you can pass in `null` and get the default value rather than having the input be actually set to `null`. This is technically a breaking change from previous versions, but since `null` was not a valid value for most of these variables, we are not considering it a truly breaking change. However, be mindful that the behavior of inputs set to `null` may change in the future, so we recommend setting them to the desired value explicitly. ## Usage This module supports full S3 [storage lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) configuration via our [s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket) module: ```hcl locals { lifecycle_configuration_rule = { enabled = true # bool id = "v2rule" abort_incomplete_multipart_upload_days = 1 # number filter_and = null expiration = { days = 120 # integer > 0 } noncurrent_version_expiration = { newer_noncurrent_versions = 3 # integer > 0 noncurrent_days = 60 # integer >= 0 } transition = [{ days = 30 # integer >= 0 storage_class = "STANDARD_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }, { days = 60 # integer >= 0 storage_class = "ONEZONE_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }] noncurrent_version_transition = [{ newer_noncurrent_versions = 3 # integer >= 0 noncurrent_days = 30 # integer >= 0 storage_class = "ONEZONE_IA" # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. }] } } module "log_storage" { source = "cloudposse/s3-log-storage/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "logs" stage = "test" namespace = "eg" versioning_enabled = true lifecycle_configuration_rules = [var.lifecycle_configuration_rule] } ``` ## Variables ### Required Variables
### Optional Variables
`abort_incomplete_multipart_upload_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Maximum time (in days) that you want to allow multipart uploads to remain in progress **Default value:** `null`
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where S3 access logs will be sent to **Default value:** `""`
`access_log_bucket_prefix` (`string`) optional
Prefix to prepend to the current S3 bucket name, where S3 access logs will be sent to **Default value:** `"logs/"`
`acl` (`string`) optional
The [canned ACL](https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl) to apply. Deprecated by AWS in favor of bucket policies. Automatically disabled if `s3_object_ownership` is set to "BucketOwnerEnforced". Defaults to "private" for backwards compatibility, but we recommend setting `s3_object_ownership` to "BucketOwnerEnforced" instead. **Default value:** `"log-delivery-write"`
`allow_encrypted_uploads_only` (`bool`) optional
Set to `true` to prevent uploads of unencrypted objects to S3 bucket **Default value:** `false`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `true`
`block_public_acls` (`bool`) optional
Set to `false` to disable the blocking of new public access lists on the bucket **Default value:** `true`
`block_public_policy` (`bool`) optional
Set to `false` to disable the blocking of new public policies on the bucket **Default value:** `true`
`bucket_key_enabled` (`bool`) optional
Set this to true to use Amazon S3 Bucket Keys for SSE-KMS, which reduce the cost of AWS KMS requests. For more information, see: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html **Default value:** `false`
`bucket_name` (`string`) optional
Bucket name. If provided, the bucket will be created with this name instead of generating the name from the context. **Default value:** `""`
`bucket_notifications_enabled` (`bool`) optional
Send notifications for the object created events. Used for 3rd-party log collection from a bucket **Default value:** `false`
`bucket_notifications_prefix` (`string`) optional
Prefix filter. Used to manage object notifications **Default value:** `""`
`bucket_notifications_type` (`string`) optional
Type of the notification configuration. Only SQS is supported. **Default value:** `"SQS"`
`enable_glacier_transition` (`bool`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Enables the transition to AWS Glacier which can cause unnecessary costs for huge amount of small files **Default value:** `null`
`enable_noncurrent_version_expiration` (`bool`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Enable expiration of non-current versions **Default value:** `null`
`expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to expunge the objects **Default value:** `null`
`force_destroy` (`bool`) optional
When `true`, permits a non-empty S3 bucket to be deleted by first deleting all objects in the bucket. THESE OBJECTS ARE NOT RECOVERABLE even if they were versioned and stored in Glacier. Must be set `false` unless `force_destroy_enabled` is also `true`. **Default value:** `false`
`glacier_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to move the data to the Glacier Flexible Retrieval storage tier **Default value:** `null`
`grants` optional
A list of policy grants for the bucket, taking a list of permissions. Conflicts with `acl`. Set `acl` to `null` to use this. Deprecated by AWS in favor of bucket policies, but still required for some log delivery services. Automatically disabled if `s3_object_ownership` is set to "BucketOwnerEnforced". **Type:** ```hcl list(object({ id = string type = string permissions = list(string) uri = string })) ``` **Default value:** `[ ]`
`ignore_public_acls` (`bool`) optional
Set to `false` to disable the ignoring of public access lists on the bucket **Default value:** `true`
`kms_master_key_arn` (`string`) optional
The AWS KMS master key ARN used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. The default aws/s3 AWS KMS master key is used if this element is absent while the sse_algorithm is aws:kms **Default value:** `""`
`lifecycle_configuration_rules` optional
A list of S3 bucket v2 lifecycle rules, as specified in [terraform-aws-s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket)" These rules are not affected by the deprecated `lifecycle_rule_enabled` flag. **NOTE:** Unless you also set `lifecycle_rule_enabled = false` you will also get the default deprecated rules set on your bucket. **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = optional(object({ object_size_greater_than = optional(number) # integer >= 0 object_size_less_than = optional(number) # integer >= 1 prefix = optional(string) tags = optional(map(string)) })) expiration = optional(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer > 0 expired_object_delete_marker = optional(bool) })) noncurrent_version_expiration = optional(object({ newer_noncurrent_versions = optional(number) # integer > 0 noncurrent_days = optional(number) # integer >= 0 })) transition = optional(list(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer >= 0 storage_class = string # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) noncurrent_version_transition = optional(list(object({ newer_noncurrent_versions = optional(number) # integer >= 0 noncurrent_days = optional(number) # integer >= 0 storage_class = string # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) })) ``` **Default value:** `[ ]`
`lifecycle_prefix` (`string`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Prefix filter. Used to manage object lifecycle events **Default value:** `null`
`lifecycle_rule_enabled` (`bool`) optional
DEPRECATED: Use `lifecycle_configuration_rules` instead. When `true`, configures lifecycle events on this bucket using individual (now deprecated) variables. When `false`, lifecycle events are not configured using individual (now deprecated) variables, but `lifecycle_configuration_rules` still apply. When `null`, lifecycle events are configured using individual (now deprecated) variables only if `lifecycle_configuration_rules` is empty. **Default value:** `null`
`lifecycle_tags` (`map(string)`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Tags filter. Used to manage object lifecycle events **Default value:** `null`
`noncurrent_version_expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies when non-current object versions expire (in days) **Default value:** `null`
`noncurrent_version_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies (in days) when noncurrent object versions transition to Glacier Flexible Retrieval **Default value:** `null`
`object_lock_configuration` optional
A configuration for S3 object locking. With S3 Object Lock, you can store objects using a `write once, read many` (WORM) model. Object Lock can help prevent objects from being deleted or overwritten for a fixed amount of time or indefinitely. **Type:** ```hcl object({ mode = string # Valid values are GOVERNANCE and COMPLIANCE. days = number years = number }) ``` **Default value:** `null`
`restrict_public_buckets` (`bool`) optional
Set to `false` to disable the restricting of making the bucket public **Default value:** `true`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and 'BucketOwnerEnforced'. **Default value:** `"BucketOwnerPreferred"`
`source_policy_documents` (`list(string)`) optional
List of IAM policy documents that are merged together into the exported document. Statements defined in source_policy_documents must have unique SIDs. Statement having SIDs that match policy SIDs generated by this module will override them. **Default value:** `[ ]`
`sse_algorithm` (`string`) optional
The server-side encryption algorithm to use. Valid values are AES256 and aws:kms **Default value:** `"AES256"`
`standard_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `null`
`versioning_enabled` (`bool`) optional
Enable object versioning, keeping multiple variants of an object in the same bucket **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
Bucket ARN
`bucket_domain_name`
FQDN of bucket
`bucket_id`
Bucket Name (aka ID)
`bucket_notifications_sqs_queue_arn`
Notifications SQS queue ARN
`enabled`
Is module enabled
`prefix`
Prefix configured for lifecycle rules
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0` - `time`, version: `>= 0.7` ### Providers - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `aws_s3_bucket` | 4.10.0 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.10.0) | n/a `bucket_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_s3_bucket_notification.bucket_notification`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_notification) (resource) - [`aws_sqs_queue.notifications`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sqs_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## s3-website~~ OBSOLETE # Module: `s3-website~~ OBSOLETE` ## Deprecated **As of July, 2023 this module is deprecated.** `terraform-aws-s3-website` offers little value beyond [ the `terraform-aws-s3-bucket` module](https://github.com/cloudposse/terraform-aws-s3-bucket), so Cloud Posse is phasing out support for this project. Users are advised to migrate to [terraform-aws-s3-bucket](https://github.com/cloudposse/terraform-aws-s3-bucket) to manage the S3 bucket (including logging) and [terraform-aws-route53-alias](https://github.com/cloudposse/terraform-aws-route53-alias) to register the website hostname in Route53. Feature requests should be directed to those modules. Terraform module to provision S3-backed Websites. **IMPORTANT:** This module provisions a globally accessible S3 bucket for unauthenticated users because it is designed for hosting public static websites. Normally, AWS recommends that S3 buckets should not publicly accessible in order to protect S3 data from unauthorized users. ## Usage #### Create s3 website bucket ```hcl module "website" { source = "cloudposse/s3-website/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" hostname = "docs.prod.cloudposse.org" deployment_arns = { "arn:aws:s3:::principal1" = ["/prefix1", "/prefix2"] "arn:aws:s3:::principal2" = [""] } } ``` #### Create S3 website bucket with Route53 DNS Required one of the `parent_zone_id` or `parent_zone_name` ```hcl module "website_with_cname" { source = "cloudposse/s3-website/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" hostname = "docs.prod.cloudposse.org" parent_zone_id = "XXXXXXXXXXXX" } ``` ## Variables ### Required Variables
`hostname` (`string`) required
Name of website bucket in `fqdn` format (e.g. `test.example.com`). IMPORTANT! Do not add trailing dot (`.`)
### Optional Variables
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `false`
`cors_allowed_headers` (`list(string)`) optional
List of allowed headers **Default value:** ```hcl [ "*" ] ```
`cors_allowed_methods` (`list(string)`) optional
List of allowed methods (e.g. GET, PUT, POST, DELETE, HEAD) **Default value:** ```hcl [ "GET" ] ```
`cors_allowed_origins` (`list(string)`) optional
List of allowed origins (e.g. example.com, test.com) **Default value:** ```hcl [ "*" ] ```
`cors_expose_headers` (`list(string)`) optional
List of expose header in the response **Default value:** ```hcl [ "ETag" ] ```
`cors_max_age_seconds` (`number`) optional
Time in seconds that browser can cache the response **Default value:** `3600`
`deployment_actions` (`list(string)`) optional
List of actions to permit deployment ARNs to perform **Default value:** ```hcl [ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:DeleteObject", "s3:ListBucket", "s3:ListBucketMultipartUploads", "s3:GetBucketLocation", "s3:AbortMultipartUpload" ] ```
`deployment_arns` (`map(any)`) optional
(Optional) Map of deployment ARNs to lists of S3 path prefixes to grant `deployment_actions` permissions **Default value:** `{ }`
`encryption_enabled` (`bool`) optional
When set to 'true' the resource will have AES256 encryption enabled by default **Default value:** `false`
`error_document` (`string`) optional
An absolute path to the document to return in case of a 4XX error **Default value:** `"404.html"`
`force_destroy` (`bool`) optional
Delete all objects from the bucket so that the bucket can be destroyed without error (e.g. `true` or `false`) **Default value:** `false`
`index_document` (`string`) optional
Amazon S3 returns this index document when requests are made to the root domain or any of the subfolders **Default value:** `"index.html"`
`lifecycle_rule_enabled` (`bool`) optional
Enable or disable lifecycle rule **Default value:** `false`
`logs_enabled` (`bool`) optional
Enable logs for s3 bucket **Default value:** `true`
`logs_expiration_days` (`number`) optional
Number of days after which to expunge the objects **Default value:** `90`
`logs_glacier_transition_days` (`number`) optional
Number of days after which to move the data to the glacier storage tier **Default value:** `60`
`logs_standard_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the glacier tier **Default value:** `30`
`noncurrent_version_expiration_days` (`number`) optional
Specifies when noncurrent object versions expire **Default value:** `90`
`noncurrent_version_transition_days` (`number`) optional
Number of days to persist in the standard storage tier before moving to the glacier tier infrequent access tier **Default value:** `30`
`parent_zone_id` (`string`) optional
ID of the hosted zone to contain the record **Default value:** `""`
`parent_zone_name` (`string`) optional
Name of the hosted zone to contain the record **Default value:** `""`
`prefix` (`string`) optional
Prefix identifying one or more objects to which the rule applies **Default value:** `""`
`redirect_all_requests_to` (`string`) optional
A hostname to redirect all website requests for this bucket to. If this is set `index_document` will be ignored **Default value:** `""`
`replication_source_principal_arns` (`list(string)`) optional
(Optional) List of principal ARNs to grant replication access from different AWS accounts **Default value:** `[ ]`
`routing_rules` (`string`) optional
A json array containing routing rules describing redirect behavior and when redirects are applied **Default value:** `""`
`versioning_enabled` (`bool`) optional
Enable or disable versioning **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`hostname`
Bucket hostname
`s3_bucket_arn`
ARN identifier of the website bucket
`s3_bucket_domain_name`
Name of the website bucket
`s3_bucket_hosted_zone_id`
The Route 53 Hosted Zone ID for this bucket's region
`s3_bucket_name`
DNS record of the website bucket
`s3_bucket_website_domain`
The domain of the website endpoint
`s3_bucket_website_endpoint`
The website endpoint URL
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 4.9` - `local`, version: `>= 1.2` ### Providers - `aws`, version: `>= 4.9` ### Modules Name | Version | Source | Description --- | --- | --- | --- `default_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `dns` | 0.13.0 | [`cloudposse/route53-alias/aws`](https://registry.terraform.io/modules/cloudposse/route53-alias/aws/0.13.0) | n/a `logs` | 1.4.2 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/1.4.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_ownership_controls.s3_bucket_ownership_controls`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_ownership_controls) (resource) - [`aws_s3_bucket_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) (resource) - [`aws_s3_bucket_public_access_block.s3_allow_public_access`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.deployment`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## security-group # Module: `security-group` Terraform module to create AWS Security Group and rules. ## Usage This module is primarily for setting security group rules on a security group. You can provide the ID of an existing security group to modify, or, by default, this module will create a new security group and apply the given rules to it. This module can be used very simply, but it is actually quite complex because it is attempting to handle numerous interrelationships, restrictions, and a few bugs in ways that offer a choice between zero service interruption for updates to a security group not referenced by other security groups (by replacing the security group with a new one) versus brief service interruptions for security groups that must be preserved. ### Avoiding Service Interruptions It is desirable to avoid having service interruptions when updating a security group. This is not always possible due to the way Terraform organizes its activities and the fact that AWS will reject an attempt to create a duplicate of an existing security group rule. There is also the issue that while most AWS resources can be associated with and disassociated from security groups at any time, there remain some that may not have their security group association changed, and an attempt to change their security group will cause Terraform to delete and recreate the resource. #### The 2 Ways Security Group Changes Cause Service Interruptions Changes to a security group can cause service interruptions in 2 ways: 1. Changing rules may be implemented as deleting existing rules and creating new ones. During the period between deleting the old rules and creating the new rules, the security group will block traffic intended to be allowed by the new rules. 2. Changing rules may alternately be implemented as creating a new security group with the new rules and replacing the existing security group with the new one (then deleting the old one). This usually works with no service interruption in the case where all resources that reference the security group are part of the same Terraform plan. However, if, for example, the security group ID is referenced in a security group rule in a security group that is not part of the same Terraform plan, then AWS will not allow the existing (referenced) security group to be deleted, and even if it did, Terraform would not know to update the rule to reference the new security group. The key question you need to answer to decide which configuration to use is "will anything break if the security group ID changes". If not, then use the defaults `create_before_destroy = true` and `preserve_security_group_id = false` and do not worry about providing "keys" for security group rules. This is the default because it is the easiest and safest solution when the way the security group is being used allows it. If things will break when the security group ID changes, then set `preserve_security_group_id` to `true`. Also read and follow the guidance below about [keys](#the-importance-of-keys) and [limiting Terraform security group rules to a single AWS security group rule](#terraform-rules-vs-aws-rules) if you want to mitigate against service interruptions caused by rule changes. Note that even in this case, you probably want to keep `create_before_destroy = true` because otherwise, if some change requires the security group to be replaced, Terraform will likely succeed in deleting all the security group rules but fail to delete the security group itself, leaving the associated resources completely inaccessible. At least with `create_before_destroy = true`, the new security group will be created and used where Terraform can make the changes, even though the old security group will still fail to be deleted. #### The 3 Ways to Mitigate Against Service Interruptions ##### Security Group `create_before_destroy = true` The most important option is `create_before_destroy` which, when set to `true` (the default), ensures that a new replacement security group is created before an existing one is destroyed. This is particularly important because a security group cannot be destroyed while it is associated with a resource (e.g. a load balancer), but "destroy before create" behavior causes Terraform to try to destroy the security group before disassociating it from associated resources, so plans fail to apply with the error ``` Error deleting security group: DependencyViolation: resource sg-XXX has a dependent object ``` With "create before destroy" and any resources dependent on the security group as part of the same Terraform plan, replacement happens successfully: 1. New security group is created 2. Resource is associated with the new security group and disassociated from the old one 3. Old security group is deleted successfully because there is no longer anything associated with it (If there is a resource dependent on the security group that is also outside the scope of the Terraform plan, the old security group will fail to be deleted and you will have to address the dependency manually.) Note that the module's default configuration of `create_before_destroy = true` and `preserve_security_group_id = false` will force "create before destroy" behavior on the target security group, even if the module did not create it and instead you provided a `target_security_group_id`. Unfortunately, just creating the new security group first is not enough to prevent a service interruption. Keep reading. ##### Setting Rule Changes to Force Replacement of the Security Group A security group by itself is just a container for rules. It only functions as desired when all the rules are in place. If using the Terraform default "destroy before create" behavior for rules, even when using `create_before_destroy` for the security group itself, an outage occurs when updating the rules or security group, because the order of operations is: 1. Delete existing security group rules (triggering a service interruption) 2. Create the new security group 3. Associate the new security group with resources and disassociate the old one (which can take a substantial amount of time for a resource like a NAT Gateway) 4. Create the new security group rules (restoring service) 5. Delete the old security group To resolve this issue, the module's default configuration of `create_before_destroy = true` and `preserve_security_group_id = false` causes any change in the security group rules to trigger the creation of a new security group. With that, a rule change causes operations to occur in this order: 1. Create the new security group 2. Create the new security group rules 3. Associate the new security group with resources and disassociate the old one 4. Delete the old security group rules 5. Delete the old security group ##### Preserving the Security Group There can be a downside to creating a new security group with every rule change. If you want to prevent the security group ID from changing unless absolutely necessary, perhaps because the associated resource does not allow the security group to be changed or because the ID is referenced somewhere (like in another security group's rules) outside of this Terraform plan, then you need to set `preserve_security_group_id` to `true`. The main drawback of this configuration is that there will normally be a service outage during an update, because existing rules will be deleted before replacement rules are created. Using keys to identify rules can help limit the impact, but even with keys, simply adding a CIDR to the list of allowed CIDRs will cause that entire rule to be deleted and recreated, causing a temporary access denial for all of the CIDRs in the rule. (For more on this and how to mitigate against it, see [The Importance of Keys](#the-importance-of-keys) below.) Also note that setting `preserve_security_group_id` to `true` does not prevent Terraform from replacing the security group when modifying it is not an option, such as when its name or description changes. However, if you can control the configuration adequately, you can maintain the security group ID and eliminate impact on other security groups by setting `preserve_security_group_id` to `true`. We still recommend leaving `create_before_destroy` set to `true` for the times when the security group must be replaced, to avoid the `DependencyViolation` described above. ### Defining Security Group Rules We provide a number of different ways to define rules for the security group for a few reasons: - Terraform type constraints make it difficult to create collections of objects with optional members - Terraform resource addressing can cause resources that did not actually change to nevertheless be replaced (deleted and recreated), which, in the case of security group rules, then causes a brief service interruption - Terraform resource addresses must be known at `plan` time, making it challenging to create rules that depend on resources being created during `apply` and at the same time are not replaced needlessly when something else changes - When Terraform rules can be successfully created before being destroyed, there is no service interruption for the resources associated with that security group (unless the security group ID is used in other security group rules outside of the scope of the Terraform plan) #### The Importance of Keys If you are using "create before destroy" behavior for the security group and security group rules, then you can skip this section and much of the discussion about keys in the later sections, because keys do not matter in this configuration. However, if you are using "destroy before create" behavior, then a full understanding of keys as applied to security group rules will help you minimize service interruptions due to changing rules. When creating a collection of resources, Terraform requires each resource to be identified by a key, so that each resource has a unique "address", and changes to resources are tracked by that key. Every security group rule input to this module accepts optional identifying keys (arbitrary strings) for each rule. If you do not supply keys, then the rules are treated as a list, and the index of the rule in the list will be used as its key. This has the unwelcome behavior that removing a rule from the list will cause all the rules later in the list to be destroyed and recreated. For example, changing `[A, B, C, D]` to `[A, C, D]` causes rules 1(`B`), 2(`C`), and 3(`D`) to be deleted and new rules 1(`C`) and 2(`D`) to be created. To mitigate against this problem, we allow you to specify keys (arbitrary strings) for each rule. (Exactly how you specify the key is explained in the next sections.) Going back to our example, if the initial set of rules were specified with keys, e.g. `[{A: A}, {B: B}, {C: C}, {D: D}]`, then removing `B` from the list would only cause `B` to be deleted, leaving `C` and `D` intact. Note, however, two cautions. First, the keys must be known at `terraform plan` time and therefore cannot depend on resources that will be created during `apply`. Second, in order to be helpful, the keys must remain consistently attached to the same rules. For example, if you did ```hcl rule_map = { for i, v in rule_list : i => v } ``` then you will have merely recreated the initial problem with using a plain list. If you cannot attach meaningful keys to the rules, there is no advantage to specifying keys at all. #### Terraform Rules vs AWS Rules A single security group rule input can actually specify multiple AWS security group rules. For example, `ipv6_cidr_blocks` takes a list of CIDRs. However, AWS security group rules do not allow for a list of CIDRs, so the AWS Terraform provider converts that list of CIDRs into a list of AWS security group rules, one for each CIDR. (This is the underlying cause of several AWS Terraform provider bugs, such as [#25173](https://github.com/hashicorp/terraform-provider-aws/issues/25173).) As of this writing, any change to any element of such a rule will cause all the AWS rules specified by the Terraform rule to be deleted and recreated, causing the same kind of service interruption we sought to avoid by providing keys for the rules, or, when create_before_destroy = true, causing a complete failure as Terraform tries to create duplicate rules which AWS rejects. To guard against this issue, when not using the default behavior, you should avoid the convenience of specifying multiple AWS rules in a single Terraform rule and instead create a separate Terraform rule for each source or destination specification. ##### `rules` and `rules_map` inputs This module provides 3 ways to set security group rules. You can use any or all of them at the same time. The easy way to specify rules is via the `rules` input. It takes a list of rules. (We will define a rule [a bit later](#definition-of-a-rule).) The problem is that a Terraform list must be composed of elements that are all the exact same type, and rules can be any of several different Terraform types. So to get around this restriction, the second way to specify rules is via the `rules_map` input, which is more complex.
Why the input is so complex (click to reveal) - Terraform has 3 basic simple types: bool, number, string - Terraform then has 3 collections of simple types: list, map, and set - Terraform then has 2 structural types: object and tuple. However, these are not really single types. They are catch-all labels for values that are themselves combination of other values. (This will become a bit clearer after we define `maps` and contrast them with `objects`) One [rule of the collection types](https://www.terraform.io/docs/language/expressions/type-constraints.html#collection-types) is that the values in the collections must all be the exact same type. For example, you cannot have a list where some values are boolean and some are string. Maps require that all keys be strings, but the map values can be any type, except again all the values in a map must be the same type. In other words, the values of a map must form a valid list. Objects look just like maps. The difference between an object and a map is that the values in an object do not all have to be the same type. The "type" of an object is itself an object: the keys are the same, and the values are the types of the values in the object. So although `{ foo = "bar", baz = {} }` and `{ foo = "bar", baz = [] }` are both objects, they are not of the same type, and you can get error messages like ``` Error: Inconsistent conditional result types The true and false result expressions must have consistent types. The given expressions are object and object, respectively. ``` This means you cannot put them both in the same list or the same map, even though you can put them in a single tuple or object. Similarly, and closer to the problem at hand, ```hcl cidr_rule = { type = "ingress" cidr_blocks = ["0.0.0.0/0"] } ``` is not the same type as ```hcl self_rule = { type = "ingress" self = true } ``` This means you cannot put both of those in the same list. ```hcl rules = tolist([local.cidr_rule, local.self_rule]) ``` Generates the error ```text Invalid value for "v" parameter: cannot convert tuple to list of any single type. ``` You could make them the same type and put them in a list, like this: ```hcl rules = tolist([{ type = "ingress" cidr_blocks = ["0.0.0.0/0"] self = null }, { type = "ingress" cidr_blocks = [] self = true }]) ``` That remains an option for you when generating the rules, and is probably better when you have full control over all the rules. However, what if some of the rules are coming from a source outside of your control? You cannot simply add those rules to your list. So, what to do? Create an object whose attributes' values can be of different types. ```hcl { mine = local.my_rules, theirs = var.their_rules } ``` That is why the `rules_map` input is available. It will accept a structure like that, an object whose attribute values are lists of rules, where the lists themselves can be different types.
The `rules_map` input takes an object. - The attribute names (keys) of the object can be anything you want, but need to be known during `terraform plan`, which means they cannot depend on any resources created or changed by Terraform. - The values of the attributes are lists of rule objects, each object representing one Security Group Rule. As explained above in "Why the input is so complex", each object in the list must be exactly the same type. To use multiple types, you must put them in separate lists and put the lists in a map with distinct keys. Example: ```hcl rules_map = { ingress = [{ key = "ingress" type = "ingress" from_port = 0 to_port = 2222 protocol = "tcp" cidr_blocks = module.subnets.nat_gateway_public_ips self = null description = "2222" }], egress = [{ key = "egress" type = "egress" from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] self = null description = "All output traffic" }] } ``` ###### Definition of a Rule For this module, a rule is defined as an object. - The attributes and values of the rule objects are fully compatible (have the same keys and accept the same values) as the Terraform [aws_security_group_rule resource](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule), except: - The `security_group_id` will be ignored, if present - You can include an optional `key` attribute. If present, its value must be unique among all security group rules in the security group, and it must be known in the Terraform "plan" phase, meaning it cannot depend on anything being generated or created by Terraform. The `key` attribute value, if provided, will be used to identify the Security Group Rule to Terraform in order to prevent Terraform from modifying it unnecessarily. If the `key` is not provided, Terraform will assign an identifier based on the rule's position in its list, which can cause a ripple effect of rules being deleted and recreated if a rule gets deleted from start of a list, causing all the other rules to shift position. See ["Unexpected changes..."](#unexpected-changes-during-plan-and-apply) below for more details. ##### `rule_matrix` Input The other way to set rules is via the `rule_matrix` input. This splits the attributes of the `aws_security_group_rule` resource into two sets: one set defines the rule and description, the other set defines the subjects of the rule. Again, optional "key" values can provide stability, but cannot contain derived values. This input is an attempt at convenience, and should not be used unless you are using the default settings of `create_before_destroy = true` and `preserve_security_group_id = false`, or else a number of failure modes or service interruptions are possible: use `rules_map` instead. As with `rules` and explained above in "Why the input is so complex", all elements of the list must be the exact same type. This also holds for all the elements of the `rules_matrix.rules` list. Because `rule_matrix` is already so complex, we do not provide the ability to mix types by packing object within more objects. All of the elements of the `rule_matrix` list must be exactly the same type. You can make them all the same type by following a few rules: - Every object in a list must have the exact same set of attributes. Most attributes are optional and can be omitted, but any attribute appearing in one object must appear in all the objects. - Any attribute that takes a list value in any object must contain a list in all objects. Use an empty list rather than `null` to indicate "no value". Passing in `null` instead of a list may cause Terraform to crash or emit confusing error messages (e.g. "number is required"). - Any attribute that takes a value of type other than list can be set to `null` in objects where no value is needed. The schema for `rule_matrix` is: ```hcl { # these top level lists define all the subjects to which rule_matrix rules will be applied key = an optional unique key to keep these rules from being affected when other rules change source_security_group_ids = list of source security group IDs to apply all rules to cidr_blocks = list of ipv4 CIDR blocks to apply all rules to ipv6_cidr_blocks = list of ipv6 CIDR blocks to apply all rules to prefix_list_ids = list of prefix list IDs to apply all rules to self = boolean value; set it to "true" to apply the rules to the created or existing security group, null otherwise # each rule in the rules list will be applied to every subject defined above rules = [{ key = an optional unique key to keep this rule from being affected when other rules change type = type of rule, either "ingress" or "egress" from_port = start range of protocol port to_port = end range of protocol port, max is 65535 protocol = IP protocol name or number, or "-1" for all protocols and ports description = free form text description of the rule }] } ``` ### Important Notes ##### Unexpected changes during plan and apply When configuring this module for "create before destroy" behavior, any change to a security group rule will cause an entire new security group to be created with all new rules. This can make a small change look like a big one, but is intentional and should not cause concern. As explained above under [The Importance of Keys](#the-importance-of-keys), when using "destroy before create" behavior, security group rules without keys are identified by their indices in the input lists. If a rule is deleted and the other rules therefore move closer to the start of the list, those rules will be deleted and recreated. This can make a small change look like a big one when viewing the output of Terraform plan, and will likely cause a brief (seconds) service interruption. You can avoid this for the most part by providing the optional keys, and [limiting each rule to a single source or destination](#terraform-rules-vs-aws-rules). Rules with keys will not be changed if their keys do not change and the rules themselves do not change, except in the case of `rule_matrix`, where the rules are still dependent on the order of the security groups in `source_security_group_ids`. You can avoid this by using `rules` or `rules_map` instead of `rule_matrix` when you have more than one security group in the list. You cannot avoid this by sorting the `source_security_group_ids`, because that leads to the "Invalid `for_each` argument" error because of [terraform#31035](https://github.com/hashicorp/terraform/issues/31035). ##### Invalid for_each argument You can supply a number of rules as inputs to this module, and they (usually) get transformed into `aws_security_group_rule` resources. However, Terraform works in 2 steps: a `plan` step where it calculates the changes to be made, and an `apply` step where it makes the changes. This is so you can review and approve the plan before changing anything. One big limitation of this approach is that it requires that Terraform be able to count the number of resources to create without the benefit of any data generated during the `apply` phase. So if you try to generate a rule based on something you are creating at the same time, you can get an error like ``` Error: Invalid for_each argument The "for_each" value depends on resource attributes that cannot be determined until apply, so Terraform cannot predict how many instances will be created. ``` This module uses lists to minimize the chance of that happening, as all it needs to know is the length of the list, not the values in it, but this error still can happen for subtle reasons. Most commonly, using a function like `compact` on a list will cause the length to become unknown (since the values have to be checked and `null`s removed). In the case of `source_security_group_ids`, just sorting the list using `sort` will cause this error. (See [terraform#31035](https://github.com/hashicorp/terraform/issues/31035).) If you run into this error, check for functions like `compact` somewhere in the chain that produces the list and remove them if you find them. ##### WARNINGS and Caveats **_Setting `inline_rules_enabled` is not recommended and NOT SUPPORTED_**: Any issues arising from setting `inlne_rules_enabled = true` (including issues about setting it to `false` after setting it to `true`) will not be addressed, because they flow from [fundamental problems](https://github.com/hashicorp/terraform-provider-aws/issues/20046) with the underlying `aws_security_group` resource. The setting is provided for people who know and accept the limitations and trade-offs and want to use it anyway. The main advantage is that when using inline rules, Terraform will perform "drift detection" and attempt to remove any rules it finds in place but not specified inline. See [this post](https://github.com/hashicorp/terraform-provider-aws/pull/9032#issuecomment-639545250) for a discussion of the difference between inline and resource rules, and some of the reasons inline rules are not satisfactory. **_KNOWN ISSUE_** ([#20046](https://github.com/hashicorp/terraform-provider-aws/issues/20046)): If you set `inline_rules_enabled = true`, you cannot later set it to `false`. If you try, Terraform will [complain](https://github.com/hashicorp/terraform/pull/2376) and fail. You will either have to delete and recreate the security group or manually delete all the security group rules via the AWS console or CLI before applying `inline_rules_enabled = false`. **_Objects not of the same type_**: Any time you provide a list of objects, Terraform requires that all objects in the list must be [the exact same type](https://www.terraform.io/docs/language/expressions/type-constraints.html#dynamic-types-the-quot-any-quot-constraint). This means that all objects in the list have exactly the same set of attributes and that each attribute has the same type of value in every object. So while some attributes are optional for this module, if you include an attribute in any one of the objects in a list, then you have to include that same attribute in all of them. In rules where the key would othewise be omitted, include the key with value of `null`, unless the value is a list type, in which case set the value to `[]` (an empty list), due to [#28137](https://github.com/hashicorp/terraform/issues/28137). ## Examples See [examples/complete/main.tf](https://github.com/cloudposse/terraform-aws-security-group/blob/master/examples/complete/main.tf) for even more examples. ```hcl module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" attributes = ["public"] delimiter = "-" tags = { "BusinessUnit" = "XYZ", "Snapshot" = "true" } } module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" cidr_block = "10.0.0.0/16" context = module.label.context } module "sg" { source = "cloudposse/security-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" # Security Group names must be unique within a VPC. # This module follows Cloud Posse naming conventions and generates the name # based on the inputs to the null-label module, which means you cannot # reuse the label as-is for more than one security group in the VPC. # # Here we add an attribute to give the security group a unique name. attributes = ["primary"] # Allow unlimited egress allow_all_egress = true rules = [ { key = "ssh" type = "ingress" from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] self = null # preferable to self = false description = "Allow SSH from anywhere" }, { key = "HTTP" type = "ingress" from_port = 80 to_port = 80 protocol = "tcp" cidr_blocks = [] self = true description = "Allow HTTP from inside the security group" } ] vpc_id = module.vpc.vpc_id context = module.label.context } module "sg_mysql" { source = "cloudposse/security-group/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" # Add an attribute to give the Security Group a unique name attributes = ["mysql"] # Allow unlimited egress allow_all_egress = true rule_matrix =[ # Allow any of these security groups or the specified prefixes to access MySQL { source_security_group_ids = [var.dev_sg, var.uat_sg, var.staging_sg] prefix_list_ids = [var.mysql_client_prefix_list_id] rules = [ { key = "mysql" type = "ingress" from_port = 3306 to_port = 3306 protocol = "tcp" description = "Allow MySQL access from trusted security groups" } ] } ] vpc_id = module.vpc.vpc_id context = module.label.context } ``` ## Variables ### Required Variables
`vpc_id` (`string`) required
The ID of the VPC where the Security Group will be created.
### Optional Variables
`allow_all_egress` (`bool`) optional
A convenience that adds to the rules specified elsewhere a rule that allows all egress. If this is false and no egress rules are specified via `rules` or `rule-matrix`, then no egress will be allowed. **Default value:** `true`
`create_before_destroy` (`bool`) optional
Set `true` to enable terraform `create_before_destroy` behavior on the created security group. We only recommend setting this `false` if you are importing an existing security group that you do not want replaced and therefore need full control over its name. Note that changing this value will always cause the security group to be replaced. **Default value:** `true`
`inline_rules_enabled` (`bool`) optional
NOT RECOMMENDED. Create rules "inline" instead of as separate `aws_security_group_rule` resources. See [#20046](https://github.com/hashicorp/terraform-provider-aws/issues/20046) for one of several issues with inline rules. See [this post](https://github.com/hashicorp/terraform-provider-aws/pull/9032#issuecomment-639545250) for details on the difference between inline rules and rule resources. **Default value:** `false`
`preserve_security_group_id` (`bool`) optional
When `false` and `create_before_destroy` is `true`, changes to security group rules cause a new security group to be created with the new rules, and the existing security group is then replaced with the new one, eliminating any service interruption. When `true` or when changing the value (from `false` to `true` or from `true` to `false`), existing security group rules will be deleted before new ones are created, resulting in a service interruption, but preserving the security group itself. **NOTE:** Setting this to `true` does not guarantee the security group will never be replaced, it only keeps changes to the security group rules from triggering a replacement. See the README for further discussion. **Default value:** `false`
`revoke_rules_on_delete` (`bool`) optional
Instruct Terraform to revoke all of the Security Group's attached ingress and egress rules before deleting the security group itself. This is normally not needed. **Default value:** `false`
`rule_matrix` (`any`) optional
A convenient way to apply the same set of rules to a set of subjects. See README for details. **Default value:** `[ ]`
`rules` (`list(any)`) optional
A list of Security Group rule objects. All elements of a list must be exactly the same type; use `rules_map` if you want to supply multiple lists of different types. The keys and values of the Security Group rule objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see the `security_group_rule` [documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule). ___Note:___ The length of the list must be known at plan time. This means you cannot use functions like `compact` or `sort` when computing the list. **Default value:** `[ ]`
`rules_map` (`any`) optional
A map-like object of lists of Security Group rule objects. All elements of a list must be exactly the same type, so this input accepts an object with keys (attributes) whose values are lists so you can separate different types into different lists and still pass them into one input. Keys must be known at "plan" time. The keys and values of the Security Group rule objects are fully compatible with the `aws_security_group_rule` resource, except for `security_group_id` which will be ignored, and the optional "key" which, if provided, must be unique and known at "plan" time. To get more info see the `security_group_rule` [documentation](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule). **Default value:** `{ }`
`security_group_create_timeout` (`string`) optional
How long to wait for the security group to be created. **Default value:** `"10m"`
`security_group_delete_timeout` (`string`) optional
How long to retry on `DependencyViolation` errors during security group deletion from lingering ENIs left by certain AWS services such as Elastic Load Balancing. **Default value:** `"15m"`
`security_group_description` (`string`) optional
The description to assign to the created Security Group. Warning: Changing the description causes the security group to be replaced. **Default value:** `"Managed by Terraform"`
`security_group_name` (`list(string)`) optional
The name to assign to the security group. Must be unique within the VPC. If not provided, will be derived from the `null-label.context` passed in. If `create_before_destroy` is true, will be used as a name prefix. **Default value:** `[ ]`
`target_security_group_id` (`list(string)`) optional
The ID of an existing Security Group to which Security Group rules will be assigned. The Security Group's name and description will not be changed. Not compatible with `inline_rules_enabled` or `revoke_rules_on_delete`. If not provided (the default), this module will create a security group. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The created Security Group ARN (null if using existing security group)
`id`
The created or target Security Group ID
`name`
The created Security Group Name (null if using existing security group)
`rules_terraform_ids`
List of Terraform IDs of created `security_group_rule` resources, primarily provided to enable `depends_on`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 3.0` - `null`, version: `>= 3.0` - `random`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` - `null`, version: `>= 3.0` - `random`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_security_group.cbd`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) (resource) - [`aws_security_group_rule.dbc`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`aws_security_group_rule.keyed`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) (resource) - [`null_resource.sync_rules_and_sg_lifecycles`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`random_id.rule_change_forces_new_security_group`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) (resource) ## Data Sources The following data sources are used by this module: --- ## security-hub(Security-hub) # Module: `security-hub` Terraform module to deploy [AWS Security Hub](https://aws.amazon.com/security-hub/). ## Introduction This module enables AWS Security Hub in one region of one account and optionally sets up an SNS topic to receive notifications of its findings. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-security-hub/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-security-hub/tree/main/test). Here's how to invoke this module in your projects: ```hcl module "securityhub" { source = "cloudposse/security-hub/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" create_sns_topic = true subscribers = { opsgenie = { protocol = "https" endpoint = "https://api.example.com/v1/" endpoint_auto_confirms = true raw_message_delivery = false } } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-security-hub/tree/master/examples/complete/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`cloudwatch_event_rule_pattern_detail_type` (`string`) optional
The detail-type pattern used to match events that will be sent to SNS. For more information, see: https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/CloudWatchEventsandEventPatterns.html **Default value:** `"Security Hub Findings - Imported"`
`create_sns_topic` (`bool`) optional
Flag to indicate whether an SNS topic should be created for notifications If you want to send findings to a new SNS topic, set this to true and provide a valid configuration for subscribers **Default value:** `false`
`enable_default_standards` (`bool`) optional
Flag to indicate whether default standards should be enabled **Default value:** `true`
`enabled_standards` (`list(any)`) optional
A list of standards/rulesets to enable See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_standards_subscription#argument-reference The possible values are: - standards/aws-foundational-security-best-practices/v/1.0.0 - ruleset/cis-aws-foundations-benchmark/v/1.2.0 - standards/pci-dss/v/3.2.1 **Default value:** `[ ]`
`finding_aggregator_enabled` (`bool`) optional
Flag to indicate whether a finding aggregator should be created If you want to aggregate findings from one region, set this to `true`. For more information, see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_finding_aggregator **Default value:** `false`
`finding_aggregator_linking_mode` (`string`) optional
Linking mode to use for the finding aggregator. The possible values are: - `ALL_REGIONS` - Aggregate from all regions - `ALL_REGIONS_EXCEPT_SPECIFIED` - Aggregate from all regions except those specified in `var.finding_aggregator_regions` - `SPECIFIED_REGIONS` - Aggregate from regions specified in `finding_aggregator_enabled` **Default value:** `"ALL_REGIONS"`
`finding_aggregator_regions` (`list(string)`) optional
A list of regions to aggregate findings from. This is only used if `finding_aggregator_enabled` is `true`. **Default value:** `[ ]`
`imported_findings_notification_arn` (`string`) optional
The ARN for an SNS topic to send findings notifications to. This is only used if create_sns_topic is false. If you want to send findings to an existing SNS topic, set the value of this to the ARN of the existing topic and set create_sns_topic to false. **Default value:** `null`
`subscribers` optional
Required configuration for subscibres to SNS topic. **Type:** ```hcl map(object({ protocol = string # The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see below) (email is an option but is unsupported, see below). endpoint = string # The endpoint to send data to, the contents will vary with the protocol. (see below for more information) endpoint_auto_confirms = bool # Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty (default is false) raw_message_delivery = bool # Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property) (default is false) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`enabled_subscriptions`
A list of subscriptions that have been enabled
`sns_topic`
The SNS topic that was created
`sns_topic_subscriptions`
The SNS topic that was created
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2` ### Providers - `aws`, version: `>= 2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `imported_findings_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sns_kms_key` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `sns_kms_key_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `sns_topic` | 0.21.0 | [`cloudposse/sns-topic/aws`](https://registry.terraform.io/modules/cloudposse/sns-topic/aws/0.21.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_event_rule.imported_findings`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_rule) (resource) - [`aws_cloudwatch_event_target.imported_findings`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_event_target) (resource) - [`aws_securityhub_account.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_account) (resource) - [`aws_securityhub_finding_aggregator.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_finding_aggregator) (resource) - [`aws_securityhub_standards_subscription.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/securityhub_standards_subscription) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.sns_kms_key_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_region.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## control-disablements # Summary There are certain controls that only make sense to monitor once per account, not once per region per account. As an example, imagine you wanted to verify an IAM Role exists for a given account. That wouldn't need to be verified in each region for the account. As such, this module takes your current AWS Provider configuration and uses it to generate and output a list of controls to disable. Because this list doesn't change much, we have a static definition of the controls to disable in the `main.tf` file. # Scope Right now, this module only targets CIS 1.2 controls. # Usage ```hcl module "control_disablements" { source = "cloudposse/security-hub/aws//modules/control-disablements" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" global_resource_collector_region = var.global_resource_collector_region # i.e. us-east-1 central_logging_account = local.central_logging_account environment = "" context = module.this.context } resource "awsutils_security_hub_control_disablement" "global" { for_each = toset(module.control_disablements.controls) control_arn = each.key reason = "Global and CloudTrail resources are not collected in this account/region" depends_on = [ module.security_hub ] } ``` # Terraform Docs ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.0 | | [aws](#requirement\_aws) | >= 2 | ## Providers | Name | Version | |------|---------| | [aws](#provider\_aws) | >= 2 | ## Modules | Name | Source | Version | |------|--------|---------| | [this](#module\_this) | cloudposse/label/null | 0.24.1 | ## Resources | Name | Type | |------|------| | [aws_caller_identity.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source | | [aws_partition.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source | | [aws_region.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional tags for appending to tags\_as\_list\_of\_maps. Not added to `tags`. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | Additional attributes (e.g. `1`) | `list(string)` | `[]` | no | | [central\_logging\_account](#input\_central\_logging\_account) | The id of the account that is the centralized cloudtrail logging account. | `string` | n/a | yes | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\}\}
| no | | [delimiter](#input\_delimiter) | Delimiter to be used between `namespace`, `environment`, `stage`, `name` and `attributes`.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | Environment, e.g. 'uw2', 'us-west-2', OR 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [global\_resource\_collector\_region](#input\_global\_resource\_collector\_region) | The region that collects AWS Config data for global resources such as IAM | `string` | n/a | yes | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for default, which is `0`.Does not affect `id_full`. | `number` | `null` | no | | [label\_key\_case](#input\_label\_key\_case) | The letter case of label keys (`tag` names) (i.e. `name`, `namespace`, `environment`, `stage`, `attributes`) to use in `tags`.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The naming order of the id output and Name tag.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 5 elements, but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | The letter case of output label values (also used in `tags` and `id`).Possible values: `lower`, `title`, `upper` and `none` (no transformation).Default value: `lower`. | `string` | `null` | no | | [name](#input\_name) | Solution name, e.g. 'app' or 'jenkins' | `string` | `null` | no | | [namespace](#input\_namespace) | Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' | `string` | `null` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Regex to replace chars with empty string in `namespace`, `environment`, `stage` and `name`.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [stage](#input\_stage) | Stage, e.g. 'prod', 'staging', 'dev', OR 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit','XYZ')` | `map(string)` | `{}` | no | ## Outputs | Name | Description | |------|-------------| | [controls](#output\_controls) | A list of controls to disable based on the the input variables | --- ## service-control-policies # Module: `service-control-policies` Terraform module to provision Service Control Policies (SCP) for AWS Organizations, Organizational Units, and AWS accounts. ## Introduction Service Control Policies are configured in YAML configuration files. We maintain a comprehensive [catalog](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/main/catalog) of SCP configurations and welcome contributions via pull request! The [example](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/main/examples/complete) in this module uses the catalog to provision the SCPs on AWS. The policies in the `catalog/*-templates` files require parameters supplied via the `parameters` input to [terraform-yaml-config](https://github.com/cloudposse/terraform-yaml-config). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on Datadog), see [test](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/main/test). ```hcl module "yaml_config" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" list_config_local_base_path = path.module list_config_paths = ["catalog/*.yaml"] context = module.this.context } module "yaml_config_with_parameters" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" list_config_local_base_path = path.module list_config_paths = ["https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/0.12.0/catalog/s3-templates/DenyS3InNonSelectedRegion.yaml"] parameters = { "s3_regions_lockdown" = "us-*,eu-north-1" } context = module.this.context } data "aws_caller_identity" "this" {} module "service_control_policies" { source = "../../" service_control_policy_statements = concat(module.yaml_config.list_configs, module.yaml_config_with_parameters.list_configs) service_control_policy_description = var.service_control_policy_description target_id = data.aws_caller_identity.this.account_id context = module.this.context } ``` ## Examples Review the [complete example](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/main/examples/complete) to see how to use this module. ## Variables ### Required Variables
`service_control_policy_statements` (`any`) required
List of Service Control Policy statements
`target_id` (`string`) required
The unique identifier (ID) of the organization root, organizational unit, or account number that you want to attach the policy to
### Optional Variables
`service_control_policy_description` (`string`) optional
Description of the combined Service Control Policy **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`organizations_policy_arn`
Amazon Resource Name (ARN) of the policy
`organizations_policy_id`
The unique identifier of the policy
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 3.0` - `local`, version: `>= 1.3` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_organizations_policy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_policy) (resource) - [`aws_organizations_policy_attachment.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/organizations_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## service-quotas # Module: `service-quotas` Terraform module to manage [AWS Service Quotas](https://docs.aws.amazon.com/servicequotas/latest/userguide/intro.html). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-service-quotas/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-service-quotas/tree/main/test). `NOTE:` Some service quotas can only be managed from specific regions (see [hashicorp/terraform-provider-aws#13075](https://github.com/hashicorp/terraform-provider-aws/issues/13075)) ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } locals { service_quotas = [ { quota_code = "L-93826ACB" # aka `Routes per route table` service_code = "vpc" value = 100 # since this is non-null, the module should try to create a service quota for this value }, { quota_name = "Subnets per VPC" # aka `L-44499CD2` service_code = "vpc" value = 250 # since this is non-null, the module will find the `quota_code` and try to create a service quota for this value }, { quota_code = "L-F678F1CE" # aka `VPC per Region` service_code = "vpc" value = null # since this is null, the module should try to lookup the value of this service quota, it should be default }, { quota_name = "VPC security groups per Region" # aka `L-E79EC296` service_code = "vpc" value = null # since this is null, the module should try to lookup the value of this service quota, it should be default } ] } module "service_quotas" { source = "cloudposse/service-quotas/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" service_quotas = local.service_quotas context = module.label.this } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-service-quotas/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`service_quotas` (`list(any)`) optional
A list of service quotas to manage or lookup. To lookup the value of a service quota, set `value = null` and either `quota_code` or `quota_name`. To manage a service quota, set `value` to a number. Service Quotas can only be managed via `quota_code`. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`service_quotas`
List of service quotas that are being managed by this module
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_servicequotas_service_quota.managed_by_code`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/servicequotas_service_quota) (resource) - [`aws_servicequotas_service_quota.managed_by_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/servicequotas_service_quota) (resource) ## Data Sources The following data sources are used by this module: - [`aws_servicequotas_service_quota.lookup_quota_by_code`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/servicequotas_service_quota) (data source) - [`aws_servicequotas_service_quota.lookup_quota_by_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/servicequotas_service_quota) (data source) - [`aws_servicequotas_service_quota.managed_by_name`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/servicequotas_service_quota) (data source) --- ## ses(Ses) # Module: `ses` Terraform module to provision Simple Email Service on AWS. ## Usage This module creates a SES domain with IAM user that is able to send emails with it. If module is provided with Route53 Zone ID it can also create verification DNS records for domain and DKIM. For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ses/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ses/tree/main/test). What's worth to note is that every AWS SES starts in Sandbox. Sending emails via it (emails not verified in AWS SES) is only allowed after support request. SES availability in regions can be checked in [latest AWS General Reference](https://docs.aws.amazon.com/general/latest/gr/ses.html) ## Variables ### Required Variables
`domain` (`string`) required
The domain to create the SES identity for.
### Optional Variables
`create_spf_record` (`bool`) optional
If provided the module will create an SPF record for `domain`. **Default value:** `false`
`custom_from_behavior_on_mx_failure` (`string`) optional
The behaviour of the custom_from_subdomain when the MX record is not found. Defaults to `UseDefaultValue`. **Default value:** `"UseDefaultValue"`
`custom_from_dns_record_enabled` (`bool`) optional
If enabled the module will create a Route53 DNS record for the `From` address subdomain. **Default value:** `true`
`custom_from_subdomain` (`list(string)`) optional
If provided the module will create a custom subdomain for the `From` address. **Default value:** `[ ]`
`iam_access_key_max_age` (`number`) optional
Maximum age of IAM access key (seconds). Defaults to 30 days. Set to 0 to disable expiration. **Default value:** `2592000`
`iam_allowed_resources` (`list(string)`) optional
Specifies resource ARNs that are enabled for `var.iam_permissions`. Wildcards are acceptable. **Default value:** `[ ]`
`iam_permissions` (`list(string)`) optional
Specifies permissions for the IAM user. **Default value:** ```hcl [ "ses:SendRawEmail" ] ```
`ses_group_enabled` (`bool`) optional
Creates a group with permission to send emails from SES domain **Default value:** `true`
`ses_group_name` (`string`) optional
The name of the IAM group to create. If empty the module will calculate name from a context (recommended). **Default value:** `""`
`ses_group_path` (`string`) optional
The IAM Path of the group to create **Default value:** `"/"`
`ses_user_enabled` (`bool`) optional
Creates user with permission to send emails from SES domain **Default value:** `true`
`verify_dkim` (`bool`) optional
If provided the module will create Route53 DNS records used for DKIM verification. **Default value:** `false`
`verify_domain` (`bool`) optional
If provided the module will create Route53 DNS records used for domain verification. **Default value:** `false`
`zone_id` (`string`) optional
Route53 parent zone ID. If provided (not empty), the module will create Route53 DNS records used for verification **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`access_key_id`
The SMTP user which is access key ID.
`custom_from_domain`
The custom mail FROM domain
`secret_access_key`
The IAM secret for usage with SES API. This will be written to the state file in plain text.
`ses_dkim_tokens`
A list of DKIM Tokens which, when added to the DNS Domain as CNAME records, allows for receivers to verify that emails were indeed authorized by the domain owner.
`ses_domain_identity_arn`
The ARN of the SES domain identity
`ses_domain_identity_verification_token`
A code which when added to the domain as a TXT record will signal to SES that the owner of the domain has authorised SES to act on their behalf. The domain identity will be in state 'verification pending' until this is done. See below for an example of how this might be achieved when the domain is hosted in Route 53 and managed by Terraform. Find out more about verifying domains in Amazon SES in the AWS SES docs.
`ses_group_name`
The IAM group name
`ses_smtp_password`
The SMTP password. This will be written to the state file in plain text.
`spf_record`
The SPF record for the domain. This is a TXT record that should be added to the domain's DNS settings to allow SES to send emails on behalf of the domain.
`user_arn`
The ARN assigned by AWS for this user.
`user_name`
Normalized IAM user name.
`user_unique_id`
The unique ID assigned by AWS.
## Dependencies ### Requirements - `terraform`, version: `>= 1.1.0` - `aws`, version: `>= 2.0` - `awsutils`, version: `>= 0.11.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ses_user` | 0.23.2 | [`cloudposse/iam-system-user/aws`](https://registry.terraform.io/modules/cloudposse/iam-system-user/aws/0.23.2) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_group.ses_users`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group) (resource) - [`aws_iam_group_policy.ses_group_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_group_policy) (resource) - [`aws_iam_user_group_membership.ses_user`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_group_membership) (resource) - [`aws_iam_user_policy.sending_emails`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_user_policy) (resource) - [`aws_route53_record.amazonses_dkim_record`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.amazonses_spf_record`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.amazonses_verification_record`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.custom_mail_from_mx`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_ses_domain_dkim.ses_domain_dkim`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_dkim) (resource) - [`aws_ses_domain_identity.ses_domain`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_identity) (resource) - [`aws_ses_domain_mail_from.custom_mail_from`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_mail_from) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.ses_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## ses-lambda-forwarder # Module: `ses-lambda-forwarder` This is a terraform module that creates an email forwarder using a combination of AWS SES and Lambda running the [aws-lambda-ses-forwarder](https://www.npmjs.com/package/aws-lambda-ses-forwarder) NPM module. ## Introduction This module provisions a NodeJS script as a AWS Lambda function that uses the inbound/outbound capabilities of AWS Simple Email Service (SES) to run a "serverless" email forwarding service. Use this module instead of setting up an email server on a dedicated EC2 instance to handle email redirects. It uses AWS SES to receive email and then trigger a Lambda function to process it and forward it on to the chosen destination. This script will allow forwarding emails from any sender to verified destination emails (e.g. opt-in). ## Limitations The SES service only allows sending email from verified addresses or domains. As such, it's mostly suitable for transactional emails (e.g. alerts or notifications). The incoming messages are modified to allow forwarding through SES and reflect the original sender. This script adds a `Reply-To` header with the original sender's email address, but the `From` header is changed to display the SES email address. For example, an email sent by `John Doe ` to `hello@example.com` will be transformed to: ``` From: John Doe at john@example.com Reply-To: john@example.com ``` To override this behavior, set a verified `fromEmail` address (e.g., `noreply@example.com`) in the config object and the header will look like this. ``` From: John Doe Reply-To: john@example.com ``` __NOTE__: SES only allows receiving email sent to addresses within verified domains. For more information, see: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/verify-domains.html ``` Initially SES users are in a sandbox environment that has a number of limitations. See: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/limits.html ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ses-lambda-forwarder/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ses-lambda-forwarder/tree/main/test). ```hcl variable "relay_email" \{ default = "example@example.com" description = "Email that used to relay from" \} variable "forward_emails" \{ type = map(list(string)) default = \{ "some_email@example.com" = ["my_email@example.com"] "other_email@example.com" = ["my_email@example.com"] \} description = "Emails forward map" \} module "ses_lambda_forwarder" \{ source = "cloudposse/ses-lambda-forwarder/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = var.namespace stage = var.stage name = var.name delimiter = var.delimiter attributes = var.attributes tags = var.tags region = var.region domain = var.domain relay_email = var.relay_email forward_emails = var.forward_emails \} ``` ## Variables ### Required Variables
`domain` (`string`) required
Root domain name
`region` (`string`) required
AWS Region the SES should reside in
`relay_email` (`string`) required
Email that used to relay from
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where s3 access log will be sent to **Default value:** `""`
`artifact_filename` (`string`) optional
Artifact filename **Default value:** `"lambda.zip"`
`artifact_url` (`string`) optional
URL template for the remote artifact **Default value:** `"https://artifacts.cloudposse.com/$${module_name}/$${git_ref}/$${filename}"`
`forward_emails` (`map(list(string))`) optional
Map of forward emails **Default value:** ```hcl \{ "ops@example.com": [ "destination@example.com" ] \} ```
`lambda_runtime` (`string`) optional
Lambda runtime **Default value:** `"nodejs12.x"`
`s3_bucket_encryption_enabled` (`bool`) optional
When set to 'true' the 'aws_s3_bucket' resource will have AES256 encryption enabled by default **Default value:** `true`
`spf` (`string`) optional
DNS SPF record value **Default value:** `"v=spf1 include:amazonses.com -all"`
`tracing_config_mode` (`string`) optional
Can be either PassThrough or Active. If PassThrough, Lambda will only trace the request from an upstream service if it contains a tracing header with 'sampled=1'. If Active, Lambda will respect any tracing header it receives from an upstream service. **Default value:** `"PassThrough"`
`versioning_enabled` (`bool`) optional
A state of versioning. Versioning is a means of keeping multiple variants of an object in the same bucket **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl \{ "additional_tag_map": \{\}, "attributes": [], "delimiter": null, "descriptor_formats": \{\}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": \{\}, "tenant": null \} ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `{ format = string labels = list(string) }` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`artifact_base64sha256`
Base64 encoded SHA256 hash of the artifact file
`artifact_file`
Full path to the locally downloaded artifact
`artifact_git_ref`
Git commit hash corresponding to the artifact
`artifact_url`
URL corresponding to the artifact
`lambda_function_arn`
Lamnda Function ARN
`lambda_function_source_code_size`
The size in bytes of the Lambda Function .zip file
`lambda_function_version`
Latest published version of the Lambda Function
`lambda_iam_policy_arn`
Lamnda IAM Policy ARN
`lambda_iam_policy_id`
Lamnda IAM Policy ID
`lambda_iam_policy_name`
Lamnda IAM Policy name
`s3_bucket_arn`
Lamnda IAM Policy ARN
`s3_bucket_domain_name`
Lamnda IAM Policy ARN
`s3_bucket_id`
Lamnda IAM Policy name
`ses_domain_identity_arn`
The ARN of the domain identity
`ses_domain_identity_verification_arn`
The ARN of the domain identity
`ses_receipt_rule_name`
The name of the SES receipt rule
`ses_receipt_rule_set_name`
The name of the SES receipt rule set
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `external`, version: `>= 1.2` - `local`, version: `>= 1.3` - `template`, version: `>= 2.2` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `artifact` | 0.8.0 | [`cloudposse/module-artifact/external`](https://registry.terraform.io/modules/cloudposse/module-artifact/external/0.8.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_lambda_alias.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_alias) (resource) - [`aws_lambda_function.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_function) (resource) - [`aws_lambda_permission.ses`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_permission) (resource) - [`aws_route53_record.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.mx`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_route53_record.txt`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) (resource) - [`aws_ses_active_receipt_rule_set.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_active_receipt_rule_set) (resource) - [`aws_ses_domain_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_identity) (resource) - [`aws_ses_domain_identity_verification.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_domain_identity_verification) (resource) - [`aws_ses_receipt_rule.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_receipt_rule) (resource) - [`aws_ses_receipt_rule_set.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ses_receipt_rule_set) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.assume`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.lambda`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_route53_zone.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) (data source) --- ## sns-cloudwatch-sns-alarms # Module: `sns-cloudwatch-sns-alarms` Terraform module to provision CloudWatch alarms for SNS ## Examples ```hcl module "sns_monitoring" { source = "git::https://github.com/cloudposse/terraform-aws-sns-cloudwatch-sns-alarms.git?ref=0.0.1" enabled = var.monitoring_enabled sns_topic_name = module.sns.sns_topic.name sns_topic_alarms_arn = module.sns.sns_topic.arn } ``` ## Variables ### Required Variables
`sns_topic_alarms_arn` (`string`) required
ARN of SNS topic that will be subscribed to an alarm.
`sns_topic_name` (`string`) required
Name of SNS topic to be monitored.
### Optional Variables
`alarm_on_sns_failed_notifications_datapoints_to_alarm` (`number`) optional
The number of datapoints in CloudWatch Metric statistic, which triggers the alarm. **Default value:** `1`
`alarm_on_sns_failed_notifications_evaluation_periods` (`number`) optional
The number of periods over which data is compared to the specified threshold. **Default value:** `1`
`alarm_on_sns_failed_notifications_period` (`number`) optional
The period, in seconds, over which to measure. **Default value:** `300`
`alarm_on_sns_failed_notifications_statistic` (`string`) optional
The statistic to apply to the alarm's associated metric. [SampleCount, Average, Sum, Minimum, Maximum] **Default value:** `"Sum"`
`alarm_on_sns_failed_notifications_threshold` (`number`) optional
Threshold for failed notifications on SNS topic. By default it will trigger on any failure. **Default value:** `0`
`alarm_on_sns_failed_notifications_treat_missing_data` (`string`) optional
Sets how this alarm is to handle missing data points. The following values are supported: missing, ignore, breaching and notBreaching. **Default value:** `"notBreaching"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_metric_alarm.alarm_on_sns_failed_notifications`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) (resource) ## Data Sources The following data sources are used by this module: --- ## sns-lambda-notify-slack # Module: `sns-lambda-notify-slack` Terraform module to provision a lambda function that subscribes to SNS and notifies to Slack. ## Usage ```hcl module "notify_slack" { source = "git::https://github.com/cloudposse/terraform-aws-sns-lambda-notify-slack?ref=tags/0.1.0" namespace = "eg" stage = "staging" name = "app" slack_webhook_url = "https://hooks.slack.com/services/AAAAAAAA/BBBBBBBB/CCCCCCC" slack_channel = "aws-services" slack_username = "reporter" } ``` ## Variables ### Required Variables
`slack_channel` (`string`) required
The name of the channel in Slack for notifications
`slack_username` (`string`) required
The username that will appear on Slack messages
`slack_webhook_url` (`string`) required
The URL of Slack webhook
### Optional Variables
`create_sns_topic` (`bool`) optional
Whether to create new SNS topic **Default value:** `true`
`kms_key_arn` (`string`) optional
ARN of the KMS key used for decrypting slack webhook url **Default value:** `""`
`lambda_source_path` (`string`) optional
The source path of the custom Lambda function **Default value:** `null`
`slack_emoji` (`string`) optional
A custom emoji that will appear on Slack messages **Default value:** `":aws:"`
`sns_topic_name` (`string`) optional
Name of the SNS topic to subscribe to. **Default value:** `""`
`vpc_security_group_ids` (`list(string)`) optional
List of security group ids when the notifying Lambda Function should run in the VPC. **Default value:** `null`
`vpc_subnet_ids` (`list(string)`) optional
List of subnet ids when the notifying Lambda Function should run in the VPC. Usually private or intra subnets. **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`lambda_cloudwatch_log_group_arn`
The Amazon Resource Name (ARN) specifying the log group
`lambda_iam_role_arn`
The ARN of the IAM role used by Lambda function
`lambda_iam_role_name`
The name of the IAM role used by Lambda function
`notify_slack_lambda_function_arn`
The ARN of the Lambda function
`notify_slack_lambda_function_invoke_arn`
The ARN to be used for invoking Lambda function from API Gateway
`notify_slack_lambda_function_last_modified`
The date Lambda function was last modified
`notify_slack_lambda_function_name`
The name of the Lambda function
`notify_slack_lambda_function_version`
Latest published version of your Lambda function
`slack_topic_arn`
The ARN of the SNS topic from which messages will be sent to Slack
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `archive`, version: `>=1.3` - `aws`, version: `>= 2.0` - `local`, version: `>= 1.3` - `null`, version: `>= 2.0` - `template`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `default_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `notify_slack` | 7.0.0 | [`terraform-aws-modules/notify-slack/aws`](https://registry.terraform.io/modules/terraform-aws-modules/notify-slack/aws/7.0.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a --- ## sns-topic(Sns-topic) # Module: `sns-topic` Terraform module to provision SNS topic ## Introduction This module provides: - SNS topic creation - SNS topic policy - SNS topic subscriptions It's possible to subscribe SQS as Dead Letter Queue. ## Usage Amazon Simple Notification Service (Amazon SNS) is a web service that coordinates and manages the delivery or sending of messages to subscribing endpoints or clients. [SNS documentation](https://docs.aws.amazon.com/sns/latest/dg/sns-how-it-works.html) There are many ways SNS can be used. As an example we can imagine CloudWatch sending alerts to SNS, by using subscribers such notifications can be sent further to PagerDuty, OpsGenie or any other oncall management tool. ## Examples ```hcl module "sns" { source = "cloudposse/sns-topic/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = var.attributes name = var.name namespace = var.namespace stage = var.stage subscribers = { opsgenie = { protocol = "https" endpoint = "https://api.example.com/v1/" endpoint_auto_confirms = true } } sqs_dlq_enabled = false } ``` ## Variables ### Required Variables
### Optional Variables
`allowed_aws_services_for_sns_published` (`list(string)`) optional
AWS services that will have permission to publish to SNS topic. Used when no external JSON policy is used **Default value:** `[ ]`
`allowed_iam_arns_for_sns_publish` (`list(string)`) optional
IAM role/user ARNs that will have permission to publish to SNS topic. Used when no external json policy is used. **Default value:** `[ ]`
`content_based_deduplication` (`bool`) optional
Enable content-based deduplication for FIFO topics **Default value:** `false`
`delivery_policy` (`string`) optional
The SNS delivery policy as JSON. **Default value:** `null`
`delivery_status` optional
Enable Message delivery status for the various SNS subscription endpoints. The success_role_arn and failure_role_arn arguments are used to give Amazon SNS write access to use CloudWatch Logs on your behalf. The success_sample_rate argument is for specifying the sample rate percentage (0-100) of successfully delivered messages. **Type:** ```hcl object({ application = optional(object({ success_role_arn = string failure_role_arn = string success_sample_rate = number })) firehose = optional(object({ success_role_arn = string failure_role_arn = string success_sample_rate = number })) http = optional(object({ success_role_arn = string failure_role_arn = string success_sample_rate = number })) lambda = optional(object({ success_role_arn = string failure_role_arn = string success_sample_rate = number })) sqs = optional(object({ success_role_arn = string failure_role_arn = string success_sample_rate = number })) }) ``` **Default value:** `{ }`
`encryption_enabled` (`bool`) optional
Whether or not to use encryption for SNS Topic. If set to `true` and no custom value for KMS key (kms_master_key_id) is provided, it uses the default `alias/aws/sns` KMS key. **Default value:** `true`
`fifo_queue_enabled` (`bool`) optional
Whether or not to create a FIFO (first-in-first-out) queue **Default value:** `false`
`fifo_topic` (`bool`) optional
Whether or not to create a FIFO (first-in-first-out) topic **Default value:** `false`
`kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SNS or a custom CMK. **Default value:** `"alias/aws/sns"`
`redrive_policy` (`string`) optional
The SNS redrive policy as JSON. This overrides the `deadLetterTargetArn` (supplied by `var.fifo_queue = true`) passed in by the module. **Default value:** `null`
`sns_topic_policy_json` (`string`) optional
The fully-formed AWS policy as JSON **Default value:** `""`
`sqs_dlq_enabled` (`bool`) optional
Enable delivery of failed notifications to SQS and monitor messages in queue. **Default value:** `false`
`sqs_dlq_max_message_size` (`number`) optional
The limit of how many bytes a message can contain before Amazon SQS rejects it. An integer from 1024 bytes (1 KiB) up to 262144 bytes (256 KiB). The default for this attribute is 262144 (256 KiB). **Default value:** `262144`
`sqs_dlq_message_retention_seconds` (`number`) optional
The number of seconds Amazon SQS retains a message. Integer representing seconds, from 60 (1 minute) to 1209600 (14 days). **Default value:** `1209600`
`sqs_queue_kms_data_key_reuse_period_seconds` (`number`) optional
The length of time, in seconds, for which Amazon SQS can reuse a data key to encrypt or decrypt messages before calling AWS KMS again **Default value:** `300`
`sqs_queue_kms_master_key_id` (`string`) optional
The ID of an AWS-managed customer master key (CMK) for Amazon SQS Queue or a custom CMK **Default value:** `"alias/aws/sqs"`
`subscribers` optional
Required configuration for subscibres to SNS topic. **Type:** ```hcl map(object({ protocol = string # The protocol to use. The possible values for this are: sqs, sms, lambda, application. (http or https are partially supported, see below) (email is an option but is unsupported, see below). endpoint = string # The endpoint to send data to, the contents will vary with the protocol. (see below for more information) endpoint_auto_confirms = optional(bool, false) # Boolean indicating whether the end point is capable of auto confirming subscription e.g., PagerDuty (default is false) filter_policy = optional(string, null) # The filter policy JSON that is assigned to the subscription. For more information, see Amazon SNS Filter Policies. filter_policy_scope = optional(string, null) # The filter policy scope that is assigned to the subscription. Whether the `filter_policy` applies to `MessageAttributes` or `MessageBody`. Default is null. raw_message_delivery = optional(bool, false) # Boolean indicating whether or not to enable raw message delivery (the original message is directly passed, not wrapped in JSON with the original message in the message property) (default is false) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`aws_sns_topic_subscriptions`
SNS topic subscription.
`dead_letter_queue_arn`
The ARN of the dead letter queue.
`dead_letter_queue_id`
The ID for the created dead letter queue. Same as the URL.
`dead_letter_queue_name`
The name for the created dead letter queue.
`dead_letter_queue_url`
The URL for the created dead letter SQS queue.
`sns_topic`
SNS topic.
`sns_topic_arn`
SNS topic ARN.
`sns_topic_id`
SNS topic ID.
`sns_topic_name`
SNS topic name.
`sns_topic_owner`
SNS topic owner.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 3.0` ### Providers - `aws`, version: `>= 3.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_sns_topic.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic) (resource) - [`aws_sns_topic_policy.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_policy) (resource) - [`aws_sns_topic_subscription.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sns_topic_subscription) (resource) - [`aws_sqs_queue.dead_letter_queue`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue) (resource) - [`aws_sqs_queue_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sqs_queue_policy) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.aws_sns_topic_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.sqs_queue_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) --- ## ssm-iam-role # Module: `ssm-iam-role` Terraform module to provision an IAM role with configurable permissions to access [SSM Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html). ## Introduction For more information on how to control access to Systems Manager parameters by using AWS Identity and Access Management, see [Controlling Access to Systems Manager Parameters](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-access.html). For more information on how to use parameter hierarchies to help organize and manage parameters, see [Organizing Parameters into Hierarchies](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-su-organize.html). __NOTE:__ This module can be used to provision IAM roles with SSM permissions for [chamber](https://docs.cloudposse.com/tools/chamber/). ## Usage This example creates a role with the name `cp-prod-app-all` with permission to read all SSM parameters, and gives permission to the entities specified in `assume_role_arns` to assume the role. ```hcl module "ssm_iam_role" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-iam-role.git?ref=master" namespace = "cp" stage = "prod" name = "app" attributes = ["all"] region = "us-west-2" account_id = "XXXXXXXXXXX" assume_role_arns = ["arn:aws:xxxxxxxxxx", "arn:aws:yyyyyyyyyyyy"] kms_key_arn = "arn:aws:kms:us-west-2:123454095951:key/aced568e-3375-4ece-85e5-b35abc46c243" ssm_parameters = ["*"] ssm_actions = ["ssm:GetParametersByPath", "ssm:GetParameters"] } ``` ## Examples ### Example With Permission For Specific Resources This example creates a role with the name `cp-prod-app-secrets` with permission to read the SSM parameters that begin with `secret-`, and gives permission to the entities specified in `assume_role_arns` to assume the role. ```hcl module "ssm_iam_role" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-iam-role.git?ref=master" namespace = "cp" stage = "prod" name = "app" attributes = ["secrets"] region = "us-west-2" account_id = "XXXXXXXXXXX" assume_role_arns = ["arn:aws:xxxxxxxxxx", "arn:aws:yyyyyyyyyyyy"] kms_key_arn = "arn:aws:kms:us-west-2:123454095951:key/aced568e-3375-4ece-85e5-b35abc46c243" ssm_parameters = ["secret-*"] ssm_actions = ["ssm:GetParameters"] } ``` ### Complete Example This example: * Provisions a KMS key to encrypt SSM Parameter Store secrets using [terraform-aws-kms-key](https://github.com/cloudposse/terraform-aws-kms-key) module * Performs `Kops` cluster lookup to find the ARNs of `masters` and `nodes` by using [terraform-aws-kops-metadata](https://github.com/cloudposse/terraform-aws-kops-metadata) module * Creates a role with the name `cp-prod-chamber-kops` with permission to read all SSM parameters from the path `kops`, and gives permission to the Kops `masters` and `nodes` to assume the role ```hcl module "kms_key" { source = "git::https://github.com/cloudposse/terraform-aws-kms-key.git?ref=master" namespace = "cp" stage = "prod" name = "chamber" description = "KMS key for SSM" } module "kops_metadata" { source = "git::https://github.com/cloudposse/terraform-aws-kops-metadata.git?ref=master" dns_zone = "us-west-2.prod.cloudposse.co" masters_name = "masters" nodes_name = "nodes" } module "ssm_iam_role" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-iam-role.git?ref=master" namespace = "cp" stage = "prod" name = "chamber" attributes = ["kops"] region = "us-west-2" account_id = "XXXXXXXXXXX" assume_role_arns = ["${module.kops_metadata.masters_role_arn}", "${module.kops_metadata.nodes_role_arn}"] kms_key_arn = "${module.kms_key.key_arn}" ssm_parameters = ["kops/*"] ssm_actions = ["ssm:GetParametersByPath", "ssm:GetParameters"] } ``` ## Variables ### Required Variables
`account_id` (`string`) required
AWS account ID
`assume_role_arns` (`list(string)`) required
List of ARNs to allow assuming the role. Could be AWS services or accounts, Kops nodes, IAM users or groups
`kms_key_reference` (`any`) required
The Key ID, Key ARN, Key Alias Name, or Key Alias ARN of the KMS key which will encrypt/decrypt SSM secret strings
`region` (`string`) required
AWS Region
`ssm_parameters` (`list(string)`) required
List of SSM parameters to apply the actions. A parameter can include a path and a name pattern that you define by using forward slashes, e.g. `kops/secret-*`
### Optional Variables
`max_session_duration` (`number`) optional
The maximum session duration (in seconds) for the role. Can have a value from 1 hour to 12 hours **Default value:** `3600`
`ssm_actions` (`list(string)`) optional
SSM actions to allow **Default value:** ```hcl [ "ssm:GetParametersByPath", "ssm:GetParameters" ] ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`name` (`string`) required
Name (e.g. `app` or `chamber`) **Required:** Yes **Default value:** ``
`namespace` (`string`) required
Namespace (e.g. `cp` or `cloudposse`) **Required:** Yes **Default value:** ``
`stage` (`string`) required
Stage (e.g. `prod`, `dev`, `staging`) **Required:** Yes **Default value:** ``
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `stage`, `name` and `attributes` **Required:** No **Default value:** `"-"`
`tags` (`map(string)`) optional
Additional tags (e.g. map(`BusinessUnit`,`XYZ`) **Required:** No **Default value:** `{ }`
## Outputs
`role_arn`
The Amazon Resource Name (ARN) specifying the role
`role_id`
The stable and unique string identifying the role
`role_name`
The name of the crated role
`role_policy_document`
A copy of the IAM policy document (JSON) that grants permissions to this role.
## Dependencies ### Providers - `aws` ### Modules Name | Version | Source | Description --- | --- | --- | --- `label` | 0.1.3 | [`git::https://github.com/cloudposse/terraform-terraform-label.git`](https://registry.terraform.io/modules/git::https:/github.com/cloudposse/terraform-terraform-label.git/0.1.3) | n/a ## Resources The following resources are used by this module: - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_kms_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) (data source) --- ## ssm-parameter-chamber-reader # Module: `ssm-parameter-chamber-reader` Terraform module read ssm paramters managed with Chamber. ## Examples ```hcl variable "rbac_enabled" { type = bool default = null description = "Override rbac enabled" } module "account_id" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-chamber-reader.git?ref=master" enabled = "true" chamber_service = "kops" parameter = "rbac_enabled" override_value = var.rbac_enabled } ``` ## Variables ### Required Variables
`chamber_service` (`string`) required
SSM parameter service name for use with chamber. This is used in chamber_format where /$chamber_service/$parameter would be the default.
`parameter` (`string`) required
SSM parameter name for use with chamber. This is used in chamber_format where /$chamber_service/$parameter would be the default.
### Optional Variables
`chamber_format` (`string`) optional
Format to store parameters in SSM, for consumption with chamber **Default value:** `"/%s/%s"`
`default_value` (`string`) optional
Use as default value in case ssm paramter is empty **Default value:** `""`
`override_key` (`string`) optional
Is specified, use as key to read from ssm parameter and ignore chamber_format. **Default value:** `""`
`override_value` (`string`) optional
Is specified, just return it as value by skipping read from ssm parameter. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `true`
## Outputs
`value`
Parameter value
## Dependencies ### Requirements - `terraform`, version: `>= 0.12` ### Providers - `aws` ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## ssm-parameter-store # Module: `ssm-parameter-store` Terraform module for providing read and write access to the AWS SSM Parameter Store. ## Introduction * [AWS Details on what values can be used](https://docs.aws.amazon.com/systems-manager/latest/userguide/sysman-paramstore-su-create.html) * [AWS API for PutParameter](https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_PutParameter.html) * [Terraform aws_ssm_parameter resource page](https://www.terraform.io/docs/providers/aws/r/ssm_parameter.html) * [Terraform aws_ssm_parameter data page](https://www.terraform.io/docs/providers/aws/d/ssm_parameter.html) ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ssm-parameter-store/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ssm-parameter-store/tree/main/test). This example creates a new `String` parameter called `/cp/prod/app/database/master_password` with the value of `password1`. ```hcl module "store_write" { source = "cloudposse/ssm-parameter-store/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" parameter_write = [ { name = "/cp/prod/app/database/master_password" value = "password1" type = "String" overwrite = "true" description = "Production database master password" } ] tags = { ManagedBy = "Terraform" } } ``` This example reads a value from the parameter store with the name `/cp/prod/app/database/master_password` ```hcl module "store_read" { source = "cloudposse/ssm-parameter-store/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" parameter_read = ["/cp/prod/app/database/master_password"] } ``` ## Variables ### Required Variables
### Optional Variables
`ignore_value_changes` (`bool`) optional
Whether to ignore future external changes in paramater values **Default value:** `false`
`kms_arn` (`string`) optional
The ARN of a KMS key used to encrypt and decrypt SecretString values **Default value:** `""`
`parameter_read` (`list(string)`) optional
List of parameters to read from SSM. These must already exist otherwise an error is returned. Can be used with `parameter_write` as long as the parameters are different. **Default value:** `[ ]`
`parameter_write` (`list(map(string))`) optional
List of maps with the parameter values to write to SSM Parameter Store **Default value:** `[ ]`
`parameter_write_defaults` (`map(any)`) optional
Parameter write default settings **Default value:** ```hcl { "allowed_pattern": null, "data_type": "text", "description": null, "overwrite": "false", "tier": "Standard", "type": "SecureString" } ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn_map`
A map of the names and ARNs created
`map`
A map of the names and values created
`names`
A list of all of the parameter names
`values`
A list of all of the parameter values
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.ignore_value_changes`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ssm_parameter.read`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ssm_parameter) (data source) --- ## ssm-parameter-store-policy-documents # Module: `ssm-parameter-store-policy-documents` This module generates JSON documents for restricted permission sets for AWS SSM Parameter Store access. Helpful when combined with [terraform-aws-ssm-parameter-store](https://github.com/cloudposse/terraform-aws-ssm-parameter-store) ## Examples Create a policy that allows access to write all parameters ```hcl module "ps_policy" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-store-policy-documents.git?ref=master" } resource "aws_iam_policy" "ps_write" { name_prefix = "write_any_parameter_store_value" path = "/" policy = "${module.ps_policy.write_parameter_store_policy}" } ``` Create a policy that allows managing all policies ```hcl module "ps_policy" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-store-policy-documents.git?ref=master" } resource "aws_iam_policy" "ps_manage" { name_prefix = "manage_any_parameter_store_value" path = "/" policy = "${module.ps_policy.manage_parameter_store_policy}" } ``` Create a policy that allows reading all parameters that start with a certain prefix ```hcl module "ps_policy" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-store-policy-documents.git?ref=master" parameter_root_name = "/cp/dev/app" } resource "aws_iam_policy" "ps_manage" { name_prefix = "write_specific_parameter_store_value" path = "/" policy = "${module.ps_policy.manage_parameter_store_policy}" } ``` Create a kms policy to allow decrypting of the parameter store values ```hcl module "kms_key" { source = "git::https://github.com/cloudposse/terraform-aws-kms-key.git?ref=master" namespace = "cp" stage = "prod" name = "app" description = "KMS key" deletion_window_in_days = 10 enable_key_rotation = "true" alias = "alias/parameter_store_key" } module "ps_policy" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-store-policy-documents.git?ref=master" parameter_root_name = "/cp/dev/app" kms_key = "${module.kms_key.key_arn}" } resource "aws_iam_policy" "ps_kms" { name_prefix = "decrypt_parameter_store_value" path = "/" policy = "${module.ps_policy.manage_kms_store_policy}" } ``` Create a policy for another account, or region ```hcl module "ps_policy" { source = "git::https://github.com/cloudposse/terraform-aws-ssm-parameter-store-policy-documents.git?ref=master" parameter_root_name = "/cp/dev/app" account_id = "783649272629220" region = "ap-southeast-2" } resource "aws_iam_policy" "ps_manage" { name_prefix = "manage_any_parameter_store_value" path = "/" policy = "${module.ps_policy.manage_parameter_store_policy}" } ``` ## Variables ### Required Variables
### Optional Variables
`account_id` (`string`) optional
The account id of the parameter store you want to allow access to. If none supplied, it uses the current account id of the provider. **Default value:** `""`
`kms_key` (`string`) optional
The arn of the KMS key that you want to allow access to. If empty it uses a wildcard resource (`*`). **Default value:** `""`
`parameter_root_name` (`string`) optional
The prefix or root parameter that you want to allow access to. **Default value:** `""`
`region` (`string`) optional
The region of the parameter store value that you want to allow access to. If none supplied, it uses the current region of the provider. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Outputs
`manage_kms_store_policy`
A JSON policy document that allows decryption access to a KMS key.
`manage_parameter_store_policy`
A JSON policy document that allows full access to the parameter store.
`put_xray_trace_policy`
A JSON policy document that allows putting data into x-ray for tracing parameter store requests.
`read_parameter_store_policy`
A JSON policy document that only allows read access to the parameter store.
`write_parameter_store_policy`
A JSON policy document that only allows write access to the parameter store.
## Dependencies ### Providers - `aws` ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.manage_kms_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.manage_parameter_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.put_xray_trace`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.read_parameter_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.write_parameter_store`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## ssm-patch-manager # Module: `ssm-patch-manager` This module provisions AWS SSM Patch manager maintenance window tasks, targets, patch baselines and patch groups and a s3 bucket for storing patch task logs. ## Introduction ## Acknowledgements This module was heavily inspired by @jparnaudeau module https://github.com/jparnaudeau/terraform-aws-ssm-patch-management ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ssm-patch-manager/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-ssm-patch-manager/tree/main/test). ```hcl module "ssm_patch_manager" { source = "cloudposse/ssm-patch-manager/aws" version = "xxxx" name = "test" } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-ssm-patch-manager/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`approved_patches` (`list(string)`) optional
A list of explicitly approved patches for the baseline **Default value:** `[ ]`
`approved_patches_compliance_level` (`string`) optional
Defines the compliance level for approved patches. This means that if an approved patch is reported as missing, this is the severity of the compliance violation. Valid compliance levels include the following: CRITICAL, HIGH, MEDIUM, LOW, INFORMATIONAL, UNSPECIFIED. The default value is UNSPECIFIED. **Default value:** `"HIGH"`
`bucket_id` (`list(string)`) optional
The bucket ID to use for the patch log. If no bucket ID is provided, the module will create a new one. This is of type `list(string)` to work around #41 / https://github.com/hashicorp/terraform/issues/28962. **Default value:** `[ ]`
`cloudwatch_log_group_name` (`string`) optional
The name of the CloudWatch log group where you want to send command output. If you don't specify a group name, Systems Manager automatically creates a log group for you. The log group uses the following naming format: aws/ssm/SystemsManagerDocumentName. **Default value:** `null`
`cloudwatch_log_output_enabled` (`bool`) optional
Enables Systems Manager to send command output to CloudWatch Logs. **Default value:** `false`
`install_maintenance_window_cutoff` (`number`) optional
The number of hours before the end of the Maintenance Window that Systems Manager stops scheduling new tasks for execution **Default value:** `1`
`install_maintenance_window_duration` (`number`) optional
The duration of the maintenence windows (hours) **Default value:** `3`
`install_maintenance_window_schedule` (`string`) optional
The schedule of the Maintenance Window in the form of a cron or rate expression **Default value:** `"cron(0 0 21 ? * WED *)"`
`install_maintenance_windows_targets` optional
The targets to register with the maintenance window. In other words, the instances to run commands on when the maintenance window runs. You can specify targets using instance IDs, resource group names, or tags that have been applied to instances. For more information about these examples formats see (https://docs.aws.amazon.com/systems-manager/latest/userguide/mw-cli-tutorial-targets-examples.html) **Type:** ```hcl list(object({ key : string values : list(string) } ) ) ``` **Default value:** `[ ]`
`install_patch_groups` (`list(string)`) optional
The targets to register with the maintenance window. In other words, the instances to run commands on when the maintenance window runs. You can specify targets using instance IDs, resource group names, or tags that have been applied to instances. For more information about these examples formats see (https://docs.aws.amazon.com/systems-manager/latest/userguide/mw-cli-tutorial-targets-examples.html) **Default value:** ```hcl [ "TOPATCH" ] ```
`install_sns_notification_enabled` (`bool`) optional
Enable/disable the SNS notification for install patches **Default value:** `false`
`max_concurrency` (`number`) optional
The maximum number of targets this task can be run for in parallel **Default value:** `20`
`max_errors` (`number`) optional
The maximum number of errors allowed before this task stops being scheduled **Default value:** `50`
`notification_arn` (`string`) optional
An Amazon Resource Name (ARN) for a Simple Notification Service (SNS) topic. Run Command pushes notifications about command status changes to this topic. **Default value:** `""`
`notification_events` (`list(string)`) optional
The different events for which you can receive notifications. Valid values: All, InProgress, Success, TimedOut, Cancelled, and Failed **Default value:** ```hcl [ "All" ] ```
`notification_type` (`string`) optional
When specified with Command, receive notification when the status of a command changes. When specified with Invocation, for commands sent to multiple instances, receive notification on a per-instance basis when the status of a command changes. Valid values: Command and Invocation **Default value:** `"Command"`
`operating_system` (`string`) optional
Defines the operating system the patch baseline applies to. Supported operating systems include WINDOWS, AMAZON_LINUX, AMAZON_LINUX_2, SUSE, UBUNTU, CENTOS, and REDHAT_ENTERPRISE_LINUX. The Default value is WINDOWS. **Default value:** `"AMAZON_LINUX_2"`
`patch_baseline_approval_rules` optional
A set of rules used to include patches in the baseline. Up to 10 approval rules can be specified. Each `approval_rule` block requires the fields documented below (unless marked optional). `approve_after_days` and `approve_until_date` conflict, do not set both in the same `approval_rule`. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_patch_baseline#approval_rule-block for full details. **Type:** ```hcl list(object({ approve_after_days : optional(number) approve_until_date : optional(string) compliance_level : string enable_non_security : bool patch_baseline_filters : list(object({ name : string values : list(string) })) })) ``` **Default value:** ```hcl [ { "approve_after_days": 7, "compliance_level": "HIGH", "enable_non_security": true, "patch_baseline_filters": [ { "name": "PRODUCT", "values": [ "AmazonLinux2", "AmazonLinux2.0" ] }, { "name": "CLASSIFICATION", "values": [ "Security", "Bugfix", "Recommended" ] }, { "name": "SEVERITY", "values": [ "Critical", "Important", "Medium" ] } ] } ] ```
`reboot_option` (`string`) optional
When you choose the RebootIfNeeded option, the instance is rebooted if Patch Manager installed new patches, or if it detected any patches with a status of INSTALLED_PENDING_REBOOT during the Install operation. Possible values : RebootIfNeeded, NoReboot **Default value:** `"RebootIfNeeded"`
`rejected_patches` (`list(string)`) optional
A list of rejected patches **Default value:** `[ ]`
`s3_bucket_prefix_install_logs` (`string`) optional
The Amazon S3 bucket subfolder for install logs **Default value:** `"install"`
`s3_bucket_prefix_scan_logs` (`string`) optional
The Amazon S3 bucket subfolder for scan logs **Default value:** `"scanning"`
`s3_log_output_enabled` (`bool`) optional
To enable or disable s3 bucket output for the runCommand logs **Default value:** `true`
`scan_maintenance_window_cutoff` (`number`) optional
The number of hours before the end of the Maintenance Window that Systems Manager stops scheduling new tasks for execution **Default value:** `1`
`scan_maintenance_window_duration` (`number`) optional
The duration of the maintenence windows (hours) **Default value:** `3`
`scan_maintenance_window_schedule` (`string`) optional
The schedule of the Maintenance Window in the form of a cron or rate expression. **Default value:** `"cron(0 0 18 ? * WED *)"`
`scan_maintenance_windows_targets` optional
The map of tags for targetting which EC2 instances will be scaned **Type:** ```hcl list(object({ key : string values : list(string) } ) ) ``` **Default value:** `[ ]`
`scan_patch_groups` (`list(string)`) optional
The targets to register with the maintenance window. In other words, the instances to run commands on when the maintenance window runs. You can specify targets using instance IDs, resource group names, or tags that have been applied to instances. For more information about these examples formats see (https://docs.aws.amazon.com/systems-manager/latest/userguide/mw-cli-tutorial-targets-examples.html) **Default value:** ```hcl [ "TOSCAN" ] ```
`scan_sns_notification_enabled` (`bool`) optional
Enable/Disable the SNS notification for scans **Default value:** `false`
`service_role_arn` (`string`) optional
The role that should be assumed when executing the task. If a role is not provided, Systems Manager uses your account's service-linked role. If no service-linked role for Systems Manager exists in your account, it is created for you **Default value:** `null`
`sns_notification_role_arn` (`string`) optional
An Amazon Resource Name (ARN) for a Simple Notification Service (SNS) topic. Run Command pushes notifications about command status changes to this topic. **Default value:** `""`
`ssm_bucket_policy` (`string`) optional
Custom bucket policy for the SSM log bucket **Default value:** `null`
`ssm_bucket_versioning_enable` (`string`) optional
To enable or disable S3 bucket versioning for the log bucket. **Default value:** `true`
`task_install_priority` (`number`) optional
The priority of the task in the Maintenance Window, the lower the number the higher the priority. Tasks in a Maintenance Window are scheduled in priority order with tasks that have the same priority scheduled in parallel. **Default value:** `1`
`task_scan_priority` (`number`) optional
The priority of the task in the Maintenance Window, the lower the number the higher the priority. Tasks in a Maintenance Window are scheduled in priority order with tasks that have the same priority scheduled in parallel. Default 1 **Default value:** `1`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`install_maintenance_window_id`
SSM Patch Manager install maintenance window ID
`install_maintenance_window_target_id`
SSM Patch Manager install maintenance window target ID
`install_maintenance_window_task_id`
SSM Patch Manager install maintenance windows task ID
`install_patch_group_id`
SSM Patch Manager install patch group ID
`patch_baseline_arn`
SSM Patch Manager patch baseline ARN
`scan_maintenance_window_target_id`
SSM Patch Manager scan maintenance window target ID
`scan_maintenance_window_task_id`
SSM Patch Manager scan maintenance windows task ID
`scan_patch_group_id`
SSM Patch Manager scan patch group ID
`ssm_patch_log_s3_bucket_arn`
SSM Patch Manager s3 log bucket ARN
`ssm_patch_log_s3_bucket_id`
SSM Patch Manager s3 log bucket ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `install_window_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `scan_window_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `ssm_patch_log_s3_bucket` | 4.0.1 | [`cloudposse/s3-bucket/aws`](https://registry.terraform.io/modules/cloudposse/s3-bucket/aws/4.0.1) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_maintenance_window.install_window`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window) (resource) - [`aws_ssm_maintenance_window.scan_window`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window) (resource) - [`aws_ssm_maintenance_window_target.target_install`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_target) (resource) - [`aws_ssm_maintenance_window_target.target_scan`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_target) (resource) - [`aws_ssm_maintenance_window_task.task_install_patches`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_task) (resource) - [`aws_ssm_maintenance_window_task.task_scan_patches`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_maintenance_window_task) (resource) - [`aws_ssm_patch_baseline.baseline`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_patch_baseline) (resource) - [`aws_ssm_patch_group.install_patchgroup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_patch_group) (resource) - [`aws_ssm_patch_group.scan_patchgroup`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_patch_group) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.bucket_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## ssm-tls-self-signed-cert # Module: `ssm-tls-self-signed-cert` This module creates a self-signed certificate and writes it alongside with its key to SSM Parameter Store (or alternatively AWS Secrets Manager). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-ssm-tls-self-signed-cert/tree/main/examples/complete). ```hcl module "self_signed_cert" { source = "cloudposse/ssm-tls-self-signed-cert/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "self-signed-cert" subject = { common_name = "example" organization = "Cloud Posse" organizational_unit = "Engineering" } validity = { duration_hours = 730 early_renewal_hours = 24 } allowed_uses = [ "key_encipherment", "digital_signature", "server_auth" ] subject_alt_names = { ip_addresses = ["10.10.10.10"] dns_names = ["example.com"] uris = ["https://example.com"] } } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-ssm-tls-self-signed-cert/tree/main/examples/complete) - complete example of using this module. ## Variables ### Required Variables
`allowed_uses` (`list(string)`) required
List of keywords each describing a use that is permitted for the issued certificate. Must be one of of the values outlined in [self_signed_cert.allowed_uses](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/self_signed_cert#allowed_uses).
### Optional Variables
`asm_recovery_window_in_days` (`number`) optional
Number of days that AWS Secrets Manager waits before it can delete the secret. This value can be `0` to force deletion without recovery or range from `7` to `30` days. This value is ignored if `var.certificate_backends` is not `ASM`, or if `var.certificate_backend_enabled` is `false`. **Default value:** `30`
`basic_constraints` optional
The [basic constraints](https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.9) of the issued certificate. Currently, only the `CA` constraint (which identifies whether the subject of the certificate is a CA) can be set. Defaults to this certificate not being a CA. **Type:** ```hcl object({ ca = bool }) ``` **Default value:** ```hcl { "ca": false } ```
`certificate_backend_kms_key_id` (`string`) optional
The KMD Key ID (ARN or ID) to use when encrypting either the AWS SSM Parameters or AWS Secrets Manager Secrets relating to the certificate. If not specified, the Amazon-managed Key `alias/aws/ssm` will be used if `var.certificate_backends` contains `SSM`, and `alias/aws/secretsmanager` will be used if `var.certificate_backends` is `ASM`. **Default value:** `null`
`certificate_backends` (`set(string)`) optional
The certificate backend to use when writing secrets related to the self-signed certificate. The value specified can either be `SSM` (AWS Systems Manager Parameter Store), `ASM` (AWS Secrets Manager), and/or `ACM` (AWS Certificate Manager). Defaults to `SSM`. **Default value:** ```hcl [ "SSM" ] ```
`certificate_backends_base64_enabled` (`bool`) optional
Enable or disable base64 encoding of secrets before writing them to the secrets store. **Default value:** `false`
`certificate_backends_enabled` (`bool`) optional
Enable or disable writing to the secrets store. **Default value:** `true`
`certificate_chain` optional
When using ACM as a certificate backend, some certificates store a certificate chain from a CA. This CA will come from another resource. **Type:** ```hcl object({ cert_pem = string private_key_pem = string }) ``` **Default value:** `null`
`private_key_algorithm` (`string`) optional
The name of the algorithm for the private key of the certificate. Currently only RSA and ECDSA are supported. If a preexisting private key is supplied via `var.private_key_contents`, this value must match that key's algorithm. Defaults to RSA as it is a more widely adopted algorithm, although ECDSA provides the same level of security and with shorter keys. **Default value:** `"RSA"`
`private_key_contents` (`string`) optional
The contents of the private key to use for the certificate. If supplied, this module will not create a private key and use these contents instead for the private key. Defaults to `null`, which means a private key will be created. **Default value:** `null`
`private_key_ecdsa_curve` (`string`) optional
When `var.cert_key_algorithm` is `ECDSA`, the name of the elliptic curve to use. May be any one of `P224`, `P256`, `P384` or `P521`. Ignored if `var.cert_key_algorithm` is not `ECDSA`, or if a preexisting private key is supplied via `var.private_key_contents`. Defaults to the `tls` provider default. **Default value:** `"P224"`
`private_key_rsa_bits` (`number`) optional
When `var.cert_key_algorithm` is `RSA`, the size of the generated RSA key in bits. Ignored if `var.cert_key_algorithm` is not `RSA`, or if a preexisting private key is supplied via `var.private_key_contents`. Defaults to the `tls` provider default. **Default value:** `2048`
`secret_extensions` optional
The extensions use when writing secrets to the certificate backend. Please refer to `var.secret_path_format` for information on how secret paths are computed. **Type:** ```hcl object({ certificate = string private_key = string }) ``` **Default value:** ```hcl { "certificate": "pem", "private_key": "key" } ```
`secret_path_format` (`string`) optional
The path format to use when writing secrets to the certificate backend. The certificate secret path will be computed as `format(var.secret_path_format, var.name, var.secret_extensions.certificate)` and the private key path as `format(var.secret_path_format, var.name, var.secret_extensions.private_key)`. Thus by default, if `var.name`=`example-self-signed-cert`, then the resulting secret paths for the self-signed certificate's PEM file and private key will be `/example-self-signed-cert.pem` and `/example-self-signed-cert.key`, respectively. This variable can be overridden in order to create more specific certificate backend paths. **Default value:** `"/%s.%s"`
`skid_enabled` (`bool`) optional
Whether or not the subject key identifier (SKID) should be included in the certificate. **Default value:** `false`
`subject` (`any`) optional
The subject configuration for the certificate. This should be a map that is compatible with [tls_cert_request.subject](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/cert_request#subject). If `common_name` is omitted, it will be set as `module.this.id`. **Default value:** `{ }`
`subject_alt_names` optional
The subject alternative name (SAN) configuration for the certificate. This configuration consists of several lists, each of which can also be set to `null` or `[]`. `dns_names`: List of DNS names for which a certificate is being requested. `ip_addresses`: List of IP addresses for which a certificate is being requested. `uris`: List of URIs for which a certificate is being requested. Defaults to no SANs. **Type:** ```hcl object({ dns_names = list(string) ip_addresses = list(string) uris = list(string) }) ``` **Default value:** ```hcl { "dns_names": null, "ip_addresses": null, "uris": null } ```
`use_locally_signed` (`bool`) optional
Create a locally signed certificate/key pair instead of a self-signed one. This is useful it a previously created certificate chain is to be used to sign a certificate. **Default value:** `false`
`validity` optional
Validity settings for the issued certificate: `duration_hours`: The number of hours from issuing the certificate until it becomes invalid. `early_renewal_hours`: If set, the resource will consider the certificate to have expired the given number of hours before its actual expiry time (see: [self_signed_cert.early_renewal_hours](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/self_signed_cert#early_renewal_hours)). Defaults to 10 years and no early renewal hours. **Type:** ```hcl object({ duration_hours = number early_renewal_hours = number }) ``` **Default value:** ```hcl { "duration_hours": 87600, "early_renewal_hours": null } ```
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`certificate_arn`
ARN of certificate stored in ACM that other services may need to refer to. This is useful when the certificate is stored in ACM.
`certificate_key_path`
Secrets store path containing the certificate private key file.
`certificate_pem`
Contents of the certificate PEM.
`certificate_pem_path`
Secrets store path containing the certificate PEM file.
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 3.0` - `tls`, version: `>= 4.0.0` ### Providers - `aws`, version: `>= 3.0` - `tls`, version: `>= 4.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_acm_certificate.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/acm_certificate) (resource) - [`aws_secretsmanager_secret.certificate`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) (resource) - [`aws_secretsmanager_secret.private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret) (resource) - [`aws_secretsmanager_secret_version.certificate`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) (resource) - [`aws_secretsmanager_secret_version.private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/secretsmanager_secret_version) (resource) - [`aws_ssm_parameter.certificate`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.private_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`tls_cert_request.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/cert_request) (resource) - [`tls_locally_signed_cert.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/locally_signed_cert) (resource) - [`tls_private_key.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) - [`tls_self_signed_cert.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/self_signed_cert) (resource) ## Data Sources The following data sources are used by this module: --- ## ssm-tls-ssh-key-pair # Module: `ssm-tls-ssh-key-pair` Terraform module that provisions an SSH TLS key pair and writes it to SSM Parameter Store. This is useful for bot accounts (e.g. for GitHub). Easily rotate SSH secrets by simply tainting the module resource and reapplying. ## Usage ```hcl module "ssm_tls_ssh_key_pair" { source = "cloudposse/ssm-tls-ssh-key-pair/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "app" ssm_path_prefix = "ssh_keys" ssh_key_algorithm = "ECDSA" } ``` ## Variables ### Required Variables
### Optional Variables
`ecdsa_curve` (`string`) optional
When ssh_key_algorithm is 'ECDSA', the name of the elliptic curve to use. May be any one of 'P256', 'P384' or P521' **Default value:** `"P256"`
`kms_key_id` (`string`) optional
KMS Key ID used for encryption **Default value:** `""`
`overwrite_ssm_parameter` (`bool`) optional
Whether to overwrite an existing SSM parameter **Default value:** `true`
`rsa_bits` (`number`) optional
When ssh_key_algorithm is 'RSA', the size of the generated RSA key in bits **Default value:** `4096`
`ssh_key_algorithm` (`string`) optional
SSH key algorithm to use. Currently-supported values are 'RSA' and 'ECDSA' **Default value:** `"RSA"`
`ssh_private_key_name` (`string`) optional
SSM Parameter name of the SSH private key **Default value:** `""`
`ssh_public_key_name` (`string`) optional
SSM Parameter name of the SSH public key **Default value:** `""`
`ssm_path_format` (`string`) optional
SSM path format **Default value:** `"/%s/%s"`
`ssm_path_prefix` (`string`) optional
The SSM parameter path prefix (e.g. /$ssm_path_prefix/$key_name) **Default value:** `"ssh_keys"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`key_name`
Name of SSH key
`public_key`
Content of the generated public key
`ssh_private_key_ssm_path`
SSM path of the generated private key
`ssh_public_key_ssm_path`
SSM path of the generated public key
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `local`, version: `>= 1.3` - `null`, version: `>= 2.1` - `tls`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` - `tls`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.private_ecdsa_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.private_rsa_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.public_ecdsa_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`aws_ssm_parameter.public_rsa_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`tls_private_key.default_ecdsa`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) - [`tls_private_key.default_rsa`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) ## Data Sources The following data sources are used by this module: - [`aws_kms_key.kms_key`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/kms_key) (data source) --- ## sso # Module: `sso` This module configures [AWS Single Sign-On (SSO)](https://aws.amazon.com/single-sign-on/). AWS SSO makes it easy to centrally manage access to multiple AWS accounts and business applications and provide users with single sign-on access to all their assigned accounts and applications from one place. With AWS SSO, you can easily manage access and user permissions to all of your accounts in AWS Organizations centrally. AWS SSO configures and maintains all the necessary permissions for your accounts automatically, without requiring any additional setup in the individual accounts. You can assign user permissions based on common job functions and customize these permissions to meet your specific security requirements. AWS SSO also includes built-in integrations to many business applications, such as Salesforce, Box, and Microsoft 365. With AWS SSO, you can create and manage user identities in AWS SSO’s identity store, or easily connect to your existing identity source, including Microsoft Active Directory, Okta Universal Directory, and Azure Active Directory (Azure AD). AWS SSO allows you to select user attributes, such as cost center, title, or locale, from your identity source, and then use them for attribute-based access control in AWS. ## Usage This module contains two sub-modules that can be used in conjunction to provision AWS SSO Permission Sets and to assign AWS SSO Users and Groups to Permissions Sets in accounts. - [modules/account-assignments](https://github.com/cloudposse/terraform-aws-sso/tree/main/modules/account-assignments) - a module for assigning users and groups to permission sets in particular accounts - [modules/permission-sets](https://github.com/cloudposse/terraform-aws-sso/tree/main/modules/permission-sets) - a module for provisioning AWS SSO permission sets ## Examples Here is a full example of using these modules to provision permission sets and assign them to accounts: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-sso/tree/main/examples/complete) - complete example of using these modules ## Variables ### Required Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Dependencies --- ## account-assignments # AWS SSO Account Assignments Module This module assigns [AWS SSO permission sets](https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html) to Users and Groups from the [AWS SSO Identity Source](https://docs.aws.amazon.com/singlesignon/latest/userguide/manage-your-identity-source.html). Some english-language examples of this would be: - users who are in the group `Administrators` should be assigned the permission set `AdmininstratorAccess` in the `production` account. - users who are in the group `Developers` should be assigned the permission set `DeveloperAccess` in the `production` account - users who are in the group `Developers` should be assigned the permission set `AdministraorAccess` in the `sandbox` account ## Usage **IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-sso/releases). For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-sso/tree/main/examples/complete). ```hcl module "sso_account_assignments" { source = "https://github.com/cloudposse/terraform-aws-sso.git//modules/account-assignments?ref=master" account_assignments = [ { account = "111111111111", permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66", permission_set_name = "Administrators", principal_type = "GROUP", principal_name = "Administrators" }, { account = "111111111111", permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-955c264e8f20fea3", permission_set_name = "Developers", principal_type = "GROUP", principal_name = "Developers" }, { account = "222222222222", permission_set_arn = "arn:aws:sso:::permissionSet/ssoins-0000000000000000/ps-31d20e5987f0ce66", permission_set_name = "Developers", principal_type = "GROUP", principal_name = "Developers" }, ] } ``` --- ## permission-sets # AWS SSO Permission Sets Module This module creates a collection of [AWS SSO permission sets](https://docs.aws.amazon.com/singlesignon/latest/userguide/permissionsetsconcept.html). A permission set is a collection of administrator-defined policies that AWS SSO uses to determine a user's effective permissions to access a given AWS account. Permission sets can contain either AWS managed policies or custom policies that are stored in AWS SSO. Policies are essentially documents that act as containers for one or more permission statements. These statements represent individual access controls (allow or deny) for various tasks that determine what tasks users can or cannot perform within the AWS account. Permission sets are stored in AWS SSO and are only used for AWS accounts. They are not used to manage access to cloud applications. Permission sets ultimately get created as IAM roles in a given AWS account, with trust policies that allow users to assume the role through AWS SSO. ## Usage **IMPORTANT:** The `master` branch is used in `source` just as an example. In your code, do not pin to `master` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-sso/releases). For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-sso/tree/main/examples/complete). ```hcl module "permission_sets" { source = "https://github.com/cloudposse/terraform-aws-sso.git//modules/permission-sets?ref=master" permission_sets = [ { name = "AdministratorAccess", description = "Allow Full Access to the account", relay_state = "", session_duration = "", tags = {}, inline_policy = "", policy_attachments = ["arn:aws:iam::aws:policy/AdministratorAccess"] customer_managed_policy_attachments = [{ name = aws_iam_policy.S3Access.name path = aws_iam_policy.S3Access.path }] }, { name = "S3AdministratorAccess", description = "Allow Full S3 Admininstrator access to the account", relay_state = "", session_duration = "", tags = {}, inline_policy = data.aws_iam_policy_document.S3Access.json, policy_attachments = [] customer_managed_policy_attachments = [] } ] context = module.this.context } data "aws_iam_policy_document" "S3Access" { statement { sid = "1" actions = ["*"] resources = [ "arn:aws:s3:::*", ] } } resource "aws_iam_policy" "S3Access" { name = "S3Access" path = "/" policy = data.aws_iam_policy_document.S3Access.json tags = module.this.tags } ``` --- ## step-functions # Module: `step-functions` Terraform module to provision [AWS Step Functions](https://docs.aws.amazon.com/step-functions/latest/dg/welcome.html). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-step-functions/tree/main/examples/complete) For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-step-functions/tree/main/test). ```hcl locals { enabled = module.this.enabled logging_configuration = { include_execution_data = true level = "ALL" } # https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html # https://docs.aws.amazon.com/step-functions/latest/dg/connect-parameters.html definition = { "Comment" = "Test Step Function" "StartAt" = "Hello" "States" = { "Hello" = { "Type" = "Pass" "Result" = "Hello" "Next" = "World" }, "World" = { "Type" = "Pass" "Result" = "World" "Next" = "Send message to SQS" }, # https://docs.aws.amazon.com/step-functions/latest/dg/connect-sqs.html "Send message to SQS" = { "Type" = "Task" "Resource" = "arn:aws:states:::sqs:sendMessage" "Parameters" = { "QueueUrl" = local.enabled ? aws_sqs_queue.default[0].url : "" "MessageBody" = "Hello World" } "Next" = "Publish to SNS" } # https://docs.aws.amazon.com/step-functions/latest/dg/connect-sns.html "Publish to SNS" = { "Type" = "Task", "Resource" = "arn:aws:states:::sns:publish" "Parameters" = { "TopicArn" = module.sns.sns_topic_arn "Message" = "Hello World" } "End" = true } } } iam_policies = { # https://docs.aws.amazon.com/step-functions/latest/dg/sns-iam.html "SnsAllowPublish" = { effect = "Allow" actions = [ "sns:Publish" ] resources = [ module.sns.sns_topic_arn ] } # https://docs.aws.amazon.com/step-functions/latest/dg/sqs-iam.html "SqsAllowSendMessage" = { effect = "Allow" actions = [ "sqs:SendMessage" ] resources = [ local.enabled ? aws_sqs_queue.default[0].arn : "" ] } } } module "step_function" { source = "cloudposse/step-functions/aws" # Cloud Posse recommends pinning every module to a specific version version = "x.x.x" type = "EXPRESS" tracing_enabled = true logging_configuration = local.logging_configuration definition = local.definition iam_policies = local.iam_policies context = module.this.context } module "sns" { source = "cloudposse/sns-topic/aws" version = "0.20.2" sqs_dlq_enabled = true fifo_topic = true fifo_queue_enabled = true context = module.this.context } resource "aws_sqs_queue" "default" { count = local.enabled ? 1 : 0 name = module.this.id fifo_queue = false visibility_timeout_seconds = 30 message_retention_seconds = 86400 max_message_size = 2048 delay_seconds = 90 receive_wait_time_seconds = 10 tags = module.this.tags } ``` ## Variables ### Required Variables
`definition` (`any`) required
The Amazon States Language definition for the Step Function. Refer to https://docs.aws.amazon.com/step-functions/latest/dg/concepts-amazon-states-language.html for more details
### Optional Variables
`cloudwatch_log_group_kms_key_id` (`string`) optional
The ARN of the KMS Key to use when encrypting log data **Default value:** `null`
`cloudwatch_log_group_name` (`string`) optional
Name of Cloudwatch Logs Group to use. If not provided, a name will be generated from the context **Default value:** `null`
`cloudwatch_log_group_retention_in_days` (`number`) optional
Specifies the number of days to retain log events in the Log Group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653 **Default value:** `null`
`existing_aws_cloudwatch_log_group_arn` (`string`) optional
The Amazon Resource Name (ARN) of the existing CloudWatch Log Group to use for the Step Function. If not provided, a new CloudWatch Log Group will be created **Default value:** `null`
`existing_iam_role_arn` (`string`) optional
The Amazon Resource Name (ARN) of the existing IAM role to use for the Step Function. If not provided, a new IAM role will be created **Default value:** `null`
`iam_policies` optional
IAM policies to attach to the created IAM role for the Step Function. The map keys will be used as the policy SIDs **Type:** ```hcl map(object({ effect = string actions = optional(list(string)) not_actions = optional(list(string)) resources = optional(list(string)) not_resources = optional(list(string)) principals = optional(list(object({ type = string identifiers = list(string) }))) not_principals = optional(list(object({ type = string identifiers = list(string) }))) condition = optional(list(object({ test = string variable = string values = list(string) }))) })) ``` **Default value:** `{ }`
`logging_configuration` optional
Defines what execution history events are logged and where they are logged **Type:** ```hcl object({ log_destination = optional(string) include_execution_data = bool level = string }) ``` **Default value:** ```hcl { "include_execution_data": false, "level": "OFF" } ```
`role_description` (`string`) optional
Description of the created IAM role **Default value:** `null`
`role_force_detach_policies` (`bool`) optional
Specifies to force detaching any policies the created IAM role has before destroying it **Default value:** `true`
`role_name` (`string`) optional
Name of the created IAM role. If not provided, a name will be generated from the context **Default value:** `null`
`role_path` (`string`) optional
Path of the created IAM role **Default value:** `null`
`role_permissions_boundary` (`string`) optional
The ARN of the policy that is used to set the permissions boundary for the created IAM role **Default value:** `null`
`step_function_name` (`string`) optional
The name of the Step Function. If not provided, a name will be generated from the context **Default value:** `null`
`tracing_enabled` (`bool`) optional
When set to true, AWS X-Ray tracing is enabled. Make sure the State Machine has the correct IAM policies for logging **Default value:** `false`
`type` (`string`) optional
Determines whether a Standard or Express state machine is created. The default is STANDARD. Valid Values: STANDARD, EXPRESS **Default value:** `"STANDARD"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`role_arn`
The ARN of the IAM role created for the Step Function
`role_name`
The name of the IAM role created for the Step Function
`state_machine_arn`
State machine ARN
`state_machine_creation_date`
State machine creation date
`state_machine_id`
State machine ID
`state_machine_status`
State machine status
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.0` ### Providers - `aws`, version: `>= 4.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `logs_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_cloudwatch_log_group.logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) (resource) - [`aws_iam_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) (resource) - [`aws_iam_policy_attachment.logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) (resource) - [`aws_iam_role.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_sfn_state_machine.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/sfn_state_machine) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.logs`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## tfstate-backend(Tfstate-backend) # Module: `tfstate-backend` Terraform module to provision an S3 bucket to store `terraform.tfstate` file and a DynamoDB table to lock the state file to prevent concurrent modifications and state corruption. The module supports the following: 1. Forced server-side encryption at rest for the S3 bucket 2. S3 bucket versioning to allow for Terraform state recovery in the case of accidental deletions and human errors 3. State locking and consistency checking via DynamoDB table to prevent concurrent operations 4. DynamoDB server-side encryption https://www.terraform.io/docs/backends/types/s3.html __NOTE:__ The operators of the module (IAM Users) must have permissions to create S3 buckets and DynamoDB tables when performing `terraform plan` and `terraform apply` __NOTE:__ This module cannot be used to apply changes to the `mfa_delete` feature of the bucket. Changes regarding mfa_delete can only be made manually using the root credentials with MFA of the AWS Account where the bucket resides. Please see: https://github.com/terraform-providers/terraform-provider-aws/issues/629 ## Usage ### Create Follow this procedure just once to create your deployment. 1. Add the `terraform_state_backend` module to your `main.tf` file. The comment will help you remember to follow this procedure in the future: ```hcl # You cannot create a new backend by simply defining this and then # immediately proceeding to "terraform apply". The S3 backend must # be bootstrapped according to the simple yet essential procedure in # https://github.com/cloudposse/terraform-aws-tfstate-backend#usage module "terraform_state_backend" { source = "cloudposse/tfstate-backend/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "terraform" attributes = ["state"] terraform_backend_config_file_path = "." terraform_backend_config_file_name = "backend.tf" force_destroy = false } # Your Terraform configuration module "another_module" { source = "....." } ``` Module inputs `terraform_backend_config_file_path` and `terraform_backend_config_file_name` control the name of the backend definition file. Note that when `terraform_backend_config_file_path` is empty (the default), no file is created. 1. `terraform init`. This downloads Terraform modules and providers. 1. `terraform apply -auto-approve`. This creates the state bucket and DynamoDB locking table, along with anything else you have defined in your `*.tf` file(s). At this point, the Terraform state is still stored locally. Module `terraform_state_backend` also creates a new `backend.tf` file that defines the S3 state backend. For example: ```hcl backend "s3" { region = "us-east-1" bucket = "< the name of the S3 state bucket >" key = "terraform.tfstate" dynamodb_table = "< the name of the DynamoDB locking table >" profile = "" role_arn = "" encrypt = true } ``` Henceforth, Terraform will also read this newly-created backend definition file. 1. `terraform init -force-copy`. Terraform detects that you want to move your Terraform state to the S3 backend, and it does so per `-auto-approve`. Now the state is stored in the S3 bucket, and the DynamoDB table will be used to lock the state to prevent concurrent modification. This concludes the one-time preparation. Now you can extend and modify your Terraform configuration as usual. ### Destroy Follow this procedure to delete your deployment. 1. In `main.tf`, change the `terraform_state_backend` module arguments as follows: ```hcl module "terraform_state_backend" { # ... terraform_backend_config_file_path = "" force_destroy = true } ``` 1. `terraform apply -target module.terraform_state_backend -auto-approve`. This implements the above modifications by deleting the `backend.tf` file and enabling deletion of the S3 state bucket. 1. `terraform init -force-copy`. Terraform detects that you want to move your Terraform state from the S3 backend to local files, and it does so per `-auto-approve`. Now the state is once again stored locally and the S3 state bucket can be safely deleted. 1. `terraform destroy`. This deletes all resources in your deployment. 1. Examine local state file `terraform.tfstate` to verify that it contains no resources. ![s3-bucket-with-terraform-state](images/s3-bucket-with-terraform-state.png) ### Bucket Replication (Disaster Recovery) To enable S3 bucket replication in this module, set `s3_replication_enabled` to `true` and populate `s3_replica_bucket_arn` with the ARN of an existing bucket. ```hcl module "terraform_state_backend" { source = "cloudposse/tfstate-backend/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "terraform" attributes = ["state"] terraform_backend_config_file_path = "." terraform_backend_config_file_name = "backend.tf" force_destroy = false s3_replication_enabled = true s3_replica_bucket_arn = "arn:aws:s3:::eg-test-terraform-tfstate-replica" } ``` ## Variables ### Required Variables
### Optional Variables
`acl` (`string`) optional
The canned ACL to apply to the S3 bucket **Default value:** `"private"`
`arn_format` (`string`) optional
ARN format to be used. May be changed to support deployment in GovCloud/China regions. **Default value:** `"arn:aws"`
`billing_mode` (`string`) optional
DynamoDB billing mode **Default value:** `"PAY_PER_REQUEST"`
`block_public_acls` (`bool`) optional
Whether Amazon S3 should block public ACLs for this bucket **Default value:** `true`
`block_public_policy` (`bool`) optional
Whether Amazon S3 should block public bucket policies for this bucket **Default value:** `true`
`bucket_enabled` (`bool`) optional
Whether to create the S3 bucket. **Default value:** `true`
`bucket_ownership_enforced_enabled` (`bool`) optional
Set bucket object ownership to "BucketOwnerEnforced". Disables ACLs. **Default value:** `true`
`deletion_protection_enabled` (`bool`) optional
A boolean that enables deletion protection for DynamoDB table **Default value:** `false`
`dynamodb_enabled` (`bool`) optional
Whether to create the DynamoDB table. **Default value:** `true`
`dynamodb_table_name` (`string`) optional
Override the name of the DynamoDB table which defaults to using `module.dynamodb_table_label.id` **Default value:** `null`
`enable_point_in_time_recovery` (`bool`) optional
Enable DynamoDB point-in-time recovery **Default value:** `true`
`enable_public_access_block` (`bool`) optional
Enable Bucket Public Access Block **Default value:** `true`
`force_destroy` (`bool`) optional
A boolean that indicates the S3 bucket can be destroyed even if it contains objects. These objects are not recoverable **Default value:** `false`
`ignore_public_acls` (`bool`) optional
Whether Amazon S3 should ignore public ACLs for this bucket **Default value:** `true`
`kms_master_key_id` (`string`) optional
AWS KMS master key ID used for the SSE-KMS encryption. This can only be used when you set the value of sse_algorithm as aws:kms. **Default value:** `null`
`logging` optional
Destination (S3 bucket name and prefix) for S3 Server Access Logs for the S3 bucket. **Type:** ```hcl list(object({ target_bucket = string target_prefix = string })) ``` **Default value:** `[ ]`
`mfa_delete` (`bool`) optional
A boolean that indicates that versions of S3 objects can only be deleted with MFA. ( Terraform cannot apply changes of this value; https://github.com/terraform-providers/terraform-provider-aws/issues/629 ) **Default value:** `false`
`permissions_boundary` (`string`) optional
ARN of the policy that is used to set the permissions boundary for the IAM replication role **Default value:** `""`
`prevent_unencrypted_uploads` (`bool`) optional
Prevent uploads of unencrypted objects to S3 **Default value:** `true`
`profile` (`string`) optional
AWS profile name as set in the shared credentials file **Default value:** `""`
`read_capacity` (`number`) optional
DynamoDB read capacity units when using provisioned mode **Default value:** `5`
`restrict_public_buckets` (`bool`) optional
Whether Amazon S3 should restrict public bucket policies for this bucket **Default value:** `true`
`role_arn` (`string`) optional
The role to be assumed **Default value:** `""`
`s3_bucket_name` (`string`) optional
S3 bucket name. If not provided, the name will be generated from the context by the label module. **Default value:** `""`
`s3_replica_bucket_arn` (`string`) optional
The ARN of the S3 replica bucket (destination) **Default value:** `""`
`s3_replication_enabled` (`bool`) optional
Set this to true and specify `s3_replica_bucket_arn` to enable replication **Default value:** `false`
`s3_state_lock_enabled` (`bool`) optional
Whether to create the S3 bucket. **Default value:** `false`
`source_policy_documents` (`list(string)`) optional
List of IAM policy documents (in JSON format) that are merged together into the generated S3 bucket policy. Statements must have unique SIDs. Statement having SIDs that match policy SIDs generated by this module will override them. **Default value:** `[ ]`
`sse_encryption` (`string`) optional
The server-side encryption algorithm to use. Valid values are `AES256`, `aws:kms`, and `aws:kms:dsse`. **Default value:** `"AES256"`
`terraform_backend_config_file_name` (`string`) optional
(Deprecated) Name of terraform backend config file to generate **Default value:** `"terraform.tf"`
`terraform_backend_config_file_path` (`string`) optional
(Deprecated) Directory for the terraform backend config file, usually `.`. The default is to create no file. **Default value:** `""`
`terraform_backend_config_template_file` (`string`) optional
(Deprecated) The path to the template used to generate the config file **Default value:** `""`
`terraform_state_file` (`string`) optional
The path to the state file inside the bucket **Default value:** `"terraform.tfstate"`
`terraform_version` (`string`) optional
The minimum required terraform version **Default value:** `null`
`write_capacity` (`number`) optional
DynamoDB write capacity units when using provisioned mode **Default value:** `5`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`dynamodb_table_arn`
DynamoDB table ARN
`dynamodb_table_id`
DynamoDB table ID
`dynamodb_table_name`
DynamoDB table name
`s3_bucket_arn`
S3 bucket ARN
`s3_bucket_domain_name`
S3 bucket domain name
`s3_bucket_id`
S3 bucket ID
`s3_replication_role_arn`
The ARN of the IAM Role created for replication, if enabled.
`terraform_backend_config`
Rendered Terraform backend config file
## Dependencies ### Requirements - `terraform`, version: `>= 1.1.0` - `aws`, version: `>= 4.9.0` - `local`, version: `>= 2.0` - `time`, version: `>= 0.7.1` ### Providers - `aws`, version: `>= 4.9.0` - `local`, version: `>= 2.0` - `time`, version: `>= 0.7.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `bucket_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `dynamodb_table_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `replication_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_dynamodb_table.with_server_side_encryption`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/dynamodb_table) (resource) - [`aws_iam_policy.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_s3_bucket.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) (resource) - [`aws_s3_bucket_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_acl) (resource) - [`aws_s3_bucket_logging.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_logging) (resource) - [`aws_s3_bucket_object_lock_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_object_lock_configuration) (resource) - [`aws_s3_bucket_ownership_controls.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_ownership_controls) (resource) - [`aws_s3_bucket_policy.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_policy) (resource) - [`aws_s3_bucket_public_access_block.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_public_access_block) (resource) - [`aws_s3_bucket_replication_configuration.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_replication_configuration) (resource) - [`aws_s3_bucket_server_side_encryption_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_server_side_encryption_configuration) (resource) - [`aws_s3_bucket_versioning.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket_versioning) (resource) - [`local_file.terraform_backend_config`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) - [`time_sleep.wait_for_aws_s3_bucket_settings`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.aggregated_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.bucket_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.replication`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.replication_sts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_region.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) --- ## transfer-sftp # Module: `transfer-sftp` This is `terraform-aws-transfer-sftp` project provides all the scaffolding for a typical well-built Cloud Posse module. It's a template repository you can use when creating new repositories. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-transfer-sftp/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-transfer-sftp/tree/main/test). ```hcl module "example" { source = "https://github.com/cloudposse/terraform-aws-transfer-sftp.git?ref=master" example = "Hello world!" } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-transfer-sftp/) - complete example of using this module ## Variables ### Required Variables
`s3_bucket_name` (`string`) required
This is the bucket that the SFTP users will use when managing files
### Optional Variables
`address_allocation_ids` (`list(string)`) optional
A list of address allocation IDs that are required to attach an Elastic IP address to your SFTP server's endpoint. This property can only be used when endpoint_type is set to VPC. **Default value:** `[ ]`
`domain` (`string`) optional
Where your files are stored. S3 or EFS **Default value:** `"S3"`
`domain_name` (`string`) optional
Domain to use when connecting to the SFTP endpoint **Default value:** `""`
`eip_enabled` (`bool`) optional
Whether to provision and attach an Elastic IP to be used as the SFTP endpoint. An EIP will be provisioned per subnet. **Default value:** `false`
`force_destroy` (`bool`) optional
Forces the AWS Transfer Server to be destroyed **Default value:** `false`
`restricted_home` (`bool`) optional
Restricts SFTP users so they only have access to their home directories. **Default value:** `true`
`security_policy_name` (`string`) optional
Specifies the name of the security policy that is attached to the server. Possible values are TransferSecurityPolicy-2018-11, TransferSecurityPolicy-2020-06, and TransferSecurityPolicy-FIPS-2020-06. Default value is: TransferSecurityPolicy-2018-11. **Default value:** `"TransferSecurityPolicy-2018-11"`
`sftp_users` optional
Map of SFTP users and their configurations. Required: user_name, public_key. Optional: s3_bucket_name, bucket_permissions, home_directory_type, home_directory, home_directory_mappings **Type:** ```hcl map(object({ user_name = string public_key = string s3_bucket_name = optional(string) bucket_permissions = optional(list(string)) home_directory_type = optional(string) home_directory = optional(string) home_directory_mappings = optional(list(object({ entry = string target = string }))) })) ``` **Default value:** `{ }`
`subnet_ids` (`list(string)`) optional
A list of subnet IDs that are required to host your SFTP server endpoint in your VPC. This property can only be used when endpoint_type is set to VPC. **Default value:** `[ ]`
`vpc_id` (`string`) optional
VPC ID that the AWS Transfer Server will be deployed to **Default value:** `null`
`vpc_security_group_ids` (`list(string)`) optional
A list of security groups IDs that are available to attach to your server's endpoint. If no security groups are specified, the VPC's default security groups are automatically assigned to your endpoint. This property can only be used when endpoint_type is set to VPC. **Default value:** `[ ]`
`zone_id` (`string`) optional
Route53 Zone ID to add the CNAME **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
ARN of the created Transfer Server
`elastic_ips`
Provisioned Elastic IPs
`endpoint_details`
Endpoints details
`host_key_fingerprint`
The message-digest algorithm (MD5) hash of the Transfer Server's host key
`id`
ID of the created example
`s3_access_role_arns`
Role ARNs for the S3 access
`transfer_endpoint`
The endpoint of the Transfer Server
## Dependencies ### Requirements - `terraform`, version: `>= 1.1.0` - `aws`, version: `>= 3.0, != 4.0.0, != 4.1.0, != 4.2.0, != 4.3.0, != 4.4.0, != 4.5.0, != 4.6.0, != 4.7.0, != 4.8.0` ### Providers - `aws`, version: `>= 3.0, != 4.0.0, != 4.1.0, != 4.2.0, != 4.3.0, != 4.4.0, != 4.5.0, != 4.6.0, != 4.7.0, != 4.8.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `iam_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `logging_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_eip.sftp`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eip) (resource) - [`aws_iam_policy.logging`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_policy.s3_access_for_sftp_users`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) (resource) - [`aws_iam_role.logging`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role.s3_access_for_sftp_users`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) (resource) - [`aws_iam_role_policy_attachment.logging`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_iam_role_policy_attachment.s3_access_for_sftp_users`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) (resource) - [`aws_route53_record.main`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) (resource) - [`aws_transfer_server.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/transfer_server) (resource) - [`aws_transfer_ssh_key.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/transfer_ssh_key) (resource) - [`aws_transfer_user.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/transfer_user) (resource) ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.assume_role_policy`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.logging`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.s3_access_for_sftp_users`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) - [`aws_s3_bucket.landing`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/s3_bucket) (data source) --- ## transit-gateway # Module: `transit-gateway` Terraform module to provision: - [AWS Transit Gateway](https://aws.amazon.com/transit-gateway/) - [AWS Resource Access Manager (AWS RAM)](https://docs.aws.amazon.com/ram/latest/userguide/what-is.html) Resource Share to share the Transit Gateway with the Organization or another AWS Account (configurable via the variables `ram_resource_share_enabled` and `ram_principals`) - [Transit Gateway route table](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-route-tables.html) - [Transit Gateway VPC attachments](https://docs.aws.amazon.com/vpc/latest/tgw/tgw-vpc-attachments.html) to connect multiple VPCs via the Transit Gateway - Transit Gateway route table propagations to create propagated routes and allow traffic from the Transit Gateway to the VPC attachments - Transit Gateway route table associations to allow traffic from the VPC attachments to the Transit Gateway - Transit Gateway static routes (static routes have a higher precedence than propagated routes) - Subnet routes to route traffic from the subnets in each VPC to the other Transit Gateway VPC attachments ## Introduction This module is configurable via the variable `transit_gateway_config` - see [usage](#usage) and [examples](#examples) below. The variable `transit_gateway_config` is a map of environment names (e.g. `prod`, `staging`, `dev`) to the environment configurations. Each environment configuration contains the following fields: - `vpc_id` - The ID of the VPC for which to create a VPC attachment and route table associations and propagations. - `vpc_cidr` - VPC CIDR block. - `subnet_ids` - The IDs of the subnets in the VPC. - `static_routes` - A list of Transit Gateway static route configurations. Note that static routes have a higher precedence than propagated routes. - `subnet_route_table_ids` - The IDs of the subnet route tables. The route tables are used to add routes to allow traffix from the subnets in one VPC to the other VPC attachments. - `route_to` - A set of environment names to route traffic from the current environment to the specified environments. In the example below, in the `prod` environment we create subnet routes to route traffic from the `prod` subnets to the VPC attachments in the `staging` and `dev` environments. Specify either `route_to` or `route_to_cidr_blocks`. `route_to_cidr_blocks` supersedes `route_to`. - `route_to_cidr_blocks` - A set of VPC CIDR blocks to route traffic from the current environment to the specified VPC CIDR blocks. In the example below, in the `staging` environment we create subnet routes to route traffic from the `staging` subnets to the `dev` VPC CIDR. Specify either `route_to` or `route_to_cidr_blocks`. `route_to_cidr_blocks` supersedes `route_to`. - `transit_gateway_vpc_attachment_id` - An existing Transit Gateway Attachment ID. If provided, the module will use it instead of creating a new one. You now have the option to have Terraform manage route table entries by key, whereas previously they were only managed by index. The advantage of managing them by key is that if a route table ID or destination CIDR changes, only that entry is affected, whereas when managed by index, all the entries after the first affected index may be destroyed and re-created at a different index. The reason this is left as an option, with the default being to manage the entries by index, is that if you are creating the VPC or subnets at the same time you are creating the Transit Gateway, then Terraform will not be able to generate the keys during the plan phase and the plan will fail with the error `The "for_each" value depends on resource attributes that cannot be determined until apply...`. We recommend setting `route_keys_enabled` to `true` unless you get this error, in which case you must leave it set to its default value of `false`. __NOTE:__ This module requires Terraform 0.13 and newer since it uses [module expansion with `for_each`](https://www.hashicorp.com/blog/announcing-hashicorp-terraform-0-13/). ## Usage Here's how to invoke this module in your projects: ```hcl locals { transit_gateway_config = { prod = { vpc_id = module.vpc_prod.vpc_id vpc_cidr = module.vpc_prod.vpc_cidr_block subnet_ids = module.subnets_prod.private_subnet_ids subnet_route_table_ids = module.subnets_prod.private_route_table_ids route_to = ["staging", "dev"] route_to_cidr_blocks = null transit_gateway_vpc_attachment_id = null static_routes = [ { blackhole = true destination_cidr_block = "0.0.0.0/0" }, { blackhole = false destination_cidr_block = "172.16.1.0/24" } ] }, staging = { vpc_id = module.vpc_staging.vpc_id vpc_cidr = module.vpc_staging.vpc_cidr_block subnet_ids = module.subnets_staging.private_subnet_ids subnet_route_table_ids = module.subnets_staging.private_route_table_ids route_to = null route_to_cidr_blocks = [module.vpc_dev.vpc_cidr_block] transit_gateway_vpc_attachment_id = null static_routes = [ { blackhole = false destination_cidr_block = "172.32.1.0/24" } ] }, dev = { vpc_id = module.vpc_dev.vpc_id vpc_cidr = module.vpc_dev.vpc_cidr_block subnet_ids = module.subnets_dev.private_subnet_ids subnet_route_table_ids = module.subnets_dev.private_route_table_ids route_to = null route_to_cidr_blocks = null transit_gateway_vpc_attachment_id = null static_routes = null } } } module "transit_gateway" { source = "cloudposse/transit-gateway/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" ram_resource_share_enabled = false config = local.transit_gateway_config context = module.this.context } ``` ## Examples Here is a working example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-transit-gateway/tree/main/examples/complete) Here are automated tests for the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS): - [`test`](https://github.com/cloudposse/terraform-aws-transit-gateway/tree/main/test) Here is an example of using this module in a multi-account environment (with the Transit Gateway in one AWS account and all the VPC attachments and routes in different AWS accounts): - [`examples/multi-account`](https://github.com/cloudposse/terraform-aws-transit-gateway/tree/main/examples/multi-account) ## Variables ### Required Variables
### Optional Variables
`allow_external_principals` (`bool`) optional
Indicates whether principals outside your organization can be associated with a resource share **Default value:** `false`
`auto_accept_shared_attachments` (`string`) optional
Whether resource attachment requests are automatically accepted. Valid values: `disable`, `enable`. Default value: `disable` **Default value:** `"enable"`
`config` optional
Configuration for VPC attachments, Transit Gateway routes, and subnet routes **Type:** ```hcl map(object({ vpc_id = string vpc_cidr = string subnet_ids = set(string) subnet_route_table_ids = set(string) route_to = set(string) route_to_cidr_blocks = set(string) transit_gateway_vpc_attachment_id = string static_routes = set(object({ blackhole = bool destination_cidr_block = string })) })) ``` **Default value:** `null`
`create_transit_gateway` (`bool`) optional
Whether to create a Transit Gateway. If set to `false`, an existing Transit Gateway ID must be provided in the variable `existing_transit_gateway_id` **Default value:** `true`
`create_transit_gateway_route_table` (`bool`) optional
Whether to create a Transit Gateway Route Table. If set to `false`, an existing Transit Gateway Route Table ID must be provided in the variable `existing_transit_gateway_route_table_id` **Default value:** `true`
`create_transit_gateway_route_table_association_and_propagation` (`bool`) optional
Whether to create Transit Gateway Route Table associations and propagations **Default value:** `true`
`create_transit_gateway_vpc_attachment` (`bool`) optional
Whether to create Transit Gateway VPC Attachments **Default value:** `true`
`default_route_table_association` (`string`) optional
Whether resource attachments are automatically associated with the default association route table. Valid values: `disable`, `enable`. Default value: `disable` **Default value:** `"disable"`
`default_route_table_propagation` (`string`) optional
Whether resource attachments automatically propagate routes to the default propagation route table. Valid values: `disable`, `enable`. Default value: `disable` **Default value:** `"disable"`
`dns_support` (`string`) optional
Whether resource attachments automatically propagate routes to the default propagation route table. Valid values: `disable`, `enable`. Default value: `enable` **Default value:** `"enable"`
`existing_transit_gateway_id` (`string`) optional
Existing Transit Gateway ID. If provided, the module will not create a Transit Gateway but instead will use the existing one **Default value:** `null`
`existing_transit_gateway_route_table_id` (`string`) optional
Existing Transit Gateway Route Table ID. If provided, the module will not create a Transit Gateway Route Table but instead will use the existing one **Default value:** `null`
`ram_principal` (`string`) optional
DEPRECATED, please use ram_principals instead. The principal to associate with the resource share. Possible values are an AWS account ID, an Organization ARN, or an Organization Unit ARN. **Default value:** `null`
`ram_principals` (`list(string)`) optional
A list of principals to associate with the resource share. Possible values are: * AWS account ID * Organization ARN * Organization Unit ARN If this (and var.ram_principal) is not provided and `ram_resource_share_enabled` is `true`, the Organization ARN will be used. **Default value:** `[ ]`
`ram_resource_share_enabled` (`bool`) optional
Whether to enable sharing the Transit Gateway with the Organization using Resource Access Manager (RAM) **Default value:** `false`
`route_keys_enabled` (`bool`) optional
If true, Terraform will use keys to label routes, preventing unnecessary changes, but this requires that the VPCs and subnets already exist before using this module. If false, Terraform will use numbers to label routes, and a single change may cascade to a long list of changes because the index or order has changed, but this will work when the `true` setting generates the error `The "for_each" value depends on resource attributes...` **Default value:** `false`
`route_timeouts` optional
aws_route resource timeouts **Type:** ```hcl object({ create = optional(string), delete = optional(string), update = optional(string) }) ``` **Default value:** `{ }`
`security_group_referencing_support_enabled` (`bool`) optional
Enable or disable support for referencing security groups across VPCs in the transit gateway. **Default value:** `false`
`transit_gateway_cidr_blocks` (`list(string)`) optional
The list of associated CIDR blocks. It can contain up to 1 IPv4 CIDR block of size up to /24 and up to one IPv6 CIDR block of size up to /64. The IPv4 block must not be from range 169.254.0.0/16. **Default value:** `null`
`transit_gateway_description` (`string`) optional
Transit Gateway description. If not provided, one will be automatically generated. **Default value:** `""`
`vpc_attachment_appliance_mode_support` (`string`) optional
Whether Appliance Mode support is enabled. If enabled, a traffic flow between a source and destination uses the same Availability Zone for the VPC attachment for the lifetime of that flow. Valid values: `disable`, `enable` **Default value:** `"disable"`
`vpc_attachment_dns_support` (`string`) optional
Whether resource attachments automatically propagate routes to the default propagation route table. Valid values: `disable`, `enable`. Default value: `enable` **Default value:** `"enable"`
`vpc_attachment_ipv6_support` (`string`) optional
Whether resource attachments automatically propagate routes to the default propagation route table. Valid values: `disable`, `enable`. Default value: `enable` **Default value:** `"disable"`
`vpn_ecmp_support` (`string`) optional
Whether resource attachments automatically propagate routes to the default propagation route table. Valid values: `disable`, `enable`. Default value: `enable` **Default value:** `"enable"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`ram_resource_share_id`
RAM resource share ID
`subnet_route_ids`
Subnet route identifiers combined with destinations
`transit_gateway_arn`
Transit Gateway ARN
`transit_gateway_association_default_route_table_id`
Transit Gateway association default route table ID
`transit_gateway_id`
Transit Gateway ID
`transit_gateway_propagation_default_route_table_id`
Transit Gateway propagation default route table ID
`transit_gateway_route_ids`
Transit Gateway route identifiers combined with destinations
`transit_gateway_route_table_id`
Transit Gateway route table ID
`transit_gateway_vpc_attachment_ids`
Transit Gateway VPC attachment IDs
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.69.0` ### Providers - `aws`, version: `>= 5.69.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `subnet_route` | latest | [`./modules/subnet_route`](https://registry.terraform.io/modules/./modules/subnet_route/) | Create routes in the subnets' route tables to route traffic from subnets to the Transit Gateway VPC attachments Only route to VPCs of the environments defined in `route_to` attribute `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `transit_gateway_route` | latest | [`./modules/transit_gateway_route`](https://registry.terraform.io/modules/./modules/transit_gateway_route/) | Static Transit Gateway routes Static routes have a higher precedence than propagated routes https://docs.aws.amazon.com/vpc/latest/tgw/how-transit-gateways-work.html https://docs.aws.amazon.com/vpc/latest/tgw/tgw-route-tables.html ## Resources The following resources are used by this module: - [`aws_ec2_transit_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway) (resource) - [`aws_ec2_transit_gateway_route_table.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table) (resource) - [`aws_ec2_transit_gateway_route_table_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_association) (resource) - [`aws_ec2_transit_gateway_route_table_propagation.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_propagation) (resource) - [`aws_ec2_transit_gateway_vpc_attachment.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_vpc_attachment) (resource) - [`aws_ram_principal_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ram_principal_association) (resource) - [`aws_ram_resource_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ram_resource_association) (resource) - [`aws_ram_resource_share.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ram_resource_share) (resource) ## Data Sources The following data sources are used by this module: - [`aws_ec2_transit_gateway.this`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ec2_transit_gateway) (data source) - [`aws_organizations_organization.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/organizations_organization) (data source) - [`aws_vpc.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## utils # Module: `utils` This `terraform-aws-utils` module provides some simple utilities to use when working in AWS. ## Introduction This `terraform-aws-utils` module provides some simple utilities to use when working in AWS. More complex utilities are available through Cloud Posse's `utils` Terraform provider [terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils). ### Compact Alternative Codes (Abbreviations) This module's primary function is to provide compact alternative codes for Regions, Availability Zones, and Local Zones, codes which are guaranteed to use only digits and lower case letters: no hyphens. Conversions to and from official codes and alternative codes are handled via lookup maps. - The `short` abbreviations for regions are variable length (generally 4-6 characters, but length limits not guaranteed) and strictly algorithmically derived so that people can more easily interpret them. The `short` region code abbreviations typically match the prefix of the Availability Zone IDs in that region, but this is not guaranteed. The `short` abbreviations for local regions are generally of the form AWS uses, with the region prefix and dashes removed. - The `fixed` abbreviations are always exactly 3 characters for regions and 4 characters for availability zones and local zones, but have some exceptional cases (China, Africa, Asia-Pacific South, US GovCloud) that have non-obvious abbreviations. If a future new region causes a conflict with an established local zone abbreviation, we may change the local zone abbreviation to keep the region mappings consistent. For example, the local zone `us-east-1-mci-1a` would have been abbreviated `mc1a` had we released it earlier, and that would have conflicted with the new (in 2022) `me-central-1a` which would also be abbreviated `mc1a` in keeping with the general pattern of using the first letter of each of the first 2 parts. We might have chosen to change the abbreviation for `us-east-1-mci-1` so we could use `mc1a` for `me-central-1a`. (As it happens, we added them both at the same time and avoided this collision.) If we were to make such a change, this would be a breaking change for people using the affected local zone, so we recommend using the `short` abbreviations if you are using local zones, which are far less likely to have conflicts in the future. - The `identity` "abbreviations" are not abbreviations but are instead the official codes (output equals input, which is why it is called "identity"). This map is provided to simplify algorithmic choice of region code abbreviation when you want to include a "no abbreviation" option. We currently support Local Zones but not Wavelength Zones. If we support Wavelength Zones in the future, it is likely that the fixed-length abbreviations for them will be non-intuitive, or we may only provide `short` and not `fixed` abbreviations for them. The intention is that existing region mappings will never change, and if new regions or zones are created that conflict with existing ones, they will be given non-standard mappings so as not to conflict. However, as stated above, we may choose to change a local region abbreviation if it conflicts with the obvious abbreviation for a newly created region. We have picked abbreviations for local zones with avoiding such future collisions in mind, but we cannot predict the future. (Both `bos` and `den` fit the pattern for region abbreviations, but we do not envision a future `bo-south-1` or `de-north-1` region.) ### ELB Logging This module provides Elastic Load Balancing Account IDs per region to be used in configuring [S3 Bucket Permissions](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-access-logs.html#access-logging-bucket-permissions) to allow access logs to be stored in S3. However, the account IDs have no other purpose, and as AWS expands, it has become more complicated to create the correct bucket policy. The policy for region `me-central-1` is different than the policy for `us-east-1`. So now this module has a new feature: you provide the full AWS region code for the region where logging is to take place (`elb_logging_region`), and the S3 bucket ARN for where logs are to be stored (`elb_logging_bucket_resource_arn`), and this module will output the appropriate S3 bucket policy (in JSON) to attach to your S3 bucket. NOTE: The region must be known at Terraform "plan" time. Use a configuration input, such as what you used to configure the Terraform AWS Provider, not an output from some resource or module. ### Region Display Names There is no AWS API that reliably returns the human-friendly display name (e.g. "Europe (Stockholm)") given the API-friendly region name. So this module provides `region_display_name_map` to implement this functionality. ### Enabled and Disabled Regions For convenience, this module provides lists of enabled and disabled regions in the current account. Note that since these lists are dynamic, they cannot be used in Terraform `count` or `for_each` resource expressions. ## Usage Here's how to invoke this example module in your projects ```hcl locals { shorten_regions = true naming_convention = local.shorten_regions ? "to_short" : "identity" az_map = module.example.region_az_alt_code_maps[local.naming_convention] } module "utils_example_complete" { source = "cloudposse/utils/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" } module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = [local.az_map["us-east-2"]] context = module.this.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-utils/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`elb_logging_bucket_resource_arn` (`string`) optional
The AWS Resource ARN to use in the policy granting access to Load Balancer Logging. Typically of the form `arn:aws:s3:::_bucket-name_/_prefix_/AWSLogs/_your-aws-account-id_/*`. Required to generate `elb_logging_s3_bucket_policy_json`. See [AWS Documentation](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/enable-access-logging.html#attach-bucket-policy). **Default value:** `""`
`elb_logging_region` (`string`) optional
Full region (e.g. `us-east-1`) where ELB logging is taking place. Required to generate `elb_s3_bucket_policy_json`. Must be known at "plan" time. **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`all_regions`
A list of all regions regardless of availability to the account
`disabled_regions`
A list of regions that are disabled in the account
`elb_logging_account`
Map of full region to ELB logging account
`elb_logging_s3_bucket_policy_json`
The S3 bucket policy (in JSON) to attach to the S3 bucket to allow Load Balancer logs to be added. Requires `elb_logging_bucket_resource_arn` and `elb_logging_region` inputs.
`enabled_regions`
A list of regions that are enabled in the account
`region_az_alt_code_maps`
Collection of maps converting between official AWS Region, Availability Zone, and Local Zone codes and shorter unofficial codes using only lower case letters and digits. Inspired for use in naming and tagging so that region or AZ code will be 1 semantic unit. - `to_fixed` = Map of regions to 3-character codes and Availability Zones to 4-character codes - `to_short` = Map of regions and Availability Zones to compact (usually 4-6 characters) codes - `from_fixed` = Map of `fixed` codes back to full region or Availability Zone codes - `from_short` = Map of `short` codes back to full region or Availability Zone codes - `identity` = Identity map of full region and Availability Zone codes back to themselves
`region_display_name_map`
Map of full region names to user-friendly display names (e.g. "eu-west-3" = "Europe (Paris)").
## Dependencies ### Requirements - `terraform`, version: `>= 0.14.0` - `aws`, version: `>= 2` ### Providers - `aws`, version: `>= 2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`aws_iam_policy_document.by_account`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.by_region`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_regions.complete`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/regions) (data source) - [`aws_regions.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/regions) (data source) - [`aws_regions.not_opted_in`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/regions) (data source) - [`aws_regions.opted_in`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/regions) (data source) --- ## vpc(Vpc) # Module: `vpc` Terraform module to provision a VPC with Internet Gateway. Contains a submodule for provisioning Interface and/or Gateway VPC Endpoints. This module also supports provisioning additional CIDR blocks for the VPC, with or without using [IPAM](https://docs.aws.amazon.com/vpc/latest/ipam/what-it-is-ipam.html). ## Examples ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" ipv4_primary_cidr_block = "10.0.0.0/16" assign_generated_ipv6_cidr_block = true } ``` Full example with [`terraform-aws-dynamic-subnets`](https://github.com/cloudposse/terraform-aws-dynamic-subnets.git): ```hcl module "vpc" { source = "cloudposse/vpc/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" ipv4_primary_cidr_block = "10.0.0.0/16" assign_generated_ipv6_cidr_block = false } module "dynamic_subnets" { source = "cloudposse/dynamic-subnets/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" availability_zones = ["us-west-2a","us-west-2b","us-west-2c"] vpc_id = module.vpc.vpc_id igw_id = [module.vpc.igw_id] cidr_block = "10.0.0.0/16" } ``` Submodule for provisioning VPC Endpoints: ```hcl module "vpc_endpoints" { source = "cloudposse/vpc/aws//modules/vpc-endpoints" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" vpc_id = module.vpc.vpc_id gateway_vpc_endpoints = { "s3" = { name = "s3" policy = jsonencode({ Version = "2012-10-17" Statement = [ { Action = [ "s3:*", ] Effect = "Allow" Principal = "*" Resource = "*" }, ] }) } } interface_vpc_endpoints = { "ec2" = { name = "ec2" security_group_ids = ["sg-12341234123412345"] subnet_ids = module.dynamic_subnets.private_subnet_ids policy = null private_dns_enabled = false } } } ``` ## Variables ### Required Variables
### Optional Variables
`assign_generated_ipv6_cidr_block` (`bool`) optional
When `true`, assign AWS generated IPv6 CIDR block to the VPC. Conflicts with `ipv6_ipam_pool_id`. **Default value:** `true`
`default_network_acl_deny_all` (`bool`) optional
When `true`, manage the default network acl and remove all rules, disabling all ingress and egress. When `false`, do not mange the default networking acl, allowing it to be managed by another component. **Default value:** `false`
`default_route_table_no_routes` (`bool`) optional
When `true`, manage the default route table and remove all routes, disabling all ingress and egress. When `false`, do not mange the default route table, allowing it to be managed by another component. Conflicts with Terraform resource `aws_main_route_table_association`. **Default value:** `false`
`default_security_group_deny_all` (`bool`) optional
When `true`, manage the default security group and remove all rules, disabling all ingress and egress. When `false`, do not manage the default security group, allowing it to be managed by another component. **Default value:** `true`
`dns_hostnames_enabled` (`bool`) optional
Set `true` to enable [DNS hostnames](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-dns.html#vpc-dns-hostnames) in the VPC **Default value:** `true`
`dns_support_enabled` (`bool`) optional
Set `true` to enable DNS resolution in the VPC through the Amazon provided DNS server **Default value:** `true`
`instance_tenancy` (`string`) optional
A tenancy option for instances launched into the VPC **Default value:** `"default"`
`internet_gateway_enabled` (`bool`) optional
Set `true` to create an Internet Gateway for the VPC **Default value:** `true`
`ipv4_additional_cidr_block_associations` optional
IPv4 CIDR blocks to assign to the VPC. `ipv4_cidr_block` can be set explicitly, or set to `null` with the CIDR block derived from `ipv4_ipam_pool_id` using `ipv4_netmask_length`. Map keys must be known at `plan` time, and are only used to track changes. **Type:** ```hcl map(object({ ipv4_cidr_block = string ipv4_ipam_pool_id = string ipv4_netmask_length = number })) ``` **Default value:** `{ }`
`ipv4_cidr_block_association_timeouts` optional
Timeouts (in `go` duration format) for creating and destroying IPv4 CIDR block associations **Type:** ```hcl object({ create = string delete = string }) ``` **Default value:** `null`
`ipv4_primary_cidr_block` (`string`) optional
The primary IPv4 CIDR block for the VPC. Either `ipv4_primary_cidr_block` or `ipv4_primary_cidr_block_association` must be set, but not both. **Default value:** `null`
`ipv4_primary_cidr_block_association` optional
Configuration of the VPC's primary IPv4 CIDR block via IPAM. Conflicts with `ipv4_primary_cidr_block`. One of `ipv4_primary_cidr_block` or `ipv4_primary_cidr_block_association` must be set. Additional CIDR blocks can be set via `ipv4_additional_cidr_block_associations`. **Type:** ```hcl object({ ipv4_ipam_pool_id = string ipv4_netmask_length = number }) ``` **Default value:** `null`
`ipv6_additional_cidr_block_associations` optional
IPv6 CIDR blocks to assign to the VPC (in addition to the autogenerated one). `ipv6_cidr_block` can be set explicitly, or set to `null` with the CIDR block derived from `ipv6_ipam_pool_id` using `ipv6_netmask_length`. Map keys must be known at `plan` time and are used solely to prevent unnecessary changes. **Type:** ```hcl map(object({ ipv6_cidr_block = string ipv6_ipam_pool_id = string ipv6_netmask_length = number })) ``` **Default value:** `{ }`
`ipv6_cidr_block_association_timeouts` optional
Timeouts (in `go` duration format) for creating and destroying IPv6 CIDR block associations **Type:** ```hcl object({ create = string delete = string }) ``` **Default value:** `null`
`ipv6_cidr_block_network_border_group` (`string`) optional
Set this to restrict advertisement of public addresses to a specific Network Border Group such as a LocalZone. Requires `assign_generated_ipv6_cidr_block` to be set to `true`. **Default value:** `null`
`ipv6_egress_only_internet_gateway_enabled` (`bool`) optional
Set `true` to create an IPv6 Egress-Only Internet Gateway for the VPC **Default value:** `false`
`ipv6_primary_cidr_block_association` optional
Primary IPv6 CIDR block to assign to the VPC. Conflicts with `assign_generated_ipv6_cidr_block`. `ipv6_cidr_block` can be set explicitly, or set to `null` with the CIDR block derived from `ipv6_ipam_pool_id` using `ipv6_netmask_length`. **Type:** ```hcl object({ ipv6_cidr_block = string ipv6_ipam_pool_id = string ipv6_netmask_length = number }) ``` **Default value:** `null`
`network_address_usage_metrics_enabled` (`bool`) optional
Set `true` to enable Network Address Usage Metrics for the VPC **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`additional_cidr_blocks`
A list of the additional IPv4 CIDR blocks associated with the VPC
`additional_cidr_blocks_to_association_ids`
A map of the additional IPv4 CIDR blocks to VPC CIDR association IDs
`additional_ipv6_cidr_blocks`
A list of the additional IPv6 CIDR blocks associated with the VPC
`additional_ipv6_cidr_blocks_to_association_ids`
A map of the additional IPv6 CIDR blocks to VPC CIDR association IDs
`igw_id`
The ID of the Internet Gateway
`ipv6_cidr_block_network_border_group`
The IPv6 Network Border Group Zone name
`ipv6_egress_only_igw_id`
The ID of the egress-only Internet Gateway
`vpc_arn`
The ARN of the VPC
`vpc_cidr_block`
The primary IPv4 CIDR block of the VPC
`vpc_default_network_acl_id`
The ID of the network ACL created by default on VPC creation
`vpc_default_route_table_id`
The ID of the route table created by default on VPC creation
`vpc_default_security_group_id`
The ID of the security group created by default on VPC creation
`vpc_id`
The ID of the VPC
`vpc_ipv6_association_id`
The association ID for the primary IPv6 CIDR block
`vpc_ipv6_cidr_block`
The primary IPv6 CIDR block
`vpc_main_route_table_id`
The ID of the main route table associated with this VPC
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `aws`, version: `>= 4.9.0` ### Providers - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `eigw_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `igw_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_default_network_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/default_network_acl) (resource) - [`aws_default_route_table.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/default_route_table) (resource) - [`aws_default_security_group.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/default_security_group) (resource) - [`aws_egress_only_internet_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/egress_only_internet_gateway) (resource) - [`aws_internet_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/internet_gateway) (resource) - [`aws_vpc.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc) (resource) - [`aws_vpc_ipv4_cidr_block_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_ipv4_cidr_block_association) (resource) - [`aws_vpc_ipv6_cidr_block_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_ipv6_cidr_block_association) (resource) ## Data Sources The following data sources are used by this module: --- ## vpc-endpoints # VPC Endpoints Submodule for provisioning Gateway and/or Interface VPC Endpoints to the VPC created by `terraform-aws-vpc`. ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.0.0 | | [aws](#requirement\_aws) | >= 4.9.0 | | [time](#requirement\_time) | >= 0.8.0 | ## Providers | Name | Version | |------|---------| | [aws](#provider\_aws) | >= 4.9.0 | | [time](#provider\_time) | >= 0.8.0 | ## Modules | Name | Source | Version | |------|--------|---------| | [gateway\_endpoint\_label](#module\_gateway\_endpoint\_label) | cloudposse/label/null | 0.25.0 | | [interface\_endpoint\_label](#module\_interface\_endpoint\_label) | cloudposse/label/null | 0.25.0 | | [this](#module\_this) | cloudposse/label/null | 0.25.0 | ## Resources | Name | Type | |------|------| | [aws_vpc_endpoint.gateway_endpoint](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint) | resource | | [aws_vpc_endpoint.interface_endpoint](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint) | resource | | [aws_vpc_endpoint_route_table_association.gateway](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_route_table_association) | resource | | [aws_vpc_endpoint_subnet_association.interface](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_endpoint_subnet_association) | resource | | [time_sleep.creation](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) | resource | | [aws_vpc_endpoint.gateway](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint) | data source | | [aws_vpc_endpoint.interface](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint) | data source | | [aws_vpc_endpoint_service.gateway_endpoint_service](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint_service) | data source | | [aws_vpc_endpoint_service.interface_endpoint_service](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc_endpoint_service) | data source | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.This is for some rare cases where resources want additional configuration of tagsand therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,in the order they appear in the list. New attributes are appended to theend of the list. The elements of the list are joined by the `delimiter`and treated as a single ID element. | `list(string)` | `[]` | no | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "descriptor_formats": \{\},  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "labels_as_tags": [    "unset"  ],  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\},  "tenant": null\}
| no | | [delimiter](#input\_delimiter) | Delimiter to be used between ID elements.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [descriptor\_formats](#input\_descriptor\_formats) | Describe additional descriptors to be output in the `descriptors` output map.Map of maps. Keys are names of descriptors. Values are maps of the form`{ format = string labels = list(string)}`(Type is `any` so the map values can later be enhanced to provide additional options.)`format` is a Terraform format string to be passed to the `format()` function.`labels` is a list of labels, in order, to pass to `format()` function.Label values will be normalized before being passed to `format()` so they will beidentical to how they appear in `id`.Default is `{}` (`descriptors` output will be empty). | `any` | `{}` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [gateway\_vpc\_endpoints](#input\_gateway\_vpc\_endpoints) | A map of Gateway VPC Endpoints to provision into the VPC. This is a map of objects with the following attributes:- `name`: Short service name (either "s3" or "dynamodb")- `policy` = A policy (as JSON string) to attach to the endpoint that controls access to the service. May be `null` for full access.- `route_table_ids`: List of route tables to associate the gateway with. Routes to the gateway will be automatically added to these route tables. |
map(object(\{    name            = string    policy          = string    route_table_ids = list(string)  \}))
| `{}` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for keep the existing setting, which defaults to `0`.Does not affect `id_full`. | `number` | `null` | no | | [interface\_vpc\_endpoints](#input\_interface\_vpc\_endpoints) | A map of Interface VPC Endpoints to provision into the VPC.This is a map of objects with the following attributes:- `name`: Simple name of the service, like "ec2" or "redshift"- `policy`: A policy (as JSON string) to attach to the endpoint that controls access to the service. May be `null` for full access.- `private_dns_enabled`: Set `true` to associate a private hosted zone with the specified VPC- `security_group_ids`: The ID of one or more security groups to associate with the network interface. The first security group will replace the default association with the VPC's default security group. If you want to maintain the association with the default security group, either leave `security_group_ids` empty or include the default security group ID in the list.- `subnet_ids`: List of subnet in which to install the endpoints. |
map(object(\{    name                = string    policy              = string    private_dns_enabled = bool    security_group_ids  = list(string)    subnet_ids          = list(string)  \}))
| `{}` | no | | [label\_key\_case](#input\_label\_key\_case) | Controls the letter case of the `tags` keys (label names) for tags generated by this module.Does not affect keys of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The order in which the labels (ID elements) appear in the `id`.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | Controls the letter case of ID elements (labels) as included in `id`,set as tag values, and output by this module individually.Does not affect values of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper` and `none` (no transformation).Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs.Default value: `lower`. | `string` | `null` | no | | [labels\_as\_tags](#input\_labels\_as\_tags) | Set of labels (ID elements) to include as tags in the `tags` output.Default is to include all labels.Tags with empty values will not be included in the `tags` output.Set to `[]` to suppress all generated tags.**Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. | `set(string)` |
[  "default"]
| no | | [name](#input\_name) | ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.This is the only ID element not also included as a `tag`.The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. | `string` | `null` | no | | [namespace](#input\_namespace) | ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique | `string` | `null` | no | | [post\_creation\_refresh\_delay](#input\_post\_creation\_refresh\_delay) | After endpoints are created via Terraform, some additional resources are provisioned by AWSthat do not immediately appear in the resource, and therefore would not appear in the outputof the resources attributes. Examples include private DNS entries and Network Interface IDs.This input (in `go` duration format) sets a time delay to allow for such activity, after whichthe endpoint data is fetched via data sources for output. | `string` | `"30s"` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.Characters matching the regex will be removed from the ID elements.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [stage](#input\_stage) | ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input\_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | | [vpc\_id](#input\_vpc\_id) | VPC ID where the VPC Endpoints will be created (e.g. `vpc-aceb2723`) | `string` | n/a | yes | ## Outputs | Name | Description | |------|-------------| | [gateway\_vpc\_endpoints\_map](#output\_gateway\_vpc\_endpoints\_map) | Map of Gateway VPC Endpoints deployed to this VPC, using keys supplied in `var.gateway_vpc_endpoints`. | | [interface\_vpc\_endpoints\_map](#output\_interface\_vpc\_endpoints\_map) | Map of Interface VPC Endpoints deployed to this VPC, using keys supplied in `var.interface_vpc_endpoints`. | --- ## vpc-flow-logs-s3-bucket # Module: `vpc-flow-logs-s3-bucket` Terraform module to create AWS [`VPC Flow logs`](https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html) backed by S3. ## Introduction The module will create: * S3 bucket with server side encryption * KMS key to encrypt flow logs files in the bucket * Optional VPC Flow Log backed by the S3 bucket (this can be disabled, e.g. in multi-account environments if you want to create an S3 bucket in one account and VPC Flow Logs in different accounts) ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-vpc-flow-logs-s3-bucket/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on Datadog), see [test](https://github.com/cloudposse/terraform-aws-vpc-flow-logs-s3-bucket/tree/main/test). ```hcl module "vpc" { source = "cloudposse/vpc/aws" version = "0.18.0" namespace = "eg" stage = "test" name = "flowlogs" cidr_block = "172.16.0.0/16" } module "flow_logs" { source = "cloudposse/vpc-flow-logs-s3-bucket/aws" version = "0.8.0" namespace = "eg" stage = "test" name = "flowlogs" vpc_id = module.vpc.vpc_id } ``` ## Variables ### Required Variables
### Optional Variables
`access_log_bucket_name` (`string`) optional
Name of the S3 bucket where S3 access log will be sent to **Default value:** `""`
`access_log_bucket_prefix` (`string`) optional
Prefix to prepend to the current S3 bucket name, where S3 access logs will be sent to **Default value:** `"logs/"`
`acl` (`string`) optional
The canned ACL to apply. We recommend log-delivery-write for compatibility with AWS services **Default value:** `"log-delivery-write"`
`allow_ssl_requests_only` (`bool`) optional
Set to `true` to require requests to use Secure Socket Layer (HTTPS/SSL). This will explicitly deny access to HTTP requests **Default value:** `true`
`bucket_key_enabled` (`bool`) optional
Set this to true to use Amazon S3 Bucket Keys for SSE-KMS, which may reduce the number of AWS KMS requests. For more information, see: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html **Default value:** `true`
`bucket_name` (`string`) optional
Bucket name. If not provided, the bucket will be created with a name generated from the context. **Default value:** `""`
`bucket_notifications_enabled` (`bool`) optional
Send notifications for the object created events. Used for 3rd-party log collection from a bucket **Default value:** `false`
`bucket_notifications_prefix` (`string`) optional
Prefix filter. Used to manage object notifications **Default value:** `""`
`bucket_notifications_type` (`string`) optional
Type of the notification configuration. Only SQS is supported. **Default value:** `"SQS"`
`expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to expunge the objects **Default value:** `null`
`flow_log_enabled` (`bool`) optional
Enable/disable the Flow Log creation. Useful in multi-account environments where the bucket is in one account, but VPC Flow Logs are in different accounts **Default value:** `true`
`force_destroy` (`bool`) optional
A boolean that indicates all objects should be deleted from the bucket so that the bucket can be destroyed without error. These objects are not recoverable **Default value:** `false`
`glacier_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days after which to move the data to the Glacier Flexible Retrieval storage tier **Default value:** `null`
`kms_key_arn` (`string`) optional
ARN (or alias) of KMS that will be used for s3 bucket encryption. **Default value:** `""`
`kms_policy_source_json` (`string`) optional
Additional IAM policy document that can optionally be merged with default created KMS policy **Default value:** `""`
`lifecycle_configuration_rules` optional
A list of lifecycle V2 rules **Type:** ```hcl list(object({ enabled = bool id = string abort_incomplete_multipart_upload_days = number # `filter_and` is the `and` configuration block inside the `filter` configuration. # This is the only place you should specify a prefix. filter_and = optional(object({ object_size_greater_than = optional(number) # integer >= 0 object_size_less_than = optional(number) # integer >= 1 prefix = optional(string) tags = optional(map(string)) })) expiration = optional(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer > 0 expired_object_delete_marker = optional(bool) })) noncurrent_version_expiration = optional(object({ newer_noncurrent_versions = optional(number) # integer > 0 noncurrent_days = optional(number) # integer >= 0 })) transition = optional(list(object({ date = optional(string) # string, RFC3339 time format, GMT days = optional(number) # integer >= 0 storage_class = string # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) noncurrent_version_transition = optional(list(object({ newer_noncurrent_versions = optional(number) # integer >= 0 noncurrent_days = optional(number) # integer >= 0 storage_class = string # string/enum, one of GLACIER, STANDARD_IA, ONEZONE_IA, INTELLIGENT_TIERING, DEEP_ARCHIVE, GLACIER_IR. })), []) })) ``` **Default value:** `[ ]`
`lifecycle_prefix` (`string`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Prefix filter. Used to manage object lifecycle events **Default value:** `null`
`lifecycle_rule_enabled` (`bool`) optional
DEPRECATED: Use `lifecycle_configuration_rules` instead. When `true`, configures lifecycle events on this bucket using individual (now deprecated) variables. When `false`, lifecycle events are not configured using individual (now deprecated) variables, but `lifecycle_configuration_rules` still apply. When `null`, lifecycle events are configured using individual (now deprecated) variables only if `lifecycle_configuration_rules` is empty. **Default value:** `null`
`lifecycle_tags` (`map(string)`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Tags filter. Used to manage object lifecycle events **Default value:** `null`
`log_format` (`string`) optional
The fields to include in the flow log record, in the order in which they should appear. **Default value:** `null`
`noncurrent_version_expiration_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies when non-current object versions expire (in days) **Default value:** `null`
`noncurrent_version_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Specifies (in days) when noncurrent object versions transition to Glacier Flexible Retrieval **Default value:** `null`
`object_lock_configuration` optional
A configuration for [S3 object locking](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html). **Type:** ```hcl object({ mode = string # Valid values are GOVERNANCE and COMPLIANCE. days = number years = number }) ``` **Default value:** `null`
`s3_object_ownership` (`string`) optional
Specifies the S3 object ownership control. Valid values are `ObjectWriter`, `BucketOwnerPreferred`, and `BucketOwnerEnforced`." Default of `BucketOwnerPreferred` is for backwards compatibility. Recommended setting is `BucketOwnerEnforced`, which will be used if you pass in `null`. **Default value:** `"BucketOwnerPreferred"`
`standard_transition_days` (`number`) optional
(Deprecated, use `lifecycle_configuration_rules` instead) Number of days to persist in the standard storage tier before moving to the infrequent access tier **Default value:** `null`
`traffic_type` (`string`) optional
The type of traffic to capture. Valid values: `ACCEPT`, `REJECT`, `ALL` **Default value:** `"ALL"`
`versioning_enabled` (`bool`) optional
Enable object versioning, keeping multiple variants of an object in the same bucket **Default value:** `false`
`vpc_id` (`string`) optional
VPC ID to create flow logs for **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`bucket_arn`
Bucket ARN
`bucket_domain_name`
FQDN of bucket
`bucket_id`
Bucket Name (aka ID)
`bucket_notifications_sqs_queue_arn`
Notifications SQS queue ARN
`bucket_prefix`
Bucket prefix configured for lifecycle rules
`flow_log_arn`
Flow Log ARN
`flow_log_id`
Flow Log ID
`kms_alias_arn`
KMS Alias ARN
`kms_alias_name`
KMS Alias name
`kms_key_arn`
KMS Key ARN
`kms_key_id`
KMS Key ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 4.9.0` ### Providers - `aws`, version: `>= 4.9.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `bucket_name` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `kms_key` | 0.12.2 | [`cloudposse/kms-key/aws`](https://registry.terraform.io/modules/cloudposse/kms-key/aws/0.12.2) | n/a `s3_log_storage_bucket` | 2.0.0 | [`cloudposse/s3-log-storage/aws`](https://registry.terraform.io/modules/cloudposse/s3-log-storage/aws/2.0.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_flow_log.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/flow_log) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_iam_policy_document.bucket`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_iam_policy_document.kms`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) (data source) - [`aws_partition.current`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) (data source) --- ## vpc-peering(Vpc-peering) # Module: `vpc-peering` Terraform module to create a peering connection between two VPCs ## Screenshots ![vpc-peering](images/vpc-peering.png) *VPC Peering Connection in the AWS Web Console* ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-vpc-peering/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-vpc-peering/tree/main/test). ### Using VPC IDs ```hcl module "vpc_peering" { source = "cloudposse/vpc-peering/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" requestor_vpc_id = "vpc-XXXXXXXX" acceptor_vpc_id = "vpc-YYYYYYYY" } ``` ### Using VPC tags ```hcl module "vpc_peering" { source = "cloudposse/vpc-peering/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" requestor_vpc_tags = { "kubernetes.io/cluster/my-k8s" = "owned" } acceptor_vpc_tags = { Name = "legacy-vpc" } } ``` ## Variables ### Required Variables
### Optional Variables
`acceptor_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow acceptor VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the requestor VPC **Default value:** `true`
`acceptor_ignore_cidrs` (`list(string)`) optional
A list of CIDR blocks from the acceptor VPC to ignore **Default value:** `[ ]`
`acceptor_route_table_tags` (`map(string)`) optional
Only add peer routes to acceptor VPC route tables matching these tags **Default value:** `{ }`
`acceptor_vpc_id` (`string`) optional
Acceptor VPC ID **Default value:** `""`
`acceptor_vpc_tags` (`map(string)`) optional
Acceptor VPC tags **Default value:** `{ }`
`auto_accept` (`bool`) optional
Automatically accept the peering (both VPCs need to be in the same AWS account) **Default value:** `true`
`create_timeout` (`string`) optional
VPC peering connection create timeout. For more details, see https://www.terraform.io/docs/configuration/resources.html#operation-timeouts **Default value:** `"3m"`
`delete_timeout` (`string`) optional
VPC peering connection delete timeout. For more details, see https://www.terraform.io/docs/configuration/resources.html#operation-timeouts **Default value:** `"5m"`
`requestor_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow requestor VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the acceptor VPC **Default value:** `true`
`requestor_ignore_cidrs` (`list(string)`) optional
A list of CIDR blocks from the requestor VPC to ignore **Default value:** `[ ]`
`requestor_route_table_tags` (`map(string)`) optional
Only add peer routes to requestor VPC route tables matching these tags **Default value:** `{ }`
`requestor_vpc_id` (`string`) optional
Requestor VPC ID **Default value:** `""`
`requestor_vpc_tags` (`map(string)`) optional
Requestor VPC tags **Default value:** `{ }`
`update_timeout` (`string`) optional
VPC peering connection update timeout. For more details, see https://www.terraform.io/docs/configuration/resources.html#operation-timeouts **Default value:** `"3m"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`accept_status`
The status of the VPC peering connection request
`connection_id`
VPC peering connection ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 5.0` ### Providers - `aws`, version: `>= 5.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route.acceptor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.requestor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_vpc_peering_connection.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_peering_connection) (resource) ## Data Sources The following data sources are used by this module: - [`aws_route_tables.acceptor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_tables) (data source) - [`aws_route_tables.requestor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_tables) (data source) - [`aws_vpc.acceptor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) - [`aws_vpc.requestor`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## vpc-peering-multi-account # Module: `vpc-peering-multi-account` Terraform module to create a peering connection between any two VPCs existing in different AWS accounts. This module supports performing this action from a 3rd account (e.g. a "root" account) by specifying the roles to assume for each member account. **IMPORTANT:** AWS allows a multi-account VPC Peering Connection to be deleted from either the requester's or accepter's side. However, Terraform only allows the VPC Peering Connection to be deleted from the requester's side by removing the corresponding `aws_vpc_peering_connection` resource from your configuration. [Read more about this](https://www.terraform.io/docs/providers/aws/r/vpc_peering_accepter.html) on Terraform's documentation portal. ## Screenshots ![vpc-peering](images/vpc-peering.png) *VPC Peering Connection in the AWS Web Console* ## Usage **IMPORTANT:** Do not pin to `master` because there may be breaking changes between releases. Instead pin to the release tag (e.g. `?ref=tags/x.y.z`) of one of our [latest releases](https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account/releases). For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-vpc-peering-multi-account/tree/main/examples/complete) ```hcl module "vpc_peering_cross_account" { source = "cloudposse/vpc-peering-multi-account/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "cluster" requester_aws_assume_role_arn = "arn:aws:iam::XXXXXXXX:role/cross-account-vpc-peering-test" requester_region = "us-west-2" requester_vpc_id = "vpc-xxxxxxxx" requester_allow_remote_vpc_dns_resolution = true accepter_aws_assume_role_arn = "arn:aws:iam::YYYYYYYY:role/cross-account-vpc-peering-test" accepter_region = "us-east-1" accepter_vpc_id = "vpc-yyyyyyyy" accepter_allow_remote_vpc_dns_resolution = true } ``` The `arn:aws:iam::XXXXXXXX:role/cross-account-vpc-peering-test` requester IAM Role should have the following Trust Policy:
Show Trust Policy ```js { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::XXXXXXXX:root" }, "Action": "sts:AssumeRole", "Condition": {} } ] } ```
and the following IAM Policy attached to it: __NOTE:__ the policy specifies the permissions to create (with `terraform plan/apply`) and delete (with `terraform destroy`) all the required resources in the requester AWS account
Show IAM Policy ```js { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:CreateRoute", "ec2:DeleteRoute" ], "Resource": "arn:aws:ec2:*:XXXXXXXX:route-table/*" }, { "Effect": "Allow", "Action": [ "ec2:DescribeVpcPeeringConnections", "ec2:DescribeVpcs", "ec2:ModifyVpcPeeringConnectionOptions", "ec2:DescribeSubnets", "ec2:DescribeVpcAttribute", "ec2:DescribeRouteTables" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AcceptVpcPeeringConnection", "ec2:DeleteVpcPeeringConnection", "ec2:CreateVpcPeeringConnection", "ec2:RejectVpcPeeringConnection" ], "Resource": [ "arn:aws:ec2:*:XXXXXXXX:vpc-peering-connection/*", "arn:aws:ec2:*:XXXXXXXX:vpc/*" ] }, { "Effect": "Allow", "Action": [ "ec2:DeleteTags", "ec2:CreateTags" ], "Resource": "arn:aws:ec2:*:XXXXXXXX:vpc-peering-connection/*" } ] } ```
where `XXXXXXXX` is the requester AWS account ID. The `arn:aws:iam::YYYYYYYY:role/cross-account-vpc-peering-test` accepter IAM Role should have the following Trust Policy:
Show Trust Policy ```js { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::XXXXXXXX:root" }, "Action": "sts:AssumeRole", "Condition": {} } ] } ```
__NOTE__: The accepter Trust Policy is the same as the requester Trust Policy since it defines who can assume the IAM Role. In the requester case, the requester account ID itself is the trusted entity. For the accepter, the Trust Policy specifies that the requester account ID `XXXXXXXX` can assume the role in the accepter AWS account `YYYYYYYY`. and the following IAM Policy attached to it: __NOTE:__ the policy specifies the permissions to create (with `terraform plan/apply`) and delete (with `terraform destroy`) all the required resources in the accepter AWS account
Show IAM Policy ```js { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:CreateRoute", "ec2:DeleteRoute" ], "Resource": "arn:aws:ec2:*:YYYYYYYY:route-table/*" }, { "Effect": "Allow", "Action": [ "ec2:DescribeVpcPeeringConnections", "ec2:DescribeVpcs", "ec2:ModifyVpcPeeringConnectionOptions", "ec2:DescribeSubnets", "ec2:DescribeVpcAttribute", "ec2:DescribeRouteTables" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AcceptVpcPeeringConnection", "ec2:DeleteVpcPeeringConnection", "ec2:CreateVpcPeeringConnection", "ec2:RejectVpcPeeringConnection" ], "Resource": [ "arn:aws:ec2:*:YYYYYYYY:vpc-peering-connection/*", "arn:aws:ec2:*:YYYYYYYY:vpc/*" ] }, { "Effect": "Allow", "Action": [ "ec2:DeleteTags", "ec2:CreateTags" ], "Resource": "arn:aws:ec2:*:YYYYYYYY:vpc-peering-connection/*" } ] } ```
where `YYYYYYYY` is the accepter AWS account ID. For more information on IAM policies and permissions for VPC peering, see [Creating and managing VPC peering connections](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_IAM.html#vpcpeeringiam). ## Variables ### Required Variables
`accepter_region` (`string`) required
Accepter AWS region
`requester_aws_assume_role_arn` (`string`) required
Requester AWS Assume Role ARN
`requester_region` (`string`) required
Requester AWS region
### Optional Variables
`accepter_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow accepter VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the requester VPC **Default value:** `true`
`accepter_aws_access_key` (`string`) optional
Access key id to use in accepter account **Default value:** `null`
`accepter_aws_assume_role_arn` (`string`) optional
Accepter AWS Assume Role ARN **Default value:** `null`
`accepter_aws_profile` (`string`) optional
Profile used to assume accepter_aws_assume_role_arn **Default value:** `""`
`accepter_aws_secret_key` (`string`) optional
Secret access key to use in accepter account **Default value:** `null`
`accepter_aws_token` (`string`) optional
Session token for validating temporary credentials **Default value:** `null`
`accepter_enabled` (`bool`) optional
Flag to enable/disable the accepter side of the peering connection **Default value:** `true`
`accepter_subnet_tags` (`map(string)`) optional
Only add peer routes to accepter VPC route tables of subnets matching these tags **Default value:** `{ }`
`accepter_vpc_id` (`string`) optional
Accepter VPC ID filter **Default value:** `""`
`accepter_vpc_tags` (`map(string)`) optional
Accepter VPC Tags filter **Default value:** `{ }`
`add_attribute_tag` (`bool`) optional
If `true` will add additional attribute tag to the requester and accceptor resources **Default value:** `true`
`auto_accept` (`bool`) optional
Automatically accept the peering **Default value:** `true`
`aws_route_create_timeout` (`string`) optional
Time to wait for AWS route creation specifed as a Go Duration, e.g. `2m` **Default value:** `"5m"`
`aws_route_delete_timeout` (`string`) optional
Time to wait for AWS route deletion specifed as a Go Duration, e.g. `5m` **Default value:** `"5m"`
`requester_allow_remote_vpc_dns_resolution` (`bool`) optional
Allow requester VPC to resolve public DNS hostnames to private IP addresses when queried from instances in the accepter VPC **Default value:** `true`
`requester_aws_access_key` (`string`) optional
Access key id to use in requester account **Default value:** `null`
`requester_aws_profile` (`string`) optional
Profile used to assume requester_aws_assume_role_arn **Default value:** `""`
`requester_aws_secret_key` (`string`) optional
Secret access key to use in requester account **Default value:** `null`
`requester_aws_token` (`string`) optional
Session token for validating temporary credentials **Default value:** `null`
`requester_subnet_tags` (`map(string)`) optional
Only add peer routes to requester VPC route tables of subnets matching these tags **Default value:** `{ }`
`requester_vpc_id` (`string`) optional
Requester VPC ID filter **Default value:** `""`
`requester_vpc_tags` (`map(string)`) optional
Requester VPC Tags filter **Default value:** `{ }`
`skip_metadata_api_check` (`bool`) optional
Don't use the credentials of EC2 instance profile **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`accepter_accept_status`
Accepter VPC peering connection request status
`accepter_connection_id`
Accepter VPC peering connection ID
`accepter_subnet_route_table_map`
Map of accepter VPC subnet IDs to route table IDs
`requester_accept_status`
Requester VPC peering connection request status
`requester_connection_id`
Requester VPC peering connection ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3` - `aws`, version: `>= 2.0` ### Providers - `aws`, version: `>= 2.0` - `aws`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `accepter` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `requester` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_route.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.accepter_ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_route.requester_ipv6`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route) (resource) - [`aws_vpc_peering_connection.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_peering_connection) (resource) - [`aws_vpc_peering_connection_accepter.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_peering_connection_accepter) (resource) - [`aws_vpc_peering_connection_options.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_peering_connection_options) (resource) - [`aws_vpc_peering_connection_options.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpc_peering_connection_options) (resource) ## Data Sources The following data sources are used by this module: - [`aws_caller_identity.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_caller_identity.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) (data source) - [`aws_region.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_region.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) (data source) - [`aws_route_table.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_table) (data source) - [`aws_route_tables.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_tables) (data source) - [`aws_route_tables.default_rts`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route_tables) (data source) - [`aws_subnet.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) - [`aws_subnet.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) (data source) - [`aws_subnets.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) (data source) - [`aws_subnets.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnets) (data source) - [`aws_vpc.accepter`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) - [`aws_vpc.requester`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/vpc) (data source) --- ## vpn-connection # Module: `vpn-connection` Terraform module to provision a [site-to-site](https://docs.aws.amazon.com/vpn/latest/s2svpn/VPC_VPN.html) [VPN connection](https://docs.aws.amazon.com/vpc/latest/userguide/vpn-connections.html) between a VPC and an on-premises network. The module can do the following: - Create a Virtual Private Gateway (VPG) and attach it to the VPC - Create a Customer Gateway (CGW) pointing to the provided Internet-routable IP address of the external interface on the on-premises network - Create a Site-to-Site Virtual Private Network (VPN) connection - Request automatic route propagation between the VPG and the provided route tables in the VPC - If the VPN connection is configured to use static routes, provision a static route between the VPN connection and the CGW Exactly what it does depends on the input parameters. The module is designed to be flexible and can be used in a variety of scenarios. - If you supply `customer_gateway_ip_address` and set `transit_gateway_enabled` to `true`, the module will create a CGW, then create a VPN connection, and then assign the connection to the Transit Gateway identified by `existing_transit_gateway_id` and the created CGW - If you supply `customer_gateway_ip_address` and set `transit_gateway_enabled` to `false`, the module will create a VPG and CGW, then create a VPN connection, and then assign it to the VPG and CGW - If you do not supply `customer_gateway_ip_address` (set it to `null`) then the module will only create a VPG The module also provides some options for adding routes to the VPC or TGW route tables. You need to use the options that correspond to the kind of attachment point (VPC or TGW) you are using. ## Usage ```hcl module "vpn_connection" { source = "cloudposse/vpn-connection/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "dev" name = "test" vpc_id = "vpc-xxxxxxxx" vpn_gateway_amazon_side_asn = 64512 customer_gateway_bgp_asn = 65000 customer_gateway_ip_address = "172.0.0.1" route_table_ids = ["rtb-xxxxxxxx", "rtb-yyyyyyyy", "rtb-zzzzzzzz"] vpn_connection_static_routes_only = "true" vpn_connection_static_routes_destinations = ["10.80.1.0/24"] } ``` ## Variables ### Required Variables
### Optional Variables
`customer_gateway_bgp_asn` (`number`) optional
The Customer Gateway's Border Gateway Protocol (BGP) Autonomous System Number (ASN) **Default value:** `65000`
`customer_gateway_device_name` (`string`) optional
The Device Name of the Customer Gateway. Set to `null` to leave unnamed. WARNING: Changing this value will cause the Customer Gateway to be replaced. **Default value:** `""`
`customer_gateway_ip_address` (`string`) optional
The Internet-routable IP address of the Customer Gateway's external interface. WARNING: If set to `null`, the module will not create a Customer Gateway *or a VPN connection* and will only create a VPN Gateway. **Default value:** `null`
`existing_transit_gateway_id` (`string`) optional
Existing Transit Gateway ID. Required if `transit_gateway_enabled` is `true`, ignored otherwise. To set up a transit gateway, one can use the`cloudposse/transit-gateway/aws` module and pass the output `transit_gateway_id` to this variable. **Default value:** `""`
`existing_vpn_gateway_id` (`string`) optional
Existing VPN Gateway ID. If provided the module will use the vpn gateway provided here. **Default value:** `null`
`route_table_ids` (`list(string)`) optional
The IDs of the route tables for which routes from the Virtual Private Gateway will be propagated **Default value:** `[ ]`
`transit_gateway_enabled` (`bool`) optional
If `true`, the module will not create a Virtual Private Gateway but instead will attach the VPN connection to the Transit Gateway specified by `existing_transit_gateway_id` **Default value:** `false`
`transit_gateway_route_table_id` (`string`) optional
The ID of the route table for the transit gateway that you want to associate + propagate the VPN connection's TGW attachment **Default value:** `null`
`transit_gateway_routes` optional
A map of transit gateway routes to create on the given TGW route table (via `transit_gateway_route_table_id`) for the created VPN Attachment. Use the key in the map to describe the route **Type:** ```hcl map(object({ blackhole = optional(bool, false) destination_cidr_block = string })) ``` **Default value:** `{ }`
`vpc_id` (`string`) optional
The ID of the VPC to which the Virtual Private Gateway will be attached. Not needed if attaching the VPN connection to a Transit Gateway. **Default value:** `null`
`vpn_acceleration_enabled` (`bool`) optional
Set to true to enable VPN acceleration for the VPN connection. Can only be set at creation time, cannot be changed later. Accelerated VPN connections come with several restrictions: consult the AWS documentation for details. **Default value:** `false`
`vpn_connection_local_ipv4_network_cidr` (`string`) optional
The IPv4 CIDR on the Customer Gateway (on-premises) side of the VPN connection **Default value:** `"0.0.0.0/0"`
`vpn_connection_log_retention_in_days` (`number`) optional
Specifies the number of days you want to retain log events **Default value:** `30`
`vpn_connection_remote_ipv4_network_cidr` (`string`) optional
The IPv4 CIDR on the AWS side of the VPN connection **Default value:** `"0.0.0.0/0"`
`vpn_connection_static_routes_destinations` (`list(string)`) optional
List of CIDR blocks to be used as destination for static routes. Routes to destinations will be propagated to the route tables defined in `route_table_ids`. **Default value:** `[ ]`
`vpn_connection_static_routes_only` (`bool`) optional
If set to `true`, the VPN connection will use static routes exclusively. Static routes must be used for devices that don't support BGP. **Default value:** `false`
`vpn_connection_tunnel1_cloudwatch_log_enabled` (`bool`) optional
Enable or disable VPN tunnel logging feature for the tunnel **Default value:** `false`
`vpn_connection_tunnel1_cloudwatch_log_group_arn` (`list(string)`) optional
The ARN of the CloudWatch log group to which the logs will be published. If the list is empty and `vpn_connection_tunnel1_cloudwatch_log_enabled` is `true`, the module will create a new log group and use it. If the list is not empty, the module will use the first ARN in the list. **Default value:** `[ ]`
`vpn_connection_tunnel1_cloudwatch_log_output_format` (`string`) optional
Set log format for the tunnel. Default format is json. Possible values are `json` and `text` **Default value:** `"json"`
`vpn_connection_tunnel1_dpd_timeout_action` (`string`) optional
The action to take after DPD timeout occurs for the first VPN tunnel. Specify `restart` to restart the IKE initiation. Specify `clear` to end the IKE session. Valid values are `clear` | `none` | `restart` **Default value:** `"clear"`
`vpn_connection_tunnel1_ike_versions` (`list(string)`) optional
The IKE versions that are permitted for the first VPN tunnel. Valid values are `ikev1` | `ikev2` **Default value:** `[ ]`
`vpn_connection_tunnel1_inside_cidr` (`string`) optional
The CIDR block of the inside IP addresses for the first VPN tunnel **Default value:** `null`
`vpn_connection_tunnel1_phase1_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase1_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase1_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase1_lifetime_seconds` (`string`) optional
The lifetime for phase 1 of the IKE negotiation for the first VPN tunnel, in seconds. Valid value is between 900 and 28800 **Default value:** `"28800"`
`vpn_connection_tunnel1_phase2_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the first VPN tunnel for phase 2 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the first VPN tunnel for phase 2 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel1_phase2_lifetime_seconds` (`string`) optional
The lifetime for phase 2 of the IKE negotiation for the first VPN tunnel, in seconds. Valid value is between 900 and 3600 **Default value:** `"3600"`
`vpn_connection_tunnel1_preshared_key` (`string`) optional
The preshared key of the first VPN tunnel. The preshared key must be between 8 and 64 characters in length and cannot start with zero. Allowed characters are alphanumeric characters, periods(.) and underscores(_) **Default value:** `null`
`vpn_connection_tunnel1_startup_action` (`string`) optional
The action to take when the establishing the tunnel for the first VPN connection. By default, your customer gateway device must initiate the IKE negotiation and bring up the tunnel. Specify start for AWS to initiate the IKE negotiation. Valid values are `add` | `start` **Default value:** `"add"`
`vpn_connection_tunnel2_cloudwatch_log_enabled` (`bool`) optional
Enable or disable VPN tunnel logging feature for the tunnel **Default value:** `false`
`vpn_connection_tunnel2_cloudwatch_log_group_arn` (`list(string)`) optional
The ARN of the CloudWatch log group to which the logs will be published. If the list is empty and `vpn_connection_tunnel2_cloudwatch_log_enabled` is `true`, the module will create a new log group and use it. If the list is not empty, the module will use the first ARN in the list. **Default value:** `[ ]`
`vpn_connection_tunnel2_cloudwatch_log_output_format` (`string`) optional
Set log format for the tunnel. Default format is json. Possible values are `json` and `text` **Default value:** `"json"`
`vpn_connection_tunnel2_dpd_timeout_action` (`string`) optional
The action to take after DPD timeout occurs for the second VPN tunnel. Specify restart to restart the IKE initiation. Specify clear to end the IKE session. Valid values are `clear` | `none` | `restart` **Default value:** `"clear"`
`vpn_connection_tunnel2_ike_versions` (`list(string)`) optional
The IKE versions that are permitted for the second VPN tunnel. Valid values are `ikev1` | `ikev2` **Default value:** `[ ]`
`vpn_connection_tunnel2_inside_cidr` (`string`) optional
The CIDR block of the inside IP addresses for the second VPN tunnel **Default value:** `null`
`vpn_connection_tunnel2_phase1_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase1_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the second VPN tunnel for phase 1 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase1_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the second VPN tunnel for phase 1 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase1_lifetime_seconds` (`string`) optional
The lifetime for phase 1 of the IKE negotiation for the second VPN tunnel, in seconds. Valid value is between 900 and 28800 **Default value:** `"28800"`
`vpn_connection_tunnel2_phase2_dh_group_numbers` (`list(string)`) optional
List of one or more Diffie-Hellman group numbers that are permitted for the first VPN tunnel for phase 1 IKE negotiations. Valid values are 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_encryption_algorithms` (`list(string)`) optional
List of one or more encryption algorithms that are permitted for the second VPN tunnel for phase 2 IKE negotiations. Valid values are AES128 | AES256 | AES128-GCM-16 | AES256-GCM-16 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_integrity_algorithms` (`list(string)`) optional
One or more integrity algorithms that are permitted for the second VPN tunnel for phase 2 IKE negotiations. Valid values are SHA1 | SHA2-256 | SHA2-384 | SHA2-512 **Default value:** `[ ]`
`vpn_connection_tunnel2_phase2_lifetime_seconds` (`string`) optional
The lifetime for phase 2 of the IKE negotiation for the second VPN tunnel, in seconds. Valid value is between 900 and 3600 **Default value:** `"3600"`
`vpn_connection_tunnel2_preshared_key` (`string`) optional
The preshared key of the second VPN tunnel. The preshared key must be between 8 and 64 characters in length and cannot start with zero. Allowed characters are alphanumeric characters, periods(.) and underscores(_) **Default value:** `null`
`vpn_connection_tunnel2_startup_action` (`string`) optional
The action to take when the establishing the tunnel for the second VPN connection. By default, your customer gateway device must initiate the IKE negotiation and bring up the tunnel. Specify start for AWS to initiate the IKE negotiation. Valid values are `add` | `start` **Default value:** `"add"`
`vpn_gateway_amazon_side_asn` (`number`) optional
The Autonomous System Number (ASN) for the Amazon side of the VPN gateway. If you don't specify an ASN, the Virtual Private Gateway is created with the default ASN. **Default value:** `64512`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`customer_gateway_device_name`
Customer Gateway Device Name
`customer_gateway_id`
Customer Gateway ID
`transit_gateway_attachment_id`
The ID of the transit gateway attachment for the VPN connection (if a TGW connection)
`vpn_acceleration_enabled`
Whether the VPN connection is enabled for acceleration
`vpn_connection_customer_gateway_configuration`
The configuration information for the VPN connection's Customer Gateway (in the native XML format)
`vpn_connection_id`
VPN Connection ID
`vpn_connection_tunnel1_address`
The public IP address of the first VPN tunnel
`vpn_connection_tunnel1_cgw_inside_address`
The RFC 6890 link-local address of the first VPN tunnel (Customer Gateway side)
`vpn_connection_tunnel1_log_group_arn`
The CloudWatch Log Group ARN for the tunnel 1 logs
`vpn_connection_tunnel1_vgw_inside_address`
The RFC 6890 link-local address of the first VPN tunnel (Virtual Private Gateway side)
`vpn_connection_tunnel2_address`
The public IP address of the second VPN tunnel
`vpn_connection_tunnel2_cgw_inside_address`
The RFC 6890 link-local address of the second VPN tunnel (Customer Gateway side)
`vpn_connection_tunnel2_log_group_arn`
The CloudWatch Log Group ARN for the tunnel 2 logs
`vpn_connection_tunnel2_vgw_inside_address`
The RFC 6890 link-local address of the second VPN tunnel (Virtual Private Gateway side)
`vpn_gateway_id`
Virtual Private Gateway ID
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 5.53.0` ### Providers - `aws`, version: `>= 5.53.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `logs` | 0.6.9 | [`cloudposse/cloudwatch-logs/aws`](https://registry.terraform.io/modules/cloudposse/cloudwatch-logs/aws/0.6.9) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_customer_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/customer_gateway) (resource) - [`aws_ec2_tag.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_tag) (resource) - [`aws_ec2_transit_gateway_route.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route) (resource) - [`aws_ec2_transit_gateway_route_table_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_association) (resource) - [`aws_ec2_transit_gateway_route_table_propagation.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ec2_transit_gateway_route_table_propagation) (resource) - [`aws_vpn_connection.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpn_connection) (resource) - [`aws_vpn_connection_route.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpn_connection_route) (resource) - [`aws_vpn_gateway.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpn_gateway) (resource) - [`aws_vpn_gateway_route_propagation.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/vpn_gateway_route_propagation) (resource) ## Data Sources The following data sources are used by this module: --- ## waf(Waf) # Module: `waf` Terraform module to create and manage AWS WAFv2 rules. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-aws-waf/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-aws-waf/tree/main/test). ```hcl module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "waf" delimiter = "-" tags = { "BusinessUnit" = "XYZ", } } module "waf" { source = "cloudposse/waf/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" geo_match_statement_rules = [ { name = "rule-10" action = "count" priority = 10 statement = { country_codes = ["NL", "GB"] } visibility_config = { cloudwatch_metrics_enabled = true sampled_requests_enabled = false metric_name = "rule-10-metric" } }, { name = "rule-11" action = "allow" priority = 11 statement = { country_codes = ["US"] } visibility_config = { cloudwatch_metrics_enabled = true sampled_requests_enabled = false metric_name = "rule-11-metric" } } ] context = module.label.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-aws-waf/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
`visibility_config` required
Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl object({ cloudwatch_metrics_enabled = bool metric_name = string sampled_requests_enabled = bool }) ```
### Optional Variables
`association_resource_arns` (`list(string)`) optional
A list of ARNs of the resources to associate with the web ACL. This must be an ARN of an Application Load Balancer, Amazon API Gateway stage, or AWS AppSync. Do not use this variable to associate a Cloudfront Distribution. Instead, you should use the `web_acl_id` property on the `cloudfront_distribution` resource. For more details, refer to https://docs.aws.amazon.com/waf/latest/APIReference/API_AssociateWebACL.html **Default value:** `[ ]`
`byte_match_statement_rules` optional
A rule statement that defines a string match search for AWS WAF to apply to web requests. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: positional_constraint: Area within the portion of a web request that you want AWS WAF to search for search_string. Valid values include the following: EXACTLY, STARTS_WITH, ENDS_WITH, CONTAINS, CONTAINS_WORD. search_string String value that you want AWS WAF to search for. AWS WAF searches only in the part of web requests that you designate for inspection in field_to_match. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) custom_response = optional(object({ response_code = string custom_response_body_key = optional(string, null) response_header = optional(object({ name = string value = string }), null) }), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`custom_response_body` optional
Defines custom response bodies that can be referenced by custom_response actions. The map keys are used as the `key` attribute which is a unique key identifying the custom response body. content: Payload of the custom response. The response body can be plain text, HTML or JSON and cannot exceed 4KB in size. content_type: Content Type of Response Body. Valid values are `TEXT_PLAIN`, `TEXT_HTML`, or `APPLICATION_JSON`. **Type:** ```hcl map(object({ content = string content_type = string })) ``` **Default value:** `{ }`
`default_action` (`string`) optional
Specifies that AWS WAF should allow requests by default. Possible values: `allow`, `block`. **Default value:** `"block"`
`default_block_custom_response_body_key` (`string`) optional
References the default response body that you want AWS WAF to return to the web request client. This must reference a key defined in a custom_response_body block of this resource. Only takes effect if default_action is set to `block`. **Default value:** `null`
`default_block_response` (`string`) optional
A HTTP response code that is sent when default block action is used. Only takes effect if default_action is set to `block`. **Default value:** `null`
`description` (`string`) optional
A friendly description of the WebACL. **Default value:** `"Managed by Terraform"`
`geo_allowlist_statement_rules` optional
A rule statement used to identify a list of allowed countries which should not be blocked by the WAF. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: country_codes: A list of two-character country codes. forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`geo_match_statement_rules` optional
A rule statement used to identify web requests based on country of origin. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: country_codes: A list of two-character country codes. forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) custom_response = optional(object({ response_code = string custom_response_body_key = optional(string, null) response_header = optional(object({ name = string value = string }), null) }), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`ip_set_reference_statement_rules` optional
A rule statement used to detect web requests coming from particular IP addresses or address ranges. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The ARN of the IP Set that this statement references. ip_set: Defines a new IP Set description: A friendly description of the IP Set addresses: Contains an array of strings that specifies zero or more IP addresses or blocks of IP addresses. All addresses must be specified using Classless Inter-Domain Routing (CIDR) notation. ip_address_version: Specify `IPV4` or `IPV6` ip_set_forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. position: The position in the header to search for the IP address. Possible values include: `FIRST`, `LAST`, or `ANY`. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) custom_response = optional(object({ response_code = string custom_response_body_key = optional(string, null) response_header = optional(object({ name = string value = string }), null) }), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`log_destination_configs` (`list(string)`) optional
The Amazon Kinesis Data Firehose, CloudWatch Log log group, or S3 bucket Amazon Resource Names (ARNs) that you want to associate with the web ACL **Default value:** `[ ]`
`logging_filter` optional
A configuration block that specifies which web requests are kept in the logs and which are dropped. You can filter on the rule action and on the web request labels that were applied by matching rules during web ACL evaluation. **Type:** ```hcl object({ default_behavior = string filter = list(object({ behavior = string requirement = string condition = list(object({ action_condition = optional(object({ action = string }), null) label_name_condition = optional(object({ label_name = string }), null) })) })) }) ``` **Default value:** `null`
`managed_rule_group_statement_rules` optional
A rule statement used to run the rules that are defined in a managed rule group. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. override_action: The override action to apply to the rules in a rule group. Possible values: `count`, `none` captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: name: The name of the managed rule group. vendor_name: The name of the managed rule group vendor. version: The version of the managed rule group. You can set `Version_1.0` or `Version_1.1` etc. If you want to use the default version, do not set anything. rule_action_override: Action settings to use in the place of the rule actions that are configured inside the rule group. You specify one override for each rule whose action you want to change. managed_rule_group_configs: Additional information that's used by a managed rule group. Only one rule attribute is allowed in each config. Refer to https://docs.aws.amazon.com/waf/latest/developerguide/aws-managed-rule-groups-list.html for more details. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number override_action = optional(string) captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = object({ name = string vendor_name = string scope_down_not_statement_enabled = optional(bool, false) scope_down_statement = optional(object({ byte_match_statement = object({ positional_constraint = string search_string = string field_to_match = object({ all_query_arguments = optional(bool) body = optional(bool) method = optional(bool) query_string = optional(bool) single_header = optional(object({ name = string })) single_query_argument = optional(object({ name = string })) uri_path = optional(bool) }) text_transformation = list(object({ priority = number type = string })) }) }), null) version = optional(string) rule_action_override = optional(map(object({ action = string custom_request_handling = optional(object({ insert_header = object({ name = string value = string }) }), null) custom_response = optional(object({ response_code = string response_header = optional(object({ name = string value = string }), null) }), null) })), null) managed_rule_group_configs = optional(list(object({ aws_managed_rules_anti_ddos_rule_set = optional(object({ sensitivity_to_block = optional(string) client_side_action_config = optional(object({ challenge = object({ usage_of_action = string sensitivity = optional(string) exempt_uri_regular_expression = optional(list(object({ regex_string = string }))) }) })) })) aws_managed_rules_bot_control_rule_set = optional(object({ inspection_level = string enable_machine_learning = optional(bool, true) }), null) aws_managed_rules_atp_rule_set = optional(object({ enable_regex_in_path = optional(bool) login_path = string request_inspection = optional(object({ payload_type = string password_field = object({ identifier = string }) username_field = object({ identifier = string }) }), null) response_inspection = optional(object({ body_contains = optional(object({ success_strings = list(string) failure_strings = list(string) }), null) header = optional(object({ name = string success_values = list(string) failure_values = list(string) }), null) json = optional(object({ identifier = string success_strings = list(string) failure_strings = list(string) }), null) status_code = optional(object({ success_codes = list(string) failure_codes = list(string) }), null) }), null) }), null) aws_managed_rules_acfp_rule_set = optional(object({ creation_path = string enable_regex_in_path = optional(bool) registration_page_path = string request_inspection = optional(object({ payload_type = string password_field = optional(object({ identifier = string }), null) username_field = optional(object({ identifier = string }), null) email_field = optional(object({ identifier = string }), null) address_fields = optional(object({ identifiers = list(string) }), null) phone_number_fields = optional(object({ identifiers = list(string) }), null) }), null) response_inspection = optional(object({ body_contains = optional(object({ success_strings = list(string) failure_strings = list(string) }), null) header = optional(object({ name = string success_values = list(string) failure_values = list(string) }), null) json = optional(object({ identifier = string success_values = list(string) failure_values = list(string) }), null) status_code = optional(object({ success_codes = list(string) failure_codes = list(string) }), null) }), null) })) })), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`nested_statement_rules` optional
Rule statement to define nested statement rules to create nested complex rules including AND, OR, NOT statements. action: The actions that AWS WAF should take on nestes requests to conditionally match different and various rules. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. statement: and_statement: Additional creation of a conditional group with AND statement See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#and-statement or_statement: Additional creation of a conditional group with OR statement See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#or-statement not_statement: Additional creation of a conditional group with NOT statement See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#not-statement visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string custom_response = optional(object({ response_code = string custom_response_body_key = optional(string, null) response_header = optional(object({ name = string value = string }), null) }), null) statement = object({ and_statement = object({ statements = list(object({ type = string statement = string })) }) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`rate_based_statement_rules` optional
A rate-based rule tracks the rate of requests for each originating IP address, and triggers the rule action when the rate exceeds a limit that you specify on the number of requests in any 5-minute time span. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement custom_response: Defines a custom response for the web request. Only valid when action is set to 'block'. response_code: The HTTP status code to return to the client. For rate limiting, use "429" for "Too Many Requests". custom_response_body_key: References a key defined in custom_response_body to use as the response body. response_header: The name and value of a custom header to add to the response. statement: aggregate_key_type: Setting that indicates how to aggregate the request counts. Possible values include: `FORWARDED_IP` or `IP` limit: The limit on requests per 5-minute period for a single originating IP address. evaluation_window_sec: The amount of time, in seconds, that AWS WAF should include in its request counts, looking back from the current time. Valid values are 60, 120, 300, and 600. Defaults to 300 (5 minutes). forwarded_ip_config: fallback_behavior: The match status to assign to the web request if the request doesn't have a valid IP address in the specified position. Possible values: `MATCH`, `NO_MATCH` header_name: The name of the HTTP header to use for the IP address. byte_match_statement: field_to_match: Part of a web request that you want AWS WAF to inspect. positional_constraint: Area within the portion of a web request that you want AWS WAF to search for search_string. Valid values include the following: `EXACTLY`, `STARTS_WITH`, `ENDS_WITH`, `CONTAINS`, `CONTAINS_WORD`. search_string: String value that you want AWS WAF to search for. AWS WAF searches only in the part of web requests that you designate for inspection in `field_to_match`. The maximum length of the value is 50 bytes. text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) custom_response = optional(object({ response_code = string custom_response_body_key = optional(string, null) response_header = optional(object({ name = string value = string }), null) }), null) statement = object({ limit = number aggregate_key_type = string evaluation_window_sec = optional(number) forwarded_ip_config = optional(object({ fallback_behavior = string header_name = string }), null) scope_down_statement = optional(object({ byte_match_statement = object({ positional_constraint = string search_string = string field_to_match = object({ all_query_arguments = optional(bool) body = optional(bool) method = optional(bool) query_string = optional(bool) single_header = optional(object({ name = string })) single_query_argument = optional(object({ name = string })) uri_path = optional(bool) }) text_transformation = list(object({ priority = number type = string })) }) }), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`redacted_fields` optional
The parts of the request that you want to keep out of the logs. You can only specify one of the following: `method`, `query_string`, `single_header`, or `uri_path` method: Whether to enable redaction of the HTTP method. The method indicates the type of operation that the request is asking the origin to perform. uri_path: Whether to enable redaction of the URI path. This is the part of a web request that identifies a resource. query_string: Whether to enable redaction of the query string. This is the part of a URL that appears after a `?` character, if any. single_header: The list of names of the query headers to redact. **Type:** ```hcl map(object({ method = optional(bool, false) uri_path = optional(bool, false) query_string = optional(bool, false) single_header = optional(list(string), null) })) ``` **Default value:** `{ }`
`regex_match_statement_rules` optional
A rule statement used to search web request components for a match against a single regular expression. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: regex_string: String representing the regular expression. Minimum of 1 and maximum of 512 characters. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl.html#field_to_match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. At least one required. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`regex_pattern_set_reference_statement_rules` optional
A rule statement used to search web request components for matches with regular expressions. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The Amazon Resource Name (ARN) of the Regex Pattern Set that this statement references. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`rule_group_reference_statement_rules` optional
A rule statement used to run the rules that are defined in an WAFv2 Rule Group. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. override_action: The override action to apply to the rules in a rule group. Possible values: `count`, `none` captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: arn: The ARN of the `aws_wafv2_rule_group` resource. rule_action_override: Action settings to use in the place of the rule actions that are configured inside the rule group. You specify one override for each rule whose action you want to change. visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number override_action = optional(string) captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = object({ arn = string rule_action_override = optional(map(object({ action = string custom_request_handling = optional(object({ insert_header = object({ name = string value = string }) }), null) custom_response = optional(object({ response_code = string response_header = optional(object({ name = string value = string }), null) }), null) })), null) }) visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`scope` (`string`) optional
Specifies whether this is for an AWS CloudFront distribution or for a regional application. Possible values are `CLOUDFRONT` or `REGIONAL`. To work with CloudFront, you must also specify the region us-east-1 (N. Virginia) on the AWS provider. **Default value:** `"REGIONAL"`
`size_constraint_statement_rules` optional
A rule statement that uses a comparison operator to compare a number of bytes against the size of a request component. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: comparison_operator: The operator to use to compare the request part to the size setting. Possible values: `EQ`, `NE`, `LE`, `LT`, `GE`, or `GT`. size: The size, in bytes, to compare to the request part, after any transformations. Valid values are integers between `0` and `21474836480`, inclusive. field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`sqli_match_statement_rules` optional
An SQL injection match condition identifies the part of web requests, such as the URI or the query string, that you want AWS WAF to inspect. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. rule_label: A List of labels to apply to web requests that match the rule match statement captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. statement: field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
`token_domains` (`list(string)`) optional
Specifies the domains that AWS WAF should accept in a web request token. This enables the use of tokens across multiple protected websites. When AWS WAF provides a token, it uses the domain of the AWS resource that the web ACL is protecting. If you don't specify a list of token domains, AWS WAF accepts tokens only for the domain of the protected resource. With a token domain list, AWS WAF accepts the resource's host domain plus all domains in the token domain list, including their prefixed subdomains. **Default value:** `null`
`xss_match_statement_rules` optional
A rule statement that defines a cross-site scripting (XSS) match search for AWS WAF to apply to web requests. action: The action that AWS WAF should take on a web request when it matches the rule's statement. name: A friendly name of the rule. priority: If you define more than one Rule in a WebACL, AWS WAF evaluates each request against the rules in order based on the value of priority. AWS WAF processes rules with lower priority first. captcha_config: Specifies how AWS WAF should handle CAPTCHA evaluations. immunity_time_property: Defines custom immunity time. immunity_time: The amount of time, in seconds, that a CAPTCHA or challenge timestamp is considered valid by AWS WAF. The default setting is 300. rule_label: A List of labels to apply to web requests that match the rule match statement statement: field_to_match: The part of a web request that you want AWS WAF to inspect. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#field-to-match text_transformation: Text transformations eliminate some of the unusual formatting that attackers use in web requests in an effort to bypass detection. See https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl#text-transformation visibility_config: Defines and enables Amazon CloudWatch metrics and web request sample collection. cloudwatch_metrics_enabled: Whether the associated resource sends metrics to CloudWatch. metric_name: A friendly name of the CloudWatch metric. sampled_requests_enabled: Whether AWS WAF should store a sampling of the web requests that match the rules. **Type:** ```hcl list(object({ name = string priority = number action = string captcha_config = optional(object({ immunity_time_property = object({ immunity_time = number }) }), null) rule_label = optional(list(string), null) statement = any visibility_config = optional(object({ cloudwatch_metrics_enabled = optional(bool) metric_name = string sampled_requests_enabled = optional(bool) }), null) })) ``` **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`arn`
The ARN of the WAF WebACL.
`capacity`
The web ACL capacity units (WCUs) currently being used by this web ACL.
`id`
The ID of the WAF WebACL.
`logging_config_id`
The ARN of the WAFv2 Web ACL logging configuration.
## Dependencies ### Requirements - `terraform`, version: `>= 1.3.0` - `aws`, version: `>= 6.2.0` ### Providers - `aws`, version: `>= 6.2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `ip_set_label` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_wafv2_ip_set.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_ip_set) (resource) - [`aws_wafv2_web_acl.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl) (resource) - [`aws_wafv2_web_acl_association.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl_association) (resource) - [`aws_wafv2_web_acl_logging_configuration.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/wafv2_web_acl_logging_configuration) (resource) ## Data Sources The following data sources are used by this module: --- ## Cloudflare import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Access a library of Terraform modules designed to help you automate and manage Cloudflare services. These modules enable efficient configuration and management of DNS, security, and performance features. --- ## waf-rulesets # Module: `waf-rulesets` Terraform module to manage CloudFlare WAF rulesets. __NOTE:__ This module is a hard fork of [Innovation Norway's](https://github.com/innovationnorway/terraform-cloudflare-waf-rulesets) terraform module and adapted to Cloud Posse conventions. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-cloudflare-waf-rulesets/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-cloudflare-waf-rulesets/tree/main/test). ```hcl module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "waf" attributes = ["cf"] delimiter = "-" } module "waf_rulesets" { source = "cloudposse/waf-rulesets/cloudflare" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" zone = "cloudposse.co" rulesets = [ { name = "OWASP ModSecurity Core Rule Set" mode = "simulate" sensitivity = "off" rule_groups = [ { name = "OWASP Bad Robots" mode = "on" rules = [ { id = "990012" # Rogue web site crawler mode = "off" }, ] }, ] }, ] context = module.label.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-cloudflare-waf-rulesets/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
`rulesets` required
A list of `rulesets` objects. name: The name of the firewall package. sensitivity: The sensitivity of the firewall package. mode: The default action that will be taken for rules under the firewall package. Possible values: `simulate`, `block`, `challenge`. rule_groups: name: The name of the firewall rule group. mode: Whether or not the rules contained within this group are configurable/usable. Possible values: `on`, `off`. rules: id: The ID of the WAF rule. mode: The mode to use when the rule is triggered. Value is restricted based on the allowed_modes of the rule. Possible values: `default`, `disable`, `simulate`, `block`, `challenge`, `on`, `off`. **Type:** ```hcl list(object({ name = string sensitivity = string mode = string rule_groups = list(object({ name = string mode = string rules = list(object({ id = string mode = string })) })) })) ```
`zone` (`string`) required
The name of the DNS zone.
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`rulesets`
A list of `rulesets` objects.
## Dependencies ### Requirements - `terraform`, version: `>= 0.13` - `cloudflare`, version: `>= 2.19` ### Providers - `cloudflare`, version: `>= 2.19` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`cloudflare_waf_group.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/waf_group) (resource) - [`cloudflare_waf_package.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/waf_package) (resource) - [`cloudflare_waf_rule.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/waf_rule) (resource) ## Data Sources The following data sources are used by this module: - [`cloudflare_waf_groups.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/data-sources/waf_groups) (data source) - [`cloudflare_waf_packages.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/data-sources/waf_packages) (data source) - [`cloudflare_zones.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/data-sources/zones) (data source) --- ## zone # Module: `zone` Terraform module to provision a CloudFlare zone with: DNS records, Argo, Firewall filters and rules. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-cloudflare-zone/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-cloudflare-zone/tree/main/test). ```hcl module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "cf" delimiter = "-" } module "zone" { source = "cloudposse/zone/cloudflare" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" account_id = "example-account-id" zone = "cloudposse.co" records = [ { name = "bastion" value = "192.168.1.11" type = "A" ttl = 3600 }, { name = "api" value = "192.168.2.22" type = "A" ttl = 3600 } ] context = module.label.context } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-cloudflare-zone/tree/main/examples/complete) - complete example of using this module ## Variables ### Required Variables
`account_id` (`string`) required
Cloudflare account ID to manage the zone resource in
`zone` (`string`) required
The DNS zone name which will be added.
### Optional Variables
`argo_enabled` (`bool`) optional
Whether to enable Cloudflare Argo for DNS zone **Default value:** `false`
`argo_smart_routing_enabled` (`bool`) optional
Whether smart routing is enabled. **Default value:** `true`
`argo_tiered_caching_enabled` (`bool`) optional
Whether tiered caching is enabled. **Default value:** `true`
`healthchecks` (`list(any)`) optional
A list of maps of Health Checks rules. The values of map is fully compliant with `cloudflare_healthcheck` resource. To get more info see https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/healthcheck **Default value:** `null`
`jump_start` (`bool`) optional
Whether to scan for DNS records on creation. **Default value:** `false`
`page_rules` (`list(any)`) optional
A list of maps of Page Rules. The values of map is fully compliant with `cloudflare_page_rule` resource. To get more info see https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/cloudflare_page_rule **Default value:** `null`
`paused` (`bool`) optional
Whether this zone is paused (traffic bypasses Cloudflare) **Default value:** `false`
`plan` (`string`) optional
The name of the commercial plan to apply to the zone. Possible values: `free`, `pro`, `business`, `enterprise` **Default value:** `"free"`
`records` (`list(any)`) optional
name: The name of the record. type: The type of the record. value: The value of the record. comment: Optional comment for the record. ttl: The TTL of the record. Default value: 1. priority: The priority of the record. proxied: Whether the record gets Cloudflare's origin protection. Default value: false. **Default value:** `[ ]`
`type` (`string`) optional
A full zone implies that DNS is hosted with Cloudflare. A `partial` zone is typically a partner-hosted zone or a CNAME setup. Possible values: `full`, `partial`. **Default value:** `"full"`
`zone_enabled` (`bool`) optional
Whether to create DNS zone otherwise use existing. **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
The zone ID.
`meta_phishing_detected`
Indicates if URLs on the zone have been identified as hosting phishing content.
`meta_wildcard_proxiable`
Indicates whether wildcard DNS records can receive Cloudflare security and performance features.
`name_servers`
A list of Cloudflare-assigned name servers. This is only populated for zones that use Cloudflare DNS.
`page_rule_targets_to_ids`
A map of the page rule targets to IDs.
`plan`
The name of the commercial plan to apply to the zone.
`record_hostnames_to_ids`
A map of the zone record hostnames to IDs.
`status`
Status of the zone.
`vanity_name_servers`
A list of Vanity Nameservers.
`verification_key`
Contains the TXT record value to validate domain ownership. This is only populated for zones of type `partial`.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `cloudflare`, version: `>= 4.0` - `time`, version: `>= 0.8` ### Providers - `cloudflare`, version: `>= 4.0` - `time`, version: `>= 0.8` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`cloudflare_argo.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/argo) (resource) - [`cloudflare_healthcheck.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/healthcheck) (resource) - [`cloudflare_page_rule.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/page_rule) (resource) - [`cloudflare_record.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/record) (resource) - [`cloudflare_zone.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/resources/zone) (resource) - [`time_sleep.wait_for_records_creation`](https://registry.terraform.io/providers/hashicorp/time/latest/docs/resources/sleep) (resource) ## Data Sources The following data sources are used by this module: - [`cloudflare_zones.default`](https://registry.terraform.io/providers/cloudflare/cloudflare/latest/docs/data-sources/zones) (data source) --- ## Datadog import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Utilize our Terraform modules to integrate and manage Datadog monitoring within your infrastructure. These modules provide a structured approach to implementing observability and monitoring solutions. --- ## platform # Module: `platform` Terraform module to provision Datadog resources. The module consists of the following submodules: - [monitors](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/monitors) - to provision Datadog [monitors](https://docs.datadoghq.com/api/v1/monitors/) - [synthetics](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/synthetics) - to provision Datadog [synthetics](https://docs.datadoghq.com/synthetics/) - [permissions](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/permissions) - to look up all available Datadog [permissions](https://docs.datadoghq.com/account_management/rbac/permissions/) - [roles](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/roles) - to provision Datadog [roles](https://docs.datadoghq.com/account_management/rbac) - [slo](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/slo) - to provision Datadog [Service Level Objectives](https://docs.datadoghq.com/monitors/service_level_objectives/) - [child_organization](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/child_organization) - to provision Datadog [child organizations](https://docs.datadoghq.com/account_management/multi_organization/) - [organization_settings](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/organization_settings) - to manage Datadog organization's settings __Notes on Datadog child organizations:__ * Users can be added to the parent-organization and/or multiple child-organizations and switch between them from the user account settings menu * The parent-organization can view the usage of individual child-organizations, allowing them to track trends in usage * The Multi-organization account feature is not enabled by default. Contact Datadog support to have it enabled * Free and Trial organizations cannot enable SAML * We can only create Datadog child organizations with terraform, but cannot destroy them. When trying to destroy, the following error is thrown: ``` Warning: Cannot delete organization. Remove organization by contacting support (https://docs.datadoghq.com/help). ``` ## Sponsorship This project is supported by the [Datadog Open Source Program](https://www.datadoghq.com/partner/open-source/). As part of this collaboration, Datadog provides a dedicated sandbox account that we use for automated integration and acceptance testing. This contribution allows us to continuously validate changes against a real Datadog environment, improving reliability and reducing the risk of regressions. We are grateful to Datadog for supporting our open source ecosystem and helping ensure that infrastructure code for Terraform remains stable and well-tested ___ ## Introduction Datadog resources (monitors, roles, etc.) are defined as [catalog](https://github.com/cloudposse/terraform-datadog-platform/tree/main/catalog) of YAML configuration files. We maintain a comprehensive [catalog](https://github.com/cloudposse/terraform-datadog-platform/tree/main/catalog) of Datadog resources and welcome contributions via pull request! The [examples/complete](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/complete) in this module uses the catalog to provision the monitors on Datadog. The [examples/synthetics](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/synthetics) shows how to provision synthetic tests on Datadog for monitoring. Consult the [synthetics README](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/synthetics) module for more details. The [examples/rbac](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/rbac) shows how to use custom RBAC to provision Datadog roles with permissions and assign roles to monitors. The [examples/slo](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/slo) shows how to provision Service Level Objectives on Datadog for SLO monitoring. The [examples/child_organization](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/child_organization) shows how to provision Datadog child organizations. The [examples/organization_settings](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples/organization_settings) shows how to provision Datadog organization settings. ## Usage Provision Datadog monitors from the catalog of YAML definitions: ```hcl module "monitor_configs" { source = "cloudposse/config/yaml" version = "1.0.2" map_config_local_base_path = path.module map_config_paths = var.monitor_paths context = module.this.context } module "datadog_monitors" { source = "cloudposse/platform/datadog//modules/monitors" # version = "x.x.x" datadog_monitors = module.monitor_configs.map_configs alert_tags = var.alert_tags alert_tags_separator = var.alert_tags_separator context = module.this.context } ``` Provision Datadog synthetics: ```hcl locals { synthetics_files = flatten([for p in var.synthetic_paths : fileset(path.module, p)]) synthetics_list = [for f in local.synthetics_files : yamldecode(file(f))] synthetics_map = merge(local.synthetics_list...) } module "datadog_synthetics" { source = "cloudposse/platform/datadog//modules/synthetics" # version = "x.x.x" datadog_synthetics = local.synthetics_map alert_tags = var.alert_tags alert_tags_separator = var.alert_tags_separator context = module.this.context } ``` Provision Datadog monitors, Datadog roles with defined permissions, and assign roles to monitors: ```hcl module "monitor_configs" { source = "cloudposse/config/yaml" version = "1.0.2" map_config_local_base_path = path.module map_config_paths = var.monitor_paths context = module.this.context } module "role_configs" { source = "cloudposse/config/yaml" version = "1.0.2" map_config_local_base_path = path.module map_config_paths = var.role_paths context = module.this.context } locals { monitors_write_role_name = module.datadog_roles.datadog_roles["monitors-write"].name monitors_downtime_role_name = module.datadog_roles.datadog_roles["monitors-downtime"].name monitors_roles_map = { aurora-replica-lag = [local.monitors_write_role_name, local.monitors_downtime_role_name] ec2-failed-status-check = [local.monitors_write_role_name, local.monitors_downtime_role_name] redshift-health-status = [local.monitors_downtime_role_name] k8s-deployment-replica-pod-down = [local.monitors_write_role_name] } } module "datadog_roles" { source = "cloudposse/platform/datadog//modules/roles" # version = "x.x.x" datadog_roles = module.role_configs.map_configs context = module.this.context } module "datadog_monitors" { source = "cloudposse/platform/datadog//modules/monitors" # version = "x.x.x" datadog_monitors = module.monitor_configs.map_configs alert_tags = var.alert_tags alert_tags_separator = var.alert_tags_separator restricted_roles_map = local.monitors_roles_map context = module.this.context } ``` Provision a Datadog child organization: ```hcl module "datadog_child_organization" { source = "cloudposse/platform/datadog//modules/child_organization" # version = "x.x.x" organization_name = "test" saml_enabled = false # Note that Free and Trial organizations cannot enable SAML saml_autocreate_users_domains = [] saml_autocreate_users_enabled = false saml_idp_initiated_login_enabled = true saml_strict_mode_enabled = false private_widget_share = false saml_autocreate_access_role = "ro" context = module.this.context } ``` ## Examples Review the [examples](https://github.com/cloudposse/terraform-datadog-platform/tree/main/examples) folder to see how to use the Datadog modules. Also checkout our [terraform-aws-components](https://github.com/cloudposse/terraform-aws-components) repository for more examples of how to use a mixture of modules to enhance monitors, slos, and synthetics with inheritence and templating! ## Variables ### Required Variables
`datadog_monitors` required
Map of Datadog monitor configurations. See catalog for examples **Type:** ```hcl map(object({ name = string type = string message = string escalation_message = string query = string tags = list(string) notify_no_data = bool new_host_delay = number evaluation_delay = number no_data_timeframe = number renotify_interval = number notify_audit = bool timeout_h = number enable_logs_sample = bool include_tags = bool require_full_window = bool locked = bool force_delete = bool threshold_windows = map(any) thresholds = map(any) })) ```
### Optional Variables
`alert_tags` (`list(string)`) optional
List of alert tags to add to all alert messages, e.g. `["@opsgenie"]` or `["@devops", "@opsgenie"]` **Default value:** `null`
`alert_tags_separator` (`string`) optional
Separator for the alert tags. All strings from the `alert_tags` variable will be joined into one string using the separator and then added to the alert message **Default value:** `"\n"`
`restricted_roles_map` (`map(set(string))`) optional
Map of monitors names to sets of Datadog roles to restrict access to each monitor **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`datadog_monitor_ids`
IDs of the created Datadog monitors
`datadog_monitor_names`
Names of the created Datadog monitors
`datadog_monitors`
Datadog monitor outputs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0.0` - `datadog`, version: `>= 3.0.0` ### Providers - `datadog`, version: `>= 3.0.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`datadog_monitor.default`](https://registry.terraform.io/providers/datadog/datadog/latest/docs/resources/monitor) (resource) ## Data Sources The following data sources are used by this module: --- ## monitors This module creates Datadog [monitors](https://docs.datadoghq.com/api/latest/monitors/). It accepts all the configuration parameters supported by the Datadog Terraform resource. ## Usage The `datadog_monitors` input takes a map of test definitions. You must supply the keys for the map, but the values can follow either of two schemas described below. There are some optional additions to the monitor definition that are not part of the Datadog API schema. These are: - `force_delete` (boolean): If true, when deleting the monitor, the monitor will be deleted even if it is referenced by other resources. - `validate` (boolean): If false, the monitor will not be validated during the Terraform plan phase. - `enabled` (boolean): If false, the monitor will not be created. This is to allow you to suppress a monitor created through merging configuration snippets. ### Tags This module provides special handling for tags. The `tags` attribute of a monitor definition in the Datadog API and the Terraform resource is a list of strings. However, the Cloud Posse (as well as AWS) default is to use a map of strings for tags. This module allows you to provide either a list or a map for the tags. If you provide a list, it will be used as is. If you provide a map, it will be converted to a list of strings in the format `key:value` (or `key` if `value` is `null`). If you provide no tag settings at all, not even an empty list or map, the module will use the default tags from the [null-label](https://github.com/cloudposse/terraform-null-label/) module. ### API Schema (preferred) Datadog provides a REST API for managing monitors. We refer to the responses to the API requests for monitor definitions (`GET https://api.datadoghq.com/api/v1/monitor/{monitor_id}`) as the "API Schema" for the tests. This should correspond to the documented API "Model" for Monitors plus additional information such as creation date, but if the documentation and the API response differ, we use the API response as the source of truth. You can retrieve Monitor definitions from Datadog via the Datadog Web Console. Navigate to the monitor you want to retrieve, click the gear icon in the upper right (representing "settings"), and select "Export". This will display a JSON representation of the monitor definition. You can then click the "Copy" button to copy the JSON to the clipboard.
Example of JSON for monitors Note that many elements of the monitor definition are optional, and the JSON representation will only include the elements that are set. This is an example of a monitor, not a comprehensive list of all possible elements. ```json { "name": "schedule-test", "type": "event-v2 alert", "query": "events(\"service:datadog-agent\").rollup(\"cardinality\", \"@evt.id\").current(\"1h\") > 2345", "message": "No message", "tags": [ "test:examplemonitor", "Terratest" ], "options": { "thresholds": { "critical": 2345, "warning": 987 }, "enable_logs_sample": false, "notify_audit": false, "on_missing_data": "default", "include_tags": false, "scheduling_options": { "custom_schedule": { "recurrences": [ { "rrule": "FREQ=DAILY;INTERVAL=1;BYHOUR=17;BYMINUTE=54", "timezone": "America/Los_Angeles" } ] }, "evaluation_window": { "hour_starts": 7 } } }, "priority": 5 } ```
You can find other examples in the [examples/complete/monitors-test/](https://github.com/cloudposse/terraform-datadog-platform/tree/main/modules/monitors/../../examples/complete/monitors-test) directory. You can then use the `jsondecode()` function in Terraform to convert the JSON to a Terraform object, and use that object as the value for the monitor definition. You can also transform the JSON to HCL other ways, however you prefer. The relevant point is that this module will accept the monitor definition in this schema. Any field in the API schema that does not have a counterpart in the Terraform schema will be ignored. #### Special Notes The `alert_tags` input is provided for convenience. It is used to add notification tags to the monitor message. However, it does not check to see if the tags are already present. If the tags are already present, they will still be added again. :::important If you define a monitor via JSON, and then you use `alert_tags` when creating it, and then export the JSON representation of the created monitor definition, it will not match because of the added tags. ::: Note that `restricted_roles_map` provides a convenient way to specify the `restricted_roles` attribute of the monitor. This is a map of monitors to sets of Datadog unique role identifiers. If provided, this will override the `restricted_roles` attribute of the monitor definition. If not provided, the `restricted_roles` attribute of the monitor definition will be used, if present. :::important ### Legacy schema (deprecated) Historically, and preserved for backward compatibility, you can configure tests using the schema used in v1.3.0 and earlier. This schema flattens the monitor definition, pulling up the `options` attributes to the top level. Note that not all fields are supported in this schema, and it is only preserved for backward compatibility. We recommend that you use the API schema going forward. ::: --- ## slo # Datadog SLO This module is responsible for creating Datadog [Service Level Objectives](https://docs.datadoghq.com/monitors/service_level_objectives/) and their related monitors and alerts. The module can create metric-based SLOs (and the corresponding alerts) and monitor-based SLOs (and the corresponding monitors). ## Alerts Datadog alerts for SLOs are terraformed through the monitor object. An SLO can have many thresholds set, but a monitor can only have one. In order to get around this, the module creates Datadog monitors for each threshold within an SLO. ## Usage Example of metric-based SLO: ```yaml metric-slo: name: "(SLO) Synthetic Checks" type: metric query: numerator: sum:synthetics.test_runs{status:success}.as_count() denominator: sum:synthetics.test_runs{*}.as_count() description: | Number of Successful Synthetic Checks. message: | ({stage} {region}) {instance_id} failed a SLO check force_delete: true validate: true thresholds: - target: "99.5" timeframe: "7d" warning: "99.9" - target: "99" timeframe: "30d" warning: "99.5" tags: ManagedBy: terraform test: true api_version: null ``` Example of monitor-based SLO: ```yaml monitor-slo: name: "(SLO) EC2 Availability" type: monitor description: | Number of EC2 failed status checks. message: | ({stage} {region}) {instance_id} failed a SLO check force_delete: true validate: true thresholds: - target: "99.5" timeframe: "7d" warning: "99.9" - target: "99" timeframe: "30d" warning: "99.5" # Either `monitor_ids` or `monitors` should be provided # `monitor_ids` is a list of externally created monitors to use for this monitor-based SLO # If `monitors` map is provided, the monitors will be created by the module and assigned to the SLO monitor_ids: null monitors: ec2-failed-status-check: name: "(EC2) Status Check" type: metric alert query: | avg(last_10m):avg:aws.ec2.status_check_failed{*} by {instance_id} > 0 message: | ({stage} {region}) {instance_id} failed a status check escalation_message: "" tags: ManagedBy: Terraform priority: 3 notify_no_data: false notify_audit: true require_full_window: true enable_logs_sample: false force_delete: true include_tags: true locked: false renotify_interval: 60 timeout_h: 0 evaluation_delay: 60 new_host_delay: 300 new_group_delay: 0 groupby_simple_monitor: false renotify_occurrences: 0 renotify_statuses: [] validate: true no_data_timeframe: 10 threshold_windows: {} thresholds: critical: 0 tags: ManagedBy: terraform test: true api_version: null ``` ## References - [Service Level Objectives](https://docs.datadoghq.com/monitors/service_level_objectives/) - [Monitor-based SLOs](https://docs.datadoghq.com/monitors/service_level_objectives/monitor/) - [Datadog Error Budget](https://docs.datadoghq.com/monitors/service_level_objectives/error_budget/) - [Monitor-based SLO example](https://github.com/DataDog/terraform-provider-datadog/issues/667) --- ## synthetics This module creates Datadog [synthetic tests](https://docs.datadoghq.com/api/latest/synthetics/). It accepts all the configuration parameters supported by the Datadog Terraform resource except `BasicAuth` authentication wherever it occurs. ## Usage The `datadog_synthetics` input takes a map of test definitions. You must supply the keys for the map, but the values can follow either of two schemas described below. ### API Schema Datadog provides a REST API for managing synthetic tests. We refer to the responses to the API requests for test definitions as the "API Schema" for the tests. (Note that some items in the response are read-only, and are ignored if included in the definition of a test.) There are errors and omissions in the documentation of the API output, and where we have found API results that differ from the documentation, we have used the API results as the source of truth. You can retrieve test definitions from the Datadog API in 3 ways: 1. You can [retrieve an individual API test](https://docs.datadoghq.com/api/latest/synthetics/#get-an-api-test) 2. You can [retrieve an individual browser test](https://docs.datadoghq.com/api/latest/synthetics/#get-a-browser-test) 3. You can [retrieve a list of all tests](https://docs.datadoghq.com/api/latest/synthetics/#get-the-list-of-all-synthetic-tests) NOTE: As of this writing (2023-10-20), the list of all tests fails to include the steps in a multistep browser test. You must use the individual browser test API to retrieve the test definition including the steps. This is a known issue with Datadog, and hopefully will be fixed soon, but verify that it is fixed before relying on the list of all tests if you are using multistep browser tests. The `datadog_synthetics` input takes a map of test definitions. You must supply the keys for the map, but the values can simply be the output of applying Terraform's `jsondecode()` to the output of the API. In the case of the list of all tests, you must iterate over the list and supply a key for each test. We recommend that you use the test's `name` as the key. For any test, you can optionally add `enabled = false` to disable/delete the test. NOTE: Since this module is implemented in Terraform, any field in the API schema that does not have a counterpart in the Terraform schema will be ignored. As of this writing (Datadog Terraform provider version 3.30.0), that includes the `metatdata` field in the steps of a multistep API test. See https://github.com/DataDog/terraform-provider-datadog/issues/2155 ### Terraform schema Historically, and preserved for backward compatibility, you can configure tests using the `datadog_synthetics` input, which takes an object that is a map of synthetic tests, each test matching the schema of the `datadog_sythetics_test` [Terraform resource](https://registry.terraform.io/providers/DataDog/datadog/latest/docs/resources/synthetics_test). See examples with suffix `-terraform.yaml` in `examples/synthetics/terraform-schema/catalog`. One distinction of this schema is that there is no `config` member. Instead, elements of `config`, such as `assertions`, are pulled up to the top level and rendered in the singular, e.g. `assertion`. Another change is that `options` is renamed `options_list`. When in doubt, refer to the Terraform code to see how the input is mapped to the Terraform resource. Note that the Terraform resource does not support the details of the `element` field for the steps of a multistep browser test. This module allows you to use an object following the API schema, or you can use a string that is the JSON encoding of the object. However, if you use the JSON string encoding, and you are using a "user locator", you must supply the `element_user_locator` attribute even though it is already included in the JSON encoding, or else the Terraform provider will show perpetual drift. As with the API schema, you can optionally add `enabled = false` to disable/delete the test. #### Unsupported inputs Any of the "BasicAuth" inputs are not supported due to their complexity and rare usage. Usually you can use the `headers` input instead. BasicAuth support could be added if there is a need. ### Locations The `locations` input takes a list of locations used to run the test. It defaults to the special value "all" which is used to indicate all _public_ locations. Any locations included in the `locations` list will be _added_ to the list of locations specified in the individual test; they will not replace the list of locations specified in the test. --- ## Example import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Explore example Terraform modules that demonstrate best practices and common patterns. These examples serve as a foundation for building your own Terraform configurations. --- ## module(Module) # Module: `module` Short description ## Introduction This is an introduction. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-example-module/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-example-module/tree/main/test). ```hcl # Create a standard label resource. See [null-label](https://github.com/cloudposse/terraform-null-label/#terraform-null-label--) module "label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version, though usually you want to use the current one # version = "x.x.x" namespace = "eg" name = "example" } module "example" { source = "cloudposse/*****/aws" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" example = "Hello world!" context = module.label.this } ``` ## Quick Start Here's how to get started... ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-example-module/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`example` (`string`) optional
Example variable **Default value:** `"hello world"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`example`
Example output
`id`
ID of the created example
`random`
Stable random number for this example
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `random`, version: `>= 2.2` ### Providers - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`random_integer.example`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/integer) (resource) ## Data Sources The following data sources are used by this module: --- ## External import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Browse our Terraform modules that facilitate interactions with external systems and APIs. These modules enable seamless integration and management of third-party services alongside your infrastructure. --- ## module-artifact # Module: `module-artifact` Terraform Module to fetch remote build artifacts via http(s) using `curl`. ## Introduction This terraform module uses the [`external`](https://www.terraform.io/docs/providers/external/data_source.html) data provider to execute `curl`, which downloads the file locally to the modules cache folder. It's ideally suited for downloading public build artifacts (e.g. for use with Lambdas) associated with terraform modules. We recommend using this pattern to avoid adding binary artifacts like `.zip` files to Git repositories. **NOTE:** The term `external` refers to the primary Terraform [provider](https://www.terraform.io/docs/providers/external/data_source.html) used in this module. ## Dependencies First make sure you have `curl` installed and that it exists in your `PATH` (E.g. `/usr/bin`). ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-external-module-artifact/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on AWS), see [test](https://github.com/cloudposse/terraform-external-module-artifact/tree/main/test). ```hcl module "external_module_artifact" { source = "cloudposse/module-artifact/external" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" filename = "test.zip" git_ref = "example" module_name = "terraform-external-module-artifact" module_path = path.module } ``` __NOTE:__ Leave `git_ref` blank and it will be automatically computed from the `module_path` __NOTE:__ We've published [an artifact](https://artifacts.cloudposse.com/terraform-external-module-artifact/example/test.zip) for purpose of testing this module ## Variables ### Required Variables
`module_name` (`string`) required
Name of the terraform module
`module_path` (`string`) required
Local path to the terraform module; e.g. `${path.module}`
### Optional Variables
`curl_arguments` (`list(string)`) optional
Arguments that should get passed to `curl` **Default value:** ```hcl [ "-fsSL" ] ```
`filename` (`string`) optional
Artifact filename **Default value:** `"lambda.zip"`
`git_ref` (`string`) optional
Git hash corresponding to the remote artifact. Leave blank and it will be computed from the `module_path` checkout **Default value:** `""`
`url` (`string`) optional
URL template for the remote artifact **Default value:** `"https://artifacts.cloudposse.com/$${module_name}/$${git_ref}/$${filename}"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`base64sha256`
Base64 encoded SHA256 hash of the local file
`file`
Full path to the locally downloaded file
`git_ref`
Git commit hash corresponding to the artifact
`url`
URL corresponding to the artifact
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `aws`, version: `>= 2.0` - `external`, version: `>= 1.2` - `local`, version: `>= 1.3` - `template`, version: `>= 2.0` ### Providers - `external`, version: `>= 1.2` - `template`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`external_external.curl`](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external) (data source) - [`external_external.git`](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external) (data source) - [`template_file.url`](https://registry.terraform.io/providers/cloudposse/template/latest/docs/data-sources/file) (data source) --- ## GitHub import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Leverage our Terraform modules to automate and manage GitHub repositories, teams, and workflows. These modules help ensure consistent and automated GitHub operations. --- ## repository # Module: `repository` Terraform Module to provision a GitHub Repository with advanced settings. ## Introduction Terraform Module to provision a GitHub Repository with advanced settings: * Repository * Environments * Deploy Keys * Rulesets * Secrets * Variables * Custom Properties * Webhooks * Autolink References * Labels * Collaborators ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-github-repository/tree/main/examples/complete). ```hcl module "github_repository" { source = "cloudposse/repository/github" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "my-repository" description = "My repository" visibility = "public" auto_init = true has_issues = true has_projects = true has_wiki = true has_downloads = true is_template = false } ``` ## Examples Here is an example of using this module: - [`examples/complete`](https://github.com/cloudposse/terraform-example-module/) - complete example of using this module ## Variables ### Required Variables
### Optional Variables
`allow_auto_merge` (`bool`) optional
Allow auto merge **Default value:** `false`
`allow_merge_commit` (`bool`) optional
Allow merge commit **Default value:** `true`
`allow_rebase_merge` (`bool`) optional
Allow rebase merge **Default value:** `true`
`allow_squash_merge` (`bool`) optional
Allow squash merge **Default value:** `true`
`allow_update_branch` (`bool`) optional
Allow updating the branch **Default value:** `false`
`archive_on_destroy` (`bool`) optional
Archive the repository on destroy **Default value:** `false`
`archived` (`bool`) optional
Whether the repository is archived **Default value:** `false`
`auto_init` (`bool`) optional
Auto-initialize the repository **Default value:** `false`
`autolink_references` optional
Autolink references **Type:** ```hcl map(object({ key_prefix = string target_url_template = string is_alphanumeric = optional(bool, false) })) ``` **Default value:** `{ }`
`custom_properties` optional
Custom properties for the repository **Type:** ```hcl map(object({ string = optional(string, null) boolean = optional(bool, null) single_select = optional(string, null) multi_select = optional(list(string), null) })) ``` **Default value:** `{ }`
`default_branch` (`string`) optional
Default branch name **Default value:** `"main"`
`delete_branch_on_merge` (`bool`) optional
Delete branch on merge **Default value:** `false`
`deploy_keys` optional
Deploy keys for the repository **Type:** ```hcl map(object({ title = string key = string read_only = optional(bool, false) })) ``` **Default value:** `{ }`
`description` (`string`) optional
Description of the repository **Default value:** `null`
`enable_vulnerability_alerts` (`bool`) optional
Enable vulnerability alerts **Default value:** `true`
`environments` optional
Environments for the repository. Enviroment secrets should be encrypted using the GitHub public key in Base64 format if prefixed with nacl:. Read more: https://docs.github.com/en/actions/security-for-github-actions/encrypted-secrets **Type:** ```hcl map(object({ wait_timer = optional(number, 0) can_admins_bypass = optional(bool, false) prevent_self_review = optional(bool, false) reviewers = optional(object({ teams = optional(list(string), []) users = optional(list(string), []) }), null) deployment_branch_policy = optional(object({ protected_branches = optional(bool, false) custom_branches = optional(object({ branches = optional(list(string), null) tags = optional(list(string), null) }), null) }), null) variables = optional(map(string), null) secrets = optional(map(string), null) })) ``` **Default value:** `{ }`
`gitignore_template` (`string`) optional
Gitignore template **Default value:** `null`
`has_discussions` (`bool`) optional
Whether the repository has discussions enabled **Default value:** `false`
`has_downloads` (`bool`) optional
Whether the repository has downloads enabled **Default value:** `false`
`has_issues` (`bool`) optional
Whether the repository has issues enabled **Default value:** `false`
`has_projects` (`bool`) optional
Whether the repository has projects enabled **Default value:** `false`
`has_wiki` (`bool`) optional
Whether the repository has wiki enabled **Default value:** `false`
`homepage_url` (`string`) optional
Homepage URL of the repository **Default value:** `null`
`ignore_vulnerability_alerts_during_read` (`bool`) optional
Ignore vulnerability alerts during read **Default value:** `false`
`is_template` (`bool`) optional
Whether the repository is a template **Default value:** `false`
`labels` optional
A map of labels to configure for the repository **Type:** ```hcl map(object({ color = string description = string })) ``` **Default value:** `{ }`
`license_template` (`string`) optional
License template **Default value:** `null`
`merge_commit_message` (`string`) optional
Merge commit message. Must be PR_BODY, PR_TITLE or BLANK. **Default value:** `"PR_BODY"`
`merge_commit_title` (`string`) optional
Merge commit title. Must be PR_TITLE or MERGE_MESSAGE. **Default value:** `"PR_TITLE"`
`rulesets` optional
A map of rulesets to configure for the repository **Type:** ```hcl map(object({ name = string // disabled, active enforcement = string // branch, tag target = string bypass_actors = optional(list(object({ // always, pull_request bypass_mode = string actor_id = optional(string, null) // RepositoryRole, Team, Integration, OrganizationAdmin actor_type = string })), []) conditions = object({ ref_name = object({ // Supports ~DEFAULT_BRANCH or ~ALL include = optional(list(string), []) exclude = optional(list(string), []) }) }) rules = object({ branch_name_pattern = optional(object({ // starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), commit_author_email_pattern = optional(object({ // starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), creation = optional(bool, false), deletion = optional(bool, false), non_fast_forward = optional(bool, false), required_pull_request_reviews = optional(object({ dismiss_stale_reviews = bool required_approving_review_count = number }), null), commit_message_pattern = optional(object({ // starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), committer_email_pattern = optional(object({ // starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), merge_queue = optional(object({ check_response_timeout_minutes = optional(number, 60) // ALLGREEN, HEADGREEN grouping_strategy = string max_entries_to_build = optional(number, 5) max_entries_to_merge = optional(number, 5) // MERGE, SQUASH, REBASE merge_method = optional(string, "MERGE") min_entries_to_merge = optional(number, 1) min_entries_to_merge_wait_minutes = optional(number, 5) }), null), pull_request = optional(object({ dismiss_stale_reviews_on_push = optional(bool, false) require_code_owner_review = optional(bool, false) require_last_push_approval = optional(bool, false) required_approving_review_count = optional(number, 0) required_review_thread_resolution = optional(bool, false) }), null), required_deployments = optional(object({ required_deployment_environments = optional(list(string), []) }), null), required_status_checks = optional(object({ required_check = list(object({ context = string integration_id = optional(number, null) })) strict_required_status_checks_policy = optional(bool, false) do_not_enforce_on_create = optional(bool, false) }), null), tag_name_pattern = optional(object({ // starts_with, ends_with, contains, regex operator = string pattern = string name = optional(string, null) negate = optional(bool, false) }), null), // Unsupported due to drift. // https://github.com/integrations/terraform-provider-github/pull/2701 # required_code_scanning = optional(object({ # required_code_scanning_tool = list(object({ # // none, errors, errors_and_warnings, all # alerts_threshold = string # // none, critical, high_or_higher, medium_or_higher, all # security_alerts_threshold = string # tool = string # })) # }), null), }), })) ``` **Default value:** `{ }`
`secrets` (`map(string)`) optional
Secrets for the repository (if prefixed with nacl: it should be encrypted value using the GitHub public key in Base64 format. Read more: https://docs.github.com/en/actions/security-for-github-actions/encrypted-secrets) **Default value:** `{ }`
`security_and_analysis` optional
Security and analysis settings **Type:** ```hcl object({ advanced_security = bool secret_scanning = bool secret_scanning_push_protection = bool }) ``` **Default value:** `null`
`squash_merge_commit_message` (`string`) optional
Squash merge commit message. Must be PR_BODY, COMMIT_MESSAGES or BLANK. **Default value:** `"COMMIT_MESSAGE"`
`squash_merge_commit_title` (`string`) optional
Squash merge commit title. Must be PR_TITLE or COMMIT_OR_PR_TITLE. **Default value:** `"PR_TITLE"`
`teams` (`map(string)`) optional
A map of teams and their permissions for the repository **Default value:** `{ }`
`template` optional
Template repository **Type:** ```hcl object({ owner = string name = string include_all_branches = optional(bool, false) }) ``` **Default value:** `null`
`topics` (`list(string)`) optional
List of repository topics **Default value:** `[ ]`
`users` (`map(string)`) optional
A map of users and their permissions for the repository **Default value:** `{ }`
`variables` (`map(string)`) optional
Environment variables for the repository **Default value:** `{ }`
`visibility` (`string`) optional
Visibility of the repository. Must be public, private, or internal. **Default value:** `"public"`
`web_commit_signoff_required` (`bool`) optional
Require signoff on web commits **Default value:** `false`
`webhooks` optional
A map of webhooks to configure for the repository **Type:** ```hcl map(object({ active = optional(bool, true) events = list(string) url = string content_type = optional(string, "json") insecure_ssl = optional(bool, false) secret = optional(string, null) })) ``` **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`collaborators_invitation_ids`
Collaborators invitation IDs
`full_name`
Full name of the created repository
`git_clone_url`
Git clone URL of the created repository
`html_url`
HTML URL of the created repository
`http_clone_url`
SSH clone URL of the created repository
`node_id`
Node ID of the created repository
`primary_language`
Primary language of the created repository
`repo_id`
Repository ID of the created repository
`rulesets_etags`
Rulesets etags
`rulesets_node_ids`
Rulesets node IDs
`rulesets_rules_ids`
Rulesets rules IDs
`ssh_clone_url`
SSH clone URL of the created repository
`svn_url`
SVN URL of the created repository
`webhooks_urls`
Webhooks URLs
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `github`, version: `>= 6.6.0` ### Providers - `github`, version: `>= 6.6.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`github_actions_environment_secret.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/actions_environment_secret) (resource) - [`github_actions_environment_variable.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/actions_environment_variable) (resource) - [`github_actions_secret.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/actions_secret) (resource) - [`github_actions_variable.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/actions_variable) (resource) - [`github_branch_default.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/branch_default) (resource) - [`github_issue_label.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/issue_label) (resource) - [`github_repository.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository) (resource) - [`github_repository_autolink_reference.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_autolink_reference) (resource) - [`github_repository_collaborators.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_collaborators) (resource) - [`github_repository_custom_property.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_custom_property) (resource) - [`github_repository_deploy_key.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_deploy_key) (resource) - [`github_repository_environment.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_environment) (resource) - [`github_repository_environment_deployment_policy.branch_pattern`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_environment_deployment_policy) (resource) - [`github_repository_environment_deployment_policy.tag_pattern`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_environment_deployment_policy) (resource) - [`github_repository_ruleset.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_ruleset) (resource) - [`github_repository_webhook.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_webhook) (resource) ## Data Sources The following data sources are used by this module: - [`github_team.environment_reviewers`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/team) (data source) - [`github_team.ruleset_rules_teams`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/team) (data source) - [`github_user.environment_reviewers`](https://registry.terraform.io/providers/integrations/github/latest/docs/data-sources/user) (data source) --- ## repository-webhooks # Module: `repository-webhooks` Terraform module to provision webhooks on a set of GitHub repositories. This is useful if you need to register a webhook en masse across dozens of repositories. ## Usage Create a GitHub Personal Access Token that has `admin:repo_hook` for full control of repository hooks; in otherwords, we need `write:repo_hook` to write repository hooks and `read:repo_hook` to read repository hooks. For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-github-repository-webhooks/tree/main/examples/complete). ```hcl module "github_webhooks" { source = "cloudposse/repository-webhooks/github" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" github_organization = "cloudposse" github_token = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" github_repositories = ["geodesic"] webhook_url = "https://atlantis.prod.company.com" webhook_content_type = "json" events = ["issues"] } ``` ## Variables ### Required Variables
`webhook_url` (`string`) required
Webhook URL
### Optional Variables
`active` (`bool`) optional
Indicate of the webhook should receive events **Default value:** `true`
`events` (`list(string)`) optional
A list of events which should trigger the webhook. **Default value:** ```hcl [ "issue_comment", "pull_request", "pull_request_review", "pull_request_review_comment" ] ```
`github_repositories` (`list(string)`) optional
List of repository names which should be associated with the webhook **Default value:** `[ ]`
`webhook_content_type` (`string`) optional
Webhook Content Type (e.g. `json`) **Default value:** `"json"`
`webhook_insecure_ssl` (`bool`) optional
Webhook Insecure SSL (e.g. trust self-signed certificates) **Default value:** `false`
`webhook_secret` (`string`) optional
Webhook secret **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`webhook_url`
Webhook URL
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `github`, version: `>= 4.2.0` - `local`, version: `>= 1.2` ### Providers - `github`, version: `>= 4.2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`github_repository_webhook.default`](https://registry.terraform.io/providers/integrations/github/latest/docs/resources/repository_webhook) (resource) ## Data Sources The following data sources are used by this module: --- ## Kubernetes import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Discover our Terraform modules for provisioning and managing Kubernetes clusters and resources. These modules streamline the deployment and scaling of Kubernetes environments. --- ## Terraform Modules import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a collection of reusable Terraform Modules. In this library you'll find real-world examples of how we've implemented reusable Terraform Modules. --- ## Module(3) import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Explore modular Terraform components that can be reused across different configurations. These modules offer flexibility and consistency in your infrastructure as code practices. --- ## module(Test) # Module: `module` This is an test project 33. :::important This repository is for experiementation ::: ## Variables ### Required Variables
### Optional Variables
`example` (`string`) optional
Example input **Default value:** `""`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`example`
Example output
`id`
ID of the created example
`random`
Stable random number for this example
`test`
Test output
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `aws`, version: `>= 3.0` - `random`, version: `>= 2.2` ### Providers - `aws`, version: `>= 3.0` - `random`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`aws_ssm_parameter.default`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ssm_parameter) (resource) - [`random_integer.example`](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/integer) (resource) ## Data Sources The following data sources are used by this module: --- ## label # Module: `label` Terraform module designed to generate consistent names and tags for resources. Use `terraform-null-label` to implement a strict naming convention. There are 6 inputs considered "labels" or "ID elements" (because the labels are used to construct the ID): 1. namespace 1. tenant 1. environment 1. stage 1. name 1. attributes This module generates IDs using the following convention by default: `{namespace}-{environment}-{stage}-{name}-{attributes}`. However, it is highly configurable. The delimiter (e.g. `-`) is configurable. Each label item is optional (although you must provide at least one). So if you prefer the term `stage` to `environment` and do not need `tenant`, you can exclude them and the label `id` will look like `{namespace}-{stage}-{name}-{attributes}`. - The `tenant` label was introduced in v0.25.0. To preserve backward compatibility, it is not included by default. - The `attributes` input is actually a list of strings and `{attributes}` expands to the list elements joined by the delimiter. - If `attributes` is excluded but `namespace`, `stage`, and `environment` are included, `id` will look like `{namespace}-{environment}-{stage}-{name}`. Excluding `attributes` is discouraged, though, because attributes are the main way modules modify the ID to ensure uniqueness when provisioning the same resource types. - If you want the label items in a different order, you can specify that, too, with the `label_order` list. - You can set a maximum length for the `id`, and the module will create a (probably) unique name that fits within that length. (The module uses a portion of the MD5 hash of the full `id` to represent the missing part, so there remains a slight chance of name collision.) - You can control the letter case of the generated labels which make up the `id` using `var.label_value_case`. - By default, all of the non-empty labels are also exported as tags, whether they appear in the `id` or not. You can control which labels are exported as tags by setting `labels_as_tags` to the list of labels you want exported, or the empty list `[]` if you want no labels exported as tags at all. Tags passed in via the `tags` variable are always exported, and regardless of settings, empty labels are never exported as tags. You can control the case of the tag names (keys) for the labels using `var.label_key_case`. Unlike the tags generated from the label inputs, tags passed in via the `tags` input are not modified. There is an unfortunate collision over the use of the key `name`. Cloud Posse uses `name` in this module to represent the component, such as `eks` or `rds`. AWS uses a tag with the key `Name` to store the full human-friendly identifier of the thing tagged, which this module outputs as `id`, not `name`. So when converting input labels to tags, the value of the `Name` key is set to the module `id` output, and there is no tag corresponding to the module `name` output. An empty `name` label will not prevent the `Name` tag from being exported. It's recommended to use one `terraform-null-label` module for every unique resource of a given resource type. For example, if you have 10 instances, there should be 10 different labels. However, if you have multiple different kinds of resources (e.g. instances, security groups, file systems, and elastic ips), then they can all share the same label assuming they are logically related. For most purposes, the `id` output is sufficient to create an ID or label for a resource, and if you want a different ID or a different format, you would instantiate another instance of `null-label` and configure it accordingly. However, to accomodate situations where you want all the same inputs to generate multiple descriptors, this module provides the `descriptors` output, which is a map of strings generated according to the format specified by the `descriptor_formats` input. This feature is intentionally simple and minimally configurable and will not be enhanced to add more features that are already in `null-label`. See [examples/complete/descriptors.tf](https://github.com/cloudposse/terraform-null-label/tree/main/examples/complete/descriptors.tf) for examples. All [Cloud Posse Terraform modules](https://github.com/cloudposse?utf8=%E2%9C%93&q=terraform-&type=&language=) use this module to ensure resources can be instantiated multiple times within an account and without conflict. The Cloud Posse convention is to use labels as follows: - `namespace`: A short (3-4 letters) abbreviation of the company name, to ensure globally unique IDs for things like S3 buckets - `tenant`: _(Rarely needed)_ When a company creates a dedicated resource per customer, `tenant` can be used to identify the customer the resource is dedicated to - `environment`: A [short abbreviation](https://github.com/cloudposse/terraform-aws-utils/#introduction) for the AWS region hosting the resource, or `gbl` for resources like IAM roles that have no region - `stage`: The name or role of the account the resource is for, such as `prod` or `dev` - `name`: The name of the component that owns the resources, such as `eks` or `rds` **NOTE:** The `null` originally referred to the primary Terraform [provider](https://www.terraform.io/docs/providers/null/index.html) used in this module. With Terraform 0.12, this module no longer needs any provider, but the name was kept for continuity. - Releases of this module from `0.23.0` onward only work with Terraform 0.13 or newer. - Releases of this module from `0.12.0` through `0.22.1` support `HCL2` and are compatible with Terraform 0.12 or newer. - Releases of this module prior to `0.12.0` are compatible with earlier versions of terraform like Terraform 0.11. ## Usage ### Defaults Cloud Posse Terraform modules share a common `context` object that is meant to be passed from module to module. The context object is a single object that contains all the input values for `terraform-null-label`. However, each input value can also be specified individually by name as a standard Terraform variable, and the value of those variables, when set to something other than `null`, will override the value in the context object. In order to allow chaining of these objects, where the context object input to one module is transformed and passed on to the next module, all the variables default to `null` or empty collections. The actual default values used when nothing is explicitly set are described in the documentation below. For example, the default value of `delimiter` is shown as `null`, but if you leave it set to `null`, `terraform-null-label` will actually use the default delimiter `-` (hyphen). A non-obvious but intentional consequence of this design is that once a module sets a non-default value, future modules in the chain cannot reset the value back to the original default. Instead, the new setting becomes the new default for downstream modules. Also, collections are not overwritten, they are merged, so once a tag is added, it will remain in the tag set and cannot be removed, although its value can be overwritten. Because the purpose of `labels_as_tags` is primarily to prevent tags from being generated that would [conflict with the AWS provider's `default_tags`](https://github.com/hashicorp/terraform-provider-aws/issues/19204), it is an exception to the rule that variables override the setting in the context object. The value in the context object cannot be changed, so that later modules cannot re-enable a problematic tag. ### Simple Example ```hcl module "eg_prod_bastion_label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" attributes = ["public"] delimiter = "-" tags = { "BusinessUnit" = "XYZ", "Snapshot" = "true" } } ``` This will create an `id` with the value of `eg-prod-bastion-public` because when generating `id`, the default order is `namespace`, `environment`, `stage`, `name`, `attributes` (you can override it by using the `label_order` variable, see [Advanced Example 3](#advanced-example-3)). Now reference the label when creating an instance: ```hcl resource "aws_instance" "eg_prod_bastion_public" { instance_type = "t1.micro" tags = module.eg_prod_bastion_label.tags } ``` Or define a security group: ```hcl resource "aws_security_group" "eg_prod_bastion_public" { vpc_id = var.vpc_id name = module.eg_prod_bastion_label.id tags = module.eg_prod_bastion_label.tags egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } } ``` ### Advanced Example Here is a more complex example with two instances using two different labels. Note how efficiently the tags are defined for both the instance and the security group.
Click to show ```hcl module "eg_prod_bastion_label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" delimiter = "-" tags = { "BusinessUnit" = "XYZ", "Snapshot" = "true" } } module "eg_prod_bastion_abc_label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = ["abc"] tags = { "BusinessUnit" = "ABC" # Override the Business Unit tag set in the base label } # Copy all other fields from the base label context = module.eg_prod_bastion_label.context } resource "aws_security_group" "eg_prod_bastion_abc" { name = module.eg_prod_bastion_abc_label.id tags = module.eg_prod_bastion_abc_label.tags ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } } resource "aws_instance" "eg_prod_bastion_abc" { instance_type = "t1.micro" tags = module.eg_prod_bastion_abc_label.tags vpc_security_group_ids = [aws_security_group.eg_prod_bastion_abc.id] } module "eg_prod_bastion_xyz_label" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" attributes = ["xyz"] context = module.eg_prod_bastion_label.context } resource "aws_security_group" "eg_prod_bastion_xyz" { name = module.eg_prod_bastion_xyz_label.id tags = module.eg_prod_bastion_xyz_label.tags ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } } resource "aws_instance" "eg_prod_bastion_xyz" { instance_type = "t1.micro" tags = module.eg_prod_bastion_xyz_label.tags vpc_security_group_ids = [aws_security_group.eg_prod_bastion_xyz.id] } ```
### Advanced Example 2 Here is a more complex example with an autoscaling group that has a different tagging schema than other resources and requires its tags to be in this format, which this module can generate via `additional_tag_map` and `tags_as_list_of_maps`:
Click to show ```hcl tags = [ { key = "Name", propagate_at_launch = true, value = "namespace-stage-name" }, { key = "Namespace", propagate_at_launch = true, value = "namespace" }, { key = "Stage", propagate_at_launch = true, value = "stage" } ] ``` Autoscaling group using propagating tagging below (full example: [autoscalinggroup](https://github.com/cloudposse/terraform-null-label/tree/main/examples/autoscalinggroup/main.tf)) ```hcl ################################ # terraform-null-label example # ################################ module "label" { source = "../../" namespace = "cp" stage = "prod" name = "app" tags = { BusinessUnit = "Finance" ManagedBy = "Terraform" } additional_tag_map = { propagate_at_launch = true } } ####################### # Launch template # ####################### resource "aws_launch_template" "default" { # terraform-null-label example used here: Set template name prefix name_prefix = "${module.label.id}-" image_id = data.aws_ami.amazon_linux.id instance_type = "t2.micro" instance_initiated_shutdown_behavior = "terminate" vpc_security_group_ids = [data.aws_security_group.default.id] monitoring { enabled = false } # terraform-null-label example used here: Set tags on volumes tag_specifications { resource_type = "volume" tags = module.label.tags } } ###################### # Autoscaling group # ###################### resource "aws_autoscaling_group" "default" { # terraform-null-label example used here: Set ASG name prefix name_prefix = "${module.label.id}-" vpc_zone_identifier = data.aws_subnet_ids.all.ids max_size = 1 min_size = 1 desired_capacity = 1 launch_template = { id = aws_launch_template.default.id version = "$$Latest" } # terraform-null-label example used here: Set tags on ASG and EC2 Servers tags = module.label.tags_as_list_of_maps } ```
### Advanced Example 3 See [complete example](https://github.com/cloudposse/terraform-null-label/tree/main/examples/complete) for even more examples. This example shows how you can pass the `context` output of one label module to the next label_module, allowing you to create one label that has the base set of values, and then creating every extra label as a derivative of that.
Click to show ```hcl module "label1" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "CloudPosse" tenant = "H.R.H" environment = "UAT" stage = "build" name = "Winston Churchroom" attributes = ["fire", "water", "earth", "air"] label_order = ["name", "tenant", "environment", "stage", "attributes"] tags = { "City" = "Dublin" "Environment" = "Private" } } module "label2" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "Charlie" tenant = "" # setting to `null` would have no effect stage = "test" delimiter = "+" regex_replace_chars = "/[^a-zA-Z0-9-+]/" additional_tag_map = { propagate_at_launch = true additional_tag = "yes" } tags = { "City" = "London" "Environment" = "Public" } context = module.label1.context } module "label3" { source = "cloudposse/label/null" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" name = "Starfish" stage = "release" delimiter = "." regex_replace_chars = "/[^-a-zA-Z0-9.]/" tags = { "Eat" = "Carrot" "Animal" = "Rabbit" } context = module.label1.context } ``` This creates label outputs like this: ```hcl label1 = { "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "-" "id" = "winstonchurchroom-hrh-uat-build-fire-water-earth-air" "name" = "winstonchurchroom" "namespace" = "cloudposse" "stage" = "build" "tenant" = "hrh" } label1_context = { "additional_tag_map" = {} "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = tostring(null) "enabled" = true "environment" = "UAT" "id_length_limit" = tonumber(null) "label_key_case" = tostring(null) "label_order" = tolist([ "name", "tenant", "environment", "stage", "attributes", ]) "label_value_case" = tostring(null) "name" = "Winston Churchroom" "namespace" = "CloudPosse" "regex_replace_chars" = tostring(null) "stage" = "build" "tags" = { "City" = "Dublin" "Environment" = "Private" } "tenant" = "H.R.H" } label1_normalized_context = { "additional_tag_map" = {} "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "-" "enabled" = true "environment" = "uat" "id_length_limit" = 0 "label_key_case" = "title" "label_order" = tolist([ "name", "tenant", "environment", "stage", "attributes", ]) "label_value_case" = "lower" "name" = "winstonchurchroom" "namespace" = "cloudposse" "regex_replace_chars" = "/[^-a-zA-Z0-9]/" "stage" = "build" "tags" = { "Attributes" = "fire-water-earth-air" "City" = "Dublin" "Environment" = "Private" "Name" = "winstonchurchroom-hrh-uat-build-fire-water-earth-air" "Namespace" = "cloudposse" "Stage" = "build" "Tenant" = "hrh" } "tenant" = "hrh" } label1_tags = tomap({ "Attributes" = "fire-water-earth-air" "City" = "Dublin" "Environment" = "Private" "Name" = "winstonchurchroom-hrh-uat-build-fire-water-earth-air" "Namespace" = "cloudposse" "Stage" = "build" "Tenant" = "hrh" }) label2 = { "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "+" "id" = "charlie+uat+test+fire+water+earth+air" "name" = "charlie" "namespace" = "cloudposse" "stage" = "test" "tenant" = "" } label2_context = { "additional_tag_map" = { "additional_tag" = "yes" "propagate_at_launch" = "true" } "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "+" "enabled" = true "environment" = "UAT" "id_length_limit" = tonumber(null) "label_key_case" = tostring(null) "label_order" = tolist([ "name", "tenant", "environment", "stage", "attributes", ]) "label_value_case" = tostring(null) "name" = "Charlie" "namespace" = "CloudPosse" "regex_replace_chars" = "/[^a-zA-Z0-9-+]/" "stage" = "test" "tags" = { "City" = "London" "Environment" = "Public" } "tenant" = "" } label2_tags = tomap({ "Attributes" = "fire+water+earth+air" "City" = "London" "Environment" = "Public" "Name" = "charlie+uat+test+fire+water+earth+air" "Namespace" = "cloudposse" "Stage" = "test" }) label2_tags_as_list_of_maps = [ { "additional_tag" = "yes" "key" = "Attributes" "propagate_at_launch" = "true" "value" = "fire+water+earth+air" }, { "additional_tag" = "yes" "key" = "City" "propagate_at_launch" = "true" "value" = "London" }, { "additional_tag" = "yes" "key" = "Environment" "propagate_at_launch" = "true" "value" = "Public" }, { "additional_tag" = "yes" "key" = "Name" "propagate_at_launch" = "true" "value" = "charlie+uat+test+fire+water+earth+air" }, { "additional_tag" = "yes" "key" = "Namespace" "propagate_at_launch" = "true" "value" = "cloudposse" }, { "additional_tag" = "yes" "key" = "Stage" "propagate_at_launch" = "true" "value" = "test" }, ] label3 = { "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "." "id" = "starfish.h.r.h.uat.release.fire.water.earth.air" "name" = "starfish" "namespace" = "cloudposse" "stage" = "release" "tenant" = "h.r.h" } label3_context = { "additional_tag_map" = {} "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "." "enabled" = true "environment" = "UAT" "id_length_limit" = tonumber(null) "label_key_case" = tostring(null) "label_order" = tolist([ "name", "tenant", "environment", "stage", "attributes", ]) "label_value_case" = tostring(null) "name" = "Starfish" "namespace" = "CloudPosse" "regex_replace_chars" = "/[^-a-zA-Z0-9.]/" "stage" = "release" "tags" = { "Animal" = "Rabbit" "City" = "Dublin" "Eat" = "Carrot" "Environment" = "Private" } "tenant" = "H.R.H" } label3_normalized_context = { "additional_tag_map" = {} "attributes" = tolist([ "fire", "water", "earth", "air", ]) "delimiter" = "." "enabled" = true "environment" = "uat" "id_length_limit" = 0 "label_key_case" = "title" "label_order" = tolist([ "name", "tenant", "environment", "stage", "attributes", ]) "label_value_case" = "lower" "name" = "starfish" "namespace" = "cloudposse" "regex_replace_chars" = "/[^-a-zA-Z0-9.]/" "stage" = "release" "tags" = { "Animal" = "Rabbit" "Attributes" = "fire.water.earth.air" "City" = "Dublin" "Eat" = "Carrot" "Environment" = "Private" "Name" = "starfish.h.r.h.uat.release.fire.water.earth.air" "Namespace" = "cloudposse" "Stage" = "release" "Tenant" = "h.r.h" } "tenant" = "h.r.h" } label3_tags = tomap({ "Animal" = "Rabbit" "Attributes" = "fire.water.earth.air" "City" = "Dublin" "Eat" = "Carrot" "Environment" = "Private" "Name" = "starfish.h.r.h.uat.release.fire.water.earth.air" "Namespace" = "cloudposse" "Stage" = "release" "Tenant" = "h.r.h" }) ```
## Variables ### Required Variables
### Optional Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`additional_tag_map`
The merged additional_tag_map
`attributes`
List of attributes
`context`
Merged but otherwise unmodified input to this module, to be used as context input to other modules. Note: this version will have null values as defaults, not the values actually used as defaults.
`delimiter`
Delimiter between `namespace`, `tenant`, `environment`, `stage`, `name` and `attributes`
`descriptors`
Map of descriptors as configured by `descriptor_formats`
`enabled`
True if module is enabled, false otherwise
`environment`
Normalized environment
`id`
Disambiguated ID string restricted to `id_length_limit` characters in total
`id_full`
ID string not restricted in length
`id_length_limit`
The id_length_limit actually used to create the ID, with `0` meaning unlimited
`label_order`
The naming order actually used to create the ID
`name`
Normalized name
`namespace`
Normalized namespace
`normalized_context`
Normalized context of this module
`regex_replace_chars`
The regex_replace_chars actually used to create the ID
`stage`
Normalized stage
`tags`
Normalized Tag map
`tags_as_list_of_maps`
This is a list with one map for each `tag`. Each map contains the tag `key`, `value`, and contents of `var.additional_tag_map`. Used in the rare cases where resources need additional configuration information for each tag.
`tenant`
Normalized tenant
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` --- ## Null import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Access utility modules that leverage the null provider in Terraform for various operations, including triggers, conditionals, and more. These modules provide additional functionality within your Terraform workflows. --- ## incident-management # Module: `incident-management` Terraform module to provision Opsgenie resources using the Opsgenie provider. The provider needs to be configured with the proper credentials before it can be used. It consist of root module which is only here as an example but can be used as a combination of all submodules. Submodules can also be combined to abstract away complexity of setting up for example a team escalation. ## Introduction Available modules: - [Alert Policy](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/alert_policy) - [API Integration](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/api_integration) - [Config](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/config) - [Escalation](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/escalation) - [Integration Action](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/integration_action) (advanced feature — not available to all OpsGenie plans) - [Notification Policy](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/notification_policy) - [Team](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/team) - [Team Routing Rule](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/team_routing_rule) - [User](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/user) - [Service](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/service) - [Service Incident Rule](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/service_incident_rule) **Note:** Root module is just an example that uses all of submodules. **Note:** See the [Advanced Features Example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/advanced_features) for features only available to some OpsGenie plans. ## Usage Here's how to invoke `team` module in your projects ```hcl module "team-name" { source = "cloudposse/incident-management/opsgenie//modules/team" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" team = { name = "team-name" description = "team-description" } } ``` ## Examples Here are examples of using the module: - [`complete`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/complete) - complete example of using this module Submodules examples: - [`alert_policy`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/alert_policy) - [`api_integration`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/api_integration) - [`escalation`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/escalation) - [`integration_action`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/integration_action) (advanced feature — not available to all OpsGenie plans) - [`notification_policy`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/notification_policy) - [`team`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/team) - [`team_routing_rule`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/team_routing_rule) - [`user`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/user) Here is an example of using the `config` module, which incorporates all resource declarations into a single module: - [`config`](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/examples/config) Here are automated tests for the examples using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and provisions the examples): - [test](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/test) ## Variables ### Required Variables
### Optional Variables
`alert_policy` (`map`) optional
Opsgenie Alert Policy configuration **Default value:** `{ }`
`api_integration` (`map(any)`) optional
Opsgenie API Integration configuration **Default value:** `{ }`
`escalation` (`map`) optional
Opsgenie Escalation configuration **Default value:** `{ }`
`integration_action` (`map`) optional
Opsgenie Integration Action configuration **Default value:** `{ }`
`notification_policy` (`map`) optional
Opsgenie Notification Policy configuration **Default value:** `{ }`
`opsgenie_provider_api_key` (`string`) optional
The API Key for the Opsgenie Integration. If omitted, the OPSGENIE_API_KEY environment variable is used **Default value:** `""`
`service` (`map`) optional
Opsgenie Service configuration **Default value:** `{ }`
`service_incident_rule` (`map`) optional
Opsgenie Service Incident Rule configuration **Default value:** `{ }`
`team` (`map`) optional
Opsgenie Team configuration **Default value:** `{ }`
`team_routing_rule` (`map`) optional
Opsgenie Team Routing Rule configuration **Default value:** `{ }`
`user` (`map`) optional
Opsgenie User configuration **Default value:** `{ }`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`alert_policy_filter`
Filters of the Opsgenie Alert Policy
`alert_policy_id`
The ID of the Opsgenie Alert Policy
`alert_policy_name`
Name of the Opsgenie Alert Policy
`alert_policy_priority`
Priority of the Opsgenie Alert Policy
`alert_policy_responders`
Responders of the Opsgenie Alert Policy.
`alert_policy_tags`
Tags of the Opsgenie Alert Policy
`api_integration_api_key`
API key of the created integration
`api_integration_id`
The ID of the Opsgenie API Integration
`api_integration_name`
The name of the Opsgenie API Integration
`escalation_id`
The ID of the Opsgenie Escalation
`escalation_name`
Name of the Opsgenie Escalation
`integration_action_id`
The ID of the Opsgenie Integration Action
`notification_policy_id`
The ID of the Opsgenie Notification Policy
`notification_policy_name`
The name of the Opsgenie Notification Policy
`service_id`
The ID of the Opsgenie Service
`service_incident_rule_id`
The ID of the Opsgenie Service Incident Rule
`service_name`
The name of the Opsgenie Service
`team_id`
The ID of the Opsgenie Team
`team_name`
The name of the Opsgenie Team
`team_routing_rule_id`
The ID of the Opsgenie Team Routing Rule
`team_routing_rule_name`
The name of the Opsgenie Team Routing Rule
`user_id`
The ID of the Opsgenie User
`user_name`
The name of the Opsgenie User
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `opsgenie`, version: `>= 0.4` ### Modules Name | Version | Source | Description --- | --- | --- | --- `alert_policy` | latest | [`./modules/alert_policy`](https://registry.terraform.io/modules/./modules/alert_policy/) | n/a `api_integration` | latest | [`./modules/api_integration`](https://registry.terraform.io/modules/./modules/api_integration/) | n/a `escalation` | latest | [`./modules/escalation`](https://registry.terraform.io/modules/./modules/escalation/) | n/a `integration_action` | latest | [`./modules/integration_action`](https://registry.terraform.io/modules/./modules/integration_action/) | n/a `notification_policy` | latest | [`./modules/notification_policy`](https://registry.terraform.io/modules/./modules/notification_policy/) | n/a `service` | latest | [`./modules/service`](https://registry.terraform.io/modules/./modules/service/) | n/a `service_incident_rule` | latest | [`./modules/service_incident_rule`](https://registry.terraform.io/modules/./modules/service_incident_rule/) | n/a `team` | latest | [`./modules/team`](https://registry.terraform.io/modules/./modules/team/) | n/a `team_routing_rule` | latest | [`./modules/team_routing_rule`](https://registry.terraform.io/modules/./modules/team_routing_rule/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `user` | latest | [`./modules/user`](https://registry.terraform.io/modules/./modules/user/) | n/a --- ## alert_policy ## Alert Policy Terraform module to configure [Opsgenie Alert Policy](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/alert_policy) ## Usage [Create Opsgenie Alert Policy example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/alert_policy/../../examples/alert_policy) ```hcl module "alert_policy" { source = "cloudposse/incident-management/opsgenie//modules/alert_policy" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" alert_policy = { name = "alert-policy" tags = ["test1", "test2"] priority = "P1" filter = { type = "match-all-conditions" conditions = [ { field = "source" operation = "matches" expected_value = ".*prod.*" }, { field = "tags" operation = "contains" expected_value = "severity:critical" } ] } } } ``` ## Inputs **Note:** `alert_policy` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `alert_policy` | `{}` | This variable is used to configure Opsgenie Alert Policy. | Yes | ## Outputs | Name | Description | |:----------------------------|:----------------------------------------| | `alert_policy_id` | The ID of the Opsgenie Alert Policy. | | `alert_policy_name` | Name of the Opsgenie Alert Policy. | | `alert_policy_filter` | Filters of the Opsgenie Alert Policy. | | `alert_policy_tags` | Tags of the Opsgenie Alert Policy. | | `alert_policy_priority` | Priority of the Opsgenie Alert Policy. | | `alert_policy_responders` | Responders of the Opsgenie Alert Policy.| --- ## api_integration ## API Integration Terraform module to configure [Opsgenie API Integration](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/api_integration) ## Usage [Create Opsgenie API Integration example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/api_integration/../../examples/alert_policy) ```hcl module "api_integration" { source = "cloudposse/incident-management/opsgenie//modules/api_integration" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" api_integration = { name = module.label.id type = "AmazonSns" owner_team_id = module.team.team_id } } ``` ## Inputs **Note:** `api_integration` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `api_integration` | `{}` | This variable is used to configure Opsgenie API Integration. | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `api_integration_api_key` | API key of the created integration | | `api_integration_name` | The name of the Opsgenie API Integration.| | `api_integration_id` | The ID of the Opsgenie API Integration. | --- ## config(Config) ## Config Terraform module that configures a multitude of [Opsgenie resources](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs). Many resources have cross-resource dependencies, which may be simpler to handle within a single module in certain cases, such as using YAML configurations. This module is designed to accept an input configuration map. One nice way of handling this is by passing resource definitions from a YAML configuration file. See below for details & examples. ## YAML Examples ### `alert_policies.yaml` ```yaml alert_policies: - name: "prioritize-env-prod-critical-alerts" owner_team_name: acme.dev tags: - "ManagedBy:terraform" filter: type: match-all-conditions conditions: - field: source operation: matches expected_value: ".*prod.acme.*" - field: tags operation: contains expected_value: "severity:critical" priority: P1 ``` ### `api_integrations.yaml` ```yaml api_integrations: - name: acme-dev-opsgenie-sns-integration type: AmazonSns owner_team_name: acme.dev ``` ### `escalations.yaml` ```yaml escalations: - name: acme.dev.some-service-escalation description: "repo: https://github.com/acme/some-service;owner:David Lightman @David Lightman" owner_team_name: acme.dev rules: - condition: if-not-acked notify_type: default delay: 0 recipient: type: team team_name: acme.dev.some-service ``` ### `notification_policies.yaml` ```yaml notification_policies: - name: auto-close-based-on-priority team_name: acme.dev auto_close_action: time_amount: 60 filter: type: match-all-conditions conditions: - field: priority operation: less-than expected_value: P3 ``` ### `schedules.yaml` ```yaml schedules: - name: acme.default description: "Acme Infrastructure Team" timezone: "America/Los_Angeles" owner_team_name: acme enabled: true ``` ### `schedule_rotations.yaml` ```yaml schedule_rotations: - name: acme.default.rotation schedule_name: acme.default start_date: "1970-01-01T00:00:00Z" type: weekly length: 1 participants: - type: user username: opsgenie-test@cloudposse.com - type: none - type: user username: opsgenie-test-2@cloudposse.com time_restriction: type: time-of-day restrictions: - start_hour: 8 start_min: 0 end_hour: 20 end_min: 0 ``` ### `team_routing_rules.yaml` ```yaml team_routing_rules: - name: some-service owner_team_name: acme.dev criteria: type: match-all-conditions conditions: - field: tags operation: contains expected_value: app:some-service not: false notify: - type: escalation name: acme.dev.some-service-escalation order: 0 ``` ### `teams.yaml` ```yaml teams: - name: acme description: Global Team for Acme Co. - name: acme.dev description: Acme Dev Team - name: acme.dev.some-service description: "repo: https://github.com/acme/some-service;owner:David Lightman @David Lightman" ``` ### `users.yaml` ```yaml users: - username: opsgenie-test@cloudposse.com full_name: Opsgenie Test User role: User locale: "en_US" timezone: "America/New_York" ``` ### `existing_schedules.yaml` ```yaml existing_schedules: - acme.default ``` ### `existing_teams.yaml` ```yaml existing_teams: - acme ``` ### `existing_users.yaml` ```yaml existing_users: - username: opsgenie-test@cloudposse.com ``` ### `services.yaml` ```yaml services: - name: frontend team_id: "..." description: Frontend service ``` ### `service_incident_rules.yaml` ```yaml service_incident_rules: - name: frontend-service-incident-rule-1 service_name: frontend incident_rule: condition_match_type: match-all conditions: - field: source operation: matches expected_value: ".*stage.*" - field: tags operation: contains expected_value: "severity:info" incident_properties: message: This is a test message priority: P3 stakeholder_properties: message: Message for stakeholders enable: true ``` ## Usage [Full Config Example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/config/../../examples/config) ```hcl locals { # Build a map of our various Opsgenie resources that will be used to iterate over each module opsgenie_resources = merge([ for resource_file in fileset(path.cwd, "resources/*.yaml") : { for k, v in yamldecode(file(resource_file)) : k => v } ]...) } module "opsgenie" { source = "cloudposse/incident-management/opsgenie//modules/config" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" opsgenie_resources = local.opsgenie_resources } ``` ## Inputs | Name | Default | Description | Required | |:-------------------------------|:----------------:|:-------------------------------------------------------------------------------|:--------:| | `opsgenie_resources` | `{}` | A map generated by sourcing YAML resource definitions (see above). | Yes | ## Outputs | Name | Description | |:----------------------------|:--------------------------------------------| | `alert_policies` | `name` and `id` of each alert policy | | `api_integrations` | `name` and `id` of each API integration | | `escalations` | `name` and `id` of each escalation | | `existing_schedules` | `name` and `id` of each existing schedule | | `existing_teams` | `name` and `id` of each existing team | | `existing_users` | `username` and `id` of each existing user | | `notification_policies` | `name` and `id` of each notification policy | | `schedules` | `name` and `id` of each schedules | | `schedule_rotations` | `name` and `id` of each schedule rotations | | `services` | `name` and `id` of each service | | `services` | `name` and `id` of each service | | `service_incident_rule_ids` | `id` of each service incident rule | | `team_routing_rules` | `name` and `id` of each team routing rule | | `teams` | `name` and `id` of each team | | `users` | `username` and `id` of each user | --- ## escalation(Escalation) ## Escalation Terraform module to configure [Opsgenie Escalation](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/escalation) ## Usage [Create Opsgenie Escalation example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/escalation/../../examples/escalation) ```hcl module "escalation" { source = "cloudposse/incident-management/opsgenie//modules/escalation" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" escalation = { name = module.label.id owner_team_id = module.owner_team.team_id rules = [{ recipient = { type = "team" id = module.escalation_team.team_id } }] } } ``` ## Inputs **Note:** `escalation` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `escalation` | `{}` | This variable is used to configure Opsgenie Escalation. | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `escalation_name` | The name of the Opsgenie Escalation.| | `escalation_id` | The ID of the Opsgenie Escalation. | --- ## integration_action ## Integration Action Terraform module to configure [Opsgenie Integration Action](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/integration_action) NOTE: your OpsGenie plan must support advanced integrations. Otherwise, you will get the following error back from the API: `Your plan does not allow saving advanced integrations.`. ## Usage [Create Opsgenie Integration Action example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/integration_action/../../examples/integration_action) ```hcl module "integration_action" { source = "cloudposse/incident-management/opsgenie//modules/integration_action" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" integration_action = { integration_id = module.api_integration.api_integration_id create = [ { name = "Create Non-informational Alerts" alias = "{{title}}" filter = { type = "match-all-conditions" conditions = [ { field = "priority" not = true operation = "equals" expected_value = "P5" } ] } } ] } } ``` ## Inputs **Note:** `integration_action` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `integration_action` | `{}` | This variable is used to configure Opsgenie Integration Action. | Yes | ## Outputs | Name | Description | |:----------------------------|:---------------------------------------------| | `integration_action_id` | The ID of the Opsgenie Integration Action. | --- ## notification_policy ## Notification Policy Terraform module to configure [Opsgenie Notification Policy](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/notification_policy) ## Usage [Create Opsgenie Notification Policy example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/notification_policy/../../examples/notification_policy) ```hcl module "notification_policy" { source = "cloudposse/incident-management/opsgenie//modules/notification_policy" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" notification_policy = { name = module.label.id team_id = module.team.team_id filter = { type = "match-all-conditions" conditions = [{ field = "tags" operation = "contains" expected_value = "recommendation:auto-close" }] } de_duplication_action = { de_duplication_action_type = "frequency-based" count = 2 duration = { time_unit = "minutes" time_amount = 5 } } delay_action = { delay_option = "for-duration" duration = { time_unit = "minutes" time_amount = 10 } } auto_close_action = { time_unit = "minutes" time_amount = 5 } } } ``` ## Inputs **Note:** `notification_policy` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `notification_policy` | `{}` | This variable is used to configure Opsgenie Notification Policy. | Yes | ## Outputs | Name | Description | |:----------------------------|:---------------------------------------------| | `notification_policy_name` | The name of the Opsgenie Notification Policy.| | `notification_policy_id` | The ID of the Opsgenie Notification Policy. | --- ## schedule Terraform module to configure [Opsgenie Schedule](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/schedule) ## Usage [Opsgenie Schedule example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/schedule/../../examples/schedule) ```hcl module "schedule" { source = "cloudposse/incident-management/opsgenie//modules/schedule" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" schedule = { name = module.label.id description = "schedule-description" } } data "opsgenie_team" "the_team" { name = var.team_name } module "team_schedule" { source = "cloudposse/incident-management/opsgenie//modules/schedule" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" schedule = { name = module.label.id description = "team-schedule-description" owner_team_id = data.opsgenie_team.the_team.id } } ``` ## Inputs **Note:** `schedule` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `schedule` | `{}` | This variable is used to configure Opsgenie schedule. | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `schedule_name` | The name of the Opsgenie schedule. | | `schedule_id` | The ID of the Opsgenie schedule. | --- ## service ## Service Terraform module to configure [Opsgenie Service](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/service) ## Usage ```hcl module "service" { source = "cloudposse/incident-management/opsgenie//modules/service" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" service = { name = "frontend" team_id = "..." description = "My company frontend service" } } ``` ## Inputs **Note:** `service` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `service` | `{}` | Opsgenie Service configuration | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `service_id` | The ID of the Opsgenie Service | | `service_name` | The name of the Opsgenie Service | --- ## service_incident_rule ## Service Incident Rule Terraform module to configure [Opsgenie Service Incident Rule](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/service_incident_rule) ## Usage [Create Opsgenie Team Routing Rule example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/service_incident_rule/../../examples/service_incident_rule) ```hcl module "service_incident_rule" { source = "cloudposse/incident-management/opsgenie//modules/service_incident_rule" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" service_incident_rule = { service_id = "..." incident_rule = { condition_match_type = "match-all" conditions = [ { field = "tags" operation = "contains" expected_value = "expected1" } ] incident_properties = { message = "This is a test message" priority = "P3" stakeholder_properties = { message = "Message for stakeholders" enable = true } } } } } ``` ## Inputs **Note:** `service_incident_rule` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:-------------------------------------------------|:--------:| | `service_incident_rule` | `{}` | Opsgenie Service Incident Rule configuration | Yes | ## Outputs | Name | Description | |:------------------------------|:-----------------------------------------------| | `service_incident_rule_id` | The ID of the Opsgenie Service Incident Rule | --- ## team ## Team Terraform module to configure [Opsgenie Team](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/team) ## Usage [Create Opsgenie Team example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/team/../../examples/team) ```hcl module "team" { source = "cloudposse/incident-management/opsgenie//modules/team" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" team = { name = module.label.id description = "team-description" } } module "ui_managed_team" { source = "cloudposse/incident-management/opsgenie//modules/team" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" team = { name = module.label.id description = "team-description" delete_default_resources = true ignore_members = true } } ``` ## Inputs **Note:** `team` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `team` | `{}` | This variable is used to configure Opsgenie Team. | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `team_name` | The name of the Opsgenie Team. | | `team_id` | The ID of the Opsgenie Team. | --- ## team_routing_rule ## Team Routing Rule Terraform module to configure [Opsgenie Team Routing Rule](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/team_routing_rule) ## Usage [Create Opsgenie Team Routing Rule example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/team_routing_rule/../../examples/team_routing_rule) ```hcl module "team_routing_rule" { source = "cloudposse/incident-management/opsgenie//modules/team_routing_rule" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" team_routing_rule = { name = module.label.id team_id = module.owner_team.team_id notify = [{ type = "escalation" id = module.escalation.escalation_id }] } } ``` ## Inputs **Note:** `team_routing_rule` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `team_routing_rule` | `{}` | This variable is used to configure Opsgenie Team Routing Rule. | Yes | ## Outputs | Name | Description | |:----------------------------|:--------------------------------------------| | `team_routing_rule_name` | The name of the Opsgenie Team Routing Rule.| | `team_routing_rule_id` | The ID of the Opsgenie Team Routing Rule. | ## Important Note Due to the Opsgenie terraform provider issue, there is a difference in the configuration of the `time_restriction` blocks based on `type`. [Github Issue #282](https://github.com/opsgenie/terraform-provider-opsgenie/issues/282) ```hcl time_restriction { type = "time-of-day" restriction { end_hour = 17 end_min = 0 start_hour = 9 start_min = 0 } } ``` vs ```hcl time_restriction { type = "weekday-and-time-of-day" restriction { end_day = "friday" end_hour = 17 end_min = 0 start_day = "monday" start_hour = 9 start_min = 0 } } ``` --- ## user ## User Terraform module to configure [Opsgenie User](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/resources/user) ## Usage [Create Opsgenie User example](https://github.com/cloudposse/terraform-opsgenie-incident-management/tree/main/modules/user/../../examples/user) ```hcl module "user" { source = "cloudposse/incident-management/opsgenie//modules/user" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" user = { username = "opsgenie-test@cloudposse.com" full_name = "Opsgenie Test User" role = "User" locale = "en_US" timezone = "America/New_York" } } ``` ## Inputs **Note:** `user` is a map for two reasons: - to be able to put whole configuration in yaml file - variables defined with type set are not robust enough (can't set default values) | Name | Default | Description | Required | |:-------------------------------|:---------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------|:--------:| | `user` | `{}` | Opsgenie User configuration | Yes | ## Outputs | Name | Description | |:----------------------------|:-----------------------------------------| | `user_id` | The ID of the Opsgenie User | | `user_name` | The name of the Opsgenie User | --- ## Opsgenie import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Implement and manage Opsgenie integrations with Terraform modules. These modules support efficient incident management and alerting workflows within your infrastructure. --- ## awsutils # Module: `awsutils` Terraform provider for performing various tasks that cannot be performed with the official [AWS Terraform Provider](https://github.com/hashicorp/terraform-provider-aws) from Hashicorp. This provider is derived in large parts from the official HashiCorp AWS provider. We copied all the boilerplate functionality so that it follows the `terraform-provider-aws` conventions, but then removed all the standard resources and added in our own. This module is intended to be used as an escape hatch to accomplish all the hard things that will never be supported by the official provider due to strong (and valid) opinions of how providers should manage the lifecycle of a resource. Unfortunately, in the real-world we have to make tradeoffs to get stuff done. That's this provider in a nutshell. ## Usage Here is how to use this provider in your own Terraform code: ```hcl terraform { required_providers { awsutils = { source = "cloudposse/awsutils" version = ">= 0.1.0" } } } provider "awsutils" { region = "us-east-2" } ``` See the [Docs](https://github.com/cloudposse/terraform-provider-awsutils/tree/main/docs) for additional information. ## Examples Here is an example of using this provider: ```hcl terraform { required_providers { awsutils = { source = "cloudposse/awsutils" } } } ``` Here are some additional examples: - [`examples/resources/awsutils_default_vpc_deletion`](https://github.com/cloudposse/terraform-provider-awsutils/tree/main/examples/resources/awsutils_default_vpc_deletion/) ## Variables ### Required Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Dependencies ## Developing the Provider If you wish to work on the provider, you'll first need [Go](http://www.golang.org) installed on your machine (see [Requirements](#requirements) above). To compile the provider, run `go install`. This will build the provider and put the provider binary in the `$GOPATH/bin` directory. To generate or update documentation, run `go generate`. In order to run the full suite of Acceptance tests, run `atmos testacc`. _Note:_ Acceptance tests create real resources, and often cost money to run. ```sh $ atmos testacc ``` ### Testing Locally You can test the provider locally by using the [provider_installation](https://www.terraform.io/docs/cli/config/config-file.html#provider-installation) functionality. For testing this provider, you can edit your `~/.terraformrc` file with the following: ```hcl provider_installation { dev_overrides { "cloudposse/awsutils" = "/path/to/your/code/github.com/cloudposse/terraform-provider-awsutils/" } # For all other providers, install them directly from their origin provider # registries as normal. If you omit this, Terraform will _only_ use # the dev_overrides block, and so no other providers will be available. direct {} } ``` With that in place, you can build the provider (see above) and add a provider block: ```hcl required_providers { awsutils = { source = "cloudposse/awsutils" } } ``` Then run `terraform init`, `terraform plan` and `terraform apply` as normal. ```sh $ terraform init Initializing the backend... Initializing provider plugins... - Finding latest version of cloudposse/awsutils... Warning: Provider development overrides are in effect The following provider development overrides are set in the CLI configuration: - cloudposse/awsutils in /path/to/your/code/github.com/cloudposse/terraform-provider-awsutils The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. ``` ```sh terraform apply Warning: Provider development overrides are in effect The following provider development overrides are set in the CLI configuration: - cloudposse/awsutils in /Users/matt/code/src/github.com/cloudposse/terraform-provider-awsutils The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: Terraform will perform the following actions: Plan: 1 to add, 0 to change, 0 to destroy. ``` --- ## context # Module: `context` Terrform provider for managing a context in Terraform. A context, in this case, is a set of key-value pairs that can be used to enable or disable a module, as well as generating consistent resource names and tags for cloud resources. ## Introduction Terrform provider for managing a context in Terraform. A context, in this case, is a set of key-value pairs that can be used to enable or disable a module, as well as generating consistent resource names and tags for cloud resources. This provider is intended to be a replacement for Cloud Posse's [terraform-null-label](https://github.com/cloudposse/terraform-null-label) Terraform module as well as the [context.tf](https://github.com/cloudposse/terraform-null-label/blob/main/exports/context.tf) export from that module, which is copied into all of Cloud Posse's modules and components (root modules) via automation. The provider is designed to be more flexible and easier to use than the `terraform-null-label` module, and to provide a consistent way to manage context across all of Cloud Posse's modules and components via a provider, rather than a Terraform module. This provider also allows flexibility in the property names that can be used to generate labels and tags, where the previous module-based solution was limited to a fixed set of properties (namespace, tenant, stage, environment, name and attributes). ## Usage Here is how to use this provider in your own Terraform code: ```hcl provider "context" { delimiter = "~" enabled = false properties = { namespace = {} tenant = {} stage = {} environment = {} name = {} } property_order = ["namespace", "tenant", "stage", "environment", "name"] values = { "namespace" = "cp" "tenant" = "core" "stage" = "prod" "environment" = "ue1" "name" = "example" } } data "context_label" "example" { values = { "tenant" = "plat" "stage" = "dev" } } data "context_label" "example" { template = "{{.namespace}}/{{.tenant}}/{{.stage}}/{{.name}}" values = { "tenant" = "plat" "stage" = "dev" } } ``` See the [Docs](https://github.com/cloudposse/terraform-provider-context/tree/main/docs) for additional information. ## Examples Here are some additional examples: - [`examples/data-sources/config`](https://github.com/cloudposse/terraform-provider-context/tree/main/examples/data-sources/config/) - [`examples/data-sources/label-delimited`](https://github.com/cloudposse/terraform-provider-context/tree/main/examples/data-sources/label-delimited/) - [`examples/data-sources/label-templated`](https://github.com/cloudposse/terraform-provider-context/tree/main/examples/data-sources/label-templated/) - [`examples/data-sources/tags`](https://github.com/cloudposse/terraform-provider-context/tree/main/examples/data-sources/tags/) ## Variables ### Required Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Dependencies --- ## Provider import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Use these Terraform modules to manage and configure various providers within your infrastructure. Providers enable Terraform to interact with cloud services, APIs, and other platforms. --- ## utils(Utils) # Module: `utils` Terraform provider for various utilities (deep merging, Atmos stack configuration management), and to add additional missing functionality to Terraform ## Usage Here is how to use this provider in your own Terraform code: ```hcl terraform { required_providers { utils = { source = "cloudposse/utils" version = ">= 1.17.0" } } } ``` See the [Docs](https://github.com/cloudposse/terraform-provider-utils/tree/main/docs) for additional information. ## Examples Here is an example of using this provider: ```hcl terraform { required_providers { utils = { source = "cloudposse/utils" } } } locals { yaml_data_1 = file("${path.module}/data1.yaml") yaml_data_2 = file("${path.module}/data2.yaml") } data "utils_deep_merge_yaml" "example" { input = [ local.yaml_data_1, local.yaml_data_2 ] } output "deep_merge_output" { value = data.utils_deep_merge_yaml.example.output } ``` Here are some additional examples: - [`examples/data-sources/utils_deep_merge_json`](https://github.com/cloudposse/terraform-provider-utils/tree/main/examples/data-sources/utils_deep_merge_json) - [`examples/data-sources/utils_deep_merge_yaml`](https://github.com/cloudposse/terraform-provider-utils/tree/main/examples/data-sources/utils_deep_merge_yaml) ## Variables ### Required Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Dependencies ## Developing the Provider If you wish to work on the provider, you'll first need [Go](http://www.golang.org) installed on your machine (see [Requirements](#requirements) above). Install [atmos](https://atmos.tools/install) To compile the provider, run `go install`. This will build the provider and put the provider binary in the `$GOPATH/bin` directory. To generate or update documentation, run `go generate`. In order to run the full suite of Acceptance tests, run `atmos testacc`. _Note:_ Acceptance tests create real resources, and often cost money to run. ```sh $ atmos testacc ``` ### Testing Locally You can test the provider locally by using the [provider_installation](https://www.terraform.io/docs/cli/config/config-file.html#provider-installation) functionality. For testing this provider, you can edit your `~/.terraformrc` file with the following: ```hcl provider_installation { dev_overrides { "cloudposse/utils" = "/path/to/your/code/github.com/cloudposse/terraform-provider-utils/" } # For all other providers, install them directly from their origin provider # registries as normal. If you omit this, Terraform will _only_ use # the dev_overrides block, and so no other providers will be available. direct {} } ``` With that in place, you can build the provider (see above) and add a provider block: ```hcl required_providers { utils = { source = "cloudposse/utils" } } ``` Then run `terraform init`, `terraform plan` and `terraform apply` as normal. ```sh $ terraform init Initializing the backend... Initializing provider plugins... - Finding latest version of cloudposse/utils... Warning: Provider development overrides are in effect The following provider development overrides are set in the CLI configuration: - cloudposse/utils in /path/to/your/code/github.com/cloudposse/terraform-provider-utils The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. ``` ```sh terraform apply Warning: Provider development overrides are in effect The following provider development overrides are set in the CLI configuration: - cloudposse/utils in /Users/matt/code/src/github.com/cloudposse/terraform-provider-utils The behavior may therefore not match any released version of the provider and applying changes may cause the state to become incompatible with published releases. An execution plan has been generated and is shown below. Resource actions are indicated with the following symbols: Terraform will perform the following actions: Plan: 0 to add, 0 to change, 0 to destroy. Changes to Outputs: + deep_merge_output = <<-EOT Statement: - Action: - s3:* Effect: Allow Resource: - '*' Sid: FullAccess - Action: - s3:* Complex: ExtraComplex: ExtraExtraComplex: Foo: bazzz SomeArray: - one - two - three Effect: Deny Resource: - arn:aws:s3:::customer - arn:aws:s3:::customer/* - foo Sid: DenyCustomerBucket Version: "2012-10-17" EOT ``` --- ## cloud-infrastructure-automation # Module: `cloud-infrastructure-automation` This repo contains a set of Terraform modules for implementing a CI/CD pipeline for Terraform infrastructure using Spacelift. - [spacelift-policy](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/main/modules/spacelift-policy) - Terraform module for creating Spacelift policies. - [spacelift-space](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/main/modules/spacelift-space) - Terraform module for creating Spacelift spaces. - [spacelift-stack](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/main/modules/spacelift-stack) - Terraform module for creating Spacelift stacks. - [spacelift-stacks-from-atmos-config](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/main/modules/spacelift-stacks-from-atmos-config) - Terraform module for extracting Spacelift stack config from Atmos config. ## Variables ### Required Variables
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
## Dependencies --- ## spacelift-policy # Module: `spacelift-policy` Terraform module to provision a [Spacelift](https://docs.spacelift.io/concepts/spacelift-policy/) policy. ## Usage Here's how to invoke this module in your project: ```hcl provider "spacelift" {} module "inline_policy" { source = "cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-policy" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" policy_name = "Test Inline Policy" policy_type = "PLAN" body = < ## Variables ### Required Variables
`policy_name` (`string`) required
The name of the policy to create. Should be unique across the spacelift account.
`space_id` (`string`) required
The `space_id` (slug) of the space the policy is in.
`type` (`string`) required
The type of the policy to create.
### Optional Variables
`body` (`string`) optional
The body of the policy to create. Mutually exclusive with `var.body_url` and `var.body_file_path`. **Default value:** `null`
`body_file_path` (`string`) optional
The local path to the file containing the policy body. Mutually exclusive with `var.body` and `var.body_url`. **Default value:** `null`
`body_url` (`string`) optional
The URL of file containing the body of policy to create. Mutually exclusive with `var.body` and `var.body_file_path`. **Default value:** `null`
`body_url_version` (`string`) optional
The optional policy version injected using a %s in `var.body_url`. This can be pinned to a version tag or a branch. **Default value:** `"master"`
`labels` (`set(string)`) optional
List of labels to add to the policy. **Default value:** `[ ]`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
The ID of the created policy.
`policy`
The created policy.
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `http`, version: `>= 3.0` - `spacelift`, version: `>= 0.1.31` ### Providers - `http`, version: `>= 3.0` - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`spacelift_policy.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/policy) (resource) ## Data Sources The following data sources are used by this module: - [`http_http.this`](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) (data source) ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.0 | | [http](#requirement\_http) | >= 3.0 | | [spacelift](#requirement\_spacelift) | >= 0.1.31 | ## Providers | Name | Version | |------|---------| | [http](#provider\_http) | >= 3.0 | | [spacelift](#provider\_spacelift) | >= 0.1.31 | ## Modules | Name | Source | Version | |------|--------|---------| | [this](#module\_this) | cloudposse/label/null | 0.25.0 | ## Resources | Name | Type | |------|------| | [spacelift_policy.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/policy) | resource | | [http_http.this](https://registry.terraform.io/providers/hashicorp/http/latest/docs/data-sources/http) | data source | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.This is for some rare cases where resources want additional configuration of tagsand therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,in the order they appear in the list. New attributes are appended to theend of the list. The elements of the list are joined by the `delimiter`and treated as a single ID element. | `list(string)` | `[]` | no | | [body](#input\_body) | The body of the policy to create. Mutually exclusive with `var.body_url` and `var.body_file_path`. | `string` | `null` | no | | [body\_file\_path](#input\_body\_file\_path) | The local path to the file containing the policy body. Mutually exclusive with `var.body` and `var.body_url`. | `string` | `null` | no | | [body\_url](#input\_body\_url) | The URL of file containing the body of policy to create. Mutually exclusive with `var.body` and `var.body_file_path`. | `string` | `null` | no | | [body\_url\_version](#input\_body\_url\_version) | The optional policy version injected using a %s in `var.body_url`. This can be pinned to a version tag or a branch. | `string` | `"master"` | no | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "descriptor_formats": \{\},  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "labels_as_tags": [    "unset"  ],  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\},  "tenant": null\}
| no | | [delimiter](#input\_delimiter) | Delimiter to be used between ID elements.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [descriptor\_formats](#input\_descriptor\_formats) | Describe additional descriptors to be output in the `descriptors` output map.Map of maps. Keys are names of descriptors. Values are maps of the form`{ format = string labels = list(string)}`(Type is `any` so the map values can later be enhanced to provide additional options.)`format` is a Terraform format string to be passed to the `format()` function.`labels` is a list of labels, in order, to pass to `format()` function.Label values will be normalized before being passed to `format()` so they will beidentical to how they appear in `id`.Default is `{}` (`descriptors` output will be empty). | `any` | `{}` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for keep the existing setting, which defaults to `0`.Does not affect `id_full`. | `number` | `null` | no | | [label\_key\_case](#input\_label\_key\_case) | Controls the letter case of the `tags` keys (label names) for tags generated by this module.Does not affect keys of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The order in which the labels (ID elements) appear in the `id`.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | Controls the letter case of ID elements (labels) as included in `id`,set as tag values, and output by this module individually.Does not affect values of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper` and `none` (no transformation).Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs.Default value: `lower`. | `string` | `null` | no | | [labels](#input\_labels) | List of labels to add to the policy. | `set(string)` | `[]` | no | | [labels\_as\_tags](#input\_labels\_as\_tags) | Set of labels (ID elements) to include as tags in the `tags` output.Default is to include all labels.Tags with empty values will not be included in the `tags` output.Set to `[]` to suppress all generated tags.**Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. | `set(string)` |
[  "default"]
| no | | [name](#input\_name) | ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.This is the only ID element not also included as a `tag`.The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. | `string` | `null` | no | | [namespace](#input\_namespace) | ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique | `string` | `null` | no | | [policy\_name](#input\_policy\_name) | The name of the policy to create. Should be unique across the spacelift account. | `string` | n/a | yes | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.Characters matching the regex will be removed from the ID elements.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [space\_id](#input\_space\_id) | The `space_id` (slug) of the space the policy is in. | `string` | n/a | yes | | [stage](#input\_stage) | ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input\_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | | [type](#input\_type) | The type of the policy to create. | `string` | n/a | yes | ## Outputs | Name | Description | |------|-------------| | [id](#output\_id) | The ID of the created policy. | | [policy](#output\_policy) | The created policy. | --- ## spacelift-space # Module: `spacelift-space` Terraform module to provisions a [Spacelift](https://docs.spacelift.io/concepts/spaces/index.html) space. ## Usage Here's how to invoke this module in your project: ```hcl provider "spacelift" {} module "space" { source = "cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-space" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" space_name = "test" description = "A space for our test infrasturcture" parent_space_id = "root" inherit_entities_from_parent = false labels = ["test", "space"] ``` ## Examples Here is an example of using this module: - [`../../examples/spacelift-space`](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/master/examples/spacelift-space) - complete example of using this module ## Variables ### Required Variables
`space_name` (`string`) required
Name of the space
### Optional Variables
`description` (`string`) optional
Description of the space **Default value:** `null`
`inherit_entities_from_parent` (`bool`) optional
Flag to indicate whether this space inherits read access to entities from the parent space. **Default value:** `false`
`labels` (`set(string)`) optional
List of labels to add to the space. **Default value:** `[ ]`
`parent_space_id` (`string`) optional
ID of the parent space **Default value:** `"root"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`space`
The created space
`space_id`
The ID of the created space
## Dependencies ### Requirements - `terraform`, version: `>= 1.0` - `http`, version: `>= 3.0` - `spacelift`, version: `>= 0.1.31` ### Providers - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`spacelift_space.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/space) (resource) ## Data Sources The following data sources are used by this module: ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 1.0 | | [http](#requirement\_http) | >= 3.0 | | [spacelift](#requirement\_spacelift) | >= 0.1.31 | ## Providers | Name | Version | |------|---------| | [spacelift](#provider\_spacelift) | >= 0.1.31 | ## Modules | Name | Source | Version | |------|--------|---------| | [this](#module\_this) | cloudposse/label/null | 0.25.0 | ## Resources | Name | Type | |------|------| | [spacelift_space.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/space) | resource | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.This is for some rare cases where resources want additional configuration of tagsand therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,in the order they appear in the list. New attributes are appended to theend of the list. The elements of the list are joined by the `delimiter`and treated as a single ID element. | `list(string)` | `[]` | no | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "descriptor_formats": \{\},  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "labels_as_tags": [    "unset"  ],  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\},  "tenant": null\}
| no | | [delimiter](#input\_delimiter) | Delimiter to be used between ID elements.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [description](#input\_description) | Description of the space | `string` | `null` | no | | [descriptor\_formats](#input\_descriptor\_formats) | Describe additional descriptors to be output in the `descriptors` output map.Map of maps. Keys are names of descriptors. Values are maps of the form`{ format = string labels = list(string)}`(Type is `any` so the map values can later be enhanced to provide additional options.)`format` is a Terraform format string to be passed to the `format()` function.`labels` is a list of labels, in order, to pass to `format()` function.Label values will be normalized before being passed to `format()` so they will beidentical to how they appear in `id`.Default is `{}` (`descriptors` output will be empty). | `any` | `{}` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for keep the existing setting, which defaults to `0`.Does not affect `id_full`. | `number` | `null` | no | | [inherit\_entities\_from\_parent](#input\_inherit\_entities\_from\_parent) | Flag to indicate whether this space inherits read access to entities from the parent space. | `bool` | `false` | no | | [label\_key\_case](#input\_label\_key\_case) | Controls the letter case of the `tags` keys (label names) for tags generated by this module.Does not affect keys of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The order in which the labels (ID elements) appear in the `id`.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | Controls the letter case of ID elements (labels) as included in `id`,set as tag values, and output by this module individually.Does not affect values of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper` and `none` (no transformation).Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs.Default value: `lower`. | `string` | `null` | no | | [labels](#input\_labels) | List of labels to add to the space. | `set(string)` | `[]` | no | | [labels\_as\_tags](#input\_labels\_as\_tags) | Set of labels (ID elements) to include as tags in the `tags` output.Default is to include all labels.Tags with empty values will not be included in the `tags` output.Set to `[]` to suppress all generated tags.**Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. | `set(string)` |
[  "default"]
| no | | [name](#input\_name) | ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.This is the only ID element not also included as a `tag`.The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. | `string` | `null` | no | | [namespace](#input\_namespace) | ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique | `string` | `null` | no | | [parent\_space\_id](#input\_parent\_space\_id) | ID of the parent space | `string` | `"root"` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.Characters matching the regex will be removed from the ID elements.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [space\_name](#input\_space\_name) | Name of the space | `string` | n/a | yes | | [stage](#input\_stage) | ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input\_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | ## Outputs | Name | Description | |------|-------------| | [space](#output\_space) | The created space | | [space\_id](#output\_space\_id) | The ID of the created space | --- ## spacelift-stack # Module: `spacelift-stack` Terraform module to provisions a [Spacelift](https://docs.spacelift.io/concepts/spaces/index.html) space. ## Usage Here's how to invoke this module in your project: ```hcl provider "spacelift" {} module "stack" { source = "cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-space" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" atmos_stack_name = "plat-ue1-prod-test-component" stack_name = "plat-ue1-prod-test-component" component_name = "test-component" component_root = "examples/test-component" repository = "spacelift-demo" branch = "main" autodeploy = true terraform_version = "1.4.6" ``` ## Examples Here is an example of using this module: - [`../../examples/spacelift-stack`](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/master/examples/spacelift-stack) - complete example of using this module ## Variables ### Required Variables
`atmos_stack_name` (`string`) required
The name of the atmos stack
`component_name` (`string`) required
The name of the concrete component (typically a directory name)
`component_root` (`string`) required
The path, relative to the root of the repository, where the component can be found
`repository` (`string`) required
The name of your infrastructure repo
`stack_name` (`string`) required
The name of the Spacelift stack
### Optional Variables
`administrative` (`bool`) optional
Whether this stack can manage other stacks **Default value:** `false`
`after_apply` (`list(string)`) optional
List of after-apply scripts **Default value:** `[ ]`
`after_destroy` (`list(string)`) optional
List of after-destroy scripts **Default value:** `[ ]`
`after_init` (`list(string)`) optional
List of after-init scripts **Default value:** `[ ]`
`after_perform` (`list(string)`) optional
List of after-perform scripts **Default value:** `[ ]`
`after_plan` (`list(string)`) optional
List of after-plan scripts **Default value:** `[ ]`
`autodeploy` (`bool`) optional
Controls the Spacelift 'autodeploy' option for a stack **Default value:** `false`
`autoretry` (`bool`) optional
Controls the Spacelift 'autoretry' option for a stack **Default value:** `false`
`aws_role_arn` (`string`) optional
ARN of the AWS IAM role to assume and put its temporary credentials in the runtime environment **Default value:** `null`
`aws_role_enabled` (`bool`) optional
Flag to enable/disable Spacelift to use AWS STS to assume the supplied IAM role and put its temporary credentials in the runtime environment **Default value:** `false`
`aws_role_external_id` (`string`) optional
Custom external ID (works only for private workers). See https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html for more details **Default value:** `null`
`aws_role_generate_credentials_in_worker` (`bool`) optional
Flag to enable/disable generating AWS credentials in the private worker after assuming the supplied IAM role **Default value:** `true`
`azure_devops` (`map(any)`) optional
Azure DevOps VCS settings **Default value:** `null`
`before_apply` (`list(string)`) optional
List of before-apply scripts **Default value:** `[ ]`
`before_destroy` (`list(string)`) optional
List of before-destroy scripts **Default value:** `[ ]`
`before_init` (`list(string)`) optional
List of before-init scripts **Default value:** `[ ]`
`before_perform` (`list(string)`) optional
List of before-perform scripts **Default value:** `[ ]`
`before_plan` (`list(string)`) optional
List of before-plan scripts **Default value:** `[ ]`
`bitbucket_cloud` (`map(any)`) optional
Bitbucket Cloud VCS settings **Default value:** `null`
`bitbucket_datacenter` (`map(any)`) optional
Bitbucket Datacenter VCS settings **Default value:** `null`
`branch` (`string`) optional
Specify which branch to use within your infrastructure repo **Default value:** `"main"`
`cloudformation` (`map(any)`) optional
CloudFormation-specific configuration. Presence means this Stack is a CloudFormation Stack. **Default value:** `null`
`commit_sha` (`string`) optional
The commit SHA for which to trigger a run. Requires `var.spacelift_run_enabled` to be set to `true` **Default value:** `null`
`component_env` (`any`) optional
Map of component ENV variables **Default value:** `{ }`
`component_vars` (`any`) optional
All Terraform values to be applied to the stack via a mounted file **Default value:** `{ }`
`context_attachments` (`list(string)`) optional
A list of context IDs to attach to this stack **Default value:** `[ ]`
`description` (`string`) optional
Specify description of stack **Default value:** `null`
`drift_detection_enabled` (`bool`) optional
Flag to enable/disable drift detection on the infrastructure stacks **Default value:** `false`
`drift_detection_reconcile` (`bool`) optional
Flag to enable/disable infrastructure stacks drift automatic reconciliation. If drift is detected and `reconcile` is turned on, Spacelift will create a tracked run to correct the drift **Default value:** `false`
`drift_detection_schedule` (`list(string)`) optional
List of cron expressions to schedule drift detection for the infrastructure stacks **Default value:** ```hcl [ "0 4 * * *" ] ```
`drift_detection_timezone` (`string`) optional
Timezone in which the schedule is expressed. Defaults to UTC. **Default value:** `"UTC"`
`github_enterprise` (`map(any)`) optional
GitHub Enterprise (self-hosted) VCS settings **Default value:** `null`
`gitlab` (`map(any)`) optional
GitLab VCS settings **Default value:** `null`
`labels` (`list(string)`) optional
A list of labels for the stack **Default value:** `[ ]`
`local_preview_enabled` (`bool`) optional
Indicates whether local preview runs can be triggered on this Stack **Default value:** `false`
`manage_state` (`bool`) optional
Flag to enable/disable manage_state setting in stack **Default value:** `true`
`policy_ids` (`list(string)`) optional
List of Rego policy IDs to attach to this stack **Default value:** `[ ]`
`protect_from_deletion` (`bool`) optional
Flag to enable/disable deletion protection. **Default value:** `false`
`pulumi` (`map(any)`) optional
Pulumi-specific configuration. Presence means this Stack is a Pulumi Stack. **Default value:** `null`
`runner_image` (`string`) optional
The full image name and tag of the Docker image to use in Spacelift **Default value:** `null`
`showcase` (`map(any)`) optional
Showcase settings **Default value:** `null`
`space_id` (`string`) optional
Place the stack in the specified space_id. **Default value:** `"root"`
`spacelift_run_enabled` (`bool`) optional
Enable/disable creation of the `spacelift_run` resource **Default value:** `false`
`spacelift_stack_dependency_enabled` (`bool`) optional
If enabled, the `spacelift_stack_dependency` Spacelift resource will be used to create dependencies between stacks instead of using the `depends-on` labels. The `depends-on` labels will be removed from the stacks and the trigger policies for dependencies will be detached **Default value:** `false`
`stack_destructor_enabled` (`bool`) optional
Flag to enable/disable the stack destructor to destroy the resources of the stack before deleting the stack itself **Default value:** `false`
`terraform_smart_sanitization` (`bool`) optional
Whether or not to enable [Smart Sanitization](https://docs.spacelift.io/vendors/terraform/resource-sanitization) which will only sanitize values marked as sensitive. **Default value:** `false`
`terraform_version` (`string`) optional
Specify the version of Terraform to use for the stack **Default value:** `null`
`terraform_workflow_tool` (`string`) optional
Defines the tool that will be used to execute the workflow. This can be one of OPEN_TOFU, TERRAFORM_FOSS or CUSTOM. Defaults to TERRAFORM_FOSS. **Default value:** `"TERRAFORM_FOSS"`
`terraform_workspace` (`string`) optional
Specify the Terraform workspace to use for the stack **Default value:** `null`
`webhook_enabled` (`bool`) optional
Flag to enable/disable the webhook endpoint to which Spacelift sends the POST requests about run state changes **Default value:** `false`
`webhook_endpoint` (`string`) optional
Webhook endpoint to which Spacelift sends the POST requests about run state changes **Default value:** `null`
`webhook_secret` (`string`) optional
Webhook secret used to sign each POST request so you're able to verify that the requests come from Spacelift **Default value:** `null`
`worker_pool_id` (`string`) optional
The immutable ID (slug) of the worker pool **Default value:** `null`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`id`
The stack id
`stack`
The created stack
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `spacelift`, version: `>= 0.1.31` ### Providers - `spacelift`, version: `>= 0.1.31` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`spacelift_aws_role.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/aws_role) (resource) - [`spacelift_context_attachment.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/context_attachment) (resource) - [`spacelift_drift_detection.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/drift_detection) (resource) - [`spacelift_environment_variable.component_env_vars`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) (resource) - [`spacelift_environment_variable.component_name`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) (resource) - [`spacelift_environment_variable.stack_name`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) (resource) - [`spacelift_mounted_file.stack_config`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/mounted_file) (resource) - [`spacelift_policy_attachment.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/policy_attachment) (resource) - [`spacelift_run.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/run) (resource) - [`spacelift_stack.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack) (resource) - [`spacelift_stack_dependency.default`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_dependency) (resource) - [`spacelift_stack_destructor.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_destructor) (resource) - [`spacelift_webhook.this`](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/webhook) (resource) ## Data Sources The following data sources are used by this module: ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 0.13.0 | | [spacelift](#requirement\_spacelift) | >= 0.1.31 | ## Providers | Name | Version | |------|---------| | [spacelift](#provider\_spacelift) | >= 0.1.31 | ## Modules | Name | Source | Version | |------|--------|---------| | [this](#module\_this) | cloudposse/label/null | 0.25.0 | ## Resources | Name | Type | |------|------| | [spacelift_aws_role.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/aws_role) | resource | | [spacelift_context_attachment.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/context_attachment) | resource | | [spacelift_drift_detection.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/drift_detection) | resource | | [spacelift_environment_variable.component_env_vars](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) | resource | | [spacelift_environment_variable.component_name](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) | resource | | [spacelift_environment_variable.stack_name](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/environment_variable) | resource | | [spacelift_mounted_file.stack_config](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/mounted_file) | resource | | [spacelift_policy_attachment.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/policy_attachment) | resource | | [spacelift_run.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/run) | resource | | [spacelift_stack.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack) | resource | | [spacelift_stack_dependency.default](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_dependency) | resource | | [spacelift_stack_destructor.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/stack_destructor) | resource | | [spacelift_webhook.this](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/resources/webhook) | resource | ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.This is for some rare cases where resources want additional configuration of tagsand therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no | | [administrative](#input\_administrative) | Whether this stack can manage other stacks | `bool` | `false` | no | | [after\_apply](#input\_after\_apply) | List of after-apply scripts | `list(string)` | `[]` | no | | [after\_destroy](#input\_after\_destroy) | List of after-destroy scripts | `list(string)` | `[]` | no | | [after\_init](#input\_after\_init) | List of after-init scripts | `list(string)` | `[]` | no | | [after\_perform](#input\_after\_perform) | List of after-perform scripts | `list(string)` | `[]` | no | | [after\_plan](#input\_after\_plan) | List of after-plan scripts | `list(string)` | `[]` | no | | [atmos\_stack\_name](#input\_atmos\_stack\_name) | The name of the atmos stack | `string` | n/a | yes | | [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,in the order they appear in the list. New attributes are appended to theend of the list. The elements of the list are joined by the `delimiter`and treated as a single ID element. | `list(string)` | `[]` | no | | [autodeploy](#input\_autodeploy) | Controls the Spacelift 'autodeploy' option for a stack | `bool` | `false` | no | | [autoretry](#input\_autoretry) | Controls the Spacelift 'autoretry' option for a stack | `bool` | `false` | no | | [aws\_role\_arn](#input\_aws\_role\_arn) | ARN of the AWS IAM role to assume and put its temporary credentials in the runtime environment | `string` | `null` | no | | [aws\_role\_enabled](#input\_aws\_role\_enabled) | Flag to enable/disable Spacelift to use AWS STS to assume the supplied IAM role and put its temporary credentials in the runtime environment | `bool` | `false` | no | | [aws\_role\_external\_id](#input\_aws\_role\_external\_id) | Custom external ID (works only for private workers). See https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html for more details | `string` | `null` | no | | [aws\_role\_generate\_credentials\_in\_worker](#input\_aws\_role\_generate\_credentials\_in\_worker) | Flag to enable/disable generating AWS credentials in the private worker after assuming the supplied IAM role | `bool` | `true` | no | | [azure\_devops](#input\_azure\_devops) | Azure DevOps VCS settings | `map(any)` | `null` | no | | [before\_apply](#input\_before\_apply) | List of before-apply scripts | `list(string)` | `[]` | no | | [before\_destroy](#input\_before\_destroy) | List of before-destroy scripts | `list(string)` | `[]` | no | | [before\_init](#input\_before\_init) | List of before-init scripts | `list(string)` | `[]` | no | | [before\_perform](#input\_before\_perform) | List of before-perform scripts | `list(string)` | `[]` | no | | [before\_plan](#input\_before\_plan) | List of before-plan scripts | `list(string)` | `[]` | no | | [bitbucket\_cloud](#input\_bitbucket\_cloud) | Bitbucket Cloud VCS settings | `map(any)` | `null` | no | | [bitbucket\_datacenter](#input\_bitbucket\_datacenter) | Bitbucket Datacenter VCS settings | `map(any)` | `null` | no | | [branch](#input\_branch) | Specify which branch to use within your infrastructure repo | `string` | `"main"` | no | | [cloudformation](#input\_cloudformation) | CloudFormation-specific configuration. Presence means this Stack is a CloudFormation Stack. | `map(any)` | `null` | no | | [commit\_sha](#input\_commit\_sha) | The commit SHA for which to trigger a run. Requires `var.spacelift_run_enabled` to be set to `true` | `string` | `null` | no | | [component\_env](#input\_component\_env) | Map of component ENV variables | `any` | `{}` | no | | [component\_name](#input\_component\_name) | The name of the concrete component (typically a directory name) | `string` | n/a | yes | | [component\_root](#input\_component\_root) | The path, relative to the root of the repository, where the component can be found | `string` | n/a | yes | | [component\_vars](#input\_component\_vars) | All Terraform values to be applied to the stack via a mounted file | `any` | `{}` | no | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "descriptor_formats": \{\},  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "labels_as_tags": [    "unset"  ],  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\},  "tenant": null\}
| no | | [context\_attachments](#input\_context\_attachments) | A list of context IDs to attach to this stack | `list(string)` | `[]` | no | | [delimiter](#input\_delimiter) | Delimiter to be used between ID elements.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [description](#input\_description) | Specify description of stack | `string` | `null` | no | | [descriptor\_formats](#input\_descriptor\_formats) | Describe additional descriptors to be output in the `descriptors` output map.Map of maps. Keys are names of descriptors. Values are maps of the form`{ format = string labels = list(string)}`(Type is `any` so the map values can later be enhanced to provide additional options.)`format` is a Terraform format string to be passed to the `format()` function.`labels` is a list of labels, in order, to pass to `format()` function.Label values will be normalized before being passed to `format()` so they will beidentical to how they appear in `id`.Default is `{}` (`descriptors` output will be empty). | `any` | `{}` | no | | [drift\_detection\_enabled](#input\_drift\_detection\_enabled) | Flag to enable/disable drift detection on the infrastructure stacks | `bool` | `false` | no | | [drift\_detection\_reconcile](#input\_drift\_detection\_reconcile) | Flag to enable/disable infrastructure stacks drift automatic reconciliation. If drift is detected and `reconcile` is turned on, Spacelift will create a tracked run to correct the drift | `bool` | `false` | no | | [drift\_detection\_schedule](#input\_drift\_detection\_schedule) | List of cron expressions to schedule drift detection for the infrastructure stacks | `list(string)` |
[  "0 4 * * *"]
| no | | [drift\_detection\_timezone](#input\_drift\_detection\_timezone) | Timezone in which the schedule is expressed. Defaults to UTC. | `string` | `"UTC"` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [github\_enterprise](#input\_github\_enterprise) | GitHub Enterprise (self-hosted) VCS settings | `map(any)` | `null` | no | | [gitlab](#input\_gitlab) | GitLab VCS settings | `map(any)` | `null` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for keep the existing setting, which defaults to `0`.Does not affect `id_full`. | `number` | `null` | no | | [label\_key\_case](#input\_label\_key\_case) | Controls the letter case of the `tags` keys (label names) for tags generated by this module.Does not affect keys of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The order in which the labels (ID elements) appear in the `id`.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | Controls the letter case of ID elements (labels) as included in `id`,set as tag values, and output by this module individually.Does not affect values of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper` and `none` (no transformation).Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs.Default value: `lower`. | `string` | `null` | no | | [labels](#input\_labels) | A list of labels for the stack | `list(string)` | `[]` | no | | [labels\_as\_tags](#input\_labels\_as\_tags) | Set of labels (ID elements) to include as tags in the `tags` output.Default is to include all labels.Tags with empty values will not be included in the `tags` output.Set to `[]` to suppress all generated tags.**Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. | `set(string)` |
[  "default"]
| no | | [local\_preview\_enabled](#input\_local\_preview\_enabled) | Indicates whether local preview runs can be triggered on this Stack | `bool` | `false` | no | | [manage\_state](#input\_manage\_state) | Flag to enable/disable manage\_state setting in stack | `bool` | `true` | no | | [name](#input\_name) | ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.This is the only ID element not also included as a `tag`.The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. | `string` | `null` | no | | [namespace](#input\_namespace) | ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique | `string` | `null` | no | | [policy\_ids](#input\_policy\_ids) | List of Rego policy IDs to attach to this stack | `list(string)` | `[]` | no | | [protect\_from\_deletion](#input\_protect\_from\_deletion) | Flag to enable/disable deletion protection. | `bool` | `false` | no | | [pulumi](#input\_pulumi) | Pulumi-specific configuration. Presence means this Stack is a Pulumi Stack. | `map(any)` | `null` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.Characters matching the regex will be removed from the ID elements.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [repository](#input\_repository) | The name of your infrastructure repo | `string` | n/a | yes | | [runner\_image](#input\_runner\_image) | The full image name and tag of the Docker image to use in Spacelift | `string` | `null` | no | | [showcase](#input\_showcase) | Showcase settings | `map(any)` | `null` | no | | [space\_id](#input\_space\_id) | Place the stack in the specified space\_id. | `string` | `"root"` | no | | [spacelift\_run\_enabled](#input\_spacelift\_run\_enabled) | Enable/disable creation of the `spacelift_run` resource | `bool` | `false` | no | | [spacelift\_stack\_dependency\_enabled](#input\_spacelift\_stack\_dependency\_enabled) | If enabled, the `spacelift_stack_dependency` Spacelift resource will be used to create dependencies between stacks instead of using the `depends-on` labels. The `depends-on` labels will be removed from the stacks and the trigger policies for dependencies will be detached | `bool` | `false` | no | | [stack\_destructor\_enabled](#input\_stack\_destructor\_enabled) | Flag to enable/disable the stack destructor to destroy the resources of the stack before deleting the stack itself | `bool` | `false` | no | | [stack\_name](#input\_stack\_name) | The name of the Spacelift stack | `string` | n/a | yes | | [stage](#input\_stage) | ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input\_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | | [terraform\_smart\_sanitization](#input\_terraform\_smart\_sanitization) | Whether or not to enable [Smart Sanitization](https://docs.spacelift.io/vendors/terraform/resource-sanitization) which will only sanitize values marked as sensitive. | `bool` | `false` | no | | [terraform\_version](#input\_terraform\_version) | Specify the version of Terraform to use for the stack | `string` | `null` | no | | [terraform\_workflow\_tool](#input\_terraform\_workflow\_tool) | Defines the tool that will be used to execute the workflow. This can be one of OPEN\_TOFU, TERRAFORM\_FOSS or CUSTOM. Defaults to TERRAFORM\_FOSS. | `string` | `"TERRAFORM_FOSS"` | no | | [terraform\_workspace](#input\_terraform\_workspace) | Specify the Terraform workspace to use for the stack | `string` | `null` | no | | [webhook\_enabled](#input\_webhook\_enabled) | Flag to enable/disable the webhook endpoint to which Spacelift sends the POST requests about run state changes | `bool` | `false` | no | | [webhook\_endpoint](#input\_webhook\_endpoint) | Webhook endpoint to which Spacelift sends the POST requests about run state changes | `string` | `null` | no | | [webhook\_secret](#input\_webhook\_secret) | Webhook secret used to sign each POST request so you're able to verify that the requests come from Spacelift | `string` | `null` | no | | [worker\_pool\_id](#input\_worker\_pool\_id) | The immutable ID (slug) of the worker pool | `string` | `null` | no | ## Outputs | Name | Description | |------|-------------| | [id](#output\_id) | The stack id | | [stack](#output\_stack) | The created stack | --- ## spacelift-stacks-from-atmos-config # Module: `spacelift-stacks-from-atmos-config` Terraform module to extract the [Spacelift Stack](https://docs.spacelift.io/concepts/stack/) configuration from atmos config. In addition, the results can be filtered by various criteria, such as tenant, environment, stack labels, etc. ## Usage Here's how to invoke this module in your project: ```hcl provider "spacelift" {} module "spacelift_stacks" { source = "cloudposse/cloud-infrastructure-automation/spacelift//modules/spacelift-stacks-from-atmos-config" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" space_name = "test" description = "A space for our test infrasturcture" parent_space_id = "root" inherit_entities_from_parent = false labels = ["test", "space"] ``` ## Examples Here is an example of using this module: - [`../../examples/spacelift-config-from-atmos-config`](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/master/examples/spacelift-config-from-atmos-config) - complete example of using this module ## Variables ### Required Variables
`context_filters` required
Context filters to output stacks matching specific criteria. **Type:** ```hcl object({ namespaces = optional(list(string), []) environments = optional(list(string), []) tenants = optional(list(string), []) stages = optional(list(string), []) tags = optional(map(string), {}) administrative = optional(bool) root_administrative = optional(bool) }) ```
### Optional Variables
`component_deps_processing_enabled` (`bool`) optional
Boolean flag to enable/disable processing stack config dependencies for the components in the provided stack **Default value:** `true`
`excluded_context_filters` optional
Context filters to exclude from stacks matching specific criteria of `var.context_filters`. **Type:** ```hcl object({ namespaces = optional(list(string), []) environments = optional(list(string), []) tenants = optional(list(string), []) stages = optional(list(string), []) tags = optional(map(string), {}) }) ``` **Default value:** `{ }`
`imports_processing_enabled` (`bool`) optional
Enable/disable processing stack imports **Default value:** `false`
`stack_config_path_template` (`string`) optional
Stack config path template **Default value:** `"stacks/%s.yaml"`
`stack_deps_processing_enabled` (`bool`) optional
Boolean flag to enable/disable processing all stack dependencies in the provided stack **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`spacelift_stacks`
Generated stacks
`stacks`
Generated stacks
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `spacelift_config` | 1.8.0 | [`cloudposse/stack-config/yaml//modules/spacelift`](https://registry.terraform.io/modules/cloudposse/stack-config/yaml/modules/spacelift/1.8.0) | Convert infrastructure stacks from YAML configs into Spacelift stacks `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Requirements | Name | Version | |------|---------| | [terraform](#requirement\_terraform) | >= 0.13.0 | ## Providers No providers. ## Modules | Name | Source | Version | |------|--------|---------| | [spacelift\_config](#module\_spacelift\_config) | cloudposse/stack-config/yaml//modules/spacelift | 1.5.0 | | [this](#module\_this) | cloudposse/label/null | 0.25.0 | ## Resources No resources. ## Inputs | Name | Description | Type | Default | Required | |------|-------------|------|---------|:--------:| | [additional\_tag\_map](#input\_additional\_tag\_map) | Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`.This is for some rare cases where resources want additional configuration of tagsand therefore take a list of maps with tag key, value, and additional configuration. | `map(string)` | `{}` | no | | [attributes](#input\_attributes) | ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`,in the order they appear in the list. New attributes are appended to theend of the list. The elements of the list are joined by the `delimiter`and treated as a single ID element. | `list(string)` | `[]` | no | | [component\_deps\_processing\_enabled](#input\_component\_deps\_processing\_enabled) | Boolean flag to enable/disable processing stack config dependencies for the components in the provided stack | `bool` | `true` | no | | [context](#input\_context) | Single object for setting entire context at once.See description of individual variables for details.Leave string and numeric variables as `null` to use default value.Individual variable settings (non-null) override settings in context object,except for attributes, tags, and additional\_tag\_map, which are merged. | `any` |
\{  "additional_tag_map": \{\},  "attributes": [],  "delimiter": null,  "descriptor_formats": \{\},  "enabled": true,  "environment": null,  "id_length_limit": null,  "label_key_case": null,  "label_order": [],  "label_value_case": null,  "labels_as_tags": [    "unset"  ],  "name": null,  "namespace": null,  "regex_replace_chars": null,  "stage": null,  "tags": \{\},  "tenant": null\}
| no | | [context\_filters](#input\_context\_filters) | Context filters to output stacks matching specific criteria. |
object(\{    namespaces          = optional(list(string), [])    environments        = optional(list(string), [])    tenants             = optional(list(string), [])    stages              = optional(list(string), [])    tags                = optional(map(string), \{\})    administrative      = optional(bool)    root_administrative = optional(bool)  \})
| n/a | yes | | [delimiter](#input\_delimiter) | Delimiter to be used between ID elements.Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. | `string` | `null` | no | | [descriptor\_formats](#input\_descriptor\_formats) | Describe additional descriptors to be output in the `descriptors` output map.Map of maps. Keys are names of descriptors. Values are maps of the form`{ format = string labels = list(string)}`(Type is `any` so the map values can later be enhanced to provide additional options.)`format` is a Terraform format string to be passed to the `format()` function.`labels` is a list of labels, in order, to pass to `format()` function.Label values will be normalized before being passed to `format()` so they will beidentical to how they appear in `id`.Default is `{}` (`descriptors` output will be empty). | `any` | `{}` | no | | [enabled](#input\_enabled) | Set to false to prevent the module from creating any resources | `bool` | `null` | no | | [environment](#input\_environment) | ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' | `string` | `null` | no | | [excluded\_context\_filters](#input\_excluded\_context\_filters) | Context filters to exclude from stacks matching specific criteria of `var.context_filters`. |
object(\{    namespaces   = optional(list(string), [])    environments = optional(list(string), [])    tenants      = optional(list(string), [])    stages       = optional(list(string), [])    tags         = optional(map(string), \{\})  \})
| `{}` | no | | [id\_length\_limit](#input\_id\_length\_limit) | Limit `id` to this many characters (minimum 6).Set to `0` for unlimited length.Set to `null` for keep the existing setting, which defaults to `0`.Does not affect `id_full`. | `number` | `null` | no | | [imports\_processing\_enabled](#input\_imports\_processing\_enabled) | Enable/disable processing stack imports | `bool` | `false` | no | | [label\_key\_case](#input\_label\_key\_case) | Controls the letter case of the `tags` keys (label names) for tags generated by this module.Does not affect keys of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper`.Default value: `title`. | `string` | `null` | no | | [label\_order](#input\_label\_order) | The order in which the labels (ID elements) appear in the `id`.Defaults to ["namespace", "environment", "stage", "name", "attributes"].You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. | `list(string)` | `null` | no | | [label\_value\_case](#input\_label\_value\_case) | Controls the letter case of ID elements (labels) as included in `id`,set as tag values, and output by this module individually.Does not affect values of tags passed in via the `tags` input.Possible values: `lower`, `title`, `upper` and `none` (no transformation).Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs.Default value: `lower`. | `string` | `null` | no | | [labels\_as\_tags](#input\_labels\_as\_tags) | Set of labels (ID elements) to include as tags in the `tags` output.Default is to include all labels.Tags with empty values will not be included in the `tags` output.Set to `[]` to suppress all generated tags.**Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. | `set(string)` |
[  "default"]
| no | | [name](#input\_name) | ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'.This is the only ID element not also included as a `tag`.The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. | `string` | `null` | no | | [namespace](#input\_namespace) | ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique | `string` | `null` | no | | [regex\_replace\_chars](#input\_regex\_replace\_chars) | Terraform regular expression (regex) string.Characters matching the regex will be removed from the ID elements.If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. | `string` | `null` | no | | [stack\_config\_path\_template](#input\_stack\_config\_path\_template) | Stack config path template | `string` | `"stacks/%s.yaml"` | no | | [stack\_deps\_processing\_enabled](#input\_stack\_deps\_processing\_enabled) | Boolean flag to enable/disable processing all stack dependencies in the provided stack | `bool` | `false` | no | | [stage](#input\_stage) | ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' | `string` | `null` | no | | [tags](#input\_tags) | Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`).Neither the tag keys nor the tag values will be modified by this module. | `map(string)` | `{}` | no | | [tenant](#input\_tenant) | ID element \_(Rarely used, not included by default)\_. A customer identifier, indicating who this instance of a resource is for | `string` | `null` | no | ## Outputs | Name | Description | |------|-------------| | [spacelift\_stacks](#output\_spacelift\_stacks) | Generated stacks | | [stacks](#output\_stacks) | Generated stacks | --- ## Spacelift(Spacelift) import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Explore modules designed to manage Spacelift configurations and workflows with Terraform. These modules enhance your continuous integration and delivery pipelines. --- ## label(Label) # Module: `label` ## Deprecated This module was an experimental fork and is now obsolete and will not be maintained. Any projects using `terraform-terraform-label` are encouraged to switch to using [terraform-null-label](https://github.com/cloudposse/terraform-null-label), which is actively maintained and used by all current Cloud Posse Terraform modules. This module was a fork of [terraform-null-label](https://github.com/cloudposse/terraform-null-label), made at a time when that project was using the Terraform `null` provider (hence the "null" in the name), in order to remove the `null` provider dependency. This was accomplished by removing outputs that required the `null` provider. With the features that became available in Terraform 0.12, the `terraform-null-label` project was able to retain all of its features and also [remove the `null` provider](https://github.com/cloudposse/terraform-null-label/commit/d6d24b80d687e76e029f39f444d0163b42c5d5e0), removing any incentive to further develop `terraform-terraform-label`. With the key distinguishing feature of `terraform-terraform-label` no longer being a distinguishing feature, this module was no longer necessary, and all focus returned to maintaining and enhancing `terraform-null-label`, which now far surpasses this module in functionality. ### Historical Description Terraform module designed to generate consistent label names and tags for resources. Use `terraform-terraform-label` to implement a strict naming convention. #### `terraform-terraform-label` is a fork of [terraform-null-label](https://github.com/cloudposse/terraform-null-label) which uses only the core Terraform provider. A label follows the following convention: `{namespace}-{stage}-{name}-{attributes}`. The delimiter (e.g. `-`) is interchangeable. It's recommended to use one `terraform-terraform-label` module for every unique resource of a given resource type. For example, if you have 10 instances, there should be 10 different labels. However, if you have multiple different kinds of resources (e.g. instances, security groups, file systems, and elastic IPs), then they can all share the same label assuming they are logically related. All [Cloud Posse modules](https://github.com/cloudposse?utf8=%E2%9C%93&q=terraform-&type=&language=) use the related [terraform-null-label](https://github.com/cloudposse/terraform-null-label) module to ensure resources can be instantiated multiple times within an account and without conflict. **NOTE:** The second `terraform` word in `terraform-terraform-label` refers to the primary Terraform provider used in this module. ## Usage ### Simple Example Include this repository as a module in your existing terraform code: ```hcl module "eg_prod_bastion_label" { source = "cloudposse/label/terraform" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" attributes = ["public"] delimiter = "-" tags = { "BusinessUnit" = "XYZ", "Snapshot" = "true" } } ``` This will create an `id` with the value of `eg-prod-bastion-public`. Now reference the label when creating an instance (for example): ```hcl resource "aws_instance" "eg_prod_bastion_public" { instance_type = "t1.micro" tags = module.eg_prod_bastion_label.tags } ``` Or define a security group: ```hcl resource "aws_security_group" "eg_prod_bastion_public" { vpc_id = var.vpc_id name = module.eg_prod_bastion_label.id tags = module.eg_prod_bastion_label.tags egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] } } ``` ### Advanced Example Here is a more complex example with two instances using two different labels. Note how efficiently the tags are defined for both the instance and the security group. ```hcl module "eg_prod_bastion_abc_label" { source = "cloudposse/label/terraform" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" attributes = ["abc"] delimiter = "-" tags = { "BusinessUnit" = "ABC" } } resource "aws_security_group" "eg_prod_bastion_abc" { name = module.eg_prod_bastion_abc_label.id tags = module.eg_prod_bastion_abc_label.tags ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } } resource "aws_instance" "eg_prod_bastion_abc" { instance_type = "t1.micro" tags = module.eg_prod_bastion_abc_label.tags vpc_security_group_ids = [aws_security_group.eg_prod_bastion_abc.id] } module "eg_prod_bastion_xyz_label" { source = "cloudposse/label/terraform" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "prod" name = "bastion" attributes = ["xyz"] delimiter = "-" tags = { "BusinessUnit" = "XYZ" } } resource "aws_security_group" "eg_prod_bastion_xyz" { name = module.eg_prod_bastion_xyz_label.id tags = module.eg_prod_bastion_xyz_label.tags ingress { from_port = 22 to_port = 22 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] } } resource "aws_instance" "eg_prod_bastion_xyz" { instance_type = "t1.micro" tags = module.eg_prod_bastion_xyz_label.tags vpc_security_group_ids = [aws_security_group.eg_prod_bastion_xyz.id] } ``` ## Variables ### Required Variables
### Optional Variables
`convert_case` (`bool`) optional
Convert fields to lower case **Default value:** `true`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`attributes` (`list(string)`) optional
Additional attributes (e.g. `1`) **Required:** No **Default value:** `[ ]`
`delimiter` (`string`) optional
Delimiter to be used between `namespace`, `stage`, `name` and `attributes` **Required:** No **Default value:** `"-"`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `true`
`name` (`string`) optional
Solution name, e.g. `app` or `jenkins` **Required:** No **Default value:** `""`
`namespace` (`string`) optional
Namespace, which could be your organization name or abbreviation, e.g. 'eg' or 'cp' **Required:** No **Default value:** `""`
`stage` (`string`) optional
Stage, e.g. 'prod', 'staging', 'dev' **Required:** No **Default value:** `""`
`tags` (`map(string)`) optional
Additional tags (e.g. `map('BusinessUnit','XYZ')`) **Required:** No **Default value:** `{ }`
## Outputs
`attributes`
Normalized attributes
`delimiter`
Delimiter between `namespace`, `stage`, `name` and `attributes`
`id`
Disambiguated ID
`name`
Normalized name
`namespace`
Normalized namespace
`stage`
Normalized stage
`tags`
Normalized Tag map
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` --- ## Terraform import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Access core Terraform modules that provide foundational resources and configurations. These modules form the basis of infrastructure management using Terraform. --- ## ssh-key-pair # Module: `ssh-key-pair` Terraform module for generating an SSH public key file. ## Usage ```hcl module "ssh_key_pair" { source = "cloudposse/ssh-key-pair/tls" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" namespace = "eg" stage = "test" name = "app" ssh_public_key_path = "/secrets" private_key_extension = ".pem" public_key_extension = ".pub" chmod_command = "chmod 600 %v" } ``` ## Variables ### Required Variables
`ssh_public_key_path` (`string`) required
Path to SSH public key directory (e.g. `/secrets`)
### Optional Variables
`chmod_command` (`string`) optional
Template of the command executed on the private key file **Default value:** `"chmod 600 %v"`
`private_key_extension` (`string`) optional
Private key extension **Default value:** `""`
`private_key_output_enabled` (`bool`) optional
Add the private key as a terraform output private_key **Default value:** `false`
`public_key_extension` (`string`) optional
Public key extension **Default value:** `".pub"`
`ssh_key_algorithm` (`string`) optional
SSH key algorithm **Default value:** `"RSA"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`key_name`
Name of SSH key
`private_key`
Content of the generated private key
`public_key`
Content of the generated public key
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `local`, version: `>= 1.3` - `null`, version: `>= 2.1` - `tls`, version: `>= 2.0` ### Providers - `local`, version: `>= 1.3` - `null`, version: `>= 2.1` - `tls`, version: `>= 2.0` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: - [`local_file.private_key_pem`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) - [`local_file.public_key_openssh`](https://registry.terraform.io/providers/hashicorp/local/latest/docs/resources/file) (resource) - [`null_resource.chmod`](https://registry.terraform.io/providers/hashicorp/null/latest/docs/resources/resource) (resource) - [`tls_private_key.default`](https://registry.terraform.io/providers/hashicorp/tls/latest/docs/resources/private_key) (resource) ## Data Sources The following data sources are used by this module: --- ## TLS import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Use our Terraform modules to manage TLS certificates and encryption settings. These modules ensure secure communication within your infrastructure. --- ## config(3) # Module: `config` Terraform module to convert local and remote YAML configuration templates into Terraform lists and maps. ## Introduction The module accepts paths to local and remote YAML configuration template files and converts the templates into Terraform lists and maps for consumption in other Terraform modules. The module can accept a map of parameters for interpolation within the YAML config templates. The module also supports a top-level `import` attribute in map configuration templates, which will include the file and perform a deep merge. Up to 10 levels of imports hierarchy are supported, and all imported maps are deep merged into a final configuration map. For example, if you have a config file like this (e.g. `myconfig.yaml`): ```yaml import: - file1 - file2 ``` Then, this module will deep merge `file1.yaml` and `file2.yaml` into `myconfig.yaml`. __Note:__ Do not include the extensions (e.g. `.yaml`) in the imports. ### Attributions Big thanks to [Imperative Systems Inc.](https://github.com/Imperative-Systems-Inc) for the excellent [deepmerge](https://github.com/Imperative-Systems-Inc/terraform-modules/tree/master/deepmerge) Terraform module to perform a deep map merge of standard Terraform maps and objects. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-yaml-config/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest) (which tests and deploys the example on Datadog), see [test](https://github.com/cloudposse/terraform-yaml-config/tree/main/test). For an example of using local config maps with `import` and deep merging into a final configuration map, see [examples/imports-local](https://github.com/cloudposse/terraform-yaml-config/tree/main/examples/imports-local). For an example of using remote config maps with `import` and deep merging into a final configuration map, see [examples/imports-remote](https://github.com/cloudposse/terraform-yaml-config/tree/main/examples/imports-remote). ## Examples ### Example of local and remote maps and lists configurations with interpolation parameters ```hcl module "yaml_config" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" map_config_local_base_path = "./config" map_config_paths = [ "map-configs/*.yaml", "https://raw.githubusercontent.com/cloudposse/terraform-opsgenie-incident-management/master/examples/config/resources/services.yaml", "https://raw.githubusercontent.com/cloudposse/terraform-opsgenie-incident-management/master/examples/config/resources/team_routing_rules.yaml" ] list_config_local_base_path = "./config" list_config_paths = [ "list-configs/*.yaml", "https://raw.githubusercontent.com/cloudposse/terraform-aws-service-control-policies/master/examples/complete/policies/organization-policies.yaml" ] parameters = { param1 = "1" param2 = "2" } context = module.this.context } ``` ### Example of local maps configurations with `import` and deep merging In the example, we use two levels of imports, and the module deep merges the local config files `imports-level-3.yaml`, `imports-level-2.yaml`, and `imports-level-1.yaml` into a final config map. See [examples/imports-local](https://github.com/cloudposse/terraform-yaml-config/tree/main/examples/imports-local) for more details. ```hcl module "yaml_config" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" map_config_local_base_path = "./config" map_config_paths = [ "imports-level-1.yaml" ] context = module.this.context } ``` ### Example of remote maps configurations with with `import` and deep merging In the example, we use two levels of imports, and the module deep merges the remote config files `globals.yaml`, `ue2-globals.yaml`, and `ue2-prod.yaml` into a final config map. See [examples/imports-remote](https://github.com/cloudposse/terraform-yaml-config/tree/main/examples/imports-remote) for more details. ```hcl module "yaml_config" { source = "cloudposse/config/yaml" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" map_config_remote_base_path = "https://raw.githubusercontent.com/cloudposse/atmos/master/example/stacks" map_config_paths = [ "https://raw.githubusercontent.com/cloudposse/atmos/master/example/stacks/ue2-prod.yaml" ] context = module.this.context } ``` ## Variables ### Required Variables
### Optional Variables
`append_list_enabled` (`bool`) optional
A boolean flag to enable/disable appending lists instead of overwriting them. **Default value:** `false`
`deep_copy_list_enabled` (`bool`) optional
A boolean flag to enable/disable merging of list elements one by one. **Default value:** `false`
`list_config_local_base_path` (`string`) optional
Base path to local YAML configuration files of list type **Default value:** `""`
`list_config_paths` (`list(string)`) optional
Paths to YAML configuration files of list type **Default value:** `[ ]`
`list_config_remote_base_path` (`string`) optional
Base path to remote YAML configuration files of list type **Default value:** `""`
`map_config_local_base_path` (`string`) optional
Base path to local YAML configuration files of map type **Default value:** `""`
`map_config_paths` (`list(string)`) optional
Paths to YAML configuration files of map type **Default value:** `[ ]`
`map_config_remote_base_path` (`string`) optional
Base path to remote YAML configuration files of map type **Default value:** `""`
`map_configs` (`any`) optional
List of existing configurations of map type. Deep-merging of the existing map configs takes precedence over the map configs loaded from YAML files **Default value:** `[ ]`
`parameters` (`map(string)`) optional
Map of parameters for interpolation within the YAML config templates **Default value:** `{ }`
`remote_config_selector` (`string`) optional
String to detect local vs. remote config paths **Default value:** `"://"`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`all_imports_list`
List of all imported YAML configurations
`all_imports_map`
Map of all imported YAML configurations
`list_configs`
Terraform lists from YAML configurations
`map_configs`
Terraform maps from YAML configurations
## Dependencies ### Requirements - `terraform`, version: `>= 0.13.0` - `http`, version: `>= 2.0` - `local`, version: `>= 1.3` - `template`, version: `>= 2.2` ### Modules Name | Version | Source | Description --- | --- | --- | --- `maps_deepmerge` | latest | [`./modules/deepmerge`](https://registry.terraform.io/modules/./modules/deepmerge/) | n/a `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a `yaml_config_1` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_10` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_2` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_3` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_4` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_5` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_6` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_7` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_8` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a `yaml_config_9` | latest | [`./modules/yaml-config`](https://registry.terraform.io/modules/./modules/yaml-config/) | n/a --- ## stack-config # Module: `stack-config` Terraform module that loads and processes an opinionated ["stack" configuration](#examples) from YAML sources using the [`terraform-provider-utils`](https://github.com/cloudposse/terraform-provider-utils) Terraform provider. It supports deep-merged variables, settings, ENV variables, backend config, remote state, and [Spacelift](https://spacelift.io/) stacks config outputs for Terraform and helmfile components. ## Introduction The module is composed of the following sub-modules: - [vars](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/vars) - accepts stack configuration and returns deep-merged variables for a Terraform or helmfile component. - [settings](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/settings) - accepts stack configuration and returns deep-merged settings for a Terraform or helmfile component. - [env](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/env) - accepts stack configuration and returns deep-merged ENV variables for a Terraform or helmfile component. - [backend](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/backend) - accepts stack configuration and returns backend config for a Terraform component. - [remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state) - accepts stack configuration and returns remote state outputs for a Terraform component. The module supports `s3` and `remote` (Terraform Cloud) backends. - [spacelift](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/spacelift) - accepts infrastructure stack configuration and transforms it into Spacelift stacks. ## Usage For a complete example, see [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/examples/complete). For automated tests of the complete example using [bats](https://github.com/bats-core/bats-core) and [Terratest](https://github.com/gruntwork-io/terratest), see [test](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/test). For an example on how to configure remote state for Terraform components in YAML config files and then read the components outputs from the remote state, see [examples/remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/examples/remote-state). For an example on how to process `vars`, `settings`, `env` and `backend` configurations for all Terraform and helmfile components for a list of stacks, see [examples/stacks](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/examples/stacks). ## Examples Here's an example of a stack configuration file: ```yaml import: - ue2-globals vars: stage: dev terraform: vars: {} helmfile: vars: {} components: terraform: vpc: backend: s3: workspace_key_prefix: "vpc" vars: cidr_block: "10.102.0.0/18" eks: backend: s3: workspace_key_prefix: "eks" vars: {} helmfile: nginx-ingress: vars: installed: true ``` The `import` section refers to other stack configurations that are automatically deep merged. ### Complete example This example loads the stack config `my-stack` (which in turn imports other YAML config dependencies) and returns variables and backend config for the Terraform component `my-vpc` from the stack `my-stack`. ```hcl module "vars" { source = "cloudposse/stack-config/yaml//modules/vars" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } module "backend" { source = "cloudposse/stack-config/yaml//modules/backend" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } module "settings" { source = "cloudposse/stack-config/yaml//modules/settings" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } module "env" { source = "cloudposse/stack-config/yaml//modules/env" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } ``` The example returns the following deep-merged `vars`, `settings`, `env`, and `backend` configurations for the `my-vpc` Terraform component: ```hcl backend = { "acl" = "bucket-owner-full-control" "bucket" = "eg-ue2-root-tfstate" "dynamodb_table" = "eg-ue2-root-tfstate-lock" "encrypt" = true "key" = "terraform.tfstate" "region" = "us-east-2" "role_arn" = "arn:aws:iam::xxxxxxxxxxxx:role/eg-gbl-root-terraform" "workspace_key_prefix" = "vpc" } backend_type = "s3" base_component = "vpc" env = { "ENV_TEST_1" = "test1_override" "ENV_TEST_2" = "test2_override" "ENV_TEST_3" = "test3" "ENV_TEST_4" = "test4" } settings = { "spacelift" = { "autodeploy" = true "branch" = "test" "triggers" = [ "1", "2", ] "workspace_enabled" = true } "version" = 1 } vars = { "availability_zones" = [ "us-east-2a", "us-east-2b", "us-east-2c", ] "cidr_block" = "10.132.0.0/18" "environment" = "ue2" "level" = 3 "namespace" = "eg" "param" = "param4" "region" = "us-east-2" "stage" = "prod" "subnet_type_tag_key" = "example/subnet/type" "test_map" = { "a" = "a_override_2" "b" = "b_override" "c" = [ 1, 2, 3, ] "map2" = { "atr1" = 1 "atr2" = 2 "atr3" = [ "3a", "3b", "3c", ] } } "var_1" = "1_override" "var_2" = "2_override" "var_3" = "3a" } ``` See [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/examples/complete) for more details. ### Remote state example This example accepts a stack config `my-stack` (which in turn imports other YAML config dependencies) and returns remote state outputs from the `s3` backend for `my-vpc` and `eks` Terraform components. __NOTE:__ The backend type (`s3`) and backend configuration for the components are defined in the stack YAML config files. ```hcl module "remote_state_my_vpc" { source = "cloudposse/stack-config/yaml//modules/remote-state" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component = "my-vpc" } module "remote_state_eks" { source = "cloudposse/stack-config/yaml//modules/remote-state" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component = "eks" } ``` See [examples/remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/examples/remote-state) for more details. ## Variables ### Required Variables
`stack_config_local_path` (`string`) required
Path to local stack configs
`stacks` (`list(string)`) required
A list of infrastructure stack names
### Optional Variables
`component_deps_processing_enabled` (`bool`) optional
Boolean flag to enable/disable processing stack config dependencies for the components in the provided stack **Default value:** `false`
`stack_deps_processing_enabled` (`bool`) optional
Boolean flag to enable/disable processing all stack dependencies in the provided stack **Default value:** `false`
### Context Variables
The following variables are defined in the `context.tf` file of this module and part of the [terraform-null-label](https://registry.terraform.io/modules/cloudposse/label/null) pattern.
`additional_tag_map` (`map(string)`) optional
Additional key-value pairs to add to each map in `tags_as_list_of_maps`. Not added to `tags` or `id`. This is for some rare cases where resources want additional configuration of tags and therefore take a list of maps with tag key, value, and additional configuration. **Required:** No **Default value:** `{ }`
`attributes` (`list(string)`) optional
ID element. Additional attributes (e.g. `workers` or `cluster`) to add to `id`, in the order they appear in the list. New attributes are appended to the end of the list. The elements of the list are joined by the `delimiter` and treated as a single ID element. **Required:** No **Default value:** `[ ]`
`context` (`any`) optional
Single object for setting entire context at once. See description of individual variables for details. Leave string and numeric variables as `null` to use default value. Individual variable settings (non-null) override settings in context object, except for attributes, tags, and additional_tag_map, which are merged. **Required:** No **Default value:** ```hcl { "additional_tag_map": {}, "attributes": [], "delimiter": null, "descriptor_formats": {}, "enabled": true, "environment": null, "id_length_limit": null, "label_key_case": null, "label_order": [], "label_value_case": null, "labels_as_tags": [ "unset" ], "name": null, "namespace": null, "regex_replace_chars": null, "stage": null, "tags": {}, "tenant": null } ```
`delimiter` (`string`) optional
Delimiter to be used between ID elements. Defaults to `-` (hyphen). Set to `""` to use no delimiter at all. **Required:** No **Default value:** `null`
`descriptor_formats` (`any`) optional
Describe additional descriptors to be output in the `descriptors` output map. Map of maps. Keys are names of descriptors. Values are maps of the form `\{ format = string labels = list(string) \}` (Type is `any` so the map values can later be enhanced to provide additional options.) `format` is a Terraform format string to be passed to the `format()` function. `labels` is a list of labels, in order, to pass to `format()` function. Label values will be normalized before being passed to `format()` so they will be identical to how they appear in `id`. Default is `{}` (`descriptors` output will be empty). **Required:** No **Default value:** `{ }`
`enabled` (`bool`) optional
Set to false to prevent the module from creating any resources **Required:** No **Default value:** `null`
`environment` (`string`) optional
ID element. Usually used for region e.g. 'uw2', 'us-west-2', OR role 'prod', 'staging', 'dev', 'UAT' **Required:** No **Default value:** `null`
`id_length_limit` (`number`) optional
Limit `id` to this many characters (minimum 6). Set to `0` for unlimited length. Set to `null` for keep the existing setting, which defaults to `0`. Does not affect `id_full`. **Required:** No **Default value:** `null`
`label_key_case` (`string`) optional
Controls the letter case of the `tags` keys (label names) for tags generated by this module. Does not affect keys of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper`. Default value: `title`. **Required:** No **Default value:** `null`
`label_order` (`list(string)`) optional
The order in which the labels (ID elements) appear in the `id`. Defaults to ["namespace", "environment", "stage", "name", "attributes"]. You can omit any of the 6 labels ("tenant" is the 6th), but at least one must be present. **Required:** No **Default value:** `null`
`label_value_case` (`string`) optional
Controls the letter case of ID elements (labels) as included in `id`, set as tag values, and output by this module individually. Does not affect values of tags passed in via the `tags` input. Possible values: `lower`, `title`, `upper` and `none` (no transformation). Set this to `title` and set `delimiter` to `""` to yield Pascal Case IDs. Default value: `lower`. **Required:** No **Default value:** `null`
`labels_as_tags` (`set(string)`) optional
Set of labels (ID elements) to include as tags in the `tags` output. Default is to include all labels. Tags with empty values will not be included in the `tags` output. Set to `[]` to suppress all generated tags. **Notes:** The value of the `name` tag, if included, will be the `id`, not the `name`. Unlike other `null-label` inputs, the initial setting of `labels_as_tags` cannot be changed in later chained modules. Attempts to change it will be silently ignored. **Required:** No **Default value:** ```hcl [ "default" ] ```
`name` (`string`) optional
ID element. Usually the component or solution name, e.g. 'app' or 'jenkins'. This is the only ID element not also included as a `tag`. The "name" tag is set to the full `id` string. There is no tag with the value of the `name` input. **Required:** No **Default value:** `null`
`namespace` (`string`) optional
ID element. Usually an abbreviation of your organization name, e.g. 'eg' or 'cp', to help ensure generated IDs are globally unique **Required:** No **Default value:** `null`
`regex_replace_chars` (`string`) optional
Terraform regular expression (regex) string. Characters matching the regex will be removed from the ID elements. If not set, `"/[^a-zA-Z0-9-]/"` is used to remove all characters other than hyphens, letters and digits. **Required:** No **Default value:** `null`
`stage` (`string`) optional
ID element. Usually used to indicate role, e.g. 'prod', 'staging', 'source', 'build', 'test', 'deploy', 'release' **Required:** No **Default value:** `null`
`tags` (`map(string)`) optional
Additional tags (e.g. `{'BusinessUnit': 'XYZ'}`). Neither the tag keys nor the tag values will be modified by this module. **Required:** No **Default value:** `{ }`
`tenant` (`string`) optional
ID element _(Rarely used, not included by default)_. A customer identifier, indicating who this instance of a resource is for **Required:** No **Default value:** `null`
## Outputs
`config`
Stack configurations
## Dependencies ### Requirements - `terraform`, version: `>= 0.14.0` - `external`, version: `>= 2.0` - `local`, version: `>= 1.3` - `utils`, version: `>= 1.7.1` ### Providers - `utils`, version: `>= 1.7.1` ### Modules Name | Version | Source | Description --- | --- | --- | --- `this` | 0.25.0 | [`cloudposse/label/null`](https://registry.terraform.io/modules/cloudposse/label/null/0.25.0) | n/a ## Resources The following resources are used by this module: ## Data Sources The following data sources are used by this module: - [`utils_stack_config_yaml.config`](https://registry.terraform.io/providers/cloudposse/utils/latest/docs/data-sources/stack_config_yaml) (data source) --- ## backend Terraform module that accepts stack configuration and returns backend config for a component. ## Usage The following example loads the stack config `my-stack` (which in turn imports other YAML config dependencies) and returns the backend config for the component `my-vpc`. ```hcl module "backend" { source = "cloudposse/stack-config/yaml//modules/backend" # version = "x.x.x" stack = "my-stack" component = "my-vpc" context = module.this.context } ``` The example returns the following `backend` configuration: ```hcl backend_type = s3 backend = { "acl" = "bucket-owner-full-control" "bucket" = "eg-ue2-root-tfstate" "dynamodb_table" = "eg-ue2-root-tfstate-lock" "encrypt" = true "key" = "terraform.tfstate" "region" = "us-east-2" "role_arn" = "arn:aws:iam::xxxxxxxxxxxx:role/eg-gbl-root-terraform" "workspace_key_prefix" = "vpc" } ``` See [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/backend/../../examples/complete) for more details. --- ## env Terraform module that accepts stack configuration and returns deep-merged ENV variables for a Terraform or helmfile component. ## Usage The following example loads the stack config `my-stack` (which in turn imports other YAML config dependencies) and returns ENV variables for Terraform component `my-vpc`. ```hcl module "vars" { source = "cloudposse/stack-config/yaml//modules/env" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } ``` See [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/env/../../examples/complete) for more details. --- ## remote-state Terraform module that accepts a component and a stack name and returns remote state outputs for the component. The module supports all backends supported by Terraform and OpenTofu, plus the Atmos-specific `static` backend. ### Errors :::note If you experience an error from the `terraform_remote_state` data source, this is most likely not an error in this module, but rather an error in the `remote_state` configuration in the referenced stack. This module performs no validation on the remote state configuration, and only modifies the configuration for the `remote` backend (to set the workspace name) and, _only when `var.privileged` is set to `true`_, the `s3` configuration (to remove settings for assuming a role). If `var.privileged` is left at the default value of `false` and you are not using the `remote` backend, then this module will not modify the backend configuration in any way. ::: ### "Local" Backend :::important If the local backend has a relative path, it will be resolved relative to the current working directory, which is usually a root module referencing the remote state. However, when the local backend is created, the current working directory is the directory where the target root module is defined. This can cause the lookup to fail if the source is not reachable from the client directory as `../source`. ::: For example, if your directory structure looks like this: ```text project ├── components │   ├── client │   │   └── main.tf │   └── complex │   └── source │   └── main.tf └── local-state └── complex └── source └── terraform.tfstate ``` Terraform code in `project/components/complex/source` can create its local state file (`terraform.tfstate`) in the `local-state/complex/source` directory using `path = "../../../local-state/complex/source/terraform.tfstate"`. However, Terraform code in `project/components/client` that references the same local state using the same backend configuration will fail because the current working directory is `project/components/client` and the relative path will not resolve correctly. ## Usage The following example accepts a stack config `my-stack` (which in turn imports other YAML config dependencies) and returns remote state outputs from the `s3` backend for `my-vpc` and `eks` Terraform components. __NOTE:__ The backend type (`s3`) and backend configuration for the components are defined in the stack YAML config files. ```hcl module "remote_state_my_vpc" { source = "cloudposse/stack-config/yaml//modules/remote-state" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" stack = "my-stack" component = "my-vpc" } module "remote_state_eks" { source = "cloudposse/stack-config/yaml//modules/remote-state" # Cloud Posse recommends pinning every module to a specific version # version = "x.x.x" stack = "my-stack" component = "eks" } ``` See [examples/remote-state](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/remote-state/../../examples/remote-state) for more details. --- ## settings Terraform module that accepts stack configuration and returns deep-merged settings for a Terraform or helmfile component. ## Usage The following example loads the stack config `my-stack` (which in turn imports other YAML config dependencies) and returns settings for Terraform component `my-vpc`. ```hcl module "vars" { source = "cloudposse/stack-config/yaml//modules/settings" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } ``` See [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/settings/../../examples/complete) for more details. --- ## spacelift(3) Terraform module that accepts infrastructure stack configurations and transforms it into Spacelift stacks. ## Usage The following example loads the infrastructure YAML stack configs and returns Spacelift stack configurations: ```hcl module "spacelift" { source = "../../modules/spacelift" stack_config_path_template = "stacks/%s.yaml" component_deps_processing_enabled = true context = module.this.context } ``` See [examples/spacelift](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/spacelift/../../examples/spacelift) for more details. --- ## stack Terraform module that constructs stack names. If `var.stack` is specified, will be returned as is. If not specified, the output will be calculated using the provided `context`. --- ## vars Terraform module that accepts stack configuration and returns deep-merged variables for a Terraform or helmfile component. ## Usage The following example loads the stack config `my-stack` (which in turn imports other YAML config dependencies) and returns variables and backend config for Terraform component `my-vpc`. ```hcl module "vars" { source = "cloudposse/stack-config/yaml//modules/vars" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } module "backend" { source = "cloudposse/stack-config/yaml//modules/backend" # version = "x.x.x" stack_config_local_path = "./stacks" stack = "my-stack" component_type = "terraform" component = "my-vpc" context = module.this.context } ``` The example returns the following `vars` and `backend` configurations for `my-stack` stack and `my-vpc` Terraform component: ```hcl vars = { "availability_zones" = [ "us-east-2a", "us-east-2b", "us-east-2c", ] "cidr_block" = "10.132.0.0/18" "environment" = "ue2" "level" = 3 "namespace" = "eg" "param" = "param4" "region" = "us-east-2" "stage" = "prod" "subnet_type_tag_key" = "example/subnet/type" "test_map" = { "a" = "a_override_2" "b" = "b_override" "c" = [ 1, 2, 3, ] "map2" = { "atr1" = 1 "atr2" = 2 "atr3" = [ "3a", "3b", "3c", ] } } "var_1" = "1_override" "var_2" = "2_override" "var_3" = "3a" } backend_type = s3 backend = { "acl" = "bucket-owner-full-control" "bucket" = "eg-ue2-root-tfstate" "dynamodb_table" = "eg-ue2-root-tfstate-lock" "encrypt" = true "key" = "terraform.tfstate" "region" = "us-east-2" "role_arn" = "arn:aws:iam::xxxxxxxxxxxx:role/eg-gbl-root-terraform" "workspace_key_prefix" = "vpc" } ``` See [examples/complete](https://github.com/cloudposse/terraform-yaml-stack-config/tree/main/modules/vars/../../examples/complete) for more details. --- ## YAML import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; Utilize these Terraform modules to manage and generate YAML configurations. These modules help in organizing and maintaining YAML files within your infrastructure as code practices. --- ## Terraform Modules(Modules) import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a collection of reusable Terraform Modules. In this library, you'll find real-world examples of how we've implemented reusable Terraform Modules. --- ## Action Items(Quickstart) import Intro from '@site/src/components/Intro'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import Admonition from '@theme/Admonition' To get a head start on your infrastructure as code journey, we recommend completing the following action items while you wait on Cloud Posse to deliver your configurations. These steps will help you set up your environment and prepare for the provisioning process. ## Prepare a New AWS Organization (root account) We recommend that you start with a new AWS Organization (e.g. a new payer account). As part of the provisioning process, you will be terraforming your entire organization, creating 12-plus accounts, and building everything from the ground up. You will be configuring SSO, fine-grained IAM roles, and more, all with Terraform. We recommend a net-new Organization, so you do not jeopardize any of your current production operations. Create a new AWS root account and add the root credentials to 1Password. ## Create GitHub Repository Create a new repository in your GitHub organization that you will use as your Infrastructure as Code repository. ## AWS IAM Identity Center (AWS SSO) In order connect your chosen IdP to AWS IAM Identity Center (AWS SSO), we will to configure your provider and create a metadata file. Please follow the relevant linked guide and follow the steps for the Identity Provider. - [GSuite](https://aws.amazon.com/blogs/security/how-to-use-g-suite-as-external-identity-provider-aws-sso/) - [Office 365](/layers/identity/aws-sso/#configure-your-identity-provider) - [JumpCloud](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center) - [Other AWS supported IdPs: Azure AD, CyberArk, Okta, OneLogin, Ping Identity](https://docs.aws.amazon.com/singlesignon/latest/userguide/supported-idps.html) - GSuite does not automatically sync Users and Groups with AWS Identity Center without additional configuration! If using GSuite as an IdP, considering deploying the [ssosync tool](https://github.com/awslabs/ssosync). - The official AWS documentation for setting up JumpCloud with AWS IAM Identity Center is not accurate. Instead, please refer to the [JumpCloud official documentation](https://jumpcloud.com/support/integrate-with-aws-iam-identity-center) ## Configure AWS SAML (Optional) If deploying AWS SAML as an alternative to AWS SSO, you will need a separate configuration and metadata file. Again, please refer to the relevant linked guide. - [GSuite](https://aws.amazon.com/blogs/desktop-and-application-streaming/setting-up-g-suite-saml-2-0-federation-with-amazon-appstream-2-0/): Follow Steps 1 through 7. This document refers to Appstream, but the process will be the same for AWS. - [Office 365](/layers/identity/tutorials/how-to-setup-saml-login-to-aws-from-office-365) - [JumpCloud](https://support.jumpcloud.com/support/s/article/getting-started-applications-saml-sso2) - [Okta](https://help.okta.com/en-us/Content/Topics/DeploymentGuides/AWS/aws-configure-identity-provider.htm) ## Purchase Domains (Optional) If you plan to use the `core-dns` account to register domains, make sure to add a credit card directly to that individual account. When the account is ready, please add a credit card to the `core-dns` account following the [AWS documentation](https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/manage-cc.html#Add-cc). ## Start Implementing the Reference Architecture Now that you're armed with everything you need, you can start implementing the reference architecture. We recommend starting with the [Foundation Layer](/layers/foundation) and working your way up from there. Get Started ## GitHub Actions ### Self Hosted Github Runners on EKS If you are deploying the Actions Runner Controller solution for Self-Hosted Github Runners, please generate the required secrets following the [GitHub Action Runner Controller setup docs](/layers/github-actions/eks-github-actions-controller/#requirements). ### Self Hosted Github Runners with Philips Labs (ECS) If you have chosen ECS as a platform, we recommend deploying Philips Labs GitHub Action Runners. Please read through the [Philips Labs GitHub Action Runners Setup Requirements](/layers/github-actions/philips-labs-github-runners#requirements). In particular, you will need a new GitHub App including a Private Key, an App ID, and an App Installation ID. We recommend that you store these secrets in 1Password. ### Atmos Component Updater Requirements The Atmos component updater GitHub Action will automatically suggest pull requests in your new repository, when new versions of Atmos components are available. If you plan to leverage it, you will need to create and install a GitHub App and allow GitHub Actions to create and approve pull requests within your GitHub Organization. For more on the Atmos Component Updater, see [atmos.tools](https://atmos.tools/integrations/github-actions/component-updater). ### Create and install a GitHub App for Atmos 1. Create a new GitHub App 2. Name this new app whatever you prefer. For example, `Atmos Component Updater`. 3. List a Homepage URL of your choosing. This is required by GitHub, but you can use any URL. For example use our documentation page: `https://atmos.tools/integrations/github-actions/component-updater/` 4. (Optional) Add an icon for your new app (example provided below) 5. Assign only the following Repository permissions: ```diff + Contents: Read and write + Pull Requests: Read and write + Metadata: Read-only ``` 6. Generate a new private key [following the GitHub documentation](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/managing-private-keys-for-github-apps#generating-private-keys). 7. Save both the App ID and the new private key in 1Password
Feel free to download and use our Atmos icon with your GitHub App! ![App Icon](/assets/refarch/github-app-icon.png)
### Allow GitHub Actions to create and approve pull requests 1. Go to `https://github.com/organizations/YOUR_ORG/settings/actions` 2. Check "Allow GitHub Actions to create and approve pull requests" ### Create `atmos` GitHub Environment We recommend creating a new GitHub environment for Atmos. With environments, the Atmos Component Updater workflow will be required to follow any branch protection rules before running or accessing the environment's secrets. Plus, GitHub natively organizes these Deployments separately in the GitHub UI. 1. Open "Settings" for your repository 1. Navigate to "Environments" 1. Select "New environment" 1. Name the new environment, "atmos". 1. In the drop-down next to "Deployment branches and tags", select "Protected branches only" 1. In "Environment secrets", create the two required secrets for App ID and App Private Key created above and in 1Password. This will be accessed from GitHub Actions with `secrets.ATMOS_APP_ID` and `secrets.ATMOS_PRIVATE_KEY` respectively.
## Optional Integrations The reference architecture supports multiple integrations. Depending on your requirements, you may need a few subscriptions set up. Please subscribe only to the services you plan to use! ### Datadog Sign up for Datadog following the [How to Sign Up for Datadog?](/layers/monitoring/datadog/tutorials/how-to-sign-up-for-datadog) documentation. --- ## Quickstart FAQ ### What is the difference between a Service Discovery Domain and a Vanity Domain? This is an extremely common question. Please see [What is the difference between a Vanity and a Service Domain?](/layers/network/faq/#what-is-the-difference-between-a-vanity-and-a-service-domain) ### Do we have to use 1Password? No, you can use whichever password manager you prefer. For Cloud Posse engagements, we use 1Password exclusively to share secrets. ### Do we have to create a new Organization? Yes! We recommend registering for a new AWS Organization. You will be terraforming your entire organization, creating 12-plus accounts, and doing everything from the ground up. You'll be configuring SSO, fine-grained IAM roles, and more. We recommend a net-new Organization, so you do not jeopardize any of your current productoin operations. ### How many email addresses do we need to create? Only one email with `+` addressing is required. This email will be used to create your AWS accounts. For example, `aws+%s@acme.com`. ### What is plus email addressing? Plus email addressing, also known as plus addressing or subaddressing, is a feature offered by some email providers that allows users to create multiple variations of their email address by adding a "+" sign and a unique identifier after their username and before the "@" symbol. For example, if the email address is "john.doe@example.com", a user can create variations such as "john.doe+newsletter@example.com" or "john.doe+work@example.com". Emails sent to these variations will still be delivered to the original email address, but the unique identifier can be used to filter or organize incoming emails. ### How can we customize our architecture? Customizations are out of scope typically, but we can assess each on a case-by-case basis. You will learn your environment and be confident to make customizations on your own. Often we can deploy an example of the customization, but it's up to you to implement the full deployment ### What if we need more help? Cloud Posse offers multiple levels of support designed to fit your needs and budget. For more information, see [Support](/support). For everything else, we provide fanatical support via our Professional Services, exclusive to reference architecture customers. We can help with anything from architecture reviews, security audits, custom development, migrations, and more. Please [book a call](https://cloudposse.com/meet) to discuss your needs. --- ## Handoffs import Handoffs from '@site/docs/jumpstart/handoffs.mdx'; --- ## Kick Off with Cloud Posse(Quickstart) import Link from "@docusaurus/Link"; import KeyPoints from "@site/src/components/KeyPoints"; import Steps from "@site/src/components/Steps"; import Step from "@site/src/components/Step"; import StepNumber from "@site/src/components/StepNumber"; import Intro from "@site/src/components/Intro"; import ActionCard from "@site/src/components/ActionCard"; import PrimaryCTA from "@site/src/components/PrimaryCTA"; import TaskList from "@site/src/components/TaskList"; import Admonition from "@theme/Admonition"; import Tabs from "@theme/Tabs"; import TabItem from "@theme/TabItem"; The kickoff call for [Quickstarts](/intro/path) ensures you get a smooth start with clear expectations. During the call, we will confirm your design decisions. After the call, you'll be ready to start with some the action items. You'll quickly receive all the configurations customized to your requirements in about 2-3 days. - **Review Design Decisions** Confirm your requirements answer any questions you may have - **Cover Next Steps** Review what to expect after the call, and what you need to get started - **Introduce Support Options** in the event you get stuck ## Preparing for the Kickoff Meeting This document outlines what to expect from your first call with Cloud Posse. In order to make the most of this meeting, please read through this document and come prepared with questions. In particular, please review the following: 1. Decide who will lead the project in your organization 2. Ensure everyone who needs to be on the call is added to the invitation 3. Read through the [Design Decisions](#-review-design-decisions) and prepare questions and decisions 4. Review the list of [Actions Items](#action-items) following this call --- ## Kickoff Meeting Agenda The following is the agenda for our Kick off call with you. We'll schedule this call as soon as you've purchased a [Quickstart](/intro/path) and submitted your [Design Decisions](#-review-design-decisions). ### Introductions Here we will review who is on the call, what their roles are, and identify our technical point of contact at Cloud Posse. We will also review the working timezones of the teams. ### Project Overview After your kick off call with Cloud Posse and receiving the configurations customized to your design decisions, you will be ready to begin deploying your infrastructure starting with the [foundation](/layers/foundation). The Reference Architecture is a collection of best practices for building a secure, scalable, and highly available infrastructure on AWS. The Reference Architecture is constantly evolving as we learn from our customers and the community. You will deploy your infrastructure in _layers_. These layers are designed to manage collections of deliverables and will be a mix of generated content from a private reference, vendored Terraform from open source libraries, and any customization for your Organization. Because we are delivering an entire infrastructure repository, these initial PRs will be massive; a complete infrastructure set up requires dozens of components, each with Terraform modules, configuration, account setup, and documentation. You are absolutely welcome to follow along, but we do not intend for your team to be required to review these massive PRs. Cloud Posse internally reviews these PRs extensively to ensure that the final product works as intended. Before you begin provisioning any layer, we recommend watching the video that explains the problems we faced, so you'll be better equipped to understand the solution. These videos go into detail on the subject to explain the problems we faced, the tradeoffs we considered and then at a high level how we solved them. **Need a hand?** Our [Essential Support](/support/essential) provides weekly guidance—whether it's answering questions or troubleshooting issues. We're here to help! ### Introduce Shared Customer Workshops > **When:** Thursdays, 7:00-7:30A PT/ 9:00-9:30A CT/ 10:00-10:30A ET > **Where:** Zoom > **Who:** [Essential Support Customers Only](/support/essential) > **When:** Wednesdays, 2:30-3:00P PT/ 4:30-5:00P CT/ 5:30-6:00P ET > **Where:** Zoom > **Who:** [Essential Support Customers Only](/support/essential) This is a great opportunity to get your questions answered and to get help with your project. ### Sign up for Community Office Hours > **When:** Wednesdays, 11:30a-12:30p PT/ 1:30p-2:30p CT/ 2:30p-3:30p ET > **Where:** Zoom > **Who:** Anyone (open to the public!) This is a good way to keep up with the latest developments and trends in the DevOps community. Sign up at [cloudposse.com/office-hours](https://cloudposse.com/office-hours/) ### Join our SweetOps Slack Community If you are looking for a community of like-minded DevOps practitioners, we invite you to join our [SweetOps Slack](https://slack.sweetops.com/). ### Review Design Decisions Review the foundational Design Decisions. - [ ] [Decide on Terraform Version](/layers/project/design-decisions/decide-on-terraform-version) - [ ] [Decide on Namespace Abbreviation](/layers/project/design-decisions/decide-on-namespace-abbreviation) - [ ] [Decide on Infrastructure Repository Name](/layers/project/design-decisions/decide-on-infrastructure-repository-name) - [ ] [Decide on Email Address Format for AWS Accounts](/layers/accounts/design-decisions/decide-on-email-address-format-for-aws-accounts) - [ ] [Decide on IdP](/layers/identity/design-decisions/decide-on-idp) - [ ] [Decide on IdP Integration Method](/layers/identity/design-decisions/decide-on-idp-integration) - [ ] [Decide on Primary AWS Region and Secondary AWS Region](/layers/network/design-decisions/decide-on-primary-aws-region) - [ ] [Decide on CIDR Allocation Strategy](/layers/network/design-decisions/decide-on-cidr-allocation) - [ ] [Decide on Service Discovery Domain](/layers/network/design-decisions/decide-on-service-discovery-domain) - [ ] [Decide on Vanity Domain](/layers/network/design-decisions/decide-on-vanity-branded-domain) These are the design decisions you can customize as part of the Quickstart package. All other decisions are pre-made for you, but you’re welcome to review them. If you’d like to make additional changes, [let us know—we’re happy to provide a quote](https://cloudposse.com/meet). ## How to Succeed Cloud Posse has noticed several patterns that lead to successful projects. ### Come to Calls Prepared Review six pagers and documentation before our calls. This will help you to know what questions to ask. Coming unprepared will lead to a lot of questions and back-and-forth. This will slow down material resulting in less time for new material. ### Take Initiative The most successful customers take initiative to make customizations to their Reference Architecture. This is a great way to make the Reference Architecture your own. It also helps to build a deeper understanding of the Reference Architecture and how it works. ### Cameras On We recommend that all participants have their cameras on during our Zoom calls. This helps to build trust and rapport. It also helps to keep everyone engaged and focused. This also lets us gauge how everyone is understanding the material. If you are having trouble understanding something, please ask questions. ### Ask Questions We encourage you to ask questions. We want to make sure that everyone understands the material. We also want to make sure that we are providing the right level of detail. Our meetings are intended to be interactive and encourage conversation. Please feel free to interject at any time if you have a question or a comment to add to the discussion. ### Participate in our Slack Community We encourage you to participate in [our public Slack channels](https://cloudposse.com/slack). This is a great way to get help and to learn from others. We have a lot of customers who have been through the same process and can provide valuable insights. We also have a lot of Cloud Posse engineers who are available to help answer questions. ### Attend Weekly Office Hours Our free weekly [Community Office Hours](#community-office-hours) are great opportunity to ask questions and get help. ### Read our Documentation You can always find how-to guides, design decisions, and other helpful pages at [docs.cloudposse.com](/) ### Take the Next Step Don't wait! Keep the momentum going by taking the next step. If you have questions, ask them. If you need help, ask for it. We are here to help you succeed. After our kickoff call, there are several action items for you to consider based on your goals. Not every item may be relevant, but please review them and take action on the ones that apply to you. Next Step --- ## Quickstart import Intro from '@site/src/components/Intro'; import ActionCard from '@site/src/components/ActionCard'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import PillBox from '@site/src/components/PillBox'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; Do it Yourself (DIY) This documentation will guide you through the end-to-end configuration of our reference architecture for your AWS organization. You can customize everything to suit your needs and implement it at your own pace. Many have completed the process in under a week, and we provide [tons of options for support](/support) if you get stuck. ## Get the Quickstart Configurations All of this documentation refers to the prebuilt configurations we provide as part of the **Quickstart Package**, which is how we fund our open-source efforts. This is optional. You are welcome to follow along with the documentation and implement your configurations from scratch, but the Quickstart Package **will save you a lot of time**. And guess what? It's just a one time fee. Buy Quickstart Configurations ## Prepare Your Design Decisions Design Decisions are how we customize the Quickstart Configurations to your needs. We'll send you a form to fill out with your requirements. Review Design Decisions ## Schedule your Kickoff Call with Cloud Posse Once you've submitted your Design Decisions, we'll schedule a call to review them with you. This is an opportunity to review them with Cloud Posse, and ask any questions before you get started. Review Agenda ## Start Building After the call, you'll receive all the configurations customized to your requirements in about 2-3 days. You'll use these configurations together with our documentation to get started with your first project. Get Started ## Get Stuck? Try our Support If you need assistance, we provide multiple options for support that fit your needs and budget. Get Support If you need more assistance, our [support options](/support) provide direct access to Cloud Posse to help you out. Review Kick-off Agenda Read the Documentation --- ## Cloud Posse Documentation License import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; This material may only be distributed subject to the terms and conditions set forth in the *[Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-nc-sa/4.0/)* or later with the restrictions noted below (the latest version of the license is presently available at [Creative Commons v4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)). - **Attribution** You must attribute the work in the manner specified by the author or licensor. - **Noncommercial** The licensor permits others to copy, distribute and transmit the work. In return, licensees may not use the work for commercial purposes — unless they get the licensor's permission. - **Share Alike** The licensor permits others to distribute derivative works only under the same license or one compatible with the one that governs the licensor's work. ## Copyright Copyright 2017-2025 © Cloud Posse, LLC. ## Distribution Distribution of the work or derivative of the work in any standard (paper) book form for commercial purposes is prohibited unless prior permission is obtained from the copyright holder. Distribution of substantively modified versions of this document is prohibited without the explicit permission of the copyright holder. ## Trademarks All other trademarks referenced herein are the property of their respective owners. :::important This documentation is provided (and copyrighted) by Cloud Posse, LLC and is released via the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-nc-sa/4.0/). The copyright holder has added the further requirement that Distribution of substantively modified versions of this document is prohibited without the explicit permission of the copyright holder. ::: --- ## Infrastructure as Code Library import DocCardList from '@theme/DocCardList'; import Intro from '@site/src/components/Intro'; Welcome to the Cloud Posse Library, your one-stop resource for all our open source infrastructure projects. Whether you're looking to deploy production-grade infrastructure, automate your workflows, or learn best practices, you'll find everything you need here. Our library is built on years of experience managing cloud infrastructure at scale, and we're excited to share these tools with you. The Cloud Posse Library is a comprehensive collection of our open source projects designed to help you build, deploy, and manage cloud infrastructure. This library contains everything you need to implement Infrastructure as Code (IaC) best practices, including: - **Terraform Components**: Production-grade, reusable "root" modules for AWS that provide complete solutions for common infrastructure patterns - **Terraform Modules**: Reusable "child" modules that can be composed together to build custom infrastructure - **GitHub Actions**: Automated workflows for continuous integration and delivery (CI/CD) - **Resources**: Additional tools, scripts, and documentation to support your cloud infrastructure journey All our projects are built with best practices in mind, including security, scalability, and maintainability. They are designed to be modular, composable, and follow the principle of least privilege. --- ## Adopted Architecture Decision Records import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; These are a collection of architectural decisions that have been adopted. --- ## Use API Gateway REST API vs HTTP API :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse ::: ## Status **DECIDED** ## Problem When using the [AWS API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html), users must choose between creating a REST API or an HTTP API. ## Context AWS supports multiple types of API Gateways and there are tradeoffs between them. This document will help decide which flavor of API Gateway is suitable for your use case. ### Differences | **Category/Feature** | **HTTP API** (formerly known as v2) | **REST API** (formerly known as v1) | | ---------------------------------------------------- | ---------------------------------------------------------------- | ----------------------------------- | | **Authorizers** | | | | Cognito | Partial Support. Cognito can be used only as a JWT token issuer. | ✔ | | Native OpenID Connect / OAuth 2.0 | ✔ | | | **Integration** | | | | Private integrations with Application Load Balancers | ✔ | | | Private integrations with AWS Cloud Map | ✔ | | | Mock | | ✔ | | **API Management** | | | | Usage plans (e.g. rate limiting) | | ✔ | | API keys | | ✔ | | TLS | ✔ (Does not support TLS 1.0) | ✔ | | **Development** | | | | API caching | | ✔ | | Request body transformation | | ✔ | | Request / response validation | | ✔ | | Test invocation (e.g. test backend via AWS console) | | ✔ | | Automatic deployments | ✔ | | | Default stage | ✔ | | | Default route | ✔ | | | Custom Gateway Responses | | ✔ | | Canary Deployments | | ✔ | | **Security** | | | | Certificates for backend authentication | | ✔ | | AWS WAF | | ✔ | | Resource Policies | | ✔ | | **Deployment Options** | | | | Regional | ✔ | ✔ | | Edge-Optimized (Cloudfront) | | ✔ | | Private | | ✔ | | **Monitoring** | | | | Access logs to Amazon Kinesis Data Firehose | | ✔ | | Execution logs | | ✔ | | AWS X-Ray | | ✔ | ## Decision **DECIDED**: Use REST API (formerly known as v1) because it checks all the boxes --- ## Use Custom AWS Region Codes import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem The AWS Public Cloud spans **25 geographic regions** around the world, with announced plans for 8 more AWS Regions in Australia, India, Indonesia, Israel, New Zealand, Spain, Switzerland, and United Arab Emirates (UAE). Regions play a big factor in naming conventions for multi-region infrastructures for disambiguation and resource residency. See [Decide on Regional Naming Scheme](/layers/project/design-decisions/decide-on-regional-naming-scheme), [Decide on Namespace Abbreviation](/layers/project/design-decisions/decide-on-namespace-abbreviation) for context. Our [Terraform](/resources/legacy/fundamentals/terraform) is used to define programmatically consistent resource names with deterministic fields separated by a common delimiter (typically `-`), including a field for region (which we call `environment`). Since the AWS regions include a `-`, we do not want our region code to include it. Additionally, many AWS resource names are restricted to 32 or 64 characters making it all the more important to conserve characters for disambiguation of resource names. ## Solution Cloud Posse provides two naming conventions to address AWS regions: `fixed` and `short`. They are defined in the `terraform-aws-utils` module, which exposes mapping outputs to use when working in AWS. It provides compact alternative codes for Regions, Availability Zones, and Local Zones that are guaranteed to use only digits and lower case letters: no hyphens. Conversions to and from official codes and alternative codes are handled via lookup maps. The `short` abbreviations are variable-length (generally 4-6 characters, but length limits not guaranteed) and strictly algorithmically derived so that people can more easily interpret them. The `fixed` abbreviations are always exactly 3 characters for regions and 4 characters for availability zones and local zones, but have some exceptional cases (China, Africa, Asia-Pacific South, US GovCloud) that have non-obvious abbreviations. We currently support Local Zones but not Wavelength Zones. If we support Wavelength Zones in the future, it is likely that the fixed-length abbreviations for them will be non-intuitive. The intention is that existing mapping will never change, and if new regions or zones are created that conflict with existing ones, they will be given non-standard mappings so as not to conflict. ## Region Codes [https://github.com/cloudposse/terraform-aws-utils/blob/master/main.tf](https://github.com/cloudposse/terraform-aws-utils/blob/master/main.tf) --- ## Use Basic Provider Block for Root-level Components **Date**: **19 Oct 2021** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse ::: ## Status **ACCEPTED** ## Context We [Use Terraform Provider Block with compatibility for Role ARNs and Profiles](/resources/adrs/adopted/use-terraform-provider-block-with-compatibility-for-role-arns-an) in all components other than the root-level components. By _root-level_ we are referring to components that are provisioned in the top-level AWS account that we generally refer to as the `root` account. The problem arises when working with the `root` account during a cold-start when there’s no SSO, Federated IAM or IAM roles provisioned, so if we used the `role_arn` or `profile` it would not work. That’s why we assume the administrator will use their current AWS session to provision these components, which is why we do not define the `role_arn` or `profile` in `provider { ... }` block for the components like [sso](/components/library/aws/identity-center/) or [account](/components/library/aws/account/) . ## Decision **DECIDED**: Use the following basic provider block in root components. ``` provider "aws" { region = var.region } ``` ## Consequences - Update any root-level components to use this block ## References - [Use Terraform Provider Block with compatibility for Role ARNs and Profiles](/resources/adrs/adopted/use-terraform-provider-block-with-compatibility-for-role-arns-an) --- ## Use Environment Variables for Configuration (12 Factor) **Date**: **14 Dec 2021** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - This page is mostly correct but may not be entirely up-to-date. ::: ## Status **ACCEPTED** ## Problem We need a simple way to pass parameters to applications without hardcoding the settings. Multiple approaches exist with tradeoffs. The 12 Factor pattern recommends using environment variables, but there are security implications everyone needs to be aware of. ## Context ## Considered Options ### **Option 1**: Use Environment Variables #### Pros - Portable configuration format supported by every mainstream language - Easily understood (key/value) pairs - Easily implemented - Supported by Kubernetes, Docker, and ECS - Compatible with SSM (via `chamber` and `external-secrets` operator), ASM (`external-secrets` operator) and HashiCorp Vault (via envconsul) - The 12 Factor pattern recommends using environment variables [https://12factor.net/](https://12factor.net/) #### Cons - Environment variables are exposed via the /proc filesystem. any process on the system can read those settings - Environment variables are harder to validate (e.g. typo an ENV, you won't get a warning in most applications, especially for optional settings) - Environment variable sprawl: over time, you may end up with hundreds of ENVs as some of our customers have. they have products that have been around for a decade or longer and gone through generations of engineers - Environment variables are harder to update. E.g. What updates the environment variables, such as CD? - If your app still consumes configs, but you are parameterizing it with ENVS, it's tedious to update both the envs and the config file templating every time you add an env - Environment variables are really only convenient for scalars. Serializing structures in YAML/JSON is ugly - ECS task definitions are capped at 64K, meaning if you use a lot of ENVs (or long ENVs), you will hit this limit when you least expect it - Kubernetes ConfigMaps are capped at 1MB, so if using ConfigMaps for ENVs, there’s still a practical limit. - Legacy applications frequently do not support environment variables ### **Option 2**: Use Configuration Files #### Pros - Compatible with Kubernetes ConfigMaps, making them easy to mount into containers [https://kubernetes.io/docs/concepts/storage/volumes/#configmap](https://kubernetes.io/docs/concepts/storage/volumes/#configmap) - Compatible even with legacy applications that depend on configuration files - Can use templates to generate configuration files from environment variables (e.g. not mutually exclusive) - Easily deployed as part of CI/CD workflow #### Cons - There are a million configuration file formats, and no standardized way of defining them - For ephemeral environments, configuration files need to be templatized, adding a layer of complexity - Configuration files should be encrypted (e.g. see `sops-operator` for kubernetes) ## Decision **DECIDED**: Use environment variables as standardized means of configuration ## Consequences - Use `external-secrets` operator with Kubernetes to mount SSM/ASM parameters as environment variables - Use ECS `envs` to pass environment variables to tasks in the service definition ## References - See also [Use SSM over ASM for Infrastructure](/resources/adrs/adopted/use-ssm-over-asm-for-infrastructure) --- ## Use One OpsGenie Integration per Team :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse ::: ## Context OpsGenie Integrates with Datadog, on the OpsGenie Platform this creates an API key that when used, sends to that specific Integration. e.g. A Key that is used by datadog to send events to OpsGenie. Many integrations can be setup on OpsGenie, allowing Datadog to specify which integration to use. This is important because an integration on OpsGenie can be specified to handle events differently. This means datadog can send an event to `@OpsGenie-1` and its routed differently in Opsgenie than if the message contained `@OpsGenie-2`. The problem with adding more integrations, is creating them in Opsgenie can be done through terraform, but adding the generated API Key must be added to DataDog manually. **TL;DR:** We support OpsGenie today and have a considerable investment in supporting it, but are open to implementing PagerDuty. ### OpsGenie Integration Per Team (Decided) Create an opsgenie api integration of `type=datadog` per team which maps to a responding team in order to enable `@opsgenie-` to be inserted within a Datadog Monitor message. This will allow us to tag resources with a team and then set the message `@opsgenie-{{team.name}}` and depending on the incident rule per team, it will or will not declare an incident. #### Pros - Simple logic - **This follows Datadog’s opsgenie guide** [https://docs.datadoghq.com/integrations/opsgenie/#create-acknowledge-and-close-opsgenie-alerts-from-datadog](https://docs.datadoghq.com/integrations/opsgenie/#create-acknowledge-and-close-opsgenie-alerts-from-datadog) **** - We would have either no global alert policy or a generic one #### Cons - For each team, we’d have to clickops a Opsgenie integration in Datadog because Datadog API doesn’t support opsgenie integration. (however, Only really a cold-start problem) ### One OpsGenie Integration with Global Routing Rules We create a single opsgenie api integration of `type=datadog` which has a global alert policy per team because the alert policy cannot templatize the responder based on the incoming key value tag #### Pros - Single generic api integration because this will have to be clickops’ed in Datadog because of no Datadog API for opsgenie integration - Terraform support for global alert policy per team #### Cons - Many global alert policies, specifically 1 per team minimum ## Decision We have decided to go with an Integration per team. This is because it follows the recommended Datadog vs OpsGenie Integration method, it provides a clean approach to team routing, and it is hopeful that eventually the clickOps portion can be terraformed once the API is exposed. --- ## Use OpsGenie for Incident Management Monitoring platforms like CloudWatch and Datadog historically provide very poor support for Incident Management. Incident Management is the art of ingesting, classifying, and escalating alerts to stakeholders based on rotations, teams, services, etc. :::tip Latest! The content in this ADR is up-to-date! For questions, please reach out to Cloud Posse ::: ## Context There are quite a few incident management platforms available, with PagerDuty being the OG. Customers often ask why we selected OpsGenie over PagerDuty, this is our current rationale. **TL;DR:** We support OpsGenie today and have a considerable investment in supporting it, but are open to implementing PagerDuty. ### OpsGenie (Decided) [https://github.com/cloudposse/terraform-opsgenie-incident-management](https://github.com/cloudposse/terraform-opsgenie-incident-management) #### Pros - Most customers use Atlassian products, including Jira, Service Desk, and Confluence which are all tightly integrated with OpsGenie - High feature parity with PagerDuty - OpsGenie is by Atlassian and tightly integrated - OpsGenie is less expensive than PagerDuty - OpsGenie is tightly integrated with StatusPage - Cloud Posse only has prior art for OpsGenie :smiley: (e.g. 20+ sprints executed on opsgenie, but none on pagerduty) #### Cons - Lacks some of the AI features now present in more modern Incident Management Platforms ### PagerDuty Customers frequently ask if we have PagerDuty support. The short answer is not yet. The longer answer is, we’re open to supporting it, if someone sponsors the development. We support OpsGenie due to customer demand. #### Pros - Arguably the dominant platform for Incident Management - Supports [Artificial Intelligence for IT operations (AIOps)](https://www.pagerduty.com/reference/learn/what-is-aiops/) #### Cons - More expensive than PagerDuty ### Datadog Incident Management Datadog released its own [incident management platform at the tail end of 2020](https://www.datadoghq.com/blog/incident-response-with-datadog/). We’ve not had a chance to evaluate the platform, mostly because as of this writing, [terraform support is non-existent](https://registry.terraform.io/providers/DataDog/datadog/latest/docs). For this reason, we ruled it out. ### Alert Panda Not Considered ### VictorOps Not Considered ## Decision - Use OpsGenie ## Consequences - Customers who want to implement OpsGenie for Incident Management should [subscribe to the Standard or Enterprise plans.](https://www.atlassian.com/software/opsgenie/pricing) --- ## Use Spacelift for GitOps with Terraform :::info A public page is available at [https://cloudposse.com/faqs/why-do-you-recommend-spacelift/](https://cloudposse.com/faqs/why-do-you-recommend-spacelift/) which shares a lot of these points. ::: Spacelift checks off all the boxes for managing extremely large environments with a lot of state management. Since Cloud Posse's focus is on deploying large-scale loosely coupled infrastructure components with Terraform, it's common to have several hundred terraform states under management. Every successful business in existence uses accounting software to manage its finances and understand the health of its business. The sheer number of transactions makes it infeasible to reconcile the books by hand. The same is true of modern infrastructure. With hundreds of states managed programmatically with terraform, and modified constantly by different teams or individuals, the same kind of state reconciliation is required to know the health of its infrastructure. This need goes far beyond continuous delivery and few companies have solved it. ## **Major Benefits** - **Reconciliation** helps you know what's deployed, what's failing, and what's queued. - **Plan Approvals** ensures changes are released when you expect them - **Policy-Driven Framework** based on OPA (open-source standard) is used to trigger runs and enforce permissions - **Drift Detection** runs on a customizable schedule surfaces inconsistencies with what's deployed and what's in git on previously successful stacks - **Terraform Graph Visualization** makes it easier to visualize the entire state across components - **Audit Logs** of every change traced back to the commit and filterable by time - **Affordable alternative** to other commercial offerings - **Works with more than Terraform** (e.g. Pulumi) - **Pull Request Previews** show what the proposed changes are before committing them - **Decoupling of Deploy from Release** ensures we can merge to trunk and still control when those changes are propagated to environments - **Ephemeral Environments** (Auto Deployment, Auto Destruction) enables us to bring up infrastructure with terraform and destroy it when it's no longer needed - **Self-hosted Runners** ensure we're in full control over what is executed in our own VPC, with no public endpoints ## Concerns ### **What level of access do the Spacelift worker pools have?** Spacelift Workers are deployed in your environment with the level of permission that we grant them via IAM instance profiles. When provisioning any infrastructure that requires modifying IAM, the minimum permission is administrative. Thus, workers are provisioned with administrative permissions in all accounts that we grant access to since the terraform we provision requires creating IAM roles and policies. Note, this is not a constraint of Spacelift; this is required regardless of the platform that performs the automation. ### **What happens if Spacelift as a product goes away?** First off, while Spacelift might be a newer brand in the infrastructure space, it's used by publicly traded companies, Healthcare companies, banks, institutions, Fortune 500 companies, etc. So, Spacelift is not going away. But just to entertain the hypothetical, let's consider what would happen. Since we manage all terraform states in S3, we have the “break glass” capability to leave the platform at any time and can always run terraform manually. Of course, we would lose all the benefits. ### **How tough would it be to move everything to a different platform?** Fortunately, with Spacelift, we can still use S3 as our standard state backend. So if at any time we need to move off of the platform, it's easy. Of course, we'd give up all the benefits but the key here is we're not locked into it. ### **Why not just use Atlantis?** We used to predominately recommend Atlantis but stopped doing so a number of years ago. The project was more or less dormant for 2-3 years, and only recently started accepting any Pull Requests. Atlantis was the first project to define a GitOps workflow for Terraform, but it's been left in the dust compared to newer alternatives. - With Alantis, there is no regular reconciliation of what terraform state has been applied or not applied. So we really have no idea in atlantis the _actual_ state of anything. With a recent customer, we helped migrate them from Atlantis to Spacelift and it took 2 months to reconcile all the infrastructure that had drifted. - With Atlantis, there's no drift detection, but with spacelift, we detect it nightly (or as frequently as we want) - With Atlantis, there's no way to manage dependencies of components, so that when one component changes, any other components that depend on it should be updated. - With Atlantis, there's no way to setup OPA policies to trigger runs. The OPA support in atlantis is very basic. - With Atlantis, [anyone who can run a plan, can exfiltrate your root credentials](https://www.youtube.com/watch?v=H9KvPe09f5A). This [talked about by others](https://alex.kaskaso.li/post/terraform-plan-rce) and was recently [highlighted at the Defcon 2021 conference](https://www.youtube.com/watch?v=3ODhxYY9-9U). - With Atlantis, there's no way to limit who can run terraform plan or apply. If you have access to the repo, you can run a terraform plan. If your plan is approved, you can run terraform apply. [Cloud Posse even tried to fix it](https://github.com/runatlantis/atlantis/issues/308) (and maintained our own fork for some time), but the discussion went nowhere and we moved on. - With Atlantis, there's no way to restrict who has access to unlock workspaces via the web GUI. The only way is to install your own authentication proxy in front of it or restrict it in your load balancer. - With Atlantis, you have to expose the webhook endpoint publicly to GitHub. ### **Why not use Terraform Cloud?** [Terraform Cloud](https://www.terraform.io/cloud) is prohibitively expensive for most non-enterprise customers we work with, and possibly 10x the cost of Spacelift. Terraform Cloud for Teams doesn't permit self-hosted runners and requires hardcoded IAM credentials in each workspace. That's insane and we cannot recommend it. Terraform Cloud for Business (and higher) support self-hosted runners that can leverage AWS IAM Instance profiles, but the number of runners is a significant factor of the cost. When leveraging several hundred loosely-coupled terraform workspaces, there is a significant need for a lot of workers for short periods of time. Unfortunately, even if those are only online for a short period of time, you need to commit to paying for them for the full month on an annualized basis. Terraform Cloud also requires that you use their state backend, which means there's no way to “break glass” and run terraform if they are down. If you want to migrate off of Terraform Cloud, you need to migrate the state of hundreds of workspaces out of the platform and into another state backend. ## References - https://www.spacelift.io/case-studies/cloud-posse - https://spacelift.io/case-studies --- ## Use SSM over ASM for Infrastructure :::tip Latest! The content in this ADR is up-to-date! For questions, please reach out to Cloud Posse ::: We primarily provision static credentials randomly generated by Terraform using the database provider and then write them to SSM, encrypted with KMS. Amazon Secrets Manager (**ASM**) was launched well after Amazon Systems Manager ([formerly AWS Simple Systems Manager](https://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html#service-naming-history), hence **SSM**) Parameter Store. Both fulfill similar use-cases, except for Parameter Store was create more generally as a Key/Value store for parameters that can also be encrypted with KMS/CMK keys. ASM on the other hand was purpose-built to manage secrets and has the concept of automatic rotation using Lambdas. Security best practices call for secrets to be rotated (changed) on a regular basis so that if a secret is improperly obtained by an adversary, the length of time the adversary can use that secret is limited. However, if everyone sharing the secret has to agree on what the secret is all the time, changing the secret presents a synchronization problem: all parties need to be updated with the new secret at exactly the same time. Parameter store only solves part of this problem: by having a single source of truth accessed by all parties, changing the secret once in parameter store makes the new secret available at the same time to all, but there is no mechanism to inform the parties of the change or to synchronize their adoption of the new secret. ASM was built to solve this synchronization problem. It allows you to store and retrieve several versions of a secret, with one being designated the “current” one and one being designated the “previous” one. To take advantage of this, servers (whatever is requiring the secret to be presented for authentication) must allow clients (whatever is presenting secrets as authentication) to present either the current or previous secret. When this is done, the synchronization problem is solved by executing the following steps in order 1. Confirm all clients and servers are using the “current” secret 2. “Rotate” the secret by changing the label of the current secret from “current” to “previous” and creating a new “current” secret. AWS provides a Lambda function that implements this. 3. Update all servers to accept either secret. This allows old clients who only know about the now “previous” secret to continue to access the servers, while allowing new clients to pick up and use the new “current” secret. Servers should be designed to be able to be updated in this way while running, without causing a service interruption. 4. Have all clients pick up and start using the new “current” secret when it is convenient for them. By maintaining 2 active secrets, clients can be designed to refresh their secrets when convenient, without significant time pressure. There is no need for special notification or synchronization features to be built into the clients. However, server-side support is critical: **If the servers do not support simultaneous use of 2 different secrets for the same purpose, there is no practical benefit to making the “previous” secret available.** One key reason to use ASM is that some Amazon services, such as RDS, have built-in integration with ASM to provide this simultaneous support. After ASM was built, SSM was enhanced to provide similar capabilities, although without as streamlined an API or a Lambda to do secrets rotation. SSM can store up to 100 versions of a parameter, and the versions can be given symbolic tags, such as “current” and “previous”. So **if you are building your own servers, you can implement the same kind of secrets rotation strategy with SSM as with ASM.** ## ASM ### Pros - Purpose-built to manage secrets - Supports automatic rotation of secrets using Lambda functions - AWS SDK support enables applications written to automatically pick up the new credentials - Multiple Kubernetes Operators exist to synchronize SSM Parameters to Kubernetes `Secrets` - Now supports 500K secrets per account [https://aws.amazon.com/about-aws/whats-new/2021/11/aws-secrets-manager-increases-secrets-limit-per-account/](https://aws.amazon.com/about-aws/whats-new/2021/11/aws-secrets-manager-increases-secrets-limit-per-account/) - Supports cross-region replication of a secret ### Cons - Only consumer applications that support dynamic credentials can take advantage of this functionality - Most systems do not support the complex pattern of key rotation; namely, concentric keys need some period of overlapping validity. Therefore, in practice it’s used mostly just for RDMS systems. - Still need to deploy the Lambda functions, which means all the CD/CD machinery to support lambdas (workflows, builds, integration tests, artifacts, deployments, etc) - No way to aggregate all secrets with a prefix - No built-in audit trail metadata (but it writes CloudTrail like any other AWS API). ### Other - 5,000 get requests per second limit: [https://aws.amazon.com/about-aws/whats-new/2020/11/aws-secrets-manager-supports-5000-requests-per-second-for-getsecretvalue-api-operation/](https://aws.amazon.com/about-aws/whats-new/2020/11/aws-secrets-manager-supports-5000-requests-per-second-for-getsecretvalue-api-operation/) ## SSM ### Pros - Very simple to operate with (true to it’s original name) - Can easily aggregate all key/values with a given prefix - Encrypted with KMS - Multiple Kubernetes Operators exist to synchronize SSM Parameters to Kubernetes `Secrets` - Built-in audit trail for every parameter in addition to CloudTrail ### Cons - No built-in concept of key rotation like ASM; however, see the “Cons” of ASM - Rate limits are lower than for ASM, but the access pattern of SSM is different so it’s not a 1:1 comparison. See `GetParametersByPath` and `GetParameters` [https://docs.aws.amazon.com/general/latest/gr/ssm.html](https://docs.aws.amazon.com/general/latest/gr/ssm.html) [https://aws.amazon.com/about-aws/whats-new/2019/04/aws_systems_manager_now_supports_use_of_parameter_store_at_higher_api_throughput/](https://aws.amazon.com/about-aws/whats-new/2019/04/aws_systems_manager_now_supports_use_of_parameter_store_at_higher_api_throughput/) ### Other - Historical rate limits for SSM were very low, now they are up to 3,000. ## Related - [Use Environment Variables for Configuration (12 Factor)](/resources/adrs/adopted/use-environment-variables-for-configuration-12-factor) - [REFARCH-210 - Decide Whether to Use RDS IAM Authentication](/layers/data/design-decisions/decide-whether-to-use-rds-iam-authentication/) --- ## Use Terraform Provider Block with compatibility for Role ARNs and Profiles **Date**: **19 Oct 2021** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - Please read through the detailed explanation in [Access Control Evolution](/layers/identity/docs/aws-access-control-evolution). ::: ## Status **ACCEPTED** ## Context Cloud Posse has used 2 conventions for assuming roles within the terraform `provider { ... }` block. ### The `role_arn` Method This was our original method of assuming roles within `terraform`. ``` provider "aws" { # The AWS provider to use to make changes in the DNS primary account alias = "primary" region = var.region assume_role { role_arn = coalesce(var.import_role_arn, module.iam_roles.dns_terraform_role_arn) } } ``` We used this for years together with AWS Federated IAM with SAML and had no issues. Then we started supporting AWS SSO and ran into some issues because with AWS SSO the role names are non-deterministic. As a result, we switched to the `profile` method below. ### The `profile` Method With the `profile` method we offload the burden of determining the `role_arn` to some external script that would generate the `~/.aws/config` with profiles and role mappings. This allowed us to support simultaneously the AWS Federated IAM alongside the AWS SSO method of authentication. The downside was we had to use the _generator_ pattern to create the `~/.aws/config`, which we generally like to avoid. We painfully upgraded all of our components to use this method since we didn’t see a path forward with the `role_arn` method at the time. ``` provider "aws" { region = var.region # `terraform import` will not use data from a data source, so on import we have to explicitly specify the profile profile = coalesce(var.import_profile_name, module.iam_roles.terraform_profile_name) } ``` ### The Hybrid Method Now we support the hybrid method after having come full circle and once again wanting to move to use the `role_arn` everywhere so we do not need to generate the AWS config. However, we also need to support customers that use the `profile` method. Fortunately, @Jeremy Grodberg found a convenient way to support both methods. ``` provider "aws" { region = var.region profile = module.iam_roles.profiles_enabled ? coalesce(var.import_profile_name, module.iam_roles.terraform_profile_name) : null dynamic "assume_role" { for_each = module.iam_roles.profiles_enabled ? [] : ["role"] content { role_arn = coalesce(var.import_role_arn, module.iam_roles.terraform_role_arn) } } } ``` ## Decision **DECIDED**: Use the Hybrid Method to support both `profile` or `role_arn` for backward compatibility Note: Until [Proposed: Use AWS Federated IAM over AWS SSO](/resources/adrs/proposed/proposed-use-aws-federated-iam-over-aws-sso) is decided otherwise, our recommendation for new projects is to use `role_arn`, but we continue to use the hybrid provider in public components, and therefore in client components. ## Consequences - Update all components to use the Hybrid Method. ## References - [Proposed: Use AWS Federated IAM over AWS SSO](/resources/adrs/proposed/proposed-use-aws-federated-iam-over-aws-sso) --- ## Use Terraform to Manage Helm Releases **Date**: **14 Dec 2021** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - Cloud Posse recommends using the `helm-release` module to manage operators and other infrastructure setup requirements but does not recommend using Terraform for application deployment. ::: ## Status **DRAFT** ## Problem Provisioning helm releases with `helmfile` has worked for a long time, but has a number of shortcomings: - No integration with Spacelift - No turnkey workflow for github actions (continuous delivery) - No way to provision IAM roles needed by services to solve these problems, we switched to using the Terraform `helm` provider, but this introduced a new set of problems: - Manual changes made to cluster resources were not detected as drift - Changes made to helm template files also were not detected as drift We want to be able to provision helm releases and all dependencies with terraform and GitOps while not compromising on drift detection. ### Related Issues - [https://github.com/databus23/helm-diff/issues/176](https://github.com/databus23/helm-diff/issues/176) - [https://github.com/hashicorp/terraform-provider-helm/pull/702](https://github.com/hashicorp/terraform-provider-helm/pull/702) - [https://github.com/databus23/helm-diff/issues/176#issuecomment-572952711](https://github.com/databus23/helm-diff/issues/176#issuecomment-572952711) - [https://github.com/databus23/helm-diff/pull/304](https://github.com/databus23/helm-diff/pull/304) (this might fix the problem in helm-diff but it was not accepted) ## Context | **Architecture** | **Detect Changes to Non-chart YAML Values?** | **Detect Changes to Local Chart Files?** | **Detect Manual Changes to Deployed Resources?**(e.g. `kubctl edit`) | | -------------------------------------- | -------------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------- | | `helm_release` with `manifest=true` | Yes | Yes | No | | `helm_release` without `manifest=true` | Yes | No | No | | `kubernetes_manifest` | Yes | No | No | | `helmfile_release` | Yes | No | No | ### Testing Methodology :::caution Note, changing the port in a running service is not a good test as it fails even with `kubectl apply` ::: #### Part 1: `echo-server` - Modify any value from any of the local template files (YAML files within `echo-server/charts/echo-server/`). Then, check that that change is detected by terraform. - Modify any value in `default.auto.tfvars`. Then, check that that change is detected by terraform. - Modify any deployed resource via `kubectl edit` and observe that that change is not detected by Terraform. #### Part 2: `cert-manager` (using the `cert-manager` component from the `>v0.185.1` releases of `cloudposse/terraform-aws-components`) - With `letsencrypt_enabled: true` and `cert_manager_issuer_selfsigned_enabled: false`, modify any value in `cert-manager-issuer/templates/letsencrypt-issuer.yaml`. Then, check that that change is detected by terraform. - With `letsencrypt_enabled: true` and `cert_manager_issuer_selfsigned_enabled: false`, modify any value in either `cert-manager-issuer/templates/selfsigning-certificate.yaml` or `cert-manager-issuer/templates/selfsigning-issuer.yaml`. Then, check that the change is not detected by Terraform, because these files will not be rendered in the deployed helm component (due to the if statements at the top of them). - Modify any value in `default.auto.tfvars`. Then, check that that change is detected by terraform. - Modify any deployed resource via `kubectl edit` and observe that that change is not detected by Terraform. ## Considered Options ### Option 1: Helm Provider `manifest=true` ### Option 2: Helm Provider with Kubernetes Provider ### Option 3: Helmfile Provider ### Option 4: ArgoCD Provider (experimental) ### Option 5: Helm Provider with Git Provider and ArgoCD (experimental) ### Option 6: Stop using Helm? 🙂 ## Decision **DECIDED**: Use the Terraform `helm` provider with the `manifest=true` flag. ## Consequences - We now have solved all the problems that motivated this design choice, and we have not sacrificed any drift detection, relative to using helmfiles for deployment. ## References - [https://github.com/cloudposse/terraform-aws-components/pull/381](https://github.com/cloudposse/terraform-aws-components/pull/381) --- ## Use Vendoring in Atmos **Date**: **21 Mar 2022** :::tip Latest! The content in this ADR is up-to-date! For questions, please reach out to Cloud Posse ::: ## Status **DECIDED** ## Problem We need a way to centralize cloudposse components for reuse across all customers. We have `cloudposse/terraform-aws-components`, but we do not use it as a source of truth. As a result, maintaining our vast library of components is challenging. We need some way to discover components to avoid duplication of effort. Additionally, we need some way to easily create new components (e.g. from a template). Also related to [Proposed: Use Mixins to DRY-up Components](/resources/adrs/proposed/proposed-use-mixins-to-dry-up-components) and [Proposed: Use Atmos Registry](/resources/adrs/proposed/proposed-use-atmos-registry) ## Context ## Considered Options ### Option 1: New configuration spec In the component directory, place a file like this to specify attribution. #### Components ``` # component.yaml — proposed name, up for debate # This configuration is deep-merged with the upstream component's configuration (component.yaml). source: # this stanza is omitted in the component.yaml of the upstream component type: git uri: github.com/cloudposse/terraform-aws-components.git//modules/argocd version: 1.2.3 ``` Similarly, in the component’s upstream repository (e.g. `cloudposse/terraform-aws-components`), we will distribute a file like this in each component. ``` # source: # type: git # uri: github.com/cloudposse/terraform-aws-components.git//modules/argocd # version: 1.2.3 ``` #### Mixins The file can also define any mixins that are to be downloaded and generated as part of this component. See also [Proposed: Use Mixins to DRY-up Components](/resources/adrs/proposed/proposed-use-mixins-to-dry-up-components) for a use-case. ``` # component.yaml — proposed name, up for debate mixins: context.tf: source: github.com/cloudposse/null-label.git//exports/context.tf # also supports local paths version: 0.25.0 filename: context.tf ``` Multiple mixins can be defined and parameterized. The parameterization will be based on Go templates. ``` mixins: infra-state: source: github.com/cloudposse/terraform-aws-components.git//mixins/infra-state.mixin.tf version: 1.2.3 filename: mixin.infra-state.tf # we should probably move to the prefix convention as a default parameters: # anything that needs to be interpolated when the file is created monorepo_uri: git::ssh://git@github.com/ACME/infrastructure.git?ref=0.1.0 sops: source: github.com/cloudposse/terraform-aws-components.git//mixins/sops.mixin.tf version: 1.2.3 filename: mixin.sops.tf ``` These could also be defined in the component’s upstream repository (e.g. `cloudposse/terraform-aws-components`) ``` # source: # type: git # uri: github.com/cloudposse/terraform-aws-components.git//modules/argocd # version: 1.2.3 mixins: context.tf: source: github.com/cloudposse/null-label.git//exports/context.tf # also supports local paths version: 0.25.0 filename: context.tf ``` A state file/manifest (not to be confused with a Terraform state file) needs to be created whenever `atmos` pulls down the mixins and components. This state file keeps track of which files were vendored and their checksums. This is so that we can identify which files came from the upstream versus files that already existed locally, and also so that we can determine if someone modifies the files upstream, and so we know when to update the mixins. ### Option 2: Extend the Stack Configuration This was the direction we thought we would take, but there’s a fundamental flaw with this approach that makes it complicated to implement: inheritance. In configuration, if multiple stacks were to define different sources of the component, which component is used? We would need to download all versions of them and couldn’t have the neat & clean layout in `components/terraform`. Thus we prefer Option 1. The inheritance issue can be mitigated by using the `metadata` section of the component configuration which is excluded from the inheritance chain. ### Option 3: Git Submodules - Complicated syntax - Doesn’t support adding files to the downstream tree (e.g. `backend.tf.json`) ### Option 4: Git Subtrees - Complicated syntax - Would be compatible with additional files added to the tree and local modifications ### Option 5: Vendir [https://github.com/vmware-tanzu/carvel-vendir](https://github.com/vmware-tanzu/carvel-vendir) Vendir was our original inclination. It’s a powerful tool, but supports too many features we don’t need and it would be hard to add what we want, since we’re not in control of the project. [https://github.com/vmware-tanzu/carvel-vendir/issues/29](https://github.com/vmware-tanzu/carvel-vendir/issues/29) [https://github.com/vmware-tanzu/carvel-vendir/pull/64](https://github.com/vmware-tanzu/carvel-vendir/pull/64) ## Decision **DECIDED**: Use Option 1 - a new configuration specification ## Consequences - Update atmos to add support for Option 1. ## References - --- ## Architectural Design Records (ADRs) import DocCardList from '@theme/DocCardList' These are the records of why Cloud Posse made various decisions in the design of the Reference Architecture. These decisions may differ from what your organization has decided due to going through the "Design Decisions" process. Send requests for additional documentation in [GitHub Discussions](https://github.com/orgs/cloudposse/discussions). ## Records --- ## Deprecated Architecture Decision Records import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; These are a collection of architectural design records that we no longer ascribe to or have abandoned for various reasons. --- ## Use Confluence for Customer Documentation :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - We are no longer using Confluence. Now all documentation is included with [docs.cloudposse.com](https://docs.cloudposse.com/) and formatted with Docusarus. ::: ## Context Cloud Posse maintains a significant amount of documentation sprawled around our public open source repos, customer repos, and platforms (e.g. LucidChart, Google Docs, GitHub, YouTube). We struggle to write and keep the documentation up to speed, communicate changes to the documentation, and disseminate the documentation. More than any other company, we integrate dozens and dozens of tools and technologies. The landscape is also continuously changing beneath our feet, forcing us to make changes and adapt. ## Options Considered ### Confluence (Decided) We debated for a long time whether or not to use Confluence, and for a long time were very much against the product because it felt clunky/slow and lacked the features of more modern documentation systems. Over the past couple of years, after significant investment by Atlassian into the product it is much faster, less clunky, and added essential features such as commenting on phrases and supports live editing. #### Pros - Extensions for most things we want (PlantUML, embeds, iframes, LucidCharts, etc) - Tightly integrates into Jira, our Workflow Management platform and where we keep our reference architecture - Updates to documentation can be worked on in a draft mode before publishing - Accessible to all customers immediately (more scalable) - Most of our customers use Atlassian products (Jira, Confluence, Service Desk) - APIs exist for updating documentation and pulling it out - We can archive pages without deleting them - Zapier automation enables us to take action when things change - Native [workflow automation coming soon](https://www.youtube.com/watch?v=N8WCeJldvFs) - We’re already paying for the additional seats - We can embed markdown content from anywhere, including our other repos using [Atlassian Marketplace Markdown Macro for Confluence](https://marketplace.atlassian.com/apps/1211438/markdown-macro-for-confluence?hosting=cloud&tab=overview) ([see example](/components/library/aws/account-settings/)) - Fully managed solution we do not need to host #### Cons - Cost per seat is not insignificant - Does not natively export content to Markdown or GitHub (however, tools exist to facilitate this and we may write one eventually specifically for our reference architecture) - Versions of documentation do not necessarily relate to a customer’s version of the software. For customers who are sensitive to this, we encourage vendoring relevant portions of documentation. - Confluence does not have a native desktop client for Mac or PC - No support for redlining or “suggest” like functionality that the author can simply accept; the workaround is to leave a comment on the section with the suggested changes ### Open Source All Documentation We strongly considered open-sourcing all of our documentation, which would provide anyone the ability to implement and use our reference architecture. Our plan is instead to eventually open-source portions of our documentation but to not let that hold us up and start aggregating it in one place. We decided against this because for the reasons below. #### Pros - One place to update the documentation for all of our customers and community - Radically increase the adoption of our process and methodology - Receive contributions via GitHub Pull Requests - Use our existing documentation infrastructure (hugo) #### Cons - No way to easily wall off portions of our documentation based on our current documentation infrastructure with Hugo and S3. - We already receive an overwhelming amount of contributions and requests from our community. If our documentation were all public, it would increase our support burden while decreasing the time we focus on customers - Difficult to share more sensitive or private information with our customers (contact information, architectural diagrams). We want to be as forthcoming as possible and make it as easy as possible to prioritize the support our customers need, over worrying about what we can publish publicly. ### Use GitHub with Markdown We started delivering documentation to all customers via Markdown in GitHub. The problem is organizing the documentation leaves a lot to be desired. #### Pros - Easily vendored into customer repositories - Use GitHub Pull Request / Code Review process - Technology neutral solution - Distribute documentation easily and securely with Private GitHub Pages #### Cons - Private Github Pages are only supported by GitHub Enterprise, which most of our customers do not use - Hosting a private documentation portal (e.g. Hugo) is even more opinionated since most customers already host documentation in some system. Plus it would require some form of authentication. - Using markdown by itself is very limiting and incorporating screenshots, images, diagrams is very tedious - since exported images are quickly out of date. - Opening Pull Requests is arguably much slower and a larger barrier to contribution ### Vendor All Documentation We have always wanted to help customers by providing documentation in their systems, but trying to serve our customers this way has not scaled with our rate of growth and the number of integrations we support. This is similar to [Use GitHub with Markdown](#) #### Pros - Customers have a version of the documentation that closely matches exactly what is deployed - Infrastructure code and documentation are alongside each other - Customers control changes to documentation #### Cons - No practical way to syndicate documentation and changes across customers - Customers miss out on corrections, updates, and improvements - Cost to customers is significantly greater ### Notion Notion is a very polarizing system. Many users using Notion came from some other system like Confluence, Evernote or Quip. We felt massive FOMO not jumping ship to Notion, but decided against it for the reasons below. #### Pros - It supports some very nice ways to provide documentation; it’s somewhere in-between Evernote meets Airtable - Integrates with systems like Jira - Nice cross-platform Desktop application #### Cons - It’s another vendor and would require additional cost-per-seat to share - Until recently, it provided no API whatsoever. - Very few of our customers use Notion compared to the alternatives - @Erik Osterman cannot stand all the cheesy UTF-8 emojis sprinkled everywhere on every page. 🚀 😵 💩 ## Decision - Use our `REFARCH` space in Confluence to aggregate, share, and disseminate documentation ## Consequences - Share `REFARCH` space with all customers --- ## Use Folder Structure for Compliance Components **Date**: **21 Mar 2022** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - We have since refactored our Compliance components. For more see, [Foundational Benchmark Compliance](/layers/security-and-compliance/). ::: ## Status **DECIDED** ## Problem - Too many files clutter the stack config folders - Files are very terse and seldom edited - So many files that code generation of YAML is the only practical way of managing them ## Context ## Considered Options ### Option 1: Use Virtual Components Define one file for each default region (17). Having multiple files is convenient for GitOps and detecting what files changed in order to trigger CI. ``` # stacks/catalog/compliance/ue1.yaml components: terraform: compliance-ue1: component: compliance vars: region: us-east-1 environment: ue1 ``` The root component should follow the same pattern, with one file for each default region. ``` # stacks/catalog/compliance/root/ue1.yaml components: terraform: compliance-root-ue1: component: compliance-root vars: region: us-east-1 environment: ue1 ``` :::caution There’s one downside with this naming convention, is that the final stack names (e.g. in Spacelift) will look like `acme-ue1-root-compliance-ue1`, with the `ue1` being repeated in the `name` because it is pulled from the `component`. This will be fixed in . ::: Define a baseline that imports all 17 default regions ``` # stacks/catalog/compliance/baseline.yaml imports: - catalog/compliance/ue1 - catalog/compliance/ue2 ... ``` Repeat for the “root” baseline should import all 17 default regions ``` # stacks/catalog/compliance/root/baseline.yaml imports: - catalog/compliance/root/ue1 - catalog/compliance/root/ue2 ... ``` Then in each account-level stack configuration, import the compliance baseline. Here are some examples: ``` # stacks/globals.yaml imports: - catalog/compliance/baseline ``` ``` # stacks/plat/prod.yaml imports: - globals ``` ``` # stacks/plat/staging.yaml imports: - globals ``` ``` # stacks/core/security.yaml imports: - globals ``` ``` # stacks/core/dns.yaml imports: - globals ``` The root account is the only exception, which would look like this: ``` # stacks/core/root.yaml imports: - globals - compliance/root/baseline ``` ### Option 2: Use YAML Separators ``` # mock stack config stages: - a - b - c components: terraform: compliance: vars: region: us-east-1 environment: ue1 --- # mock stack config stages: - a - b - c components: terraform: compliance: vars: environment: ue2 region: us-east-1 --- # mock stack config stages: - a - b - c components: terraform: compliance: vars: region: us-west-1 environment: uw1 ``` ### Option 2: Current Solution ``` ### ### stack: stacks/mdev/euw3/mdw3-audit.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml ### ``` ``` import: - mdev/mdev-globals - euw3/euw3-audit ``` ``` ### ### stack: stacks/mdev/mdev-globals.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml > stacks/mdev/mdev-globals.yaml ### import: - globals vars: tenant: mdev terraform: backend: s3: bucket: "vygr-mdev-use2-root-tfstate" dynamodb_table: "vygr-mdev-use2-root-tfstate-lock" role_arn: "arn:aws:iam::807952753552:role/vygr-mdev-gbl-root-terraform" remote_state_backend: s3: bucket: "vygr-mdev-use2-root-tfstate" dynamodb_table: "vygr-mdev-use2-root-tfstate-lock" role_arn: "arn:aws:iam::807952753552:role/vygr-mdev-gbl-root-terraform" settings: spacelift: worker_pool_name: vygr-mdev-use2-auto-spacelift-worker-pool ``` ``` ### ### stack: stacks/globals.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml > stacks/mdev/mdev-globals.yaml > stacks/globals.yaml ### vars: namespace: vygr required_tags: - Team - Service tags: # We set the default team here, this means everything will be tagged Team:sre unless otherwise specified. # This is used because it is the default alerted team. Team: sre terraform: vars: label_order: ["namespace", "tenant", "environment", "stage", "name", "attributes"] descriptor_formats: stack: format: "%v-%v-%v" labels: ["tenant", "environment", "stage"] # This is needed for the transit-gateway component account_name: format: "%v" labels: ["stage"] backend_type: s3 # s3, remote, vault, etc. backend: s3: encrypt: true key: "terraform.tfstate" acl: "bucket-owner-full-control" region: "us-east-2" remote_state_backend_type: s3 # s3, remote, vault, etc. remote_state_backend: s3: encrypt: true key: "terraform.tfstate" acl: "bucket-owner-full-control" region: "us-east-2" ``` ``` ### ### stack: stacks/euw3/euw3-audit.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml > stacks/euw3/euw3-audit.yaml ### import: - euw3/euw3-globals vars: stage: audit terraform: vars: {} helmfile: vars: {} components: terraform: compliance: settings: spacelift: workspace_enabled: false aws-inspector: settings: spacelift: workspace_enabled: false ``` ``` ### ### stack: stacks/euw3/euw3-globals.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml > stacks/euw3/euw3-audit.yaml > stacks/euw3/euw3-globals.yaml ### import: - catalog/compliance/compliance # - catalog/aws-inspector # @TODO aws-inspector is not yet supported in Paris, it is likely this will change in the future # https://docs.aws.amazon.com/inspector/v1/userguide/inspector_rules-arns.html vars: region: eu-west-3 environment: euw3 components: terraform: vpc: vars: availability_zones: - "eu-west-3a" - "eu-west-3b" - "eu-west-3c" eks/eks: vars: availability_zones: - "eu-west-3a" - "eu-west-3b" - "eu-west-3c" ``` ``` ### ### stack: stacks/catalog/compliance/compliance.yaml ### chain: stacks/mdev/euw3/mdw3-audit.yaml > stacks/euw3/euw3-audit.yaml > stacks/euw3/euw3-globals.yaml > stacks/catalog/compliance/compliance.yaml ### components: terraform: compliance: settings: spacelift: workspace_enabled: true vars: enabled: true tags: Team: sre Service: compliance config_bucket_env: use2 config_bucket_stage: audit config_rules_paths: - https://raw.githubusercontent.com/cloudposse/terraform-aws-config/0.14.1/catalog/account.yaml ... central_logging_account: audit central_resource_collector_account: security cloudtrail_bucket_stage: audit cloudtrail_bucket_env: use2 create_iam_role: true global_resource_collector_region: us-east-2 guardduty_admin_delegated: true securityhub_admin_delegated: true securityhub_create_sns_topic: true securityhub_enabled_standards: - ruleset/cis-aws-foundations-benchmark/v/1.2.0 securityhub_opsgenie_sns_topic_subscription_enabled: false # TODO, enable this once /opsgenie/opsgenie_securityhub_uri SSM param is set securityhub_opsgenie_integration_uri_ssm_account: corp securityhub_opsgenie_integration_uri_ssm_region: us-east-2 default_vpc_deletion_enabled: true az_abbreviation_type: short ``` ## Decision **DECIDED**: Use **Option 1** with virtual components ## Consequences - Refactor configurations to be more DRY - to avoid duplication of the environment in stack names ## References - [Compliance Setup](/layers/security-and-compliance/) --- ## Use IPAM for IP Address Management and Allocation **Date**: **29 Apr 2022** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - Too expensive without significant customer interest or value. ::: ## Status **DRAFT** ## Problem ## Context It’s not required to create subpools. Subpools are used when you need a logical grouping. Management of Subnets of a VPC - Large enterprises want to do as much route aggregation as possible. ### Today Today, without IPAM, for existing clients, we manage “pools” in terraform using straight subnet math: - Pool: One supernet for the AWS organization - Pool: Per account - Pool: Per region VPC (with typically only one VPC per account, per region) - Per availability zone - public - private ### Future We propose managing pools in a similar manner, but introducing a pool for the OU. - One supernet for the AWS organization - Per OU - Per region - Per VPC (with typically only one VPC per account, per region) - final pool - Per availability zone - all AZ subnets are siblings of each other, and children of the VPC - public - private The more pools we create, the hard it is to leverage route aggregation. ### Use-case: Grant VPN Access in Zscaler to all non-production networks ### Use-case: Production VPC has reached 90% capacity in us-east-1 and need to add IPs ### Use-case: New production account added and needs VPCs in 2 regions Proposal 1 ``` components: terraform: # manage the organization's IPAM ipam: vars: organization_admin_account: network organization_pool_cidr: 10.0.0.0/8 operating_regions: - name: ue1 cidr: 10.0.0.0/12 - name: ec1 cidr: 10.16.0.0/12 - name: ap1 cidr: 10.32.0.0/12 pools: - name: ue1-phi-data cidr_range: 10.0.0.0/13 parent: ue1 - name: ue1-non-phi-data cidr_range: 10.8.0.0/13 parent: ue1 - name: ec1-phi-data cidr_range: 10.16.0.0/13 parent: ec1 - name: ec1-non-phi-data cidr_range: 10.24.0.0/13 parent: ec1 - name: ap1-phi-data cidr_range: 10.32.0.0/13 parent: ap1 - name: ap1-non-phi-data cidr_range: 10.40.0.0/13 parent: ap1 ``` ## Considered Options ### Option 1: ### Option 2: ### Option 3: ## Decision **DECIDED**: ## Consequences - ## References - --- ## Jumpstart Design Records import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; These are the design records for the Jumpstart architecture. They are helpful for understanding some of the more low-level aspects of our Jumpstart implementation. --- ## Proposed: Atmos Workflows v2 **Date**: **26 Jan 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal has been already been adopted, and this ADR needs to be updated to reflect the final decision. Cloud Posse recommends use of Workflows generally with [Atmos Workflows](https://atmos.tools/core-concepts/workflows/) and specifically with the Reference Architecture. ::: ## Status **DRAFT** ## Problem In the original `variant` version of `atmos` we had the concept of workflows. These were a simple set of steps that could be executed in order to bring up an environment or execute some kind of operation. When we ported `atmos` to Golang, we didn’t carry over this functionality because it was seldom used as implemented. Updating a workflow with all the steps was cumbersome. If a workflow failed, there was no way to restart back at the last failed step. To define a workflow to build and destroy an environment, would require defining two workflows (E.g. `create-env` and `destroy-env`). ## Context ## Considered Alternatives - Use `make` (or other task runner) to call `atmos` - Use shell scripts ## Other examples - See `astro` by Uber (abandoned) [https://github.com/uber/astro](https://github.com/uber/astro) - Atlantis workflows: [https://www.runatlantis.io/docs/custom-workflows.html#use-cases](https://www.runatlantis.io/docs/custom-workflows.html#use-cases) - Terragrunt dependencies [https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/](https://terragrunt.gruntwork.io/docs/features/execute-terraform-commands-on-multiple-modules-at-once/) ## Considered Options ### Option 1: Maintain the exact same interface ``` import: [] vars: {} terraform: vars: {} helmfile: vars: {} components: terraform: fetch-location: vars: {} fetch-weather: vars: {} output-results: vars: {} print_users_weather_enabled: true helmfile: {} workflows: deploy-all: description: Deploy terraform projects in order steps: - job: terraform deploy fetch-location - job: terraform deploy fetch-weather - job: terraform deploy output-results ``` ### Option 2: Workflows with parameters ``` import: [] vars: {} terraform: vars: {} helmfile: vars: {} components: terraform: fetch-location: vars: {} fetch-weather: vars: {} output-results: vars: {} print_users_weather_enabled: true helmfile: {} workflows: deploy-all: description: Deploy terraform projects in order steps: - subcommand: terraform apply fetch-location vars: enabled: true - subcommand: terraform apply fetch-weather - subcommand: terraform apply output-results destroy-all: description: Destroy terraform projects in order steps: - subcommand: terraform apply output-results vars: enabled: false - subcommand: terraform apply fetch-weather vars: enabled: false - subcommand: terraform apply fetch-location vars: enabled: false ``` ### Option 3: Support native dependencies between components and a `--reverse` flag First, we add an an official `depends-on` field to our stack configuration. In this configuration `echo` → `vpc` → `eks` → `external-dns` → `cert-manager` ``` components: terraform: echo: metadata: type: abstract hooks: before: - echo "Hello world!" vpc: component: vpc depends-on: echo eks: component: eks depends-on: vpc external-dns: depends-on: eks hooks: before-deploy: - sleep 100 cert-manager: depends-on: external-dns alb-controller: depends-on: eks ``` Provision the eks component’s workflow (and everything that depends on `eks`) ``` atmos workflow terraform apply eks ``` Decommission the eks component ``` atmos workflow terraform apply eks --reverse ``` Provision everything that depends on `eks` , which will deploy `external-dns` and then `cert-manager` ``` atmos workflow terraform apply --depends-on external-dns ``` ### Option 4: Leverage Existing Go-based Task Runner Framework Use something like `gotask` to add rich support into atmos stack configurations, without reinventing the wheel. gotask: [https://taskfile.dev/#/](https://taskfile.dev/#/) variant: [https://github.com/mumoshu/variant](https://github.com/mumoshu/variant) mage: [https://github.com/magefile/mage](https://github.com/magefile/mage) ``` # Gotask example tasks: deploy-all: cmds: - echo 'Hello World from Task!' - atmos terraform apply eks - sleep 100 - atmos terraform apply external-dns silent: true ``` ## Decision **DECIDED**: ## Consequences - ## References - --- ## Proposed: Distribution Method for GitHub Actions **Date**: **22 Jun 2022** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - After this proposal was created, GitHub announced support for reusable workflows. ::: ## Status **DRAFT** ## Problem We need a reliable way to distribute GitHub Actions workflows to repos and keep these workflows up to date with any changes made to the GitHub Actions themselves. An example of such a workflow might be [https://github.com/cloudposse/github-action-ci-terraform/blob/main/.github/workflows/ci-terraform.yml](https://github.com/cloudposse/github-action-ci-terraform/blob/main/.github/workflows/ci-terraform.yml). Here are 6 factors to keep in mind when designing a solution: 1. Customers (not just Cloud Posse) need to be able to use this solution to both initialize actions in their repos and update their actions later on. 2. We need our solution to be able to update all of our repos easily. 3. Not all repos need the same workflows; some repos might need all of our GitHub Actions workflows, but others might only need a subset. Our solution should distribute workflow files accordingly. (E.g., some actions might be specific to Terraform projects, and non-Terraform repos don’t need these.) 4. Our solution needs to include a (possibly, one-time) strategy for pushing out actions en masse to our Cloud Posse repos (e.g., git-xargs). 5. As a rule, too many PRs can be noisy, so ideally our solution will minimize the number of PRs needed to keep things up to date. 6. We want our solution to auto-merge PRs if the tests pass. ## Context I, Dylan, have been writing a bunch of CI/CD-related GitHub Actions ([https://github.com/cloudposse/github-action-ci-terraform](https://github.com/cloudposse/github-action-ci-terraform), [https://github.com/cloudposse/github-action-auto-format](https://github.com/cloudposse/github-action-auto-format), [https://github.com/cloudposse/github-action-auto-release](https://github.com/cloudposse/github-action-auto-release), [https://github.com/cloudposse/github-action-validate-codeowners](https://github.com/cloudposse/github-action-validate-codeowners)) and other people, like @Igor Rodionov have been working on GitHub Actions, too. We need a reliable distribution and maintenance strategy for them in order for them to be usable. As it is now, the easiest way to add them to a repo is for someone to manually copy a workflow file from each action repo into the repo of interest. For maintenance, the state of the art is manually checking whether there have been updates to each GitHub Action repo. Needless to say, these strategies could be improved on. ## Considered Options ### Option 1: Pull distribution (w/ or w/o centralized workflow file repo) The key here is creating a GitHub Action whose whole purpose is to distribute and keep current the workflows of all other GitHub Actions in all repos: `github-action-distributor`. (Very similar actions already exist, e.g., [https://github.com/marketplace/actions/repo-file-sync-action](https://github.com/marketplace/actions/repo-file-sync-action) and [https://github.com/marketplace/actions/push-a-file-to-another-repository](https://github.com/marketplace/actions/push-a-file-to-another-repository).) In this proposal, the `github-action-distributor` would copy all GitHub Actions workflows directly from their home repos (e.g., [https://github.com/cloudposse/github-action-validate-codeowners/blob/main/.github/workflows/validate-codeowners.yml](https://github.com/cloudposse/github-action-validate-codeowners/blob/main/.github/workflows/validate-codeowners.yml)) to their destination repos (e.g., [https://github.com/cloudposse/terraform-example-module](https://github.com/cloudposse/terraform-example-module)). - **One-time, internal org-level distribution strategy:** - Use a `git-xargs` command to distribute a GitHub Actions workflow for a `github-action-distributor`. The purpose of the `github-action-distributor` is to propagate all the appropriate GitHub Actions workflows to a repo and keep them up to date using a cron job. - **Customer repo-level distribution strategy:** - Customers can manually (or using a tool like `git-xargs`, I suppose) distribute the `github-action-distributor` workflow to each repository they would like to add GitHub Actions workflows to. - **Internal and customer repo-level update strategy:** - Whenever new a new version of a GitHub Actions workflow is released, a change can be made to the `github-action-distributor` (using `renovate.json` ideally, or manually) to distribute the new version of that workflow from now on, and this change will be reflected in all downstream repos the next time their `github-action-distributor` cron job runs. Alternatively, we can pin the version of the `github-action-distributor` action used in a given repo, so that the versions of all GitHub Actions workflows in that repo are known and stable. There are two **variants** of this option, depending on whether we copy the workflow files for the GitHub Actions into their own repo: 1. We could have action maintainers manually copy the workflow file(s) for the action(s) they maintain to a centralized repo (e.g., `cloudposse/actions`) and have the `distributor` action pull whatever is in that repo into the end-user repos. In order to update what workflows are being distributed, someone would just copy new workflows to the centralized workflow repo. 2. We could have the `distributor` action pull workflows from their home repos (e.g., pulling `.github/workflows/auto-format.yml` from `cloudposse/github-action-auto-format`). In order to update what workflows are being distributed, someone would update the version tags that would be hardcoded into the `distributor` action. **NB:** It should be possible to implement the `github-action-distributor` as just a piece of functionality within a larger, existing GitHub Action. For example, the `github` [target](https://github.com/cloudposse/github-action-auto-format/blob/main/scripts/github/format.sh) inside the `auto-format` action essentially fulfills this role right now by copying the desired workflows from the `cloudposse/.github` repo. ### Option 2: Push distribution (w/ or w/o centralized workflow file repo) Similar to option 1 above, there would be a GitHub Action, `github-action-distributor`, whose purpose is to distribute GitHub Action workflow files to end-user repos. Also similar to Option 1, this action would be compatible with centralized and decentralized workflow organizational strategies (see “**variants**” above). In this proposal, though, the `distributor` would behave differently. Instead of being added to each end-user repository (e.g., `cloudposse/terraform-aws-components`, `cloudposse/build-harness`, `cloudposse/atmos`, etc.), it would only run in one repository (the centralized workflow file repo, in the case where we use that strategy), or in a small number of repositories (each of the `github-action-*` repositories, if we decline to use a centralized workflow file repo). Whenever a workflow file is updated inside a repo that has the `distributor` action added, that updated workflow will be pushed out to either all `cloudposse/*` repos, or a logical subset of them, depending on the specific action. The net result is that PRs would be opened in the end-user repos and automatically merged, all by the `distributor` action. This option comes with the advantage, relative to Option 1, of being much simpler for Cloud Posse to bootstrap, since the manual distribution of the `distributor` workflow file is limited to < ~10 repos, enough to easily be done by hand. However, the bootstrapping process for third-parties would be non-existent. They would need to find their own methods, likely by implementing something like Option 1 for themselves. ### Option 3: Using internal GitHub functionality It looks like there may be a way to distribute sample actions to an org’s repos via the `[org]/.github` repo, but this functionality is not well-documented, if it does exist, and even if it does, it probably requires opening a number of PRs on each repo to bootstrap. [If there is more interest in this, I (Dylan) can look into it further.) One point worth noting is that this approach would lead to the same workflows being distributor to all (or nearly all) repos in the `cloudposse` GitHub org. This means that all actions/workflows need to detects early as possible whether they’re going to do anything useful on a given repo (e.g., running `terraform/fmt` would be completely unnecessary in a non-Terraform repo) and exit asap, to not tie up GitHub runners unnecessarily. ## Decision **DECIDED**: ## Consequences - ## References - - --- ## Proposed: Spacelift Admin Stack Architecture **Date**: **19 Oct 2021** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - Cloud Posse refactored our Spacelift implementation with [PR #717](https://github.com/cloudposse/terraform-aws-components/pull/717) and since updated the documentation included with the components. [More on Spacelift](/components/library/aws/spacelift/). ::: ## Status **DRAFT** ## Problem 1. Human error - Stages that do not have spacelift access, will accidentally have spacelift enabled for stacks in those stages. This is due to human error which is an easy mistake to make. 2. High blast radius - Spacelift admin stacks have significant blast radius as we have a single admin stack that controls all of the infrastructure in an entire organization 3. Admin stack errors - Spacelift admin stack may error out in the middle of the apply due to one of the stacks currently in use, preventing the following stacks from modification. This requires rerunning the admin stack. 4. High priority fixes can be queued - Need to get a high priority fix out and if the change is in `globals.yaml` or similar import and we want the fix to be deployed to prod first, how would we prioritize that up the chain instead of waiting for all the queued stacks to be finished first. No sense of priority. 5. Same policies and configurations that’s associated with every spacelift stack 6. All stacks cannot be shown in Spacelift ## Context ## Considered Options ### Option 1: Single admin stack Pros: - Consistency Cons: - Significant blast radius (problem 2) - Forces a single worker pool (problem 4) - Same policies and configuration that’s associated with every spacelift stack (problem 5) ### Option 2: Multi admin stack Segment on `-` We can use [var.context_filters](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation#input_context_filters) on the spacelift component to explicitly say that we only want to capture spacelift stacks for stages like `auto`, `corp`, `dev`, `staging`, etc. Pros: - Using context_filters, we can capture specific stages for specific admin stacks. e.g. admin stack for `auto` can use a filter for only `auto` stacks (solves problem 1) - Limited blast radius (solves problem 2) - Reduces admin stack issues (reduces problem 3) - Option to use multiple worker pools so for high priority items you can have a separate worker pool for prod vs dev (solves problem 4) - Allows policy and configuration on a per stage basis (solves problem 5) Consequences - To make admin stack creations easier, we would need to codify the spacelift admin stack and reuse the [stack submodule](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/tree/master/modules/stack) ### Option 3: Read-write only policies Pros: - Consistency Cons: - Have to ensure that security, audit, root, identity do not have spacelift stacks enabled (problem 6) ### Option 4: Read-write policies for some and Read for others Option 2 (multiple admin stacks) could be a prereq for this however, with some terraform magic, we could add different policies for different accounts. Pros: - All stacks can be shown in spacelift (solves problem 6) - Allows read-write policies for dev, qa, auto, corp, etc - Allows read for security, audit, root, identity Cons: - If went with a single admin stack, this would require changes to the cloudposse spacelift module to change policies based on some input. ### Option 5: Single worker pool Pros: - Single worker pool so costs don’t have to be managed across multiple ASGs Cons: - High priority changes may be queued behind other changes (problem 4) ### Option 6: Multiple worker pools This is currently solved in one of our customers using multiple admin stacks. Option 2 (multiple admin stacks) could be a prereq for this however, with some terraform magic, we could add different policies for different accounts. Pros: - High priority changes can be delivered faster if changes need to go into prod first. Prod can have its own worker pool and non-prod can all share a separate pool. Cons: - Costs could get out of control if too many worker pools. More workers, higher cost. - If went with a single admin stack, this would require changes to the cloudposse spacelift module to change worker pools based on some input. ### Option 7: Combination - Multi admin stacks, read&write policies for some and read policies for others, and multiple worker pools We could do option 2, have multiple admin stacks to solve problems 1 to 5. We could do option 4, have read/write policies for and read only for others to solve problem 6. We could do option 6, multiple worker pools. One worker pool for prod (min 1, max 10) and one worker pool (min 1, max 10) for all others. This solves problem 4. ## Decision **DECIDED**: Loose decision for cplive - Organize admin stacks around teams because - overall we’re adopting a strategy where components are organized around teams where it makes sense e.g. opsgenie-team, datadog, and soon iam - Teams are an easy construct for people to grok - (optional) single spacelift worker pool - ideally each admin stack associated with a dedicated worker pool - worker pools can more easily be granted more narrowly scoped iam roles e.g. security stack mapped to security team with a security worker pool - Spacelift is introducing spaces (end of july 2022) which map to teams which map to worker pools ## Consequences - - - ## References - --- ## Proposed: Use Atmos Registry :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - Cloud Posse uses the [terraform-aws-components](https://github.com/cloudposse/terraform-aws-components) repository with vendoring similarly to how a registry might behave. For more on vendoring, see [atmos vendor pull](https://atmos.tools/cli/commands/vendor/pull). ::: ## Problem We need a way to centralize cloudposse components for reuse across all customers. We have cloudposse/terraform-aws-components, but we do not use it as a source of truth. As a result, maintaining our vast library of components is challenging. We need some way to discover components to avoid duplication of effort. Providing a registry is a common characteristic among successful languages or tools (E.g. `python` has PyPi, `perl` has CPAN, `ruby` has RubyFoge, `docker` has DockerHub, `terraform` has the Terraform Registry). Additionally, we need some way to easily create new components (e.g. from a template). ## Solution Implemnet a GitHub based “registry” for components, stacks and mixins. Use a generator pattern (e.g. like cookiecuter), but make it natively built-in to atmos. (E.g. see [https://github.com/tmrts/boilr](https://github.com/tmrts/boilr) for inspiration, but anything we do should be a re-implementation with a very nice UI). ### TODO: inconsistencies - Mixins (YAML vs Terraform) - Registry usage (search, generating components) ### TODO: explanations - why even care about templating? - why add components to `--stack` ### Use-case #1: Pull down existing components Imagine the command like this... ``` atmos component generate terraform/aurora-postgres \ --stack uw2-dev \ --source cloudposse/terraform-aws-components//modules/aurora-postgres \ --version 1.2.3 ``` 1. It will download the component 2. It will prompt the user for any information needed, providing sane defaults. It will save the user’s answers, so subsequent generations persist state. 3. It will add a component configuration to the `uw2-dev` stack if none found 4. User commits to VCS ``` atmos component generate terraform/eks \ --source cloudposse/terraform-aws-components//modules/eks --version 1.2.3 atmos stacks generate stacks/catalog/eks \ --source cloudposse/terraform-aws-stacks//catalog/eks-pci --version 1.2.3 ``` ### Use-case #2: Initialize a new component from the component template for AWS This will create a new component from some boilerplate template and add it to the `uw2-dev` stack. ``` atmos component generate terraform/my-new-component \ --stack uw2-dev \ --source cloudposse/terraform-aws-component-template ``` ### Use-case #3: Mixins This will create a `context.tf` in the `components/my-new-component` ``` atmos mixins generate components/terraform/my-new-component/context.tf \ --source cloudposse/terraform-aws-components/mixins/context.tf ``` ### Use-case #4: List & Search for Components, Stacks and Mixins. The `--filter` argument is used to filter the results. Search for all EKS components. ``` atmos component registry list --filter eks ``` Search for all EKS stacks ``` atmos stack registry list --filter eks ``` Search for all mixins ``` atmos mixin registry list --filter context ``` ### Use-case #5: Add registries ``` atmos component registry add cloudposse/terraform-aws-components ``` Add our reference architecture registry ``` atmos stack registry add cloudposse/refarch ``` The `atmos.yml` contains: ``` components: terraform: registries: - cloudposse/terraform-aws-components stacks: registries: - cloudposse/refarch ``` ### Use-case #5: Configuration ``` import: - uw2-globals vars: stage: testplatform terraform: vars: {} helmfile: vars: account_number: "199589633144" components: terraform: # this will download all components into `aws-component/0.141.0` # it's abstract because: atmos terraform apply aws-component doesn't make sense "cloudposse/terarform-aws-components/0.141.0": metadata: type: abstract source: https://github.com/cloudposse/terraform-aws-components//modules version: 0.141.0 # this will run aurora-postgres from `aws-component/0.141.0/aurora-postgres` aurora-postgres: component: "cloudposse/terraform-aws-components/0.141.0/aurora-postgres" mixins: # calling this mixins is confusing # this will upgrade the context.tf - file: context.tf source: https://github.com/cloudposse/terraform-aws-components/mixins/context.tf version: 1.2.3 vars: # https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html instance_type: db.r4.large cluster_size: 1 cluster_name: main database_name: main # this will run aurora-postgres from `aws-component/0.141.0/aurora-postgres` eks: component: "eks" mixins: metadata: type: real source: https://github.com/gruntwork/terraform-aws-components//modules/eks version: 1.0 vars: # https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html instance_type: db.r4.large cluster_size: 1 cluster_name: main database_name: main helmfile: cert-manager: metadata: type: real source: https://github.com/cloudposse/helmfiles/cert-manager/ version: 1.0 vars: .. ``` ### Use-case #6: easy upgrades ``` atmos component upgrade terraform/aws-component --version latest ``` 1. This will update `version: 0.141.0` to the latest (e.g. `0.142.0`) 2. Then pull down the `latest` version of the component (in this case, the entire component library) and write it to `components/terraform/cloudposse/terraform-aws-components/0.142.0` 3. User commits to VCS Optionally, the old version can be purged: ``` atmos component upgrade terraform/cloudposse/terraform-aws-components \ --version latest \ --update \ # update any derived component versions to use 0.142.0 --purge \ # delete previous versions --commit # idea: git commit these changes (what about branch, pr, etc) ``` ### Use-case #7: diverge from cloudposse component ``` cp -a components/terraform/cloudposse/terraform-aws-modules/aurora-postgres \ components/terraform/aurora-postgres ``` ### Use-case #8: diff ``` atmos component upgrade --diff --use-defaults ``` ### Use-case #6: refarch ``` atmos generate cloudposse/refarch/multi-account-eks-pci ``` 1. pull down the refarch for a multi-account PCI compliance refarch 2. It will prompt the user for all the inputs 3. It will generate all the configs and components 4. User commits to VCS --- ## Proposed: Use AWS Federated IAM over AWS SSO **Date**: **19 Oct 2021** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - Customers overwhelmingly prefer AWS SSO. We continue to use both AWS Federated IAM with the `aws-saml` component and use AWS SSO with the `aws-sso` component. However, customers typically use AWS SSO themselves and grant Cloud Posse access by AWS Federated IAM. ::: ## Status **IN PROGRESS** @Jeremy Grodberg working on this one. ## Context :::info AWS Federated IAM and AWS SSO can coexist and are not mutually exclusive. ::: ### AWS SSO #### Pros - Native support via the AWS cli (e.g. `aws sso login` command) - It is nice that you can define a permission set once and deploy it to all the accounts (but we can do that with Terraform about as easily) #### Cons - Must have a profile to log into and use a permission set - Cannot set IAM permissions boundary - Cannot attach customer-managed IAM policies - Cannot set `SourceIdentity` - Cannot use as a Principal in IAM policies because they are transient (**a particular problem with EKS access**) - Cannot have more than one IdP - In our use cases, the IdP is still a SAML app requiring a SAML connector ### AWS Federated IAM with SAML ``` # Note that you cannot update the aws-sso component if you are logged in via aws-sso. # Use the SuperAdmin credentials mentioned in docs/cold-start.md (as of now, stored in 1password) aws-saml-sso: settings: spacelift: workspace_enabled: false vars: idps: cloudposse: acme: roles: terraform-prod: policy: .. terraform-nonprod: .. account_assignments: artifacts: grants: - cloudposse: - terraform-nonprod - acme - terraform-nonprod - terraform-prod dev: direct_idp: - cloudposse: - terraform-nonprod grants: - acme - terraform-nonprod - terraform-prod ``` #### Pros - Full control over the implementation - No problems working with EKS and terraform #### Cons - GSuite does not support mapping group attributes to SAML attributes (But they don't really solve it for AWS SSO either, and if you can script the GSuite API you can achieve the same effect.) - Our current implementation with [iam-primary-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-primary-roles) and [iam-delegated-roles](https://github.com/cloudposse/terraform-aws-components/tree/main/deprecated/iam-delegated-roles) is outdated and should be updated to use the interface we developed for AWS [sso](/components/library/aws/identity-center/). ### When to use AWS SSO? lnmgafj/....[;..................................................................................................................................................................................................../PLU5D5YT6FTYUNFVHGT6FHGNU VDYBHBGYBDFR G YGDAWS SSO is ideally suited for business users of AWS that interact with the AWS Web Console. It does work well with the `aws` CLI, but not together with EKS. ### When to use AWS Federated IAM with SAML? AWS Federated IAM is ideally suited for organizations that need to use multiple Identity Providers (IdP). It’s also better suited for managing the IAM RBAC mapping with EKS due to limitations in AWS managing the `auth` `ConfigMap` which has no AWS API. [https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html](https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html) ## Decision **DECIDED**: ## Consequences - ## References - --- ## Proposed: Use Defaults for Components :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal has already been adopted, and this ADR needs to be updated to reflect the final decision. ::: ## Status **DRAFT** ## Problem There are many places to set configurations with various levels of prescedence: - As defaults to `variables { ... }` in `.tf` files - As `defaults.auto.tfvars` (or similar file `.tfvars` file) - As configuration in `stacks/` or catalogs like`stacks/catalog/...` - As environment variables (e.g. `TF_VAR_...`) - As arguments to `terraform plan` or `terraform apply` (e.g. `terraform plan -var foo=bar` Developers are confused about where to set variables without a consistent, opinionated convention on how to do it. Running `atmos describe` it’s easy to see the deep-merged configuration after all imports have been consumed, however, it doesn’t show what defaults are set in `variables { ... }` or `.tfvars` files. ``` # Ideal outcome (not yet supported) vars: enabled: true # default.auto.tfvars nodes: 10 # stacks/catalog/eks/defaults.yaml min_size: 3 # stacks/uw2-prod.yaml name: eks # components/terraform/variables.tf ``` ## Context In Terraform 0.11, regular `*.tf` files were [loaded in alphabetical order](https://www.terraform.io/docs/configuration-0-11/load.html), and then override files were applied. When invoking any command that loads the Terraform configuration, Terraform loads all configuration files within the directory specified in alphabetical order. Override files are the exception, as they're loaded after all non-override files, in alphabetical order. In the newer Terraform 0.12, the load order of `*.tf` files is [no longer specified](https://www.terraform.io/docs/configuration/index.html#configuration-ordering). Behind the scenes (in both versions), Terraform reads all of the files in a directory and then determines a resource order that makes sense ignoring the order the files were actually read. Terraform automatically processes resources in the correct order based on relationships defined between them in configuration, and so you can organize resources into source files in whatever way makes sense for your infrastructure. In TT 0.11, the `auto.tfvars` files were loaded in alphabetical order. In TF 0.12 and newer, it says they are loaded in random order, but that depends on many things, depends on the file system, and we can assume that they can be loaded in the alphabetical order as well. **NOTE:** This is a convincing reason to define ALL config in YAML files and not to have `default.auto.tfvars` at all, especially not to have many `auto.tfvars` in the same folder with different names and conflicting settings inside them. Because in this case, the order of operations is NOT defined, and it could succeed in one place (`atmos`), but fail in another (Spacelift). Thus our current recommendation is to remove all `*.auto.tfvars` (e.g. `default.auto.tfvars` and `variables-helm.auto.tfvars`) and put all the configuration for the component into the YAML stack config. This does not only solve the issue described above but allows seeing ALL variables for the component when executing the `atmos describe component argo-workflows --stack mgmt-uw2-sandbox` command (if some variables are in `auto.tfvars` files, the command will not see them) ## Considered Options ### Option 1: Only use Stacks and Catalogs Avoid all use of `.tfvars` and just use stacks and catalogs. Avoid any defaults in `variable { ... }` blocks. Default enabled in `default.auto.tfvars` to `false`. ### Option 2: Use Stacks, Catalogs and `.tfvars` Place the majority defaults in `.tfvars` with sane defaults and only create archetypes in catalogs. Default archetypes to be `enabled`. ## Decision - Do not put defaults in `defaults.auto.tfvars` - **Exception**: The `spacelift` component which requires it - **Exception**: Helm releases should have chart name, repo and version in `default.auto.tfvars` - Add defaults to `catalog/$component/baseline.yaml` where `$component` usually refers to a component in `components/terraform/$component` - These would generally be of `metadata.type=abstract` - Every component needs an `enabled` flag which gets passed to all modules as well as toggles any resources in the component itself - The `enabled` flag should be set to `true` in the `catalog/$component/baseline.yaml` ### Consequences - Provide a way to generate the baseline configurations from the `.tfvars` and/or the `variables { ... }` - Provide a way to visualize where all the imports happen (almost like a `git blame`) ### References - [https://stackoverflow.com/questions/59515702/multiple-tf-files-in-a-folder](https://stackoverflow.com/questions/59515702/multiple-tf-files-in-a-folder) - [https://www.terraform.io/language/configuration-0-11/load#load-order-and-semantics](https://www.terraform.io/language/configuration-0-11/load#load-order-and-semantics) --- ## Proposed: Use GitHub Actions with Atmos **Date**: **14 Apr 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal has already been adopted, and this ADR needs to be updated to reflect the final decision. ::: ## Status **DRAFT** ## Problem Smaller, bootstrappy startups don’t have the budget for Spacelift. There are things in atmos that we can do (e.g. workflows) that are easier to implement in conventional manners than the rego-based approach in Spacelift. ## Context ## Considered Options - Update atmos to natively support `git diff` to strategically determine what changed in a branch HEAD commit relative to any other BASE commit for the purpose of strategic plan/apply of YAML stack configurations - For our GitHub Action, we’ll want to detect the last successful commit applied against the default branch. This can be accomplished by querying the GitHub API. - Provision private, restricted S3 bucket to store terraform `planfiles` (or use dynamodb? to facilitate locking, and discovery of the latest planfile and invalidating old planfiles) - Use lifecycle rules to expunge old plans - Institute branch protections with `CODEOWNERS` - Implement support for manual deployment approvals [https://docs.github.com/en/enterprise-cloud@latest/actions/managing-workflow-runs/reviewing-deployments](https://docs.github.com/en/enterprise-cloud@latest/actions/managing-workflow-runs/reviewing-deployments) - Implement environment protection rules [https://docs.github.com/en/enterprise-cloud@latest/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-protection-rules](https://docs.github.com/en/enterprise-cloud@latest/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-protection-rules) - Create private, shared GitHub Actions workflows for: - `atmos terraform plan` - Develop a github action that will use a service account role for the runner (or OIDC) to run a terraform plan - Store the planfile in S3 (or dynamo) - Comment on the PR with a pretty depiction of what will happen for every affected stack - Upon merge to main, trigger a GitHub `deployment` with approval step See [https://github.com/suzuki-shunsuke/tfcmt](https://github.com/suzuki-shunsuke/tfcmt) for inspiration - Include support for `terraform plan -destroy` for deleted stacks or components - `atmos terraform apply` - Upon approval, trigger an “apply” by pulling down the corresponding “planfile” artifact from S3; they may be more than one planfile; abort if no planfile - Run atmos terraform apply on the planfiles in the appropriate order - Discard planfiles upon completion - Workflows for complex, coordinated sequences of operations (e.g. to bring up a full stack, one component at time) - Conflict resolution - Locking strategy for components / planfiles. How do we determine the latest planfile? - Implement a GitHub Action that when used together with branch protections prevents merging of pull requests if other unconfirmed changes are pending deployment and affect the same stacks. - This is the most complicated part of the solution. A “one cancels all” type of strategy will probably need to be implemented. We have to ensure planfiles are applied in sequential order, and any planfiles needs to be invalidated (or replanned) if upstream commits are made affecting the stacks - Drift Detection - Implement cron-based automatic replans of all infrastructure under management. [https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/workflow-syntax-for-github-actions#onschedule](https://docs.github.com/en/enterprise-cloud@latest/actions/using-workflows/workflow-syntax-for-github-actions#onschedule) - Trigger webhook callbacks when drift is detected (e.g. escalate to Datadog) ### Risks - GitHub Actions does not provide a great dashboard overview of workflow runs. Mitigated by something like [https://github.com/chriskinsman/github-action-dashboard](https://github.com/chriskinsman/github-action-dashboard) ## Mocking GitHub Action Workflows Checks UI. Each job is a component. Each step is an environment. ``` name: "plan" on: pull_request: types: [opened, synchronize, reopened] paths: - stacks/* - components/* jobs: atmos-plan: runs-on: self-hosted-runner steps: - name: "Checkout source code at current commit" uses: actions/checkout@v2 - name: Atmos do everything runs: atmos plan do-everything - name: id: prepare env: LATEST_TAG_OS: 'alpine' BASE_OS: ${{matrix.os}} run: | ``` ## Decision **DECIDED**: ## Consequences - ## References - [https://blog.symops.com/2022/04/14/terraform-pipeline-with-github-actions-and-github-oidc-for-aws/](https://blog.symops.com/2022/04/14/terraform-pipeline-with-github-actions-and-github-oidc-for-aws/) --- ## Proposed: Use Global Filename Convention **Date**: **29 Apr 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal has already been adopted, and this ADR needs to be updated to reflect the final decision. We decided on changing the convention to `_defaults`. ::: ## Status **DRAFT** ## Problem - There are a lot of `globals` files scattered across many folders and subfolders. The meaning of the globals file is the same everywhere, but the context in which it is used is by convention based on it’s location in the filesystem. - For some people, working in an IDE (e.g. JetBrains, VSCode), it’s easy to get confused on which globals file is being edited. - For some people, working on the command line, if the prompt doesn’t show the full hierarchy, knowing which globals file is being edited is also not clear. Including the full path in the prompt, could get too long. ## Context ## Considered Options ### Option 1: `globals.yaml` (what we have today) :heavy_minus_sign: easily confused with stacks; globals are intended to only be imported ### Option 2: `_globals.yaml` :heavy_plus_sign: disambiguate easily between globals (or imports) from stacks using a `_` prefix convention :heavy_minus_sign: doesn’t solve the filename disambiguation ### Option 3: `$subpath1-$subpath2-globals.yaml` (e.g. `eg-prod-globals.yaml`) - or something like it :heavy_minus_sign: adds to the naming convention overhead. Moving files around requires renaming files. ### Option 4: Find alternatives for IDE :heavy_plus_sign: Enable sorting files first over folders, and using `_globals` will place them at the top :heavy_plus_sign: Enable the path depth :heavy_plus_sign: Rainbow plugin :heavy_plus_sign: Ship a `.vimrc`, `.VSCcode`, `.emacs`, etc file as a baseline ## Decision **DECIDED**: ## Consequences - ## References - JetBrains has a setting to order --- ## Proposed: Use ISO-8601 Date Index for ADRs **Date**: **19 Oct 2021** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. ::: ## Status **PROPOSED** ## Context Using the auto-incrementing index for ADRs is the conventional way of indexing them. The problem is when we have multiple open PRs, the indexes frequently end up conflicting, forcing team members to update the PRs. Using ISO-8601 dates will accomplish the same purpose of an incrementing ID, but avoid needing to renumber all ADRs based on PR merge order. ### Pros - Auto incrementing based on date - No conflicts, assuming we consider the (date, summary) is the ID. ### Cons - We can no longer refer to ADRs by a single number (E.g. `ADR 0007`). Maybe we refer to the corresponding `REFARCH` ticket instead. ## Decision **DECIDED**: Use ISO-8601 dates for ADR index ## Consequences - Write all new ADRs using date format. - Recommend updating existing ADRs with date - Regenerate the table of contents ## References - [Architectural Design Records (ADRs)](/resources/adrs) --- ## Proposed: Use Mixins to DRY-up Components **Date**: **11 Mar 2022** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - Cloud Posse does use mixins, but generally they are avoided. Instead, we recommend [using the override pattern](/learn/component-development#how-can-terraform-modules-or-resources-be-added-to-a-component). This ADR should be updated to reflect the latest decision. ::: ## Status **DRAFT** ## Problem Many Terraform components are not [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) component-to-component because they contain the same boilerplate for achieving similar functions: Configuring variables or the Helm Provider for an EKS Component, Creating a SopsSecret for an EKS Component, etc. ## Considered Options A Terraform mixin (inspired by the [concept of the same name in OOP languages such as Python and Ruby](https://en.wikipedia.org/wiki/Mixin)) is a Terraform configuration file that can be dropped into a root-level module, i.e. a component, in order to add additional functionality. Mixins are meant to encourage code reuse, leading to more simple components with less code repetition between component to component. ### Proposed Mixins #### Mixin: `infra-state.mixin.tf` Code: [https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/infra-state.mixin.tf](https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/infra-state.mixin.tf) This mixin is meant to be placed in a Terraform configuration outside the organization's infrastructure monorepo in order to: 1. Instantiate an AWS Provider using roles managed by the infrastructure monorepo. This is required because Cloud Posse's `providers.tf` pattern requires an invocation of the `account-map` component’s `iam-roles` submodule, which is not present in a repository outside of the infrastructure monorepo. Retrieve outputs from a component in the infrastructure monorepo. This is required because Cloud Posse’s `remote-state` module expects a `stacks` directory, which will not be present in other repositories, the monorepo must be cloned via a `monorepo` module instantiation. Because the source attribute in the `monorepo` and `remote-state` modules cannot be interpolated and refers to a monorepo in a given organization, the following dummy placeholders have been put in place upstream and need to be replaced accordingly when "dropped into" a Terraform configuration: 1. Infrastructure monorepo: `github.com/ACME/infrastructure` Infrastructure monorepo ref: `0.1.0` #### Mixin: `introspection.mixin.tf` Code: [https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/introspection.mixin.tf](https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/introspection.mixin.tf) This mixin is meant to be added to Terraform components in order to append a `Component` tag to all resources in the configuration, specifying which component the resources belong to. It's important to note that all modules and resources within the component then need to use `module.introspection.context` and `module.introspection.tags`, respectively, rather than `module.this.context` and `module.this.tags`. #### Mixin: `sops.mixin.tf` Code: [https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/sops.mixin.tf](https://github.com/cloudposse/terraform-aws-components/blob/6dc766d848306d6ce3ddb1a86bc26822b30ce56f/mixins/sops.mixin.tf) This mixin is meant to be added to Terraform EKS components which are used in a cluster where sops-secrets-operator (see: [https://github.com/isindir/sops-secrets-operator](https://github.com/isindir/sops-secrets-operator) ) is deployed. It will then allow for SOPS-encrypted SopsSecret CRD manifests (such as `example.sops.yaml`) placed in a `resources/` directory to be deployed to the cluster alongside the EKS component. This mixin assumes that the EKS component in question follows the same pattern as `alb-controller`, `cert-manager`, `external-dns`, etc. That is, that it has the following characteristics: 1. Has a `var.kubernetes_namespace` variable. 2. Does not already instantiate a Kubernetes provider (only the Helm provider is necessary, typically, for EKS components). #### Mixin: `helm.mixin.tf` Code: TODO This mixin is meant to be added to Terraform EKS components and performs the following functions: 1. It provides consistent boilerplate for Helm charts, i.e. all of the Terraform variables required to configure a Helm chart and its version. 2. It instantiates the Helm provider and the Kubernetes provider, and all of the variables to override it, including toggling of the Helm Provider’s experimental manifest feature. This mixin does _not_ instantiate the `helm-release` module itself. Rather, it encapsulates all of the boilerplate required to do so. The reason for this is because the module instantiation is unique to the component, and has an intuitive interface to set up policies for the IRSA role, etc. This mixin also assumes that EKS components will contain some values in `defaults.auto.tfvars`, which do not frequently change but can still be overridden by YAML stack configs. This includes things such as the chart repository and chart version. The benefit of this is that tools such as [renovatebot](https://github.com/renovatebot) can automatically increment the Helm chart version if these values are within `defaults.auto.tfvars`, rather than the YAML stack config. Additionally, the variables within `helm.mixin.tf` need defaults for these values, but these defaults should not exist within the variable declaration blocks themselves as they are unique per component, and the end user of the component should not always have to provide a YAML stack config with values for these variables, if they do not frequently change from user-to-user. ``` name = "alb-controller" chart = "aws-load-balancer-controller" chart_repository = "https://aws.github.io/eks-charts" chart_version = "1.4.0" kubernetes_namespace = "kube-system" resources = { limits = { cpu = "200m" memory = "256Mi" }, requests = { cpu = "100m" memory = "128Mi" } } ``` ### Additional Considerations #### Versioning See [Use Vendoring in Atmos](/resources/adrs/adopted/use-vendoring-in-atmos) #### Mixin Best Practices - Whenever a Terraform mixin contains a Terraform Provider, it must set an alias for it. Otherwise, mixins will conflict with each other. #### Unit Testing Mixins - `terraform-aws-components` will contain both the mixins and components using them. The component configuration schema allows for referencing mixins using relative paths. Thus, the component can reference the mixin in the same repository. This provides an integration test for both the components and the mixins they use, ensuring both are functioning. ## Decision **DECIDED**: ## Consequences - TODO: Waiting on Decision ## References - [https://github.com/cloudposse/terraform-aws-components/pull/385](https://github.com/cloudposse/terraform-aws-components/pull/385) --- ## Proposed: Use More Flexible Resource Labels **Date**: **19 Apr 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - No pushback from the team. Overall, we know we need to support arbitrary label fields and don't like how we use environment to represent region. Note, this suggestion also matches (is consistent with) our filesystem organization: `///`. [Internal discussion reference](https://github.com/cloudposse/knowledge-base/discussions/120) - Decision is to adopt: `---` ::: ## Status **PROPOSAL** ## Problem Currently, we use a fixed set of labels, dictated by the [terraform-null-label](https://github.com/cloudposse/terraform-null-label/) component, for labeling everything provisioned by IoC. This set of labels is also treated specially by `atmos` and includes labeling IAM roles and both `atmos` and Spacelift “stacks”. 1. **The choice of label names has proven to be confusing and unpopular.** 2. The set of labels is fixed. When we added “tenant” as a possible label it was a major undertaking to upgrade `terraform-null-label` to handle it. 3. Because the label names are fixed, and `atmos` does not have access to the outputs of `terraform-null-label` (because `atmos` is written in `go` and not Terraform), adding or changing label names requires code changes to both `terraform-null-label` and `atmos` 4. The use of `name` as a label name is a particular problem as it conflicts with AWS' usage of the tag key “Name” as the UI display name of a resource. 5. We have come to rely on `atmos` as a tool, and it needs to parse labels to determine the Atmos “stack” name, the Terraform backend configuration, the Terraform workspace name, the EKS cluster name, and possibly other resources, but `atmos` is written in `go` and cannot use `terraform-null-label`, which is a Terraform module, to generate these items, but nevertheless we want some of them to be available in Terraform so that components can access configuration data generated by other Terraform components. 6. We have some components, such as Kubernetes deployments, that have additional configuration labels/variants, such as `color` for blue/green deployments or `ip` for IPv4/IPv6 variants. We would like to be able to flexibly use or not use these additional labels to distinguish deployed deployments where applicable, without requiring them for other components (e.g. `cloudtrail_bucket`) where they are not needed. Currently we are doing this by manually altering the component names to include the variant labels, but this practice is not DRY and eliminates many of the advantages `atmos` gives us through importing configurations, since all configurations are, in the end, tied to a component name. The proper Atmos model is to have a single component name with variable Terraform workspaces selected by variable labels. ## Context Early on, Cloud Posse decided that consistent labeling was important and implemented a mechanism for it in the form of `terraform-null-label`. (`terraform-null-label`, or `null-label` for short, was first released in 2017.) At the time it was first released, Terraform itself was in the early stages of development and lacked many essential features, so the capabilities of the module were limited. In particular, there was no way to iterate over lists or maps. This imposed a practical requirement that inputs to `null-label` be known in advance (hardcoded). The original set of labels was: - `namespace` - `stage` - `name` Over time, we added - `environment` - `tenant` ...to get to the current set of 5 labels. (`null-label` also accepts a list of `attributes` and a map of `tags`, which are outside the scope of this ADR.) Unfortunately, except for the `tenant`, there are issues with all of these label names. - `namespace` collides with Kubernetes' use of “namespace” as a mechanism for isolating groups of resources within a single cluster, and we have had problems due to the `$NAMESPACE` shell variable being set to indicate our version of “namespace” while being interpreted by some tools as Kubernetes' version. - `environment` is not bad, but a lot of people use it in a way we do not use it. We use it as a region code (abbreviation for a particular AWS Region) while most people use it to indicate a functional role or AWS account, such as “production” or “staging”. - `stage` is a bit confusing, and in the end more generic than we allow. We use it the way many people use “environment”, but because we typically have a 1-to-1 mapping of `stage` to AWS Account, our code frequently assumes that “stage” is the same as “account”. This breaks, however, in multi-tenant environments where tenants have multiple accounts, such as `tenant-dev`, `tenant-stage` and `tenant-production`. - `name` is a problem in that AWS reserves that for the tag key whose value is displayed in the web UI. For all our other labels, we add a tag with the (capitalized) label name as tag key and (normalized) label value as the tag value. We make an exception for “Name”, setting that value to the the `id` (the fully formed identifier combining all the labels), not the value of the `name` label, which confuses everyone. - Atmos separately has (in `atmos.yaml`) configuration for `helm_aws_profile_pattern`, EKS `cluster_name_pattern`, and Stack `name_pattern`, along with separate configuration for Component name (directory) and Terraform workspace name. Currently these are either completely hard coded (Component name) or are configured using a template based on the above listed special label names, which works completely separately from `null-label` and must be kept in sync. Now (April 2022), Terraform version 1.1 has several features that enable us to use an arbitrary set of label names. On the drawing board (but for no earlier than Terraform version 1.3) is also an additional feature we would like, allowing input objects to have optional attributes. This suggests we can create a new `null-label` version with 1.1 features and again enhance it after optional attributes have been released. [https://github.com/hashicorp/terraform/pull/31154](https://github.com/hashicorp/terraform/pull/31154) ## Considered Options ### Option 1: #### Null Label Going forward, I suggest Cloud Posse use different label names in its engagements: - `company` instead of `namespace`, to provide a global prefix that makes the final ID unique despite our reuse of all the other label values - `region_code` or `reg` instead of `environment` to indicate the abbreviated AWS Region - `tenant` can remain, or be changed to `ou` for organizational unit. - `env` instead of `stage`, to indicate the function of the environment, such as “development”, “sandbox”, or “production”. In environments where `env` always equals `account`. We would specify only one and have the other be a generated label (see below). Which one to specify should be based on a survey of clients' preferences. - `account` instead of `stage` to indicate the name of the AWS account. `account` would never be specified directly, it would generally be either `env` or `tenant-env`. - `component_name` instead of `name` (and to avoid overloading `name` used by AWS and `component` which has special meaning to `atmos`). - Possibly an additional label component, such as `net` or `ip` that can be used to allow us to create IPv4 and IPv6 versions of components like EKS clusters or ALBs in the same account and region and yet still distinguish them. It label component would ideally have an optional attribute that removes the delimiter before it, so if `name` is `eks` and `ip` is `6`, we can get a name like `{namespace}-{tenant}-{environment}-{stage}-eks6-cluster` instead of `{namespace}-{tenant}-{environment}-{stage}-eks-6-cluster` To facilitate this, I suggest an overhaul of `terraform-null-label`. We can use the existing `label_order` input to take an arbitrary list of label names. We can deprecate the existing hard-coded label names in favor of a new input, called `label_input` (to allow us to have an output named `labels` which has the normalized label values, and a separate output named `label_input` which preserves the input untransformed) or `labels` (where either we do not care about the output `labels` being different than the input or we are satisfied that `module.this.labels` is normalized while `module.this.context.labels` gets you back exactly what was input, as is currently the case with the special label names., e.g `module.this.stage` vs `modules.this.context.stage`) which is a `map(string)` where the keys are label names and the values are label values. (This is exactly like the `tags` input, but the tags are not altered, while labels are.) Additionally, we deprecate the existing `descriptor_format` input and `descriptors` output in favor of a `label_generator` input which adds labels to the `labels` output. This would allow us to have an `account` output that by default is the same as the `env` or `stage` output (and for that matter, allow us to preserve the `namespace`, `environment`, and `name` outputs even though we have stopped using them as inputs), and also handle the case where `account` is a composite of 2 labels like `tenant-dev`. #### Future Possibilities **Once** [**Terraform supports optional object members**](https://github.com/hashicorp/terraform/issues/19898#issuecomment-1101853833)[,](https://github.com/hashicorp/terraform/issues/19898#issuecomment-1101853833) I would propose `label_generator` be a `map(object)` that has: - key is name of label to generate - `labels = list(string)` list of label to construct the label from, in order - `delimiter = optional(string)` the delimiter to use when joining the labels, defaults to label `delimiter` - `value_case = optional(string)` the case formatting of the label values, one of `lower`, `title`, `upper` or `none` (no transformation), defaults to `label_value_case` - `regex_remove_chars = optional(string)` regex specifying characters to remove from the value, defaults to top level `regex_replace_chars` (which I would deprecate and replace with `regex_remove_chars` since we do not provide the capability to replace the characters and no one has asked for that). - `length_limit = optional(number)` the limit on the length of the value, or 0 for unlimited, defaults to 0. - `truncation_mode = optional(string)` one of "beginning", "middle", or "end". Where to place the hash that substitutes for the extra characters in the label. Allows you to decide to truncate `foo-bar-baz` as `foo-bar-` (the only mode we allow today), `-bar-baz`, or `foo--baz`. I would also add `id_truncation_mode` to the top-level and default `truncation_mode` to whatever `id_truncation_mode` is set to. Unfortunately, `id_truncation_mode` would need to default to `end` for backward compatibility, but I think `middle` is the better default. ``` locals { # Create a default format map so it can be reused, optionally with changes applied. # This is in part to deal with the Terraform requirement that all values of a map # must have the exact same type. default_format = { delimiter = "-" value_case = "lower" regex_remove_chars = "/[^a-zA-Z0-9-]/" length_limit = 64 truncation_mode = "middle" } } # Advanced example, more like what we would probably use module "this" { source = "cloudposse/label/null" label_order = [ "org", "ou", "reg", "env", "component"] label_format = local.default_format label_generator = { # This is how we would generate the "id" output if it were not hardcoded for backward compatibility id = merge(local.default_format, { labels = [ "org", "ou", "reg", "env", "component"] }) # Generate an output named "account" of the form "${ou}_${env}" account = merge(local.default_format, { # Specify the value inputs and the order labels = ["ou", "env"] # Change the delimiter to "_" instead of "-" delimiter = "_" # By default, we remove underscores, so we need to alter the list of characters to remove regex_remove_chars = "/[^a-zA-Z0-9-_]/" }) } # In practice, the "values" input would be generated by Atmos # For example, in stacks/orgs/cplive/_defaults.yaml # vars: # label_values: # org: cplive label_values = merge ({component = var.component_name} , { org = "cplive", ou = "plat", reg = "ue1" }) } locals { id = module.this.id org = module.this.labels["org"] account_name = module.this.labels["account"] } ``` ``` # Simpler example module "this" { source = "cloudposse/label/null" label_order = [ "org", "ou", "reg", "env", "component"] label_format = local.default_format label_generator = { account = { labels = ["ou", "env"] delimiter = "_" regex_remove_chars = "/[^a-zA-Z0-9-_]/" } } label_values = { org = "cplive", ou = "plat", reg = "ue1" } } ``` ``` # Simplest example module "this" { source = "cloudposse/label/null" label_order = [ "org", "ou", "reg", "env", "component"] format = local.default_format values = { org = "cplive", ou = "plat", reg = "ue1" } } ``` ``` # In stacks/orgs/cplive/_defaults.yaml using current labels # (Compare to https://github.com/cloudposse/infra-live/blob/8754dc3d1e938c31387bc704ef361fc476fe28e5/stacks/orgs/cplive/_defaults.yaml#L9-L28 ) vars: label_values: namespace: cplive label_order: - namespace - tenant - environment - stage - name - attributes label_format: &default_label_format delimiter: "-" value_case: "lower" regex_remove_chars: "/[^a-zA-Z0-9-]/" length_limit: 64 truncation_mode: "middle" label_generator: account_name: <<: *default_label_format labels: - tenant - stage stack: <<: *default_label_format labels: - tenant - environment - stage # In stacks/orgs/cplive/core/_defaults.yaml vars: label_values: tenant: cplive # et cetera ``` For now (April 2022) with no ETA on that feature, I would limit `label_generators` to `map(list(string))`: - key is name of label to generate - `labels = list(string)` list of label to construct the label from, in order The generated label will be the normalized values of the labels named in the list, in that order, joined by the same `delimiter` used for the `id`. Likewise, we would deprecate the named outputs (and `descriptors`) in favor of a `labels` output which is a map of label names to normalized label outputs. So instead of `module.this.stage` we would reference `modules.list.labels["stage"]` #### Atmos Changes We need to update atmos to support a flexible set of labels. ##### Atmos option 1 Instead of specifying a template for each configuration value, such as `cluster_name_pattern`, Atmos could configure a `labels` output to use as `cluster_name_pattern` (e.g. `cluster`) and then both `atmos` and `terraform` will have access to exactly the same information in the same way (e.g. `module.this.labels["cluster"]`). ##### Atmos option 2 Right now, there are the top level `namespace`, `stage`, `name`, `tenant`, `environment` labels. We could put these now under a new section in the stacks or in `atmos.yaml`: ``` terraform: backend: backend_pattern: {foo}-{bar}-{baz} labels: - foo - bar - baz ``` For compatibility with `null-label`, `atmos` should populate the labels based on the fully merged `vars` section of the stack configuration, supporting both the old variables as it does now and the new `label_input` (or whatever we call it) map. ### Option 2: ``` module "this" { label = "camelcase(id)-lowercase(name)-uppercase(company)" # camelcaseHyphenFoobarFormat(....) context = var.context } ``` ### Option 3: We predefine a named set of formats and allow additional custom formats to be defined ``` # Simpler example module "this" { source = "cloudposse/label/null" label_order = [ "org", "ou", "reg", "env", "component"] label_format = "kebab" label_generator = { account = { labels = ["ou", "env"] format = "snake" } } label_values = { org = "cplive", ou = "plat", reg = "ue1" } } ``` ## Decision **DECIDED**: ## Consequences - ## References - --- ## Proposed: Use Multiple Terraform State Bucket Backends **Date**: **25 Mar 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal remains in draft and needs context updated. ::: ## Status **DRAFT** ## Problem - Terraform state backend has sensitive information e.g. RDS master credentials - Using multiple state backends would alleviate some of these concerns, but introduce new problems with how to manage access to the bucket as well as how remote-state lookups know where to find the state ## Context ## Considered Options ### Option 1: Use AWS SSO with Standalone IAM Roles Create a `terraform-prod` `PermissionSet` and create a `terraform-non-prod` PermissionSet. #### Pros #### Cons - We create permission sets, not roles. We only create permission sets in the `aws-sso`, standalone components cannot create permission sets. - Introducing more permission sets pollutes the global namespace with roles that are only really relevant in a couple of accounts - Delegation of PermissionSets cannot be given to other components ### Option 2: Use Federated IAM with SAML ### Option 3: ## Decision **DECIDED**: ## Consequences - ## References - --- ## Proposed: Use Private and Public Hosted Zones **Date**: **11 Feb 2022** :::warning Rejected! The proposal in this ADR was rejected! For questions, please reach out to Cloud Posse. - Context for rejection is needed. Overally this proposal adds complexity and cost for using private zones, as well as has a lack of customer demand. ::: ## Status **DRAFT** ## Problem There is confusion regarding service discovery and vanity domains. Hisotically, service discovery domains are on privately hosted DNS zones, yet Cloud Posse typically advocates using public DNS zones for everything, including internal load balancers. Using public hosted zones leaks information about the cloud architecture, which is why some advocate for strictly using private zones. ## Context ## Considered Options ### Option 1: Public zones only #### Pros - This is what we’re currently doing and this has been tried and tested - ACM certs can be used without PCA - LetsEncrypt can be an issuer for cert-manager (when using `ingress-nginx`) #### Cons - Route53 hostnames might be leaked (**although services won’t be accessible**). - Security through Obscurity. ### Option 2: Private zones only This is an “old school” security best practice, which in principle is great but in practice rather limiting. #### Pros - Route53 hostnames cannot be leaked, making it more difficult for adversaries to map out the infrastructure for targeted attacks #### Cons - We’ve recently added support for this in a customer engagement. There could be “underwater stones” (credit to @Igor Rodionov for this term). - **IMPORTANT** Services cannot be exposed in any way to external third-party integrations (e.g. webhook callbacks with Twillio, GitHub, etc - Requires cross association of VPCs with private hosted zones - **IMPORTANT** Can only be associated with exactly one VPC - Is this true? I’ve been able to associate a private zone with multiple VPCs. @RB (Ronak Bhatia) - Requires a VPN solution like AWS Client VPN Endpoint in order to resolve any names - Private CA or a public hosted zone for ACM verification - Public hosted zone for acm verification which would require a split-view setup - or Private CA to sign certificates and sign ACM certs and `cert-manager`. - At least 2 Private CAs are recommended (one for prod, one for non-prod). Each private CA is $400/mo. [https://aws.amazon.com/certificate-manager/pricing/](https://aws.amazon.com/certificate-manager/pricing/) - Troubleshooting DNS lookups would require ssh'ing via SSM to an instance in a VPC that is associated with a private Hosted Zone to check if DNS lookups work as expected ### Option 3: Hybrid of public and private zones This has all the pros of Option 1 and Option 2 and mitigates some of the Cons. #### Pros - Best of both worlds - Public for vanity domains - Private for service discovery domains - Both public and private can co-exist #### Cons - Still requires either a public hosted zone for certificate verification OR the costly Private CA at $400/mo/ea x2. --- ## Proposal: Use Stack Filesystem Layout That Follows AWS Organization Conventions **Date**: **27 May 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse - The proposal has already been adopted, and this ADR needs to be updated to reflect the final decision. ::: ## Status **DRAFT** ## Decision Use Option 4: Use an organization directory. ## Problem We have stacks defined all over the place. It’s not clear what is a top-level stack and what is imported. It’s not clear where to define something and where a service is deployed. There are too many ways to do things and we haven’t standardized how we organize configurations across customers. ## Context ## Considered Options ### Option 1: Current - `stacks/catalog` - `stacks/` - `stacks/` ``` ✗ tree -L 2 stacks stacks ├── catalog │ ├── account-map.yaml │ ├── ... │ └── waf.yaml ├── core │ ├── gbl │ ├── globals.yaml │ ├── ue1 │ └── ue2 ├── gbl │ ├── artifacts.yaml │ ... │ └── staging.yaml ├── globals.yaml ├── plat │ ├── gbl │ ├── globals.yaml │ └── ue2 ├── ue1 │ └── globals.yaml └── ue2 ├── audit.yaml ... └── staging.yaml ``` ### Option 2: Put region within catalog - `stacks/catalog/` - `stacks/` ``` ✗ tree -L 3 stacks --dirsfirst stacks ├── catalog │ ├── argocd │ │ └── repo ... │ ├── gbl │ │ ├── artifacts.yaml ... │ │ └── staging.yaml │ ├── s3-bucket │ ├── ue1 │ │ └── globals.yaml │ ├── ue2 │ │ ├── audit.yaml │ │ ├── auto.yaml │ │ ├── corp.yaml │ │ ├── dev.yaml │ │ ├── globals.yaml │ │ ├── marketplace.yaml │ │ ├── network.yaml │ │ ├── prod.yaml │ │ ├── root.yaml │ │ ├── sandbox.yaml │ │ └── staging.yaml │ ├── account-map.yaml ... │ └── waf.yaml ├── core │ ├── gbl │ │ ├── artifacts.yaml ... │ │ └── security.yaml │ ├── ue1 │ │ ├── globals.yaml │ │ └── public.yaml │ ├── ue2 │ │ ├── audit.yaml ... │ │ └── root.yaml │ └── globals.yaml ├── plat │ ├── gbl │ │ ├── dev.yaml ... │ │ └── staging.yaml │ ├── ue2 │ │ ├── dev.yaml ... │ │ └── staging.yaml │ └── globals.yaml └── globals.yaml ``` ### Option 3: Put root level stacks in order tenant/account/region instead of tenant/region/account - `stacks/catalog` - `stacks/mixins/` - `stacks/` ``` ✗ tree -L 3 stacks --dirsfirst stacks ├── catalog │ ├── argocd ... │ └── waf.yaml ├── mixins │ ├── gbl │ │ ├── artifacts.yaml ... │ │ └── staging.yaml │ ├── ue1 │ │ └── globals.yaml │ └── ue2 │ ├── audit.yaml ... │ └── staging.yaml ├── plat │ ├── gbl │ │ ├── dev.yaml ... │ │ └── staging.yaml │ ├── ue2 │ │ ├── dev.yaml ... │ │ └── staging.yaml │ └── globals.yaml ├── tenants │ ├── core │ │ ├── gbl │ │ ├── ue1 │ │ ├── ue2 │ │ └── globals.yaml │ └── plat │ ├── gbl │ ├── ue2 │ └── globals.yaml └── globals.yaml ``` ### Option 4: Use an organization directory Use a filesystem hierarchy that mirrors the AWS hierarchy: Organization → OU → Account → Region → Resources - `stacks/catalog/` - e.g. `eks/cluster.yaml` - `stacks/mixins/` - e.g. `regions/us-east-1.yaml` - `stacks/orgs////.yaml` | | | | | --------- | ---------------------------------- | --- | | namespace | The namespace for the organization | | | ou | Typically the tenant name | | | account | The stage within the tenant | | | region | The canonical AWS region | | Use fully spelled out canonical region name (e.g. `us-east-1`) Use `global-region.yaml` for resources that are not tied to any particular region (e.g. Route53). Use `_defaults.yaml` for any other default settings. ## Decision **DECIDED**: ## Consequences - ## References - --- ## Proposed: Use Strict Provider Pinning in Components **Date**: **11 Feb 2022** :::info Needs Update! The content in this ADR may be out-of-date and needing an update. For questions, please reach out to Cloud Posse ::: ## Status **DRAFT** ## Problem New major provider versions can break planning and applying of terraform components/modules. [https://github.com/hashicorp/terraform-provider-aws/releases/tag/v4.0.0](https://github.com/hashicorp/terraform-provider-aws/releases/tag/v4.0.0) While we wait to upgrade broken modules to newer provider versions, we should come up with a solution that won’t break customer workflows. Currently, we set this in both modules and components for most customers ```hcl required_providers { aws = { source = "hashicorp/aws" version = ">= 3.0" } } ``` ## Considered Options ### Option 1: Pinning Providers in Components to Major, Minor Versions - The easiest way for Cloud Posse to distribute components with versions that have been tested in a particular configuration (E.g. as opposed to using `.terraform.lock.hcl`) - Pin providers to major versions in downstream components for clients ```hcl required_providers { aws = { source = "hashicorp/aws" version = "~> 3.0" } } ``` ### Option 2: Pinning Providers in Upstream Modules :::info Cloud Posse needs to do lower-bound pinning for terraform core version and AWS provider in our open source modules on `github.com/cloudposse`. We learned this lesson the hard way. The essence of the problem is that terraform computes the intersection of all supported provider versions and takes the highest number supported. If the sets are disjoint, it errors. - ``` Error: Failed to query available provider packages Could not retrieve the list of available versions for provider hashicorp/aws: no available releases match the given constraints ~> 3.0, ~> 2.0 ``` If we pin a module to `~> 3.0` it presents several problems when version 4.0 comes out: - We cannot tell (without removing the version constraint) whether or not the module works with 4.0 as-is or needs modification. - Anyone wanting to use version 4.0 cannot do so with this module or any module that uses it, even if it would otherwise work. - This includes other Cloud Posse modules: if we want to upgrade another module to work with 4.0, we cannot do that if it uses any other modules pinned to `~> 3.0` The net result is that as soon as 4.0 comes out, in practice we need to remove the `~> 3.0` pin anyway, just to see if the module needs modification, and more often than not it does not, so the pin has just created a lot of extra work for no real benefit. (If the code still works, then the pin did nothing but break it. If the code is broken at 4.0, then the best we can say is the pin makes it a little easier to see why the previously working code is now broken, but either way it is broken, so that is not great consolation.) Therefore, Cloud Posse today by convention only does lower bound pinning in our open source module until all the modules are updated. We only bump the the lower bound when the code takes advantage of a new feature that requires it. ::: - Pin providers to major versions in upstream modules ```hcl required_providers { aws = { source = "hashicorp/aws" version = "~> 3.0" } } ``` ### Option 3: Pinning Providers in Components to Exact Versions - This can also be done by committing the `.terraform.lock.hcl` file instead of ignoring it in `.gitignore` and using a github action to periodically update it. - This can be done using a `required_providers` and using something like rennovatebot to update it ``` required_providers { aws = { source = "hashicorp/aws" version = "= 3.70" } } ``` ## Decision - Use Option 1 ## Consequences - Update `providers.tf` in all components to follow this convention - Something should still update the pinning when new providers are available. ## Related Documentation - [Components](/components) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) --- ## Proposed Architecture Decision Records --- ## Architecture Diagrams import Intro from '@site/src/components/Intro'; We provide a number of boilerplate architecture diagrams. Think of them as templates that can be copied and used throughout your organization. Reach out to Cloud Posse PMs if you’d like a copy of any one of them. ## Available Diagrams Don’t see the diagram you need? Open a [GitHub Discussion](https://github.com/orgs/cloudposse/discussions) to raise the request! ## 4 Layers of Infrastructure The 4 Layers of Infrastructure depict the various layers and lifecycles associated with provisioning infrastructure from the bottom up. Each layer introduces new tools and builds upon the previous layers. The SDLC of each layer is independent from the other layers, and each layer must exist before the subsequent layers can be provisioned. As we approach the top of the stack, the layers change more frequently. The lower down we go, the more seldom layers change and frequently more challenging to modify in place. ## 8 Layers of Security The 8 Layers of Security depict security in depth. Cloud Posse has Terraform support for provisioning the most essential security-oriented products, mostly AWS managed services like AWS SecurityHub or AWS WAF. ## Big Picture The Big Picture helps paint the story of how there are dozens of services in play. Where possible, we opt for fully managed services by AWS or best-of-breed SaaS alternatives. We reserve the platform (EKS or ECS) for running and operating your applications, which is your competitive advantage. ## Security Escalation Architecture Our approach to Security Escalations has everything flow through SecurityHub and then to Amazon SNS then through to OpsGeneie for Incident Management. --- ## Alerting import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Learn how to set up an effective alerting system that notifies the right people when issues arise. This involves configuring OpsGenie for incident management and integrating it with Datadog for monitoring, ensuring alerts are properly escalated and managed to prevent alert fatigue and system failures. Alerting notifies users about changes in a system’s state, typically via emails or text messages sent to specific individuals or groups. Alerting goes hand in hand with [monitoring](/layers/monitoring). When a system is monitored properly, only the right people are notified when something goes wrong. :::important This Quick Start assumes you have read through [the Monitoring Quick Start](/layers/monitoring) ::: ## The Problem When a system is not monitored properly, it's easy to get overwhelmed with alerts. This leads to alert fatigue. Alert fatigue is a real problem that can lead to a system being ignored and eventually failing. Furthermore, a system must be in place for alerts to be escalated to the right team and people. When a single system goes down, there are often cascading effects on other services. ## Our Solution Our solution begins with [OpsGenie](https://www.opsgenie.com/). OpsGenie is a modern incident management platform for operating always-on services, empowering Dev and Ops teams to plan for service disruptions and stay in control during incidents. We integrate [OpsGenie with Datadog](https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-datadog/) to pair it with our monitoring solution. ### Implementation We have a singular component that can be instanced for every team. This component is called [`opsgenie-team`](/components/library/aws/opsgenie-team/) and it handles the surrounding work of setting up a team in OpsGenie. To get started with this component, you'll need an OpsGenie API key, which you can get from the [OpsGenie API page](https://support.atlassian.com/opsgenie/docs/api-key-management/). Follow the Component README to get started, this will create a catalog entry which should be configured to your company's defaults for every team. Then start by creating a team for each team in your organization. ### How it works Our Monitors have a global configurable variable called `alert_tags`, this should be set to include `@opsgenie-{{team.name}}`, such as: ```yaml alert_tags: ["@opsgenie-{{team.name}}"] ``` This creates a message on the Datadog monitor that uses the team tag to send the alert to the correct team in OpsGenie. When an event or alert is triggered, the **data's** tag of `team` will be used to dynamically send an alert to the corresponding team in OpsGenie. An important distinction in that the tag is not fetched from the monitor, but the data sent to the monitor. Having the data's tag of team being used is beneficial, monitors can be configured to send alert to different teams. For example, you can have a single monitor for pods crashlooping on EKS, if each deployment is properly labeled with the `team:foo` or `team:bar` tag, then the alert will be sent to the correct team. ### Service Level Indicators (SLIs) and Service Level Objectives (SLOs) SLIs and SLOs are a way to measure the reliability of a service. They are a way to measure the quality of a service. Sometimes a business has contractual obligations to their SLOs. For example, a business may have a contractual obligation to have 99.9% uptime. This means that the service must be available 99.9% of the time. Datadog supports SLOs, they can be a set of metrics or monitors. For example, you can have a monitor that checks if a service is up or down. This monitor can be used as an SLO. You can then use SLO Monitors to report Incidents in OpsGenie, when you create a team, you can specify the level of priority that is considered an **Incident**. When the alert matches the priority level of an Incident, an Incident is created in OpsGenie. An [**Incident**](https://support.atlassian.com/opsgenie/docs/what-is-an-incident/) is a specialized alert that is used to track the progress of an issue, it can have cascading effects on other services. ## References :::tip [This article](/resources/deprecated/alerting/opsgenie) goes more in-depth on some of the above topics. ::: - [OpsGenie](https://www.opsgenie.com/) - [How to Sign Up for OpsGenie?](/resources/deprecated/alerting/opsgenie/how-to-sign-up-for-opsgenie) - [How to Create New Teams in OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-create-new-teams-in-opsgenie) - [How to Add Users to a Team in OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-add-users-to-a-team-in-opsgenie) - [How to Onboard a New Service with Datadog and OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-onboard-a-new-service-with-datadog-and-opsgenie) - [Component `opsgenie-team`](/components/library/aws/opsgenie-team/) - [Datadog: How to Pass Tags Along to Datadog](/layers/monitoring/datadog/tutorials/how-to-pass-tags-along-to-datadog) ## FAQ ### How do I set up SSO with OpsGenie? There are [official docs on how to configure SSO/SAML](https://support.atlassian.com/opsgenie/docs/configure-saml-based-sso/). Those should suffice for using AWS Identity Center. AWS also has [docs on adding SAML applications](https://docs.aws.amazon.com/singlesignon/latest/userguide/saasapps.html) which includes an official config for OpsGenie already. If you don't plan on using AWS IC, there are [docs on configuring SSO with other identity providers](https://support.atlassian.com/opsgenie/docs/configure-sso-for-opsgenie/). --- ## Decide on Default Schedules ## Context and Problem Statement By default, an opsgenie team comes with its own schedule. Sometimes however we want different schedules for different timezones. A team spread across the world would have to manually keep track of the schedule to make sure individuals are only on call for particular hours. ## Considered Options ### Option 1 - Use one default Schedule (Recommended) :::tip Our Recommendation is to use Option 1 because.... ::: #### Pros - One single pane of glass for whose on call #### Cons - Ensuring people in different timezones are on call at the right times is a manual process ### Option 2 - Many Schedules to follow the sun #### Pros - Sets default routing based on timezones to particular schedules. #### Cons - Slightly more complex setup ## References - Links to any research, ADRs or related Jiras --- ## Decide on Incident Ruleset ## Context and Problem Statement We need to decide the rules that make an alert an incident. This ruleset could be based on priority-level of the alert, message, or by tag. Opsgenie can escalate an alert into an incident, this marks the alert as more severe and needs more attention than a standard alert. See [How to Implement Incident Management with OpsGenie](/resources/deprecated/alerting/opsgenie/#terminology) for more details on what an **Incident** is. :::info Picking a standard here provides a clear understanding to when an alert should become an incident, ideally this is not customized by each team. ::: ## Considered Options ### Option 1 - Priority Level Based (P1 & P2) (Recommended) :::tip Recommended because maps 1-1 with Datadog Severity and provides a clear understanding ::: #### Pros - Priority is a first-class field in Datadog and Opsgenie - Directly maps to Datadog severity level in monitors. - P1 & P2 Are considered Critical and High priority, allowing slight variation in the level of incidents. - Dynamic based on the Monitoring Platform (e.g. Datadog can say if this alert happens 5x in 1 min, escalate priority) ### Option 2 - Priority Level Based (Other) This could be only **P1** or any range. #### Pros - Directly maps to Datadog severity level in monitors. - Dynamic based on the Monitoring Platform (e.g. Datadog can say if this alert happens 5x in 1 min, escalate priority) ### Option 3 - Tag Based Tag based approach would mean any monitor that sends an alert with a tag `incident:true` becomes an incident. #### Pros - Dynamic based on the Monitoring Platform (e.g. Datadog can say if this alert happens 5x in 1 min, escalate priority) #### Cons - Incidents can now be defined in more than one way - An extra field must be passed - Puts definition of an incident on the monitoring platform. ## References - [How to Implement Incident Management with OpsGenie](/resources/deprecated/alerting/opsgenie/) - [How to Implement SRE with Datadog](/layers/monitoring/datadog) --- ## Decide on Teams for Escalations ## Problem Teams need to be notified of incidents tied to services that affect them. ## Solution Come up with a table of services and the teams or business units responsible for them. Services are associated with incidents Incidents are escalated to teams ## Other Considerations - Should the teams map to products, services or business units? - Should we map the teams to existing teams in IdP or directly associate users to teams in OpsGenie? #### Here’s how we think about teams: :::note Members can also be handled by the IdP integration, but the teams still need to be defined in OpsGenie) ::: ```yaml teams: - name: cloudplatform description: "Cloud Platform Team" members: - username: user@ourcompany.com role: admin - username: user@ourcompany.com role: admin - name: security description: "Security Team" members: - username: user@ourcompany.com role: admin - username: user@ourcompany.com role: admin - name: compliance-engineering description: "Compliance Engineering Team" members: - username: user@ourcompany.com role: admin - username: user@ourcompany.com role: admin ``` --- ## Design Decisions(8) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions to determine how you'll implement incident management, escalations, and alerting. --- ## How to Add Users to a Team in OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Adding users to a team in OpsGenie ensures that the right team members are notified when alerts occur. This guide will help you update your team's YAML stack configuration to include new members, specify their roles, and integrate the changes seamlessly using the `opsgenie-team` component. Whether you're managing a Site Reliability Engineering (SRE) team or any other team, this process ensures efficient alert handling and response. ## Problem We often need to change who on a team responds to particular alerts. ### Prerequisites Assuming you are using the [opsgenie-team](/components/library/aws/opsgenie-team/) component with `ignore_team_members` set to `false` ## Solution :::tip **TL;DR** In your team’s YAML stack configuration, add users to the `members` array block. ::: Example Configuration: ``` members: - user: erik@cloudposse.com role: admin - user: ben@cloudposse.com ``` ``` components: terraform: opsgenie-team-sre: component: opsgenie-team settings: spacelift: workspace_enabled: true vars: enabled: true name: sre description: "SRE team" members: - user: erik@cloudposse.com role: admin - user: ben@cloudposse.com ``` --- ## How to Create Escalation Rules in OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Creating escalation rules in OpsGenie allows you to control the actions taken when an alert isn’t acknowledged. This guide will walk you through configuring escalation rules in the stack configuration of the `opsgenie-team` component, ensuring timely response and proper alert handling. By defining escalation conditions and actions, you can efficiently manage alert escalations within your teams. ## Problem You want to control what to do when an alert isn’t acknowledged. ## Solution :::tip **TL;DR** This is controlled by escalation rules in the stack configuration of the `opsgenie-team`. ::: An Escalation resource for a team is directly exposed via a map. Have a look at [escalations in terraform for exact variable names](https://registry.terraform.io/providers/opsgenie/opsgenie/latest/docs/reference/escalation) and [How do escalations work in opsgenie](https://support.atlassian.com/opsgenie/docs/how-do-escalations-work-in-opsgenie/) to determine how you want to configure your escalations. An example is below ``` components: terraform: opsgenie-team-my-team: component: opsgenie-team ... escalations: my-team_escalate_to_sre: enabled: true description: "Escalate to 'sre' team if 'my-team' team does not acknowledge" rule: condition: if-not-acked notify_type: all delay: 5 recipients: - type: team team_name: sre repeat: wait_interval: 10 count: 2 reset_recipient_states: false close_alert_after_all: false ``` --- ## How to Create New Teams in OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import TaskList from '@site/src/components/TaskList'; As your company grows, you'll need a streamlined way to configure new teams with alerting on specific resources. This guide demonstrates how to use the `opsgenie-team` component to create a new team in OpsGenie. By tagging resources appropriately, you can ensure that alerts are directed to the right team through Datadog. ## Problem As a company grows so does its number of teams. We need a way to be easily able to configure a new team with alerting on particular resources. ## Solution The [opsgenie-team](/components/library/aws/opsgenie-team/) component can be used as a virtual component to create a new team. :::tip **TL;DR** Create a new opsgenie-team component. Then tag resources with the `team: ` to start sending alerts through datadog. ::: ### Prerequisites - [How to Implement Incident Management with OpsGenie](/resources/deprecated/alerting/opsgenie) - [How to Implement SRE with Datadog](/layers/monitoring/datadog) - Deploy [`opsgenie-team`](/components/library/aws/opsgenie-team/) component ### Basic Configuration ``` components: terraform: opsgenie-team-sre: component: opsgenie-team vars: enabled: true # Name of the team and the name of the team tag value # Teams must be globally unique name: sre description: "SRE team" # Services can only be owned by a single team, so we put the services with the team # Service names are globally unique or will conflict with other teams # Services may not map directly to github repos, as multiple microservices might make up one service services: frontend: description: "Front End Lorem Ipsum" backend: description: "Back End Lorem Ipsum" members: - user: erik@cloudposse.com role: admin integrations: {} routing_rules: {} ``` ### Tagging Resources should then be tagged with the `team:team` name tag. This would look like ``` components: terraform: aurora-postgres: vars: tags: team: sre service: aurora-postgres ``` --- ## How to Onboard a New Service with Datadog and OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; Onboarding a new service with Datadog and OpsGenie ensures that monitoring and alerting are properly configured from the start. This guide will walk you through tagging your application, setting up OpsGenie teams, and adding necessary monitors and alerts to keep your service running smoothly and ensure the right team is notified in case of issues. ## Problem When spinning up a new service you want to quickly get a set of monitors up and running on the app, even if more detailed APMs are to come. Similarly, we immediately want a particular team to be considered the app owner, who will be notified if the app has an alert. By default, we should know if the new app is deployed and healthy, and its current load. We should also be able to quickly spin up new monitors for the application that are more specific to logs or other exposed metrics. ## Solution 1. Tag your Application with `service` and `team` 2. Ensure the team with users exists in OpsGenie 3. Add Monitors, Synthetics and SLOs for your Application. ## Ensure Tags are being sent to Datadog Ensure the deployment location is fetching tags to datadog. This is usually in Kubernetes or an AWS Service such as ECS or Elastic Bean Stalk. Follow this guide to ensure tags are being sent to datadog. [Datadog: How to Pass Tags Along to Datadog](/layers/monitoring/datadog/tutorials/how-to-pass-tags-along-to-datadog). ## Ensure Applications are Properly Tagged & labeled Ensure that your application is deployed with the following tags: 1. **required**: `team:` 2. **required**: `service:` 3. **optional**: `app:` In Kubernetes this might look like: ``` [...] metadata: labels: team: sre service: my-api app: my-app ``` ## Setup Teams in OpsGenie If Tag fetching is setup (#1) and your app is tagged (#2) then the datadog side is complete (excluding additional monitors). The next step would be to ensure that a team exists in opsgenie to receive an alert. We need to ensure an opsgenie-team component instance for `` exists. This would be a team with `name: ` to be set. **If no team already exists** follow this guide to create a new opsgenie team [How to Create New Teams in OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-create-new-teams-in-opsgenie). ## Add Users to Teams Once a team exists we need people to respond to the alert. Follow [How to Add Users to a Team in OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-add-users-to-a-team-in-opsgenie) to add people to a team. At this point the default suite of monitors will create a base system of monitoring and alerting for a new service. The default suite of monitors includes: - `(k8s) - ImagePullBackOff detected` - `(k8s) - CrashloopBackOff detected` Which will be triggered if the app fails to pull an image or crashes on startup. The healthcheck defined in k8s will crashloop the pod if it fails (such as a `/health` on an actuator springboot application not being able to be reached.) ## Add Additional Monitors This Guide walks through the different ways to monitor a new service [How to Monitor a new Service](/layers/monitoring/datadog/tutorials/how-to-monitor-a-new-service) Follow this guide to create new monitors: [datadog-monitor](/components/library/aws/datadog-monitor/) ## Add SLOs and Synthetic Checks (Optional) [How to create a Synthetic and SLO](/layers/monitoring/datadog/tutorials/how-to-create-a-synthetic-and-slo) :::info The reason for all these steps is to set it up once, and be able to reuse the existing configuration for new services. Once this process has been ran through once, only steps **#2 (Tagging your app)**, and **#3, #4 (Opsgenie Ensuring a team exists, with users exists).** This makes the process easy to get started with new applications. Once the app is deployed you can start investigating Key Metrics you want to record for your app and start developing SLOs ::: --- ## How to Setup Rotations in OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Setting up rotations in OpsGenie ensures that on-call schedules are managed effectively, allowing team members to rotate between on-call and off-call periods. This guide will show you how to configure on-call rotations through the OpsGenie UI, enabling seamless schedule management without the need for GitOps. ## Problem You need to configure rotations to allow people to be on call for periods of time before rotating to off-call and having someone else cover. ## Solution :::tip This is done through the UI, because typically managers move schedules around and using GitOps is overkill. ::: This is done through ClickOps by going to: **Teams** → My-Team → On-Call → **On-call schedules** → + Add Rotations. Fill this out with the information of whose on call when! --- ## How to Sign Up for OpsGenie? import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; Signing up for OpsGenie involves choosing the right plan based on your organization’s needs. This guide will help you understand the features and limitations of the different OpsGenie plans, with a recommendation to use the Enterprise plan for its comprehensive capabilities, particularly in managing service alerts and incidents. ## Problem You’ve been asked to [sign up for OpsGenie](https://www.atlassian.com/software/opsgenie/pricing), but are unsure of what plan is needed for your organization. The pricing increases depending on features, but you’re not sure what features will be required. ## Solution [https://www.atlassian.com/software/opsgenie/pricing](https://www.atlassian.com/software/opsgenie/pricing) :::tip We recommend the Enterprise Plan. ::: Some resources cannot be provisioned unless on a Standard or Enterprise plan. While our terraform modules support all plans, what we will do will be limited. Using Standard is ok to start, but we really recommend Enterprise has some more features we recommend leveraging. ### Option 1: Enterprise Plan (Recommended) The main thing we can use from the Enterprise plan (and supported by the component) is “Service Subscriptions”. > Opsgenie enables you to map alerts to the business services they impact and have a clear understanding of which teams need to respond and who needs to be kept up to date on the progress towards resolution. Disparate teams are notified simultaneously and presented with the tools they need to collaborate during resolution.With this, it basically means that the services we create (e.g. backend, a website, an app, a k8s deployemnt, or really any abstraction we can call a service) can be managed like this: ``` Service → Datadog alert → Opsgenie business service → Opsgenie alert → Opsgenie incident → Team notification → Escalation ``` This entire chain can be used only in the Enterprise plan. All of these things are only available in the Enterprise Plan: [https://www.atlassian.com/software/opsgenie/service-aware-incident-management](https://www.atlassian.com/software/opsgenie/service-aware-incident-management) So in short, if you have 20 microservices that send metrics to Datadog, Datadog monitors send alerts to Opsgenie, Opsgenie creates alerts per service (which is your microservice). If incidents are enabled and configured, then Opsgenie creates an incident from an alert and assigns it to the team. The last part (alert → incident → team notification → incident status page → acknowledgment → postmortem) is only in Enterprise plans. ### Option 2: Standard Plan In Standard, all parts can be deployed and used separately, e.g. ``` Service → Datadog alert → Opsgenie alert → Team notification → Escalation ``` Or... ``` External alert → Opsgenie alert → Opsgenie incident → Team notification → Escalation ``` They don’t allow to use of Opsgenie services to create incidents and notify teams in the Standard plan. No `Opsgenie business service` involved, which is > Opsgenie enables you to map alerts to the business services they impact and have a clear understanding of which teams need to respond and who needs to be kept up to date on the progress towards resolution. Disparate teams are notified simultaneously and presented with the tools they need to collaborate during resolution. ## Troubleshooting ### Error: `You do not have the right to use Service API, Took: 0.002000, RequestId: f4550375-efa7-49bc-ac04-3a0d6709f7d1` This is related to [https://community.atlassian.com/t5/Opsgenie-questions/Using-Incident-API-trial-account/qaq-p/1455015](https://community.atlassian.com/t5/Opsgenie-questions/Using-Incident-API-trial-account/qaq-p/1455015) for the [https://docs.opsgenie.com/docs/service-api](https://docs.opsgenie.com/docs/service-api). Upgrade the Opsgenie trial if your plan is currently on a trial. > Service API is only available to Standard and Enterprise plans --- ## How to Implement Incident Management with OpsGenie import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import CategoryList from '@site/src/components/CategoryList'; Implementing incident management with OpsGenie involves setting up a standardized, reusable method to handle incidents effectively. This guide outlines the strategy used by Cloud Posse to integrate OpsGenie with Datadog for incident management, ensuring rapid response and remediation while minimizing alert fatigue. ## Introduction Cloud Posse advises customers with very similar monitoring requirements because we have standardized our delivery model for our customers. This is our strategy for implementing OpsGenie for Incident Management. The actual fine-tuning of any implementation needs to be tailored to the customer’s needs based on the [Design Decisions](#design-decisions). Our goal with this guide is to define the reusable method for incident management we use with our customers that we can readily implement time and time again. Also, make sure to review [How to Implement SRE with Datadog](/layers/monitoring/datadog). ## Problem The number of events and alerts happening in modern systems easily overwhelms the Signal-to-Noise Ratio (SNR) making them generally ineffective as a means to escalate. We can adjust thresholds all day long and still never silence the system. And even if we silence them, it only reduces our visibility but not the fact that they are still happening. All we really care about is what measurably and adversely affects the business. When the business is affected, we need to rapidly respond, triage and remediate the issue. Tools like Datadog, Sentry, NewRelic are primarily concerned with (and very good at!) raising alerts, but not yet mature at handling complex escalations. [Datadog's Incident Management](https://www.datadoghq.com/blog/incident-response-with-datadog/) is not yet a mature product and [there’s no Terraform support for managing it](https://registry.terraform.io/providers/DataDog/datadog/latest/docs%20). ## Solution Route all events through Datadog. Leverage the advanced capabilities of Datadog to surface meaningful alerts that are enriched with tags used by OpsGenie to escalate based on a matrix of rules. This Document describes the OpsGenie side of things and how it works with Datadog, also see, why we [Use OpsGenie for Incident Management](/resources/adrs/adopted/use-opsgenie-for-incident-management) and [How to Implement SRE with Datadog](/layers/monitoring/datadog). ## Terminology The terms _event, alert_ and _incident_ are frequently used interchangeably, but we need to define them strictly for our purposes.
Event
An **event** is an observed change to the normal behavior of a system, environment, process, workflow or person. All events have tags. See Tag Mapping. Example: AWS Console Login by IAM User
Alert
An **alert** is a notification that a particular type of event (or series of events) has occurred, which is sent to responsible parties for the purpose of spawning action. Example: AWS Console Login using Root User Credentials
Incident
An **incident** is a type of event that negatively affects the Confidentiality, Integrity, and/or Availability (CIA) at an organization in a way that impacts the business. Read more [here](https://support.atlassian.com/opsgenie/docs/what-is-an-incident/)
Escalation
An **escalation** is a means of responding to an event based on schedules, rotations, and routing rules that deliver notifications via multiple mediums (email, phone, SMS, app, webhook, etc) Example: Send a notification event to the mobile device of the on-call engineer, if no acknowledgement after 5 minutes, fallback to the backup on-call engineer
Service
A **service** is a deployable instance, or part, of an application as a whole. A **service** is deployed to many environments. This is important to understand because it follows the Datadog Service Definition [[1](https://docs.datadoghq.com/tracing/visualization)] [[2](https://docs.datadoghq.com/getting_started/tagging/unified_service_tagging/?tab=kubernetes#overview)], and maps directly to an OpsGenie Service (see below). [OpsGenie Official Service Definition](https://support.atlassian.com/opsgenie/docs/what-are-services-in-opsgenie/) Example: Cert-Manager is a Service, as part of an EKS app, deployed to many environments.
App
An **App** is a collection of services that represent a group of functionality for the end user. This acts as a logical grouping and is on the roadmap for cascading impact. Example: My-EComm-Store is an app, EKS is an app (end user is developers)
Team
A **Team** is a group of users that an alert can be routed to. [Opsgenie Team Definition](https://support.atlassian.com/opsgenie/docs/what-are-teams-in-opsgenie/) Example: SRE is a team
Routing Rules
A team has Routing Rules which controls the logic of where an event goes within a team.
Escalations
A team has Escalations, as described above.
Schedules
A team has Schedules which allows users to have times that they are on call.
Owned Services
A team has owned Services, as described above.
Some definitions borrowed from [Daniel Miessler's Event Alert Incident](https://danielmiessler.com/study/event-alert-incident/) :::caution All incidents are events, but not all events and alerts are incidents. ::: :::info Since SLOs are by definition business-critical, any SLO violation is automatically an incident. ::: :::info An important note here is the **Service** mapping from **Datadog** to **OpsGenie**. It is a 1-1 Mapping, Datadog has a very easy to view graphing and listing of these services. In OpsGenie this requires _explicit_ owning of a service by a team. ::: ## Incidents An Incident is an escalated alert with a business impact. All SLOs by definition have business impacts and therefore when violated should be considered an incident. We define anything of severity P1 or P2 as an incident and therefore having business impact. An alert is automatically escalated to an incident by its severity as defined in the monitor. ## Component: `opsgenie-team` ### Component Page [opsgenie-team](/components/library/aws/opsgenie-team/) ### How to Guides ### Goals - We want to be able to define a simple DSL in YAML for expressing an opinionated way to define escalations. - We want to be able to define an opinionated tag-based approach to OpsGenie so that any integration (e.g. Datadog) can send alerts and incidents, by simply adding tags like `team:sre`, `service:eks`. - We want to define a clear line between a constant alerting ruleset (Opsgenie), and the data being fed to it (Datadog) - OpsGenie Component shouldn’t need to change unless teams or services change. - New alerts, monitors, and SLOs do not require any change to the OpsGenie configuration. - It is the responsibility of the integration (e.g. Datadog) to determine the tags. If the integration cannot enrich its own events with tags, that integration should pass through something like Datadog which can enrich the events. :::caution OpsGenie cannot escalate based on any mathematical expressions (e.g. rate or frequency of alerts). As an alternative, consider using Datadog to determine the rate or frequency of some event and alert on that. ::: ### What does it do Our **OpsGenie-Team** Component handles the creation of many resources. The component is highly configurable to allow full reference to the underlying resources. However, we also want to have sane defaults that allow short snippets to quickly spin up new fully-fledged teams in OpsGenie. :::info The component follows the **DataDog recommended way for integrating with OpsGenie**. By creating one integration per team. [https://docs.datadoghq.com/integrations/opsgenie/](https://docs.datadoghq.com/integrations/opsgenie/) ::: ### Limitations ##### Cannot Add dependent Services - Incident Routing cannot add dependent services - in api versions `v1` and `v2` a `service_incident_rule` object has `serviceId` as type string, in `webapp` this becomes `serviceIds` of type `list(string)` [https://github.com/opsgenie/terraform-provider-opsgenie/issues/282](https://github.com/opsgenie/terraform-provider-opsgenie/issues/282) ##### Cannot Add Stakeholders [https://github.com/opsgenie/terraform-provider-opsgenie/issues/278](https://github.com/opsgenie/terraform-provider-opsgenie/issues/278) ##### No Resource to create Slack Integration - [https://github.com/DataDog/terraform-provider-datadog/issues/67](https://github.com/DataDog/terraform-provider-datadog/issues/67) - Full list of integrations supported [https://docs.opsgenie.com/docs/integration-types-to-use-with-api](https://docs.opsgenie.com/docs/integration-types-to-use-with-api) - The rules (e.g. match conditions, like priority) for the Slack integration are in the `opsgenie_integration_action` but since there isn’t a data source to pull the integration id, if we wanted to create this resource in terraform, we’d need the user to manually grab the guid integration id in order to create the rule. ##### User management can be cumbersome - SAML/SSO integration in OpsGenie does not retrieve any specific attributes - There is no way to map a SAML/SSO user to an OpsGenie team or role - Creating users at the moment is done manually using yaml which may not be desired - An alternative is a separate point of truth such as Okta or Github where teams are already managed. If this is the case, we can use the `okta_user` or `github_user` data source, grab a specific attribute from okta or team associated to the gh user, and map that to the same in opsgenie using `opsgenie_user` data source. ### How it Works #### Alerting a Team A team is alerted by tagging their datadog integration name in the **message** of an alert. e.g. adding `@opsgenie-`. This can be done automatically by a team tag. In code this looks like `@opsgenie-${team.name}` where datadog fetches the team from the label applied to the resource. :::caution The tag must be fed to a monitor via the data source (like aws integration - and the aws resource), Datadog will not use a monitor tag. ::: Thus if a message for datadog is `@opsgenie-${team.name}` datadog looks for the team tag and will change the message to be `@opsgenie-sre` if the team was `sre`. Datadog knows to look at an opsgenie integration because of the `@opsgenie` portion, then under that integration configuration, looks for a `Datadog Integration Name` with the team specified, `sre` in this case. If it’s found then Datadog now knows what integration to use! If it doesn’t find it, datadog looks for a Datadog Integration name called `datadog`, which acts as the default. Each Monitor has a default Integration to use. Each Integration has a default team associated to it. A `monitor`s message is the `message` field + the `alert_tag` of the component. Where the alert tag appends the `@opsgenie-${team.name}` ### Incidents Rules The OpsGenie Abstract component creates the rules that specify when an alert is an incident. :::tip We recommend that all **P1** and **P2** alerts **are Incidents**. We recommend all teams use company defaults and only in very special cases do teams override this setting. This helps promote an understanding of how severe a problem is. ::: ### Severity Mapping The severity levels (P1-P5) should be standardized at an organizational level and documented in an ADR. The severity level determines the escalation path. We recommend for consistency that given any severity, the escalation mechanism is identical, but who it escalates to will vary based on service. :::tip In general, reserving **P1** and **P2** for reactive responses and escalate 24x7. **P3**-**P5** are for proactive responses and only escalate during business hours, if at all. ::: It’s up to every organization to define these mappings for their organization as it will form the basis of the escalation matrix. The severity should be scaled based on the impact to the business, not necessarily the impact to the service. For example, if a non-essential microservice is entirely offline, but is does not effect the delivery of core business services, it should still not be a P1. :::info Datadog priorities map one-to-one to OpsGenie severity. ::: #### Recommendations | **Severity** | **Response** | **Event** | **Description** | **Examples** | | ------------ | ------------ | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | P1 | Reactive | Incident | SLO violation. Someone’s getting paged 24x7. All hands on deck. Production is down. All customers are affected. Critical 24x7 | - Production Backend API has been down for 3 hours this week (Failed SLO)- Production Backend API is down (Over 5 min) (Synthetic Failed)→ SLO not broken yet but in violation (SLO Trending to failure) | | P2 | Reactive | Incident | SLO violation. Error budget in jeopardy of being depleted. Some customers are being severely affected. Someone’s getting paged 24x7. Important 24x7 | - Production API Down 1 Hour this week (Warning SLO)- Production API Down 1 Minute This week (SLO Trending to Warning)- Customers in `us-west-2` Are experiencing significant latency (Breaks SLO on response time but not all customers) | | P3 | Proactive | Alert | Alert Created Regular Business Hours:App Failures, Crashes, Minor Downtime(non-critical apps), SLI Violation.Lesser Violations that could lead to P1/P2 | - Pods of deployment X are crashlooping- High Response Time Backend Service | | P4 | Proactive | Alert | Things that need to be acknowledged by a team but will not necessarily lead to failures. | - Number of Kube nodes reached > 10 | | P5 | Proactive | Event | Informational Alarms that self resolve. Metrics could create be created off number of these. | - Feature Flag enabled in environment X | ## References - [https://support.atlassian.com/opsgenie/](https://support.atlassian.com/opsgenie/) - [https://en.wikipedia.org/wiki/Incident_management_(ITSM)](https://en.wikipedia.org/wiki/Incident_management_(ITSM)) - [https://sre.google/sre-book/monitoring-distributed-systems/](https://sre.google/sre-book/monitoring-distributed-systems/) --- ## Setup Alerting import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Quick Start You will need a OpsGenie account! Please follow the sign up steps in [How to Sign Up for OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-sign-up-for-opsgenie/) ## How To ### Vendor Like most other components we need to vendor in the component. This can be done with the following: ### Deploy #### Setting up OpsGenie Teams Teams are the foundation of our OpsGenie implementation. Each team has schedules, rotations, integrations, and more. We need teams to be able to be easily spun up, but also be fully customizable. To accomplish this we have setup a reference team YAML file with many options written out but left disabled. You can find it here `stacks/catalog/opsgenie-team/_defaults.yaml`. Please read [How to Create New Teams in OpsGenie](/resources/deprecated/alerting/opsgenie/how-to-create-new-teams-in-opsgenie/) and [How to Implement Incident Management with OpsGenie](/resources/deprecated/alerting). Once your company defaults have been setup and your teams customized, simply deploy the `opsgenie-team` component. `atmos terraform deploy opsgenie-team/ -s core-gbl-auto`. we have provided 2 sample teams `opsgenie-team/sre` and `opsgenie-team/app-team-1`. Their definitions can be found `stacks/catalog/opsgenie-team` --- ## Deprecated Documentation import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This section contains documentation for deprecated features and tools. While these features are no longer actively maintained, their documentation is preserved here for reference. ## Deprecated Features --- ## Decide on how Spacelift will use external private modules import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Sometimes there will be a need to host private Terraform modules that may exist outside the infrastructure monorepo. When this is the case, Spacelift needs access to these module registries or repositories. There are a few ways to go about it, with various tradeoffs to ensure secure and efficient module management while minimizing complexity and risk. **Date**: **19 Oct 2021** ## Status **DRAFT** ## Problem App (or other) repositories outside the infrastructure monorepo may require access to other GitHub repositories, such as an in-house terraform module. By default, Spacelift only has access to a single repository when cloning and only clones the app repository. ## Considered Options #### Option 1: Use a GitHub Personal Access Token (PAT) **RECOMMENDED** Cloud Posse usually uses this approach on private runners. - `.netrc` would require a read-only PAT (spenmo doesn’t like PATs) on a shared bot user - PAT could have access to multiple repositories - Private terraform modules should be referenced like `git::https://github.com/your-org/terraform-aws-example` #### Option 2: Use the Spacelift private module registry - This uses the Spacelift Github App permissions - The infrastructure module would be turned into a terraform module in the private registry. - The `source` of the terraform module would point to spacelift i.e. `spacelift.io///` e.g. `spacelift.io/client-org/infrastructure/aws` - Local spacelift API creds would be needed to iterate to retrieve the module source - or we could set up the terraform for local sources and use something like an on-the-fly terraform change in a `before_init` hook - [https://docs.spacelift.io/vendors/terraform/module-registry](https://docs.spacelift.io/vendors/terraform/module-registry) #### Option 3: Use a Github App - Private terraform modules should be referenced like `git::https://github.com/your-org/terraform-aws-example` #### Option 4: Use a GitHub “deploy” key **NOT RECOMMENDED** Cloud Posse recommends avoiding this approach. - Private terraform modules should be referenced like `git::https://github.com/your-org/terraform-aws-example` - Configure an SSH “deploy” key on a shared bot user - Deploy key will only have access to a single repository - [https://docs.github.com/en/rest/deploy-keys](https://docs.github.com/en/rest/deploy-keys) - Unfortunately, deploy keys cannot be shared across repositories, meaning many keys will need to be managed if there are a lot of private repositories. ### Options on how to use the key #### Option 1: Private worker pools bake a static token in userdata Cloud Posse usually uses this approach on private runners. - Retrieve the token from SSM - Save it in the userdata of the spacelift worker pool - Mount the token as an environment variable for all runs #### Option 2: Use only a `before_init` hook - The app id and app pem are used to create a JWT which is used to create a temporary 1hr access token - The spacelift worker pool uses a `before_init` hook - to retrieve the token from SSM to create the `.netrc` file on-the-fly - to run `git config url.insteadof` to replace the source with the user and token #### Option 3: Use a mounted file per stack Cloud Posse recommends avoiding this approach. - This would require retrieving the secret during the admin stack `spacelift` component run and adding the secret to each stack as a file - Then once the secret is added, a `before_init` hook would be required ## References - [https://docs.spacelift.io/vendors/terraform/external-modules](https://docs.spacelift.io/vendors/terraform/external-modules) - [https://superuser.com/questions/1314064/read-only-access-to-github-repo-via-ssh-key](https://superuser.com/questions/1314064/read-only-access-to-github-repo-via-ssh-key) - [https://docs.github.com/en/developers/overview/managing-deploy-keys#deploy-keys](https://docs.github.com/en/developers/overview/managing-deploy-keys#deploy-keys) - [https://www.terraform.io/language/state/remote-state-data](https://www.terraform.io/language/state/remote-state-data) - [https://maelvls.dev/gh-actions-with-tf-private-repo/](https://maelvls.dev/gh-actions-with-tf-private-repo/) --- ## Decide on Spacelift Administrative Stack Auto-deployment import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Spacelift administrative stacks support auto-deployment to streamline the process of provisioning new component stacks, but the ease of use and automation must be balanced against the risk of accidental deletions due to misconfiguration. Reviewing these considerations will help decide the best approach for your organization’s needs. ## Problem Spacelift (and for that matter, pretty much any CI/CD platform including Atlantis) will only run workflows that have been configured to run on the platform (e.g. there is no auto-discovery of stacks). While systems like GitHub Actions will automatically run workflows in a branch or Pull Request from any repo, Spacelift by design does not do this for security reasons, since that would mean any rogue repo or branch could modify infrastructure. In order for the stack to run in spacelift, the configuration needs to be merged into `main` and confirmed (e.g. applied). Frequently, users forget about this, so after merging to main the infrastructure stack is not confirmed and then they are left scratching their heads why their stack didn’t run. ## Context There are 2 kinds of Spacelift stacks: administrative stacks and component stacks (which are created by an admin stack). The “infrastructure” stack is an _administrative stack_ that deploys and manages component stacks. This means this stack has special permissions to manage spacelift itself, but actually no credentials to manage AWS (or other services). In the screenshot, you can see that the stack is _Administrative_ on the status line under Resources. This admin stack is manually created in Spacelift (it’s how it knows about your infrastructure repository) and it runs the Terraform code in the `components/terraform/spacelift` component to create all your other component stacks. When making changes or adding a new component to a stack configuration in Spacelift, those changes must be applied first by the administrative stack before Spacelift can do anything with it (e.g. show a terraform plan of this stack). ## Considered Options ### Option 1: Autodeploy Spacelift Administrative Stack Usually, CloudPosse sets this admin stack to `autodeploy=true` for customers so that new stacks are provisioned when detected otherwise the risk is that the admin stack is unconfirmed and forgotten about. :::note Note that by turning on auto deploy on the admin stack, it does not turn auto deploy on the component stacks. We default component stacks to be manually confirmed before the changes are applied. ::: #### Pros - Adding stacks feels more natural. As soon as you merge the PR with the new stack, Spacelift will configure it without further intervention by users. #### Cons - Stacks can be accidentally deleted by misconfiguration (E.g. user error) in a PR. When that happens, the history of that stack will be lost in the Spacelift UI. The AWS stacks themselves are not affected at all. We are working with Spacelift to add a deletion protection flag to avoid this from happening. ### Option 2: Manually Confirm Spacelift Administrative Stack :::caution If `autodeploy=false` on the admin stack, then new stacks will have to be manually confirmed after each PR merge before any changes are planned, which is frequently confusing for developers. ::: #### Pros - The risk of accidentally deleting a stack configuration in the UI is reduced #### Cons - Users are frequently confused why their stack does not show any plans or why nothing is terraformed. - Users forget to confirm the changes to the Spacelift configuration because this implementation detail is not obvious to them. --- ## Decide on Spacelift Worker Pool Architecture import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; Spacelift Workers are deployed in VPCs with scoped IAM permissions to ensure robust automation capabilities while minimizing security risks. By default, we avoid automating certain core system accounts, but this can be adjusted as needed. Review these considerations to determine the best approach for your organization. ## Problem In order to provision infrastructure, we need to sufficient IAM permissions (e.g. Service Accounts) for Terraform to modify infrastructure. If we give it too much power, very bad things can happen; if we don’t give it enough, then we have no way to run automation in an unattended fashion. In an ideal world, Spacelift would use the credentials of the person making the changes, but that is not the case and this is not a problem unique to Spacelift (e.g. Terraform Cloud, Atlantis, all work this way). Instead, Terraform runs with the permissions granted to the Spacelift runners and we use Rego policies to control what a user can or cannot do. ## Context Spacelift can use workers run Terraform inside your VPCs much like GitHub Action runners. These workers are then provisioned with IAM service account profiles that determine what Spacelift can or cannot do. :::caution Spacelift Enterprise subscription (or trial) is required to use self-hosted workers. See [How to Sign Up for Spacelift](/resources/deprecated/spacelift) for details. ::: ## Considered Options ### **Option 1:** Use Spacelift to Manage _Almost_ Everything :::tip Our Recommendation is to use Option 3 because it exposes the organization to less risk by not permitting Spacelift to modify anything in these core system accounts, while still permitting drift detection and accounting of what’s been deployed using Spacelift. ::: Configure IAM role for Spacelift with mostly read-only, view-only access in the root, security and audit accounts. Prevent any account deletion, cloud trail disablement, etc. We would need to more precisely define what capabilities we tolerate in these accounts. #### Pros - View everything provisioned by terraform under a single pane of glass, including outputs - Automatic Drift detection #### Cons - Still need to manually run terraform in the accounts which have read-only permissions under Spacelift ### **Option 2:** Use Spacelift to Manage Non-root, Security and Audit Accounts :::info We typically deploy this configuration, but are leaning towards Option 1 instead. ::: #### Pros - Enhanced security posture restricts the damage that can be done to the organization #### Cons - Cannot use Spacelift for accounting purposes to reconcile what has been deployed in these accounts - Cannot use Spacelift to release changes in these accounts ### **Option 3:** Use Spacelift to Manage Everything #### Pros - 💯 automation of all infrastructure via Spacelift and Terraform - Total accountability for everything under management, including drift detection #### Cons - The instance profile on the Spacelift workers enables _God_ mode for the entire AWS organization. Anyone with access to the worker can do _anything_ in the organization, including traverse accounts or possibly delete accounts. ## References - [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) - [How to Sign Up for Spacelift](/resources/deprecated/spacelift) - [spacelift-worker-pool](//components/library/aws/spacelift/worker-pool/) (component) --- ## Design Decisions(9) import DocCardList from "@theme/DocCardList"; import Intro from "@site/src/components/Intro"; Review the key design decisions for how you'll leverage Spacelift for continuous delivery of your infrastructure with Terraform and Atmos. --- ## Spacelift FAQ import Note from '@site/src/components/Note'; ## Why do we need Spacelift? See [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) to learn why. ### Can we still use Spacelift despite Terraform licensing changes? :::important This is not a technical question. This is a matter of legal interpretation of the [HashiCorp BSL license (aka BUSL)](https://www.hashicorp.com/bsl) (and [binding FAQ](https://www.hashicorp.com/license-faq)), and we are not lawyers. We recommend consulting with your legal team to understand the implications of using Spacelift with Terraform. **Cloud Posse cannot provide legal advice on this matter.** ::: It depends. - You can use Spacelift with MPL licensed Terraform (e.g. versions including 1.5.7 and earlier) - You can use Spacelift with all versions OpenTofu But you cannot use Spacelift with BSL licensed versions of Terraform. ## Why do we need an Enterprise Agreement? For Cloud Posse engagements, you’ll need to have the **Enterprise-tier** agreement which enables these features. - **“Self-hosted private worker pools”** which will execute `terraform plan/apply` on the EC2 instances that we provision and control, not on the Spacelift cloud -- better, or required in many cases, for security and compliance. - This is deployed to private subnets and its IPs are allowlisted in security groups so it can manage EKS clusters, helm charts, aurora/rds cluster resources (databases, users), and other private resources. - **“Webhook-based audit logging”** which is used to send notifications from Spacelift to external systems like Datadog or Opsgenie. These notifications include audit login, the status of each stack, each run, etc., which is useful for compliance and stacks’ status visibility. - For a full breakdown of the differences, check out the [spacelift pricing](https://spacelift.io/pricing). This is required in order to provision any database users or schemas, manage EKS clusters on a private subnet, or in general manage any resources residing in private subnets in a VPC. The other benefit with an Enterprise Agreement is that it will fix your costs, even if you burst your Spacelift Workers. If you were on a Metered (aka “pay-as-you-go plan”), you will be charged for those bursts and that will add up to thousands per month. The Spacelift Enterprise 30 day trial (with an option to extend it) should be signed up to take advantage of these features. :::tip Referrals Cloud Posse has a partnership with Spacelift that can save you cost! Please reach out to Cloud Posse for a referral. ::: ## Who should you contact regarding Spacelift contracts and pricing? Reach out to Pawel Hawtry [pawelh@spacelift.io](mailto:pawelh@spacelift.io) and mention you are working with Cloud Posse. Alternatively, we’re happy to facilitate an introduction directly. ## What if Spacelift is too expensive? First, make sure you understand why we chose to [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform). Spacelift wants to earn your business and they only want happy customers. Work with them to reach a middle ground that makes everyone happy. Alternatively, you can use our [free GitHub Actions for Atmos with Terraform](/layers/gitops). This is a great way to get started with GitOps and Terraform without any additional cost. ## Why can’t we start with an entry-level plan? (See [How to Sign Up for Spacelift](/resources/deprecated/spacelift) ) - Only enterprise plans support self-hosted workers which are required in order to manage any infrastructure provisioned inside of VPCs (E.g. Databases, non-public EKS clusters, terraform provisioners, etc). Additionally, using EC2 Instance Profiles we’re able to grant the precise roles we want to give Spacelift without any hardcoded credentials. - Only enterprise plans support fixed costs. Cloud is metered billing, which means if we scale up our workers temporarily to flush out a large queue of plans, they will charge for the additional workers on a monthly rate. They do not have the ability to disable this and it happens automatically. Several customers have accidentally encountered _Bill Shock_ as a result of this! (we got it all sorted out for them in the end) ## Is Spacelift SOC2 Compliant? Yes, Spacelift has SOC2 Type II certification performed by an independent external auditor who confirms the effectiveness of internal controls in terms of Spacelift security, availability, processing integrity, confidentiality, and privacy of customer data. [https://spacelift.io/security](https://spacelift.io/security) Spacelift’s SOC2 Type 2 report and the pentest report from April 2021 is attached below. If anyone ever has any security-related questions, they can reach out to [@Wojciech Syrkiewicz-Trepiak](mailto:wojciechs@spacelift.io), Spacelift’s security engineer. Spacelift also has D&O and Cyber & Tech Insurance in place. Please reach out to Spacelift to request a report. ## How many runners will we need? it really varies based on your usage patterns and size of your infrastructure, but start with a minimum of 2-3. Since Spacelift permits you to scale up for short periods of time without metering, we recommend starting with a commitment of 2-3 runners online at any time. We’ll configure your workers to autoscale up as necessary. ### Can we consolidate multiple Spacelift Organizations under one bill? Yes! Spacelift can consolidate the billing. For example, if you have multiple GitHub Organizations, each one associated with a different Spacelift Organization, Spacelift can consolidate the invoice. ### Where do we sign up? How you sign up depends on if you’re using GitHub or GitLab. --- ## Setup import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; import AtmosWorkflow from '@site/src/components/AtmosWorkflow'; ## Quick Start | Steps | | | :------------------------------------- | :---------------------------------------------------- | | 1. Sign up for Spacelift | | | 2. Create an API Key | | | 3. Vendor Spacelift components | `atmos workflow vendor -f spacelift` | | 4. Push API Key to AWS SSM | `atmos workflow init -f spacelift` | | 5. Set Spacelift Environment variables | `source $(which set-spacelift-environment)` | | 6. Deploy `spacelift/spaces` | `atmos workflow deploy/spaces -f spacelift` | | 7. Deploy `spacelift/worker-pool` | `atmos workflow deploy/worker-pool -f spacelift` | | 8. Deploy Spacelift Root Admin stack | `atmos workflow deploy/root-admin-stack -f spacelift` | | 9. Deploy Spacelift Child Admin stack | `atmos workflow deploy/admin-stack -f spacelift` | | 10. Open the Spacelift UI | https://acme.app.spacelift.io | ## Requirements ### Sign Up for Spacelift First, sign up for Spacelift. Follow these steps, and see [the detailed documentation](/resources/deprecated/spacelift/tutorials/how-to-sign-up/) for additional information. 1. Sign up [Get started with Spacelift](https://spacelift.io/free-trial) 1. Install Spacelift GitHub app into the customer GitHub organization 1. Once the app is installed into the customer GitHub organization, the GitHub organization admin will receive an email from Spacelift 1. Click on the link in the email from Spacelift 1. After the Spacelift app has been installed and enabled, only the integration creator (the admin of the customer GitHub org) gets administrative permissions by default, so all other users and administrators must be granted their access using a login policy. 1. Add the following login policy. 1. Click `Policies` at the top 1. Click `Add Policies` 1. Set a name like `Login Policy` 1. Change the policy type to `Login policy` 1. Copy and paste the policy below 1. Change the `user_collaborators` to the customer github org 1. Add any additional `admin_collaborators`, if required 1. Add the Cloud Posse team `CLOUDPOSSE-CASE-SENSITIVE-TEAM-NAME` and non cloudposse `YOUR-CASE-SENSITIVE-TEAM-NAME` team names. 1. Finally, click `Create Policy` ```rego package spacelift # See https://docs.spacelift.io/concepts/policy/login-policy for implementation details. # Note: Login policies don't affect GitHub organization or SSO admins. # Note 2: Enabling SSO requires that all users have an IdP (G Suite) account, so we'll just use # GitHub authentication in the meantime while working with external collaborators. # Map session input data to human friendly variables to use in policy evaluation username := input.session.login member_of := input.session.teams github_org := input.session.member # Define GitHub usernames of non-github_org org external collaborators with admin vs. user access admin_collaborators := {} # case sensitive GitHub handles of additional admin collaborators user_collaborators := { "acme" } # case sensitive name of the github org # Grant admin access to github_org org members in the non cloud posse case-sensitive team # Do not use the slug here, use the name shown in github.com/org/teams admin { github_org member_of[_] == "YOUR-CASE-SENSITIVE-TEAM-NAME" } # Grant admin access to github_org org members in the Cloud Posse group # Do not use the slug here, use the name shown in github.com/org/teams admin { github_org member_of[_] == "CLOUDPOSSE-CASE-SENSITIVE-TEAM-NAME" } # Grant admin access to non-github_org org accounts in the admin_collaborators set admin { # not github_org admin_collaborators[username] } # Grant user access to accounts in the user_collaborators set allow { # not github_org user_collaborators[username] } # Deny access to any non-github_org org accounts who aren't defined in external collaborators sets deny { not github_org not user_collaborators[username] not admin_collaborators[username] } ``` 5. Verify Cloud Posse users have administrative access 1. Create an administrative API key 1. Click the three horizontal bars on the top right hand corner 1. Click `Settings` 1. Click `API KEYS` 1. Click Add new API Key 1. Give it a name like `Spacelift worker pool API admin key` 1. Enable `Admin` 1. Click `Add Key` 1. A file will download. Save the contents to our shared 1Password vault. This will give us the `key_secret` and the `token`. 1. Also share the 26 character string next to the name of the key which gives us the `key_id`. ### Upload the API Key In order for the `spacelift/worker-pool` component to connect to the given Spacelift account, we need to add the previously created Spacelift API Key to SSM. Create the following two parameters in AWS SSM Parameter Store in the same AWS account and region where the Spacelift Worker Pool will be deployed. In this case, that is `core-use1-auto`. | Key | SSM Path | Type | | :------ | :---------------------- | :------------- | | API ID | `/spacelift/key_id` | `SecureString` | | API Key | `/spacelift/key_secret` | `SecureString` | _HINT: The API key ID is displayed as an upper-case, 26-character alphanumeric value next to the key name in the API key list._ This step can be executed with an Atmos Workflow! Run the follow and enter the API Key ID and Secret when prompted: ## How To ### Vendor Spacelift adds three components, `spacelift/worker-pool`, `spacelift/admin-stack`, and `spacelift/spaces`. Vendor these components either with the included Atmos Workflows or with Atmos Vendor. ### Deploy Spacelift can be deployed with Atmos Workflows or by manual steps. If using Workflows, you must first upload the Spacelift API Key to AWS SSM, described above. Once the API Key is created in SSM, connected to Geodesic and run: ```shell set-spacelift-environment ``` All other configuration should be preset, but if you encounter errors, verify the following manual steps have been set up correctly. #### Setting up the Spacelift Worker Pool The `spacelift/worker-pool` component is responsible for deploying an Auto Scaling Group in AWS that will launch instances for Spacelift to attach a number of runners onto. For additional details, see [worker-pool](//components/library/aws/spacelift/worker-pool/). #### Deploy `spacelift/worker-pool` Deploy the component into the `auto` account in the primary region: After Terraform finishes applying, take note of the Terraform outputs `worker_pool_id`, `worker_pool_name`, and `trusted_role_arns`. ##### IAM configuration **Note:** This step should already be done by the identity automated setup. If you are having trouble with IAM permission and access in Spacelift, verify the follow role has been set correctly. You must give the created instance role permission to assume the Spacelift worker role. This is done by adding `iam_role_arn` from the output to the `trusted_role_arns` list for the `spacelift` role in `aws-teams` in `stacks/catalog/aws-teams.yaml`. #### Deploying Spacelift Stacks The `spacelift` component is responsible for provisioning the Admin Spacelift stacks. These Admin stacks are then responsible for deploying _all_ other Spacelift stacks. This is where each component stack is created. ##### Validate the `spacelift` catalog These configuration settings should be set already, but in case of errors, these are the dynamic variables set for a particular customer. 1. `repository` = Name of `infrastructure` repository 1. `branch` = Name of main/master branch 1. Set `components.terraform.spacelift.settings.spacelift.worker_pool_name` to the name of the worker pool you want to use for the `spacelift` component. This is typically set with default Spacelift settings at `stacks/orgs/acme/_defaults.yaml` ##### Set the Spacelift Environment To assume the Spacelift provider, we need to add the Spacelift API Key to our environment. Run the included script from Geodesic to assume the proper IAM role and pull the API Key from SSM: ``` set-spacelift-environment ``` Or manually set these ENV vars: ```bash export SPACELIFT_API_KEY_ENDPOINT=https://acme.app.spacelift.io export SPACELIFT_API_KEY_ID=... export SPACELIFT_API_KEY_SECRET=... ``` ##### Deploy the admin stacks Note that this is the only manually operation you need to perform in `geodesic` using `atmos` to create the initial admin stack. All other infrastructure stacks will be created in Spacelift by this admin stack. # Related Topics ## Pull Request Workflow 1. Create a new branch & make changes 2. Create a new pull request (targeting the `main` branch) 3. View the modified resources directly in the pull request 4. View the successful Spacelift checks in the pull request 5. Merge the pull request and check the Spacelift job ## How to use `spacectl`? See docs https://github.com/spacelift-io/spacectl ### Install ```shell ⨠ apt install -y spacectl -qq ``` Setup a profile ```shell ⨠ spacectl profile login gbl-identity Enter Spacelift endpoint (eg. https://unicorn.app.spacelift.io/): https://.app.spacelift.io Select credentials type: 1 for API key, 2 for GitHub access token: 1 Enter API key ID: 01FKN... Enter API key secret: ``` ### Listing stacks ```bash spacectl stack list ``` Grab all the stack ids (use the JSON output to avoid bad chars) ```bash spacectl stack list --output json | jq -r '.[].id' > stacks.txt ``` If the latest commit for each stack is desired, run something like this. NOTE: remove the `echo` to remove the dry-run functionality ```bash cat stacks.txt | while read stack; do echo $stack && echo spacectl stack set-current-commit --sha 25dd359749cfe30c76cce19f58e0a33555256afd --id $stack; done ``` --- ## Spacelift(4) import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import PartialSpaceliftRootStack from '@site/examples/legacy/snippets/stacks/orgs/acme/spacelift.yaml'; import PartialSpaceliftTenantStack from '@site/examples/legacy/snippets/stacks/orgs/acme/plat/spacelift.yaml' import CodeBlock from '@theme/CodeBlock'; import CategoryList from '@site/src/components/CategoryList'; GitOps is a cloud-native continuous deployment methodology that uses Git as the single source of truth for declarative infrastructure and applications. Changes to infrastructure or applications are made through Git commits, and the actual state is automatically adjusted to match the desired state expressed in the Git repository. This approach provides an audit trail for changes, simplifies rollback, and enhances collaboration and visibility across teams. ## The Problem Terraform itself focuses on managing the state of infrastructure. It does not provide a way of indexing, CI/CD, or collaborating on state. In fact, Terraform really wants to just be a CLI tool that you run locally. The problem is that with all our infrastructure spanning hundreds of stacks we cannot easily see the state of our infrastructure. Moreover, there could be many different teams of engineers who are responsible for different parts of the infrastructure. How do we enable them to collaborate on the same infrastructure? You will likely need to set up permissions and roles for that, and to future-proof those you'll need someone to govern those roles as teams change and infrastructure evolves. Lastly we want a way to automate our infrastructure as code. It does little good if code changes are not immediately reflected in the infrastructure. If several code changes happen before infrastructure updates, you could be left with difficult to resolve conflicts. ## Our Solution Spacelift is a hosted service that indexes the state of your infrastructure and provides a way to collaborate on it. It also provides a way to automate gitops and will continuously monitor your infrastructure for changes at a schedule you can configure. Spacelift breaks down access and gitops with policies that govern what can trigger changes and who can either read or update infrastructure. The policies use rego, an industry standard language for policy as code. Infrastructure is indexed by resource, label, status, or other facets. Dependencies can also be created between stacks, so that changes in one stack will lock other stacks until changes are done and then ensure that the dependent stacks are updated. Overall, Spacelift is a comprehensive way to make sure that your infrastructure is easy to automate and manage. Because our solution keeps the state of your infrastructure in S3, you also are not locked in and can use other tools to manage your infrastructure in case either GitHub or Spacelift go down. ## Spacelift Stack Lifecycle Spacelift has its own notion of a "Stack", which is a workspace for Terraform with a slug identitifier. Stacks have a lifecycle that is triggered by events like a git commit or a drift schedule. The lifecycle is as follows: ```mermaid --- title: Basic Spacelift Stack Lifecycle --- stateDiagram-v2 direction LR [*] --> Triggered : Git Commit Triggered --> Run state Run { direction LR state if_changes <> [*] --> Planning Planning --> if_changes if_changes --> Confirm: if changes if_changes --> Applied: if no changes Confirm --> Applied Applied --> [*] } ``` Runs are what Spacelift uses to scope changes to a given event like a commit SHA. Within the run, Spacelift uses a plan policy to dictate what changes require confirmation and what changes can be applied automatically. After a run is complete, Spacelift will evaluate other trigger policies and dependencies to determine if other stacks need to be updated. ## Implementation We have three components that implement Spacelift. The first is the `spacelift/admin-stack` component which creates admin stacks in Spacelift. The second is the `spacelift/spaces` component which creates Spacelift Spaces and manages policies in those Spaces. The third is the `spacelift/worker-pool` component which creates a worker pool for Spacelift to use self-hosted workers. ### Global Configuration In order to apply common Spacelift configuration to all stacks, we need to set a few global Spacelift settings. The `pr-comment-triggered` label will be required to trigger stacks with GitHub comments but is not required otherwise. More on triggering Spacelift stacks to follow. Add the following to `stacks/orgs/NAMESPACE/_defaults.yaml`: ```yaml settings: spacelift: workspace_enabled: true # enable spacelift by default before_apply: - spacelift-configure-paths before_init: - spacelift-configure-paths - spacelift-write-vars - spacelift-tf-workspace before_plan: - spacelift-configure-paths labels: - pr-comment-triggered ``` Furthermore, specify additional tenant-specific Space configuration for both `core` and `plat` tenants. For example, for `core` add the following to `stacks/orgs/NAMESPACE/core/_defaults.yaml`: ```yaml terraform: settings: spacelift: space_name: core ``` And for `plat` add the following to `stacks/orgs/NAMESPACE/plat/_defaults.yaml`: ```yaml terraform: settings: spacelift: space_name: plat ``` ### Admin Stacks Spacelift allows some stacks to manage other stacks (or even itself). This is useful for making new component instances show up in the Spacelift UI. The `spacelift/admin-stack` component takes the `atmos.yaml` and uses it to derive what stacks need to be created in Spacelift. By updating your atmos stacks, you'll see the changes reflected in Spacelift. Not all stacks are managed by Spacelift. If a management stack sees that a stack does not have `workspace_enabled` set to true, it will be ignored by the admin stack.
Managed vs. Unmanaged Components ```yaml components: terraform: vpc: settings: spacelift: workspace_enabled: true vars: enabled: true ``` ```yaml components: terraform: aws-sso: settings: spacelift: workspace_enabled: false vars: enabled: true ```
### Spaces The `spacelift/spaces` component maintains Spacelift spaces and configures all policies in those given Spaces. Policy labels can allow policies to be automatically enforced on a given group of stacks. A Spacelift Space is that group. We deploy `spacelift/spaces` three times. First we deploy a `root` Space for all Spacelift administrative resources, and then we deploy two more Spaces for the `plat` and `core` tenants. Spacelift exists outside of the AWS ecosystem, so we define these components as outside our standard stack organization. ```diff + stacks/orgs/NAMESPACE/spacelift.yaml + stacks/orgs/NAMESPACE/core/spacelift.yaml + stacks/orgs/NAMESPACE/plat/spacelift.yaml ``` The `root` Space in Spacelift is responsible for deploying the root administrator stack, `admin-stack`, and the Spaces component, `spaces`. Since the root administrator stack is unique to tenants, we modify the stack context to create a unique stack slug, `root-gbl-spacelift`. A tenant-specific Space in Spacelift, such as `core` or `plat`, includes the administrator stack for that specific Space and _all_ components in the given tenant. This administrator stack uses `var.context_filters` to select all components in the given tenant and create Spacelift stacks for each. Similar to the root administrator stack, we again create a unique stack slug for each tenant. For example `core-gbl-spacelift` or `plat-gbl-spacelift`.
Spacelift Spaces {PartialSpaceliftRootStack} {PartialSpaceliftTenantStack}
In the Spacelift UI, you should see the administrator stacks created. Typically these should look similar to the following: ```diff + root-gbl-spacelift-admin-stack + root-gbl-spacelift-spaces + core-gbl-spacelift-admin-stack + plat-gbl-spacelift-admin-stack + core-ue1-auto-spacelift-worker-pool ``` ### Worker Pools The `spacelift/worker-pool` component creates a worker pool for Spacelift to use. It manages an ASG (Autoscaling Group) in AWS and the instances effectively run drift detection and regular Spacelift stack runs. The component lives in the `auto` stage since its considered automation infrastructure. Some common things to tweak while working with this component include maximum instances (in-case runs are often blocked by busy workers), and spot pricing (in-case runs are interrupted too frequently during busy times). If you see a stack in a locked state with the run as `worker failed`, often the instance was interrupted and the ASG events can be investigated for next steps. :::caution Spacelift Worker Pools can quickly become expensive. Spacelift bills per Worker total, and each instance in the Auto Scaling Group can have a number of Spacelift Workers. By default, we set the max instance count in the Auto Scaling Group to 2 and set the number of Spacelift Workers per instance to 1. This means that the total number of Spacelift Workers can scale to `2 x 1 = 2`. Once you become more familiar with Spacelift, scale the workers per instance or scale the number of instances with the `spacelift/worker-pool` catalog. ```yaml spacelift_agents_per_node: 1 # This is the number of Spacelift Workers for each instance min_size: 1 # This is the minimum number of instances in the Auto Scaling Group max_size: 2 # This is the maximum number of instances in the Auto Scaling Group ``` ::: ## Triggering Spacelift Runs Cloud Posse recommends two options to trigger Spacelift stacks. ### Triggering with Policy Attachments Historically, all stacks were triggered with three `GIT_PUSH` policies: 1. [GIT_PUSH Global Administrator](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.administrative.rego) triggers admin stacks 2. [GIT_PUSH Proposed Run](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.proposed-run.rego) triggers Proposed runs (typically Terraform Plan) for all non-admin stacks on Pull Requests 3. [GIT_PUSH Tracked Run](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/main/catalog/policies/git_push.tracked-run.rego) triggers Tracked runs (typically Terraform Apply) for all non-admin stacks on merges into `main` Attach these policies to stacks and Spacelift will trigger them on the respective git push. ### Triggering with GitHub Comments (Preferred) Atmos support for `atmos describe affected` made it possible to greatly improve Spacelift's triggering workflow. Now we can add a GitHub Action to collect all affected components for a given Pull Request and add a GitHub comment to the given PR with a formatted list of the affected stacks. Then Spacelift can watch for a GitHub comment event and then trigger stacks based on that comment. In order to set up GitHub Comment triggers, first add the following `GIT_PUSH Plan Affected` policy to the `spaces` component. For example, ```yaml title="stacks/catalog/spacelift/spaces.yaml" components: terraform: spaces: metadata: component: spacelift/spaces settings: spacelift: administrative: true space_name: root vars: spaces: root: policies: ... # This policy will automatically assign itself to stacks and is used to trigger stacks directly from the `cloudposse/github-action-atmos-affected-trigger-spacelift` GitHub action # This is only used if said GitHub action is set to trigger on "comments" "GIT_PUSH Plan Affected": type: GIT_PUSH labels: - autoattach:pr-comment-triggered body_url: https://raw.githubusercontent.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/%s/catalog/policies/push.git_comment.rego ``` This policy will automatically attach itself to _all_ components that have the `pr-comment-triggered` label, already defined in `stacks/orgs/NAMESPACE/_defaults.yaml` under `settings.spacelift.labels`. Next, create two new GitHub Action workflows: ```diff + .github/workflows/atmos-trigger-spacelift-feature-branch.yaml + .github/workflows/atmos-trigger-spacelift-main-branch.yaml ``` The feature branch workflow will create a comment event in Spacelift to run a Proposed run for a given stack. Whereas the main branch workflow will create a comment event in Spacelift to run a Deploy run for those same stacks. #### Feature Branch ```yaml name: "Plan Affected Spacelift Stacks" on: pull_request: types: - opened - synchronize - reopened branches: - main jobs: context: runs-on: ["self-hosted"] steps: - name: Atmos Affected Stacks Trigger Spacelift uses: cloudposse/github-action-atmos-affected-trigger-spacelift@v1 with: atmos-config-path: ./rootfs/usr/local/etc/atmos github-token: ${{ secrets.GITHUB_TOKEN }} ``` This will add a GitHub comment such as: ``` /spacelift preview plat-ue1-sandbox-foobar ``` #### Main Branch ```yaml name: "Deploy Affected Spacelift Stacks" on: pull_request: types: [closed] branches: - main jobs: run: if: github.event.pull_request.merged == true runs-on: ["self-hosted"] steps: - name: Atmos Affected Stacks Trigger Spacelift uses: cloudposse/github-action-atmos-affected-trigger-spacelift@v1 with: atmos-config-path: ./rootfs/usr/local/etc/atmos deploy: true github-token: ${{ secrets.GITHUB_TOKEN }} head-ref: ${{ github.sha }}~1 ``` This will add a GitHub comment such as: ``` /spacelift deploy plat-ue1-sandbox-foobar ``` ### Component Documentation - [`spacelift`](//components/library/aws/spacelift/) ## References - [Spacelift Documentation](https://docs.spacelift.io/) - [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) - [Proposed Spacelift admin stack architecture](/resources/adrs/proposed/proposed-spacelift-admin-stack-architecture) --- ## How to Destroy a Component using `atmos` or Spacelift import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem You want to destroy a component that was provisioned with terraform (i.e. its is no longer needed or for development purposes want to destroy and recreate it). There are a few different ways to destroy a component in a stack using `atmos` or Spacelift, depending on what you want to achieve. It’s important to understand the tradeoffs and why there isn’t just one way of doing it. In this guide, we explain the various options and the gotchas you might encounter along the way. ## Considerations - **Auto destruction.** If we simply remove the component from the stack configuration, it will orphan resources whether or not auto destruction is enabled. - **Guard rails.** If we enable auto destruction across the board, it too easy for accidental deletion of production resources. - **Audit trails.** If we use `atmos` on the command line, the space lift audit trail is bypassed (there’s still cloud trail logs) - **Component dependencies.** If we destroy a component that other components depend on via remote-state, then when the component is destroyed the remote-state will be lost and those dependent components will be unable to plan (or destroy). This means the order of operations is very important. ## Solution These are the main ways of destroying a component in a stack (each one described in more details in its own section below): :::tip - Run `atmos terraform destroy -s ` in `geodesic` on local computer - Run `terraform apply -destroy -auto-approve` from the Spacelift “Tasks” tab for the corresponding stack - Set `enabled: false` in the component YAML config and run `atmos terraform apply -s ` in `geodesic` on local computer - Set `enabled: false` in the component YAML config and run `terraform apply -auto-approve` from the Spacelift “Tasks” tab for the corresponding stack - A (potentially) 2-step process: - Set `settings.spacelift.stack_destructor_enabled: true` in the component YAML config if it was not set already, then open and merge a PR with that change and update the administrative stack. - In a subsequent PR, set `settings.spacelift.workspace_enabled: false` in the component YAML config to destroy the Spacelift stack and the component at the same time **IMPORTANT NOTES:** 1. In all of the above scenarios, the component configuration must remain present in the stack. This is a requirement of Terraform: all the required variables must have values supplied, even when destroying a component. Only after following the above steps to destroy the components can you then remove the component configuration. 2. You may choose to set `settings.spacelift.stack_destructor_enabled: true` in the stacks in advance, perhaps at a global level. If you do so, you still must merge and apply a PR setting `settings.spacelift.workspace_enabled: false` before removing the component from the stack, or else Spacelift will try to destroy the resources (even if none exist) and be unable to, which will block the deletion of the Spacelift stack. ::: ### **Option 1**: Run `atmos terraform destroy -s ` in `geodesic` on local computer In `geodesic`, run the following command: ``` atmos terraform destroy -s ``` #### Pros 1. One of the fastest ways of destroying a component in a stack #### Cons 1. No audit records at all of who did what, when and why 2. Someone can accidentally run the command and destroy the component without other people knowing about it, reviewing and approving it Not recommended. ### **Option 2**: Run `terraform destroy -auto-approve` from the Spacelift “Tasks” tab for the corresponding stack From the Spacelift “Tasks” tab for the corresponding stack, run the following command: ``` terraform destroy -auto-approve ``` #### Pros 1. One of the fastest ways of destroying a component in a stack 2. The record (who did it and the result of the command execution) will remain in Spacelift UI #### Cons 1. Disconnect from the Source Control System, no audit records in Git/GitHub - this will create a situation when the component is destroyed, but the repository has no corresponding record about it, which will force Terraform to recreate the component again on `terraform plan` 2. Someone (with Spacelift access) can accidentally run the command and destroy the component without other people knowing about it, reviewing and approving it Not recommended. ### **Option 3**: Set `enabled: false` in the component YAML config and run `atmos terraform apply -s ` in `geodesic` on local computer Open a pull request and set the variable `enabled: false` in the component YAML config: ``` components: terraform: my-component: settings: spacelift: workspace_enabled: true vars: enabled: false # set this to `false` ``` After the PR is reviewed, approved and merged, run the following command from `geodesic`: ``` atmos terraform apply -s ``` Note that the component needs to support the `enabled` variable on all modules and resources to be able to destroy it by setting `enabled` to `false`. #### Pros 1. There will be an audit record in Git of who did it, when, and the most importantly, why the component was destroyed 2. Can be easily reverted by reviewing the pull request that destroyed the component, and then setting `enabled` back to `true` and running `terraform apply` again #### Cons 1. It’s a multi-step process: 1) open a PR with the change; 2) wait for the PR review and approval; 3) merge the PR; 4) apply the change 2. The component needs to support the `enabled` pattern on all modules and resources ### **Option 4**: Set `enabled: false` in the component YAML config and run `terraform apply -auto-approve` from the Spacelift “Tasks” tab for the corresponding stack Open a pull request and set the variable `enabled: false` in the component YAML config: ``` components: terraform: my-component: settings: spacelift: workspace_enabled: true vars: enabled: false # set this to `false` ``` After the change has been pushed to the PR, Spacelift will trigger a proposed run on the stack, and you can review the result of `terraform plan` in the “PRs” tab for the corresponding stack in the Spacelift UI. Note that since the component has been disabled, the plan should show all the steps Terraform would take to destroy all the resources for the component. After `terraform plan` is reviewed in the Spacelift UI, and the PR is reviewed, approved and merged into the default branch, Spacelift will trigger a tracked run on the stack which will run `terraform apply` (you can view it in the "Runs" tab), and will destroy the component. #### Pros 1. There will be an audit record in Git of who did it, when, and the most importantly, why the component was destroyed 2. There will be an audit record in Spacelift of who triggered the stack and when, and from what Git commit 3. Can be easily reverted by setting `enabled` back to `true` and running the steps described above again #### Cons 1. It’s a multi-step process: 1) open a PR with the change; 2) review the result of `terraform plan` in Spacelift “PRs” tab; 3) wait for the PR review and approval; 4) merge the PR; 5) review the result of `terraform apply` in the “Runs” tab of the Spacelift UI 2. The component needs to support the `enabled` pattern on all modules and resources ### **Option 5**: Set `settings.spacelift.workspace_enabled: false` in the component YAML config to destroy the Spacelift stack and the component at the same time If you want to destroy a component and the corresponding Spacelift stack at the same time, you can set `settings.spacelift.workspace_enabled: false` in the component YAML config. Note that for this to work, Spacelift stack destructor ([https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/reference/stack_destructor](https://registry.terraform.io/providers/spacelift-io/spacelift/latest/docs/reference/stack_destructor)) must have been already enabled on the component by setting `settings.spacelift.stack_destructor_enabled` to `true` in YAML config (or alternatively, the destructor can be enabled on all stacks by setting the variable `stack_destructor_enabled` to `true` in the `spacelift` component). If the stack destructor was not enabled before, you can do so by setting `stack_destructor_enabled` to `true` in a separate PR. Open a pull request and set `settings.spacelift.workspace_enabled: false` in the component YAML config: ``` components: terraform: my-component: settings: spacelift: stack_destructor_enabled: true # this must have been already set to `true` workspace_enabled: false # set this to `false` ``` After the change has been pushed to the PR, Spacelift will trigger a proposed run on the admin `infrastructure` stack, and you can review the result of `terraform plan` in the “PRs” tab for the corresponding stack in the Spacelift UI. Note that it will show a `tarrform plan` to destroy the component’s Spacelift stack, not the component itself. After `terraform plan` is reviewed in the Spacelift UI, and the PR is reviewed, approved and merged into the default branch, Spacelift will trigger a tracked run on the `infrastructure` stack which will run `terraform apply` (you can view it in the "Runs" tab), and will destroy the Spacelift stack. After that, Spacelift will trigger the component’s stack and destroy the component itself and all the compoenet’s resources. #### Pros 1. There will be an audit record in Git of who did it, when, and the most importantly, why the component was destroyed 2. There will be an audit record in Spacelift of who triggered the stack and when, and from what Git commit 3. Can be easily reverted by setting `settings.spacelift.workspace_enabled` back to `true` and running the steps described above again #### Cons 1. It’s a multi-step process: 1) open a PR with the change; 2) review the result of `terraform plan` in Spacelift “PRs” tab for the `infrastructure` stack; 3) wait for the PR review and approval; 4) merge the PR; 5) review the result of `terraform apply` in the “Runs” tab of the Spacelift UI or the `infrastructure` stack; 6) review the component’s stack triggered run in the “Runs” tab of the component’s stack 2. Spacelift stack destructor must have been already enabled on the component ## Additional Notes Pay attention to the following: 1. If the component you want to destroy has dependencies (other components that depend on it), destroying the component will throw an error. For example, if you try to destroy a VPC and you have other resources deployed into it (e.g. EKS or Aurora Postgres clusters), destroying the VPC will fail. Likewise, if you have a Security Group that is already attached to the ENIs of some EC2 instances, destroying the Security Group will fail. 2. Components can be divided into two categories: 1) AWS resources (e.g. EKS or database clusters); 2) internal resources that are deployed inside AWS resources (e.g. `helm` or `kubernetes` releases deployed into the EKS clusters using Terraform `helm` and `kubernetes` providers, or additional users and databases deployed to Aurora Postgres clusters using the Terraform `postgres` provider). If you destroy the AWS resources (EKS or Aurora clusters), the internal resources will be destroyed as well, but they will remain in the Terraform state of the corresponding components. To get around this, when you destroy AWS resources like EKS or Aurora clusters, destroy the internal components (helm/kubernetes releases, additional databases and users) first. If you don’t destroy the internal components first, you’ll have to manually fix the Terraform state. 3. If you have AWS resources deployed into VPC private subnets (for example, Aurora Postgres clusters), and you use a VPN to access them (for example, to access databases in the cluster), make sure you are on the VPN before attempting to destroy the resources. Otherwise, Terraform will not be able to access the resources in the private subnets, and any attempt to destroy will fail (in this case, `terraform plan` will fail as well). --- ## How to Develop with Spacelift and Atmos import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem - Spacelift is great for day-2 operations, such as making changes to existing components and changing settings, but it’s not so great for the execution of workflow needed to bring up an entire stack or modify lots of components within a single Pull Request. - Atmos is very convenient for development but doesn’t provide the team collaboration benefits, policy controls, audit trails & accounting that we get when we use Spacelift - Iterating on development with Spacelift is a very slow feedback cycle (save work, commit, push, wait, repeat) compared to local development with `atmos` - Performing state migrations (e.g. `terraform mv`) and state imports using Spacelift Tasks is great for auditability, but cumbersome/slow as a developer who can just run `atmos` locally in a terminal - It’s complicated if you need to perform more than one operation affecting more than one component in a PR or commit that depends on other actions happening first - If operations are performed in `atmos` on the command line, those changes are unavailable to Spacelift; concurrent development in both Spacelift and `atmos` will lead to conflicts in terraform state and rolling back to one or the other. Spacelift is always watching the `main` branch and drift detection will automatically propose a plan to remediate the drift. Changes made by atmos (e.g. from other branches) will appear as drift. - Dependencies between components are enforced by a _best-effort_ using labels, but not guaranteed and don't work in all situations and are not enforced in `atmos` (only Spacelift). See [How to Manage Explicit Component Dependencies with Spacelift](/resources/deprecated/spacelift/tutorials/how-to-manage-explicit-component-dependencies-with-spacelift) - Destroying a component is not always easy (for good and bad reasons). See [How to Destroy a Component using `atmos` or Spacelift](/resources/deprecated/spacelift/tutorials/how-to-destroy-a-component-using-atmos-or-spacelift) for details. - Spacelift doesn’t have a visual way of releasing/promoting a change across environments; the user must confirm the plan for each stack in the order they must be applied ## Prerequisites Review why we [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) and [Atmos](/resources/legacy/fundamentals/atmos) documentation. ## Workflow :::caution **Practice “Inbox Zero” with Spacelift** Maintaining good Spacelift status for our stacks is **very important.** If we see stacks that are “Failed” or “Unconfirmed”, it is everyone's responsibility to ensure they are remediated in a tie mely manner. The longer plans go unconfirmed or remain in a failed state, the less confident we are about the state of the infrastructure and able to react to events as they happen. ::: Here’s the general workflow when working with infrastructure: 1. Apply your changes through Spacelift where/whenever possible. (open PR → spacelift proposed run looks OK → approve & merge → spacelift plan, confirm, apply, party on) 2. When/where not possible to run via Spacelift: after applying manually, backfill changes into source control and trigger a spacelift run to ensure stack is successfully finished ASAP (spacelift can also import out-of-band changes via Tasks!). Even better, if you put your manual changes as an "I'm going to do _this_" in your PR along with any TF to backfill, and use that as a proposed change record. 3. Use `dev` as the environment to test manual changes / or develop new modules, if not ready for spacelift before promoting to other envs (dev → staging → prod) The general workflow, however, doesn’t address all use-cases. Please review the various use-cases and follow the best practices for tackling each scenario. ## Use-cases Here are the most common scenarios developers will encounter and how to approach them. ### Use-case #1: Developing a New Component Document: How to develop a new component and add to a stack configuration ### Use-case #2: Modifying/Updating an Existing Component or Stack Document: how to make changes (e.g. with PR) and see what the impact is of those changes Document: how to debug with Spacelift when things go wrong ### Use-case #2: Destroying Components [How to Destroy a Component using `atmos` or Spacelift](/resources/deprecated/spacelift/tutorials/how-to-destroy-a-component-using-atmos-or-spacelift) How to use `spacelift_stack_destructor` with Spacelift. @Andriy Knysh ### Use-case #3: Creating/Destroying a Stack (e.g. Complete Environment) Document: How to generally approach the creation or destruction of multiple components with dependencies on the order of operation This is the use-case for workflows. Mostly it needs to be done on the command line using `atmos`. ## FAQ ## How to get access to Spacelift? Access is frequently given by including the GitHub user in the GitHub org which then gives access to the org Spacelift using URL: `https://.app.spacelift.io` Alternatively, for companies using SSO together with Spacelift, the Developer needs to be added to the appropriate group in the IdP. ## How to apply all the changes in my PR? ## How to set up Spacelift? If you haven't yet signed up for Spacelift, see [How to Sign Up for Spacelift](/resources/deprecated/spacelift). Once the Spacelift account is set up, follow the ## How to Destroy a Spacelift stack and the components? See [How to Destroy a Component using `atmos` or Spacelift](/resources/deprecated/spacelift/tutorials/how-to-destroy-a-component-using-atmos-or-spacelift) --- ## How to Enable Spacelift Drift Detection import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem The state of infrastructure may drift from what is defined in Infrastructure as Code via Terraform. This happens for multiple reasons: someone performs clickops in the AWS Web UI, Terraform introduces changes to resources, someone runs terraform or `atmos` locally without upstreaming their changes, an adversary exploits some mechanism to alter the state of your infrastructure, or simply you depend on some remote state not defined in the source control (e.g. [using the data source to pull the latest AMI](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/ami)). In any one of these situations, the state of our infrastructure has diverged from what we last knew, which is why we should perform regular reconciliation. ## Solution :::tip Use spacelift drift detection to continuously reconcile the state of your infrastructure with what’s in GitHub ::: Spacelift supports automatically detecting drift by re-running any stack on a cron schedule. When it detects changes (e.g. `terraform plan` indicates there are changes), then it can propose a run to remediate the change. The configuration can be controlled by updating the stack configuration for a component by toggling the `drift_detection_enabled` and `drift_detection_reconcile` settings. ```yaml components: terraform: your-component: settings: spacelift: workspace_enabled: true drift_detection_enabled: true drift_detection_reconcile: true ``` ### Default Settings The defaults for all components are set in `spacelift` component in `default.auto.tfvars`file. ```ini # This will enable the detection of drift. Use with `drift_detection_reconcile` to ensure automatic reconciliation. drift_detection_enabled = true # Enable automatic reconciliation (this will propose a run if there are changes) drift_detection_reconcile = true # Run every day at 04:00 am GMT drift_detection_schedule = [ "0 4 * * *" ] ``` ## Related - [Decide on Spacelift Administrative Stack Auto-deployment](/resources/deprecated/spacelift/design-decisions/decide-on-spacelift-administrative-stack-auto-deployment) --- ## How to Manage Explicit Component Dependencies with Spacelift import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem When Spacelift deploys a component into a particular environment it doesn’t natively understand any HCL or what that component needs. Spacelift does not compute any sort of DAG between components the way that Terraform does between resources in the graph of a single root module. If a component relates to other components via a remote state (E.g. our convention using the `remote-state.tf` file in each component's folder), this is not Spacelift’s concern to handle it. Nonetheless, we need to be able to explicitly define cross-component dependencies in order to do things with remote-state. ## Solution :::note Our spacelift module will automatically derive dependencies based on `imports` , which means that and stack configuration (E.g. in `catalog/*`) that is imported by any other stack configuration, is now an implicit (or derived) dependency of that stack. So that means that when that upstream file that is imported is modified, it will also trigger an plan of any stack that imports it. - For example, if you modify `globals.yaml`, since that file is imported by every single stack, it will trigger every single stack to plan. - For example, if you modify something like `catalog/rds.yaml`, it will only trigger stacks that import that config. ::: We have a very primitive dependency system based on a custom to `depends-on` label (`depends_on` spacelift setting in component configuration) which gets added to `labels` for the Spacelift stacks to express dependencies. For each stack, in YAML stack config, we allow specifying the stack dependencies on other components in the same environment and on other stacks. We automatically detect if the dependency is a stack or a component in the current stack, and throws an error if it's not a valid stack and not a valid component. Using Spacelift workflows which are simply Rego policies under the wood, we trigger the dependent stacks after the "parent" stacks finish running. The Rego policy is defined here [https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/0.42.0/catalog/policies/trigger.dependencies.rego](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/blob/0.42.0/catalog/policies/trigger.dependencies.rego) For details, see PR that implemented it [https://github.com/cloudposse/terraform-provider-utils/pull/43](https://github.com/cloudposse/terraform-provider-utils/pull/43)) :::caution Processing Dependencies Spacelift can only operate on dependency relationships within a single commit. There’s no way to express a dependency relationship for what should happen between multiple commits, e.g. multiple Pull Requests are merged around the same time and requiring components in one stack be applied before other stacks is not possible. ::: ### Confifgure Dependencies Between Components with Labels Use labels in Spacelift to define the dependencies between components There are many ways to express dependencies. Which to use depends on what you need to accomplish. You can see the dependencies easily in the Spacelift UI by looking at the labels. Not all labels are dependencies, only the ones that begin with `deps:`or `depends-on:`. (note, the folders are created similarly by prefixing labels with `folder:`) We support globbing with `*` and `**` so you can express things like `stacks/uw2/**` to match all files recursively in the `stacks/uw2` folder. A component can depend on any folder as part of the VCS. If a change is detected in that folder, then it will trigger a plan. Just prefix the label with `deps:` and add a `/*` at the end to match all files. ```yaml labels: - deps:component/aurora-postgres-2/* - deps:uw2/dev/* ``` A component can express explicit dependencies on other files. ```yaml labels: - deps:stacks/catalog/rds-defaults.yaml - deps:stacks/globals.yaml - deps:stacks/uw2-dev.yaml - deps:stacks/uw2-globals.yaml - deps:config/secrets/dev-internal-secrets.yml ``` A component can express explicit dependencies on other components in other stacks (e.g. the `vpc` component in the `uw2-dev` stack) ```yaml labels: - depends-on:uw2-dev-vpc - depends-on:uw2-dev-dns-delegated - depends-on:gbl-dns-dns-primary ``` A component can express explicit dependencies on the current stack (E.g. the `vpc`) without needing to express the canonical component name. ```yaml labels: - depends-on:vpc - depends-on:dns-delegated - depends-on:dns-primary ``` ### Step-by-Step Process 1. You use `depends_on` in YAML like this: ```yaml dms-iam: settings: spacelift: workspace_enabled: true policies_by_name_enabled: - "access.dms" dms-replication-instance: settings: spacelift: workspace_enabled: true depends_on: # Stack dependencies. This stack will be triggered after the parent stack finishes running - "dms-iam" ``` 2. When your `spacelift` component (which uses [https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation)) parses the YAML, it adds the label to the Spacelift stacks 3. Spacelift checks the rego policy `trigger.dependencies.rego` to determine if the label exists on the stack and triggers the dependent stack(s) ### Example ```yaml aurora-postgres-2: component: aurora-postgres settings: spacelift: workspace_enabled: true autodeploy: true labels: - "deps:config/secrets/dev-internal-secrets.yml" depends_on: - vpc - dns-delegated - gbl-dns-dns-primary ``` Will associate the following labels in Spacelift with the Stack: ```yaml labels: - deps:stacks/catalog/rds-defaults.yaml - deps:stacks/globals.yaml - deps:stacks/uw2-dev.yaml - deps:stacks/uw2-globals.yaml - deps:config/secrets/dev-internal-secrets.yml - depends-on:uw2-dev-vpc - depends-on:uw2-dev-dns-delegated - depends-on:gbl-dns-dns-primary - folder:component/aurora-postgres-2 - folder:uw2/dev ``` ## Troubleshooting If any of the entries in `depends_on` is not a valid stack and not a valid component in the current stack, the following errors will be thrown: ```console Error: Component 'aurora-postgres-2' in stack 'uw2-dev' specifies 'depends_on' dependency 'vpc3', but 'vpc3' is not a stack and not a terraform component in 'uw2-dev' stack Error: Component 'aurora-postgres-2' in stack 'uw2-dev' specifies 'depends_on' dependency 'gbl-ops-dns-primary', but 'gbl-ops-dns-primary' is not a stack and not a terraform component in 'uw2-dev' stack ``` --- ## How to Scale Spacelift Runners import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem There are hundreds or thousands of proposed runs pending in Spacelift. Autoscaling will eventually scale up to handle capacity but want faster turnaround time. ## Solution :::note If there are too many Spacelift runs triggered by PRs that do not require Spacelift to run at the moment. The user can add a label called `spacelift-no-trigger` on the PR to prevent Spacelift stacks from running on each commit. This label should be removed before the last commit (before approval and merge) so the Spacelift stacks can be validated. ::: :::note To reduce Spacelift runs triggered by PRs, ensure that the `spacelift` component is using the [latest upstream module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation/releases) or `0.46.0` at the very least. This release has an updated policy that cancels previous commit runs if a new commit is pushed. ::: ### Scaling EC2 Runners via Stack Configurations :::caution While we always recommend making all changes to IaC via Pull Request, if the desire is to increase the size of the Auto Scale Group and there are hundreds or thousands of pending runs, then the Pull Request will suffer the same fate as the other runs and be blocked from execution until the other runs complete. ::: The parameters that affect scale up behavior are: | **Setting** | **Description** | | ---------------------------------------- | --------------------------------------------------------------------------------------------------------- | | `min_size` | The minimum of the instances | | `max_size` | The maximum number of instances | | `desired_capacity` | The number of instances desired online all the time. | | `health_check_grace_period` | | | `spacelift_agents_per_node` | The number of Spacelift agents running per EC2 instance. Increase for greater density of agents per node. | | `cpu_utilization_high_threshold_percent` | Percent CPU utilization (integer between 0 and 100) required to trigger a scale up event | | `wait_for_capacity_timeout` | Before scaling further, wait for this many seconds for capacity requirements to be met. | The parameters that affect scale-down behavior are: | **Setting** | **Description** | | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `default_cooldown` | | | `scale_down_cooldown_seconds` | | | `cpu_utilization_low_threshold_percent` | Percent CPU utilization (integer between 0 and 100) required to trigger a scale down event. The higher this number, the more likely the cluster will be scaled down because nodes are typically not very CPU intensive _on average._ | | `max_size` | | Here’s a sample `spacelift-worker-pool` configuration. ```yaml components: terraform: spacelift-worker-pool: settings: spacelift: workspace_enabled: true vars: enabled: true ecr_stage_name: artifacts ecr_tenant_name: mgmt ecr_environment_name: uw2 ecr_repo_name: infrastructure ecr_region: us-west-2 ecr_account_id: "1234567890" spacelift_api_endpoint: https://acme.app.spacelift.io spacelift_runner_image: 1234567890.dkr.ecr.us-west-2.amazonaws.com/infrastructure:latest instance_type: "c5a.xlarge" # cost-effective x86 instance with 4vCPU, 8GB (see: https://aws.amazon.com/ec2/instance-types/) wait_for_capacity_timeout: "10m" spacelift_agents_per_node: 2 min_size: 3 max_size: 50 desired_capacity: null default_cooldown: 300 scale_down_cooldown_seconds: 2700 # Set a low scaling threshold to ensure new workers are launched as soon as the current one(s) are busy cpu_utilization_high_threshold_percent: 10 cpu_utilization_low_threshold_percent: 5 health_check_type: EC2 health_check_grace_period: 300 termination_policies: - OldestLaunchConfiguration ebs_optimized: true block_device_mappings: - device_name: "/dev/xvda" no_device: null virtual_name: null ebs: delete_on_termination: null encrypted: false iops: null kms_key_id: null snapshot_id: null volume_size: 100 volume_type: "gp2" iam_attributes: - admin instance_refresh: # https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-instance-refresh.html # https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/autoscaling_group#instance_refresh strategy: Rolling preferences: # The number of seconds until a newly launched instance is configured and ready to use # Default behavior is to use the Auto Scaling Group's health check grace period instance_warmup: null # The amount of capacity in the Auto Scaling group that must remain healthy during an instance refresh to allow the operation to continue, # as a percentage of the desired capacity of the Auto Scaling group min_healthy_percentage: 50 triggers: null ``` ### Scaling EC2 Runners Immediately (via AWS Console) #### Scaling Up - Login to the AWS console for the automation account. - Find the autoscale group under the EC2 menu and increase the autoscaling group size. #### Scaling Down :::caution **IMPORTANT** Before scaling spacelift workers down, check the “**Pending runs**” and “**busy workers**” are both **0.** In other words do not scale down when spacelift is busy. If the worker is terminated in the middle of a job, it will leave a stale terraform state lock and maybe broken state. Stack lock can be fixed using “terraform force-unlock“ as suggested in a command line prompt. ::: - Login to the AWS console for the automation account. - Find the autoscale group under the EC2 menu decrease the autoscaling group size ### Scaling EC2 Runners Immediately (via `aws` CLI) Get the name of the ASG ```bash AWS_PROFILE=$NAMESPACE-gbl-auto-admin # Get ASG name ASG_NAME=$(aws autoscaling describe-auto-scaling-groups \ --query 'AutoScalingGroups | [?contains(AutoScalingGroupName, `spacelift`)] | [0].AutoScalingGroupName' \ --output text) # Set the max of the ASG aws autoscaling update-auto-scaling-group --auto-scaling-group-name $ASG_NAME --max-size 10 # Set the desired capacity of the ASG aws autoscaling set-desired-capacity --auto-scaling-group-name $ASG_NAME --desired-capacity 10 ``` ### k8s (Not Yet Supported) Cloud Posse has not yet implemented Kubernetes Spacelift runners. --- ## How to Sign Up for Spacelift import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; import Steps from '@site/src/components/Steps'; ## Problem You’ve been asked to signup for the Spacelift service and you’re not sure which plan to choose or where to go next. You have questions on pricing and want to better understand your options. You already use GitHub Actions, or some other CI/CD platform and want to understand why you need yet another platform. ## Solution :::tip Signup for any trial Spacelift account and it will automatically be enabled as an **Enterprise Trial**. We’ll request via our partern-channel that Spacelift bump it up to an extended **Enterprise Trial**, while you work with them to reach a fair price. ::: #### GitHub Organizations [https://spacelift.io/free-trial](https://spacelift.io/free-trial) After signing up using your GitHub account, you should be prompted to install the Spacelift GitHub App. Or install it by going here: [https://github.com/apps/spacelift-io/installations/new](https://github.com/apps/spacelift-io/installations/new) [https://github.com/apps/spacelift-io](https://github.com/apps/spacelift-io) #### GitLab Organizations For organizations running GitLab, register with your work email address instead. [https://spacelift.io/free-trial](https://spacelift.io/free-trial) Once you’ve signed up for the Spacelift trial, we’ll configure GitLab as source control integration. To learn more about setting up the GitLab integration, see the [https://docs.spacelift.io/integrations/source-control/gitlab](https://docs.spacelift.io/integrations/source-control/gitlab) integration documentation. ## Signup Process :::info Sign up for Spacelift after the cold start components and `vpc` component have been provisioned in the automation account ::: 1. Sign up [https://spacelift.io/free-trial](https://spacelift.io/free-trial) 2. Install [Spacelift GitHub app](https://github.com/apps/spacelift-io/installations/new) into the customer GitHub organization 1. Once the app is installed into the customer GitHub organization, the GitHub organization admin will receive an email from Spacelift 3. Click on the link in the email from Spacelift 1. After the Spacelift app has been installed and enabled, only the integration creator (the admin of the customer GitHub org) gets administrative permissions by default, so all other users and administrators must be granted their access using a login policy. 4. Add the following login policy. 1. Click `Policies` at the top 2. Click `Add Policies` 3. Set a name like `Login Policy` 4. Change the policy type to `Login policy` 5. Copy and paste the policy below 6. Change the `user_collaborators` to the customer github org 7. Add any additional `admin_collaborators` from the Cloud Posse team 8. Add the names of the Cloud Posse `CLOUDPOSSE-CASE-SENSITIVE-TEAM-NAME` and non cloudposse `YOUR-CASE-SENSITIVE-TEAM-NAME` team names. 9. Finally, click `Create Policy` ```rego package spacelift # See https://docs.spacelift.io/concepts/policy/login-policy for implementation details. # Note: Login policies don't affect GitHub organization or SSO admins. # Note 2: Enabling SSO requires that all users have an IdP (G Suite) account, so we'll just use # GitHub authentication in the meantime while working with external collaborators. # Map session input data to human friendly variables to use in policy evaluation username := input.session.login member_of := input.session.teams github_org := input.session.member # Define GitHub usernames of non-github_org org external collaborators with admin vs. user access admin_collaborators := { "osterman", "aknysh", "Nuru", "nitrocode" } # case sensitive names of collaborators user_collaborators := { "Customer Github Org" } # case sensitive name of the github org # Grant admin access to github_org org members in the non cloud posse case-sensitive team # Do not use the slug here, use the name shown in github.com/org/teams admin { github_org member_of[_] == "YOUR-CASE-SENSITIVE-TEAM-NAME" } # Grant admin access to github_org org members in the Cloud Posse group # Do not use the slug here, use the name shown in github.com/org/teams admin { github_org member_of[_] == "CLOUDPOSSE-CASE-SENSITIVE-TEAM-NAME" } # Grant admin access to non-github_org org accounts in the admin_collaborators set admin { # not github_org admin_collaborators[username] } # Grant user access to accounts in the user_collaborators set allow { # not github_org user_collaborators[username] } # Deny access to any non-github_org org accounts who aren't defined in external collaborators sets deny { not github_org not user_collaborators[username] not admin_collaborators[username] } ``` 5. Verify Cloud Posse users have administrative access 6. Create an administrative api key 1. Click the three horizontal bars on the top right hand corner 2. Click `Settings` 3. Click `API KEYS` 4. Click `Add new API Key` 5. Give it a name like `Spacelift worker pool API admin key` 6. Enable `Admin` 7. Click `Add Key` 8. A file will download. Save the contents to our shared 1Password vault. This will give us the `key_secret` and the `token`. 9. Also share the 16 character string next to the name of the key which gives us the `key_id`. :::important After signing up and following the above steps, let our team know what your Spacelift account slug is and we’ll get it upgraded to an Enterprise trial via our partner channels with Spacelift. ::: --- ## How to Use Multiple Infrastructure Repositories with Spacelift import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem Using a single infrastructure repository makes a lot of sense for foundational infrastructure, but sometimes it would be nicer to colocate infrastructure code alongside other products. For example, a data science team might work out of a separate repository where they store all their airflow DAGs or snowflake queries. In that case, it might be nice to manage the terraform code outside of the infrastructure repository. ## Solution :::tip Our stack configurations can reference terraform components from any repository in the GitHub organization ::: The Cloud Posse convention is to use the `infrastructure` repository as your centralized registry of stack configurations so you know where everything that’s deployed across your organization resides. It helps avoid code sprawl where you have hundreds of repositories and no record of how it’s managed. But just like any good registry, the stack configuration can point to other GitHub repositories that contain a terraform component. The great thing about this is that there’s centralized governance over what is managed by Spacelift via Rego policies, and decentralized location of the infrastructure code granting teams the autonomy to build and manage their own services. This strategy is useful in combination with deploying infrastructure code for entirely different systems like Cloudflare or Snowflake via terraform. ### Example Component In the following snippet, we’re deploying a component named `snowflake` defined in the `snowflake` repository under the `terraform` folder from the `main` branch. ```yaml components: terraform: snowflake: component: spacelift-stack settings: spacelift: workspace_enabled: true autodeploy: true repository: snowflake branch: main component_root: terraform vars: enabled: true name: snowflake ``` ## Related - [How to terraform non-AWS infrastructure?](/learn/maintenance/tutorials/how-to-terraform-non-aws-infrastructure) --- ## How to use Spacectl import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; ## Problem You have many stacks that you need to manipulate in some way that the Spacelift Web UI doesn’t support or makes very cumbersome to do one by one. For example, maybe the stacks are pointing to the wrong `HEAD` commit. You need to get some data out of Spacelift for reporting purposes. ## Solution :::tip Use `spaceftl` to automate certain mundane tasks in Spacelift and/or get an overview of Spacelift. ::: Long explanation for how to solve the problem. See docs [https://github.com/spaceone-dev/spacectl](https://github.com/spaceone-dev/spacectl) ### Install ```shell ⨠ apt install -y spacectl -qq ``` ### Setup a profile ```shell ⨠ spacectl profile login acme Enter Spacelift endpoint (eg. https://unicorn.app.spacelift.io/): https://acme.app.spacelift.io Select credentials type: 1 for API key, 2 for GitHub access token: 1 Enter API key ID: 01FKN... Enter API key secret: ``` ### Listing stacks ```shell spacectl stack list ``` Grab all the stack ids (use the JSON output to avoid bad chars) ```shell spacectl stack list --output json | jq -r '.[].id' > stacks.txt ``` ### Setting stacks to a specific commit If the latest commit for each stack is desired, run something like this. NOTE: remove the `echo` to remove the dry-run functionality ```shell cat stacks.txt | while read stack; do echo $stack && echo spacectl stack set-current-commit --sha snip --id $stack; done ``` --- ## Spacelift Tutorials import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a collection of tutorials for Spacelift. --- ## 12-Factor The 12 Factor Pattern is a software methodology for building cloud-friendly (or cloud-native), scalable, maintainable applications that deploy easily on a Platform-as-a-Service (aka PaaS). It is a language agnostic, “Best Practice” for writing Cloud Native applications that can be easily and consistently deployed using Continuous Integration and Continuous Delivery (“CI/CD”). The pattern can be summed up as: - (a) treat all apps as disposable services that receive their configuration via environment variables; - (b) rely on backing services to provide durability; - (c) script all changes; and - (d) treat all environments (dev, prod, qa, etc) as identical. --- ## Amazon Certificate Manager (ACM) [Amazon Certificate Manager](https://aws.amazon.com/certificate-manager/) is a service that lets you easily provision, manage, and deploy TLS certificates for use with AWS services such as ELBs and CloudFront." --- ## Amazon Machine Image (AMI) An [Amazon Machine Image](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) provides the information required to launch an EC2 instance, which is a virtual server in the Amazon public cloud." --- ## Anti-Pattern An anti-pattern is a common response to a recurring problem that is usually ineffective and risks being highly counterproductive. See [best-practices](/resources/glossary/best-practices). --- ## Availability Zone (AZ) An [Availability Zone](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-availability-zones) is an isolated data center within a geographic region that is interconnected with other AZs through low-latency links. --- ## aws-vault *(Deprecated)* `aws-vault` is a utility for securely managing secrets with AWS Systems Manager (SSM) Parameter Store and KMS. It used to be Cloud Posse's recommended tool for this purpose, but for various reasons we now recommend using [Leapp](htpps://leapp.cloud) instead. --- ## AWS(Glossary) Amazon Web Services is a public cloud offering from Amazon. It's also a command line tool (`aws`) use to control services running on the platform. --- ## Bastion(Glossary) A bastion host is the only host permitted to be directly addressed via SSH from the internet. --- ## Best Practices(Glossary) A well-understood and often published technique or methodology that, through experience and research, has proven to reliably lead to a desired result. --- ## BeyondCorp BeyondCorp is an enterprise security model pioneered by Google that enables every employee to work from untrusted networks without the use of a VPN. One critical component is an Identity Aware Proxy. --- ## Build Harness The primary benefit of using a [`build-harness`](/learn/toolchain/#build-harness) is it allows for the consolidation of business logic related to building software. This allows to keep things DRY. Using a centralized repo that can be versioned and shared across multiple projects reduces long-term technical debt associated with building and releasing software by reducing maintenance effort. We provide one that we use in nearly all of our projects. It's available here: https://github.com/cloudposse/build-harness --- ## Business Logic [Business Logic](https://en.wikipedia.org/wiki/Business_logic) is the code that describes real-world business rules that determine how things should work. --- ## Chamber [Chamber](https://github.com/segmentio/chamber) is a tool by Segment IO for managing secrets with AWS SSM+KMS and exposing them as environment variables. --- ## Change Management A formal process followed in order to make changes in a document, systematic process. --- ## Chart Registry A Chart Registry is responsible for storing and serving Helm chart packages (`.tar.gz`) to the helm tiller running in the kubernetes cluster. --- ## Continuous Integration / Continuous Delivery (CI/CD) CI/CD is the practice of combining “Continuous Integration” with “Continuous Delivery” (aka “Continuous Deployment”) --- ## CLI `cli` is common abbreviation for `client` and usually refers to some kind of command line tool. --- ## ClickOps ClickOps is the error-prone and time-consuming process of having people click-through various menu options in cloud providers’ websites, to select and configure the correct automated computing infrastructure. This is typically coupled with WikiOps processes and is considered bad practice in the modern cloud era. --- ## CloudFront Amazon CloudFront is a content delivery network (CDN) used to improve latency for end users by hosting cacheable content on distributed global edge locations. --- ## Cloud Posse, LLC Cloud Posse is a DevOps Accelerator. Let us know how we can help. Reach us at [hello@cloudposse.com](mailto:hello@cloudposse.com) --- ## CloudTrail(Glossary) --- ## CloudWatch Logs Amazon CloudWatch Logs is a central store for managing logs from AWS cloud resources and applications. --- ## CloudWatch(Glossary) Amazon CloudWatch is a monitoring service for AWS cloud resources and applications. --- ## Container Management Platform (CMP) --- ## CodeBuild(Glossary) [Amazon CodeBuild](https://aws.amazon.com/codebuild/) is a fully managed build service that compiles source code, runs tests, and produces software packages that are ready to deploy. With CodeBuild, you don’t need to provision, manage, and scale your own build servers. CodeBuild scales continuously and processes multiple builds concurrently, so your builds are not left waiting in a queue. Because CodeBuild runs in AWS, it can leverage IAM Instance Profiles to assume roles to securely provision resources without exposing AWS access credentials. --- ## CodeDeploy Amazon CodeDeploy is a service that automates software deployments to AWS cloud resources. --- ## CodePipeline [Amazon CodePipeline](https://aws.amazon.com/codepipeline/) is a continuous integration and continuous delivery service for fast and reliable application and infrastructure updates. --- ## Code Review (CR) A process of reviewing Pull Requests to enable collaboration between team members, improves code quality/stability through creation of alternative solutions to problems, increase transparency through proof of work, increases team awareness by involvement, and improves business continuity through knowledge transfer. --- ## Codefresh Codefresh is CI/CD as a service, built for containers with native support for Docker, Kubernetes and Helm. --- ## Continuous Delivery (CD) A process in which immutable code artifacts (E.g. packages, rpms, images, etc) are shipped to permanent storage and optionally deployed to an environment such as staging or production. This process is usually associated with “Continuous Deployment” or “Automated Deployment” --- ## Continuous Integration (CI) A process in which code is automatically checked out from version control (e.g. git) and submitted to a battery of automated tests to ensure that the recent changes will not have adverse effects on the code base and product stability. --- ## Create, Read, Update, Delete (CRUD) CRUD is a frequently used acronym which means Create, Read, Update and Destroy. It refers to the standard life-cycle events for data manipulation. --- ## The curl-bash pattern This is a common approach to installing various tools via a terminal prompt. You see this commonly done like so: `curl example.com/install.sh | bash` The curl-bash pattern is an approach to installing software on your local machine via your terminal. It utilizes `curl` to download a script which is then piped (i.e. `|` ) into `bash`. This enables the script to execute code on your machine which then does whatever setup or installation steps it needs to do to install the target software. Example: ``` curl -sSL https://get.rvm.io | bash ``` --- ## Declarative Declaration ## References - https://en.wikipedia.org/wiki/Infrastructure_as_Code#Types_of_approaches --- ## Shared Memory Filesystem (/dev/shm) `/dev/shm` is an ephemeral in-memory filesystem useful for caching temporary files that should not be persisted. --- ## The docker-bash pattern This is a SweetOps pattern used to install tooling via a terminal prompt. The primary usage is in Geodesic, which looks like: `docker run --rm cloudposse/geodesic:latest-debian init | bash -s latest-debian` The docker-bash pattern is an approach to installing software on your local machine via your terminal and docker. It utilizes `docker run` to output a script which is then piped (i.e. `|` ) into `bash`. This enables the script to execute code on your machine which then does whatever setup or installation steps it needs to do to install the target software. Geodesic utilizes this pattern via the `init` script which is expected to be piped into bash: ``` docker run --rm cloudposse/geodesic:latest-debian init | bash -s latest-debian ``` --- ## Docker Compose Docker Compose is a tool for defining and running multi-container Docker applications. Typically used for local development. --- ## Docker Image A [docker image](https://docs.docker.com/engine/reference/commandline/images/) is a self-contained, layered archive containing an application and all of its OS dependencies and is the artifact of running a `docker build`. The image is what gets stored in a [docker registry](https://docs.docker.com/registry). --- ## Docker Registry A Docker registry is a place to store and distribute Docker images. It works together with the `docker push` and `docker pull` commands for standard CRUD operations. :::info Examples of Docker registry: * [Docker Hub](https://hub.docker.com/) * [Codefresh Registry](https://codefresh.io/docs/docs/docker-registries/codefresh-registry/) * [AWS Elastic Container Registry](https://aws.amazon.com/ecr/) * [Quay Registry](https://quay.io/) ::: --- ## Dockerfile A [Dockerfile](https://docs.docker.com/engine/reference/builder/) is a lightweight DSL that contains all the commands a user could call on the command line to assemble an image in order to run an application. --- ## Don't Repeat Yourself (DRY) [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself) is an acronym for \"Don't Repeat Yourself\", which refers to the aim of reducing repetition of software code through copy-and-paste. --- ## End-to-end (e2e) The term *end-to-end* encompasses an entire workflow from start to finish. --- ## ElasticBeanstalk (EB) [Amazon Elastic Beanstalk](https://aws.amazon.com/elasticbeanstalk/) is an orchestration service offered from AWS for deploying infrastructure which orchestrates various services, including EC2, S3, ELB, SNS, CloudWatch, auto-scaling, auto-healing, and Elastic Load Balancers. --- ## Elastic Container Registry (ECR) Amazon ECR is a fully managed Docker container registry to store, manage, and deploy Docker container images on AWS. --- ## ECS Agent The ECS Agent is a component of Amazon Elastic Container Service (ECS) and is responsible for managing containers on behalf of Amazon ECS. --- ## ECS Service An ECS Service is a specified number (the "desired count") of instances of an ECS Task simultaneously running in an Amazon ECS cluster. --- ## ECS Task An ECS Task is a JSON document that describes one or more containers, that form an application. It can be thought of as a blueprint for an application. --- ## ECS(Glossary) --- ## Environment Variable An Environment Variable is an an interface provided by nearly every OS to pass configuration information to a process. --- ## Executable Documentation Executable Documentation is documentation that is so actionable you can just execute it directly or run it manually if you so choose. We treat `Makefiles` as executable documentation. That is, for every target we add a meaningful annotations (e.g. `## Build docker container`). The end-user can then choose to run the target (e.g. `make build`) or run the command described in the target (e.g. `docker build -t example/image .`). --- ## FUSE [FUSE](https://en.wikipedia.org/wiki/Filesystem_in_Userspace) stands for Filesystem in Userspace and is an interface that allows developers to implement custom filesystems without requiring complex kernel modules. --- ## Geodesic Module A geodesic module is a docker image that extends the geodesic base image and implements functions specific to that stage or account. Usually we create geodesic modules that correspond to each AWS organization. For example, the standard geodesic modules are: - `root.cloudposse.org` - a module which is responsible for administering the root AWS account and provisioning all subaccounts (organizations). - `prod.cloudposse.org` - a module which is responsible for provisioning all production infrastructure including production kops clusters and backing services (E.g. rds) - `staging.cloudposse.org` - a module which is responsible for provisioning all staging resources - `dev.cloudposse.org` - a module which is responsible for provisioning a sandbox environment for developers - `testing.cloudposse.org` - a module which is responsible for provisioning the organization used for testing of infrastructure modules (E.g. testing terraform modules as part of CI/CD) --- ## Geodesic Shell A geodesic shell is an invocation of a geodesic module. Said differently, it's when you run a geodesic docker image and enter into the bash shell. --- ## Geodesic [Geodesic](https://github.com/cloudposse/geodesic/) is an interactive command-line shell which bundles all essential open source cloud orchestration tools needed administer clusters from the command line. The only dependency is that docker is installed. The tools provided include `kops`, `terraform`, `eb` cli, `aws` cli, etc --- ## Git Workflow A form of Change Control that uses Git as the system of record. In the Git Workflow, the `master` branch is often treated as the pristine copy of the code base and considered always safe to deploy to production. Every time a change is needed, a developer will open up a new branch against `master` and push their changes up to the `origin`. When the developer is ready to merge their changes, they open up a Pull Request and request one of their peers to perform a Code Review. Once the Pull Request is approved, it may be merged into master. --- ## Glossary of Terms import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This is a glossary of terms used in our documentation. --- ## Goofys Goofys is a utility that implements S3-backed filesystems using FUSE. [`goofys`](/learn/toolchain/#geodesic) is a a high-performance, POSIX-ish Amazon S3 file system written in Go. We use `goofys` in [Geodesic](/resources/legacy/tools#geodesic) --- ## HashiCorp Language (HCL) HCL is [HashiCorp's configuration language](https://github.com/hashicorp/hcl) used in multiple products, but most notably in `terraform`. The HashiCorp Configuration language was designed to be both human readable and machine friendly. It used by most HashiCorp tools such as `terraform`. The nice thing about HCL is also fully JSON compatible, which means that JSON can be used anywhere HCL is expected. By supporting JSON backwards compatibility, HCL remains interoperable with other systems. Here's an example of HCL: ```hcl variable "ami" { description = "the AMI to use" } ``` --- ## Helm Chart A *Helm Chart* is a package that defines all the kubernetes resources necessary for deploying an application to kubernetes. --- ## Helm Tiller The helm tiller is the server-side component (API) for helm that manages all CRUD operations. --- ## Helm Helm is one of the predominant package managers for kubernetes which is used for installing applications on the cluster. --- ## Identity Access Management (IAM) Amazon's IAM is a service that helps you securely control access to AWS resources. --- ## Identity Aware Proxy An Identity-Aware Proxy enables an organization to control access to cloud applications (e.g. SaaS). An Identity-Aware Proxy enables an organization to control access to cloud applications (e.g. SaaS). Typically, an IAP works together with an organization’s Single Sign-on (SSO) provide for verifying a user’s identity and determining if that user should be allowed to access the application. The IAP is part of the BeyondCorp enterprise security model pioneered by Google. It enables every employee to work from untrusted networks without the use of a VPN. Another way of thinking about IAP is it’s a point-to-point VPN for every single service. Unlike a traditional VPN, which typically grants a wide range of access to internal networks, an IAP only grants access to one specific resource. IAP Providers and Implementations: - https://www.cloudflare.com/teams-access/ - https://cloud.google.com/iap/ - https://beyondcorp.com/ - https://github.com/bitly/oauth2_proxy --- ## Imperative Declaration The imperative approach focuses on how precisely the infrastructure should be defined. ## References - https://en.wikipedia.org/wiki/Infrastructure_as_Code#Types_of_approaches --- ## Infrastructure as Code (IaC) [Infrastructure as Code](https://en.wikipedia.org/wiki/Infrastructure_as_Code) (IaC) is the process of managing/provisioning Infrastructure as a Service (IaaS) using machine-readable definition files (usually DSLs), rather than rely on humans doing physical/manual hardware configuration. --- ## Infrastructure Infrastructure is everything that supports running your software. --- ## Ingress Controller An Ingress Controller is a native resource type in Kubernetes that functions like a Layer 7 Load Balancer (e.g. HTTP Load Balancer) to route requests to various backend services based on incoming hostname (e.g. `Host` header) and request path (e.g. `/foo`). The Kubernetes Ingress Controller is a native resource type in Kubernetes that functions like a Layer 7 Load Balancer (e.g. HTTP Load Balancer) to route requests to various backend services based on incoming hostname (e.g. `Host` header) and request path (e.g. `/foo`). The default Ingress Controller in Kubernetes is powered by Nginx, but this is an implementation detail that is entirely abstracted away from the end user. There are many vendors, in addition to Nginx providing alternative implementations. --- ## init-terraform The `init-terraform` script is a helper for configuring and then initializing terraform remote state in combination with the terraform-aws-tfstate-backend module. :::info - This [`init-terraform`](https://github.com/cloudposse/geodesic/blob/master/rootfs/usr/local/bin/init-terraform) script is provided as part of geodesic. - The [terraform-aws-tfstate-backend](https://github.com/cloudposse/terraform-aws-tfstate-backend) module provides an encrypted S3 bucket for persisting state and a DynamoDB table for state locking. ::: --- ## String Interpolation [String Interpolation](https://en.wikipedia.org/wiki/String_interpolation) is the process of evaluating a string containing one or more placeholders (e.g. `$FOOBAR` or `{{...}}`) and replacing the placeholders with their corresponding values. --- ## jq(Glossary) [`jq`](https://github.com/stedolan/jq) is a Go-based command line tool for JSON that supports standard CRUD operations. --- ## Kanban [Kanban](https://en.wikipedia.org/wiki/Kanban) is a popular framework used to implement agile software development that leverages cards and boards to visually communicate the status of a project. --- ## Key Management Service (KMS) A managed service that makes it easy for you to create and control the encryption keys used to encrypt your data, and uses FIPS 140-2 validated hardware security modules to protect the security of your keys. AWS Key Management Service is integrated with most other AWS services to help you protect the data you store with these services. AWS Key Management Service is also integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs. --- ## Kubernetes Ops (kops) Kops ships with geodesic and is the easiest way to get a production grade Kubernetes cluster up and running on AWS. ![Kops Logo](/assets/kops-logo.png) --- ## Key Performance Indicator (KPI) A Key Performance Indicator is a metric (e.g. number of requests per second) that indicates if some key business objective is being satisfied. Since it’s a metric, it means that it must be able of being measured or quantified. Businesses usually use multiple KPIs to evaluate their success at reaching concrete targets. --- ## kubectl [`kubectl`](https://kubernetes.io/docs/reference/kubectl/overview/) is a cli for controlling Kubernetes clusters. It's officially pronounced as "kube control", but we'll always refer to it as "kube cuddle". --- ## Kubernetes(Glossary) Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon 15 years of experience of running production workloads at Google, combined with best-of-breed ideas and practices from the community. --- ## Lambda(Glossary) AWS serverless (functions as a service) offering. --- ## Layer 7 The “Application Layer” (e.g. HTTP) --- ## Multi-Factor Authentication (MFA) Multifactor authentication is a security "Best Practice" of requiring more than one method to verify access credentials during authentication There are a lot of terms that roughly mean the same thing. - *MFA* - Multi-factor Authentication - *OTP* - One-time password (~MFA Token) - *2FA* - Two-factor authentication --- ## Monorepo Monorepo refers to a strategy of storing all code for possibly unrelated applications within the same source code repository. See also [polyrepo](/resources/glossary/polyrepo). --- ## On-call Engineer (OCE) The on-call engineer is the person currently assigned to take-point if any serious issues arise. They are typically the ones on “pager duty” who get a phone call in the middle of the night if/when things break. --- ## OSI Model [OSI](https://en.wikipedia.org/wiki/OSI_model) is a conceptual model consisting of 7 abstraction layers that represent the various functions of a computing system without regard to its underlying internal structure and technology. --- ## Platform-as-a-Service (PaaS) A Platform-as-a-Service is a type of cloud platform which offers black-box services that enable developers to build applications on top of the compute infrastructure without needing to deal with the day-to-day maintenance of the infrastructure. This might include developer tools that are offered as a service to build services, or data access and database services, or billing services. --- ## PagerDuty An incident management platform that provides reliable incident notifications via email, push, SMS, and phone, as well as automatic escalations, on-call scheduling, and other functionality to help teams detect and fix infrastructure problems quickly. --- ## Parameter Store The [Amazon Systems Manager Parameter Store](https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html) provides secure, hierarchical storage for configuration data management and secrets management. You can store data such as passwords, database strings, and license codes as parameter values. You can store values as plain text or encrypted data using KMS. You can then reference values by using the unique name that you specified when you created the parameter. Highly scalable, available, and durable, Parameter Store is backed by the AWS Cloud. Parameter Store is offered at no additional charge. --- ## Pingdom Pingdom is a service that tracks the availability (e.g. uptime & downtime) as well as the performance of websites. It’s been around for ages and is a tried-and-true method for being alerted when there’s an outage. They also have a feature called “Real User Monitoring” that uses a little bit of javascript to determine how fast your pages are loading for real end-users. Real-user-monitoring is essential for catching brown-outs where the site might be “up” but degraded. The company is based in Sweden, but they monitor your websites from multiple locations around the world so that they can distinguish between outages from localized network routing problems. They will also produce a `traceroute` from a location that has connectivity problems so you can help identify where in the network path traffic is affected. --- ## Polyrepo Polyrepo describes an approach of using multiple, independent source code repositories that are independently versioned and controlled. See also [monorepo](/resources/glossary/monorepo). --- ## Relational Database Service (RDS) Amazon Relational Database Service is a service that makes it easier to set up, operate, and scale a relational database in the cloud. It provides cost-efficient, resizable capacity for an industry-standard relational database and manages common database administration tasks such as backups, restores, and automatic failover. --- ## Release Engineering A sub-discipline of software engineering concerned with the compilation, assembly, and delivery of source code into finished products or other software components that are subsequently shipped to production --- ## Terraform Root Module Every Terraform configuration has at least one module, known as its root module, which consists of the resources defined in the `.tf` files in the main working directory. Root modules are the terraform configuration that we actually **apply** and have terraform state. Terraform has two types of modules; the top-level module is always called the "root" module and the modules that are called from the root module are called "child" modules. Root modules are the most opinionated. They describe the architecture you want to deploy. It's these "root" modules that we actually deploy when we run the `terraform apply` command. The "root" modules may contain many child modules or none at all. The "child" modules reusable modules that we invoke in root modules. [You can read more about this in Terraform documentation](https://www.terraform.io/docs/language/modules/index.html). Generally, when we refer to "modules" we mean "child modules". --- ## S3 Bucket An S3 bucket is a logical unit of storage in S3 that stores collections of objects. Think of an S3 bucket as a top-level folder. It must be globally unique on AWS across all customers and accounts. --- ## S3 [Amazon S3](https://aws.amazon.com/s3/) is an object storage service with a simple web service interface and API capable of storing and retrieving any amount of data from anywhere on the web. It is designed to deliver 99.999999999% durability, and scale past trillions of objects worldwide. --- ## s3fs S3FS refers both to an application, script and the concept of mounting a remote S3 bucket as a local filesystem. The geodesic implements an s3fs using [`goofys`](/learn/toolchain/#goofys). --- ## Software-as-a-Service (SaaS) Sofware-as-a-Service is a form of a cloud services platform, whereby the computing platform (operating system and associated services) is delivered as a service over the Internet by the provider. --- ## Sandbox Environment A sandbox environment is a place where developers can play around with new technologies without risk of impacting staging or production environments. --- ## Software Development Lifecycle (SDLC) The SDLC describes the process for planning, developing, testing, and deploying an application. --- ## Semantic Version Semantic versioning (e.g. `1.0.3`) is the most widely adopted scheme for assigning unique version numbers to software releases. --- ## Sidekick Containers A container that performs other duties that are related to our main application but shouldn't be directly built into that application. Examples of common sidekick containers are for service discovery or loading fixtures into a development database. --- ## Service Level Agreement A contract or agreement offered by a service provider that defines the expected level of service, responsibilities, priorities, and guarantees regarding availability, performance, and other aspects of the service. --- ## Slack Slack is a cloud-based service for team collaboration (chat, voice, video, screensharing, etc). Visit [Slack.com](https://slack.com) for more details. --- ## Subject Matter Expertise (SME) A subject-matter expert (SME) is a person who is an authority (domain expert) in a particular area or topic, which is referred to as the domain such as DevOps, Kubernetes, Terraform, etc. --- ## Simple Notification Service (SNS) [Amazon Simple Notification Service](https://aws.amazon.com/sns/) is a fast, flexible, fully managed push notification service that lets you send individual messages or to fan-out messages to large numbers of recipients. --- ## Amazon Systems Manager (SSM) [Amazon's Systems Manager](https://aws.amazon.com/systems-manager/) provides a unified user interface so you can view operational data from multiple AWS services and allows you to automate operational tasks across your AWS resources such as configurations with Parameter Store. Systems Manager simplifies resource and application management, shortens the time to detect and resolve operational problems, and makes it easy to operate and manage your infrastructure securely at scale. --- ## Single Sign-on (SSO) Single sign-on (SSO) is an authentication system that allows a user to login to multiple applications with one set of credentials. --- ## Stage One of the phases in the SDLC whereby software is deployed to an environment. Common stages are “Production”, “Staging”, “QA” or “Development” It's important to note that within a given stage, there might be multiple environments. We always prescribe separating multiple stages by using multiple AWS accounts or organizational units. Then provisioning multiple environments within that stage as necessary. For example, the "staging" account might run "pre-production" and "QA" environments. --- ## Synthetic Monitoring Synthetic monitoring is a style of monitoring that attempts to closely emulate the behavior of an end-user, often by using scripted recordings of web transactions played back through a web browser using plugin like Selenium. The scripts carry out behaviors (such as traversing paths) to simulate an action that a customer or end-user would take on a site. E.g. we highly recommend testing all user registration flows and password reset forms using this methodology. --- ## Technical Debt Technical debt represents all the things that need to be redone later due to poor choices or necessary tradeoffs made today. It should be noted that the cost of tech debt is very real. It manifests in the form of engineering costs to address the problem, opportunity costs of fixing it versus working on new features, and business costs such as lost revenue from frustrated customers. :::info The best article ever written on "Tech Debt", is by Bill Clark at Riot Games. - https://engineering.riotgames.com/news/taxonomy-tech-debt ::: --- ## HashiCorp Terraform [HashiCorp Terraform](https://www.terraform.io/) is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers such as AWS as well as custom in-house solutions. Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. --- ## Topology A topology is the arrangement of various elements (links, nodes, etc.) in a network architecture. --- ## Unlimited Staging Environments The ability to run complete, disposable apps on Kubernetes for Staging and Development. --- ## Virtual Private Cloud (VPC) A virtual private cloud (“VPC”) is an isolated environment running inside of a public cloud. You can think of it like a private data center with its own private subnets and resources. A VPC can also be an extension of a physical datacenter. The most common example of a VPC is the service that Amazon offers called “Amazon VPC” which allows customers to make Amazon EC2 an extension to their physical infrastructure using an IPsec VPN tunnel. --- ## WikiOps Wiki documentation driven operation processes. WikiOps is the pattern of following Wiki-style documentation as the means of building and operating your infrastructure. This is to be avoided as it relies on ClickOps and documentation that becomes quickly outdated and inconsistent. --- ## YAML(Glossary) YAML (or YAML Ain't Markup Language) is a human-readable data-serialization language. YAML is a configuration syntax (aka data-serialization format) that is pervasive throughout the DevOps community. It is a superset of the popular JSON configuration format but provides a more concise, human-readable interface while also supporting value inheritance via [anchors](https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors). --- ## yq [`yq`](https://github.com/mikefarah/yq) is a Go-based command line tool for YAML that supports standard CRUD operations. --- ## AWS Feature Requests and Limitations import Intro from '@site/src/components/Intro'; import KeyPoints from '@site/src/components/KeyPoints'; # AWS Feature Requests and Limitations ### ALB/ELB Default TLS Support **Use-case:** e2e encryption behind CDN (E.g. CloudFlare) **Use-case:** turnkey CloudFormation template to create a secure stack without requiring DNS or email validation (difficult to obtain in enterprise environments) Resources like RDS clusters/instances and Elasticache Redis support TLS out-of-the-box without a dependency on ACM. Load balancers should support the same to facilitate end-to-end encryption by default. For example, origins fronted by a CDN do not need any sort of branded/vanity hostnames and would suffice with an [amazonaws.com](http://amazonaws.com) TLS certificate. ### Facilitate AWS Account Automation **Use-case:** e2e automation of AWS accounts (full SDLC) AWS accounts are today treated as scarce quantities and pets. Net-new AWS accounts are not permitted to raise the limit, yet a standard well-architected infrastructure includes about a dozen accounts (master, identity, prod, staging, uat, dev, network, workspaces, sandbox, audit, security, data, etc) Once accounts are created, they cannot be easily destroyed and require human intervention. This precludes e2e CI/CD testing of account automation. Account email addresses are one-time-use. After destroying an account, it cannot be recreated using the same email address, requiring a roundabout way of first renaming the address on-file before destroying. AWS already has the concept of Organizations with master accounts and child accounts. This could be extended to allow for automation accounts, which would inherit limits from parent accounts (or perhaps take a fraction of the parent account’s limit) when they are created and restore the limits to the parent account when they are deleted. Similar to KMS keys, they could pass through a frozen stage where they have a limited lifespan where they can be revived, but preferably, accounts less than 1 day old could be deleted immediately. ### [Launched] Managed Prometheus-compatible Time Series Database :::info AWS Launched Amazon Managed Service for Prometheus which provides highly available, secure, and managed monitoring for your containers [https://aws.amazon.com/prometheus/](https://aws.amazon.com/prometheus/) ::: ~~AWS provides a time series service called Timestream, but it does not support a format compatible with Prometheus. Prometheus~~ [~~won’t support it~~](https://github.com/prometheus/prometheus/issues/5086) ~~out-of-the-box. Given that Prometheus is the most popular open source monitoring framework, this would be a big win and a great way to increase adoption for Timestream.~~ ### More flexible high-level CDK constructs We were severely hampered in our use of high-level CDK constructs because we wanted to create an environment-agnostic CFT that deploys into an existing VPC. We created the CFT using low level constructs that took VPC ID and subnets as parameters, but it would have been much easier for us if we could have fed those parameters into the high level constructs where they currently require a full VPC object created at synth time. One specific example, we cannot create CloudWatch Metrics or Alarms on the Application Load Balancer we create in our CFT because that requires text processing of the Load Balancer ARN, which the CDK’s `Arn.parse` method is [explicitly documented as being unable to do](https://docs.aws.amazon.com/cdk/api/latest/python/aws_cdk.core/Arn.html#aws_cdk.core.Arn.parse). If we used the high-level `ApplicationLoadBalancer` construct, it would provide us with `Metric` objects, but we cannot use it because [it requires an](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-elasticloadbalancingv2.ApplicationLoadBalancer.html#metricmetricname-props) `IVpc` object. ### Better CDK documentation We understand documentation is hard, and there is voluminous documentation for the CDK, but at the same time, the documentation is largely at a very low level that leaves a lot of gaps. For example [https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-deletionpolicy.html) explains that `AWS::EC2::Volume` supports a DeletionPolicy of Snapshot, but we have been stumped about how to use that in conjunction with an EC2 instance set to “delete on termination” an EBS data volume, to the point where we are not even sure it is truly supported. Similarly, we could not find examples about how to set up, using the Python CDK, a stack that has CloudWatch metrics and alarms that would monitor instance metrics for EC2 instances launched as part of an AutoScaling Group. All we could find were alarms on the AutoScaling Group itself, such as `autoscaling:EC2_INSTANCE_LAUNCH_ERROR`. We found [this note](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-autoscaling-readme.html#future-work) but disagreed about what it means: `Future Work: CloudWatch Events (impossible to add currently as the AutoScalingGroup ARN is necessary to make this rule and this cannot be accessed from CloudFormation).` Does that mean we also cannot set up CloudWatch Metrics and Alarms? Here is another [random example](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-cloudwatch.CreateAlarmOptions.html#evaluatelowsamplecountpercentile): `evaluateLowSampleCountPercentile` is a `string` that “specifies whether to evaluate the data and potentially change the alarm state if there are too few data points to be statistically significant.” Why is that a `string`? How is the string interpreted? I would expect it to be a boolean given the variable name. ### Account Limit operations should be available via API Currently, AWS does not provide a generally available way to alert when limits are reached. This is adverse to providing an ultimately stable platform for your customers. Reaching limits is something that can happen when you least expect it and lead to forced outages. AWS needs to make it easier to programmatically increase these limits (e.g. via an API we can call from terraform) and trigger alerts when thresholds are met. [https://aws.amazon.com/solutions/implementations/limit-monitor/](https://aws.amazon.com/solutions/implementations/limit-monitor/) [https://github.com/awslabs/aws-limit-monitor](https://github.com/awslabs/aws-limit-monitor) ### [Launched] EKS Managed Node Groups Custom Userdata support **~~Use-case~~**~~: building containers with “Docker in Docker” with Jenkins on managed node groups~~ ~~[https://github.com/aws/containers-roadmap/issues/596](https://github.com/aws/containers-roadmap/issues/596)~~ ~~Also want to be able to set Kubernetes node labels and taints at instance launch.~~ ### Next Generation AWS VPC CNI Plugin **Use-case:** achieve higher container density per node [https://github.com/aws/containers-roadmap/issues/398](https://github.com/aws/containers-roadmap/issues/398) The problem is that our customers frequently run less CPU-intensive workloads and could achieve greater efficiency and economies of scale by packing more containers on an instance. EKS restricts the number of pods per node implicitly by limiting the number of ENIs. In order to run more pods, a larger instance is required that supports more ENIs, but then the instances are underutilized. In the end, it feels like a backhanded way of forcing customers to pay for more than they need - like enterprise software licensing charging per core. No updates on this issue for over a year. ### CloudFront Distributions with EdgeLambdas take too long to destroy **Use-case**: e2e automation testing The problem is that Edge Lambdas once provisioned can take hours if not days to destroy. Implementing any kind of automated testing (e.g. using [terratest](https://github.com/gruntwork-io/terratest)) for this is complicated then by the fact we cannot iterate. Infrastructure on Amazon must be considered ephemeral and be built with testing in mind from the ground up, especially as Infrastructure as Code is becoming more mainstream. ### CloudFront Distributions are Too Slow to Modify Use-case: ### [Launched] DNSSEC on Route53 [https://aws.amazon.com/about-aws/whats-new/2020/12/announcing-amazon-route-53-support-dnssec/](https://aws.amazon.com/about-aws/whats-new/2020/12/announcing-amazon-route-53-support-dnssec/) ### NLB Health Checks Overwhelm Origins Something needs to be done with Network Load Balancer TCP health checks. The health checks easily overload origin servers when they fail. It is bad enough that the health check interval cannot be configured and that when healthy we are getting an average of 120 health checks per minute, but when they fail the number skyrockets to ~3,000 per minute. The linked issue has details. [https://github.com/kubernetes/ingress-nginx/issues/5425](https://github.com/kubernetes/ingress-nginx/issues/5425) ## Bugs These are some bugs that affect us. The bugs might be mitigated by the Terraform providers or are upstream and may not be AWS bugs. `Error: Creating CloudWatch Log Group failed: ResourceAlreadyExistsException: The specified log group already exists: The CloudWatch Log Group '/aws/eks/eg-test-eks-cluster/cluster' already exists` [https://github.com/cloudposse/terraform-aws-eks-cluster/issues/67](https://github.com/cloudposse/terraform-aws-eks-cluster/issues/67) This is happening because EKS Cluster gets destroyed after Terraform deletes the Cloudwatch Log Group. The `AmazonEKSServicePolicy` IAM policy (that is assigned to EKS Cluster role by default within this module) has permissions to `CreateLogGroup` and anything else needed to continue logging correctly. When the Terraform destroys the Cloudwatch Log Group, the EKS Cluster that is running recreates it. Then, when you run Terraform Apply again, the Cloudwatch Log Group doesn't exist in your state anymore (because the Terraform actually destroyed it) and the Terraform doesn't know this resource created outside him. --- ## Code of Conduct(Legacy) This code of conduct governs all documentation and related [Open Source Projects](https://github.com/cloudposse). **TL;DR:** Cloud Posse is a DevOps Accelerator founded to help organizations collaborate on DevOps. We want to make sure our community is as constructive as possible in order to help each other solve cool DevOps problems. Anything that goes against the spirit of this, isn’t cool. If you see something suspicious, derogatory, or otherwise inappropriate, etc. please send an email to [abuse@cloudposse.com](mailto:abuse@cloudposse.com) with details. ### Introduction We strive to be inclusive to the largest number of contributors, with as diverse backgrounds as possible. That is why we are committed to providing an environment that encourages friendly, safe, and welcoming discussions for everyone interested in learning and sharing. This code of conduct outlines our expectations for all those who participate in our community, as well as the consequences for unacceptable behavior. We invite all those who participate in SweetOps to help us create safe and positive experiences for everyone. ### Etiquette - Do not cross-post questions across multiple channels (pick one) - Use the professional language you would use at work. Avoid using profanity in any language. - Praise publicly (with tons of emojis!) - Criticize constructively, but never disparagingly (e.g. there are no “stupid” questions) - Practice empathy (e.g. “Ahk, that shouldn’t happen”) - Correct delicately (e.g. “Have you considered...”) - Avoid sarcasm, or feigning surprise (e.g. Don’t say “I’m surprised you didn’t know that”) - Seek to understand, not to embarrass, discredit or humiliate. (e.g. “Hrmm... could you explain that another way? I’m not following.”) - When posting for the first time, review the channel history to get a sense of how others interact. Also, check out our “Guidelines When Asking for Help.” - Check out the channel’s topic and purpose in order to stay on point - Use threads when responding, especially in busy channels - Don’t use `@channel` and `@here` (we've disabled them) ### Unacceptable Behavior The following are behaviors we consider forms of harassment that are unacceptable within our community: - Comments that may be considered sexist, racist, homophobic, transphobic or otherwise discriminatory or offensive in nature. (Everything you say in public is on the record) - Usernames that contain unwelcome, suggestive, derogatory or inappropriate terms. - Disrespect towards others is not tolerated. - Unprofessional jokes, innuendo, dismissive attitudes. - All forms of intimidation or harassment are prohibited. - Foul or obscene language (in any language!) - Disrespect towards differences of opinion. - Disparagement. - Deceptive or fraudulent activity. - Sustained disruptions (like spamming channels). - All forms of violence, threats of violence or violent language. This is not an exhaustive list of things we prohibit. In order to preserve the spirit of this community, we reserve the right to use our discretion when applying disciplinary actions to behaviors we deem inappropriate. ### Solicitation We strictly prohibit companies or individuals to solicit work, post jobs, or engage in self-promotion. It is not acceptable to engage in this behavior in any channel or to send unsolicited direct messages concerning job opportunities or job marketplaces. Under no circumstances is it acceptable to solicit payment from members for any reason (e.g. recruiting fees). Additionally, the promotion of any kind of third-party job marketplace is not permitted. Any direct messages to members should be in relation to specific messages posted about job opportunities or candidates seeking work, but absolutely not for promoting other services. Recruiters may only reach out to individual candidates who have posted in the `#jobs` channel but are not permitted to directly solicit members or companies in this community. If you’ve received an unwelcome solicitation, please email [abuse@cloudposse.com](mailto:abuse@cloudposse.com) with details including screenshots. ### Guidelines When Asking for Help **Tips to follow when asking for help:** We appreciate that not everyone has all the answers every time. That’s exactly why our community is here to help. Keep in mind, we are all volunteers. Everyone donates their time; no one receives any payment for helping answer your questions. 1. Research the problem or question. Be prepared to share links of what you’ve already tried and considered. We expect you’ve “Googled it” before we do. If you don’t know where to begin, that’s okay. In that case, ask for help on how to approach the problem. 2. State your problem, not just your solution. The [XY Problem](http://xyproblem.info/) is the fallacy of asking for help about your _attempted solution_ rather than ask for help solving your _actual_ _problem_. This leads to enormous amounts of wasted time and energy, both on the part of people asking for help, and on the part of those providing help. 3. Search our Slack for the answer. We see a lot of the same questions asked over and over again. Do a quick search before asking your question. 4. Ask Constructive Questions. When asking your question, mention any research you’ve done, relevant links and GitHub issues, as well as provide any relevant code samples. 5. Use Markdown Formatting. When pasting code samples, make sure you use markdown formatting. Code that is not formatted makes it difficult for others to read and reduce the likelihood of getting an answer. For longer blobs of text, make sure you use snippets. **IMPORTANT**: Make sure you have permission to share any code samples because these will be public. 6. Use the Appropriate Channel. We have dozens of channels that help us organize all the information and questions asked. The #general channel is only for welcoming new users or asking for help on where to ask a question. 7. Do Not Cross-Post. Use one channel at a time to avoid spamming our community. Keep in mind, users are in 57 timezones. You may not get an answer right away. 8. Keep Conversations in Threads. Our channels get pretty busy. When multiple concurrent conversations are happening, it’s hard to keep it straight. Also, it’s helpful if you add supplementary information in the thread of the question, rather than posting a new message. 9. Give Copious Praise. Did you get the answer you were looking for? Did someone help you? Let them know! We operate on praise. There’s never enough. ### Privacy All conversations that happen in our Slack channels are shared with other customers. For this reason, members should not have any expectation of privacy in their use. It’s the responsibility of our members to avoid sharing any information that might lead to loss or damages, such as a breach of their company’s security protocols or sharing of intellectual property. All direct messages are private. They are not accessible by anyone on the Slack team, including moderators and administrators. Your email address is private and will not be shared with anyone outside of Cloud Posse. From time to time, Slack (the Company) or Cloud Posse may send you notifications. If you are unable to opt-out for any reason, please let us know at [abuse@cloudposse.com](mailto:abuse@cloudposse.com). Additionally, Slack (the Company) has its own [Privacy Policy](https://slack.com/privacy-policy) which affects all communities hosted on the platform. ### Logs and Records **Please be mindful that anything you say in our community is a matter of public record**. We cannot prevent people from taking screenshots or otherwise logging conversations in this slack team. We also can’t guarantee that every member’s login credentials and logged-in devices are secure and not compromised. Keep in mind that all files uploaded to this slack team can be viewed and downloaded by anyone both via Slack and our archives. Please exercise caution and refrain from sharing sensitive information that could harm you or others if it became public. ### Message Retention Slack (the company) retains a complete log of all channels and direct messages back to the inception of the SweetOps Slack team. Slack is a U.S. company and therefore subject to subpoenas from U.S. courts. Our logs may be subject to subpoena and could become public as part of legal proceedings. ### Reporting Guidelines Please let us know if you encounter any unacceptable behavior by sending us an email at [abuse@cloudposse.com](mailto:abuse@cloudposse.com). We would appreciate it if you would include any screenshots that provide a full context of whatever transpired. We might not be able to see what you see, especially if what was sent was in a private direct message. ### Moderation We perform both automated and human moderation. To some degree, we expect our community to self-moderate and let us know when they see unacceptable behavior. Moderators reserve the right to delete excessive self-promotional or commercial posts. Similarly, any objectionable, inappropriate or off-topic comments may be deleted. Members posting this content will receive a warning and risk being blocked from the team if the unacceptable behavior persists. Official moderators of our community are: - Erik Osterman - Andriy Knysh ### Enforcement & Consequences Any unacceptable behavior will not be tolerated. If you are the subject of or witness to any violations of this Code of Conduct, please email us at [abuse@cloudposse.com](mailto:abuse@cloudposse.com). If violations occur, our community moderators will take necessary and appropriate actions, up to and including expulsion from the community. ### Membership Our customer community is private and restricted to paying customers. We reserve the right to remove anyone for any reason. Our goal is to facilitate collaboration across organizational boundaries so we can achieve SweetOps. ### Open Source Collaboration One of the big goals of our community is to increase the amount of Open Source collaboration in the DevOps community. We hope that by encouraging participants to recognize and strengthen the relationships between companies, organizations and individuals that their effects on our community will be amplified. This is why we go the extra mile to ensure our community is open, welcoming, friendly, and encourages all participants to contribute to the fullest extent possible. ### “Office Hours” Support Every week we hold public [community “office hours”](https://cloudposse.com/office-hours) which are an opportunity for members to get to know each other face-to-face, ask questions, and get help. These sessions abide by this Code of Conduct. These sessions may be recorded, published and shared. ### Projects We support a lot of Open Source projects. These projects abide by this Code of Conduct. - [https://github.com/cloudposse](https://github.com/cloudposse) ### Contact We want to hear from you. Please let us know if there’s something we should add, amend or update as part of our Code of Conduct. **Email:** [community@cloudposse.com](mailto:community@cloudposse.com) **Phone:** +1 (310) 496-6556 **Mailing Address:** 340 S Lemon Ave #1430 Walnut, CA 91789 --- ## Demo Applications Here’s a list of fancy demo applications, predominantly for Kubernetes. 1. [https://github.com/GoogleCloudPlatform/microservices-demo](https://github.com/GoogleCloudPlatform/microservices-demo) (by Google) 2. [https://github.com/microservices-demo/microservices-demo](https://github.com/microservices-demo/microservices-demo) (by Weaveworks) 3. [https://github.com/dotnet-architecture/eShopOnContainers](https://github.com/dotnet-architecture/eShopOnContainers) (by Microsoft) 4. [https://github.com/istio/istio/tree/master/samples/bookinfo](https://github.com/istio/istio/tree/master/samples/bookinfo) (by Istio)[https://github.com/kubernetes/examples](https://github.com/kubernetes/examples) 5. [https://github.com/kubernetes/examples](https://github.com/kubernetes/examples) Guestbook - classic demo (by Kubernetes) --- ## Archived Decisions :::info These Design Decisions have been superseded and are no longer used. ::: ## Related Decisions - [Decide on Status Page Requirements](/resources/legacy/design-decisions/archived-decisions/decide-on-status-page-requirements) - [Decide on RDS requirements](/resources/legacy/design-decisions/archived-decisions/decide-on-rds-requirements) --- ## Decide on RDS requirements Superseded by [Decide on RDS Technology and Architecture](/layers/data/design-decisions/decide-on-rds-technology-and-architecture) --- ## Decide on Status Page Requirements ## Problem The business may be obligated or want to communicate transparently to customers the availability of all services affecting the platforms/products stability. ## Solution Use [StatusPage.io](http://StatusPage.io) by Atlassian to integrate with OpsGenie and other dependent services to communicate overall availability. ## Other Considerations - Should it be **Public** (customer-facing) or **Private** (internal facing only)? - How to host it? - What to expose to customers? - Should it pull in the availability of third-party dependencies? e.g. Twillio API --- ## Decide on API Gateway Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement ## Overview Amazon API Gateway is an AWS service designed to simplify publishing highly-scalable REST, HTTP, and WebSocket APIs. These API Gateways act as a central point of access at the edge and can access backend services running on EC2, EKS, ECS, Lambda, and AWS Services directly, such as DynamoDB, S3, or SQS. The API Gateway has several benefits over a conventional ALB in that it’s optimized for APIs: namely, it can authenticate requests, cache, rate-limiting, feature flagging, a/b testing, rewrite requests/responses, aggregate requests, etc. It’s arguably a simpler alternative to using something like a Service Mesh. ## Common Scenarios There are several common use cases for API Gateway, some of the most common of which are detailed below. ### REST API :::info Choose REST APIs if you need features such as API keys, per-client throttling, request validation, AWS WAF integration, or private API endpoints. [https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html) ::: At its core, an API Gateway REST API comprises resources (e.g. customers) and methods. A resource is a logical entity that can be accessed through a resource path. A method corresponds to a REST API request submitted by the user of your API and the response returned to the user. > [RESTful APIs have strong controls](https://aws.amazon.com/blogs/compute/from-poll-to-push-transform-apis-using-amazon-api-gateway-rest-apis-and-websockets/#:~:text=RESTful%20APIs%20have%20strong%20controls%20to%20ensure%20that%20user%20requests%20for%20data%20are%20validated%20and%20given%20guardrails%20for%20the%20intended%20query%20purpose.%20This%20helps%20prevent%20rogue%20requests%20and%20unforeseen%20impacts%20to%20the%20analytics%20environment%2C%20especially%20when%20exposed%20to%20a%20large%20number%20of%20users.) to ensure that user requests for data are validated and given guardrails for the intended query purpose. This helps prevent rogue requests and unforeseen impacts to the analytics environment, especially when exposed to a large number of users. > For example, `/customers` could be the path of a resource representing the business’s customers. A resource supports one or more operations defined by standard HTTP verbs such as GET, POST, PUT, PATCH, and DELETE. A combination of a resource path and an operation identifies a method of the API. For example, a `POST /customers` method could add a new customer, and a `GET /customers` method could return a list of all of the customers. The API caller doesn't need to know where the requested data is stored and fetched from on the backend. In API Gateway REST APIs, the front-end interface is encapsulated by _method requests_ and _method responses_. The API interfaces with the backend by means of _integration requests_ and _integration responses_. For example, with DynamoDB as the backend, the API developer sets up the integration request to forward the incoming method request to the chosen backend (DynamoDB). The setup includes specifications of a DynamoDB action, required IAM role and policies, and required input data transformation. The backend returns the result to API Gateway as an integration response. To return the integration response to the client (method response), you can configure the integration response to map response parameters from integration to method. You can also translate the output data format of the backend to that of the front end (e.g. map specific DynamoDB columns to the JSON response), if necessary. API Gateway enables you to define a schema or model for the [payload]() to facilitate setting up the body mapping template. In addition to the functionality listed above, API Gateway REST APIs also provide additional management functionality such as: - Generating custom SDKs and creating API documentation using API Gateway extensions to OpenAPI - API Key management - Throttling of HTTP requests ### HTTP API :::info Choose HTTP APIs if you don't need the features included with REST APIs. ::: HTTP APIs enable you to create RESTful APIs with lower latency and cost than REST APIs. You can use HTTP APIs to forward requests to any publicly routable HTTP endpoint or AWS Lambda Functions. For example, you can create an HTTP API that integrates with a Lambda function on the backend. When a client calls your API, API Gateway sends the request to the Lambda function and returns the function's response to the client. The [Use API Gateway REST API vs HTTP API](/resources/adrs/adopted/use-api-gateway-rest-api-vs-http-api) document compares and contrasts the differences between REST and HTTP API Gatewayways. ### WebSocket API :::info Choose a WebSocket API to push results to your clients. Note, consider mixing and matching REST APIs and WebSocket APIs. See [https://aws.amazon.com/blogs/compute/from-poll-to-push-transform-apis-using-amazon-api-gateway-rest-apis-and-websockets/](https://aws.amazon.com/blogs/compute/from-poll-to-push-transform-apis-using-amazon-api-gateway-rest-apis-and-websockets/) ::: A WebSocket API is a special kind of API where the connection stays open between the client and the server so they can send messages anytime. Backend servers can easily push data to connected users and devices, avoiding implementing complex polling mechanisms. For example, you could build a serverless application using an API Gateway WebSocket API and AWS Lambda to send and receive messages to and from individual users or groups of users in a chat room. Or you could invoke backend services such as AWS Lambda, Amazon Kinesis, or an HTTP endpoint based on message content. You can use API Gateway WebSocket APIs to build secure, real-time communication applications without having to provision or manage any servers to manage connections or large-scale data exchanges. Targeted use cases include real-time applications such as the following: - Chat applications - Real-time dashboards such as stock tickers - Real-time alerts and notifications ## Considerations ### Authentication Authentication and Authorization is one of the more complex topics when using API Gateway because there are so many different options that vary depending on the API Gateway you are deploying, or may not even be deployed at all, in the case of a public API. #### REST API REST APIs support the following authentication types: - IAM Authentication - AWS Cognito - API Key - Lambda Authorizer ### Monitoring and Tracing **TODO** ### Web Application Firewall (WAF) **TODO** See also:[Decide on WAF Requirements/Strategy](/layers/security-and-compliance/design-decisions/decide-on-waf-requirements-strategy) ## References - [Use API Gateway REST API vs HTTP API](/resources/adrs/adopted/use-api-gateway-rest-api-vs-http-api) - [Decide on WAF Requirements/Strategy](/layers/security-and-compliance/design-decisions/decide-on-waf-requirements-strategy) - [https://aws.amazon.com/blogs/compute/from-poll-to-push-transform-apis-using-amazon-api-gateway-rest-apis-and-websockets/](https://aws.amazon.com/blogs/compute/from-poll-to-push-transform-apis-using-amazon-api-gateway-rest-apis-and-websockets/) - [https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html) --- ## Decide on CloudFront Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement A CDN such as CloudFront speeds up the user experience primarily in 2 ways. 1. **Caching.** By caching content on a server closer to the user (called the “edge server”), the user can benefit from the much shorter connection path to the nearby server, obtaining responses much more quickly. 2. **Connection Pooling & TLS offloading.** By maintaining a long-lived multiplexed connection to the origin (such as HTTP/2), the edge server can avoid the overhead of creating a new connection for each request and make other optimizations so that it can retrieve the data from the origin server considerably faster than an end user who makes a new connection for each request. Even compared to an end user using HTTP/2, a CloudFront edge server connected to an AWS origin benefits from traffic traveling over the optimized high-speed AWS backbone network and other optimizations built into CloudFront. CloudFront also offers the ability to limit access to content based on the geographic location of the requester, field-level encryption of HTTP `POST` content, and other features. Typically, there are 3 common use-cases for CloudFront that need to each be addressed in a different way. ## Considered Options In order to implement your CDN strategy with CloudFront, we’ll need to know a little bit more about how you currently use it and how you intend to use it. Any combination of one or more of the options below is supported but will need to be implemented differently. Keep in mind, as soon as you introduce a CDN, you have to solve the cache invalidation problem. > _There are only two hard things in Computer Science: cache invalidation and naming things._ > _-- Phil Karlton_ > [https://www.karlton.org/2017/12/naming-things-hard/](https://www.karlton.org/2017/12/naming-things-hard/) ### Option 1: Origin Acceleration for Dynamic Content :::caution If you intend to also serve static assets behind a CDN, see _Option 2._ ::: #### Pros - Accelerate delivery of frequently requested content - Reduce the compute capacity required to serve cacheable content - Improve SEO [https://www.semrush.com/blog/how-fast-is-fast-enough-page-load-time-and-your-bottom-line/](https://www.semrush.com/blog/how-fast-is-fast-enough-page-load-time-and-your-bottom-line/) - Relatively easy to deploy (effectiveness will depend on cache rules and cachabilty of content) - Handle spikey request traffic - Speed up TLS connections - Pipelining requests from the edge to your origins - Conceal your origin servers from attackers - More easily mitigate DoS/DDoS attacks #### Cons - More difficult, less payoff for caching private assets - Accidentally cache private content and serve it to another session or leak it publicly - Not all content is “cacheable” (content with a short lifetime, private content, large content e.g. assets that are GBs in size) - In frequently accessed content frequently expunged from the cache before getting subsequent hits ### Option 2: Origin Acceleration for Static Assets in S3 (e.g. Images, CSS) The most common pattern we deploy is supporting [`aws-spa-s3-cloudfront`](https://github.com/cloudposse-terraform-components/aws-spa-s3-cloudfront). We’ll need to know if this is a pattern you intend to utilize, as it has an impact on the strategies for CI/CD. Each strategy will at a minimum require a different terraform component. :::caution If you intend to serve static assets behind a CDN, this is the only advisable way of doing it. Rolling updates associated with deployments to Kubernetes or ECS means that serving static assets from your application servers will likely yield inconsistent results during deployments. ::: #### Recommendations - Version your assets by release version or commit SHA. - Release your CSS alongside your images/fonts with relative paths - Ensure your applications are loading the assets for their version - Do not use query string style cache busting (easy to cache the wrong version of the asset) #### Pros - Relatively easy to implement - S3 storage is very inexpensive relative to the typical scale of data for static assets - Easier rollouts/rollbacks (no need to worry about clients requesting the wrong version of the assets) #### Cons - An additional deployment target - If your application was not built with this in mind, it could be costly to update all the hardcoded references to assets. The good news is most modern frameworks when implemented correctly handle this for you automatically. ### Option 3: Edge Lambda Dynamic Processing (Lambda@Edge) Lambda@Edge lets you run code closer to users of your application, which improves performance and reduces latency. The major benefit is that with Lambda@Edge, you don't have to provision or manage multi-region infrastructure, and you pay only for the compute time you consume - there is no charge when your code is not running. Common use-cases include Website Security & Privacy, Dynamics server-side applications at the edge, Search Engine Optimization (SEO), Data localization or Intelligent Routing to origins and datacenters, bot mitigation, image transformations (e.g. resizing), A/B Testing, real-time authentication and authorization, user prioritization, user tracking, and analytics. [https://aws.amazon.com/lambda/edge/](https://aws.amazon.com/lambda/edge/) #### Pros - Serve content dynamically generated at the edge without relying on client capabilities - Reduce overhead on origin servers to generate dynamic content - See _common use_ _cases above._ #### Cons - There are tons of restrictions on Edge lambda functions intended to ensure they are ultra-performant and do not tax edge locations. Make sure you’re aware of them. [https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/edge-functions-restrictions.html](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/edge-functions-restrictions.html) - Websites that rely on edge content manipulation will require some compensating controls for local development (harder to test and debug) - The functions are more difficult to implement for _Preview Environments_ due to the complexity around managing lambda functions and deployments with the CDN. ## References - **Lambda@Edge Design Best Practices** [https://aws.amazon.com/blogs/networking-and-content-delivery/lambdaedge-design-best-practices/](https://aws.amazon.com/blogs/networking-and-content-delivery/lambdaedge-design-best-practices/) - **Service Quotas (Limitations)** [https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-limits.html](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cloudfront-limits.html) --- ## Decide on Cognito Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Overview Amazon Cognito provides authentication, authorization, and user management for your web and mobile apps. Users can sign in directly with a user name and password stored in Cognito or through a third party such as a SAML IdP, OIDC, Facebook, Amazon, Google, or Apple. There are two main components to Amazon Cognito: User Pools and Identity Pools. User Pools are directories of your applications’ users while Identity Pools are used to grant application users access to other AWS services such as S3. User Pools and Identity Pools can be used together or separately, depending on your needs. ## Common Scenarios There are several common use cases for Cognito, some of the most common of which are detailed below. To configure the Cognito component, we’ll need to know as much as possible about your configuration. Ideally, you’ve already set it up previously and can share the precise configuration you’re using. ### App Authentication and Authorization In this scenario, Cognito becomes your application's authentication and authorization service. You allow users to log in with a username and password (stored in a Cognito User Pool), social provider (Google, Facebook, Apple, etc), OIDC Provider (GitHub), or SAML IdP (Okta) and your app trusts Cognito to verify the identity of the user. After successful authentication, your app will receive a signed JSON Web Token (JWT) which it can use to control access to features and functionality within your application. Groups may be set up within the User Pools (and sent along to your application in the JWT after login) to represent different user Roles within the application. ### API Gateway Access In this scenario, a user of your application logs in to the User Pool via one of the methods discussed previously (U/P, Social, IdP, OIDC). Then the application makes requests to your API Gateway, providing the Cognito-issued JWT in the request via HTTP headers. The API Gateway, via an Amazon Cognito authorizer Lambda function, would then use the information contained in the JWT to determine if the API operation is allowed or denied. ### AWS Service Access For this use case, your application obtains a third-party identity token (from a Social, SAML IdP, or OIDC provider), or if your use case requires, generates an anonymous token, and exchanges that token for temporary AWS Credentials that allow direct access to certain AWS services (via IAM roles). One common scenario for this type of access would be granting the user the ability to upload files directly to an S3 bucket. ## Considerations ### Amazon Cognito Authorizer Lambda Function We’ll need to have a lambda authorizer function deployed. This will also require a strategy for deploying the Lambda function and the function must be deployed before provisioning Cognito. ### Migrations :::caution **No Easy Migration Path** If you are already using Cognito and are allowing users to sign into a User Pool with Username and Password, one of Cognito's major limitations is that you cannot migrate the passwords from those User Pools to a new User Pool. ::: We have seen customers take two approaches to migration: - Require all users to reset their password through the “forgot password” functionality - Create application logic that first checks to see if the Username exists in the new User Pool. If the user already exists, then authentication proceeds. Suppose the user does not exist in the new User Pool. In that case, the application validates the password against the old User Pool, and assuming a successful authentication, creates a user in the new User Pool and sets its password to the password it successfully validated against the old User Pool. ### User Pools & Identity Providers Before implementing Cognito, we need to document the User Pool(s) you would like to provision along with any Identity Providers you would like to use. --- ## Decide on IAM Roles for GitHub Action Runners import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem In order for GitHub Actions runners to be able to access AWS resources, they need to be able to assume an IAM role. The trust relationship of this IAM role depends on how the Runners are hosted: - GitHub-hosted Runners: [Need to GitHub's OIDC Provider](https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect) - Self-hosted Runners on EKS: Need to use the EKS OIDC Provider (See: [IRSA](https://docs.aws.amazon.com/emr/latest/EMR-on-EKS-DevelopmentGuide/setting-up-enable-IAM.html)) - Self-hosted Runners on EC2: Need to have an Instance Profile Regardless of the trust relationship, the IAM roles themselves need to be defined.GitHub Action Workflows regularly use third-party actions. These actions execute on the self-hosted runners and have whatever access the runners have, including VPC connectivity and IAM permissions. ## Considered Options Some possible configurations (and combinations thereof) include: - A Runner without any IAM roles (e.g. for unit tests in a CI pipeline). - A Runner with access to `dev` and/or `sandbox` (e.g. for integration tests). - A Runner with access to ECR in the `artifacts` account. - A Runner with access to S3 buckets in SDLC accounts, and those S3 buckets allowing the role via an S3 bucket policy. - A Runner with access to EKS (e.g. if ArgoCD is not used and push-based deployments are required). ## Related Decisions - [Decide on IAM Roles for GitHub Action Runners](/resources/legacy/design-decisions/decide-on-iam-roles-for-github-action-runners) - [Decide on Self-Hosted GitHub Runner Strategy](/layers/software-delivery/design-decisions/decide-on-self-hosted-github-runner-strategy) - [Decide on Strategy for Continuous Integration](/layers/software-delivery/design-decisions/decide-on-strategy-for-continuous-integration) - [Decide on GitHub Actions Workflow Organization Strategy](/layers/software-delivery/design-decisions/decide-on-github-actions-workflow-organization-strategy) --- ## Decide on Kinesis Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Context and Problem Statement Amazon Kinesis collects, processes, and analyzes real-time, streaming data to get timely insights and react quickly to new information. It provides key capabilities to cost-effectively process streaming data at any scale, along with the flexibility to choose the tools that best suit the requirements of your application. Frequently, it’s used to ingest real-time data such as video, audio, application logs, website clickstreams, and IoT telemetry data for machine learning, analytics, and other applications. With Kinesis, information can be acted on as soon as it arrives, so apps can respond instantly instead of waiting until all data is collected before processing. ## Considered Options Kinesis has 4 major use-cases: streaming, firehose, analytics, and video. We have experience working with streams and firehose, but not the newer analytics and video offerings. **AWS Kinesis Data Streams** is for real-time data streaming. It can continuously capture gigabyte-scale data every second from multiple sources. It’s basically Amazon’s proprietary alternative to Kafka. We have terraform support for [https://github.com/cloudposse/terraform-aws-kinesis-stream](https://github.com/cloudposse/terraform-aws-kinesis-stream) **AWS Kinesis Data Firehose** allows loading data streams into AWS data stores. This is the simplest approach for capturing, transforming, and loading data streams into AWS-specific data stores like RedShift, S3, or ElasticSearch service. The service can automatically scale to handle gigabytes of data per second and supports batching, encryption, and streaming data compression. We have previously implemented data firehose for customer-specific applications, but do not have a generalized component for this. ### Option 1: AWS Kinesis Data Streams In order to provision the kinesis streams component, we’ll need to know more about how it will be used. | | | | | ----------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | --- | | **Name(s) of the Streams** | What are the names of the streams? or just provide some examples | | | **Region** | AWS Region for the cluster | | | **Number of Shards** | The number of shards to provision for the stream. | | | **Retention Period** | Length of time data records are accessible after they are added to the stream. The maximum value is 168 hours. Minimum value is 24. | | | **Shard Level Metrics** | A list of shard-level CloudWatch metrics to enabled for the stream. Options are IncomingBytes, OutgoingBytes | | | **Enforce Consumer Deletion** | Forcefully delete stream consumers before destroying the stream | | | **Encryption Type** | The encryption type to use. Acceptable values are `NONE` and `KMS` | | | **Steaming Mode** | The capacity mode of the stream. Must be either `PROVISIONED` or `ON_DEMAND`. | | ### Option 2: AWS Kinesis Data Firehose We’ll need more information about how it will be used to provision the firehose. Implementing the component will likely be highly custom to your use case. Standard use-cases are: - Extended S3 Destination with Lambdas - Extended S3 Destination with native support for shipping to S3 - Redshift - Elasticsearch - Splunk - HTTP Endpoint ## References - [https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/kinesis_stream](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/kinesis_stream) - [https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/kinesis_firehose_delivery_stream](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/reference/kinesis_firehose_delivery_stream) --- ## Decide on KMS Requirements import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; AWS Key Management Service (AWS KMS) makes it easy to create and manage cryptographic keys and control their use across various AWS services and in your applications. AWS KMS is a secure and resilient service that uses hardware security modules that have been validated under FIPS 140-2, or are in the process of being validated, to protect your keys. AWS KMS is integrated with AWS CloudTrail to provide you with logs of all key usage to help meet your regulatory and compliance needs. ## Context and Problem Statement Best practice, security certifications, and compliance regulations require that data be “encrypted at rest.” AWS provides options for encrypting data at rest virtually anywhere AWS provides for data storage. However, there remain considerations regarding managing the encryption _keys_ for the encrypted data. Consideration should begin with a review of governmental, industry, and corporate compliance requirements, standards, and goals regarding encryption at rest, auditability, data localization, data preservation and recovery, high availability, and disaster recovery. Cloud Posse does not provide advice on compliance requirements, it is up to the client to determine their own needs. For example, here is AWS' approach to HIPAA compliance, summarized: :::info There is no HIPAA certification for a cloud service provider (CSP) such as AWS. In order to meet the HIPAA requirements applicable to AWS' operating model, AWS aligns their HIPAA risk management program with FedRAMP and NIST 800-53, which are higher security standards that map to the HIPAA Security Rule. NIST supports this alignment and has issued [**SP 800-66 An Introductory Resource Guide for Implementing the HIPAA Security Rule**](http://csrc.nist.gov/publications/nistpubs/800-66-Rev1/SP-800-66-Revision1.pdf), which documents how NIST 800-53 aligns to the HIPAA Security Rule. ::: In other words, because of the lack of specificity in the HIPAA standard, AWS has decided to follow a stricter, more detailed standard and rely on a ruling that asserts that the stricter standard is sufficient to comply with HIPAA. The client organization needs to make such determinations for itself regarding its compliance goals. For clients operating outside of specialized regulatory environments (like PCI, HIPAA, or FedRamp), it can be helpful to keep in mind that encryption is not, in general, a required part of the access control system restricting access to data. In most cases, a malicious user who can gain access to the data via the API (by stealing or elevating credentials) will also gain access to the decryption service via the same mechanism. The primary value of encryption in the AWS environment is to enforce access via authorized APIs and prevent unauthorized or accidental access via some kind of bypass of the authorized APIs. For example, the logical model of a file stored in S3 is that there is a single file in a single place containing the data. In reality, the data is duplicated on as many as five (5) devices to provide high availability and protect against loss. If one of those devices is a standard magnetic disk that is replaced due to age or in anticipation of failure, and that device is carelessly discarded and ends up in the hands of a third party, that party would now have access to your data. If that data were, say, a CSV file of Personally Identifiable Information (PII), this would be a serious data breach because the third party could likely figure out what the data is and make use of it, despite the lack of context, just by reading the raw blocks from the drive. Encryption at rest protects against this and similar scenarios by ensuring that access such as this, which bypasses the APIs and takes advantage of the physical implementation of the logical model, will only yield encrypted and therefore useless data. Nevertheless, because access to data is difficult to control, some certifications focus on encryption plus access controls on the encryption keys to assure and monitor access to the data. The client’s auditors and compliance officers might prefer to audit and monitor use of encryption keys versus access to encrypted data. So the client must review their requirements and preferences to inform the choices regarding KMS usage. ### KMS Keys are Locked to a Region KMS keys reside in AWS-managed hardware security modules that are physically tied to a single region. Each key can only be used in the region in which it was created. This can, in some sense, enforce Data Localization requirements, although it remains possible to decrypt data in the region and then transport the decrypted data anywhere on the internet. :::info - **Data residency** refers to where a business, industry body or government specifies that their data is stored in a geographical location of their choice, usually for regulatory or policy reasons. - **Data sovereignty** refers to who has power over data. For example, data stored in Germany is subject to the laws of both German and the European Union. - **Data localization** is the most stringent and restrictive concept of the three, and like data sovereignty, is a version of data residency predicated on legal obligations. It requires that data created within certain borders stay within them. For example, if an organization has personal data about Russian citizens, Russia’s “On Personal Data” (OPD) law states that such data must reside in data centers or other facilities within the Russian Federation. ::: KMS _multi-region keys_ are actually a synchronized collection of single region keys, each one individually provisioned. The difference is that they have the same key material and same key ID, so they can be used interchangeably. Data encrypted by a multi-region key in one region can be transported to another region in encrypted form and then decrypted by the replica key in that region. This may sound attractive, but it is rarely needed in practice and comes with several issues. (See [Security considerations for multi-Region keys](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html#mrk-when-to-use).) Although a single region key cannot be converted into a multi-region key, the choice to use a multi-region key is generally one that can be deferred until a strong use case emerges for one, and even then can be used just for that specific use case ### Multiple Types of KMS Keys Amazon documents three (3) types of AWS KMS: - **Customer managed key** – (Also called Customer Master Key, CMK, AWS KMS key, and KMS key.) Customers create and control the lifecycle and key policies of customer managed keys. All requests made against these keys are logged as CloudTrail events. - **AWS managed keys** – (Also referred to as Default Keys.) AWS creates and controls the lifecycle and key policies of AWS managed keys, which are resources in a customer’s AWS account. Customers can view access policies and CloudTrail events for AWS managed keys, but cannot manage any aspect of these keys. All requests made against these keys are logged as CloudTrail events. - **AWS owned keys** – These keys are created and exclusively used by AWS for internal encryption operations across different AWS services. Customers do not have visibility into key policies or AWS owned key usage in CloudTrail. As documented, AWS owned keys are practically invisible to customers, so your only real choice is between AWS managed keys and customer managed keys. With Customer managed keys, you also have the some control over the key type (symmetric or asymmetric), cipher, and how the key is generated, but Cloud Posse recommends using the defaults (symmetric AES-256 generated by KMS). ### Key Identifiers KMS keys can have a few different [identifiers](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#key-id), usually called `KeyId`. When created, they are immediately given a _Key ID_ (not what is meant by `KeyId`) which uniquely identifies a KMS key within an account and Region. That Key ID, plus account and region info, constitutes a _Key ARN_, a unique, fully qualified identifier for the KMS key. Additionally, you can create an _Alias Name_ (actually, you can create several) that provides a friendly name for the key. Furthermore, that Alias Name can be associated with different keys at different points in time. More importantly, while you cannot control the Key ID when you create a customer managed key, you can control the value of the Alias Name for the key. This allows you to have the same Alias in every account and region, which then allows you to have the same configuration (referencing the Alias rather than the Key ID or ARN) in every account and every region even though they all point to different keys. Like a Key ID, an Alias can be used to constitute an Alias ARN, which is also a unique, fully qualified identifier for the alias, and for the KMS key, it represents. AWS Managed keys are created automatically and are tied to a specific service, such as S3 or SSM, and are usually used by default when you specify encryption but do not specify a KMS key. They all have Alias Names beginning with `aws/` and are named according to the service or usage. For example `aws/s3` is used to encrypt S3 bucket data, while `aws/ebs` is used to encrypt EBS volumes. ### Cost of KMS KMS is not free. Each AWS KMS key you create costs $1/month (prorated hourly). The $1/month charge is the same for symmetric keys, asymmetric keys, HMAC keys, each multi-Region key (each primary and each replica multi-region key), keys with imported key material, and keys in custom key stores. If you enable automatic key rotation, each newly generated backing key costs an additional $1/month (prorated hourly). This covers the cost to AWS KMS of retaining all versions of the key material so they can be used to decrypt older ciphertexts. Each API request to the AWS Key Management Service (outside of the free tier) costs. [https://aws.amazon.com/kms/pricing/](https://aws.amazon.com/kms/pricing/) :::caution **S3 Encryption** Every encrypted object in an S3 bucket is encrypted with a different data key. The default/legacy method of S3 encryption uses KMS to generate each data key. For buckets with a large number of objects and/or lots of traffic, this can create a burdensome amount of KMS activity. To reduce the impact on KMS (latency, cost of operations), AWS introduced the [Bucket Keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html#using-bucket-key) feature. The documentation is a bit unclear on the topic, but it appears that using a bucket key allows S3 to create the data keys directly, without going to the KMS service, speeding up the creation of new objects. It is not clear if it also speeds up decryption operations. However, as long as you are comfortable with using a single KMS key to encrypt all the objects in a single S3 bucket, using a Bucket Key will at best be a significant win (99% reduction in KMS operations) and at worst be no better than not, so **we recommend using S3 Bucket Keys**. ::: ## Considered Options ### Option 1: Use AWS Managed Keys (Recommended where applicable) :::tip The simplest solution is to use encryption keys as defense in depth, without attempting to leverage keys as additional access controls or monitoring opportunities. ::: Use default keys by alias (e.g. `aws/s3`, `aws/ssm`) everywhere possible. Create a single key per account, per region, with the same alias, e.g. `default_key`, for services that do not create their own default keys, such as EKS. Transfer encrypted data across accounts and regions by decrypting in the source region and re-encrypting under new keys in the target region. Use aliases wherever possible to reduce confusion. #### Pros - Default keys are automatically created by AWS with appropriate key policies and are automatically managed and rotated by AWS. - Per-region, per-account keys provide full isolation for high-availability and redundancy. #### Cons - Default keys are created on demand, and Terraform cannot reference them before they are created. - Default keys can only be used by the service they were created for. You cannot alter the key policies for the default keys to allow them to be used elsewhere. - Not every service has a default key, so, you may still need to create at least one customer managed key, and it has to be created exactly once in some component before any other component needs it. - In some situations, a configuration may end up transferring encrypted data to a new region that will not have access to the key to decrypt it. ### Option 2: Use Customer Managed Keys :::tip If it is desired to use access control on encryption keys to limit access to data, use a key per category of data, identified by alias, per account and region. ::: - Create keys for each regulation or category of protected information, such as PCI, PHI, PII, assigning aliases based on intended usage. - Create an additional `default_key` for use for encryption of data not requiring special handling. - Use alias in configuration wherever possible. - Use separate keys per region and per account, but use the same alias everywhere for the same purpose. - Use access control on the keys to keep the data from being transferred to other accounts (or regions, if necessary). #### Pros - Easier to audit, monitor, and manage access to protected information - Explicit creation of keys allows explicit creation of key policies and grants, providing an easy-to-view/review/manage point of control - Immediately and permanently revoke access to all data simply by deleting the key. See [Deleting AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/deleting-keys.html). #### Cons - No clear, intuitive way to handle mixed data. For example, what encryption key do you use for a database that holds both PHI and PCI data? Or do you use two (2) separate databases so you can separate the data by category/KMS key? - Extra work to configure and deploy keys outside of components where they will be used. May end up creating keys that are never used. - Immediately and permanently lose access to all data if the KMS key is deleted. See [Deleting AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/deleting-keys.html). ### Option 3: Use a Customer Managed Key per Workload Key per workload Multi-region keys #### Pros - Key is managed by same component as the one using it, no order-of-operations issues in Terraform - Data encrypted in one region can be decrypted in a different region (whether this is pro or con is a matter of perspective) #### Cons - No clear justification for treating secrets in S3 different from secrets in a database or EKS cluster. - No audit trail or precise control over data by regulatory or compliance category. - Controlling access and enforcing data security policy is more complex with multi-Region keys. You must ensure that policy is audited consistently on key across multiple, isolated regions. And you need to use policy to enforce boundaries instead of relying on separate keys. For example, you need to set policy conditions on data to prevent payroll teams in one Region from being able to read payroll data for a different Region. Also, you must use access control to prevent a scenario where a multi-Region key in one Region protects one tenant's data and a related multi-Region key in another Region protects a different tenant's data. - Auditing keys across Regions is also more complex. With multi-Region keys, you need to examine and reconcile audit activities across multiple Regions to understand key activities on protected data. - Compliance with data residency mandates can be more complex. With isolated Regions, you can ensure data residency and data sovereignty compliance. KMS keys in a given Region can decrypt sensitive data only in that Region. Data encrypted in one Region can remain completely protected and inaccessible in any other Region. To verify data residency and data sovereignty with multi-Region keys, you must implement access policies and compile AWS CloudTrail events across multiple Regions. - AWS Services that use KMS keys do not take advantage of multi-Region keys. They still decrypt data in one region, use encrypted communication to move it to another region, and re-encrypt the data in the new region, even though the keys are the same. ## References - [AWS KMS FAQs](https://aws.amazon.com/kms/faqs/) - [AWS Key Management Service details](https://docs.aws.amazon.com/kms/latest/cryptographic-details/intro.html) - [Does data localization cause more problems than it solves?](https://d1.awsstatic.com/institute/AWS-Sovereignty-and-Data-Localization-2022.pdf) - [AWS CloudHSM](https://aws.amazon.com/cloudhsm/) - [Security considerations for multi-Region keys](https://docs.aws.amazon.com/kms/latest/developerguide/multi-region-keys-overview.html#mrk-when-to-use) - [KMS Key IDs](https://docs.aws.amazon.com/kms/latest/developerguide/concepts.html#key-id) - [Reducing the cost of SSE-KMS with Amazon S3 Bucket Keys](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html) - [Deleting AWS KMS keys](https://docs.aws.amazon.com/kms/latest/developerguide/deleting-keys.html) --- ## Decide on Transactional Email (SMTP) Provider for Operational Emails import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; ## Problem We’ll want to have an SMTP email gateway that’s used by services like Alert Manager, Sentry, Printunl, et al to send emails. ## Considered Options #### AWS SES Our recommended option is to provision an SES gateway using our [https://github.com/cloudposse/terraform-aws-ses](https://github.com/cloudposse/terraform-aws-ses) module for this. It’s also worth noting is that every AWS SES starts in Sandbox. Sending emails via it (emails not verified in AWS SES) is only allowed after support requests (so requires some human intervention to automate). SES is also only available in the following regions: - `us-east-1` - `us-east-2` - `us-west-2` - `ap-south-1` - `ap-southeast-2` - `eu-west-1` - `eu-west-2` - `sa-east-1` This will use the vanity branded domain in [Decide on Vanity (Branded) Domain](/layers/network/design-decisions/decide-on-vanity-branded-domain) if SES is used for customer-facing emails. If these are for internal domains then we can use the service discovery domains. #### Mailgun The other option we typically recommend is mailgun. It’s very economical and easy to integrate with. There’s also a terraform provider for managing mailgun configurations. #### Bring-your-own We’ll happily integrate with whatever SMTP system your company uses today. :::caution Gmail is not a good option because they will rate limit sending. ::: --- ## Design Decisions(10) import Intro from "@site/src/components/Intro"; import KeyPoints from "@site/src/components/KeyPoints"; import DocCardList from "@theme/DocCardList"; Design Decisions are architectural considerations for how to approach or implement some sort of functionality. The decision outcomes should be documented as Architectural Design Records (ADRs). See [how to document a new design decision](/learn/maintenance/tutorials/how-to-document-a-new-design-decision) to this reference architecture as well as [How to write ADRs](/learn/maintenance/tutorials/how-to-write-adrs) . :::note This is our entire reference catalog of Design Decisions and not all may be relevant to the scope of a particular engagement. ::: ## All Decisions They are broken down in the following way: --- ## Atmos `atmos` is both a command-line tool and Golang module for provisioning, managing and orchestrating workflows across various toolchains including `terraform` and `helmfile`. The `atmos` tool is part of the SweetOps toolchain and was built to make DevOps and Cloud automation easier across multiple tools. It has direct support for automating Terraform, Helmfile. By utilizing [Stacks](/resources/legacy/fundamentals/stacks), `atmos` enable you to effortlessly manage your Terraform and Helmfile [Components](/components) from your local machine, in your CI/CD pipelines, or using [spacelift](//components/library/aws/spacelift/). ## Problem A modern infrastructure depends on lots of various tools like terraform, packer, helmfile, helm, kubectl, docker, etc. All these tools have varying degrees of configuration support, but most are not optimized for defining DRY configurations across dozens or hundreds of environments. Moreover, the configuration format is very different between the tools, but usually, boils down to some kind of key-value configuration in either JSON or YAML. This lack of configuration consistency poses a problem when we want to make it easy to declaratively define the settings that end-users should care about. ## Solution We defined a “universal” configuration format that works for all the tools we use. When using terarform, helmfile, etc we design our components as reusable building blocks that accept simple declarative parameters and offload all business logic to the tools themselves. [We designed this configuration schema in YAML](https://learningactors.com/what-is-infrastructure-as-code-automating-your-infrastructure-builds/#:~:text=Infrastructure%20as%20code%20defined,a%20vastly%20larger%20scale.) and added convenient and robust deep-merging strategies that allow configurations to extend to other configurations. As part of this, we support OOP concepts of mixins, inheritance, and multiple inheritances - but all applied to the configuration. We support YAML anchors to clean up complex blocks of configuration, folder structures, environment variables, and all kinds of tool-specific settings. ## Alternatives There are a number of alternative tools to atmos, that accomplish some aspect of it. | **Tool** | **Description** | **Website** | | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | terragrunt | | [https://github.com/gruntwork-io/terragrunt](https://github.com/gruntwork-io/terragrunt) | | astro | | [https://github.com/uber/astro](https://github.com/uber/astro) | | terraspace | | [https://github.com/boltops-tools/terraspace](https://github.com/boltops-tools/terraspace) | | leverage | The Leverage CLI intended to orchestrate Leverage Reference Architecture for AWS | [https://github.com/binbashar/leverage](https://github.com/binbashar/leverage) | | opta | The next generation of Infrastructure-as-Code. Work with high-level constructs instead of getting lost in low-level cloud configuration | [https://github.com/run-x/opta](https://github.com/run-x/opta)[https://docs.opta.dev/](https://docs.opta.dev/) | | pterradactyl | Pterradactyl is a library developed to abstract Terraform configuration from the Terraform environment setup. | [https://github.com/nike-inc/pterradactyl](https://github.com/nike-inc/pterradactyl) | | terramate | Terramate is a tool for managing multiple terraform stacks | [https://github.com/mineiros-io/terramate](https://github.com/mineiros-io/terramate) | | `make` (honorable mention) | Many companies (including cloudposse) start by leveraging `make` with `Makefile` and targets to call terraform. This is a tried and true way, but at the scale we help our customer operate didn’t work. We know, because we tried it for ~3 years and suffocated under the weight of environment variables and stink of complexity only a mother could love. | [https://www.gnu.org/software/make/](https://www.gnu.org/software/make/) | What `atmos` is not: - An alternative to chef, puppet, or ansible. Instead, `atmos` is the type of tool that would call these tools. - An alternative to CI or CD systems. If anything, those systems will call `atmos`. ## Design Considerations - Keep it strictly declarative (no concept of iterators or interpolations) - Offload all imperative design to the underlying tools - Do not write a programming language in YAML (e.g. CloudFormation) or JSON (e.g. terraform or JSONNET, KSONNET) - Do not use any esoteric expressions (e.g. JSONNET) - Keep it Simple Stupid (KISS) - Ensure compatibility with multiple tools, not just `terraform` - Define all configuration in files and not based on filesystem conventions. ## Usage `atmos help` (actually, we still need to implement this 😵 after porting to golang) :::info **IMPORTANT** Atmos underwent a complete rewrite from an esoteric task runner framework called `variant2` into native Golang as of version 1.0. The documentation is not updated everywhere. The interface is identical/backward compatible (and enhanced), but some references to `variant2` are inaccurate. You can assume this documentation is for the latest version of atmos. ::: Subcommands are positional arguments passed to the `atmos` command. ### Subcommand: `version` Show the current version ### Subcommand: `describe` Show the deep-merged configuration for stacks and components. ### Subcommand: `terraform` - Supports all built-in [Terraform Subcommands](https://www.terraform.io/docs/cli/commands/index.html) (we essentially pass them through to the `terraform` command) - `deploy` is equivalent to `atmos terraform apply -auto-approve` - `generate backend` is used to generate the static `backend.tf.json` file that should be committed to VCS - `generate varfile` (deprecated command: `write varfile`) — This command generates a varfile for a terraform component: `atmos terraform generate varfile -s -f ` - `clean` deletes any orphaned varfiles or planfiles ### Subcommand: `helmfile` - Supports all `helmfile` subcommands - `describe` - `generate varfile` — This command generates a varfile for a helmfile component: `atmos helmfile generate varfile -s -f ` ### Subcommand: `workflow` This subcommand is temporarily unavailable as a result of a major refactor from variant2 to golang. We will reintroduce the subcommand and it **has not** been _officially_ deprecated. [https://github.com/cloudposse/atmos](https://github.com/cloudposse/atmos) **Latest Releases** [https://github.com/cloudposse/atmos/releases](https://github.com/cloudposse/atmos/releases) **Open Issues** [https://github.com/cloudposse/atmos/issues](https://github.com/cloudposse/atmos/issues) ## Examples ### Provision Terraform Component To provision a Terraform component using the `atmos` CLI, run the following commands in the `geodesic` container shell: ``` atmos terraform plan eks --stack=ue2-dev atmos terraform apply eks --stack=ue2-dev ``` Where: - `eks` is the Terraform component to provision (from the `components/terraform` folder) that is defined in the stack. If the component is not defined in the stack, it will error. - `--stack=ue2-dev` is the stack to provision the component into (or in other words, where to read the configuration) :::info You can pass _any_ argument supported by `terraform` and it will be passed through to the system call to `terraform`. e.g. We can pass the `-destroy` flag to `terraform plan` by running `atmos terraform plan -destroy --stack=uw2-dev` ::: Short versions of the command-line arguments can also be used: ``` atmos terraform plan eks -s ue2-dev atmos terraform apply eks -s ue2-dev ``` To execute `plan` and `apply` in one step, use `terrafrom deploy` command: ``` atmos terraform deploy eks -s ue2-dev ``` ### Provision Terraform Component with Planfile You can use a terraform `planfile` (previously generated with `atmos terraform plan`) in `atmos terraform apply/deploy` commands by running the following: ``` atmos terraform plan test/test-component-override -s tenant1/ue2/dev atmos terraform apply test/test-component-override -s tenant1-ue2-dev --from-plan atmos terraform deploy test/test-component-override -s tenant1-ue2-dev --from-plan ``` ### Provision Helmfile Component To provision a helmfile component using the `atmos` CLI, run the following commands in the container shell: ``` atmos helmfile diff nginx-ingress --stack=ue2-dev atmos helmfile apply nginx-ingress --stack=ue2-dev ``` Where: - `nginx-ingress` is the helmfile component to provision (from the `components/helmfile` folder) - `--stack=ue2-dev` is the stack to provision the component into Short versions of the command-line arguments can be used: ``` atmos helmfile diff nginx-ingress -s ue2-dev atmos helmfile apply nginx-ingress -s ue2-dev ``` To execute `diff` and `apply` in one step, use `helmfile deploy` command: ``` atmos helmfile deploy nginx-ingress -s ue2-dev ``` ### View Deep-merged CLI Configs Use `atmos describe config` command to show the effective CLI configuration. Use `--format` of `json` or `yaml` to alter the output to structured data. The deep-merge processes files from these locations: - system dir (`/usr/local/etc/atmos/atmos.yaml` on Linux, `%LOCALAPPDATA%/atmos/atmos.yaml` on Windows) - home dir (`~/.atmos/atmos.yaml`) - `atmos.yaml` in the current directory Here are some more examples: ``` atmos describe config -help atmos describe config atmos describe config --format=json atmos describe config --format json atmos describe config -f=json atmos describe config -f json atmos describe config --format=yaml atmos describe config --format yaml atmos describe config -f=yaml atmos describe config -f yaml ``` ### Example Commands ``` atmos version atmos describe config # Describe components and stacks atmos describe component -s atmos describe component --stack # Generate atmos terraform generate backend -s atmos terraform write varfile -s # this command will be changed to `terraform generate varfile` atmos terraform write varfile -s -f ./varfile.json # supports output file # Terraform # (almost) all native Terraform commands supported # https://www.terraform.io/docs/cli/commands/index.html atmos terraform plan -s atmos terraform apply -s -auto-approve atmos terraform apply -s --from-plan atmos terraform deploy -s atmos terraform deploy -s --from-plan atmos terraform deploy -s -deploy-run-init=true atmos terraform workspace -s atmos terraform validate -s atmos terraform output -s atmos terraform graph -s atmos terraform show -s atmos terraform clean -s # Helmfile # All native helmfile commands supported including [global options] # https://github.com/roboll/helmfile#cli-reference atmos helmfile diff -s atmos helmfile apply -s # Helmfile with [global options] atmos helmfile diff -s --global-options "--no-color --namespace=test" atmos helmfile diff -s --global-options="--no-color --namespace test" ``` ### Workflows :::danger **IMPORTANT** This is in atmos 0.x and while this functionality has not been deprecated, it also **has not** been ported over to atmos 1.x yet. ::: Workflows are a way of combining multiple commands into one executable unit of work, kind of like a basic task-runner. In the CLI, workflows can be defined using two different methods: - In the configuration file for a stack (see [workflows in dev/us-east-2.yaml](https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/orgs/cp/tenant1/dev/us-east-2.yaml) for an example) - In a separate file (see [workflows.yaml](https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/workflows/workflow1.yaml) In the first case, we define workflows in the configuration file for the stack (which we specify on the command line). To execute the workflows from [workflows in dev/us-east-2.yaml](https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/orgs/cp/tenant1/dev/us-east-2.yaml), run the following commands: ``` atmos workflow deploy-all -s ue2-dev ``` Note that workflows defined in the stack config files can be executed only for the particular stack (environment and stage). It's not possible to provision resources for multiple stacks this way. In the second case (defining workflows in a separate file), a single workflow can be created to provision resources into different stacks. The stacks for the workflow steps can be specified in the workflow config. For example, to run `terraform plan` and `helmfile diff` on all terraform and helmfile components in the example, execute the following command: ``` atmos workflow plan-all -f workflows ``` where the command-line option `-f` (`--file` for long version) instructs the `atmos` CLI to look for the `plan-all` workflow in the file [workflows](https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/workflows/workflow1.yaml). As we can see, in multi-environment workflows, each workflow job specifies the stack it's operating on: ``` workflows: plan-all: description: Run 'terraform plan' and 'helmfile diff' on all components for all stacks steps: - job: terraform plan vpc stack: ue2-dev - job: terraform plan eks stack: ue2-dev - job: helmfile diff nginx-ingress stack: ue2-dev - job: terraform plan vpc stack: ue2-staging - job: terraform plan eks stack: ue2-staging ``` You can also define a workflow in a separate file without specifying the stack in the workflow's job config. In this case, the stack needs to be provided on the command line. For example, to run the `deploy-all` workflow from the [workflows](https://github.com/cloudposse/atmos/blob/master/examples/complete/stacks/workflows/workflow1.yaml) file for the `ue2-dev` stack, execute the following command: ``` atmos workflow deploy-all -f workflows -s ue2-dev ``` ## Recommended Filesystem Layout :::info For an example of what this looks like within [Geodesic](/resources/legacy/fundamentals/geodesic) see the section on “Filesystem Layout” ::: Our general recommended filesystem layout looks like this. It can be customized using the CLI Configuration file. ``` # Your infratructure repository infrastructure/ │ │ # Centralized components configuration ├── stacks/ │ │ └── catalog/ │ │ │ └── $stack.yaml │ │ # Components are broken down by tool ├── components/ │ │ │ ├── terraform/ # root modules in here │ │ ├── vpc/ │ │ ├── eks/ │ │ ├── rds/ │ │ ├── iam/ │ │ ├── dns/ │ │ └── sso/ │ │ │ └── helmfile/ # helmfiles are organized by chart │ ├── cert-manager/helmfile.yaml │ └── external-dns/helmfile.yaml │ │ # Makefile for building the CLI ├── Makefile │ │ # Docker image for shipping the CLI and all dependencies └── Dockerfile (optional) ``` ## CLI Configuration Atmos supports a CLI configuration to define configure the behavior working with stacks and components. In [Geodesic](/resources/legacy/fundamentals/geodesic) we typically put this in `/usr/local/etc/atmos/atmos.yaml` (e.g. in `rootfs/...` in the `infrastructure` repository). Note this file uses the stack config format for consistency, but we do not consider it a stack configuration. The CLI config is loaded from the following locations (from lowest to highest priority): - system dir (`/usr/local/etc/atmos` on Linux, `%LOCALAPPDATA%/atmos` on Windows) - home dir (`~/.atmos`) - current directory (`./`) - ENV vars - Command-line arguments It supports [POSIX-style Globs for file names/paths](https://en.wikipedia.org/wiki/Glob_(programming)) (double-star `**` is supported) ### Environment Variables Most YAML settings can be defined also as environment variables. This is helpful while doing local development. For example, setting `ATMOS_STACKS_BASE_PATH` to a path in `/localhost` to your local development folder, will enable you to rapidly iterate. | **Variable** | **YAML Path** | **Description** | | ---------------------------------------------------- | ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ | | `ATMOS_COMPONENTS_TERRAFORM_BASE_PATH` | `components.terraform.base_path` | | | `ATMOS_COMPONENTS_TERRAFORM_APPLY_AUTO_APPROVE` | `components.terraform.apply_auto_approve` | | | `ATMOS_COMPONENTS_TERRAFORM_DEPLOY_RUN_INIT` | `components.terraform.deploy_run_init` | | | `ATMOS_COMPONENTS_HELMFILE_BASE_PATH` | `components.helmfile.base_path` | | | `ATMOS_COMPONENTS_HELMFILE_KUBECONFIG_PATH` | `components.helmfile.aws_profile_pattern` | | | `ATMOS_COMPONENTS_HELMFILE_HELM_AWS_PROFILE_PATTERN` | `components.helmfile.helm_aws_profile_pattern` | | | `ATMOS_COMPONENTS_HELMFILE_CLUSTER_NAME_PATTERN` | `components.helmfile.cluster_name_pattern` | | | `ATMOS_STACKS_BASE_PATH` | `stacks.base_path` | | | `ATMOS_STACKS_INCLUDED_PATHS` | `stacks.included_paths` | | | `ATMOS_STACKS_EXCLUDED_PATHS` | `stacks.excluded_paths` | | | `ATMOS_STACKS_NAME_PATTERN` | `stacks.name_pattern` | | | `ATMOS_LOGS_VERBOSE` | | For more verbose output, set this environment variable to `true` to see the logs how the CLI finds the configs and performs merges. | ### Example `atmos.yaml` Configuration File (see: [https://github.com/cloudposse/atmos/blob/master/examples/complete/atmos.yaml#L30](https://github.com/cloudposse/atmos/blob/master/examples/complete/atmos.yaml#L30)) ``` components: # Settings for all terraform components terraform: # Can also be set using `ATMOS_COMPONENTS_TERRAFORM_BASE_PATH` ENV var, or `--terraform-dir` command-line argument # Supports both absolute and relative paths base_path: "/atmos_root/components/terraform" # Can also be set using `ATMOS_COMPONENTS_TERRAFORM_APPLY_AUTO_APPROVE` ENV var apply_auto_approve: false # Can also be set using `ATMOS_COMPONENTS_TERRAFORM_DEPLOY_RUN_INIT` ENV var, or `--deploy-run-init` command-line argument deploy_run_init: true # Can also be set using `ATMOS_COMPONENTS_TERRAFORM_AUTO_GENERATE_BACKEND_FILE` ENV var, or `--auto-generate-backend-file` command-line argument auto_generate_backend_file: false # Settings for all helmfile components helmfile: # Can also be set using `ATMOS_COMPONENTS_HELMFILE_BASE_PATH` ENV var, or `--helmfile-dir` command-line argument # Supports both absolute and relative paths base_path: "/atmos_root/components/helmfile" # Can also be set using `ATMOS_COMPONENTS_HELMFILE_KUBECONFIG_PATH` ENV var kubeconfig_path: "/dev/shm" # Can also be set using `ATMOS_COMPONENTS_HELMFILE_HELM_AWS_PROFILE_PATTERN` ENV var helm_aws_profile_pattern: "{namespace}-{tenant}-gbl-{stage}-helm" # Can also be set using `ATMOS_COMPONENTS_HELMFILE_CLUSTER_NAME_PATTERN` ENV var cluster_name_pattern: "{namespace}-{tenant}-{environment}-{stage}-eks-cluster" # Settings for all stacks stacks: # Can also be set using `ATMOS_STACKS_BASE_PATH` ENV var, or `--config-dir` and `--stacks-dir` command-line arguments # Supports both absolute and relative paths base_path: "/atmos_root/stacks" # Can also be set using `ATMOS_STACKS_INCLUDED_PATHS` ENV var (comma-separated values string) included_paths: - "**/*" # Can also be set using `ATMOS_STACKS_EXCLUDED_PATHS` ENV var (comma-separated values string) excluded_paths: - "globals/**/*" - "catalog/**/*" - "**/*globals*" # Can also be set using `ATMOS_STACKS_NAME_PATTERN` ENV var name_pattern: "{tenant}-{environment}-{stage}" logs: verbose: false colors: true ``` ## Troubleshooting :::info For more verbose output, you can always set the environment variable `ATMOS_LOGS_VERBOSE=true` to see the logs how the CLI finds the configs and performs merges. ::: ### **Error:** `stack name pattern must be provided in 'stacks.name_pattern' config or 'ATMOS_STACKS_NAME_PATTERN' ENV variable` This means that you are probably missing a section like this in your `atmos.yml`. See the instructions on CLI Configuration for more details. ``` stacks: name_pattern: "{tenant}-{environment}-{stage}" ``` ### **Error:** `The stack name pattern '{tenant}-{environment}-{stage}' specifies 'tenant`, but the stack ue1-prod does not have a tenant defined` This means that your `name_pattern` declares a `tenant` is required, but not specified. Either specify a `tenant` in your `vars` for the stack configuration, or remove the `{tenant}` from the `name_pattern` ``` stacks: name_pattern: "{tenant}-{environment}-{stage}" ``` ## How-to Guides - [How to Upgrade Atmos](/learn/maintenance/upgrades/how-to-upgrade-atmos) - [How to use Atmos](/learn/maintenance/tutorials/how-to-use-atmos) ## Concepts - [Stacks](/resources/legacy/fundamentals/stacks) - [Components](/components) --- ## Building Blocks The SweetOps solution by Cloud Posse is a powerful framework for DevOps, designed with multiple building blocks that offer various levels of abstraction. This modular approach allows for complete customization to meet the unique requirements of any business, ensuring a tailored solution that aligns with specific needs and objectives. To understand how it works, it's important to know how these parts work together: [Geodesic](/resources/legacy/tutorials/geodesic-getting-started), [Atmos](https://atmos.tools), [Terraform Components](/components/) (root modules), [Terraform Modules](/modules/), and [GitHub Actions](/github-actions/). ## Geodesic **Your DevOps Toolbox** [Geodesic](/resources/legacy/tutorials/geodesic-getting-started) is the ultimate DevOps toolbox, encapsulated within a Docker image. It eliminates the need for individual installations of Terraform, Kubernetes, Helmfile, AWS CLI, and other core tools in the SweetOps methodology. Geodesic is versatile, serving as an interactive cloud automation shell, a base image for other containers, or a crucial element in CI/CD workflows. ## Atmos **Your Workflow Automation Conductor** [Atmos](https://atmos.tools) takes the reins as a DevOps workflow automation tool, adept at managing large-scale and intricate infrastructures. It empowers users to manage DRY configurations hierarchically, foster team collaboration, and enforce guardrails through policies as code. Atmos relies on YAML configuration and supports multiple inheritance, simplifying the process of defining and reusing logical groups of configuration. We usually run Atmos inside of Geodesic, but it's not a requirement. ## Terraform Components **Blueprints for Infrastructure Patterns** [Terraform Components](/components/) (Root Modules) act as the foundational Terraform configurations for high-level infrastructure elements, such as Kubernetes clusters or VPCs. These root modules enable reusability across different environments, and frequently leverage Terraform child modules, streamlining infrastructure management. ## Terraform Modules **Bricks of Infrastructure Business Logic** [Terraform Modules](/modules/) are smaller building blocks of infrastructure that are invoked by components to dictate the business logic associated with configuring infrastructure resources. These versatile modules can be shared among multiple Terraform configurations, allowing for the creation, updating, and deletion of infrastructure resources. ## GitHub Actions **Your Deployment Automation Platform** [GitHub Actions](/github-actions/) play a pivotal role as the CI/CD tool, automating software workflows to build, test, and deploy code changes. This powerful platform integrates seamlessly with Geodesic, Atmos and Terraform components and modules, automating deployment to infrastructure environments. ## TL;DR The various building blocks we leverage are [Geodesic](/resources/legacy/tutorials/geodesic-getting-started) as a comprehensive DevOps toolbox, [Atmos](https://atmos.tools) as a workflow automation conductor, [Terraform components](/components/) and [modules](/modules/) as the architects of infrastructure elements and resources, and [GitHub Actions](/github-actions/) as the deployment automation platform. This unified approach empowers teams to efficiently manage and scale their cloud infrastructure. --- ## Concepts import ReactPlayer from 'react-player' # Concepts SweetOps is built on top of a number of high-level concepts and terminology that are critical to understanding prior to getting started. In this document, we break down these concepts to help you better understand our conventions as we introduce them. ### Components [Components](/components) are opinionated, self-contained units of infrastructure as code that solve one, specific problem or use-case. SweetOps has two flavors of components: 1. **Terraform:** Stand-alone root modules that implement some piece of your infrastructure. For example, typical components might be an EKS cluster, RDS cluster, EFS filesystem, S3 bucket, DynamoDB table, etc. You can find the [full library of SweetOps Terraform components here](/components/). We keep these types of components in the `components/terraform/` directory within the infrastructure repository. 2. **Helmfiles**: Stand-alone, applications deployed using `helmfile` to Kubernetes. For example, typical helmfiles might deploy the DataDog agent, cert-manager controller, nginx-ingress controller, etc. Similarly, the [full library of SweetOps Helmfile components is on GitHub](https://github.com/cloudposse/helmfiles). We keep these types of components in the `components/helmfile/` directory within the infrastructure repository. One important distinction about components that is worth noting: components are opinionated “root” modules that typically call other child modules. Components are the building blocks of your infrastructure. This is where you define all the business logic for how to provision some common piece of infrastructure like ECR repos (with the [ecr](/components/library/aws/ecr/) component) or EKS clusters (with the [eks/cluster](/components/library/aws/eks/cluster/) component). Our convention is to stick components in the `components/terraform` directory and to use a `modules/` subfolder to provide child modules intended to be called by the components. :::caution We do not recommend consuming one terraform component inside of another as that would defeat the purpose; each component is intended to be a loosely coupled unit of IaC with its own lifecycle. Further more, since components define a state backend, it’s not supported in terraform to call it from other modules. ::: ### Stacks Stacks are a way to express the complete infrastructure needed for an environment using a standard YAML configuration format that has been developed by Cloud Posse. Stacks consist of components and the variables inputs into those components. For example, you configure a stack for each AWS account and then reference the components which comprise that stack. The more modular the components, the easier it is to quickly define a stack without writing any new code. Here is an example stack defined for a Dev environment in the us-west-2 region: ```yaml # Filename: stacks/uw2-dev.yaml import: - eks/eks-defaults vars: stage: dev terraform: vars: {} helmfile: vars: account_number: "1234567890" components: terraform: dns-delegated: vars: request_acm_certificate: true zone_config: - subdomain: dev zone_name: example.com vpc: vars: cidr_block: "10.122.0.0/18" eks: vars: cluster_kubernetes_version: "1.19" region_availability_zones: ["us-west-2b", "us-west-2c", "us-west-1d"] public_access_cidrs: ["72.107.0.0/24"] aurora-postgres: vars: instance_type: db.r4.large cluster_size: 2 mq-broker: vars: apply_immediately: true auto_minor_version_upgrade: true deployment_mode: "ACTIVE_STANDBY_MULTI_AZ" engine_type: "ActiveMQ" helmfile: external-dns: vars: installed: true datadog: vars: installed: true datadogTags: - "env:uw2-dev" - "region:us-west-2" - "stage:dev" ``` Great, so what can you do with a stack? Stacks are meant to be a language and tool agnostic way to describe infrastructure, but how to use the stack configuration is up to you. We provide the following ways to utilize stacks today: 1. [Atmos](https://atmos.tools): Atmos is a command-line tool that enables CLI-driven stack utilization and supports workflows around `terraform`, `helmfile`, and many other commands 2. [`terraform-provider-utils`](https://github.com/cloudposse/terraform-provider-utils): is our Terraform provider for consuming stack configurations from within HCL/Terraform. 3. [Spacelift](https://spacelift.io/): By using the [terraform-spacelift-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) you can configure Spacelift continuously deliver components. Read up on why we [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) . ### Catalogs Catalogs in SweetOps are collections of sharable and reusable configurations. Think of the configurations in catalogs as defining archetypes (a very typical example of a certain thing) of configuration (E.g. `s3/public` and `s3/logs` would be two kinds of archetypes of S3 buckets). They are also convenient for managing [Terraform](/resources/legacy/fundamentals/terraform). These are typically YAML configurations that can be imported and provide solid baselines to configure security, monitoring, or other 3rd party tooling. Catalogs enable an organization to codify its best practices of configuration and share them. We use this pattern both with our public terraform modules as well as with our stack configurations (e.g. in the `stacks/catalog` folder). SweetOps provides many examples of how to use the catalog pattern to get you started. Today SweetOps provides a couple important catalogs: 1. [DataDog Monitors](https://github.com/cloudposse/terraform-datadog-monitor/tree/master/catalog/monitors): Quickly bootstrap your SRE efforts by utilizing some of these best practice DataDog application monitors. 2. [AWS Config Rules](https://github.com/cloudposse/terraform-aws-config/tree/master/catalog): Quickly bootstrap your AWS compliance efforts by utilizing hundreds of [AWS Config](https://aws.amazon.com/config/) rules that automate security checks against many common services. 3. [AWS Service Control Policies](https://github.com/cloudposse/terraform-aws-service-control-policies/tree/master/catalog): define what permissions in your organization you want to permit or deny in member accounts. In the future, you’re likely to see additional open-source catalogs for OPA rules and tools to make sharing configurations even easier. But it is important to note that how you use catalogs is really up to you to define, and the best catalogs will be specific to your organization. ### Collections Collections are groups of stacks. ### Segments Segments are interconnected networks. For example, a production segment connects all production-tier stacks, while a non-production segment connects all non-production stacks. ### Primary vs Delegated Primary vs Delegated is an implementation pattern in SweetOps. This is most easily described when looking at the example of domain and DNS usage in a mutli-account AWS organization: SweetOps takes the approach that the root domain (e.g. `example.com`) is owned by a **primary** AWS account where the apex zone resides. Subdomains on that domain (e.g. `dev.example.com`) are then **delegated** to the other AWS accounts via an `NS` record on the primary hosted zone which points to the delegated hosted zone’s name servers. You can see an example of this pattern in the [dns-primary](/components/library/aws/dns-primary/) and [dns-delegated](/components/library/aws/dns-delegated/) components. ### Live vs Model (or Synthetic) Live represents something that is actively being used. It differs from stages like “Production” and “Staging” in the sense that both stages are “live” and in-use. While terms like “Model” and “Synthetic” refer to something which is similar, but not in use by end-users. For example, a live production vanity domain of `acme.com` might have a synthetic vanity domain of `acme-prod.net`. ### Docker Based Toolbox (aka Geodesic) In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one convenient image that you can use as your infrastructure automation toolbox. We call it [Geodesic](/resources/legacy/fundamentals/geodesic) and we use it as our DevOps automation shell and as the base Docker image for all of our DevOps tooling. Geodesic is a DevOps Linux Distribution packaged as a Docker image that provides users the ability to utilize `atmos`, `terraform`, `kubectl`, `helmfile`, the AWS CLI, and many other popular tools that compromise the SweetOps methodology without having to invoke a dozen `install` commands to get started. It’s intended to be used as an interactive cloud automation shell, a base image, or in CI/CD workflows to ensure that all systems are running the same set of versioned, easily accessible tools. ### Vendoring Vendoring is a strategy of importing external dependencies into a local source tree or VCS. Many languages (e.g. NodeJS, Golang) natively support the concept. However, there are many other tools which do not address how to do vendoring, namely `terraform`. There are a few reasons to do vendoring. Sometimes the tools we use do not support importing external sources. Other times, we need to make sure to have full-control over the lifecycle or versioning of some code in case the external dependencies go away. Our current approach to vendoring of thirdparty software dependencies is to use [vendir](https://github.com/vmware-tanzu/carvel-vendir) when needed. Example use-cases for Vendoring: 1. Terraform is one situation where it’s needed. While terraform supports child modules pulled from remote sources, components (aka root modules) cannot be pulled from remotes. 2. GitHub Actions do not currently support importing remote workflows. Using `vendir` we can easily import remote workflows. ### Generators Generators in SweetOps are the pattern of producing code or configuration when existing tools have shortcomings that cannot be addressed through standard IaC. This is best explained through our use-cases for generators today: 1. In order to deploy AWS Config rules to every region enabled in an AWS Account, we need to specify a provider block and consume a compliance child module for each region. Unfortunately, [Terraform does not currently support the ability loop over providers](https://github.com/hashicorp/terraform/issues/19932), which results in needing to manually create these provider blocks for each region that we’re targeting. On top of that, not every organization uses the same types of accounts so a hardcoded solution is not easily shared. Therefore, to avoid tedious manual work we use the generator pattern to create the `.tf` files which specify a provider block for each module and the corresponding AWS Config child module. 2. Many tools for AWS work best when profiles have been configured in the AWS Configuration file (`~/.aws/config`). If we’re working with dozens of accounts, keeping this file current on each developer’s machine is error prone and tedious. Therefore we use a generator to build this configuration based on the accounts enabled. 3. Terraform backends do not support interpolation. Therefore, we define the backend configuration in our YAML stack configuration and use `atmos` as our generator to build the backend configuration files for all components. ### The 4-Layers of Infrastructure We believe that infrastructure fundamentally consists of 4-layers of infrastructure. We build infrastructure starting from the bottom layer and work our way up. Each layer builds on the previous one and our structure is only as solid as our foundation. The tools at each layer vary and augment the underlying layers. Every layer has it’s own SDLC and is free to update independently of the other layers. The 4th and final layer is where your applications are deployed. While we believe in using terraform for layers 1-3, we believe it’s acceptable to introduce another layer of tools to support application developers (e.g. Serverless Framework, CDK, etc) are all acceptable since we’ve built a solid, consistent foundation. --- ## Geodesic v3 (Obsolete) import ReactPlayer from 'react-player' import Note from '@site/src/components/Note'; This documentation is for Geodesic v3, which is now obsolete. Please refer to the [Geodesic project](https://github.com/cloudposse/geodesic/) or the updated documentation on this site for current documentation on Geodesic v4 or later. ## Introduction In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one convenient image that you can use as your infrastructure automation toolbox. We call it Geodesic and we use it as our DevOps automation shell and as the base Docker image for all of our DevOps tooling. Geodesic is a DevOps Linux Distribution packaged as a Docker image that provides users the ability to utilize `atmos`, `terraform`, `kubectl`, `helmfile`, the AWS CLI, and many other popular tools that compromise the SweetOps methodology without having to invoke a dozen `install` commands to get started. It’s intended to be used as an interactive cloud automation shell, a base image, or in CI/CD workflows to ensure that all systems are running the same set of versioned, easily accessible tools. Most commonly, it is used as a base image for individual user, project, or company Docker images that have additional tools or specific versions of tools pre-installed. For example, Geodesic does not have `atmos` pre-installed, so that you can install the version of `atmos` that you want to use in your project update that version independently of updating Geodesic. These days, the typical software application is distributed as a docker image and run as a container. Why should infrastructure be any different? Since everything we write is "Infrastructure as Code", we believe that it should be treated the same way. This is the "Geodesic Way". Use containers+envs instead of unconventional wrappers, complicated folder structures, and symlink hacks. Geodesic is the container for all your infrastructure automation needs that enables you to truly achieve SweetOps. An organization may choose to leverage all of these components or just the parts that make their life easier. We recommend starting by using geodesic as a Docker base image (e.g. `FROM cloudposse/geodesic:...` pinned to a release and base OS) in your projects. :::info **Apple Silicon (M1, M2, etc. Chips) Support** Geodesic is comprised of a large collection of open-source tools, most of them not created or maintained by Cloud Posse. (Most of them and many more can be installed individually using packages from our [`packages` repository](https://github.com/cloudposse/packages).) As such, **support for Macs with Apple chips (M1, M2, etc.) is fully not under Cloud Posse's control**, rather it depends on each tool author updating each tool for the Apple chips, referred to as the `arm64` architecture. **The Debian-based Geodesic is provided as a multi-architecture Docker image, supporting both Intel chips (`amd64`) and Apple chips (`arm64`)**, but the (deprecated) Alpine-based Geodesic is only provided for Intel chips (`amd64`). The image built for `arm64` does not include a few tools that the maintainers have not yet updated for `linux/arm64`, but those tools are not widely used anymore, so your experience should be good on Apple silicon. (See the current list of unsupported tools [here](https://github.com/cloudposse/geodesic/blob/master/packages-amd64-only.txt)) ::: ## Use-cases Since `geodesic` is at its heart just a dockerized toolbox, it can be used anywhere docker images can be run. It supports both headless and interactive terminals. ### Use a Local Development Environment Running `geodesic` as a local development environment ensures all team members on the team can get up and running quickly using the same versions of the tools. The only requirement is having Docker installed. :::info **Pro Tip!** When Geodesic is started using the wrapper script, it mounts the host’s `$HOME` directory as `/localhost` inside the container and creates a symbolic link from `$HOME` to `/localhost` so that files under `$HOME` on the host can be referenced by the exact same absolute path both on the host computer and inside Geodesic. For example, if the host `$HOME` is `/Users/fred`, then `/Users/fred/src/example.sh` will refer to the same file both on the host and from inside the Geodesic shell. This means you can continue editing files using your favorite IDE (e.g. VSCode, IntelliJ, etc) and interact with your local filesystem within the docker container. ::: ### Use as a Remote Development Environment Running `geodesic` as a remote development environment is as easy as calling `kubectl run` on the geodesic container. You’ll be able then to remotely interact with the container to debug within a kubernetes cluster. ### Use as a Base Image for Automation Running `geodesic` as the base image for Spacelift or with GitHub Actions ensures you can use the same exact tooling in an automated fashion. ## How-to Guides - [How to Upgrade or Install Versions of Terraform](/learn/maintenance/upgrades/how-to-upgrade-or-install-versions-of-terraform) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [How to run Docker-in-Docker with Geodesic?](/learn/maintenance/tutorials/how-to-run-docker-in-docker-with-geodesic) - [How to Customize the Geodesic Shell](/learn/maintenance/tutorials/how-to-customize-the-geodesic-shell) - [How to use Atmos](/learn/maintenance/tutorials/how-to-use-atmos) ## Debian and Alpine Base Images Starting with Geodesic version 0.138.0, we distribute 2 versions of Geodesic Docker images, one based on Debian and one based on Alpine, tagged `VERSION-BASE_OS`, e.g. `0.138.0-alpine`. Prior to this, all Docker images were based on Alpine only and simply tagged `VERSION`. We encourage people to use the Debian version and report any issues by opening a GitHub issue. We will continue to maintain the `latest-alpine` and `latest-debian` Docker tags for those who want to commit to using one base OS or the other but still want automatic updates. However, our primary focus will be on the Debian-based images. The Alpine-based images are deprecated and issues unique to Alpine (i.e. not affecting the Debian version) may not be addressed. ## Packages Central to `geodesic` is its rich support for the latest version of [the most popular packages](https://github.com/cloudposse/packages/tree/master/vendor) for DevOps. We maintain hundreds of packages that are graciously hosted by Cloud Smith. Our packages are updated nightly as soon as new releases are made available by vendors. As such, we strongly recommend version pinning packages installed via the `Dockerfile`. Also unique about our packages is that for `kubectl` and `terraform` we distribute all major versions with `dpkg-alternative` support so they can be concurrently installed without the use of version managers. For example, to install the latest version of `kubectl` v1.28 (to match the version of the cluster it is controlling), you can install package `kubectl-1.28`. Package repository hosting is graciously provided by [cloudsmith](https://cloudsmith.io/). Cloudsmith is the only fully hosted, cloud-native, universal package management solution, that enables your organization to create, store and share packages in any format, to any place, with total confidence. We believe there’s a better way to manage software assets and packages, and they’re making it happen! ## Filesystem Layout Here’s a general filesystem layout for an infrastructure repository leveraging `geodesic` with `atmos` together with stacks and components. Note, individual customer repos will resemble this layout but will not be identical. ``` infrastructure/ # GitHub Repository ├── Dockerfile # Dockerfile uses `cloudposse/geodesic` as base Image ├── Makefile # Makefile to build custom Geodestic-based Docker image and install wrapper script for `geodesic` ├── README.md ├── components # Location for all reusable component building blocks │ └── terraform/ # Location for all terraform (HCL) components │ └── foobar/ # Example `foobar` component │ ├── README.md # Every component has a well-maintained `README.md` with usage instructions │ ├── context.tf # Standard context interface for all cloud posse modules. │ ├── main.tf # Standard `main.tf` based on HashiCorp best-practices │ ├── modules/ # Example of submodules within a component (aka child modules) │ │ ├── baz/ # Submodule named `baz/` │ │ │ ├── context.tf # Submodules should use the same standard interface with `context.tf` │ │ │ ├── main.tf # Submodules should also follow HashiCorp best practices for module layout │ │ │ ├── outputs.tf │ │ │ └── variables.tf # Submodules should define variables in `variables.tf` and not modify `context.tf` │ │ └── bar/ # Example of another submodule named `bar/` │ │ ├── context.tf │ │ ├── main.tf │ │ ├── outputs.tf │ │ └── variables.tf │ ├── outputs.tf # Outputs exported by this component in the remote state │ ├── providers.tf # Providers used by this component │ ├── remote-state.tf # Remote state leveraged by the component using the `remote-state` module │ ├── variables.tf # Variables used by the component │ └── versions.tf # Version pinning for providers used by the component │ ├── docs/ # Location for documentation specific to this repository │ ├── adr/ # Home for all Architectural Design Records for your organization │ │ ├── 0001-namespace-abbreviation.md │ │ ├── 0002-infrastructure-repository-name.md │ │ ├── 0003-email-addresses-for-aws-accounts.md │ │ ├── 0004-secure-channel-secrets-sharing.md │ │ ├── 0005-primary-aws-region.md │ │ ├── README.md # Index of all READMEs │ │ └── template.md # Markdown template file to create new ADRs │ │ │ └── cold-start.md │ ├── rootfs/ # The `rootfs` pattern overlays this filesystem on `/` (slash) inside the docker image (e.g. `ADD /rootfs /`) │ ├── etc/ # The `/etc/` inside the container │ │ ├── aws-config/ │ │ │ └── aws-config-teams # The AWS config used from within Geodesic by setting `AWS_CONFIG_PATH` │ │ │ └── aws-extend-switch-roles # The config used by browser plugin AWS Extend Switch Roles │ │ └── motd # Message of the Day (MOTD) displayed to `stdout` on interactive shell logins │ │ │ └── usr/ # The `usr/` tree inside the docker image │ └── local/ │ ├── bin/ # Stick all scripts in `/usr/local/bin` │ │ ├── aws-config # Script used to generate the files in `/rootfs/etc/aws-config` │ │ ├── eks-update-kubeconfig # Helper script to export the `kubeconfig` for EKS using the `aws` CLI │ │ ├── spacelift-git-use-https # Helper script to configure git to use HTTPS rather than SSH for Spacelift │ │ ├── spacelift-tf-workspace # Helper script to configure Spacelift Terraform workspace │ │ └── spacelift-write-vars # Helper script to write Terraform variables to a file for Spacelift to use │ └── etc/ │ └── atmos/ │ └── atmos.yaml # Atmos CLI configuration. Instructs where to find stack configs and components. │ └── stacks/ # Location of all stack configurations ├── catalog/ # Location where to store catalog imports. See our catalog pattern. │ ├── account-map.yaml # Catalog entry for [account-map](/components/library/aws/account-map/) │ ├── account-settings.yaml # Catalog entry for [account-settings](/components/library/aws/account-settings/) │ ├── account.yaml # ... │ ├── aws-team-roles.yaml │ ├── aws-teams.yaml │ ├── cloudtrail.yaml │ ├── dns-delegated.yaml │ ├── dns-primary.yaml │ ├── ecr.yaml │ ├── eks │ │ ├── alb-controller.yaml │ │ ├── cert-manager.yaml │ │ ├── eks.yaml │ │ ├── external-dns.yaml │ │ ├── metrics-server.yaml │ │ └── ocean-controller.yaml │ ├── s3 │ │ ├── alb-access-logs.yaml │ │ └── s3-defaults.yaml │ ├── sso.yaml │ ├── tfstate-backend.yaml │ ├── transit-gateway.yaml │ └── vpc.yaml └── orgs/ # Environment-specific configuration └── eg/ # Example organization ├── corp/ # Example organizaional unit (OU) │ ├── auto/ # Example environment │ │ ├── _defaults.yaml # Default settings for all stacks in this environment │ │ ├── global-region.yaml # Settings for regionless or all-region components (e.g. IAM roles) │ │ └── us-west-2.yaml # Settings for infrastructure deployed in us-west-2 region │ ├── identity/ │ │ ├── _defaults.yaml │ │ ├── global-region.yaml │ │ └── us-west-2.yaml │ ├── root/ │ │ ├── _defaults.yaml │ │ ├── global-region.yaml │ │ └── us-west-2.yaml └── plat/ ├── dev/ │ ├── _defaults.yaml │ ├── global-region.yaml │ └── us-west-2.yaml ├── stage/ │ ├── _defaults.yaml │ ├── global-region.yaml │ └── us-west-2.yaml └── prod/ ├── _defaults.yaml ├── global-region.yaml └── us-west-2.yaml ``` ## Build and Run Geodesic Prerequisites for your host computer: - Docker installed - `make` installed, preferably GNU Make - `git` installed - Infrastructure Git repo cloned If all goes well, you should be able to build and run the Infrastructure Docker image from your host by executing `make all` from the command line in the root directory of your Git repo clone. If you have issues at this step, contact Cloud Posse or look for help in the Cloud Posse [Geodesic](https://github.com/cloudposse/geodesic/) or [Reference Architecture](https://docs.cloudposse.com/) repos. At this point (after `make all` concludes successfully) you should be running a `bash` shell inside the Infrastructure Docker container (which we will also call the "Geodesic shell") and your prompt should look something like this: ``` ⧉ Infrastructure ✗ . [none] / ⨠ ``` From here forward, any command-line commands are meant to be run from within the Geodesic shell. ## Troubleshooting ### Command-line Prompt Ends with a Unicode Placeholder If your command-line prompt inside of the `geodesic` shell ends with a funky Unicode placeholder, then chances are the default character we use for the end of the command line prompt ([Unicode Z NOTATION SCHEMA PIPING](https://www.compart.com/en/unicode/U+2A20)) is not present in the font library you are using. On the Mac, Terminal (at least) falls back to some other font when the character is missing, so it's not a problem. On other systems, we recommend installing the freely available Noto font from Google, whose mission is to supply workable characters for every defined Unicode code point. On Ubuntu, it is sufficient to install the Noto core fonts, via ``` apt install fonts-noto-core ``` Another option is to switch to a different command prompt scheme, by adding ``` export PROMPT_STYLE="fancy" # or "unicode" or "plain" ``` to your Geodesic customizations. See [How to Customize the Geodesic Shell](/learn/maintenance/tutorials/how-to-customize-the-geodesic-shell) for more detail, and also to see how you can completely customize the prompt decorations. ### Files Written to Mounted Linux Home Directory Owned by Root User If a user runs the Docker daemon as `root`, files may fail to be written to the mounted Linux home directory. The recommended solution for Linux users is to run Docker in ["rootless"](https://docs.docker.com/engine/security/rootless/) mode. In this mode, the Docker daemon runs as the host user (rather than as root) and files created by the root user in Geodesic are owned by the host user on the host. Not only does this configuration solve this issue, but it provides much better system security overall. See [this issue](https://github.com/cloudposse/geodesic/issues/594) for more details. --- ## Introduction # What is SweetOps? SweetOps is a methodology for building modern, secure infrastructure on top of Amazon Web Services (AWS). It provides a toolset, library of reusable Infrastructure as Code (IaC), and opinionated patterns to help you bootstrap robust cloud native architectures. Built in an Open Source first fashion by [Cloud Posse](https://cloudposse.com/), it is utilized by many high performing startups to ensure their cloud infrastructure is an advantage instead of a liability. In short, SweetOps makes working in the DevOps world **Sweet**! ## Who is this for? SweetOps is for DevOps or platform engineering teams that want an opinionated way to build software platforms in the cloud. If the below sounds like you, then SweetOps is what you're looking for: 1. You're on AWS 1. You're using Terraform as your IaC tool 1. Your platform needs to be secure and potentially requires passing compliance audits (PCI, SOC2, HIPAA, HITRUST, FedRAMP, etc.) 1. You don't want to reinvent the wheel With SweetOps you can implement the following complex architectural patterns with ease: 1. An AWS multi-account Landing Zone built on strong, well-established principles including Separation of Concerns and Principle of Least Privilege (POLP). 1. Multi-region, globally available application environments with disaster recovery capabilities. 1. Foundational AWS-focused security practices that make complex compliance audits a breeze. 1. Microservice architectures that are ready for massive scale running on Docker and Kubernetes. 1. Reusable service catalogs and components to promote reuse across an organization and accelerate adoption ## What are the alternatives? SweetOps is similar to [Gruntwork](https://gruntwork.io/)'s Terragrunt + subscription plan, [the Terraspace framework](https://terraspace.cloud/), and [BinBash's Leverage](https://leverage.binbash.com.ar/). How does it differentiate from these solutions? 1. It's 100% Open Source: SweetOps [is on GitHub](https://github.com/cloudposse) and is free to use with no strings attached under Apache 2.0. 1. It's comprehensive: SweetOps is not only about Terraform. It provides patterns and conventions for building cloud native platforms that are security focused, Kubernetes-based, and driven by continuous delivery. 1. It's community focused: SweetOps has [over 9000 users in Slack](https://sweetops.com/slack/), well-attended weekly office hours, and a [budding community forum](https://ask.sweetops.com/). --- ## Leapp [https://github.com/Noovolari/leapp](https://github.com/Noovolari/leapp) Leapp is a Desktop Dev Tool that handles the management and security of your cloud credentials for you so you can log into any AWS account with the click of a button using your native OS keychain. ## How-to Guides - [How to Use Leapp to Authenticate with AWS](/layers/identity/tutorials/leapp/) ### Reference - [How to Log into AWS](https://docs.cloudposse.com/layers/identity/how-to-log-into-aws/) --- ## Philosophy SweetOps is built on a foundational philosophy that ensures the methodology is comprehensive and reusable across organizations. To fully understand what SweetOps is and how to utilize it effectively, it's critical to understand the backing principles that define it. ## 100% Open Source All of the primary technology that enables SweetOps is Open Sourced by [Cloud Posse](https://cloudposse.com) and the community under Apache 2.0 Open Source Software License. Using open source licensing is a practical, deliberate strategy. It enables SweetOps to be easily adopted by organizations by using a well-understood license that carries less risk than proprietary licenses. With SweetOps, everything is permissively licensed, which makes Infrastructure as Code more straightforward to adopt. It protects all parties to reuse the software and fosters greater collaboration. ## Automate Everything An important idea from the beginning has been to ensure that all pieces of the process are automated. This includes everything from AWS account creation to configuring incident response management tooling. Ensuring that all aspects of SweetOps are automated makes processes repeatable, transparent, and reliable without having to depend on quickly outdated documentation (aka WikiOps). ## Embrace the UNIX Philosophy for the Cloud Era The SweetOps infrastructure as code library, purpose built tools, and reusable catalogs are all crafted to follow the [UNIX Philosophy](https://en.wikipedia.org/wiki/Unix_philosophy); they're intended to be small, simple, readable, and modular in nature which encourages composition. SweetOps never intends to provide one tool that does it all. It is the polar opposite of platforms like OpenShift and Rancher which attempt to do everything. Instead, SweetOps provides small tools that do one thing well and then stitch them together to achieve the best results. This enables SweetOps to swap in & out tools as better ones come along. ## YAML Configuration Drives As Much As Possible SweetOps has adopted YAML as its configuration syntax of choice. YAML is a portable, well-understood configuration format that is widely known and accepted within the DevOps community. It easily used in any language and has many advantages over using configuration formats. Particularly of note is the comparison of YAML to Terraform's `.tfvar` files which are a form of HCL. With `.tfvar` files it is not possible to load them from remote locations (e.g. over `https://`), do interpolations, or control which files we want to load and when to load them. On the other hand, using YAML, we're able to address all of these shortcomings while also supporting [anchors](https://helm.sh/docs/chart_template_guide/yaml_techniques/#yaml-anchors) and permits us to define additional behaviors like imports and inheritance. Utilizing YAML also enables SweetOps to separate code from configuration. Many will agree that Terraform has graduated from being purely configuration into a full-fledged programming language. With that being the case we focus on putting our business logic into Terraform and keep our configuration separate in YAML. This results in more reusable, clearly configured usage of our tools. ## 4 Layers of Infrastructure ![4 Layers of Infrastructure](https://lucid.app/publicSegments/view/dc705e05-cf3e-4e03-9029-acd8c4b4812f/image.png) SweetOps breaks down cloud infrastructure into 4 discrete layers: 1. **Foundation**: Your AWS Accounts, VPCs, IAM, and DNS architecture. 1. **Platform**: Your EKS / ECS Clusters, Load Balancers, IAM Service Accounts, and Certificates. 1. **Shared Services**: Your CI/CD pipelines, BeyondCorp solution, and Monitoring and Logging tooling. 1. **Application**: Your Backend and Frontend Applications. We delineate between these layers as they have different Software Development Life Cycles (SDLC) and tools that are responsible for managing them. For example, Terraform is great for building your Foundation layer, but other tools might be better for managing the continuous delivery of the Application layer (e.g. ArgoCD). It's also important to note that each layer builds on the previous and the lower layers are less likely to change over time. At the bottom, you won't frequently change your AWS accounts and VPCs in your Foundation layer, just like when operating a platform you won't be rebuilding EKS clusters in your Platform layer without disrupting all tenants of the platform. You might, however, add HashiCorp Vault to your Shared Services layer to increase your security posture or add a new API microservice to your Application layer. The Application Layer is ultimately the most important layer of them all: it's what drives your business. It's where your applications live, which is why it will change continuously. ## Optimize for Day 2 Operations One important part we emphasize within SweetOps is that we optimize the methodology for [day 2+ operations](https://dzone.com/articles/defining-day-2-operations) and not the cold-start. The process to create the bottom layers of your platform is day 1 or your cold-start process. You will generally only perform a cold-start once so it's not logical or economical to focus on agility in this phase of building the platform. Instead, SweetOps focuses on optimizing the processes that come after your cold-start: GitOps, release engineering, monitoring/SRE, and ensuring your applications run efficiently on top of the base platform. ## Share Nothing SweetOps takes the principle of [Separation of Concerns](https://en.wikipedia.org/wiki/Separation_of_concerns) to heart when it comes to infrastructure. We believe strongly in fine grained separations of AWS Accounts, AWS Organizational Units, environments, and access credentials. We utilize a separate AWS account to manage DNS, logs, auditing, and identity in practice of this principle. These practices ensure optimal security and agility if any part of the system is compromised or needs to change in some way. ## Secure by Default The SweetOps infrastructure as code library focuses on being secure by default rather than requiring additional configuration to secure it. We think about this as being implicitly secure and if your use-case requires insecurity for one reason or another then you need to be explicit about that. One way we practice our "Secure by Default" process is by utilizing [BridgeCrew](https://bridgecrew.io/)'s security scanning tools [on our Terraform modules and components](https://github.com/Cloudposse/terraform-aws-vpc#security--compliance-) to ensure they're passing many of the leading security compliance testing frameworks. ## Delegate Thought Leadership As much as possible, SweetOps aims to identify thought leaders and mimic them instead of reinventing the wheel with our own solutions. This is why SweetOps chooses to use best in class tools like Terraform, Helmfile, ArgoCD, etc. Instead of creating those types of tools ourselves, we aim to provide the thought leadership on how to tie and orchestrate those tools together (e.g. via [Stacks](/resources/legacy/fundamentals/concepts#stacks)). --- ## Stacks Stacks are a way to express the complete infrastructure needed for an environment composed of [Components](/components) using a standard YAML configuration ## Background Stacks are a central SweetOps abstraction layer that is used to instantiate [Components](/components). They’re a set of YAML files [that follow a standard schema](https://github.com/cloudposse/atmos/blob/master/docs/schema/stack-config-schema.json) to enable a **fully declarative description of your various environments**. This empowers you with the ability to separate your infrastructure’s environment configuration settings from the business logic behind it (provided via components). SweetOps utilizes a custom YAML configuration format for stacks because it’s an easy-to-work-with format that is nicely portable across multiple tools. The stack YAML format is natively supported today via [Atmos](/resources/legacy/fundamentals/atmos) , [the terraform-yaml-stack-config module](https://github.com/cloudposse/terraform-yaml-stack-config), and [Spacelift](https://spacelift.io/) via [the terraform-spacelift-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation). :::note Stacks define a generic schema for expressing infrastructure ::: ## How-to Guides - [How to add or mirror a new region](/layers/accounts/tutorials/how-to-add-or-mirror-a-new-region) - [How to Upgrade or Install Versions of Terraform](/learn/maintenance/upgrades/how-to-upgrade-or-install-versions-of-terraform) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to Use Terraform Remote State](/learn/maintenance/tutorials/how-to-use-terraform-remote-state) - [How to Manage Explicit Component Dependencies with Spacelift](/resources/deprecated/spacelift/tutorials/how-to-manage-explicit-component-dependencies-with-spacelift) - [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [How to Define Stacks for Multiple Regions?](/learn/maintenance/tutorials/how-to-define-stacks-for-multiple-regions) - [How to Version Pin Components in Stack Configurations](/learn/maintenance/tutorials/how-to-version-pin-components-in-stack-configurations) - [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks) ## Conventions We have a number of important conventions around stacks that are worth noting. :::info Make sure you’re already familiar with the core [Concepts](/resources/legacy/fundamentals/concepts). ::: ### Stack Files Stack files can be very numerous in large cloud environments (think many dozens to hundreds of stack files). To enable the proper organization of stack files, SweetOps recommends the following: - All stacks should be stored in a `stacks/` folder at the root of your infrastructure repository. - Name individual environment stacks following the pattern of `$environment-$stage.yaml` - For example, `$environment` might be `ue2` (for `us-east-2`) and `$stage` might be `prod` which would result in `stacks/ue2-prod.yaml` - For any **global** resources (as opposed to _regional_ resources), such as Account Settings, IAM roles and policies, DNS zones, or similar, the `environment` for the stack should be `gbl` to connote that it’s not tied to any region. - For example, to deploy the `iam-delegated-roles` component (where all resources are global and not associated with an AWS region) to your production account, you should utilize a `stacks/gbl-prod.yaml` stack file. ### Catalogs When you have a configuration that you want to share across various stacks, use catalogs. Catalogs are the SweetOps term for shared, reusable configuration. By convention, all shared configuration for stacks is put in the `stacks/catalog/` folder, which can then be used in the root `stacks/` stack files via `import`. These files use the same stack schema. Learn more about [How to Use Imports and Catalogs in Stacks](/learn/maintenance/tutorials/how-to-use-imports-and-catalogs-in-stacks). There are a few suggested shared catalog configurations that we recommend adopting: - **Global Catalogs**: For any configuration to share across **all** stacks. - For example, you create a `stacks/catalog/globals.yaml` file and utilize `import` wherever you need that catalog. - **Environment Catalogs**: For any configuration you want to share across `environment` boundaries. - For example, to share configuration across `ue2-stage.yaml` and `uw2-stage.yaml` stacks, you create a `stacks/catalog/stage/globals.yaml` file and utilize `import` in both the `ue2-stage.yaml` and `uw2-stage.yaml` stacks to pull in that catalog. - **Stage Catalogs**: For any configuration that you want to share across `stage` boundaries. - For example, to share configuration across `ue2-dev.yaml`, `ue2-stage.yaml`, and `ue2-prod.yaml` stacks, you create a `stacks/catalog/ue2/globals.yaml` file and `import` that catalog in the respective `dev`, `stage`, and `prod` stacks. - **Base Components**: For any configuration that you want to share across all instances of a component. - For example, you’re using the `eks` component and you want to ensure all of your EKS clusters are using the same Kubernetes version, you create a `stacks/catalog/component/eks.yaml` file which specifies the `eks` component’s `vars.kubernetes_version`. You can then `import` that base component configuration in any stack file that uses the [eks/cluster](/components/library/aws/eks/cluster/) component. - More information in the below section. ### Component Inheritance Using a component catalog, you can define the default values for all instances of a component across your stacks. But it is also important to note that you can provide default values for multiple instances of a component in a single stack using the component inheritance pattern via the `component` key / value: :::info **Pro tip**: You can also use our inheritance model for the [polymorphism](https://en.wikipedia.org/wiki/Polymorphism_(computer_science)) of components. ::: ``` # stacks/catalog/component/s3-bucket.yaml components: terraform: s3-bucket: vars: enabled: false user_enabled: false acl: private grants: null versioning_enabled: true # stacks/uw2-dev.yaml import: - catalog/component/s3-bucket - catalog/dev/globals - catalog/uw2/globals components: terraform: public-images: component: s3-bucket vars: enabled: true acl: public name: public-images export-data: component: s3-bucket vars: enabled: true name: export-data # ... ``` In the above example, we’re able to utilize the default settings provided via the `s3-bucket` base component catalog, while also creating multiple instances of the same component and providing our own overrides. This enables maximum reuse of global component configuration. ### Terraform Workspace Names In `atmos` and the accompanying terraform automation modules like [terraform-spacelift-cloud-infrastructure-automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) the terraform [workspaces](https://www.terraform.io/docs/language/state/workspaces.html) will be automatically created when managing components. These workspaces derive their names from the stack name and the component name in question following this pattern: `$env-$stage-$component`. The result is workspace names like `ue2-dev-eks` or `uw2-prod-mq-broker`. ## Pro Tips Here are some tips to help you write great stacks: 1. Use `{}` for empty maps, but not just a key with an empty value. 2. Use consistent data types for deep merging to work appropriately with imports (e.g. don’t mix maps with lists or scalars). 3. Use [YAML anchors](https://blog.daemonl.com/2016/02/yaml.html) to DRY up a config within a single file. :::caution **IMPORTANT** Anchors work only within the scope of a single file boundary and not across multiple imports. ::: ## Stack Schema [The official JSON Schema document for Stacks can be found here](https://github.com/cloudposse/atmos/blob/master/docs/schema/stack-config-schema.json). The below is a walk-through of a complete example utilizing all capabilities. ``` # stacks/ue2-dev.yaml # `import` enables shared configuration / settings across different stacks # The referenced files are deep merged into this stack to support granular configuration capabilities import: # Merge the below `stacks/catalog/*.yaml` files into this stack to provide any shared `vars` or `components.*` configuration - catalog/globals - catalog/ue2/globals - catalog/dev/globals # `vars` provides shared configuration for all components -- both terraform + helmfile vars: # Used to determine the name of the workspace (e.g. the 'dev' in 'ue2-dev') stage: dev # Used to determine the name of the workspace (e.g. the 'ue2' in 'ue2-dev') environment: ue2 # Define cross-cutting terraform configuration terraform: # `terraform.vars` provides shared configuration for terraform components vars: {} # `backend_type` + `backend` provide configuration for the terraform backend you # would like to use for all components. This is typically defined in `globals.yaml` # atmos + our modules support all options that can be configured for a particular backend. # `backend_type` defines which `backend` configuration is enabled backend_type: s3 # s3, remote, vault backend: s3: encrypt: true bucket: "eg-uw2-root-tfstate" key: "terraform.tfstate" dynamodb_table: "eg-uw2-root-tfstate-lock" role_arn: "arn:aws:iam::999999999999:role/eg-gbl-root-terraform" acl: "bucket-owner-full-control" region: "us-east-2" remote: {} vault: {} # Define cross-cutting helmfile configuration helmfile: # `helmfile.vars` provides shared configuration for terraform components vars: account_number: "999999999999" # Components are all the top-level units that make up this stack components: # All terraform components should be listed under this section. terraform: # List one or more Terraform components here first-component: # Provide automation settings for this component settings: # Provide spacelift specific automation settings for this component # (Only relevant if utilizing terraform-spacelift-cloud-infrastructure-automation) spacelift: # Controls whether or not this workspace should be created # NOTE: If set to 'false', you cannot reference this workspace via `triggers` in another workspace! workspace_enabled: true # Override the version of Terraform for this workspace (defaults to the latest in Spacelift) terraform_version: 0.13.4 # Which git branch trigger's this workspace branch: develop # Controls the `autodeploy` setting within this workspace (defaults to `false`) auto_apply: true # Add extra 'Run Triggers' to this workspace, beyond the parent workspace, which is created by default # These triggers mean this component workspace will be automatically planned if any of these workspaces are applied. triggers: - ue2-dev-second-component - gbl-root-example1 # Set the Terraform input variable values for this component. vars: my_input_var: "Hello world! This is a value that needs to be passed to my `first-component` Terraform component." bool_var: true number_var: 47 # Complex types like maps and lists are supported. list_var: - example1 - example2 map_var: key1: value1 key2: value2 # Every terraform component should be uniquely named and correspond to a folder in the `components/terraform/` directory second-component: vars: my_input_var: "Hello world! This is another example!" # You can also define component inheritance in stacks to enable unique workspace names or multiple usages of the same component in one stack. # In this example, `another-second-component` inherits from the base `second-component` component and overrides the `my_input_var` variable. another-second-component: component: second-component vars: my_input_var: "Hello world! This is an override." # All helmfile components should be listed under this section. helmfile: # Helmfile components should be uniquely named and correspond to a folder in the `components/helmfile/` directory # Helmfile components also support virtual components alb-controller: # Set the helmfile input variable values for this component. vars: installed: true chart_values: enableCertManager: true # `workflows` enable the ability to define an ordered list of operations that `atmos` will execute. These operations can be any type of component such as terraform or helmfile. # See "Getting started with Atmos" documentation for full details: /tutorials/atmos-getting-started/ workflows: # `workflows` is a map where the key is the name of the workflow that you're defining deploy-eks-default-helmfiles: # `description` should provide useful information about what this workflow does description: Deploy helmfile charts in the specific order # `steps` defines the ordering of the jobs that you want to accomplish steps: # `job` entries defined `atmos` commands that you want to execute as part of the workflow - job: helmfile sync cert-manager - job: helmfile sync external-dns - job: helmfile sync alb-controller - job: helmfile sync metrics-server - job: helmfile sync ocean-controller - job: helmfile sync efs-provisioner - job: helmfile sync idp-roles - job: helmfile sync strongdm - job: helmfile sync reloader - job: helmfile sync echo-server ``` --- ## Terraform(Fundamentals) import ReactPlayer from 'react-player' For the most part, we assume users have a solid grasp of `terraform`. Cloud Posse has adopted a number of conventions for how we work with `terraform` that we document here. Review [our opinionated public “best practices” as it relates to terraform](/best-practices/terraform). We use [Atmos](/resources/legacy/fundamentals/atmos) together with [Stacks](/resources/legacy/fundamentals/stacks) to call [Components](/components) that provision infrastructure with `terraform`. :::caution Be aware of [Terraform Environment Variables](https://www.terraform.io/docs/cli/config/environment-variables.html) that can alter the behavior of `terraform` when run outside of what you see in `atmos` or `geodesic`. These are also helpful to change default behavior as well, such as by setting the `TF_DATA_DIR`. ::: ## How-to Guides - [How to Upgrade or Install Versions of Terraform](/learn/maintenance/upgrades/how-to-upgrade-or-install-versions-of-terraform) - [How to Manage Terraform Dependencies in Micro-service Repositories](/learn/maintenance/tutorials/how-to-manage-terraform-dependencies-in-micro-service-repositori) - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to Use Terraform Remote State](/learn/maintenance/tutorials/how-to-use-terraform-remote-state) - [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) - [How to support GovCloud and Other AWS Partitions with Terraform](/learn/maintenance/tutorials/how-to-support-govcloud-and-other-aws-partitions-with-terraform) ## Architectural Design Records - [Proposed: Use Strict Provider Pinning in Components](/resources/adrs/proposed/proposed-use-strict-provider-pinning-in-components) - [Use Basic Provider Block for Root-level Components](/resources/adrs/adopted/use-basic-provider-block-for-root-level-components) - [Use Terraform Provider Block with compatibility for Role ARNs and Profiles](/resources/adrs/adopted/use-terraform-provider-block-with-compatibility-for-role-arns-an) - [Use Spacelift for GitOps with Terraform](/resources/adrs/adopted/use-spacelift-for-gitops-with-terraform) - [Use SSM over ASM for Infrastructure](/resources/adrs/adopted/use-ssm-over-asm-for-infrastructure) - [Proposed: Use Defaults for Components](/resources/adrs/proposed/proposed-use-defaults-for-components) ## Conventions ### Mixins Terraform does not natively support the object-oriented concepts of multiple inheritances or [mixins](https://en.wikipedia.org/wiki/Mixin), but we can simulate by using convention. For our purposes, we define a mixin in terraform as a controlled way of adding functionality to modules. When a mixin file is dropped into a folder of a module, the code in the mixin starts to interact with the code in the module. A module can have as many mixins as needed. Since terraform does not directly, we instead use a convention of exporting what we want to reuse. We achieve this currently using something we call an `export` in our terraform modules, which publish some reusable terraform code that we copy verbatim into modules as needed. We use this pattern with our `terraform-null-label` using the `context.tf` file pattern (See below). We also use this pattern in our `terraform-aws-security-group` module with the [https://github.com/cloudposse/terraform-aws-security-group/blob/main/exports/security-group-variables.tf](https://github.com/cloudposse/terraform-aws-security-group/blob/main/exports/security-group-variables.tf). To follow this convention, create an `export/` folder with the mixin files you wish to export to other modules. Then simply copy them over (E.g. with `curl`). We recommend calling the installed files something `.mixin.tf` so it’s clear it's an external asset. ### Resource Factories Resource Factories provide a custom declarative interface for defining multiple resources using YAML and then terraform for implementing the business logic. Most of our new modules are developed using this pattern so we can decouple the architecture requirements from the implementation. See [https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c) for a related discussion. To better support this pattern, we implemented native support for deep merging in terraform using our [https://github.com/cloudposse/terraform-provider-utils](https://github.com/cloudposse/terraform-provider-utils) provider as well as implemented a module to standardize how we use YAML configurations [https://github.com/cloudposse/terraform-yaml-config](https://github.com/cloudposse/terraform-yaml-config). Examples of modules using Resource Factory convention: - [https://github.com/cloudposse/terraform-aws-service-control-policies](https://github.com/cloudposse/terraform-aws-service-control-policies) - [https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) - [https://github.com/cloudposse/terraform-datadog-platform](https://github.com/cloudposse/terraform-datadog-platform) - [https://github.com/cloudposse/terraform-opsgenie-incident-management](https://github.com/cloudposse/terraform-opsgenie-incident-management) - [https://github.com/cloudposse/terraform-aws-config](https://github.com/cloudposse/terraform-aws-config) ### Naming Conventions (and the `terraform-null-label` Module) Naming things is hard. We’ve made it easier by defining a programmatically consistent naming convention, which we use in everything we provision. It is designed to generate consistent human-friendly names and tags for resources. We implement this using a terraform module which accepts a number of standardized inputs and produces an output with the fully disambiguate ID. This module establishes the common interface we use in all of our terraform modules in the Cloud Posse ecosystem. Use `terraform-null-label` to implement a strict naming convention. We use it in all of our [Components](/components) and export something we call the `context.tf` pattern. [https://github.com/cloudposse/terraform-null-label](https://github.com/cloudposse/terraform-null-label) Here’s video where we talk about it. There are 6 inputs considered "labels" or "ID elements" (because the labels are used to construct the ID): 1. `namespace` 2. `tenant` 3. `environment` 4. `stage` 5. `name` 6. `attributes` This module generates IDs using the following convention by default: `{namespace}-{environment}-{stage}-{name}-{attributes}`. However, it is highly configurable. The delimiter (e.g. `-`) is configurable. Each label item is optional (although you must provide at least one). #### Tenants `tenants` are a Cloud Posse construct used to describe a collection of accounts within an Organizational Unit (OU). An OU may have multiple tenants, and each tenant may have multiple AWS accounts. For example, the `platform` OU might have two tenants named `dev` and `prod`. The `dev` tenant can contain accounts for the `staging`, `dev`, `qa`, and `sandbox` environments, while the `prod` tenant only has one account for the `prod` environment. By separating accounts into these logical groupings, we can organize accounts at a higher level, follow AWS Well-Architected Framework recommendations, and enforce environment boundaries easily. ### The `context.tf` Mixin Pattern Cloud Posse Terraform modules all share a common `context` object that is meant to be passed from module to module. A `context` object is a single object that contains all the input values for `terraform-null-label` and every `cloudposse/terraform-*` module uses it to ensure a common interface to all of our modules. By convention, we install this file as `context.tf` which is why we call it the `context.tf` pattern. By default, we always provide an instance of it accessible via `module.this`, which makes it always easy to get your _context._ 🙂 Every input value can also be specified individually by name as a standard Terraform variable, and the value of those variables, when set to something other than `null`, will override the value in the context object. In order to allow chaining of these objects, where the context object input to one module is transformed and passed on to the next module, all the variables default to `null` or empty collections. ### Stacks and Components We use [Stacks](/resources/legacy/fundamentals/stacks) to define and organize configurations. We place terraform “root” modules in the `components/terraform` directory (e.g. `components/terraform/s3-bucket`). Then we define one or more catalog archetypes for using the component (e.g. `catalog/s3-bucket/logs.yaml` and `catalog/s3-bucket/artifacts`). ### Atmos CLI We predominantly call `terraform` from `atmos`, however, by design all of our infrastructure code runs without any task runners. This is in contrast to tools like `terragrunt` that manipulate the state of infrastructure code at run time. See [How to use Atmos](/learn/maintenance/tutorials/how-to-use-atmos) ## FAQ ### How to upgrade Terraform? See [How to Switch Versions of Terraform](/learn/maintenance/tutorials/how-to-switch-versions-of-terraform) for a more complete guide. TL;DR: - Note the version you want to use - Make sure the version is available in [cloudposse/packages](https://github.com/cloudposse/packages/pulls?q=terraform) to see if the version desired is in a merged PR for terraform - Make sure the version is available in Spacelift by editing an existing stack and see if the new version is available - Update Terraform in `Dockerfile` - Update Terraform in `.github/workflows/pre-commit.yaml` github action - Update Terraform in `components/terraform/spacelift/default.auto.tfvars` ### How to use `context.tf`? Copy this file from `https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf` and then place it in your Terraform module to automatically get Cloud Posse's standard configuration inputs suitable for passing to Cloud Posse modules. ``` curl -sL https://raw.githubusercontent.com/cloudposse/terraform-null-label/master/exports/context.tf -o context.tf ``` Modules should access the whole context as `module.this.context` to get the input variables with nulls for defaults, for example `context = module.this.context`, and access individual variables as `module.this.`, with final values filled in. [https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf](https://github.com/cloudposse/terraform-null-label/blob/master/exports/context.tf) For example, when using defaults, `module.this.context.delimiter` will be `null`, and `module.this.delimiter` will be `-` (hyphen). :::caution ONLY EDIT THIS FILE IN [http://github.com/cloudposse/terraform-null-label](http://github.com/cloudposse/terraform-null-label) . All other instances of this file should be a copy of that one. ::: ## Learning Resources If you’re new to terraform, here are a number of resources to check out: - [https://learn.hashicorp.com/terraform](https://learn.hashicorp.com/terraform) are the official classes produced by HashiCorp - [https://acloudguru.com/search?s=terraform](https://acloudguru.com/search?s=terraform) - [https://www.pluralsight.com/courses/terraform-getting-started](https://www.pluralsight.com/courses/terraform-getting-started) - [https://www.youtube.com/watch?v=wgzgVm7Sqlk](https://www.youtube.com/watch?v=wgzgVm7Sqlk) ## Troubleshooting ### **Prompt**: `Do you want to migrate all workspaces to "s3"?` If you get this message, it means you have local state (e.g. a `terraform.tfstate` file) which has not been published to the S3 backend. This happens typically when the backend was not defined (e.g. `backend.tf.json`) prior to running `terraform init`. :::caution **WARNING** This will overwrite any state currently in S3 for this component. If you were not expecting the state to be completely new, this prompt is unexpected. Working with any existing component shouldn't involve migrating a workspace and further investigation is warranted. ::: As far as I know, this shouldn't involve migrating a workspace, since this state should be completely new. Should I say yes? Is this just a misleading warning? or indicative that I'm about to mess something up? ## Reference - [AWS Region Codes](/resources/adrs/adopted/use-aws-region-codes/) - [Structure of Terraform S3 State Backend Bucket](/layers/accounts/tutorials/terraform-s3-state) --- ## Helm(Legacy) Helm makes it easy to install `charts` (an application) on kubernetes clusters. Just like `npm` or `apt` make it easy to install NodeJS modules and Debian packages, `helm` makes it easy to deploy a full-fledged application architecture with all of its dependencies on Kubernetes. It’s one of the original tools to deploy applications to Kubernetes and has the widest support amongst tools. One of the problem though is that something still needs to define the `values` you need to deploy a Helm release. That’s why there are other tools (e.g. terraform, helmfile, and argocd) required in order to integrate helm with your environment. ## Alternatives - Kustomize [https://github.com/kubernetes-sigs/kustomize](https://github.com/kubernetes-sigs/kustomize) - Ksonnet (deprecated) [https://github.com/ksonnet/ksonnet](https://github.com/ksonnet/ksonnet) - Kapitan [https://github.com/deepmind/kapitan](https://github.com/deepmind/kapitan) ## How-to Guides - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) ## Charts Charts are the deployable artifact used to create helm releases. ### Our Charts Cloud Posse maintains a dozen or so charts. In general, we prefer _not_ to maintain any charts and our goal is to leverage as many open source charts as possible. Our charts are available via `https://charts.cloudposse.com` and browsable via [https://artifacthub.io/packages/search?repo=cloudposse&sort=relevance&page=1](https://artifacthub.io/packages/search?repo=cloudposse&sort=relevance&page=1) . [https://github.com/cloudposse/charts](https://github.com/cloudposse/charts) ### Open Source Charts There’s been a lot of churn in the open-source community regarding Helm Charts. Historically, the `helm/charts` repo was the “official” source of charts managed by the community at large. Maintenance became a serious problem and bottleneck, so the repo has been deprecated. [https://helm.sh/blog/charts-repo-deprecation](https://helm.sh/blog/charts-repo-deprecation) [https://github.com/helm/charts](https://github.com/helm/charts) Today, the best place to search for charts is via [https://artifacthub.io/](https://artifacthub.io/) , which is like the “DockerHub” for helm charts. We also really like the well-maintained charts from Bitnami. [https://github.com/bitnami/charts](https://github.com/bitnami/charts) ## Conventions ### Charts as an Interface Typically, in a service-oriented architecture (SOA) aka microservices architecture, there will be dozens of very similar services. Traditionally, companies would develop a “unique” helm chart for each of these services. In reality, the charts were generated by running the `helm create` ([https://helm.sh/docs/helm/helm_create/](https://helm.sh/docs/helm/helm_create/) ) command that would generate all the boilerplate. As a result, the services would share 99% of their DNA with each other (e.g. like monkeys and humans), and 1% would differ. This led to a lot of tech debt, sprawl, and copy & paste 🍝 mistakes. For proprietary apps deployed by your organization, we recommend taking a different tactic when developing helm charts. Instead, treat charts like an interface - the way you want to deploy apps to Kubernetes. Develop 1-2 charts based on the patterns you want your developers to use (e.g. microservice, batchjob, etc). Then parameterize things like the `image`, `env` values, `resource` limits, `healthcheck` endpoints, etc. Think of charts like developing your own Heroku-like mechanism to deploy an application. Instead of writing 2 dozen charts, maintain one. Make your apps conform to the convention. Push back on changes to the convention unless necessary. **What if we need more than one deployment (or XYZ) in a chart?** That’s easy. You have a few options: a) Deploy the same chart twice; b) Decide if as an organization you want to support that interface and then extend the chart; c) Develop an additional chart interface. **What if we want to add a feature to our chart and don’t want to disrupt other services?** No problem. Charts are versioned. So think of the version of a chart as the version of your interface. Every time you change the chart, bump the version. Ensure all your services pin to a version of the chart. Now when you change the version of the chart in your service, you know that your upgrading your interface as well. **What if we need some backing services?** Good question. You can still use the features of umbrella charts, and even feature flag common things like deploying a database backend for development environments by using a `condition` in the `requirements.yaml` that can be toggled in the `values.yaml`. _Pro-tip:_ Use [https://artifacthub.io/](https://artifacthub.io/) to find ready-made charts you can use. ``` - name: elasticsearch version: ^1.17.0 repository: https://kubernetes-charts.storage.googleapis.com/ condition: elasticsearch.enabled - name: kibana version: ^1.1.0 repository: https://kubernetes-charts.storage.googleapis.com/ condition: kibana.enabled ``` ## Usage ### Command Line Helm is conveniently operated from the command line using the `helm` cli. We provide a package in our `packages` repository for multiple versions of helm (`helm2` and `helm3`) so they can be concurrently installed using `dpkg-alternative` [https://github.com/cloudposse/packages/tree/master/vendor](https://github.com/cloudposse/packages/tree/master/vendor) ### Helmfile For many years, we relied on `helmfile` as the primary delivery tool for `helm`. Using `helmfile` we could declaratively express all the applications and their settings by the environment. Starting in 2021, we started shifting our focus on using `terraform` and/or ArgoCD for deployments as many of the features of `helmfile` are now adequately supported by them. [https://github.com/roboll/helmfile](https://github.com/roboll/helmfile) [https://github.com/cloudposse/helmfiles](https://github.com/cloudposse/helmfiles) [https://archive.sweetops.com/helmfile/](https://archive.sweetops.com/helmfile/) ### Terraform There are two predominant ways to deploy `helm` chart releases with terraform. The primary benefit to using `terraform` over any other tool like `helmfile` or ArgoCD, is it’s the ability to facilitate integration with other services like AWS IAM, S3, SAML providers, etc. For example, a very common requirement is to provision IRSAs needed by pods to perform certain operations in AWS. #### Terraform `helm` provider :::caution Out-of-the-box the `helm` provider does not detect any changes made to the cluster, helm charts or values. [https://github.com/hashicorp/terraform-provider-helm/issues/372](https://github.com/hashicorp/terraform-provider-helm/issues/372) This means that changes that are made manually in the cluster are not detected as drift. Unlike with helmfile, which performs a `diff` (using `helm-diff` plugin), the terraform `helm` provider does not. #### Mitigations - Use the `helm_template` data source ([https://registry.terraform.io/providers/hashicorp/helm/latest/docs/data-sources/template](https://registry.terraform.io/providers/hashicorp/helm/latest/docs/data-sources/template)) to render the helm release and deploy it with the new `kubernetes_manifest` resource ([https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/reference/manifest](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/reference/manifest)). Terraform will then be able to detect changes from the desired state. There’s one trade-off, however, and that’s there will no longer be any helm history on the server since the manifests will be managed directly by terraform. - Use the `repository_file` resource together with ArgoCD to write `Application` Custom Resource manifests for deployment. This has the benefit of coupling all the benefits of terraform’s ability to provision backing services, with ArgoCDs abilities to reconcile kubernetes state. ::: The `helm` provider works conveniently in terraform to deploy helm releases. [https://github.com/hashicorp/terraform-provider-helm](https://github.com/hashicorp/terraform-provider-helm) We use our `terraform-aws-helm-release` module to facilitate the most common deployment patterns that we use, including the ability to automatically provision the IAM roles required by the service. [https://github.com/cloudposse/terraform-aws-helm-release](https://github.com/cloudposse/terraform-aws-helm-release) As well as publish ready-made [Components](/components) for services like `external-dns`, `cert-manager`, `aws-load-balancer-controller`, and `argocd` (and many more) leveraging this pattern. [https://github.com/cloudposse/terraform-aws-components](https://github.com/cloudposse/terraform-aws-components) #### Terraform `helmfile` provider Usage of the `helmfile` provider is primarily advised for customers with a sizable investment into `helmfile`, but who also want to consolidate deployments with terraform. The `helmfile` provider simply wraps the `helmfile` command-line tool and ties into the terraform life cycle hooks for plan, apply and destroy. In order for it to work, it depends on multiple binaries being installed: `helm`, `helmfile`, and the `helm-diff` plugin. In addition, the `KUBECONFIG` must be set and pointed to an active session. ### ArgoCD TL;DR: Even if you use ArgoCD, you’ll still need something that manages ArgoCD itself, any integrations with other kubernetes clusters, GitHub, and your IdP. For this reason we deploy ArgoCD with terraform and your apps _optionally_ with ArgoCD. ## Learning Resources - [https://helm.sh/docs/learn/quickstart/](https://helm.sh/docs/learn/quickstart/) - [https://kube.academy/courses/helm-101](https://kube.academy/courses/helm-101) - [https://jfrog.com/blog/10-helm-tutorials-to-start-your-kubernetes-journey/](https://jfrog.com/blog/10-helm-tutorials-to-start-your-kubernetes-journey/) - [https://www.udemy.com/course/helm-package-manager-for-kubernetes-complete-master-course/](https://www.udemy.com/course/helm-package-manager-for-kubernetes-complete-master-course/) 💰 ## References - [https://helm.sh/docs/howto/charts_tips_and_tricks/](https://helm.sh/docs/howto/charts_tips_and_tricks/) - [Handling CRDs in Helm 3](https://github.com/helm/community/blob/main/hips/hip-0011.md) [https://github.com/kubernetes/helm](https://github.com/kubernetes/helm) --- ## Provision Status Page ## Problem Statuspage provides either a private or public endpoint that shows incidents relating to SLOs. With an internal page, developers and engineers within an organization can see any incidents that are affecting SLOs. With a public page, users can see these incidents as well, which means that if they are experiencing issues with the organization’s services, they can validate that the issue is indeed acknowledged and is being remediated. For Statuspage to work seamlessly, it needs to be updated it automatically. ## Solution :::tip Create Statuspage Components in relation to SLOs, integrate OpsGenie with Statuspage, and then create OpsGenie Integration Actions to tag Datadog Synthetics alerts when they come in from Datadog, such that Statuspage will create incidents for its corresponding components. ::: ### Use of Datadog Synthetics Alerts Datadog Synthetics are the preferable Datadog alerts to integrate with OpsGenie and Statuspage. This is because Datadog Synthetics are directly related to Datadog SLOs, and because Datadog Synthetics tests check for uptime, which is closer to any disruption in functionality that a user might experience, when compared to things such as Kubernetes pods being down (Datadog Monitors). This matches the “alert at the faucet” principle described in the [Google SRE Handbook](https://sre.google/sre-book/table-of-contents/). In order for these Datadog alerts to affect Statuspage components, a `cmp_[Component Name]:[Component Status]` tag must be set in the Datadog Synthetics. For Datadog Synthetics, the component status should be either `partial_outage` or `major_outage`. It’s safer to choose the former when in doubt, because usually these components are part of larger system, and hence their outage is only a partial outage. The Major Outage status in Statuspage can be set for a component by an operator manually, at their discretion, whenever it is necessary to convey a major service disruption. For example, within `datadog-synthetics/catalog/example-api.yaml`: ``` example-api: name: "Example API" message: "Example API is not reachable" type: api subtype: http tags: managed-by: Terraform tenant: conch "cmp_Example API": partial_outage ... ``` ### Integrate OpsGenie with Statuspage Follow the official [documentation for integrating OpsGenie and Statuspage](https://support.atlassian.com/opsgenie/docs/integrate-opsgenie-with-statuspage/). This enables bi-directional communication between OpsGenie and Statuspage. Once the two are integrated, configure the Statuspage Integration as follows: This step is done via “ClickOps”, because there is no Terraform resource for managing the Statuspage integration. The aforementioned settings allow for the Statuspage incident to be updated when the alert is modified on the OpsGenie side. Once the alert is closed in OpsGenie, the corresponding event is resolved in Statuspage. ### Create OpsGenie Integration Actions for Datadog Synthetics Alerts In order for the Statuspage integration to function correctly, Integration Actions must be in place to clean up the alert message and description, and also prevent Datadog from inserting a large number of tags (in particular, since we are ) into the alert that will cause the tag count limit to be exceeded and preventing the Statuspage integration This should not be manually configured via “ClickOps”, and is instead configured using the `opsgenie-team` component (`integration` submodule). --- ## Authenticate with AWS inside of Geodesic using 'aws-vault' (Deprecated) ## Intro In this how-to, we'll provide a step-by-step guide on how we used to recommend authenticating with AWS inside of Geodesic, using `aws-vault`. This remains for historical reference and for companies who have been using `aws-vault` for a while and want to train new users. **Cloud Posse no longer recommends this workflow. See** [Authenticate with AWS using Leapp](/resources/legacy/howto/geodesic/authenticate-with-leapp) **for the currently recommended procedure.** ## Prerequisites ### System Requirements You'll need to have an AWS Account with the ability to create credentials and have [Docker installed](https://docs.docker.com/get-docker/) on your local machine. **That's all**. ### SweetOps Know-how We expect you've gone through the tutorial on ["Getting started with Geodesic"](/resources/legacy/tutorials/geodesic-getting-started) prior to this How-To since that contains some important understanding of what Geodesic is, how it works, and what it's doing. ## How-To ### 1. Install and start Geodesic First, at your terminal, let's install Geodesic! ```bash docker run cloudposse/geodesic:latest-debian | bash ``` This will install Geodesic as a script at `/usr/local/bin/geodesic`, which you can then invoke like so: ```bash geodesic ``` This small script is just a wrapper around running Geodesic via `docker` with a couple extra options (like mounting `$HOME` to `/localhost`). It should drop you into the Geodesic shell like you would expect. ### 2. Authenticate with AWS + aws-vault Geodesic ships with [`aws-vault`](/learn/toolchain/#aws-vault) to help manage our credentials and retrieve access tokens from AWS to provide us with authenticated sessions. To set up a new profile, first [create a new IAM user and programmatic Access Key ID and Secret Key](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html#id_users_create_console) and be sure to copy those values down somewhere. Now, in your Geodesic shell, let's do the following: ```bash # Since our Geodesic shell is on Linux, let's use the file backend which Linux supports. # NOTE: It's not possible to use the OSX Keychain with `aws-vault` running inside of a Docker container. export AWS_VAULT_BACKEND=file # Add our new credentials to aws-vault under whatever profile name you would like. i.e. replace $YOUR_PROFILE_NAME # This will prompt you for the Access Key ID and Secret Key that you copied down earlier, which you should input. aws-vault add $YOUR_PROFILE_NAME ``` Now we've added our credentials to `aws-vault` and we can easily use `aws-vault exec` to execute an authenticated command on the AWS CLI like so: ```bash # List all the buckets in your account: aws-vault exec $YOUR_PROFILE_NAME -- aws s3 ls # Or get some information on your user: aws-vault exec luke.skywalker -- aws sts get-caller-identity ``` ### 3. Start a AWS Profile Session That's cool... but what about if you want to start a full blown session as the profile you added so you don't need to keep typing `aws-vault exec`? Well, Geodesic comes bundled with a handy `assume-role` utility that you can use to do that. First though, we need to update our `/localhost/.aws/config` file to support `aws-vault`. To do that, on your local machine, open up `$HOME/.aws/config` with your favorite editor and add the following config entry: ```properties # ... your existing AWS Config file contents ... [profile $YOUR_PROFILE_NAME] credential_process = aws-vault exec $YOUR_PROFILE_NAME --json ``` Great, now that we've set that up, our profile is ready to use with our `assume-role` utility: ```bash # Now we run `assume-role` with our newly created profile and this will start a new shell session which is authenticated as that profile for us. assume-role $YOUR_PROFILE_NAME # We should now see our profile name in our command line prompt and we can now run our AWS CLI # commands without having to manually invoke `aws-vault exec` each time aws s3 ls aws sts get-caller-identity ``` ## Conclusion Here is a simple use-case of setting up a set of AWS credentials within Geodesic, but there is plenty more you can do here since Geodesic also comes bundled with [`saml2aws`](https://github.com/Versent/saml2aws) and [`aws-google-auth`](https://github.com/cevoaustralia/aws-google-auth). Use what works for you and your organization! --- ## Authenticate with AWS using Leapp ## Intro In this how-to, we will help you get started using Geodesic to work with AWS resources by helping you set up and use [Leapp](https://leapp.cloud) to handle credentials and authentication. Leapp is an open-source tool that makes this easier. ## Prerequisites ### SweetOps Know-how We expect you've gone through the tutorial on ["Getting started with Geodesic"](/resources/legacy/tutorials/geodesic-getting-started) prior to this How-To since that contains some important understanding of what Geodesic is, how it works, and what it's doing. ### AWS Region and Credentials #### Region It will be helpful to know which AWS Region you will be primarily working in. This should be fairly common knowledge among people with AWS access at your company. If you are doing this on your own, choose the region closest to you geographically. If you are still in doubt, pick - `ca-central-1` if in Canada, - `us-east-2` if in the United States east of the Mississippi, - `us-west-2` if anywhere else in the North America or anywhere in Central America, - `sa-east-1` if in South America, - `eu-north-1` if in Europe, - `af-south-1` if in Africa, - `ap-south-1` if in Asia, or - `ap-southeast-2` if in Australia or Antarctica. #### Credentials You will have to have some way of authenticating to AWS already set up, and you need to know the nature of your credentials. Whoever gave you access to AWS should be able to tell you what kind of access you have. It should be one of the following: - **Federated Role** (recommended). This means you log into something you use to identify yourself to a lot of places (usually referred to as Single Sign-On or SSO), and it is set up to log you into AWS with a specific IAM Role as well. At a company, this is usually Google Workspaces (formerly GSuite), Okta, or Active Directory. This is also known as a SAML IdP. - **AWS SSO**. This means your company has configured AWS as a Single Sign-On provider. This is AWS _as_ a Single Sign-On provider, allowing you to access multiple _permission sets_ within AWS, not using some other Single Sign-On provider to sign in to AWS as a single IAM Role. Please note that even if your company has set up AWS _as_ a Single Sign-On provider, you still may be using your company's primary SSO provider to authenticate to AWS SSO. - **AWS IAM User**. This is the older way of authenticating to AWS, with a basic username and password to log into the AWS console, and a long-lived "Access Key" for API access. If you are going to use this method, we strongly recommend that you enable multi-factor authentication (MFA). Based on which kind of credentials you have, you will need to gather different information with which to configure Leapp. Whoever is in charge of setting up your access to AWS should be able to give you the information you need. ##### Federated Role If using Federated Login, you will need - Your IdP Single Sign-On URL. - For Google Workspaces, it looks something like `https://accounts.google.com/o/saml2/initsso?idpid=C0asdfasdfal&spid=12344321`. - For Okta, it looks something like `https://company.okta.com/app/company_samlidp_1/Hka1abcke6h4P1WQr5d7/sso/saml`. - Most importantly,it should not be confused with the AWS Single Sign-On URLs like these (**do not use these**) - `https://signin.aws.amazon.com/saml` (this is used by your IdP admin) - `https://my-sso-portal.awsapps.com/start` (this means you are using AWS SSO) - The AWS Identity Provider ARN assigned by AWS to your IdP. Something like - `arn:aws:iam::123434211234:saml-provider/company-name` - Your assigned/authorized AWS IAM Role ARN. Something like - `arn:aws:iam::123434211234:role/role-name` ##### AWS SSO If using AWS SSO, you will need: - Your AWS SSO "start URL", also known as your "portal URL". It should be very close to: - `https://something.awsapps.com/start` - The region in which AWS SSO has been deployed. This may or may not be the same region you will be working in. ##### AWS IAM User If you are a regular IAM User who can log into the AWS Console, you should log into the AWS Console while setting up Leapp. Choose "My Security Credentials" from the Account drop-down menu: - Copy and paste your MFA ARN or Serial Number from "Assigned MFA device" - Click on "Create access key" to create a new access key and copy the Access Key ID and Secret Access Key from the console and paste directly into the appropriate fields in Leapp. ## How-To ### Start Geodesic If you are a Cloud Posse client, you will have a customized version of Geodesic that sets an initial value for the environment variable `AWS_PROFILE`. Alternatively, you may have customized it yourself, or you may want to. In any case, we will go with what you have. So start Geodesic, and at the command prompt, type ```bash echo $AWS_PROFILE ``` This is the value you will give to the profile in Leapp when you configure it. If the output of `echo` is blank, as would be expected if you are running our public tutorial image, use "default" for the profile name. ### Install and Configure Leapp Please refer to the official Leapp documentation for the latest instructions on installing and configuring Leapp. Now that you are armed with the information from the previous steps, it should be pretty easy. - Visit the [Leapp website](https://leapp.cloud) - Download and install Leapp as instructed by the website - Follow the instructions (under "Docs") for configuring AWS Below is some guidance to the Leapp documentation that applies as of the time of this writing. By the time you read this it may be out of date, but hopefully will still be of help in guiding you through the Leapp site. ##### Key Points - The "AWS Profile" setting in Leapp must match _exactly_ the value of `$AWS_PROFILE` you found in Geodesic in the earlier step. - The "AWS Region" you set in the Leapp session should be the AWS Region you most often use, as discussed [above](#aws-region-and-credentials). - The "Session Alias" is completely up to you: it is the name for this set of credentials that you will see in the UI. The Leapp documentation is at [https://docs.leapp.cloud/](https://docs.leapp.cloud/) and the best set of instructions to follow are the ones under a sub-menu on the left side of the page: **Tutorials > AWS** - If you have a **Federated Login**, pick "AWS IAM Federated Role". Most of the tutorial is about how to configure Federated Login itself, and you can skip all that. Just follow the last step: "Setup in Leapp". - Otherwise, pick "AWS SSO" or "AWS IAM User" and follow the steps. ### Log in using Leapp Leapp provides 2 UIs for logging into or out of a session. There is a system-wide menu item in the taskbar on Windows systems or as a "status menu" on the Mac menu bar. Click on it and a menu will appear with all your configured session names and their corresponding profile names. An icon for each session will appear either in gray if logged out or orange if logged in (at least those are the current defaults for the "Light" color scheme on macOS). Just select the menu item to toggle the state. Alternately, you can select "Show" from the menu and be shown a richer UI where you can do more, but still the main thing is that you click on a session name to change its state and the indicator shows gray for logged out and orange for logged in. If you are not using IAM user access keys then in order to access AWS you will have to log in to your "identity provider" (e.g. Okta, Google) like you do for access to other resources. Therefore, when you try to activate a session in Leapp, it may open a small web browser window popup or open a window or tab in your default browser so you can complete the login flow, including supplying a second factor if you are using MFA, and perhaps solving a CAPTCHA puzzle. This is normal and expected behavior, not a virus or hacking attempt. When you finish logging in via the web browser, Leapp uses the credentials provided by your identity provider to authenticate to AWS, just as if you were logging into the AWS web console or SSO portal. Like your web browser, Leapp can store cookies and other information, and in addition, Leapp is able to use your system keychain for secure storage of other things like your Secret Access Key. Because of this, and depending on your identity provider and their settings, simply being logged into your computer may be enough authentication for Leapp to grant you access when you enable a session without asking you for anything. However, AWS requires periodic refreshing of session keys and if you are using MFA with AWS, you should enable Leapp notifications so you can receive prompts when your AWS session expires and Leapp needs a new MFA token in order to start a new session for you. ### It should just work Once you log in to AWS using Leapp, go back into Geodesic and simply hit return at the prompt. The prompt should change from having a red ✗ and `[none]` to having a green √ and `[default]` or whatever your profile name is. This is Geodesic's way of letting you know you are authorized with AWS, and what profile you have active. You can always confirm your current authentication status by running: ```bash aws sts get-caller-identity ``` When you are properly authenticated, it will output your IAM role and other information. When you are logged out, it will print some kind of error message, usually: ```text Unable to locate credentials. You can configure credentials by running "aws configure". ``` --- ## Updating Modules for Terraform 0.14 ## Updating Cloud Posse Terraform modules for Terraform 0.14 We have some tools you can use for updating our Terraform modules to support Terraform 0.14. This should work for any Terraform modules we have published that currently support Terraform 0.12 or later and contain a `context.tf` file. - Check out/fork the current version of the Terraform module - Create a `terraform-0.14-upgrade` branch and check it out: `git checkout -b terraform-0.14-upgrade` - Run the update: `make tf14-upgrade` - **Important**: Manually verify the changes to the Terraform code. The scripts are not foolproof. - *Optional:* Update any internal label modules to refer to version `0.24.1` of `cloudposse/label/null` - If you have made any changes, run `make pr/auto-format` to rebuild the README - Check in the changes and open a PR request with the name "Terraform 0.14 upgrade" ``` ## what - Upgrade to support Terraform 0.14 and bring up to current Cloud Posse standard ## why - Support Terraform 0.14 ``` - *Optional:* If the core Terraform provider version did not change, label the PR with "patch" - Not all of our repositories have a "patch" label configured. If you need to create the label, here is the configuration: - name: "patch" - description: "A minor, backward compatible change" - color: "#0e8a16" - *If you are a Cloud Posse Contributor:* Run the tests by adding a comment "/test all" to the PR - Fix problems causing tests to fail until they all pass - If `validate-codeowners` fails, you will not be able to fix it. Request a review from `cloudposse/admins` - Post a link to the PR in the #pr-reviews channel of the [SweetOps Slack](https://slack.cloudposse.com/) --- ## Getting Started(Legacy) ## Geodesic Start with getting familiar with the [geodesic](/learn/toolchain/#geodesic). Get intimately familiar with docker inheritance and [multi-stage docker builds](/best-practices/docker#multi-stage-builds). We use this pattern extensively. Check out our [terraform-aws-components](https://github.com/cloudposse/terraform-aws-components) for reference architectures to easily provision infrastructure. ## Tools Tons of tools/clis are used as part of our solution. We distribute these tools in a couple of different ways. * Geodesic bundles most of these tools as part of the geodesic base image * Our [packages repo](/learn/toolchain/#packages) provides an embeddable `Makefile` system for installing packages in other contexts (e.g. [`build-harness`](/resources/legacy/tools#build-harness)). This can also be used for local ("native") development contexts. Here are some of the most important tools to be aware of: - [`make`](/learn/toolchain/#make) - [`chamber`](/learn/toolchain/#chamber) - [`terraform`](/learn/toolchain/#terraform) - [`gomplate`](/learn/toolchain/#gomplate) - [Leapp](/learn/toolchain/#leapp) If using kubernetes, then also review these tools: - [`helm`](/learn/toolchain/#helm) - [`helmfile`](/learn/toolchain/#helmfile) ## Kubernetes Kubernetes is a massive part of our solutions. Our Kubernetes documentation is geared towards leveraging our [Terraform EKS modules](https://github.com/cloudposse?q=terraform-aws-eks) and [`helmfile`](/learn/toolchain/#helmfile). ### Helm Helm is central to how we deploy all services on kubernetes. * [helm](/learn/toolchain/#helm) is essentially the package manager for Kubernetes (like `npm` for Node, `gem` for Ruby, and `rpm` for RHEL) * [helm charts](https://helm.sh/docs/topics/charts/) are how kubernetes resources are templatized using Go templates * [helmfiles](/learn/toolchain/#helmfile) are used to define a distribution of helm charts. So if you want to install prometheus, grafana, nginx-ingress, kube-lego, etc, we use a `helmfile.yaml` to define how that's done. ## Terraform Study up on our [Best Practices](/best-practices/terraform) for working with terraform. Get started quickly provisioning infrastructure by referencing our [terraform-aws-components](https://github.com/cloudposse/terraform-aws-components). ### Terraform Modules [We provide a staggering number of Terraform modules in our GitHub](https://github.com/cloudposse?q=&type=&language=hcl). This number is growing every week and we're also [accepting module contributions](/community/contribute/our-github#contributing). Before writing your own modules, review our [Best Practices](/best-practices/terraform) for working with Terraform modules. ## Contributing Back Everything we provide on our [GitHub](https://github.com/cloudposse/) wouldn't have been possible if it weren't for our [phenomenal customers](https://cloudposse.com/) and the support of the [community](https://cloudposse.com/slack/) contributing bug-fixes, [filing issues](https://github.com/search?q=org%3Acloudposse+type%3Aissue) and submitting a steady stream of [Pull Requests](https://github.com/search?q=org%3Acloudposse+type%3Apr). We welcome any Terraform module submissions, Helm charts, and generally any other useful tools that others could benefit from. Our only requirement is that they be licensed under `APACHE2`. Drop us a line at [hello@cloudposse.com](mailto:hello@cloudposse.com) to get in touch with us about contributing. ## Getting Help Review our [glossary](/resources/glossary/) if there are any terms that are confusing. File issues anywhere you find the documentation lacking by going to our [docs repo](https://github.com/cloudposse/docs). Join our [Slack Community](https://cloudposse.com/slack/) and speak directly with the maintainers. We provide "white glove" DevOps support. [Get in touch](/community/contact-us) with us today! [Schedule Time](https://calendly.com/cloudposse/) with us. --- ## Kubernetes(Legacy) ## Design Decisions - [Decide on EKS Node Pool Architecture](/layers/eks/design-decisions/decide-on-eks-node-pool-architecture) - [Decide on Kubernetes Ingress Controller(s)](/layers/eks/design-decisions/decide-on-kubernetes-ingress-controller-s) - [Decide on Host OS Flavor for EKS](/layers/eks/design-decisions/decide-on-host-os-flavor-for-eks) - [Decide on Kubernetes Platform Compliance Strategy](/layers/security-and-compliance/design-decisions/decide-on-kubernetes-platform-compliance-strategy) - [Decide on Kubernetes Application Artifacts](/layers/software-delivery/design-decisions/decide-on-kubernetes-application-artifacts) ## Related Guides - [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date) - [How to use Datadog Metrics for Horizontal Pod Autoscaling (HPA)](/layers/monitoring/datadog/tutorials/how-to-use-datadog-metrics-for-horizontal-pod-autoscaling-hpa) ## Related Components - [eks/cluster](/components/library/aws/eks/cluster/) ## Learning Resources - Start by checking out this long blog post which is the best introduction, summary, and reference cheat sheet on Kubernetes we have yet to find. Reviewing this will shave weeks off the learning curve. [http://emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html](http://emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html) - [http://kubernetesbyexample.com/](http://kubernetesbyexample.com/) - [http://learnk8s.io/blog/kubectl-productivity](http://learnk8s.io/blog/kubectl-productivity) - [https://aws.github.io/aws-eks-best-practices/](https://aws.github.io/aws-eks-best-practices/) ## FAQ ### What’s the difference between Controllers and Operators? [https://stackoverflow.com/questions/47848258/what-is-the-difference-between-a-kubernetes-controller-and-a-kubernetes-operator](https://stackoverflow.com/questions/47848258/what-is-the-difference-between-a-kubernetes-controller-and-a-kubernetes-operator) ## Managing CRDs For operators of any Kubernetes platform, it’s critical that you are aware of all the pitfalls of using CRDs. Cloud Posse 💯 supports the use of and loves to use operators (E.g. `external-dns`, `cert-manager`, `alb-load-balancer-controller`, `istio`, etc), they come with a big warning. While [this document is by the Helm community](https://github.com/helm/community/blob/main/hips/hip-0011.md), the problems are not in any way unique to Helm. ## References - [Demo Applications](/resources/legacy/demo-applications) - [Lens](/resources/legacy/lens) - [Helm](/resources/legacy/helm) --- ## Learning Resources import ReactPlayer from 'react-player' ## Terraform If you’re new to terraform, here are a number of resources to check out: - [https://learn.hashicorp.com/terraform](https://learn.hashicorp.com/terraform) are the official classes produced by HashiCorp - [https://acloudguru.com/search?s=terraform](https://acloudguru.com/search?s=terraform) - [https://www.pluralsight.com/courses/terraform-getting-started](https://www.pluralsight.com/courses/terraform-getting-started) - [https://www.youtube.com/watch?v=wgzgVm7Sqlk](https://www.youtube.com/watch?v=wgzgVm7Sqlk) ## Kubernetes - Start by checking out this long blog post which is the best introduction, summary, and reference cheat sheet on Kubernetes we have yet to find. Reviewing this will shave weeks off the learning curve. [http://emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html](http://emprovisetech.blogspot.com/2018/12/kubernetes-container-orchestration-at.html) - [http://kubernetesbyexample.com/](http://kubernetesbyexample.com/) - [http://learnk8s.io/blog/kubectl-productivity](http://learnk8s.io/blog/kubectl-productivity) - [https://aws.github.io/aws-eks-best-practices/](https://aws.github.io/aws-eks-best-practices/) ## Helm - [https://helm.sh/docs/learn/quickstart/](https://helm.sh/docs/learn/quickstart/) - [https://kube.academy/courses/helm-101](https://kube.academy/courses/helm-101) - [https://jfrog.com/blog/10-helm-tutorials-to-start-your-kubernetes-journey/](https://jfrog.com/blog/10-helm-tutorials-to-start-your-kubernetes-journey/) - [https://www.udemy.com/course/helm-package-manager-for-kubernetes-complete-master-course/](https://www.udemy.com/course/helm-package-manager-for-kubernetes-complete-master-course/) 💰 ## Helpful Videos Here are some helpful videos/outtakes from our weekly Office Hours. We call this series “Cloud Posse Explains” and we have [a playlist on YouTube we regularly update](https://www.youtube.com/playlist?list=PLhRztDM6Uvnf40vPxs9nP09ZTr7tQQ96_) (with [RSS feed](https://www.youtube.com/feeds/videos.xml?playlist_id=PLhRztDM6Uvnf40vPxs9nP09ZTr7tQQ96_)) ## The 4 Layers of Infrastructure ## The Difference between Vanity Domains and Service Discovery Domains ## The `context.tf` Pattern ## The Geodesic Docker Toolbox --- ## Legacy Documentation import Intro from '@site/src/components/Intro'; import DocCardList from '@theme/DocCardList'; This section contains some documentation we've kept around for historical purposes. This documentation is no longer maintained and is out of date. --- ## Lens import ReactPlayer from 'react-player' # Lens Download from [k8slens.dev](https://k8slens.dev/) or install the [Homebrew formula](https://formulae.brew.sh/cask/lens#default). ### Generating a Cluster’s `kubeconfig` Once Lens is downloaded, it needs a `kubeconfig` file to authenticate with a server. Using a Geodesic shell, run `set-cluster [cluster-name]` to generate a `kubeconfig` in `/conf/.kube`. The file will be named after the cluster and current user’s role; e.g. `set-cluster platform-use2-dev` will generate a file named like `/conf/.kube/kubecfg.platform-use2-dev-admin`. The easiest way to access this cluster from Lens is to copy the `kubeconfig` to the current system user's `~/.kubeconfig` folder which is represented as `/localhost/.kubeconfig` inside of the Geodesic shell. #### Example ``` set-cluster platform-use2-dev cp /conf/.kube/kubecfg.platform-use2-dev-admin /localhost/.kubeconfig/ ``` :::note The filename does not include specific context to the project. They can be renamed on the local user’s system as needed, if naming conflicts arise. Example: `mv /localhost/.kubeconfig/kubecfg.platform-use2-dev-admin localhost/.kubeconfig/kubecfg.namespace-platform-use2-dev-admin` where `namespace` is a relevant, distinguishable attribute for that config. ::: ### Importing `kubeconfig` On the Catalog view, click the `+` in the bottom corner and select the config file imported in the previous step. The configs can also be synced by hovering over the `+` button and clicking “Sync kubeconfig(s)” button. ### Connecting to a Cluster Click cluster within the list on the specific cluster. :::note Click the pin icon to place the cluster in the sidebar. ::: Once the cluster connects, the following screen will be available to use. --- ## Operational Readiness **WORK IN PROGRESS** This is an estimation of the operational readiness required to claim ownership over the systems and processes. Depending on your roles and responsibilities, the level of ownership changes. :::info If you’re new to all of this, you will want to start with [Choose Your Path](/intro/path/). ::: ## Organization As an organization, you’ve conducted the following - Production Readiness Review (PRR) [https://sre.google/sre-book/evolving-sre-engagement-model/#the-prr-model-DVsGhWZ](https://sre.google/sre-book/evolving-sre-engagement-model/#the-prr-model-DVsGhWZ) - Roles, Responsibilities, & Ownership (RACI) for all [Components](/components) in-use - Ensure you’ve validated all playbooks (especially important for certain compliance frameworks) - You have established SLOs in Datadog for all mission-critical business functions and educated the teams on their value - You are aware of your AWS spend and have budgets in place - You have a process in place for managing all SaaS/Vendor relationships, enterprise agreements - You have a formalized incident response process for Application, Infrastructure and Security - You have a post-mortem review process ## Developer - You are comfortable with adding new services to the CI/CD pipeline - You are leveraging preview environments (if applicable) as part of your development workflow - You know where to look for logs emitted from your service - You know how to add telemetry to your services - You are able to debug/triage services without direct access to the cluster or pods - You are able to login to AWS using Leapp and experiment in the sandbox account - You are familiar with the [https://cloudposse.com/12-factor-app/](https://cloudposse.com/12-factor-app/) - You have developed a `Dockerfile` for your service(s) and understand what every line in that file does - You have developed a [Helm](/resources/legacy/helm) chart (if you’re using EKS) ## Operations - You are able to debug/triage services without direct access to the cluster or pods - You are able to find the logs for services and filter for what you need - You are comfortable upgrading EKS with Addons - You know how to develop (and have developed) a new terraform component and add them to stacks - You can explain the core [Concepts](/resources/legacy/fundamentals/concepts) and use them as part of your design process - You are current on all of the [Conventions](/learn/conventions) used - You are familiar with all the backup policies and how to restore backups - You are familiar with the upgrade process for all managed services (E.g. RDS, MSK, etc) - You are regularly performing upgrades to keep everything current (see [How to Keep Everything Up to Date](/learn/maintenance/upgrades/how-to-keep-everything-up-to-date)) - You are using Spacelift on a daily basis as part of your job - You are able to define new IAM roles and configure/manage AWS SSO/SAML - You are able to authenticate with all required systems (E.g. AWS, Datadog, OpsGenie, Spacelift). - You are familiar with the DNS architecture and how to work with vanity domains and service discovery domains ## SRE - You are able to debug/triage services without direct access to the cluster or pods - You are able to authenticate with all required systems (E.g. AWS, Datadog, OpsGenie, Spacelift). - You are able to open up AWS support tickets as needed - You are able to open up support tickets with all external vendors - You are able to open up internal support tickets and know how to triage them - You are aware of and help maintain SLOs - You are writing RCA and post mortems for incidents - You are able to develop new monitors, dashboards, SLOs, synthetics in Datadog - You are able to configure incident management in OpsGenie IaC configurations to control escalation paths ## Security - You are able to ingest cloudtrail logs and identify, classify and attribute events - You are familiar with the IAM architecture and implementation of custom roles - You are aware of and able to manage AWS SSO or AWS Federated IAM with integrations to your IdP - You know where to look and are able to respond to events surfaced by AWS SecurityHub, GuardDuty, etc ## Release Engineer - You are able to debug/triage workflows - You are able to tune and optimize Github Actions (if applicable) ## QA - --- ## GOST Framework ![GOST Framework](/assets/3feff88-ghost.png) The GOST Framework is a method of clearly articulating a process for achieving a goal. It does this by decomposing it into it's various parts: [G]oals, [O]bjectives, [S]trategies, [T]actics. It helps distinguish between the objectives, strategies and tactics so that relevant stakeholders can focus on what's relevant to them. ## Goals A goal defines a broad primary outcome. This is the business driver and should relate to some specific benefit to the business. Strongly recommend defining a single goal. **Example**: "Improve user productivity for most popular pages across our site" :::tip The goal should not include any objectives, strategies or tactics. It's very high-level and should be easily understood by anyone in the business. ::: ## Objectives List of measurable outcomes that will be achieved by executing the strategies. Objectives describe *what* will be achieved by the goal. This is the barometer for success. It's how we will know we've achieved our goal. **Example**: "Reduce page load times by 25%" :::tip Typically, these objectives are described using the following terms: "minimize", "increase", "reduce", "eliminate". ::: ## Strategies The specific approach that will be taken to achieve a goal and objectives. They describe *why* the objectives will be met, therefore try to map all the strategies to the objectives. **Example**: "Use caching to reduce page load times" :::tip Strategies should not encompass any tactics. That is, there might be multiple tacts that can be executed to fulfill the strategy. This is geared towards the CIO/CTO audience that might not be well versed on the tactics. For example, "Use Caching to reduce page load times" ::: ## Tactics Tactics describe *how* specific tools will be used to implement the strategies. Try to map the tools to the strategies. **Example**: "Use Memcache to cache API responses" :::tip Tactics describe the exact solution that will be implemented by the engineering team. Typically, this is a list of technologies (E.g. MySQL, Memcache, Kubernetes, Helm, Docker, etc) ::: --- ## Managers vs Makers ![Paul Graham](/assets/e5ae645-paul_graham.png) ## TL;DR Developers (Makers) and Managers work differently. - **Makers** plan in half-day blocks of time. These are developers. - **Managers** plan to minimize empty 15-minute slots in their calendar. Interrupts are costly for developers and therefore the business. http://www.paulgraham.com/makersschedule.html (Paul Graham - YCombinator Founder) --- ## RACI Matrix The [RACI Responsibility assignment matrix](https://en.wikipedia.org/wiki/Responsibility_assignment_matrix) is used to clarify the participation by various roles in completing tasks or deliverables as it relates to projects or teams. * **Responsible:** Those who perform or recommend the work to achieve the task. * **Accountable:** The one ultimately answerable for the correct and thorough completion of the deliverable or task, and the one who delegates the work to those responsible. They are required to sign off (approve) work that the Responsible party provides. There must be only one accountable party specified for each Role or Responsibility. * **Consulted:** Those whose opinions are sought, typically Subject Matter Experts (SME) and with whom there is two-way communication. * **Informed:** Those who are kept up-to-date on progress, often only on completion of the task or deliverable; and with whom there is just one-way communication. --- ## Remote Developer Environments: Comparative Analysis of Tools ### Overview Comparative Analysis of Skaffold, Garden, and Telepresence with Helmfile :::info [Tilt.dev](https://tilt.dev/) was not evaluated because it is primarily focused on building and deploying to a _local_ kubernetes cluster. While a local kubernetes cluster can work for a local environment it often consumes the majority of the developer’s resources on their computer. This makes other tasks more difficult. Other tools Such as Garden, Skaffold, and Telepresence, focus on deploying to remote clusters but could support a local cluster if desired. ::: ### Remote Environment | **Requirement** | **Skaffold** | **Garden** | | ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **single application deployment** | `skaffold deploy` Skaffold makes it very easy to build and/or deploy a single application. Ran from app repository. | `garden dev my-app` Builds Tests and Deploys `my-app` where it is defined in the project. **Requires directory structure or linked folders (more later)** | | **dependency deployment - DBs, Helper Containers** | Can be defined in the skaffold.yml as an additional release (or profile) or in the app chart dependencies. Must exist as a chart. can deploy helm charts from | each **module**[[1](https://docs.garden.io/using-garden/modules)] can contain a depends on section. This means any number of dependencies can be added which include Terraform, Helm Charts, Kubernetes Manifests, or Docker containers. Can point to modules from remote sources | | **deploy group of applications** | Can be defined similarly as above through a profile as a set of releases, or through dependencies in the helm chart. Groups can also be defined through remote skaffold configurations which can live remotely and contain groups of releases [[2](https://skaffold.dev/docs/design/config/#remote-config-dependency)]. Other configs can be local paths as well - meaning no commit hassel [[3](https://skaffold.dev/docs/design/config/#local-config-dependency)]. | `garden dev` (or `deploy`) will deploy all modules in a project. This means an entire environment could be deployed in one command, a `--skip module-b` exists to help manually not deploy particular modules. | | **application versioning by environment** | With additional config dependencies (more **skaffold.yml** files) could be committed to a repository and versioning could point to a branch to set the starting point for environments as the latest. A ref [[2](https://skaffold.dev/docs/design/config/#remote-config-dependency)] to another skaffold config would set other applications default versions. | Garden will build any module with a Dockerfile next to it. Deployment does not require a Helm chart, and will create kubernetes manifests to fit a docker image with configurations for port and ingresses. | | **application version overrides** | Overriding Other Applications (e.g. `dev` environment for all except `svc-a` and `svc-b` where `svc-a` is your local repo and `svc-b` is a dependency you’ve customized) would require a local directory set. Or you change the helm chart version that is fetched for that service. Note: It looks like you cannot override Skaffold Yamls fetched remotely. | Garden allows `link`ing a service, which overrides a remote configuration with a local directory. This allows you to use your local as a deployment for customized versioning. For a different published version modules allow you to specify a `repositoryUrl` and it will use the local configuration. | | **customize environment variables** | Environment Variables can easily be added once the helm chart forwards an env block of values yaml to the deployment definition. Skaffold allows easy overriding of default values.yaml files, and merging other values.yaml files. | This is especially easy in Garden, for modules deployed as a container type (has some default k8 deployments) you can add an `env` block to define key value pairs. For kubernetes or helm type modules it must again be specified in the helm chart or in the deployment env block. | | **New environment** | Using Skaffold Required Configs (Perhaps a config in an environment repository that has required configs in every microservice repository) we could define an entire environment and deploy it to a new environment with `skaffold deploy` | `garden deploy` will deploy and entire project, with a git controlled repository this would be extremely easy to stand up a clean new environment. | | **Teardown environment** | Similar to spinning up a new environment there is a command that tears down the environment based on the current configuration, which is `skaffold delete` | `garden delete environment` will tear down an environment. you can specify with `--env foo`A default environment can be specified for a project. It appears you could nest a `garden.yml` in a parent directory to override specific values. | | **Re-create environment** **(teardown and re-create environment bringing all versioned apps and dependencies back to a clean state)** | Skaffold does not natively have this. However with local configs and version overrides we could tear down an environment, respin it up, and override with our versioned apps. This would be ran via `skaffold deploy` | `garden delete env foo; garden deploy --env=foo` will reset an environment to a clean slate. This includes any linked local repositories. Since it is deploying it will rebuild and keep any changes to the code. | | **Environment integration** **(Connecting development environment to dependencies on other environments)** | Not natively supported. This would require custom Services and values overrides to point to k8 services in other namespaces. | Not natively supported. | ### Development Cycle | | **Skaffold** | **Garden** | | ------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Creating a new application** | Skaffold does not support creating a new application. The tool does not support this type of generation. However there are other tools that can help with this. | Garden does not support creating a new application, however it does have a very easy to use onboarding to create a new module (App, Tests, Tasks, Deployments) via the CLI that includes auto detection. | | **On-boarding an existing application** | Once a project exists. Github Workflows and Skaffold.yaml files would need to be added, these would likely need minor changes from other projects, such as project name. | Using `garden create module` a new module can be spun up pretty easily to be added to the current project. | | **Starting the development** | Skaffold.yaml can be configured so it always points to the running developers workspace (namespace). Simply running `skaffold dev` should then be able to build, test, and deploy their code to their workspace. This command includes hot reloading. | `garden dev` starts by deploying the environment and also starting up the garden dashboard, which lets you look at pod logs, dependency graph, and an overview, including accessible endpoints. | | **Hot reloading code** | Skaffold supports this by noticing file changes, it can either sync the files to the running container or can build a new image and redeploy it. | supported on deploy and dev with flags. `--watch # watch for changes to code``--hot=my-service` `--hot=*` | | **Hot reloading packages** | Skaffold allows explicitly declaring folders and file types that should be synced to a deployment. This is done automatically during the `dev` command [[https://skaffold.dev/docs/filesync/](https://skaffold.dev/docs/filesync/)] | Garden supports File Sync and Command Sync. Where when files change they will either be volume mounted or a command will be run. [[https://docs.garden.io/v/docs%252Fexperimental/guides/hot-reload#basic-example](https://docs.garden.io/v/docs%252Fexperimental/guides/hot-reload#basic-example)] | | **Debugging** | `skaffold debug` runs `skaffold dev` (build test deploy) but also forwards debugging ports. Currently Supports - NodeJS (runtime ID: `nodejs`) using the NodeJS Inspector (Chrome DevTools)- Java and JVM languages (runtime ID: `jvm`) using JDWP | No Native Support | | **Add/Change Environment Variables** | Requires upfront work - the helm chart update requires a block definition that can import a map of env variables from a values file. Skaffold can then override these variables very easily | Same as skaffold - can override helm deployment env block with values yamls. For kubernetes modules there is a directly supported `env` block in the `garden.yml` | | **Add/Change Environment Secrets** | Similar to Env Variable changes, Secrets would need to be deployed in a helm chart somewhere, skaffold can then deploy it and override variables | Done through Kubernetes secrets, Only supported on Enterprise level for management. | | **Helm Chart Changes** **(if we change SOPS, does the helm chart get reloaded)** | Skaffold `dev` allows rapid redeploy of helm charts. However if the helm chart syntax is broken the dev command will return non-zero exit code. This means pulling helm chart changes will be immediately deployed. | `helm` modules point to either a remote chart, or a local directory and would be redeployed through `garden deploy`. Kubernetes Modules and Container Modules deploy either set manifests or are auto detected. | :::caution **Telepresence Note** Telepresence V2 is tightly coupled with Ambassador, this could potentially cause issues when using other service meshes (e.g. Istio, Linkerd, Kong, etc). Telepresence V1 could be used, though it would not be worth the investment for a tool that is no longer supported. ::: --- ## Tune SpotInst Parameters for EKS ## Problem SpotInst (SaaS) replaces the need to deploy the [standard kubernetes cluster auto-scaler](https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler), so the mechanisms of tuning the scaling parameters are different than out-of-the-box EKS clusters. The autoscaling capability is handled by the `ocean-controller` together with the Spot platform, and not by Auto Scale Groups, so there’s no ASG parameters that we can toggle. Autoscaling is further constrained to by the number of spot instances of a given instance type in AWS as well as [Spot.io](http://Spot.io) has limits on your account. While SpotInst (aka [http://spot.io](http://spot.io) ) is responsible for efficiently managing Spot Instances, it can actually manage on-demand and reserved instances as well. Spot Instances can sometimes cause problems for workloads that are more sensitive to interruptions, such as CI/CD (GitHub Action Runners) and WebSockets. :::caution AWS Reserved Instances are not a “type” of instance like Spot. AWS Reserved Instances are a billing arrangement that reduces costs for on-demand instance types. So to leverage “reserved instances”, simply launch on-demand instances of the type you have purchased through the RI marketplace. [https://aws.amazon.com/ec2/pricing/reserved-instances/](https://aws.amazon.com/ec2/pricing/reserved-instances/) ::: ## Solution ### Option 1: Tune Pod Annotations See: [https://docs.spot.io/ocean/features/labels-and-taints](https://docs.spot.io/ocean/features/labels-and-taints) #### `spotinst.io/restrict-scale-down` Some workloads are not as resilient to spot instance replacements as others, so you may wish to lower the frequency of replacing the nodes they are running on as much as possible, while still getting the benefit of spot instance pricing. For these workloads, use the `spotinst.io/restrict-scale-down` label (set to `true`) to block the proactive scaling down of the instance for the purposes of more efficient bin packing. This will leave the instance running as long as possible. The instance will be replaced only if it goes into an unhealthy state or if forced by a cloud provider interruption. #### `spotinst.io/node-lifecycle` Ocean labels all nodes it manages with a label key `spotinst.io/node-lifecycle`. The label value is either `od` (on-demand) or `spot`, according to the lifecycle of the instance, and can assist when monitoring the cluster’s nodes in different scenarios. Some workloads are mission-critical and are not resilient to spot instance interruptions. These workloads have to run on on-demand instances at all times. To ensure that, apply node affinity to the `spotinst.io/node-lifecycle` label with value `od`. ### Option 2: Tune Parameters in Stack Configuration :::tip The [eks/cluster](/components/library/aws/eks/cluster/) component supports exposes most (but maybe not all) parameters of the [https://github.com/cloudposse/terraform-aws-eks-spotinst-ocean-nodepool](https://github.com/cloudposse/terraform-aws-eks-spotinst-ocean-nodepool) module. Before setting any variables make sure your component is passing those variables through to the upstream `terraform-aws-eks-spotinst-ocean-nodepool` module, or they will have no effect. ::: ``` components: terraform: eks: settings: spacelift: workspace_enabled: true terraform_version: "1" command: "/usr/bin/terraform-1" vars: aws_ssm_enabled: true cluster_encryption_config_enabled: true cluster_kubernetes_version: "1.18" spotinst_oceans: main: max_group_size: 20 min_group_size: 3 desired_group_size: 3 disk_size: 200 # Can only set one of ami_release_version or kubernetes_version # Leave both null to use latest AMI for Cluster Kubernetes version kubernetes_version: null # use cluster Kubernetes version ami_release_version: null # use latest AMI for Kubernetes version attributes: null ami_type: null # use "AL2_x86_64" for standard instances, "AL2_x86_64_GPU" for GPU instances tags: null instance_types: # m5n.large and r5n.large equivalents according to ec2-instance-selector ~8 GiB memory and 2 vCPUs # ec2-instance-selector --hypervisor nitro --memory-min 7GiB --memory-max 25GiB -a amd64 --network-performance-min 25 --vcpus-min 2 -g 0 # Limited to Nitro Instances with Transit Encryption (not all nitro instances support it) # https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/data-protection.html - "c5n.2xlarge" - "c5n.xlarge" # d3en not available in ue2 # - "d3en.xlarge" - "i3en.large" # inf1 does not support network encryption # - "inf1.2xlarge" # - "inf1.xlarge" - "m5dn.large" - "m5dn.xlarge" - "m5n.large" - "m5n.xlarge" - "m5zn.large" - "m5zn.xlarge" - "r5dn.large" - "r5n.large" # The metrics server is needed by the `ocean-controller` metrics-server: vars: installed: true # The `ocean-controller` is what feeds spotinst (SaaS) the information it needs to make informed decisions on scaling your cluster ocean-controller: vars: installed: true chart_version: "1.0.85" ssm_region: "us-west-2" ``` --- ## # How to Sign Up for Spot.io :::caution Before registering for [Spot.io](http://Spot.io) request Cloud Posse initiate the deal registration via our partner portal. This is so we can help manage the account and get credit for the referral. ::: ## Problem Cost management on AWS is tricky. There are reserved instances, AWS savings plans, Enterprise Agreements, and the Spot Instance marketplace. To get the best deal on compute, everything needs to be taken into account. It’s possible to roll your own with Spot Fleets, AWS lambdas, but the engineering effort may not be worth it. Learn more about the differences between[https://spot.io/blog/aws-spot-instances-and-diy/](https://spot.io/blog/aws-spot-instances-and-diy/) . ## Solution :::tip Use the fully managed solution for [http://spot.io](http://spot.io) of EKS clusters (and more) ::: 1. **Deal registration.** Request Cloud Posse initiate the deal registration via our partner portal before registering for an account. This is so we can help manage the account. 2. **Account Registration.** Go to [https://console.spotinst.com/spt/auth/signUp](https://console.spotinst.com/spt/auth/signUp) to sign up. 3. **Setup billing.** The way it works is that you set up a paid account on [Spot.io](http://spot.io/). No credit card is required, just enter all the billing details for where to send invoices (person and company name, mailing address, email, phone for who to make invoice out to and who to send it to). 4. **Add users.** Go to your avatar in the top right of the screen, click “Your Organization” → “Users” → “ADD USER”, turn on the "Add existing user" toggle, and add Cloud Posse personnel at their Cloud Posse email addresses (`@cloudposse.com`) with the Organization Role "Admin". IMPORTANT: anyone invited must already have a [http://spot.io](http://spot.io) account. You’ll get an error trying to invite anyone who does not already have an account. 5. **Skip AWS Integration.** DO NOT "Finish setting up your account by connecting a cloud provider." Cloud Posse will configure the rest of [Spot.io](http://spot.io/) via Terraform. 6. **Optionally, take a walkthrough.** DO take the "Console Walkthrough" if you are interested. :::info **Spot Support** While Cloud Posse is available to provide support, [Spot.io](http://spot.io/) has its own support channel available via web chat and email, and they are the best ones to answer questions about how [Spot.io](http://spot.io/) works and what features it does and does not support. ::: ## FAQ ### Why are we signing up for Spot.io? [Spot.io](http://Spot.io) intelligently manages [Spot.io](http://Spot.io) so you don’t have to. The way we use it is to manage fleets of heterogeneous EC2 instances in EKS node pools. Spot is smarter than the AWS cluster auto-scaler because it schedules workloads based on what is needed. It can mix and match EC2 instance types, on-demand/reserved, and spot. Workloads can target on-demand instances simply by adding an annotation that tells the scheduler where to place the pod. A much more comprehensive write up is here: [https://spot.io/blog/aws-spot-instances-and-diy/](https://spot.io/blog/aws-spot-instances-and-diy/) ### How much does Spot.io cost? Technically, [Spot.io](http://Spot.io) is free. They charge a percentage of your estimated savings against on-demand pricing. [https://spot.io/pricing/](https://spot.io/pricing/) ### Do we have to use Spot.io? No, this is entirely optional. We recommend using it though because it’s the easiest path to _Continuous Optimization_ ### Can we trust Spot.io? While every company should perform its own vendor evaluation and due diligence, it’s worth noting that [Spot.io](http://Spot.io) was acquired by NetApp. ### What are spot instances? Spot.io provides the best explanation of [https://spot.io/what-are-ec2-spot-instances/](https://spot.io/what-are-ec2-spot-instances/) ### How should we share access to Spot.io? Our recommendation is to invite individual users. But if you want, a shared account exchanged via 1Password to log in will suffice to get started. This is of course a reversible decision. ### Do AWS Savings Plans cover Spot Instances? Spot instances are not covered by [AWS Savings Plans](https://aws.amazon.com/savingsplans/faq/). That said, they are not mutually exclusive, and both provide savings and benefits. Learn more [https://docs.aws.amazon.com/savingsplans/latest/userguide/what-is-savings-plans.html](https://docs.aws.amazon.com/savingsplans/latest/userguide/what-is-savings-plans.html) ### Should we use both AWS Savings Plans and Spot Instances? AWS Savings Plans provide significant savings up to 72% of the on-demand rates in exchange for a commitment that can't be canceled during the term. AWS Savings plans are much easier to manage than purchasing Reserved Instances, plus you can pay for your commitment using **All Upfront**, **Partial Upfront**, or **No Upfront** payment options. In order to realize significant cost savings, however, you need to pay everything upfront which can be a lot of money 🤑 . Spot Instances on the other hand require zero upfront commitments and provide almost identical savings, if not greater, but with the trade-off that they can be preempted in just 2 minutes. [Spot.io](http://Spot.io) however uses a combination of AI and market intelligence to better predict fluctuations in Spot market pricing and can help reschedule workloads well in advance of the 2 minute warning AWS provides. [https://aws.amazon.com/savingsplans/compute-pricing/](https://aws.amazon.com/savingsplans/compute-pricing/) ### How reliable are Spot Instances? AWS provides extremely short notice of Spot Instance interruption. **A warning that is issued two minutes before Amazon EC2 stops or terminates your Spot Instance**. [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-interruptions.html) But this is where the true value of [Spot.io](http://Spot.io) comes out. With [https://spot.io/blog/predictive-rebalancing/](https://spot.io/blog/predictive-rebalancing/), Spot is able to reschedule workloads before market shifts. Additionally, since EKS node pools run on a mixed fleet of instance types, the likelihood of losing bids on more than one instance type at the same time is reduced, and the larger the cluster, the less the impact of losing a few nodes. In our experience, the Spot market volatility is very low. The [AWS Spot Instance Advisor](https://aws.amazon.com/ec2/spot/instance-advisor/) will help you see just how low that volatility is. For small, mission-critical clusters with less than 5 nodes, it’s best to run a mixture of on-demand and spot instances. :::caution For services that depend on long-lived connections like WebSockets and do not handle interruptions well, then we recommend either using an [API Gateway with WebSockets](https://aws.amazon.com/blogs/compute/announcing-websocket-apis-in-amazon-api-gateway/) support or scheduling pods on Reserved Instances. The AWS API Gateway handles all the long-lived connections between the client and service, so you can focus on building your business logic using plain HTTP-based backends such as AWS Lambda, or any other HTTP endpoint. ::: Check out [How to Tune SpotInst Parameters for EKS](/resources/legacy/spotinst/how-to-tune-spotinst-parameters-for-eks) to ensure pods run On-Demand instances (which would be covered by RI reservations, if available). --- ## Stacks(Legacy) # Background Stacks are a central SweetOps abstraction layer that are used to instantiate components. They're a set of YAML files [that follow a standard schema](https://github.com/cloudposse/atmos/blob/master/docs/schema/stack-config-schema.json) to enable a **fully declarative description of your various environments**. This empowers you with the ability to separate your infrastructure's environment configuration settings from the business logic behind it (provided via components). SweetOps utilizes YAML configuration for stacks because it's an easy to work with format that is nicely portable across multiple tools. The stack yaml format is natively supported today via [atmos](https://github.com/cloudposse/atmos), [the terraform-yaml-stack-config module](https://github.com/cloudposse/terraform-yaml-stack-config), and [Spacelift](https://spacelift.io) via [the terraform-spacelift-cloud-infrastructure-automation module](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation). # Conventions We have a number of important conventions around stacks that are worth noting. ## Stack Files Stack files can be very numerous in large cloud environments (think many dozens to hundreds of stack files). To enable proper organization of stack files, SweetOps recommends the following: * All stacks should be stored in a `stacks/` folder at the root of your infrastructure repository. * Name individual environment stacks following the pattern of `$environment-$stage.yaml` * For example, `$environment` might be `ue2` (for `us-east-2`) and `$stage` might be `prod` which would result in `stacks/ue2-prod.yaml` * For any **global** resources (as opposed to _regional_ resources), such as Account Settings, IAM roles and policies, DNS zones, or similar, the `environment` for the stack should be `gbl` to connote that it's not tied to any region. * For example, to deploy the `iam-delegated-roles` component (where all resources are global and not associated with an AWS region) to your production account, you should utilize a `stacks/gbl-prod.yaml` stack file. ## Shared Stack Configuration When you have configuration that you want to share across various stacks, use catalogs. Catalogs are the SweetOps term for shared, reusable configuration. By convention, all shared configuration for stacks is put in the `stacks/catalog/` folder, which can then be used in the root `stacks/` stack files via `import`. These files use the same stack schema. There are a few suggested shared catalog configurations that we recommend adopting: * **Global Catalogs**: For any configuration to share across **all** stacks. * For example, you create a `stacks/catalog/globals.yaml` file and utilize `import` wherever you need that catalog. * **Environment Catalogs**: For any configuration you want to share across `environment` boundaries. * For example, to share configuration across `ue2-stage.yaml` and `uw2-stage.yaml` stacks, you create a `stacks/catalog/stage/globals.yaml` file and utilize `import` in both the `ue2-stage.yaml` and `uw2-stage.yaml` stacks to pull in that catalog. * **Stage Catalogs**: For any configuration that you want to share across `stage` boundaries. * For example, to share configuration across `ue2-dev.yaml`, `ue2-stage.yaml`, and `ue2-prod.yaml` stacks, you create a `stacks/catalog/ue2/globals.yaml` file and `import` that catalog in the respective `dev`, `stage`, and `prod` stacks. * **Base Components**: For any configuration that you want to share across all instances of a component. * For example, you're using the `eks` component and you want to ensure all of you EKS clusters are using the same Kubernetes version, you create a `stacks/catalog/component/eks.yaml` file which specifies the `eks` component's `vars.kubernetes_version`. You can then `import` that base component configuration in any stack file that uses the `eks` component. * More information in the below section. ## Component Inheritance Using a component catalog, you can define the default values for all instances of a component across your stacks. But it is also important to note that you can provide default values for multiple instances of a component in a single stack using the component inheritance pattern via the `component` key / value: ```yaml # stacks/catalog/component/s3-bucket.yaml components: terraform: s3-bucket: vars: enabled: false user_enabled: false acl: private grants: null versioning_enabled: true ``` ```yaml # stacks/uw2-dev.yaml import: - catalog/component/s3-bucket - catalog/dev/globals - catalog/uw2/globals components: terraform: public-images: component: s3-bucket vars: enabled: true acl: public name: public-images export-data: component: s3-bucket vars: enabled: true name: export-data # ... ``` In the above example, we're able to utilize the default settings provided via the `s3-bucket` base component catalog, while also creating multiple instances of the same component and providing our own overrides. This enables maximum reuse of global component configuration. ## Terraform Workspace Names In `atmos` and the accompanying terraform automation modules like [terraform-spacelift-cloud-infrastructure-automation](https://github.com/cloudposse/terraform-spacelift-cloud-infrastructure-automation) the terraform [workspaces](https://www.terraform.io/docs/language/state/workspaces.html) will be automatically created when managing components. These workspaces derive their names from the stack name and the component name in question following this pattern: `$env-$stage-$component`. The result is workspace names like `ue2-dev-eks` or `uw2-prod-mq-broker`. # Pro Tips Here are some tips to help you write great stacks: 1. Use `{}` for empty maps, but not just a key with an empty value. 1. Use consistent datatypes for deep merging to work appropriately with imports (e.g. don't mix maps with lists or scalars). 1. Use [YAML anchors](https://blog.daemonl.com/2016/02/yaml.html) to DRY up a config within a single file. 1. **IMPORTANT**: Anchors work only within the scope of a single file boundary and not across multiple imports. # Stack Schema [The official JSON Schema document for Stacks can be found here](https://github.com/cloudposse/atmos/blob/master/docs/schema/stack-config-schema.json). The below is a walk through of a complete example utilizing all capabilities. ```yaml # stacks/ue2-dev.yaml # `import` enables shared configuration / settings across different stacks # The referenced files are deep merged into this stack to support granular configuration capabilities import: # Merge the below `stacks/catalog/*.yaml` files into this stack to provide any shared `vars` or `components.*` configuration - catalog/globals - catalog/ue2/globals - catalog/dev/globals # `vars` provides shared configuration for all components -- both terraform + helmfile vars: # Used to determine the name of the workspace (e.g. the 'dev' in 'ue2-dev') stage: dev # Used to determine the name of the workspace (e.g. the 'ue2' in 'ue2-dev') environment: ue2 # Define cross-cutting terraform configuration terraform: # `terraform.vars` provides shared configuration for terraform components vars: {} # `backend_type` + `backend` provide configuration for the terraform backend you # would like to use for all components. This is typically defined in `globals.yaml` # atmos + our modules support all options that can be configured for a particular backend. # `backend_type` defines which `backend` configuration is enabled backend_type: s3 # s3, remote, vault backend: s3: encrypt: true bucket: "eg-uw2-root-tfstate" key: "terraform.tfstate" dynamodb_table: "eg-uw2-root-tfstate-lock" role_arn: "arn:aws:iam::999999999999:role/eg-gbl-root-terraform" acl: "bucket-owner-full-control" region: "us-east-2" remote: {} vault: {} # Define cross-cutting helmfile configuration helmfile: # `helmfile.vars` provides shared configuration for terraform components vars: account_number: "999999999999" # Components are all the top-level units that make up this stack components: # All terraform components should be listed under this section. terraform: # List one or more Terraform components here first-component: # Provide automation settings for this component settings: # Provide spacelift specific automation settings for this component # (Only relevant if utilizing terraform-spacelift-cloud-infrastructure-automation) spacelift: # Controls whether or not this workspace should be created # NOTE: If set to 'false', you cannot reference this workspace via `triggers` in another workspace! workspace_enabled: true # Override the version of Terraform for this workspace (defaults to the latest in Spacelift) terraform_version: 0.13.4 # Which git branch trigger's this workspace branch: develop # Controls the `autodeploy` setting within this workspace (defaults to `false`) auto_apply: true # Add extra 'Run Triggers' to this workspace, beyond the parent workspace, which is created by default # These triggers mean this component workspace will be automatically planned if any of these workspaces are applied. triggers: - ue2-dev-second-component - gbl-root-example1 # Set the Terraform input variable values for this component. vars: my_input_var: "Hello world! This is a value that needs to be passed to my `first-component` Terraform component." bool_var: true number_var: 47 # Complex types like maps and lists are supported. list_var: - example1 - example2 map_var: key1: value1 key2: value2 # Every terraform component should be uniquely named and correspond to a folder in the `components/terraform/` directory second-component: vars: my_input_var: "Hello world! This is another example!" # You can also define component inheritance in stacks to enable unique workspace names or multiple usages of the same component in one stack. # In this example, `another-second-component` inherits from the base `second-component` component and overrides the `my_input_var` variable. another-second-component: component: second-component vars: my_input_var: "Hello world! This is an override." # All helmfile components should be listed under this section. helmfile: # Helmfile components should be uniquely named and correspond to a folder in the `components/helmfile/` directory # Helmfile components also support virtual components alb-controller: # Set the helmfile input variable values for this component. vars: installed: true chart_values: enableCertManager: true # `workflows` enable the ability to define an ordered list of operations that `atmos` will execute. These operations can be any type of component such as terraform or helmfile. # See "Getting started with Atmos" documentation for full details: https://docs.cloudposse.com/tutorials/atmos-getting-started/ workflows: # `workflows` is a map where the key is the name of the workflow that you're defining deploy-eks-default-helmfiles: # `description` should provide useful information about what this workflow does description: Deploy helmfile charts in the specific order # `steps` defines the ordering of the jobs that you want to accomplish steps: # `job` entries defined `atmos` commands that you want to execute as part of the workflow - job: helmfile sync cert-manager - job: helmfile sync external-dns - job: helmfile sync alb-controller - job: helmfile sync metrics-server - job: helmfile sync ocean-controller - job: helmfile sync efs-provisioner - job: helmfile sync idp-roles - job: helmfile sync strongdm - job: helmfile sync reloader - job: helmfile sync echo-server ``` --- ## Tools ## 1Password The [1Password for Teams](https://1password.com/teams/) product by AgileBits is arguably the most popular SaaS-based password management tool. In our opinion, it offers a better user experience over apps like LastPass as well as provides integration with [Duo](https://duo.com/docs/1password) and [Slack](https://support.1password.com/slack/) for real-time activity notifications. ### Best Practices Here are some of our recommended practices for working with 1Password. If your organization choses an alternative password management solution, we recommend implementing compensating controls. #### Enable Real-time Slack Notifications With real-time slack notifications, you'll be able to monitor logins for anomalies. ![Real-time Slack Notifications](/assets/1password-e3bc9e9c.png) #### Duo Integration for MFA Leverage Duo push notifications for MFA. With Duo, you can do full-on geofencing to ensure logins do not come from untrusted locations. ![Duo Push Notifications](/assets/1password-57e89599.png) #### Create Role-based Teams Define teams in terms of roles (E.g. `production-admin`, `staging-admin`, `dns`, `finance`, etc). #### AWS Master Credentials We use 1Password to store the AWS Master Account "root" credentials. Share OTP (MFA) codes with trusted admins. ![1Password MFA Tokens](/assets/1password-ca184eb3.png) ## AWS CLI The AWS Command Line Interface (CLI) is a command line tool to manage multiple AWS services and is useful for shell automation using scripts. ### Delete all versions of objects in an S3 bucket ``` export BUCKET=foobar aws s3api delete-objects --bucket $BUCKET \ --delete "$(aws s3api list-object-versions --bucket $BUCKET | \ jq -M '{Objects: [.["Versions","DeleteMarkers"][]|select(.Key == "key-value")| {Key:.Key, VersionId:.VersionId}], Quiet: false}')" ``` via: [stackoverflow](https://stackoverflow.com/a/36604650/1237191) ## *(Deprecated)* AWS Vault *(Deprecated)* The [`aws-vault`](https://github.com/99designs/aws-vault) command line tool by [99 Designs](https://99designs.com/) is a utility for securely storing and accessing encrypted AWS credentials for use in development environments. This tool makes it easy to work with IAM assumed roles across multiple AWS organizations. Cloud Posse no longer recommends this tool or workflow now that AWS supports federated login (meaning users never need long-lived AWS API keys) and the current AWS SDKs take a Role ARN as a parameter and automatically assume the role. We recommend [Leapp](#leapp) for managing initial credentials. With the AWS SDK enhancements, Terraform and the `aws` CLI, and by extension EKS (which we now recommend over `kops`), will automatically assume the roles as needed, removing the need to manually assume a role. :::info `aws-vault` has no relationship to the HashiCorp Vault. ::: ### Features - Encrypted vault for IAM credentials (OSX KeyChain or file) - IAM Metadata server (mocks the [EC2 API](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html)) to simulate instance profiles for local development - Prompts for MFA Token - Variable-length session TTLs - Compatible with `~/.aws/config` - Automatic logins to AWS Web Console ### *(Deprecated)* Local Configuration We recommend using the `file` type backend for `aws-vault` because this is compatible with Linux, which is needed for [Geodesic](#geodesic) sessions. Add the following to your `~/.bashrc`: ``` export AWS_VAULT_BACKEND="file" ``` Then `source ~/.bashrc` to update your current session. 1. Generate IAM Access Key ID/Secret on your AWS root account via IAM management page in the AWS Console. :::caution Do not define the source profile in `~/.aws/credentials`; we're going to use `aws-vault add` for that. ::: 2. Using the IAM Access Key ID/Secret generated in Step 1, add the `source_profile`: ```bash $ aws-vault add example ``` 3. Add the `source_profile` created in Step 2 to your `~/.aws/config`. ``` [profile example] region=us-west-2 ``` 4. Setup your `~/.aws/config` by adding a profile entry for each AWS account: :::caution Remember to replace the `$aws_account_id`s with your account ids and `user@example.com` with your IAM username below. We recommend using email addresses for all IAM user accounts associated with human users. ::: ``` [profile example-staging-admin] region=us-west-2 role_arn=arn:aws:iam::$aws_account_id_for_staging:role/OrganizationAccountAccessRole mfa_serial=arn:aws:iam::$aws_account_id_for_root:mfa/user@example.com source_profile=example ``` 5. Test that it is all set up properly: ``` $ aws-vault login example-staging-admin ``` This should open a browser and log you into the AWS console as the assumed role `example-staging-admin`. ### *(Obsolete)* Using with Geodesic `aws-vault` is no longer preinstalled in the Geodesic shell. It is available as a package named "aws-vault" in the Cloud Posse package repository, and you can install it at run time by running the appropriate commands for your Geodesic Linux distro (e.g. `apt-get install aws-vault` for Debian), or you can put a corresponding `RUN` command in your Dockerfile to install it at build time. To start the shell, run: ```bash $CLUSTER_NAME ``` #### Add your profile to AWS Vault Now we are ready to configure your AWS credentials. To add your AWS credentials to the encrypted vault run the following command. Remember to replace `example` with your source profile name. ``` aws-vault add example ``` #### Troubleshooting Most problems stem from misconfiguration. _**Note:** `aws-vault` configuration conflicts with the currently recommended Leapp workflow and configuration._ - **Do not** define a `[default]` profile in `~/.aws/credentials` or `[profile default]` in `~/aws/config` - **Do not** set `AWS_SDK_LOAD_CONFIG` - **Do not** set `AWS_SHARED_CREDENTIALS_FILE` If using `--server` mode, ensure the following credentials are not exported: :::caution Since running `aws-vault` using `--server` binds to the `169.254.169.254` local ip address to mock the AWS metadata server, you can run only one process per host machine. ::: - `AWS_ACCESS_KEY_ID` - `AWS_SECRET_ACCESS_KEY` - `AWS_SECURITY_TOKEN` - `AWS_SESSION_TOKEN` Use `unset` to delete each of the above variables from your environment and ensure they aren't exported in your `~/.bashrc` or `~/.profile`. ## Build Harness The `build-harness` is a collection of [Makefiles](/learn/toolchain/#make) to facilitate building stuff. It supports Golang projects, Dockerfiles, Helm charts, and much more. | | | | :----------- | :------------------------------------------------------------------------------------------------------------------------------------- | | GitHub Repo | https://github.com/cloudposse/build-harness/ | | Build Status | [![Build Status](https://travis-ci.org/cloudposse/build-harness.svg)](https://travis-ci.org/cloudposse/build-harness) | | Release | [![Release](https://img.shields.io/github/release/cloudposse/build-harness.svg)](https://github.com/cloudposse/build-harness/releases) | It's designed to work with CI/CD systems such as GitHub Action, Codefresh, Jenkins, and others. It's 100% Open Source and licensed under APACHE2. ### Usage At the top of your `Makefile` add, the following... ```shell -include $(shell curl -sSL -o .build-harness "https://cloudposse.tools/build-harness"; echo .build-harness) ``` This will download a `Makefile` called `.build-harness` and include it at run-time. We recommend adding the `.build-harness` file to your `.gitignore`. This automatically exposes many new targets that you can leverage throughout your build & CI/CD process. Run `make help` for a list of available targets. :::info The `/` in target names is interchangeable with the `:` in target names ::: ### Real World Examples We leverage the `build-harness` in nearly every project on our [GitHub](/community/contribute/our-github). | Example Repo | Usage | | :---------------------------------------------------------------------------- | :------------------------------------------------------------------------- | | [`charts`](https://github.com/cloudposse/charts/) | A collection of Helm Charts that leverages `docker/%` and `helm/%` targets | | [`terraform-null-label`](https://github.com/cloudposse/terraform-null-label/) | A terraform module that leverages `terraform/%` targets | ## Chamber Chamber is a CLI for managing secrets stored in [AWS Systems Manager Parameter Store](https://aws.amazon.com/systems-manager/features/#Parameter_Store). In the article [The Right Way to Store Secrets using Parameter Store](https://aws.amazon.com/blogs/mt/the-right-way-to-store-secrets-using-parameter-store/), `AWS` recommends using `chamber` for secrets management. ### Usage Using chamber you can perform all standard CRUD operations on secrets stored in [AWS SSM](/resources/glossary/ssm) and execute commands with environment variables populated from the secrets. For a complete description, check out the [official documentation](https://github.com/segmentio/chamber#usage). ## Geodesic ![Geodesic Logo](/assets/638d917-geodesic-small.png) | | | | :----------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------- | | GitHub Repo | https://github.com/cloudposse/geodesic | | Release | [![Release](https://img.shields.io/github/release/cloudposse/geodesic.svg)](https://github.com/cloudposse/geodesic/releases) | | Build Status | [![Build Status](https://github.com/cloudposse/geodesic/workflows/docker/badge.svg)](https://github.com/cloudposse/geodesic/actions?query=workflow%3Adocker) | ### Introduction Geodesic is our **Docker Based Toolbox**. It's a DevOps Linux Distro distributed as a docker image that has all the essential tools for automation of [AWS](https://aws.amazon.com/), [kubernetes](https://kubernetes.io/), etc. The toolbox can be used as a docker base image in order to ship consistent tooling and we provide packages for all [popular devops tools](https://github.com/cloudposse/packages). ### Demo import AsciinemaWidget from '../../../src/components/AsciinemaWidget'; At its core, Geodesic is a DevOps toolkit Linux Distro distributed via Docker for provisioning cloud infrastructure and the applications that sit on top of it. We leverage as many existing tools as possible to facilitate cloud fabrication and administration. Geodesic is like the connective tissue that sits between all of the components of a modern cloud. ### Technologies | Tool | Purpose | | :------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------- | | [`terraform`](https://github.com/hashicorp/terraform/) | for provisioning miscellaneous resources on pretty much any cloud | | [`aws-cli`](https://github.com/aws/aws-cli/) | for interacting directly with the AWS APIs (E.g. s3, ec2, rds) | | [`helm`](https://github.com/kubernetes/helm/) | for installing packages like nginx-ingress or datadog agent on the Kubernetes cluster | | [`kubectl`](https://kubernetes.io/docs/user-guide/kubectl-overview/) | for controlling kubernetes resources like deployments, pods, configmaps, etc. | | [`gomplate`](https://github.com/hairyhenderson/gomplate/) | for template rendering configuration files using the GoLang template engine. Supports lots of local and remote data sources | | [`goofys`](https://github.com/kahing/goofys/) | for mounting encrypted S3 buckets that store cluster configurations and secrets | | [`chamber`](https://github.com/segmentio/chamber) | for managing secrets with AWS SSM+KMS | | [`atmos`](https://github.com/cloudposse/atmos) | for orchestrating terraform or helmfile automation | | [`turf`](https://github.com/cloudposse/turf) | for orchestrating automation tasks that are difficult to automate via IaC | ### Our Logo In mathematics, a geodesic line is the shortest distance between two points on a sphere. It's also a solid structure composed of geometric shapes such as hexagons. We like to think of geodesic as the shortest path to a rock-solid cloud infrastructure. The geodesic logo is a hexagon with a cube suspended at its center. The cube represents this geodesic container, which is central to everything and at the same time is what ties everything together. But look a little closer and you'll notice there's much more to it. It's also an isometric shape of a cube with a missing piece. This represents its pluggable design, which lets anyone extend it to suit their vision. ## Gomplate ![Gomplate Template Renderer](/assets/gomplate-5869374e.png) A flexible commandline tool for template rendering. Supports lots of local and remote datasources. ### Why Gomplate? The Go programming language supports [native templating](https://golang.org/pkg/text/template/). The problem is that there aren't very many functions supported out-of-the-box and it cannot be called from the command line. Gomplate is a CLI tool that addresses this need. That is, it can be used as an alternative to [`envsubst`](https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html), but also supports additional template data-sources such as: JSON, YAML, and even AWS EC2 metadata. We love `envsubst` for its super-minimalist template processing, but sometimes more advanced templating with conditions is necessary. We use `gomplate` to parameterize a number of critical files across our architectures. Gomplate is an alternative to `envsubst` that provides some useful built-in functions to make templates even more expressive. ### References - [Official Documentation](https://docs.gomplate.ca/) ## Goofys [Goofys](https://github.com/kahing/goofys) a high-performance, POSIX-ish Amazon S3 file system written in Go. We use `goofys` as part of [Geodesic](#geodesic) to provide our own S3 filesystem interface for mounting buckets securely inside a container for the purpose of storing master SSH keys needed by EC2 instances. ### Why Goofys? Goofys is similar to the original [`s3fs`](https://github.com/s3fs-fuse/s3fs-fuse) FUSE filesystem, but written in Golang and much, much faster. Also, it works seamlessly with EC2 instance profiles for assumed-roles, which `s3fs-fuse` does not support as of `v1.82`. The reason why `goofys` is faster is that it implements a subset of the POSIX filesystem semantics. Namely, it only supports `read`, `write` and `append` operations. This makes it suitable for many simple use-cases, but not suitable for running things like a database. ## Helm ![Helm Logo](/assets/helm-d7685fec.png) Helm makes it easy to install `charts` (an application) on kubernetes clusters. Just like `npm` or `apt` make it easy to install NodeJS modules and Debian packages, `helm` makes it easy to deploy a full-fledged application with all of its dependencies on kubernetes. ### Delete All Releases in Namespace To delete all helm releases in `NAMESPACE`, run the following command: ``` helm list --namespace $NAMESPACE --short | xargs helm delete --purge ``` :::info Deleting releases will not delete the namespace itself. This is because there may be other artifacts. For a more detailed discussion on the pros & cons of this, check out the [GitHub issue](https://github.com/kubernetes/helm/issues/1464) tracking this topic. ::: Then after running this command, to delete the namespace, run: ``` kubectl delete namespace $NAMESPACE ``` ## Helmfile We use `helmfile` to deploy collections of charts. The `helmfile.yaml` is a declarative configuration file that makes it easier to deploy and manage a large number of helm charts. Another way to think about it is like this: > If `helm` is responsible for deploying a single application to kubernetes, then `helmfile` is responsible for deploying multiple applications by calling `helm`. ### Features - **12-Factor Style Configurations** - Parameterize all charts using Environment Variables - **Inline `values.yaml`** - Maintain changes to chart configurations in version control - **Chamber Compatibility** - Use [`chamber`](/learn/toolchain/#chamber) to deploy helm charts with secrets pulled from SSM - **CI/CD Integration** - Easily integrate `helmfile` with CI/CD pipelines using our [`build-harness`](/learn/toolchain/#build-harness) - **Synchronize Environments** - Rapidly deploy or update all services for a cluster - **Go Templating** - Configuration is templatized using [Go template](https://godoc.org/text/template) and supports all the [Sprig](https://godoc.org/github.com/Masterminds/sprig) interpolation functions. ### Use-cases - We recommend using a `helmfile` anywhere you need to deploy a helm chart. This is because `helm` does not support environment variables out-of-the-box. - The `helmfile` reduces the need for complex umbrella charts that are more difficult to manage. ### Dependencies Helmfile depends on the following `cli` tools. - [`helm`](/learn/toolchain/#helm) - for managing helm packages - [`kubectl`](/learn/toolchain/#kubectl) - for interfacing with the Kubernetes API :::info [`geodesic`](#geodesic) shell ships with all dependencies. ::: Prior to using `helmfile`, you'll need a valid `kubectl` context. Alternatively, set the `KUBE_CONTEXT` when using `helmfile` with a Codefresh pipeline. ### Configuration File The `helmfile.yaml` is a [go-template](https://golang.org/pkg/text/template/) formatted "YAML" file. Note, this means that it is first evaluated as a plain-text go-template before getting processed as YAML. It essential that the go-template result in well-formed YAML with properly escaped values. For complete examples, review our comprehensive distribution of [helmfiles](https://github.com/cloudposse/helmfiles/tree/master/releases). ### Example `helmfile.yaml` Here's an example `helmfile.yaml`. Note that it's possible to use conditionals (e.g. `if` / `else`). ```yaml title="Helmfile Example" # # Helm Chart Repositories # repositories: # Stable repo of official helm charts - name: "stable" url: "https://kubernetes-charts.storage.googleapis.com" {{ if env "CHART_REPO_URL" }} # Local chart repo - name: "chart-repo" url: '{{ env "CHART_REPO_URL" }}' {{ end }} # # Kubernetes # context: '{{ env "KUBE_CONTEXT" }}' # # Helm Releases # releases: # Postgres - name: '{{ env "RELEASE_NAME" }}-pg' labels: chart: "postgresql" component: "database" chart: "stable/postgresql" version: "0.11.0" namespace: '{{ env "NAMESPACE" }}' set: - name: "image" value: "cloudposse/postgres" - name: "imageTag" value: '{{ coalesce (env "POSTGRES_IMAGE_TAG") "0.2.1" }}' - name: "imagePullSecrets" value: '{{ env "RELEASE_NAME" }}-pull-secret-dockercfg' - name: "postgresDatabase" value: '{{ env "POSTGRES_DB" }}' - name: "postgresUser" value: '{{ env "POSTGRES_USER" }}' - name: "postgresPassword" value: '{{ env "POSTGRES_PASSWORD" }}' ``` ### Environment Variables There are two options for retrieving environment variables. The `env` function will simply retrieve the environment variable. If it does not exist, it will return an empty string. Combined with the `default` function, it's possible to set a default value. ``` {{ env "POSTGRES_USER" }} ``` The other option is to call `requiredEnv`. This method is not recommended. ``` {{ requiredEnv "POSTGRES_USER" }} ``` This will throw an exception if the environment variable (`POSTGRES_USER`) is not set. While this sounds like a great feature, it's actually problematic because sometimes we only want to `sync` a particular chart using the `--selector` argument. In this case, we shouldn't need to set all environment variables - only the ones relevant to the chart we're installing. If the `helmfile.yaml` is using `requiredEnv`, all those environment variables will need to be defined. For this reason, we do not recommend using `requiredEnv. ### Default Values ``` {{ env "POSTGRES_USER" | default "postgres" }} ``` An alternative way to express this is with the `coalesce` function (Sprig function). ``` {{ coalesce (env "POSTGRES_USER") "postgres" }} ``` ### Arguments ```txt title="Helmfile Usage" NAME: helmfile - USAGE: helmfile [global options] command [command options] [arguments...] COMMANDS: repos sync repositories from state file (helm repo add && helm repo update) charts sync charts from state file (helm upgrade --install) diff diff charts from state file against env (helm diff) sync sync all resources from state file (repos, charts and local chart deps) status retrieve status of releases in state file delete delete charts from state file (helm delete) test test releases from state file (helm test) GLOBAL OPTIONS: --file FILE, -f FILE load config from FILE (default: "helmfile.yaml") --quiet, -q silence output --namespace value, -n value Set namespace. Uses the namespace set in the context by default --selector,l value Only run using the releases that match labels. Labels can take the form of foo=bar or foo!=bar. A release must match all labels in a group in order to be used. Multiple groups can be specified at once. --selector tier=frontend,tier!=proxy --selector tier=backend. Will match all frontend, non-proxy releases AND all backend releases. The name of a release can be used as a label. --selector name=myrelease --kube-context value Set kubectl context. Uses current context by default --help, -h show help --version, -v print the version ``` ### Example: Sync All Charts To deploy all charts defined in the `helmfile.yaml`, simply run `helmfile sync`. Typically, we do not run this, however, because a `helmfile.yaml` will define many more charts than installed on the cluster. ### Example: Sync Specific Charts A `helmfile.yaml` may contain dozens of charts. To target specific charts, just pass the `--selector` argument to `helmfile`. Multiple label constraints can be specified by concatenating them with a `,` (comma). This works like a logical "and". Alternatively, the `--selector` argument may be passed multiple times which works like a logical "or"; each parameter is resolved independently so a release that matches any parameter will be used. For example, to sync all charts that have the `default: true` label, we would run: ``` helmfile --selector default=true sync ``` To sync all default charts, but exclude one in particular (e.g. `chartmuseum`), we could run something like: ``` helmfile --selector 'default=true,chart!=chartmuseum' sync ``` To sync all default charts *or* `chartmuseum`, we would run: ``` helmfile --selector 'default=true' --selector 'chart=chartmuseum' sync ``` ### Example: Helmfile with Chamber We prefer to use SSM as the system of record for all secrets. To export those secrets as environment variables for usage with helm, run the following command: ``` chamber exec $service -- helmfile sync ``` ### References - [Official Helmfile documentation](https://github.com/roboll/helmfile) - [Sprig functions documentation](http://masterminds.github.io/sprig/) - [Helmfiles](https://github.com/cloudposse/helmfiles/tree/master/releases/) ## Hugo ![/assets/hugo-7303d89d.png](Hugo Static Site Generator) [Hugo](https://gohugo.io/) is one of the most popular open-source static site generators. It also happens to be a rediculously fast framework for building static websites. We use it to build [our documentation](https://github.com/cloudposse/docs/). What we like about it is that it's written in Go, speaks "markdown" and uses Go-templates for rendering all pages. ## Kubectl `kubectl` is the command line tool (cli) for running commands against Kubernetes clusters. Think of it like the AWS CLI to Kubernetes clusters. ## Leapp [Leapp](https://leapp.cloud) is a GUI tool run on the host computer that manages cloud credentials such as AWS keys. It integrates with Geodesic via the host file system's `$HOME` directory. Leapp stores _temporary_ credentials in the standard AWS shared credentials file `$HOME/.aws/credentials` (note that all AWS libraries cache temporary credentials on the local file system), which Geodesic makes available inside the running container. Leapp runs a daemon process on the host to refresh the credentials as needed, allowing them to be as short-lived as possible without requiring human intervention to refresh them. ## Make The `make` command has been around for decades. Predominantly, it was used to build complex projects in C & C++, but it's seen a resurgence in the ops world for automating routine tasks and orchestration of infrastructure-as-code. In the 12-factor pattern, these are your “admin tasks”. ![Make is the only orchestrator you'll ever need.](/assets/4132caf-image_3.png) ### Understanding Makefiles Because `make` is a very simple language, there's not very much you need to learn to get started. Some things, however, might seem unintuitive when approaching it from the perspective of other languages. Namely, `make` is really a template engine that renders “stubs” and then executes those stubs in a shell. Environment variables can be considered as the arguments to those targets. The reason for using `make` is that it's supposed to be less magical. It's such a basic language it should be hard to do something wrong. It defines every action you could possibly want to take and provides a consistent interface for calling those targets. Environment variables are used everywhere, so it's very configurable and also 12-factor-esque. On the other hand, something like `bash` is more full-featured, but suffers from some of the early criticisms of PHP. Most shell-scripts suck. They don't check exit codes. They don't accept arguments in consistent fashion. They don't use environment variables. They hardcode settings. They end up looking like spaghetti. ### Variables Part of the confusion around make is that make variables (e.g. `$(FOO)`) resemble shell-like variables (e.g. `$FOO` or `${FOO}`). Note, that in bash using `$(....)` is synonymous using ``...`` (which will run the command). In `make`, variables are synonymous to environment variables. They can be defined ad-hoc on the command line or at the top of the Makefile. ``` make something FOO=bar ``` or Makefile: ``` FOO=bar ``` It's important to note that everything after the = is treated literally. Thus quotes (“) are not necessary, unlike in bash. ### Built-in Functions There are many built-in functions. A few of the most common uses are explained below. `$(call foobar,arg1,arg2)` This will call a macro called "foobar" `$(shell /bin/ls)` This will run the command “/bin/ls” and output the results. Results always have new-lines & carriage returns stripped off. There's no way around it. `$(eval FOO=bar)` This will evaluate the “`FOO=bar`” as “`make`” language formatted code. In this case, since “`FOO=bar`” is how a variable is defined, the result is that a variable named `FOO` gets set to bar in the global make namespace. ### Macros TBD ### Targets Targets are like “functions”. They can run any shell command. Those shell commands can be other scripts or apps like terraform or system commands. The `make` command is basically a “`bash`” template engine: every line of a make target is a one-line shell script. Think of the contents of a target as “stubs” or “scriptlets”. #### Make Evaluation 1. Treat entire target contents as a template 2. Render all variables, functions and macros 3. Then execute each line as a standalone shell script, which is executed in a sub-shell (`/bin/bash`` -c “......”`) #### Definition There are 3 ways target names can be defined. 1. `:` - the default way. allows exactly one target to use this name. if another target attempts to use the same name, you'll see a warning 2. `::` - this works like #1, but allows a target to be redefined in multiple places. This is nice when you want to decompose a Makefile into many sub-makefiles. e.g. `deps::` is a good target to define this way, since multiple makefiles might want to define dependencies. 3. `%:` - This is a wildcard. You might want to do this if you want to write one target for multiple use-cases. For example, the following example could be used to create a target that installs a brew package (e.g. `make install-vim`) ``` install-%: brew install $(subst,install-,) ``` #### Dependencies Targets require other targets. Just add those targets after the colon (`:`) For example.... ``` deps: @which -s envsubst || brew install gettext replace: deps @envsubst < something.sh > something.txt ``` **WARNING:** If your target name contains an escaped color (\:), builds deps do not work. *The following example does not work.* ``` example\:test: @echo "Hello world!" example\:broken: example:text ``` **Workaround:** call make as part of your target. This work basically the sameway, however, since it will be called in a subshell, it cannot modify the parent process environment in anyway. ``` example\:test: @echo "Hello world!" example\:broken: @make example:text ``` ### Examples Below are some annotated examples with explanations of what's going on. #### **Example 1** The basisc syntax is like this: ``` TMP_FILE = /tmp/test foobar: touch $(TMP_FILE) rm $(TMP_FILE) ``` In this example, `foobar` is the target name. The leading spaces are all tabs (e.g. `^I`) 1. It runs a command called “`touch /tmp/test`”. If that command exits 0 (success), then it will proceed to the next line. If the command exits non-zero, it's considered a failure and everything aborts at that point. 2. It runs the command “rm /tmp/test” But behind the scenes though, a lot more is happening. **This is the ROOT OF ALL MISUNDERSTANDINGS.** The contents of a `Makefile` target are a template. Think “`envsubst`”, only instead of simply replacing environment variables, the template supports make syntax. The make syntax has been described above and includes variables, functions, and macros. Let's break down the execution of the example above. ``` touch $(TMP_FILE) rm $(TMP_FILE) ``` Gets first rendered as a template. The output would look like this: ``` touch /tmp/test rm /tmp/test ``` Then, make will proceed to process the output data. Every line output is executed in it's own shell, unless the line ends with a backslash (e.g. `\`). The shell is determined by the value of the SHELL variable. We usually default this to `bash`. Make then executes the following 1. `/bin/bash -c “touch /tmp/test”` 2. `/bin/bash -c “rm /tmp/test”` Note, by default, make will output each command run. This can be turned off by adding an @ at the beginning of the lines. This is a make-ism and nothing to do with shell. Remember, make is really just a template language. #### **Example 2** Now that we've explained how commands are executed, let's introduce a more complicated example that uses `$(eval ...)` ``` foobar: @echo "TODAY=wednesday" > /tmp/env $(eval -include /tmp/env) @echo "Today is: '$(TODAY)'" ``` This would output: ``` Today is: '' ``` Why? Let's re-evaluate this file the way make looks at it. Here's what our template looks like: ``` @echo "TODAY=wednesday" > /tmp/env $(eval -include /tmp/env) @echo "Today is: '$(TODAY)'" ``` 1. Make will first render the template. It does this by first processing all make directives. These are everything in `$(....)`. 2. It finds` $(eval -include /tmp/env)`. This says to include the contents of `/tmp/env` into the current scope. If the file `/tmp/env` does not exist, it will not error and silently continue. Without the leading `-`, it would exit if the file does not exist (which it doesn't in this example - readon for more details). 3. `/tmp/env` does not exist, so nothing got included. Even if it did exist, it would be from some previous execution 4. Then it finds `$(TODAY)` which is not set, so it's evaluated to an empty string. 5. All `$(...)` processing is complete, so the rendered contents look like: ``` @echo "TODAY=wednesday" > /tmp/env @echo "Today is: ''"` ``` 6. Make proceeds to execute the contents, line by line 7. `/bin/bash -c 'echo “TODAY=wednesday” > /tmp/env'` 8. `/bin/bash -c 'echo "Today is: \'\'"'` 9. And finally, we see something on our terminal: `Today is: ''` #### **Example 3** Here's another example that demonstrates how literally these templates are interpreted. ``` foobar: #@echo "Hello $(shell date)" ``` Will first render the` $(shell date)` function which calls the `date` command and returns `Thu Feb 23 07:01:14 UTC 2017`, then execute the line in a sub-shell like this: ``` /bin/bash -c '#@echo "Hello Thu Feb 23 07:01:14 UTC 2017"' ``` Thus, to avoid this unnecessary execution, it's recommend to place all comments (`#`) at the very beginning of the line. ``` foobar: ## @echo "Hello" ``` #### **Example 4** Here's an example of a multi-line bash script ``` foobar: @if [ -d /tmp/test ]; then \ echo "Directory exists"; \ exit 0; \ else \ echo "Directory does not exist"; \ exit 1; \ fi ``` Here we execute in a single bash execution, the inline script above. Note that all the line endings use a combination of `;` and `\` depending on context. #### **Example 5** ``` foobar: @export MONTH=February @echo $${MONTH} ``` This will output *nothing* but a new line `\n` because what happens is `export MONTH=February` is executed and exported in the first subshell, but a subshell cannot modify the environment of the parent process (`make`). Then when the `echo` runs, we try and output `${MONTH}` (in bash-syntax) and it's empty. **Note:** we escape the dollar-sign with an extra `$` to avoid make from attempting to interpret it. Make always has priority since it's preprocessing the contents of the template looking for functions and variables. To correct this, we could instead write: ``` foobar: @export MONTH=February; \ echo $${MONTH} ``` ## Packages ["Packages"](https://github.com/cloudposse/packages) is the Cloud Posse distribution of native apps. Use this repo to easily install binary releases of popular apps such as all of our standard tools mentioned in this doc. This is useful for inclusion into a Dockerfile to install dependencies. ### Usage Clone the repo. ``` git clone git@github.com:cloudposse/packages.git ``` See all available packages: ``` make -C packages/install help ``` Install everything... ``` make -C packages/install all ``` Install specific packages: ``` make -C packages/install atmos chamber ``` Install to a specific folder: ``` make -C packages/install atmos INSTALL_PATH=/usr/bin ``` Add this to a `Dockerfile` to easily install packages: ``` RUN git clone --depth=1 -b master https://github.com/cloudposse/packages.git /packages && \ rm -rf /packages/.git && \ make -C /packages/install kubectl ``` Here's a [real example](https://github.com/cloudposse/geodesic/blob/0.9.17/Dockerfile#L37-L46) of how we use it in `geodesic`. Uninstall a specific package ``` make -C uninstall yq ``` ### Makefile Inclusion Sometimes it's necessary to install some binary dependencies when building projects. For example, we frequently rely on `gomplate` or `helm` to build chart packages. Here's a stub you can include into a `Makefile` to make it easier to install binary dependencies. ``` export PACKAGES_VERSION ?= master export PACKAGES_PATH ?= packages/ export INSTALL_PATH ?= $(PACKAGES_PATH)/vendor ## Install packages packages/install: @if [ ! -d $(PACKAGES_PATH) ]; then \ echo "Installing packages $(PACKAGES_VERSION)..."; \ rm -rf $(PACKAGES_PATH); \ git clone --depth=1 -b $(PACKAGES_VERSION) https://github.com/cloudposse/packages.git $(PACKAGES_PATH); \ rm -rf $(PACKAGES_PATH)/.git; \ fi ## Install package (e.g. helm, helmfile, kubectl) packages/install/%: packages/install @make -C $(PACKAGES_PATH)/install $(subst packages/install/,,$@) ## Uninstall package (e.g. helm, helmfile, kubectl) packages/uninstall/%: @make -C $(PACKAGES_PATH)/uninstall $(subst packages/uninstall/,,$@) ``` Here's a [real example](https://github.com/cloudposse/build-harness/blob/0.5.5/modules/packages/Makefile) of how we use it. ## Teleport [Teleport](https://goteleport.com/) is a BeyondCorp solution to provide the unified access plane for infrastructure. ## Terraform We use `terraform` as one of our most central tools for automating infrastructure. It's the predominate, cloud agnostic [Infrastructure as Code (IaC)](https://en.wikipedia.org/wiki/Infrastructure_as_code) tool. We leverage `terraform` wherever possible to create composable, automated infrastructure architectures. Learn more: 1. [Our Terraform Best Practices](/best-practices/terraform) 1. [Our Terraform Modules](https://github.com/cloudposse?q=terraform-) 1. [Our Terraform Tips & Tricks](/learn/tips-and-tricks/terraform) --- ## Troubleshooting Here are some common errors and fixes. ## Terraform ### Error: `Could not retrieve the list of available versions for provider` If you get an error like this one, it’s usually because of the `.terraform.lock.hcl` file. :::tip Either run `terraform init -upgrade` to upgrade pinning for all providers or simply run `rm -f .terraform.lock.hcl` to have the file rereated on next run of `terraform init`. ::: ``` Error: Failed to query available provider packages Could not retrieve the list of available versions for provider cloudposse/utils: locked provider registry.terraform.io/cloudposse/utils 0.3.1 does not match configured version constraint >= 0.8.0, ~> 0.11.0; must use terraform init -upgrade to allow selection of new versions Error: 1 error occurred: * step "plan init": job "terraform init": job "terraform subcommand": command "terraform init" in "./components/terraform/foobar": exit status 1 ``` ### Error: `ResourceNotFoundException: Requested resource not found` :::tip This usually happens when the `role_arn: null` still exists in the stack and/or `backend.tf.json` files from the cold start. ::: _If using the_ `aws-sso` _component, then:_ 1. remove `role_arn: null` from all the stack yaml 2. replace `"role_arn": null,` with `"profile": "-gbl-root-terraform",` in the `backend.tf.json` _If not using the_ `aws-sso` _component:_ 1. replace `"role_arn": null` with the full role arn of `-gbl-root-terraform` in the `backend.tf.json` :::caution You may have to run a `atmos terraform plan clean -s ` or it may complain that the state needs to be migrated. ::: ### Error: `The security token included in the request is invalid` :::tip If using **Leapp during the cold-start process**, first try settings the environment variables in the shell (e.g. AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) instead of using config file with session token. ::: If you see this error during the **cold-start** process. This usually means one of the following: 1. Terraform is trying to assume a role while targeting the **root** account 2. Terraform is not properly assuming the **OrganizationAccountAccessRole** when targeting a **non-root** account 3. The session token that **Leapp** generates for the AWS config file does not play well with Terraform Make sure that the **Leapp** credentials are not a problem by using the environment variables (e.g. AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY). If that doesn’t work, **org_role_arn** (returned by account-map.modules.iam-roles) should empty for the **root** account or populated with the **OrganinizationAccountAccessRole** ARN when targeting a **non-root** account. ## Kubernetes ### Error: `Kubernetes cluster unreachable: the server has asked for the client to provide credentials` > `Error: Kubernetes cluster unreachable: the server has asked for the client to provide credentials` :::tip **If you see this error on Spacelift**, it probably means the plan was created some time ago, so the EKS token has already expired. Simply retrigger the stack and it should work. ::: This is almost certainly due to Kubernetes Terraform Provider issues. There are various subtle problems with how it handles EKS authentication. The long-term solution will be for the provider to fix the problems. In the short term, retriggering the stack usually works. ##### Background An EKS cluster, like any Kubernetes cluster, has its own authentication mechanism (known in Kubernetes as Role-Based Access Control) to limit who can take what actions. In EKS, the mechanism for authentication, in brief, is that the user presents IAM authentication credentials to a special API endpoint, and receives a short-lived token that it can present to EKS for authentication. Because this API call is expensive, the token is cached somewhere, and authentication operations use the cached token. In a normal operation under `kubectl`, you create a so-called `KUBECONFIG` file that instructs `kubectl` how to get this token, and `kubectl` will respond to an authentication failure (which happens regularly when the cached token expires) by asking for another token. In Terraform, the mainstream way to get this authentication token is by directly calling the API via the `aws_eks_cluster_auth` Data Source. Then Terraform caches the token in its state file. The problem here is that Terraform is a general-purpose resource manager, and is not equipped with a retry mechanism to refresh the token after it expires. The current workaround is to execute some command which causes Terraform to re-read the Data Source rather than use the cached token. At Cloud Posse, we have developed an alternative mechanism that leverages `kubectl`, but it requires that the `aws` CLI be installed, which is not something we want to require for our open source modules for various reasons. We are considering making it standard on our closed-source installations where we use Spacelift and can require `aws`. Inquire about it if you run into this problem. ### Error: `Kubernetes cluster unreachable: Get "https://xyz.gr7.region.eks.amazonaws.com/version?timeout=32s": dial tcp a.b.c.d:443: i/o timeout` This error is most likely due to a security group issue. The [transit-gateway](/components/library/aws/tgw/) component controls ingress from `auto` and `corp` into the other accounts. Check the `transit-gateway`'s input var `accounts_with_eks` is filled in correctly and redeploy the component. ### Error: `Post "http://localhost/api/v1/namespaces/kube-system/configmaps": dial tcp 127.0.0.1:80: connect: connection refused` This could also be a `Error: Get` Do not remove the kubeconfig map from the state as that will cause a cluster recreation Instead, run a `deploy` to refresh the token and then a `plan` will work on the eks cluster. The `deploy` may fail but try a `plan` after and it should work. ``` atmos terraform deploy eks --stack ue1-auto atmos terraform plan eks --stack ue1-auto ``` This may also be an issue with using only private subnets and not using the vpn. ### Error: `initContainers` does not accept `imagePullSecrets` This issue is **CLOSED** but still affects versions of Kubernetes up to 1.22.0. [https://github.com/kubernetes/kubernetes/issues/70732](https://github.com/kubernetes/kubernetes/issues/70732) ### Error: configmaps "aws-auth" is forbidden: User "system:anonymous" cannot get resource "configmaps" in API group "" in the namespace "kube-system" This input for the eks module should fix it ``` kube_exec_auth_enabled = !var.kubeconfig_file_enabled ``` And/or use all the best practices for the eks module in the eks component ``` kube_data_auth_enabled = false # exec_auth is more reliable than data_auth when the aws CLI is available # Details at https://github.com/cloudposse/terraform-aws-eks-cluster/releases/tag/0.42.0 kube_exec_auth_enabled = !var.kubeconfig_file_enabled # If using `exec` method (recommended) for authentication, provide an explicit # IAM role ARN to exec as for authentication to EKS cluster. kube_exec_auth_role_arn = coalesce(var.import_role_arn, module.iam_roles.terraform_role_arn) kube_exec_auth_role_arn_enabled = true # Path to KUBECONFIG file to use to access the EKS cluster kubeconfig_path = var.kubeconfig_file kubeconfig_path_enabled = var.kubeconfig_file_enabled ``` ## Other ### Error: `pre-commit` hook for `terraform-docs` is not working :::tip Try running `precommit autoupdate` to fix the issue. ::: When `pre-commit` fails, it doesn’t produce much output to `stdout`. ``` Terraform docs...........................................................Failed ``` Instead, `pre-commit` will direct the output to a logfile, `$HOME/.cache/pre-commit/pre-commit.log`. To re-run the `terraform-docs` pre-commit hook, run `pre-commit run terraform_docs -a`. Others have reported that errors were caused by a local version of `terraform-docs` that was ahead of our `pre-commit` version in the infrastructure repository. Running `precommit autoupdate` fixed the issue. ### Error: Leapp is Unstable on Windows If Leapp is unstable on Windows, the workaround, for the time being, may be to use WSL and copy over the `.aws` directory from Windows home user to Linux home user. --- ## Getting started with Atmos ## Atmos Documentation for Atmos has moved to [atmos.tools](https://atmos.tools) --- ## (Obsolete) Getting started with Geodesic v3 import Note from '@site/src/components/Note'; This documentation is for Geodesic v3, which is now obsolete. Please refer to the [Geodesic project](https://github.com/cloudposse/geodesic/) or the updated documentation on this site for current documentation on Geodesic v4 or later. ## Intro In the landscape of developing infrastructure, there are dozens of tools that we all need on our personal machines to do our jobs. In SweetOps, instead of having you install each tool individually, we use Docker to package all of these tools into one convenient image that you can use as your infrastructure automation toolbox. We call it [Geodesic](/learn/toolchain/#geodesic) and we use it as our DevOps automation shell and as the base Docker image for all of our DevOps scripting / CI jobs. In this tutorial, we'll walk you through how to use Geodesic to execute Terraform and other tooling. We'll be sure to talk about what is going on under the hood to ensure you're getting the full picture. ## Prerequisites ### System Requirements To accomplish this tutorial, you'll need to have [Docker installed](https://docs.docker.com/get-docker/) on your local machine. **That's all**. ### Docker Primer Before we jump in, it's important to note that Geodesic is built around some advanced features of Docker and the Docker CLI that are worth understanding. If you know the docker CLI well then feel free to skip over this section. 1. Geodesic overrides the default entrypoint with a custom script that enables starting a login shell or installing geodesic as an executable on your local machine. If you don't know much about Docker's `ENTRYPOINT` or `CMD` capabilities then [we recommend reading this article before moving forward](https://phoenixnap.com/kb/docker-cmd-vs-entrypoint). 1. Geodesic can be thought of as just another shell that you open up on your machine like `zsh` or `bash`, but when running it inside of Docker means that it doesn't have access to your home directory or the projects you're looking to work on. To get around that, Geodesic is typically run using Docker volume bind mounts via the `--volume` / `-v` flag. You'll see this in our examples below, but if you would like to understand more about what is going on under the hood then we recommend [reading up quickly on mounting local volumes via `docker run`](https://docs.docker.com/engine/reference/commandline/run/#mount-volume--v---read-only). ### Geodesic Usage Patterns Let's talk about a few of the ways that one can run Geodesic. Our toolbox has been built to satisfy many use-cases, and each result in a different pattern of invocation: 1. You can **run standalone** Geodesic as a standard docker container using `docker run`. This enables you to quickly use most of the built-in tools. (Some tools require installing the wrapper script first, as explained in the next step.) 1. Example: `docker run -it --rm --volume $HOME:/localhost cloudposse/geodesic:latest-debian --login` opens a bash login shell (`--login` is our Docker `CMD` here; it's actually just [the arguments passed to the `bash` shell](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html) which is our `ENTRYPOINT`) in our Geodesic container. 1. Example: `docker run --rm cloudposse/geodesic:latest-debian -c "terraform version"` executes the `terraform version` command as a one-off and outputs the result. 1. You can **install** Geodesic onto your local machine using what we call the docker-bash pattern (e.g. `docker run ... | bash`). Similar to above, this enables a quickstart process but supports longer lived usage as it creates a callable script on your machine that enables reuse any time you want to start a shell. 1. Example: `docker run --rm cloudposse/geodesic:latest-debian init | bash -s latest-debian` installs `/usr/local/bin/geodesic` on your local machine which you can execute repeatedly via simply typing `geodesic`. In this example, we're pinning the script to use the `cloudposse/geodesic:latest-debian` docker image, but we could also pin to our own image or to a specific version. 1. Lastly, you can **build your own toolbox** on top of Geodesic. This is what SweetOps generally recommends to practitioners. We do this when we want to provide additional packages or customization to our team while building on the foundation that geodesic provides. This is simple to do by using Geodesic as your base image (e.g. `FROM cloudposse/geodesic:latest-debian`) in your own `Dockerfile`, adding your own Docker `RUN` commands or overriding environment variables, and then using `docker build` to create a new image that you distribute to your team. This is more advanced usage and we'll cover how to do this in a future how-to article. In this tutorial, we'll be running Geodesic standalone using `docker run` to allow us to get up and running quickly. ## Tutorial ### 1. Start the Geodesic Shell First, at your terminal, let's start up the Geodesic shell! ```bash docker run -it --rm --volume $HOME:/localhost cloudposse/geodesic:latest-debian --login ``` There are a few things going on there, so let's break that down a bit: 1. We're using `docker run` to start a new container using the image `cloudposse/geodesic:latest-debian` which is [hosted on Docker Hub](https://hub.docker.com/r/cloudposse/geodesic) 1. We're using the `-it` flags (i.e. `--interactive` and `--tty`) to start an interactive terminal session [with a TTY](https://stackoverflow.com/questions/22272401/what-does-it-mean-to-attach-a-tty-std-in-out-to-dockers-or-lxc). 1. We're using the `--rm` flag to ensure that we clean up this container after we exit out of the session. 1. We're using the `--volume` flag to mount our `$HOME` directory to `/localhost` in our new container. This is a Geodesic standard practice which enables the container and your corresponding shell session to have access to your dotfiles, configurations, and the projects that you'll work on. 1. **NOTE**: If you're running on Linux and using Geodesic, any files written to the `--volume` mounts will be owned by the user inside the container, which is `root`. [See here for potential workarounds](https://github.com/moby/moby/issues/3124#issuecomment-104936573). 1. Finally, after the image name, we're passing `--login`. This is the Docker `CMD` that we're passing to our image. Since we override the Docker `ENTRYPOINT` with a small bash script, our `--login` `CMD` results in calling `/bin/bash --login` which creates a new [bash login shell](https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html). 1. It's worth noting that since Geodesic `v0.143.2` ([PR #693](https://github.com/cloudposse/geodesic/pull/693)), you can now drop `--login` as the default `CMD` will provide the same functionality. The result of running this command should look like this: ![Geodesic Login Shell](/assets/geodesic-login-shell.png) ### 2. Download our Tutorial Project Great -- we've started up Geodesic so now let's do something with it. How about we pull a terraform project and apply it? To accomplish this, let's do the following: ```bash # Change to our /localhost directory so that we can pull our project's code to our # local machine as well as our docker container cd /localhost # Clone our tutorials repository git clone https://github.com/cloudposse/tutorials # Change to our tutorial code cd tutorials/01-geodesic ``` Easy! And since we changed into our `/localhost` directory inside Geodesic, the `tutorials` project that we git cloned is available both in the container that we're running our shell in **and** on our local machine in our `$HOME` directory. This enables us to share files between our local machine and our container, which should start to give you an idea of the value of mounting `$HOME` into Geodesic. ### 3. Apply our Terraform Project Now that we've got some code to work with, let's apply it... ```bash # Setup our terraform project terraform init # Apply our terraform project terraform apply -auto-approve ``` Sweet, you should see a successful `terraform apply` with some detailed `output` data on the original star wars hero! 😎 Just to show some simple usage of another tool in the toolbox, how about we parse that data and get that hero's name? ### 4. Read some data from our Outputs Let's utilize [`jq`](https://github.com/stedolan/jq) to grab some info from that terraform project's output: ```bash # Pipe our terraform project's output into jq so we can pull out our hero's name terraform output -json | jq .star_wars_data.value.name ``` Again, without having to install anything, we've grabbed a tool from our toolbox and were able to use it without a second thought. ## Conclusion The beautiful thing about all of this is that we didn't need to install anything except Docker on our local machine to make this happen. Tools like `git`, `terraform`(all versions), and `jq` all involve specific installation instructions to get up and running using the correct versions across various machines/teams, but by using Geodesic we're able to quickly skip over all of that and use a container that includes them out of the box alongside [dozens of other tools as well](https://github.com/cloudposse/packages/tree/master/vendor). And with the mounting of our `$HOME` directory to `/localhost` of the container, our Geodesic shell just ends up being an extension of our local machine. That is why we call it a toolbox as it enables consistent usage of CLI tools across your entire organization! If you want to see another usage of Geodesic, [read our next tutorial in the SweetOps series about one of our most important tools: `atmos`.](https://atmos.tools/quick-start/introduction) --- ## Resources import DocCardList from '@theme/DocCardList' These are additional resources that may be helpful to you. ## Resources --- ## Get AWS Support import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Note from '@site/src/components/Note'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Admonition from '@theme/Admonition'; There are many ways to get support from AWS, but navigating their organization can be daunting. Understanding these various roles can help you effectively navigate the AWS structure and get the most out of your relationship with AWS. AWS will not be able to provide support for the Cloud Posse reference architecture. If you have any questions or need support, please reach out to Cloud Posse directly. ### Account Manager The Account Manager is part of the AWS Sales group and their main focus is on increasing your AWS consumption, but in a good way! 🙂 Account Managers are very interested in seeing happy AWS customers and knowing at a high level as much as possible about what you want to achieve by running on AWS. Then they lookout for how they can help out on their end. Account Managers will work with APN Partners like Cloud Posse to achieve customer business objectives. If you need an introduction to your AWS Account Manager, Cloud Posse can assist you with that. ### AWS Technical Account Managers (TAM) AWS Enterprise Support provides you with a concierge-like service with a dedicated TAM who will maintain context about what your organization is trying to achieve and look out for your best interests navigating the organization. A TAM will be able to connect you with Subject Matter Experts (SMEs) throughout the organization. For example, if you have some highly specialized questions on an AWS product, they can bring in an SME to address those questions. If you need an introduction to your AWS Technical Account Manager, Cloud Posse can assist you with that. While we’ve never had a customer replace their TAM, in the event you are ever unhappy with your TAM, Cloud Posse can reach out discretely using AWS back channels via our AWS partner status. Our partner can then bypass the TAM and raise any issues or concerns directly with the TAM’s manager. Of course, this is the last resort, and we hope to resolve any issues by working with the TAM. ### AWS Solutions Architects (SA) AWS assigns every account a Solutions Architect. Depending on your support agreement with AWS, the SA may even be able to join our regular Design Decision calls. Typically, we see this more often with our customers who have an Enterprise Support agreement. Looping in an SA is a great way to keep AWS informed on your technology roadmap. The SAs frequently bring deep industry knowledge of what other companies in your technology sector are doing (E.g. Healthcare or Financial Services), which is helpful for validation. ### Product Teams (aka “Service Teams”) AWS has countless services and each one is managed by a Product Team. As necessary, as your AWS Partner we can reach out to teams so that you can learn about upcoming features, make feature requests, and even participate in betas of early features. Customers under an NDA with AWS will also be able to learn about upcoming features not yet announced. This can save your company the cost of implementing workarounds for features that are already planned to be released. --- ## Get Essential Support import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Note from '@site/src/components/Note'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import SecondaryCTA from '@site/src/components/SecondaryCTA'; import Admonition from '@theme/Admonition'; import FeatureList from '@site/src/components/FeatureList'; import Steps from '@site/src/components/Steps'; import Step from '@site/src/components/Step'; import StepNumber from '@site/src/components/StepNumber'; import ActionCard from '@site/src/components/ActionCard'; Essential Support provides **priority asynchronous support** for self-sufficient teams building on Cloud Posse’s reference architecture and open source components. It includes prioritized responses in GitHub Discussions and access to twice-weekly shared customer workshops, where teams can get help with questions about Cloud Posse architecture, components, and implementation patterns. ## What's Included? Your subscription includes **priority access to Cloud Posse expertise**, providing: - **[2x Weekly Customer Workshops](#-join-weekly-customer-workshops-via-zoom):** Join live group sessions twice a week to get expert guidance, troubleshoot challenges, and refine your architecture. - **[GitHub Discussions](#-ask-questions-on-github-discussions):** Get expert responses and leverage our knowledge base for self-service troubleshooting. :::tip Need hands-on help or engineering time? For **scheduled engineering work**—bug fixes, upgrades, Atmos enhancements, new components, and integration work—check out our [**Flexible Support**](/support/flexible). If you're facing a **high-stakes deadline** or need **private access** to senior engineers to move fast, consider [**Platform Advisory**](/support/platform-advisory). ::: ## How it Works We provide two ways to get direct access to Cloud Posse so you can get the help you need in a timely manner. Whether you prefer to ask questions asynchronously or discuss them live, we have options to support you. ## Ask Questions on GitHub Discussions The best way to start is by asking your questions asynchronously via **GitHub Discussions**. This allows us to provide detailed responses, including links to documentation and code examples that you can follow at your own pace. GitHub Discussions works best when: - You encounter a **specific error** that you can share. - You have **questions about configuring components**. - You need technical quidance on [**Atmos**](https://atmos.tools), [**Geodesic**](https://github.com/cloudposse/geodesic), or any of our **GitHub Actions**. - You run into **documentation issues**. - You need **help troubleshooting** a setup. Not only will you receive a response from the Cloud Posse team (typically within a day or by the next business day, as part of our two-day SLA), but your question also becomes part of a **searchable knowledge base**. Chances are, someone has already asked a similar question, and you can quickly find answers by searching past discussions. As part of the onboarding process, we'll ask for a list of GitHub usernames that should have **prioritized access** to Cloud Posse expertise in these public discussions. See Examples ## Join Weekly Customer Workshops via Zoom If you find it difficult to articulate your question in writing, need real-time guidance, or have a more nuanced issue, our **Weekly Customer Workshops** are a great way to get help. These live Zoom sessions provide an interactive space where we can: - Workshop your challenges together - Answer complex, high-level architectural questions - Jump into breakout rooms for **pair programming** and troubleshooting - Discuss edge cases that are hard to explain in text While these workshops are **invite-only and exclusive to customers**, they are not strictly confidential. If you need to discuss **sensitive or proprietary topics**, we recommend using [**Platform Advisory**](/support/platform-advisory) for private access to Cloud Posse engineers, or [**Flexible Support**](/support/flexible) for hands-on development work. By coming prepared to the Customer Workshops, we can dedicate more time to answering your questions and diving into hands-on exercises, making the most of our time together! ## Why Cloud Posse? Our team has helped hundreds of companies implement AWS services using Infrastructure as Code (IaC) with Terraform, to deploy and manage AWS services such as Amazon EKS, AWS ECS, AWS Lambda, AWS IAM, Amazon RDS, Amazon S3, AWS Security Hub, and Multi-account Organizations. Whether you need PR reviews, bug fixes, custom Terraform module development, or assistance optimizing your AWS architecture, we provide structured, scheduled support to help you achieve your goals. We’ve built more **Terraform modules for AWS** than anyone else—[160+ and counting](https://github.com/cloudposse/). If it’s AWS and Terraform, we’ve probably already solved it. Hit a snag? You don’t have to figure it out alone. Cloud Posse has your back—get priority responses in GitHub Discussions and join a live customer workshop to keep moving forward. --- ## Get Flexible Support import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Note from '@site/src/components/Note'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Admonition from '@theme/Admonition'; import FeatureList from '@site/src/components/FeatureList'; import ActionCard from '@site/src/components/ActionCard'; Need help with bigger problems? Flexible Support is the most cost-effective way to get expert guidance from [Cloud Posse](https://cloudposse.com) for hands-on help and non-urgent but important requests—like bug fixes, PR reviews, and custom development. Support is scheduled to fit into your timeline, so you get the predictable help you need. Unlike [Essential Support](/support/essential), Flexible Support includes direct engineering work by Cloud Posse, so you can stay focused while we handle the heavy lifting. Many teams waste weeks trying to figure things out themselves—time that could have been spent shipping features and scaling their business. If you want clear answers, expert guidance, and hands-on support, our **Flexible Support** helps you move faster. ## Includes - **Hands-on Support** – Bug fixes, upgrades, Atmos enhancements, improvements to our open source components, new components, and integration work. - **Private Slack Channel** – Coordinate active requests with Cloud Posse engineers during business hours. - **Scheduled Delivery** – All work is scheduled in advance via Slack or email to align with your priorities and ensure predictable delivery. - **No Project Management Overhead** – Cloud Posse manages and coordinates the work—no extra effort required from your team. - **Retainer & Pay-as-you-go Options** – Flexible purchasing options to fit your needs and budget. We offer two convenient ways to get Flexible Support, including **Hourly Retainers** and **Pay-as-you-go** options. Choose the plan that fits your needs and budget. ## Who's it for? **Flexible Support** is designed for **hands-on engineering work** that is planned and scheduled in advance. It’s ideal for: - **Implementation work** that requires significant engineering effort—bug fixes, upgrades, new components, integrations, and more - **Non-urgent tasks** that don't require same-day turnaround :::tip In a pinch or under pressure? When the clock is ticking, leadership is watching, and you need to move fast—**[Platform Advisory](/support/platform-advisory)** gives you private access to senior Cloud Posse engineers via Slack & Zoom. Perfect for audits, migrations, architecture challenges, and high-stakes delivery. ::: ## Why Cloud Posse? Our team has helped hundreds of companies implement AWS services using Infrastructure as Code (IaC) with Terraform, to deploy and manage AWS services such as Amazon EKS, AWS ECS, AWS Lambda, AWS IAM, Amazon RDS, Amazon S3, AWS Security Hub, and Multi-account Organizations. Whether you need PR reviews, bug fixes, custom Terraform module development, or assistance optimizing your AWS architecture, we provide structured, scheduled support to help you achieve your goals. We’ve built more **Terraform modules for AWS** than anyone else—[160+ and counting](https://github.com/cloudposse/). If it’s AWS and Terraform, we’ve probably already solved it. Hit a snag? You don’t have to figure it out alone. Our experts have your back—so you can stay focused on what matters. Get the help you need, exactly when you need it. --- ## Get Support import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Admonition from '@theme/Admonition'; import FeatureList from "@site/src/components/FeatureList"; import Note from "@site/src/components/Note"; Many teams waste weeks trying to figure things out themselves—time that could have been spent shipping features and scaling their business. If you want clear answers, expert guidance, and hands-on support, our support helps you move faster. We provide a variety of support options to help you get the most out of our Reference Architecture. Mix and match our support options to fit your team’s needs. Support is available for customers building on Cloud Posse’s reference architecture and open source components. We offer flexible purchasing options to accommodate a variety of procurement processes. Free Tier Support is available for anyone using Cloud Posse’s **open source projects**. It’s a great way to get **public, best-effort help** from the Cloud Posse team and the broader community. You can ask questions and connect through **SweetOps Slack**, **Public Office Hours**, and **GitHub Discussions & Issues**. - **Public, best-effort help**—available to all users of our open source projects. - **Engage with the community**—via SweetOps Slack, Public Office Hours, and GitHub Discussions & Issues. - **Learn from past discussions**—browse previously answered questions and solutions. Just a heads up: Free Tier Support **doesn’t include guaranteed response times or confidentiality**. It’s not intended for support of our commercial reference architectures (Quickstart, Jumpstart) or any paid services. The Cloud Posse team participates when we can—but responses are always best-effort. Learn More Essential Support is for **self-sufficient teams** building on Cloud Posse’s **reference architecture and open source components** who want **priority asynchronous support** to stay unblocked and save time. It includes prioritized responses in **GitHub Discussions** and access to twice-weekly **shared Customer Workshops**—a great way to learn and get help with common questions about architecture, components, and implementation patterns. - **Priority responses** in GitHub Discussions. - **2x Weekly shared Customer Workshops** via Zoom. Learn More Flexible Support is for teams running on Cloud Posse’s **reference architecture and open source components** who need **hands-on engineering work**—such as bug fixes, upgrades, Atmos enhancements, new components, and integration work. Work is scheduled in advance to fit your team’s priorities and ensure predictable delivery. - **Scheduled hands-on engineering**—bug fixes, upgrades, Atmos enhancements, new components, and integration work. - **Private Slack Connect channel**—coordinate active requests and schedule new work. We offer two purchasing options for Flexible Support: **Hourly Retainers** and **Pay-as-you-go**. Choose the option that fits your needs and budget. Learn More Platform Advisory is for teams running on Cloud Posse’s **reference architecture and open source components** who need **priority access** to senior Cloud Posse engineers. It’s ideal for **high-stakes work**—such as audits, migrations, architecture changes, or board-level deadlines—where delays or mistakes would be costly. - **Private Slack Connect channel**—direct access to staff-to-principal-level Cloud Posse engineers. - **On-demand Zoom sessions**—for architecture reviews, migration planning, compliance discussions, and more. - **Same-day response (4h SLA)**—for high-impact requests. - **Includes 10 hrs/month of Flexible Support**—for bug fixes, upgrades, Atmos enhancements, new components, and integration work. Platform Advisory is a **premium support level** for teams that want to move fast and reduce risk on mission-critical projects. Learn More ## Restrictions To ensure the highest level of service, we exclusively support customers using [Atmos](https://atmos.tools), our [AWS Reference Architecture](https://docs.cloudposse.com), and our [Terraform modules](https://docs.cloudposse.com/modules/) or [components for AWS](https://docs.cloudposse.com/components/). This focus allows us to provide expert guidance, maintain best practices, and deliver predictable outcomes tailored to AWS environments. --- ## Get Platform Advisory import Intro from '@site/src/components/Intro'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Note from '@site/src/components/Note'; import PrimaryCTA from '@site/src/components/PrimaryCTA'; import Admonition from '@theme/Admonition'; import FeatureList from '@site/src/components/FeatureList'; import ActionCard from '@site/src/components/ActionCard'; When you’re in a pinch, the clock is ticking, and leadership is watching—**Platform Advisory** gives you private access to senior Cloud Posse engineers who know your architecture and can chart the best path forward. Get answers, guidance, and clarity—without waiting for workshops or scheduling. When your team faces **high-stakes migrations, audits, architecture challenges, or board-level deadlines**, you can’t afford delays or costly missteps. Our **Platform Advisory** service gives your team priority access to Cloud Posse’s most senior engineers—so you can move forward with confidence. ## Includes Platform Advisory provides **on-demand access** to Cloud Posse’s senior engineering team for your mission-critical needs: - **Private Slack Connect Channel** – On-demand access to staff-to-principal Cloud Posse engineers. - **On-Demand Zoom Calls** – Architecture reviews, migration planning, compliance guidance, and more. - **4-hour SLA** – Guaranteed same-day response for high-impact issues. - **Includes 10 hrs/month of Flexible Support** – For bug fixes, PR reviews, Atmos enhancements, new components, and integration work. ## Why Cloud Posse? Our team has helped hundreds of companies implement AWS services using Infrastructure as Code (IaC) with Terraform, to deploy and manage AWS services such as Amazon EKS, AWS ECS, AWS Lambda, AWS IAM, Amazon RDS, Amazon S3, AWS Security Hub, and multi-account Organizations. We’ve built more **Terraform modules for AWS** than anyone else—[160+ and counting](https://github.com/cloudposse/). If it’s AWS and Terraform, we’ve probably already solved it. When leadership is watching, deadlines are looming, and your team can’t afford to get stuck—**Platform Advisory** is the safety net that helps your platform teams deliver under pressure. When your team is under pressure, you don’t have time to waste. Get private access to the Cloud Posse engineers who know your architecture inside and out. --- ## Terraform Component GitHub Repository Has Moved! import Intro from '@site/src/components/Intro'; import Steps from '@site/src/components/Steps'; The GitHub repository for Cloud Posse's Terraform components has migrated to a dedicated GitHub organization. All documentation remains here, but all future updates, contributions, and issue tracking for the source code should now be directed to the respective repositories in the new organization. # Hello, Cloud Posse Community! We're excited to announce that starting on November 12, 2024, we will begin migrating each component in the `cloudposse/terraform-aws-components` repository to individual repositories under a new GitHub organization. This change aims to improve the stability, maintainability, and usability of our components. ## Why This Migration? Our goal is to make each component easier to use, contribute to, and maintain. This migration will allow us to: - Leverage terratest automation for better testing - Implement semantic versioning to clearly communicate updates and breaking changes - Improve PR review times and accelerate community contributions - Enable Dependabot automation for dependency management - And much more! ## What to Expect Starting November 12, 2024 ### Migration Timeline The migration will begin on November 12 and is anticipated to finish by the end of the following week. ### Code Freeze Starting on November 12, this repository will be set to read-only mode, marking the beginning of a code freeze. No new pull requests or issues will be accepted here after that date. ### New Contribution Workflow After the migration, all contributions should be directed to the new individual component repositories. ### Updated Documentation To support this transition, we are updating our documentation and cloudposse-component updater. ### Future Archiving In approximately six months, we plan to archive this repository and transfer it to the cloudposse-archives organization. ## Frequently Asked Questions ### Does this affect Terraform modules? No, only the `terraform-aws-components` repository is affected. Our Terraform modules will remain where they are. We are committed to making this transition as seamless as possible. If you have any questions or concerns, please feel free to post them in this issue. Your feedback is important to us, and we appreciate your support as we embark on this new chapter! --- ## Automated Component Testing import Intro from '@site/src/components/Intro'; We're excited to announce the completion of the second phase of our Component Testing project, which has added automated testing for 27 components. This milestone follows our successful migration of 160+ Terraform Components from a monorepo to individual repositories, making them more maintainable and testable. Hello SweetOps! A few months ago, we embarked on a MASSIVE project to enable Component Testing. The goal is to improve the stability of our components, detect and fix integration errors, and pave the way for confident delivery of new features. In the first phase, we split the [`cloudposse/terraform-aws-components`](https://github.com/cloudposse/terraform-aws-components) monorepo consisting of 160+ Terraform Components into individual repositories in the [`cloudposse-terraform-components`](https://github.com/cloudposse-terraform-components) GitHub organization. We updated the [`cloudposse/github-action-atmos-component-updater`](https://github.com/cloudposse/github-action-atmos-component-updater) GitHub action to rewrite URLs in component manifests automatically, allowing you to smoothly migrate to new repositories. ## Current Status Now, we are happy to announce that we have completed the second phase of this project, introducing automated tests for the first 27 components. Hopefully, you are already using components from the [new organization](https://github.com/cloudposse-terraform-components)! The complete list of covered components can be found [here](https://github.com/orgs/cloudposse-terraform-components/projects/1/views/3). We've developed a Go-based [testing framework](https://github.com/cloudposse/test-helpers) built on top of Terratest, optimized specifically for testing Atmos components. Additionally, we created a [step-by-step guide](https://docs.cloudposse.com/community/contribute/component-testing/) to help you write effective component tests. You can track the project's progress on [this board](https://github.com/orgs/cloudposse-terraform-components/projects/1/views/1). We invite everyone to contribute to this project. Please like the "Add component tests" issue in the corresponding component repository for which you are interested in prioritizing test coverage. If you want to contribute more, we have the opportunity. ## How can you help? We really need help writing tests. You can take any "Add component tests" issue with the "Good First Question" label and contribute to the test following our [documentation](https://docs.cloudposse.com/community/contribute/component-testing/). We will prioritize reviewing your PRs in the `#pr-reviews` channel and help ensure they get merged smoothly.. Feel free to DM to `@Erik Osterman` or `@Igor Rodionov` in Slack with any questions or feedback. :::tip Join the Conversation! Want to help shape the future of our Terraform components? We're building it *in the open* and you're invited. Join us in the [SweetOps Slack](https://cloudposse.com/slack) to chat about component testing, automation, and all things Terraform. ::: P.S.: Huge thanks to `@RoseSecurity` for the first community-driven component test contribution [here](https://github.com/cloudposse-terraform-components/aws-vpc/pull/25). --- ## Announcing Platform Advisory import FeatureList from '@site/src/components/FeatureList'; import Intro from '@site/src/components/Intro'; We’re excited to announce our new **Platform Advisory** service—now available to Cloud Posse customers. Private access to Cloud Posse engineers. Many of our larger customers—especially in **fintech, health tech**—who operate in regulated industries have asked for a way to get **private, real-time access** to senior Cloud Posse engineers for their most critical projects. These teams often run into scenarios where: - **Delays, mistakes, or failed migrations would cost big** - They need to **de-risk complex platform changes** - They want trusted guidance on **Cloud Posse architecture and components**—from the engineers who built it **Platform Advisory** was designed specifically to address these needs. --- ## What is Platform Advisory? Platform Advisory gives your team: - **Private Slack Connect** → direct access to Cloud Posse’s staff-to-principal-level engineers - **On-demand Zoom sessions** → architecture reviews, migration planning, compliance discussions, and more - **Same-day response (4-hour SLA)** → for high-impact requests - **10 hrs/month of Flexible Support included** → for bug fixes, upgrades, Atmos enhancements, new components, and integration work It’s designed for teams running on **Cloud Posse’s reference architecture and open source components** who need priority access to expert guidance—especially when getting it wrong isn’t an option. --- ## Why we built this As more of our customers adopt Cloud Posse architecture for **mission-critical platforms**, they’ve asked for a way to engage more deeply—especially for projects where **de-risking migrations and accelerating delivery** matters. When the team is facing a complex migration, rolling out new environments, or making high-impact platform changes—**waiting days for answers isn’t good enough**. Platform Advisory gives them **priority access** to engineers who: - Know Cloud Posse architecture inside and out - Understand their environments and goals - Can chart the best path forward quickly and safely --- ## How it fits with our other support options | Support Option | Best For | | --------------------- | ----------------------------------------------------------------- | | **Essential Support** | Self-service teams who want async guidance | | **Flexible Support** | Scheduled hands-on engineering work | | **Platform Advisory** | Teams where delays, mistakes, or failed migrations would cost big | ## Where to learn more You can explore all the details on the [Platform Advisory support page](/support/platform-advisory). If you’re unsure whether Platform Advisory is the right fit for your team, [reach out to us](/support)—we’re happy to help. **Remember**: When you invest in Cloud Posse, you’re not just helping your team—you’re strengthening the ecosystem your business depends on. We’re excited to make **Platform Advisory** available—and we look forward to helping more teams succeed on Cloud Posse architecture. --- ## Introducing Our Component Deprecation Process import Intro from '@site/src/components/Intro'; We've documented our formal process for deprecating and archiving components to ensure transparency and give our community adequate notice when repositories are being sunset. Hello SweetOps! As part of our commitment to maintaining [300+ open source projects](https://github.com/cloudposse/) across Terraform modules, [components](https://github.com/cloudposse-terraform-components), and other tooling, we occasionally need to deprecate repositories that are no longer actively maintained or have been superseded by better alternatives. ## What to Expect We've added comprehensive documentation outlining our [Deprecation and Archival Process](/community/contribute/our-github#deprecation-and-archival-process) to ensure this transition is as smooth as possible for everyone in our community. When we deprecate a repository, here's what you can expect: 1. **GitHub Issue Created**: A pinned issue with detailed explanation, timeline, and migration guidance 2. **README Warnings Added**: Prominent deprecation notices at the top of documentation 3. **Blog Post Published**: Announcement in our changelog/blog about the deprecation 4. **Pull Request Submitted**: All changes announced via PR for community visibility 5. **Grace Period**: Typically 90+ days for the community to migrate and ask questions 6. **Repository Archived**: After the grace period, repos are archived (not deleted) and remain publicly accessible 7. **Blog Post Updated**: Announcement updated to reflect the archival completion ## Why This Matters This structured approach ensures that: - You have advance notice before any repository is archived - Migration paths and alternatives are clearly documented - Historical access to code is preserved - The community can provide feedback during the deprecation period ## Our Commitment As stated in our [GitHub documentation](/community/contribute/our-github), we commit to always provide free and public access to our Open Source repositories. Even when archived, repositories remain accessible for historical reference and continued use. :::tip Questions? If you have questions about deprecated components or need migration assistance, reach out in the [SweetOps Slack](https://cloudposse.com/slack) or [GitHub Discussions](https://github.com/orgs/cloudposse/discussions). ::: --- ## Why We Recommend Managed Node Groups Over Fargate for EKS Add-Ons import FeatureList from '@site/src/components/FeatureList'; import Intro from '@site/src/components/Intro'; When simplicity meets automation, sometimes it's the hidden complexity that bites back. For a while, running Karpenter on AWS Fargate sounded like a perfect solution. No nodes to manage, automatic scaling, and no EC2 lifecycle headaches. The [AWS EKS Best Practices Guide](https://aws.github.io/aws-eks-best-practices/karpenter/#run-the-karpenter-controller-on-eks-fargate-or-on-a-worker-node-that-belongs-to-a-node-group) and [Karpenter's official documentation](https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/) both present Fargate as a viable option for running the Karpenter controller. But in practice, that setup started to cause problems for certain EKS add-ons. Over time, those lessons led us — and our customers — to recommend using a small managed node group (MNG) instead of relying solely on Fargate. **This recommendation diverges from some official AWS guidance**, and we acknowledge that. Here's why we made this decision. ## Why Fargate Was Attractive (and Still Is, Sometimes) The appeal of Fargate for Karpenter is understandable: - No need to bootstrap a managed node group before deploying Karpenter - Simpler initial setup for teams not using Infrastructure-as-Code frameworks - Karpenter's early versions had limited integration with managed node pools - It showcased Karpenter's capabilities in the most dramatic way possible For teams deploying clusters manually or with basic tooling, Fargate eliminates several complex setup steps. But when you're using sophisticated Infrastructure-as-Code like [Cloud Posse's Terraform components](https://docs.cloudposse.com/components/), that initial complexity is already handled—and the operational benefits of a managed node group become far more valuable. ## The Problem with "No Nodes" (and the Terraform Catch-22) EKS cluster creation with Terraform requires certain managed add-ons — like CoreDNS or the EBS CSI driver — to become active before Terraform considers the cluster complete. But Fargate pods don't exist until there's a workload that needs them. That means when Terraform tries to deploy add-ons, there are no compute nodes for the add-ons to run on. Terraform waits… and waits… until the cluster creation fails. Terraform enforces a strict dependency model: it won't complete a resource until it's ready. Without a static node group, Terraform can't successfully create the cluster (because the add-ons can't start). And without those add-ons running, Karpenter can't launch its first node (because Karpenter itself is waiting on the cluster to stabilize). This circular dependency means your beautiful "fully automated" Fargate-only cluster gets stuck in the most ironic place: **bootstrap deadlock**. You can manually retry or patch things later, but that defeats the purpose of automation. We build for repeatability — not babysitting. ## The Hidden Cost of "Serverless Nodes" Even after getting past cluster creation, there are subtle but serious issues with high availability. By AWS and Cloud Posse best practices, production-grade clusters should span three availability zones, with cluster-critical services distributed across them. However, during initial scheduling with **managed node groups**, Karpenter might spin up just one node large enough to fit all your add-on pods — even if they request three replicas with anti-affinity rules. Kubernetes will happily co-locate them all on that single node. Once they're running, those pods don't move automatically, even as the cluster grows. The result? **A deceptively healthy cluster with all your CoreDNS replicas living on the same node in one AZ — a single point of failure disguised as a distributed system.** While [topologySpreadConstraints](https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/) can help encourage multi-AZ distribution, they don't guarantee it during the critical cluster bootstrap phase when Karpenter is creating its first nodes. ## The Solution: A Minimal Managed Node Pool Our solution is simple: **Deploy a tiny managed node group — one node per availability zone — as part of your base cluster.** - This provides a home for cluster-critical add-ons during creation - It ensures that CoreDNS, EBS CSI, and other vital components are naturally distributed across AZs - It gives Karpenter a stable platform to run on - And it eliminates the bootstrap deadlock problem entirely You can even disable autoscaling for this node pool. One node per AZ is enough. Think of it as your cluster's heartbeat — steady, predictable, and inexpensive. ## Additional Fargate Constraints Beyond the HA challenges, [Fargate has architectural constraints](https://docs.aws.amazon.com/eks/latest/userguide/fargate-pod-configuration.html) that can affect cluster add-ons: - Each Fargate pod runs on its own isolated compute resource (one pod per node) - No support for EBS-backed dynamic PVCs; only EFS CSI volumes are supported - Fixed CPU and memory configurations with coarse granularity - 256 MB memory overhead for Kubernetes components While these constraints don't necessarily prevent Fargate from working, they add complexity when running cluster-critical infrastructure that needs precise resource allocation and high availability guarantees. ## Cost and Flexibility Fargate offers convenience, but at a premium. A pod requesting 2 vCPUs and 4 GiB of memory costs about **$0.098/hour**, compared to **$0.076/hour** for an equivalent EC2 c6a.large instance. And because [Fargate bills in coarse increments](https://docs.aws.amazon.com/eks/latest/userguide/fargate-pod-configuration.html), you often overpay for partial capacity. By contrast, the hybrid approach unlocks significant advantages: - Static MNG with On-Demand instances provides a stable foundation for cluster add-ons - Use cost-effective Graviton instances (c7g.medium) to reduce baseline costs - Karpenter provisions Spot instances exclusively for application workloads (not add-ons) - Achieve cost savings on application pods while maintaining reliability for cluster infrastructure The result: **stable cluster services on On-Demand, cost-optimized applications on Spot**. ## The Evolution of Karpenter's Recommendations Interestingly, the Karpenter team's own guidance has evolved over time. [Karpenter's current getting started guide](https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/) now defaults to using **EKS Managed Node Groups** in its example configurations, with Fargate presented as an alternative that requires uncommenting configuration sections. While we can't pinpoint exactly when this shift occurred, it suggests the Karpenter team recognized that managed node groups provide a more reliable foundation for most production use cases. ## Lessons Learned At Cloud Posse, we love automation — but we love reliability through simplicity even more. Running Karpenter on Fargate works for proof-of-concepts or ephemeral clusters. But for production systems where uptime and high availability matter, a hybrid model is the clear winner: - Static MNG with On-Demand instances for cluster-critical add-ons (CoreDNS, Karpenter, etc.) - Karpenter provisioning Spot instances for dynamic application workloads - Fargate only when you truly need pod-level isolation It's not about Fargate being bad — it's about knowing where it fits in your architecture. ## When Fargate-Only Might Still Work To be fair, there are scenarios where running Karpenter on Fargate might make sense: - Long-lived development environments where the $120/month MNG baseline cost matters more than availability - Clusters deployed manually (not via Terraform) where bootstrap automation isn't critical - Proof-of-concept deployments demonstrating Karpenter's capabilities - Organizations that have accepted the operational trade-offs and built workarounds **However**, be aware that development clusters that are frequently rebuilt will hit the Terraform bootstrap deadlock problem more often—making automation failures a regular occurrence rather than a one-time setup issue. ## Your Mileage May Vary It's worth noting that [experienced practitioners in the SweetOps community](https://sweetops.slack.com/) have successfully run Karpenter on Fargate for years across multiple production clusters. Their setups work, and they've built processes around the constraints. This proves our recommendation isn't absolute—some teams make Fargate work through careful configuration and accepted trade-offs. However, these same practitioners acknowledged they'd likely choose MNG if starting fresh today with modern tooling. > "Karpenter doesn't use voting. Leader election uses Kubernetes leases. There's no strict technical requirement to have three pods — unless you actually care about staying up." > > — Ihor Urazov, SweetOps Slack That's the key insight. The technical requirements are flexible—it's your operational requirements that determine the right choice. If staying up matters, if automation matters, if avoiding manual intervention matters, then give your cluster something solid to stand on. A small, stable managed node pool does exactly that. ## What About EKS Auto Mode? It's worth mentioning that AWS introduced [EKS Auto Mode](https://docs.aws.amazon.com/eks/latest/userguide/automode.html) in December 2024, which takes a fundamentally different approach to solving these problems. EKS Auto Mode runs Karpenter and other critical cluster components (like the EBS CSI driver and Load Balancer Controller) **off-cluster** as AWS-managed services. This elegantly sidesteps the bootstrap deadlock problem entirely—there's no chicken-and-egg dependency because the control plane components don't need to run inside your cluster. The cluster starts with zero nodes and automatically provisions compute capacity as workloads are scheduled. While this solves the technical bootstrap challenge we've discussed, it comes with trade-offs: - Additional 12-15% cost premium on top of EC2 instance costs - Lock-in to AWS VPC CNI (can't use alternatives like Cilium or Calico) - Less control over cluster infrastructure configuration - Available only for Kubernetes 1.29+ and not in all AWS regions For organizations willing to accept these constraints in exchange for fully managed operations, EKS Auto Mode may address many of the concerns raised in this post. However, for teams requiring fine-grained control, cost optimization, or running on older Kubernetes versions, the MNG + Karpenter approach remains highly relevant. --- ## Making Our Docs AI-Friendly with llms.txt import Intro from '@site/src/components/Intro'; We've implemented the llms.txt standard to make our documentation more accessible to AI assistants, ensuring better responses when you ask ChatGPT, Claude, or other LLMs about Cloud Posse tools and best practices. Hello SweetOps! As AI assistants become increasingly integrated into developer workflows, we're excited to announce support for the [llms.txt standard](https://llmstxt.org/) across our documentation site. ## What is llms.txt? Think of `llms.txt` as the AI equivalent of `robots.txt` for search engines. It's an emerging standard that provides LLMs with structured, curated documentation in a format optimized for their understanding. Rather than crawling entire websites and hitting context window limits, AI assistants can now access our documentation in two curated formats: - **[/llms.txt](https://docs.cloudposse.com/llms.txt)** - A compact list of important documentation links - **[/llms-full.txt](https://docs.cloudposse.com/llms-full.txt)** - Full documentation content in markdown format ## Why This Matters When you ask an AI assistant about Atmos, Terraform components, or Cloud Posse best practices, you'll get more accurate and up-to-date responses. The assistant can reference our curated documentation directly instead of relying on training data that may be outdated or incomplete. ### Benefits for the Community - **Better AI Assistance**: More accurate responses when asking AI tools about our projects - **Efficient Context Usage**: LLMs can access precisely what they need without crawling - **Up-to-Date Information**: Always references current documentation, not stale training data - **Developer Velocity**: Faster answers means less time searching, more time building ## How It Works We're using the [`docusaurus-plugin-llms`](https://github.com/rachfop/docusaurus-plugin-llms) to automatically generate these files from our Docusaurus site. The plugin: 1. Prioritizes core documentation sections (Atmos, components, tutorials) 2. Includes blog content for recent updates and announcements 3. Generates both compact and full-text versions 4. Automatically updates with each documentation deployment ## Try It Out Next time you're working with an AI assistant, try asking questions about: - "How do I configure Atmos stacks?" - "What's the latest on Cloud Posse component deprecation?" - "Show me examples of Terraform component patterns" The assistant will have direct access to our structured documentation, leading to better, more accurate responses. ## Join the Movement The llms.txt standard is gaining adoption across the developer community. If you maintain documentation, consider implementing it for your projects. It's a simple way to make your content more accessible to the AI tools your users are already using. :::tip Questions? Have feedback about our AI-friendly documentation? Join us in the [SweetOps Slack](https://cloudposse.com/slack) or [GitHub Discussions](https://github.com/orgs/cloudposse/discussions). :::