Our automated testing strategy and resources
Nearly all of our Terraform modules are updated with automated tests. We have 2 general strategies. First is by running some
bats tests that do some basic sanity checks.
These tests are defined in our
test-harness and generally don’t require any credentials to run. Since that’s the case, they can really only do basic linting and static analysis.
Then we have some tests which are based on the
terratest library for infrastructure testing that do more in-depth integration tests of module functionality.
Philosophy of Testing
Here’s our philosophy on infrastructure testing. At a minimum, we like to test to ensure that all of our modules cleanly
destroy. This catches 80% of the problems with only 20% of the effort. We also want to test that the outputs are what we would expect. On the other hand, we don’t want to retest what already is tested by HashiCorp in their providers. So for example, we have our
terraform-aws-s3-bucket module that creates an S3 bucket. We don’t need to test that a bucket is created; we assume that would be caught by the upstream terraform tests. But we do want to test that the bucket name is what we expect it to be, since this is an opinionated aspect of the module.
We’ve setup some basic “ChatOps” style functionality to run tests. This is required in order to securely run infrastructure integration tests without exposing integration secrets. More than this, it’s a hard limitation imposed by GitHub and the only way we can run the tests from Pull Requests originating from forks.
Our tests run against some testing infrastructure that we host in an isolated account on AWS, strictly for the purposes of testing infrastructure.
The workflow is defined in the
cloudposse/actions repository. The benefit with this is we have one place to control the testing
workflow for all of our hundreds of terraform modules. The downside, however, with dispatched workflows is that the workflows always run from the
To run the tests, you’ll need to issue some commands by posting GitHub comments on the PR. Note, only users that have been added to our
cloudposse organization as a contributor are able to run these commands. If you’re not a contributor,
the commands will have no effect and there will be no error messages or warnings.
||Run bats automated tests|
All tests run from the
If you’re a contributor who wants to intialize one of our terraform modules, this is the process. Note, if a repo has already been configured for chatops, there’s no need to proceed with these steps.
To initialize one of our modules with chatops, run the following commands:
git clonethe terraform module repository
cd $repoto enter the repository directory
make initto initialize the build-harness
make github/initto write all the scaffolding
git add *to add the changes
git rm -rf codefresh/to remove legacy codefresh pipelines no longer needed
- Add the build badge to the
badgessection. Also, remove any Codefresh badge, since that’s also no longer needed
make readmeto rebuild the
README.md(remember, never edit the
README.mdmanually since it’s generated from the
- Open up a Pull Request with the changes. Here is a good example.
- Request a Code Review in the
#pr-reviewsSlack channel (and big thanks for your contribution!)