Using Secrets with CI/CD

Learn how we recommend managing secrets as part of CI/CD pipelines.

We advocate using chamber with all CI/CD pipelines to access secrets. Chamber provides a universal interface and a single “system of record” for storing and rotating secrets.

Enabling CI/CD systems to access chamber depends on the CI/CD solution. For CodeBuild/CodePipeline, it’s possible to leverage IAM Roles to grant access to KMS+SSM, while for other systems which are external to AWS, it’s necessary to provision a chamber IAM user with a very limited scope (the link provides an example for how we do this using terraform).

Using Chamber with Codefresh

If using Codefresh, there a few things that need to happen.

  1. Provision a terraform-aws-iam-chamber-user for each AWS account that Codefresh should deploy to.
  2. Add the chamber user’s IAM credentials as encrypted environment variables in the Codefresh pipeline.
  3. Leverage chamber in one of your build-steps using our build-harness.

In the example below, we assume the AWS credentials have already been exported in the Codefresh pipeline. We then proceed to invoke chamber exec kops $NAMESPACE from the cloudposse/build-harness which will pull in the secrets from the kops service namespace as well as the $NAMESPACE service namespace, if any exist. This strategy allows any particular namespace to define it’s own specific environment variables.

Using Chamber with Codefresh

  deploy_helmfile:
    title: Deploy app with helmfile
    image: cloudposse/build-harness:${{BUILD_HARNESS_VERSION}}
    environment:
      - AWS_REGION=us-west-2
      - AWS_ACCESS_KEY_ID=${{AWS_ACCESS_KEY_ID}}          # for chamber
      - AWS_SECRET_ACCESS_KEY=${{AWS_SECRET_ACCESS_KEY}}  # for chamber
      - CHART_NAME=example
      - CHART_REPO_URL=${{STAGING_REPO_ENDPOINT}}
      - CHART_VERSION=${{SEMVERSION_COMMIT_SHORT}}
      - IMAGE_TAG=${{SEMVERSION_COMMIT_SHORT}}
      - INGRESS_HOST=${{APP_HOST}}
      - KUBE_CONTEXT=us-west-2-example-cloudposse-org
    commands:
      # Install or upgrade tiller
      - "make helm/toolbox/upsert"
      # Update values.yaml with envs from chamber
      - "chamber exec kops ${NAMESPACE} -- envsubst < config/chart.yaml > config/${RELEASE_NAME}.yaml"
      # Deploy chart to cluster using helmfile (with chamber secrets)
      - "chamber exec kops ${NAMESPACE} -- helmfile --file config/helmfile.yaml --selector component=app sync --concurrency 1 --args '--wait --timeout=600 --recreate-pods --force --reset-values'"
    when:
      condition:
        all:
          executeForNamespace: "'${{NAMESPACE}}' != ''"