External DNS

The external-dns controller synchronizes exposed Kubernetes Services and Ingresses with DNS providers like Route53.

Prerequisites

This assumes you’ve followed the Geodesic Module Usage with Terraform guide which covers all the scaffolding necessary to get started.

Dependencies

Installation

Provision IAM Role

Create a file in /conf/kops-aws-platform/external-dns.tf with the following content

External DNS IAM Role

module "kops_external_dns" {
  source       = "git::https://github.com/cloudposse/terraform-aws-kops-external-dns.git?ref=tags/0.1.2"
  namespace    = "${module.identity.namespace}"
  stage        = "${module.identity.stage}"
  name         = "external-dns"
  cluster_name = "${module.identity.aws_region}.${module.identity.zone_name}"

  tags = {
    Cluster = "${module.identity.aws_region}.${module.identity.zone_name}"
  }
}

output "kops_external_dns_role_name" {
  value = "${module.kops_external_dns.role_name}"
}

output "kops_external_dns_role_unique_id" {
  value = "${module.kops_external_dns.role_unique_id}"
}

output "kops_external_dns_role_arn" {
  value = "${module.kops_external_dns.role_arn}"
}

output "kops_external_dns_policy_name" {
  value = "${module.kops_external_dns.policy_name}"
}

output "kops_external_dns_policy_id" {
  value = "${module.kops_external_dns.policy_id}"
}

output "kops_external_dns_policy_arn" {
  value = "${module.kops_external_dns.policy_arn}"
}

Rebuild the Geodesic Module

Rebuild the module

> make build

Start the Geodesic Shell

Run the Geodesic shell followed by assume-role

sh-3.2$ $CLUSTER_NAME

Run the Geodesic Shell

sh-3.2$ staging.example.com
# Mounting /home/goruha into container
# Starting new staging.example.com session from cloudposse/staging.example.com:dev
# Exposing port 41179
* Started EC2 metadata service at http://169.254.169.254/latest

         _              _                                              _
     ___| |_ __ _  __ _(_)_ __   __ _    _____  ____ _ _ __ ___  _ __ | | ___
    / __| __/ _` |/ _` | | '_ \ / _` |  / _ \ \/ / _` | '_ ` _ \| '_ \| |/ _ \
    \__ \ || (_| | (_| | | | | | (_| | |  __/>  < (_| | | | | | | |_) | |  __/
    |___/\__\__,_|\__, |_|_| |_|\__, |  \___/_/\_\__,_|_| |_| |_| .__/|_|\___|
                  |___/         |___/                           |_|


IMPORTANT:
* Your $HOME directory has been mounted to `/localhost`
* Use `aws-vault` to manage your sessions
* Run `assume-role` to start a session


-> Run 'assume-role' to login to AWS
 ⧉  staging example
❌   (none) ~ ➤

Then login to AWS by running assume-role:

Assume role

❌   (none) conf ➤  assume-role
Enter passphrase to unlock /conf/.awsvault/keys/:
Enter token for arn:aws:iam::xxxxxxx:mfa/goruha: 781874
* Assumed role arn:aws:iam::xxxxxxx:role/OrganizationAccountAccessRole
-> Run 'init-terraform' to use this project
 ⧉  staging example
✅   (example-staging-admin) conf ➤

Provision Chamber Resources

Change directory to /conf/kops-aws-platform and run there commands to provision the external-dns backend.

init-terraform
terraform plan
terraform apply

From the Terraform outputs, copy the kops_external_dns_role_nameinto the ENV var EXTERNAL_DNS_IAM_ROLE with Chamber.

terraform apply

✅   (example-staging-admin) kops-aws-platform ➤  terraform apply
data.aws_caller_identity.current: Refreshing state...
data.aws_caller_identity.default: Refreshing state...
data.aws_iam_role.nodes: Refreshing state...
data.aws_route53_zone.default: Refreshing state...
data.aws_vpc.kops: Refreshing state...
data.aws_iam_role.masters: Refreshing state...
data.aws_iam_policy_document.assume_role: Refreshing state...
data.aws_iam_policy_document.default: Refreshing state...
data.aws_security_group.bastion: Refreshing state...
data.aws_subnet_ids.private: Refreshing state...
data.aws_security_group.masters: Refreshing state...
data.aws_security_group.nodes: Refreshing state...
data.aws_subnet_ids.utility: Refreshing state...

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + module.kops_external_dns.aws_iam_policy.default
      id:                    <computed>
      arn:                   <computed>
      description:           "Grant permissions for external-dns"
      name:                  "example-staging-external-dns"
      path:                  "/"
      policy:                "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"GrantModifyAccessToDomains\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"route53:ChangeResourceRecordSets\",\n      \"Resource\": \"arn:aws:route53:::hostedzone/Z2PQD3A7ZVDAIH\"\n    },\n    {\n      \"Sid\": \"GrantListAccessToDomains\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"route53:ListResourceRecordSets\",\n        \"route53:ListHostedZones\"\n      ],\n      \"Resource\": \"*\"\n    }\n  ]\n}"

  + module.kops_external_dns.aws_iam_role.default
      id:                    <computed>
      arn:                   <computed>
      assume_role_policy:    "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"sts:AssumeRole\",\n      \"Principal\": {\n        \"AWS\": \"arn:aws:iam::XXXXXXXXXXXX:role/masters.us-west-2.staging.example.com\",\n        \"Service\": \"ec2.amazonaws.com\"\n      }\n    }\n  ]\n}"
      create_date:           <computed>
      description:           "Role that can be assumed by external-dns"
      force_detach_policies: "false"
      max_session_duration:  "3600"
      name:                  "example-staging-external-dns"
      path:                  "/"
      unique_id:             <computed>

  + module.kops_external_dns.aws_iam_role_policy_attachment.default
      id:                    <computed>
      policy_arn:            "${aws_iam_policy.default.arn}"
      role:                  "example-staging-external-dns"

  + module.kops_external_dns.module.label.null_resource.default
      id:                    <computed>
      triggers.%:            "5"
      triggers.attributes:   ""
      triggers.id:           "example-staging-external-dns"
      triggers.name:         "external-dns"
      triggers.namespace:    "example"
      triggers.stage:        "staging"


Plan: 4 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.kops_external_dns.module.label.null_resource.default: Creating...
  triggers.%:          "" => "5"
  triggers.attributes: "" => ""
  triggers.id:         "" => "example-staging-external-dns"
  triggers.name:       "" => "external-dns"
  triggers.namespace:  "" => "example"
  triggers.stage:      "" => "staging"
module.kops_external_dns.module.label.null_resource.default: Creation complete after 0s (ID: 6903789342022752579)
module.kops_external_dns.aws_iam_role.default: Creating...
  arn:                   "" => "<computed>"
  assume_role_policy:    "" => "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"sts:AssumeRole\",\n      \"Principal\": {\n        \"AWS\": \"arn:aws:iam::XXXXXXXXXXXX:role/masters.us-west-2.staging.example.com\",\n        \"Service\": \"ec2.amazonaws.com\"\n      }\n    }\n  ]\n}"
  create_date:           "" => "<computed>"
  description:           "" => "Role that can be assumed by external-dns"
  force_detach_policies: "" => "false"
  max_session_duration:  "" => "3600"
  name:                  "" => "example-staging-external-dns"
  path:                  "" => "/"
  unique_id:             "" => "<computed>"
module.kops_external_dns.aws_iam_policy.default: Creating...
  arn:         "" => "<computed>"
  description: "" => "Grant permissions for external-dns"
  name:        "" => "example-staging-external-dns"
  path:        "" => "/"
  policy:      "" => "{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"GrantModifyAccessToDomains\",\n      \"Effect\": \"Allow\",\n      \"Action\": \"route53:ChangeResourceRecordSets\",\n      \"Resource\": \"arn:aws:route53:::hostedzone/Z2PQD3A7ZVDAIH\"\n    },\n    {\n      \"Sid\": \"GrantListAccessToDomains\",\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"route53:ListResourceRecordSets\",\n        \"route53:ListHostedZones\"\n      ],\n      \"Resource\": \"*\"\n    }\n  ]\n}"
module.kops_external_dns.aws_iam_role.default: Creation complete after 2s (ID: example-staging-external-dns)
module.kops_external_dns.aws_iam_policy.default: Creation complete after 2s (ID: arn:aws:iam::XXXXXXXXXXXX:policy/example-staging-external-dns)
module.kops_external_dns.aws_iam_role_policy_attachment.default: Creating...
  policy_arn: "" => "arn:aws:iam::XXXXXXXXXXXX:policy/example-staging-external-dns"
  role:       "" => "example-staging-external-dns"
module.kops_external_dns.aws_iam_role_policy_attachment.default: Creation complete after 3s (ID: example-staging-external-dns-20180523104524972800000001)

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

Outputs:

kops_external_dns_policy_arn = arn:aws:iam::XXXXXXXXXXXX:policy/example-staging-external-dns
kops_external_dns_policy_id = arn:aws:iam::XXXXXXXXXXXX:policy/example-staging-external-dns
kops_external_dns_policy_name = example-staging-external-dns
kops_external_dns_role_arn = arn:aws:iam::XXXXXXXXXXXX:role/example-staging-external-dns
kops_external_dns_role_name = example-staging-external-dns
kops_external_dns_role_unique_id = XXXXXXXXXXXXXXXXXXXXX
 ⧉  staging example
✅   (example-staging-admin) kops-aws-platform ➤

In the example the iam role name is kops_external_dns_role_name = example-staging-external-dns.

Install Chart

You can install external-dns in a few different ways, but we recomend to use the Master Helmfile.

Install with Master Helmfile

  1. Set with chamber the EXTERNAL_DNS_IAM_ROLE secret to IAM role name from previous step.
  2. Set with chamber the EXTERNAL_DNS_TXT_OWNER_ID secret to cluster name.
  3. Set with chamber the EXTERNAL_DNS_TXT_PREFIX secret to random string.
  4. Run then install external-dns using helmfile sync.

Install external-dns

chamber write kops EXTERNAL_DNS_IAM_ROLE example-staging-external-dns
chamber write kops EXTERNAL_DNS_TXT_OWNER_ID us-west-2.staging.example.com
chamber write kops EXTERNAL_DNS_TXT_PREFIX "$(uuidgen)-"
chamber exec kops -- helmfile -f /conf/kops/helmfile.yaml --selector namespace=kube-system,chart=external-dns sync

Replace with values to suit your specific project.

Install with Custom Helmfile

Add to your Kubernetes Backing Services Helmfile this code

helmfile.yaml

repositories:
- name: "stable"
  url: "https://kubernetes-charts.storage.googleapis.com"

releases:
- name: "dns"
  namespace: "kube-system"
  labels:
    chart: "external-dns"
    component: "ingress"
    namespace: "kube-system"
    vendor: "kubernetes-incubator"
    default: "true"
  chart: "stable/external-dns"
  version: "0.5.4"
  set:
    - name: "extraEnv.EXTERNAL_DNS_SOURCE"
      value: "service\ningress"

    - name: "txtOwnerId"
      value: 'us-west-2.staging.example.com'

    - name: "txtPrefix"
      value: '11591833-F9CE-407C-8519-35A947DB1D87-'

    - name: "publishInternalServices"
      value: "true"

    - name: "provider"
      value: "aws"

    - name: "podAnnotations.iam\\.amazonaws\\.com/role"
      value: 'example-staging-external-dns'

Then follow the instructions for running helmfile sync.

Usage

To leverage external-dns, you will need to add annotations (e.g. kubernetes.io/tls-acme: "true") to the Ingress resource.

With these in place, then kube-lego will handle issuing of TLS certificates from Let’s Encrypt and saving them to a Kubernetes secret specificied by the tls config parameter.

Here are some examples:

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: chartmuseum
  annotations:
    ingress.annotations.external-dns.alpha.kubernetes.io/target: "ingress.us-west-2.staging.example.com"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: chartmuseum-service
          servicePort: 80

values.yaml

## Ingress for load balancer
ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: nginx
    ingress.annotations.external-dns.alpha.kubernetes.io/target: "ingress.us-west-2.staging.example.com"
  hosts:
    example.com:
      - /charts
      - /index.yaml

helmfile.yaml

repositories:
- name: stable
  url: https://kubernetes-charts.storage.googleapis.com

releases:
- name: charts
  chart: stable/chartmuseum
  version: 1.3.1
  set:
  - name: ingress.enabled
    value: true
  - name: ingress.annotations.kubernetes\.io/ingress\.class
    value: nginx
  - name: ingress.annotations.external-dns\.alpha\.kubernetes\.io/target
    value: ingress.us-west-2.staging.example.com
  - name: ingress.hosts.example\.com[0]
    value: /

Note

There is no unified specification on how to structure helm chart values. Different charts may have very different structures of the value parameters. The only way to know for sure what is supported is to refer to the chart manifests.

The examples provided here are based on the stable/chartmuseum chart https://github.com/kubernetes/charts/blob/master/stable/chartmuseum