Skip to main content
Latest Documentation
This is the latest documentation for the Cloud Posse Reference Architecture. To determine which version you're currently using, please see Version Identification.
Version: Latest
Sponsor @cloudposse

How to Setup Vanity Domains with an ALB on EKS

Learn how to setup vanity domains on an existing ALB together with Service Discovery Domains.

Pre-requisites​

Context​

After setting up your Network Architecture you will have 2 hosted zones in each platform account.

In dev for example, you will have Hosted Zones for dev-acme.com and dev.platform.acme.com.

You should also have an ACM certificate that registers *.dev-acme.com and *.dev.platform.acme.com.

We also should've deployed applications to your EKS cluster and have an ALB for service discovery. For example the echo-server component.

Now we want to set up a vanity subdomain for dev-acme.com that will point to the ALB used for service discovery. This saves us money by not requiring a new ALB for each vanity domain.

EKS Auto Mode

With EKS Auto Mode, AWS manages the ALB controller natively. Use ingressClassName: alb in your Ingress spec instead of the kubernetes.io/ingress.class annotation. The standard alb.ingress.kubernetes.io/* annotations are fully supported.

Implementation​

This is fairly simple to implement. All we need to do is set up our Kubernetes ingresses and ensure ACM doesn't have duplicate certs for domains.

1 Setup Ingresses​

Ingresses for your applications can use several different .spec.rules to provide access to the application via many different URLs. With EKS Auto Mode, the ALB controller is managed by AWS and supports all standard ALB annotations.

Example​

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/group.name: ingress-group
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS":443}]'
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/ssl-redirect: "443"
    alb.ingress.kubernetes.io/target-type: ip
    external-dns.alpha.kubernetes.io/hostname: my-app-api.dev.plat.acme-svc.com
  name: my-app-api
  namespace: dev
spec:
  ingressClassName: alb
  rules:
    # new Vanity Domain
    - host: api.dev-acme.com
      http:
        paths:
          - backend:
              service:
                name: my-app-api
                port:
                  number: 8081
            path: /api
            pathType: Prefix
    # Existing Service discovery domain
    - host: my-app-api.dev.plat.acme-svc.com
      http:
        paths:
          - backend:
              service:
                name: my-app-api
                port:
                  number: 8081
            path: /
            pathType: Prefix

Key annotations:

AnnotationPurpose
alb.ingress.kubernetes.io/schemeinternet-facing for public ALBs, internal for private
alb.ingress.kubernetes.io/target-typeip routes traffic directly to pod IPs (required for Auto Mode)
alb.ingress.kubernetes.io/group.nameGroups multiple Ingresses to share a single ALB
alb.ingress.kubernetes.io/listen-portsConfigure HTTP/HTTPS listeners
alb.ingress.kubernetes.io/ssl-redirectRedirect HTTP to HTTPS on the specified port
alb.ingress.kubernetes.io/backend-protocolProtocol for communicating with pods (HTTP or HTTPS)
external-dns.alpha.kubernetes.io/hostnameTells external-dns to create DNS records for this host
tip

Use spec.ingressClassName: alb instead of the deprecated kubernetes.io/ingress.class annotation. The eks/ingress-class component sets up the alb IngressClass automatically.

2 Setup ACM Certs​

By default, our dns-primary component and dns-delegated component will create ACM certs for each Hosted Zone in the platform account, along with an additional cert for *.dev-acme.com. Depending on the level of subdomains you want, you may need to disable this with the variable request_acm_certificate: false

If a single subdomain is sufficient. e.g. api.dev-acme.com then you can leave this enabled.

The important thing to note is that you cannot have duplicate certs in ACM. So if you want to add a new subdomain, you will need to delete the existing cert for *.dev-acme.com and create a new one with the new subdomain. This can lead to issues when trying to delete certificates, as they are in use by the ALB. You will need to delete the ALB first, then delete the certificate.

See the troubleshooting section if you run into issues with recreating resources.

How it works​

With a single valid ACM cert for your domains, the ALB controller (managed by EKS Auto Mode) registers your domain to the ALB. The ALB recognizes the valid certificate in ACM automatically. This is why we need to ensure we have a valid certificate for our domains.

You can validate your cert is picked up by the ALB by checking the ALB's target group. You should see the certificate listed under the Certificates tab.

Troubleshooting​

The problem with this comes when you need to remove a subdomain or ACM certificate. By running atmos terraform deploy dns-delegated -s plat-<region>-dev with request_acm_certificate: false, you are trying to destroy a single ACM certificate in an account. While this is a small scope deletion, the ACM certificate is in use by the ALB, and the ALB has many different targets. Thus Terraform will stall out.

You need to:

  1. Delete the listeners and targets of the ALB that are using the certificate
  2. Delete the ALB
  3. Terraform will then successfully delete the ACM certificate.

You will notice:

  1. The ALB will be recreated
  2. Ingresses should reconcile for service discovery domains
  3. ALB Targets should be recreated pointing at service discovery domains.

Once you recreate the correct ACM certificates and have valid ingresses you should be able to access your applications via the vanity domain.