Decide on S3 Bucket Requirements
Problem
Your organization depends on many S3 buckets, but they all have different purposes, with different requirements. Some are public for serving assets, while others are storing sensitive uploads for customer data. You want to reduce the friction for developers to create new buckets while ensuring some uniformity and standardized policies for how buckets are created. Buckets will be created in multiple stages and seldom if ever shared across stages. Bucket names in AWS S3 are globally unique, so we’ll need to have a convention to name them.
Considerations
We’ll use the Terraform to generate bucket names, so a short name for each bucket is all that is required. If you don’t yet know what buckets you will need, then we can provision some dummy buckets as examples.
-
Short name to describe the bucket (without stage or account)
-
Define the archetypes/classes of buckets used. The most common types we see are:
-
Static configurations (e.g. downloaded by mobile clients or EC2 instances)
-
Static assets (e.g. images, videos, thumbnails, uploads)
-
Websites, SPAs, Cloudfront origins
-
Log buckets (e.g. ALB access logs, Cloudtrail Logs, VPC Flow Logs, etc)
-
Artifact buckets (E.g. for CI/CD, binary executables)
-
SFTP/upload buckets
-
Define the lifecycle requirements of objects
-
Public/private ACLs
-
Encryption at Rest
-
Access logs? aggregation of access logs to a centralized location across all accounts?
-
Cloudfront integration
Consequences
-
Catalog configurations will be created for each bucket archetype.
-
Buckets will be provisioned using the s3-bucket component