Scalable Systems on AWS

AWS sums up how to build good cloud systems in six pillars. Treat them as a review checklist: go through each one and ask how your workload measures up. Scaling mostly lives under Performance Efficiency and Reliability, but it affects all six.

• Operational excellence: run and improve the system, with automation in place of manual steps.
• Security: protect data and access, in every layer.
• Reliability: recover from failure and keep working as demand changes.
• Performance efficiency: use resources well as your needs grow.
• Cost optimization: avoid paying for what you do not use.
• Sustainability: reduce the energy and hardware your workload needs.

AWS is divided into Regions around the world, and each Region contains several Availability Zones: separate groups of data centres with their own power and networking. Spreading a system across zones is the cheapest big reliability win on AWS.

• Region: a geographic area, such as Singapore or Ireland. Choose one close to your users.
• Availability Zone: an isolated location inside a Region, close enough for very fast connections.
• Multi-AZ: run in at least two zones so one failure does not take you offline.
• Multi-Region: much harder and more expensive. Keep it for strict uptime or legal needs.

The cheapest request is the one that never reaches your servers. Route 53 answers DNS, CloudFront caches content at edge locations near users, and S3 stores your files.

• Amazon Route 53: DNS that can route by health, latency or weight.
• Amazon CloudFront: a CDN that serves cached content from locations close to each user.
• Amazon S3: object storage for images, downloads and whole static sites, with no servers to run.
• Offloading: moving images and files out of your app lets it scale far further for the same cost.

A load balancer sits in front of your servers and spreads requests across them. It also runs health checks, so traffic stops going to a server that has died. AWS runs it for you, across zones.

• Application Load Balancer: works at layer 7, reads HTTP, and routes by path or host.
• Network Load Balancer: works at layer 4 for very high throughput and low latency.
• Target group: the set of servers or containers that receive the traffic.
• Health checks: regular requests to each target. Failing targets are taken out of rotation.

Auto Scaling adds servers when demand rises and removes them when it falls, so you pay for what you use and survive traffic spikes. It needs your app to be stateless, so that any server can handle any request.

• Auto Scaling group: a fleet of EC2 instances with a minimum, a maximum and a desired size.
• Target tracking: you pick a target, such as 50% average CPU, and AWS adds or removes instances to hold it.
• Launch template: the recipe each new instance is built from.
• Stateless servers: keep sessions in ElastiCache or DynamoDB and files in S3, because any instance can disappear.
• Scaling up first: a larger instance needs no code change, but every instance size has a limit.

Containers package your app with everything it needs, so it runs the same everywhere. AWS has two main ways to run them, plus an option that removes the servers from view.

• Amazon ECR: a private registry where your container images are stored.
• Amazon ECS: the AWS container orchestrator. Simpler, and a good default on AWS.
• Amazon EKS: managed Kubernetes. Pick this when you need Kubernetes or already know it.
• AWS Fargate: runs containers for ECS or EKS with no EC2 instances to manage.

Lambda runs your code in response to events, with nothing to provision. Pair it with API Gateway and DynamoDB, and you get a whole backend that scales up on its own and costs almost nothing when idle.

• AWS Lambda: runs a function per request or event. Each run can last up to 15 minutes.
• Amazon API Gateway: gives your functions an HTTP front door.
• Concurrency: each request gets its own environment, so extra load means extra copies. Account limits still apply.
• Cold start: the extra delay when a new environment has to start.

If one part of a system must wait for another, a slowdown spreads. Queues and topics put a buffer in the middle, so a traffic spike becomes a backlog instead of an outage.

• Amazon SQS: a queue. Each message is handled by one consumer. Standard queues deliver at least once, so consumers must be safe to run twice.
• Amazon SNS: publish/subscribe. One message fans out to every subscriber.
• Amazon EventBridge: an event bus that routes events by rules.
• Dead-letter queue: messages that keep failing are parked for inspection instead of blocking the rest.

A cache keeps a copy of expensive results in memory. Done well, it cuts latency and takes load off your database. Done badly, it serves stale data with great confidence.

• Amazon ElastiCache: managed in-memory caching with engines such as Valkey and Redis OSS.
• DynamoDB Accelerator: DAX is an in-memory cache built specifically for DynamoDB.
• Cache-aside: check the cache, read the database on a miss, then store the result.
• Eviction: LRU drops the least recently used entry when full. A TTL expires entries after a set time.

Amazon RDS runs relational databases for you. Two separate features are often confused: Multi-AZ is for surviving failure, and read replicas are for handling more reads.

• Multi-AZ: keeps a synchronous standby in another zone and fails over automatically. It is for availability.
• Read replicas: asynchronous copies that take read traffic. They are for scale, and they can lag.
• Amazon Aurora: a MySQL and PostgreSQL compatible database with shared storage across zones and up to 15 read replicas.
• Connection pooling: RDS Proxy shares connections, which helps when Lambda or many instances connect at once.

DynamoDB is a key-value and document database built to stay fast at any size. It spreads data across partitions by hashing your partition key, so the design of that key decides how well it scales.

• Partition key: decides which partition holds an item. Choose one with many distinct values.
• Hot partition: one popular key can overload its partition however large the table is.
• On-demand mode: pay per request and let the table absorb traffic changes.
• Access patterns: design tables around your queries up front, because joins do not exist here.

Disks die, networks drop and deploys go wrong. Reliable systems expect all of this and limit how far the damage spreads.

• Redundancy: run at least two of everything important, ideally in different zones.
• Timeouts: never wait forever. One slow dependency can otherwise tie up every thread you have.
• Backoff and jitter: wait longer between retries and randomise the delay, or retries become their own traffic spike.
• Circuit breaker: after repeated failures, stop calling the sick service for a while and fail fast.
• Graceful degradation: turn off recommendations and keep checkout working.

You do not build all of this on day one. A scalable AWS system usually grows in stages, each one fixing the bottleneck you actually have. Watch one app grow from a single server.

• Stage 1: one EC2 instance and a managed database. Fine for your first users.
• Stage 2: a load balancer and an Auto Scaling group across two zones.
• Stage 3: CloudFront and S3 for files, ElastiCache for repeated reads.
• Stage 4: read replicas for the database and SQS for slow background work.