System Design for Beginners

Tech concepts & system design components covered (beginner-focused)

System design mental model

• A scalable system can be thought of as two main parts:
  • Clients: mobile phones, laptops, IoT devices, etc.
  • Servers: always-on machines/VMs that handle requests.
• DNS (DNS resolution) translates human-friendly domains (e.g., amazon.com) into IP addresses.

Server capacity & scaling

• A single server can bottleneck and crash when traffic spikes.
• Vertical scaling (scale up): increase CPU/RAM of the same server.
  • Pros: keeps the same IP.
  • Cons: typically requires a restart, causing downtime—risky for “instant peak” events (e.g., ecommerce sale start).
• Horizontal scaling (scale out): add more server replicas instead of upgrading one server.
  • Can be done with no downtime by bringing new instances online while existing ones keep handling traffic.

Load balancing

• With horizontal scaling, DNS cannot directly return multiple server IPs.
• A load balancer sits in front to distribute traffic to replicas.
• Load balancer responsibilities:
  • Distribute incoming requests across multiple replicas
  • Use algorithms like round robin
  • Check instance health and route traffic only to healthy targets
• The summary mentions Elastic Load Balancer (ELB) conceptually (AWS terminology).
• It also notes load balancers can become bottlenecks/failure points—later addressed with CDN.

Microservices + routing

• Microservices architecture splits functionality into separate services (examples: auth, orders, payments, API).
• Requires request routing based on URL paths, such as:
  • /auth → auth service
  • /orders → order service
  • /payments → payments service
• A layered routing approach is described:
  • API Gateway as the central entry point
  • API gateway routes to service-specific load balancers
  • Service replicas often run on EC2 (VMs)

API Gateway

• A centralized reverse proxy that receives API calls and routes them to backend services.
• Can also host authentication/authorizers.
• Example behaviors mentioned:
  • Routing to different compute/storage targets depending on the endpoint (e.g., Lambda, S3).

Background work & batch processing

• Some tasks shouldn’t block the main request path, such as:
  • emailing many users
  • processing bulk uploads
  • long-running worker jobs

---

Asynchronous communication, queues, pub/sub, and workers

Why synchronous processing is bad

• If a request handler (e.g., payment service) waits for a slow downstream action (e.g., sending email), throughput suffers.

Queue system (pull-based worker model)

• Uses an SQS-like model:
  • Payment events are placed into a queue
  • Worker(s) poll the queue for messages (pull)
  • Each worker processes one message and then discards/acknowledges it
• Scaling advantage: add more worker instances for parallelism.
• Rate limiting advantage: workers can enforce limits to protect dependencies (e.g., “only N emails per second” for an external provider).

Polling vs push

• Pull: workers repeatedly ask the queue for messages
• Push: queue delivery triggers workers
• Mentions short polling vs long polling:
  • short polling = frequent checks
  • long polling = wait longer to reduce empty polls/cost

Pub/Sub model (SNS-like)

• Publish/subscribe via an SNS-like mechanism:
  • One payment event is broadcast so multiple services can react (email, WhatsApp/SMS, etc.)
• Contrast with queue semantics:
  • Queue: typically one-to-one consumption (each message handled once)
  • Pub/Sub: one-to-many delivery (multiple subscribers receive the event)
• Reliability/semantics note:
  • Pub/sub may not provide straightforward acknowledgement/retention like queues.
  • Queue-style failure handling may require concepts like dead-letter queues (DLQ) and retries.

Fan-out architecture (broadcast + reliable processing)

• Fan-out combines broadcasting with reliable queue processing:
  • Event triggers (SNS/events) lead to multiple SQS queues
    • e.g., WhatsApp queue, email queue, SMS queue
  • Dedicated workers consume each queue
• Benefits:
  • multiple downstream actions
  • queue-based retry/ack handling

Event-driven architecture summary

• Overall pattern: emit an event → services react via queues/pub-sub.
• Key reliability considerations:
  • acknowledgements
  • DLQs
  • retries

---

Rate limiting & protecting databases

Rate limiting

• Rate limiting helps mitigate overload and abuse (e.g., fake requests, DDoS).
• Example enforcement: allow “only X requests per second” and block excess.
• Common strategies:
  • Token bucket
  • Leaky bucket

Database scaling concepts

• Increased traffic volume can overwhelm the database.
• Mentions replication:
  • Read replicas for analytics/reads (can have replication lag)
  • Primary/master for real-time reads/writes
• Mentions caching:
  • Cache-first approach using an in-memory cache (e.g., Redis)
  • If cached data exists, serve it; otherwise fetch from DB and cache the result

---

CDN / CloudFront to reduce latency and load

Why add a CDN

• Notes load balancers can become problematic at scale.
• Adds CDN (Amazon CloudFront) in front of the origin/load balancing layer.

How CDN helps

• Users are routed (via anycast) to the nearest edge location.
• Edge cache serves popular content (e.g., photos/videos), reducing origin hits.
• Benefits:
  • lower latency
  • fewer origin hits
  • reduced server load
  • reduced bandwidth usage

---

Mentioned tutorial/resources (explicitly recommended)

• Rate limiting strategies:
  • Suggests searching for “API Rate Limiting Strategies”
  • Mentions an article by Piyush Garg that covers:
    • token bucket / leaky bucket workings
    • diagrams
    • code
    • real-world examples

---

Main speakers/sources

• Speaker: The YouTube narrator (referred to as Piyush Garg when referencing the rate-limiting article).