RETURN TO SYSTEM DESIGN VAULT (55)BLUEPRINT #09 // Hard
#09HardCore Distributed Systems
Design an API Rate Limiter
ASKED IN INTERVIEWS AT:AmazonAtlassianUberPatreon
EXECUTIVE SUMMARY & PROBLEM STATEMENT
Design a distributed, highly available rate limiter capable of enforcing tiered per-user and per-IP rate limits with atomic counters across distributed nodes.
SCALE ESTIMATION & TRAFFIC ENVELOPE
DAILY ACTIVE USERS / TPS1 Million requests/sec across API gateway
READ / WRITE TRAFFIC RATIO1:1 read & increment on every request
5-YEAR STORAGE CAPACITYRedis RAM: ~20 bytes per user key * 100M active tokens = ~2 GB RAM
FUNCTIONAL REQUIREMENTS
- [1]Throttle requests exceeding threshold (e.g. 100 req/min)
- [2]Return HTTP 429 Too Many Requests with reset timestamp
NON-FUNCTIONAL REQUIREMENTS
- •Sub-millisecond overhead to API requests
- •Distributed synchronization across multiple edge proxy nodes
- •Resilience to Redis downtime (fail open/closed)
CORE ARCHITECTURAL COMPONENTS
Envoy / NGINX API Gateway
Redis Cluster Cache
Lua Atomic Pipeline Scripts
Local In-Memory Cache (L1) with Sync
DATA SCHEMA & PERSISTENCE
Redis Hash: `rl:{userId}:{window}` -> { count, last_refill }
DISTRIBUTED TRADE-OFFS & FAILURE MODES
- ⟡Sliding Window Counter (accurate, low memory) vs Token Bucket (burst-tolerant, memory-efficient).