RETURN TO SYSTEM DESIGN VAULT (55)BLUEPRINT #29 // Hard
#29HardSearch, Autocomplete & Big Data
Design an A/B Testing System (like Optimizely)
ASKED IN INTERVIEWS AT:AffirmFAANG
EXECUTIVE SUMMARY & PROBLEM STATEMENT
Design an enterprise experimentation platform supporting multi-layered A/B tests, deterministic bucket hashing, statistical significance calculations, and zero client latency degradation.
SCALE ESTIMATION & TRAFFIC ENVELOPE
DAILY ACTIVE USERS / TPS100 Million daily active users across thousands of active experiments
READ / WRITE TRAFFIC RATIOContinuous analytics write stream
5-YEAR STORAGE CAPACITYExperiment configurations distributed via CDN to clients
FUNCTIONAL REQUIREMENTS
- [1]Assign users consistently to test variants (A, B, or C)
- [2]Support targeting rules (country, device, user attributes)
- [3]Track metrics and compute conversion p-values
NON-FUNCTIONAL REQUIREMENTS
- •Zero network latency for user bucket evaluation (evaluate in client SDK)
- •Deterministic bucketing without storing user assignments in DB
- •Zero test interference
CORE ARCHITECTURAL COMPONENTS
Experiment Config Management UI
Global CDN Config Distributor
In-Memory Client SDK Evaluator
Clickstream Exposure Event Logger
Analysis Engine (Spark)
DATA SCHEMA & PERSISTENCE
Experiments: experiment_id, layer_id, traffic_allocation_percent, salt, variants: [{ name, weight }]
DISTRIBUTED TRADE-OFFS & FAILURE MODES
- ⟡Hashing `hash(userId + experimentSalt) % 100` guarantees deterministic assignment without requiring database lookups.