RETURN TO SYSTEM DESIGN VAULT (55)BLUEPRINT #13 // Hard
#13HardSearch, Autocomplete & Big Data
Identify the K Most Shared Articles in Various Time Windows
ASKED IN INTERVIEWS AT:LinkedInMetaTwitter
EXECUTIVE SUMMARY & PROBLEM STATEMENT
Design a real-time analytics streaming engine that computes the top K most viral/shared articles across sliding time windows (past 5 minutes, 1 hour, 24 hours).
SCALE ESTIMATION & TRAFFIC ENVELOPE
DAILY ACTIVE USERS / TPS500 Million share events per day = ~6,000 events/sec
READ / WRITE TRAFFIC RATIOHigh volume event ingestion with cached dashboard reads
5-YEAR STORAGE CAPACITYApproximate probabilistic counters fit in <500 MB RAM
FUNCTIONAL REQUIREMENTS
- [1]Track share events per article ID
- [2]Query top K articles across 5min, 1hr, 24hr sliding windows
NON-FUNCTIONAL REQUIREMENTS
- •Real-time window update latency < 5s
- •Memory bounded regardless of millions of unique articles
- •High availability
CORE ARCHITECTURAL COMPONENTS
Event Ingestion Gateway
Kafka Event Stream
Flink Sliding Window State Machine
Redis Top-K Cache
Dashboard API
DATA SCHEMA & PERSISTENCE
Count-Min Sketch array + Min-Heap storing `{ article_id, frequency }`
DISTRIBUTED TRADE-OFFS & FAILURE MODES
- ⟡Probabilistic data structures (Count-Min Sketch) trade 1% counting precision for 99% RAM reduction.