RETURN TO SYSTEM DESIGN VAULT (55)BLUEPRINT #53 // Hard
#53HardMedia Streaming & Storage
Create a Distributed File Transfer System like Bittorrent
ASKED IN INTERVIEWS AT:GoogleAtlassianTwitch
EXECUTIVE SUMMARY & PROBLEM STATEMENT
Design a decentralized peer-to-peer file sharing protocol featuring torrent metadata parsing, distributed hash table (DHT) peer discovery, and rarest-first chunk exchange.
SCALE ESTIMATION & TRAFFIC ENVELOPE
DAILY ACTIVE USERS / TPSMillions of concurrent peers sharing multi-gigabyte files
READ / WRITE TRAFFIC RATIOP2P symmetric upload and download
5-YEAR STORAGE CAPACITYZero central storage needed; storage distributed across peer disks
FUNCTIONAL REQUIREMENTS
- [1]Publish file as torrent with SHA-1 piece hashes
- [2]Discover swarm peers via tracker and DHT
- [3]Download chunks from multiple peers and seed to others
NON-FUNCTIONAL REQUIREMENTS
- •Download speed scales proportionally with swarm popularity
- •Self-healing peer churn
- •Integrity verified per chunk
CORE ARCHITECTURAL COMPONENTS
Torrent Client Engine
Tracker Server (HTTP/UDP)
Kademlia DHT Peer Discovery
Piece Exchange Manager
Local Disk Cache
DATA SCHEMA & PERSISTENCE
TorrentMeta: info_hash, piece_length, pieces_sha1_array, announce_url
DISTRIBUTED TRADE-OFFS & FAILURE MODES
- ⟡Rarest-first piece selection prevents swarm stagnation where common chunks are plentiful but unique pieces vanish.