#53HardMedia Streaming & Storage

Create a Distributed File Transfer System like Bittorrent

ASKED IN INTERVIEWS AT:GoogleAtlassianTwitch

EXECUTIVE SUMMARY & PROBLEM STATEMENT

Design a decentralized peer-to-peer file sharing protocol featuring torrent metadata parsing, distributed hash table (DHT) peer discovery, and rarest-first chunk exchange.

SCALE ESTIMATION & TRAFFIC ENVELOPE

DAILY ACTIVE USERS / TPSMillions of concurrent peers sharing multi-gigabyte files
READ / WRITE TRAFFIC RATIOP2P symmetric upload and download
5-YEAR STORAGE CAPACITYZero central storage needed; storage distributed across peer disks

FUNCTIONAL REQUIREMENTS

  • [1]Publish file as torrent with SHA-1 piece hashes
  • [2]Discover swarm peers via tracker and DHT
  • [3]Download chunks from multiple peers and seed to others

NON-FUNCTIONAL REQUIREMENTS

  • •Download speed scales proportionally with swarm popularity
  • •Self-healing peer churn
  • •Integrity verified per chunk

CORE ARCHITECTURAL COMPONENTS

Torrent Client Engine
Tracker Server (HTTP/UDP)
Kademlia DHT Peer Discovery
Piece Exchange Manager
Local Disk Cache

DATA SCHEMA & PERSISTENCE

TorrentMeta: info_hash, piece_length, pieces_sha1_array, announce_url

DISTRIBUTED TRADE-OFFS & FAILURE MODES

  • ⟡Rarest-first piece selection prevents swarm stagnation where common chunks are plentiful but unique pieces vanish.