Functional Requirements–

• User should be able to search and find products based on product titles or names.

• User should be able to view the details of the product (Description, image, available quantity, review)

• User should be able to select the quantity and move the item to cart.

• User should be able to make the payment and do the checkout.

• User should be able to check the status of the order

• Manage purchase of items having limited stock.

Non-Functional Requirements—

• Scale: 10M MAU and 10 orders/sec

• CAP Theorem: Highly available with respect to searching and viewing the items, and highly consistent with respect to placing the order and payment.

Core Entities—

• User

• Product

• Cart

• Order

• Checkout

API–

1. // Search Product

1. GET: /products/search?q={seachTerm} → List <ProductId> list : Pagination

2. // Product Detail

1. GET:  /products/{productId}→ return the product details (json)

3. // Insert Data

1. POST: /cart {body: all product Id} → return cartId

4. // checkout and payment

1. POST: /checkout {body: all product Id with wty and price} → orderId

2. POST: /payment {body: orderId and payment details} → confirmation

5. // order Status

1. GET: /status/{orderId}

High-Level Design–

Problem Definition–

• Design a web crawler to extract text data from the web and store it to train an LLM.

• Your crawler can run for only 5 days.

Functional Requirements–

• Crawl the full web starting from seed urls

• Extract text data and store

Scale–

• 10B web pages

• 2MB per page avg.

• 5 days to scrape

• Unlimited resources (withing reason)

Non-Functional Requirements–

• Fault tolerant

• Politeness

• Scale to 10B pages

• Efficiently crawl in under 5 days

Core Entities—

• Text Data

• URL metadata

• Domain metadata

Interface–

input – set of seed urls

Output – text data

Data Flow–

1. Take seed urls from a frontier and the IP from DNS

2. Fetch HTML

3. Extract text from HTML

4. Store that text in database

5. Extract the urls in the text and add to our frontier

6. Repeat steps 1-5 until all urls have been crawled

High-Level Design–

Efficiency–

1. Don’t crawl a URL that has already been added.

2. Don’t parse a webpage with duplicate content that has already been parsed.

Functional Requirements–

1. Users should be able to upload videos.

2. Users should be able to watch/stream videos.

Scale 

~1M uploads/day

100M DAU

Max video size of 256GB

Non-Functional Requirements–

• Availability >> Consistency for video uploads

• Support uploading/streaming for large videos (256GB)

• Low latency streaming (<500 ms) is true in low bandwidth

• Scalability to scale to 1M uploads per day and 100M views

Core Entities—

• User

• Video

• Video Metadata

API

//upload a video

POST /videos

{

videoMetadata

}

//watch a video

GET /videos/:videoId → video & videoMetadata

High-Level Design–

Requirements–

1. Start group chat

2. send/receive messages

3. send/receive media

4. Access messages after I've been offline.

Non-Functional Requirements—

1. Delivered with low latency < 500ms

2. Guarantee delivery messages

3. Billions of users, high throughput

4. Messages not stored unnecessarily.

5. Fault-tolerant

Core Entities—

• Messages

• Users

• Chat

• Client

API-

Commands Sent Commands Received

• createChat - newMessage

• sendMessage - chatUpdate

• createAttachment

• modifyParticipants

High-Level Design–

The system allows users to create, edit, delete, like, retweet, and reply to tweets. Users can follow other users and see their tweets on a personalized Timeline (Home Feed). It also supports tweet search and user profile management.

• User should be able to create, edit, and delete tweets.

• Users should be able to like, retweet, and reply to tweets.

• Users should be able to search for tweets.

• Users should be able to view tweets from users they follow on their homepage.

• Users should be able to create an account and log in.

• Users should be able to follow other users.

• High Throughput: Capable of handling thousands of tweets, replies, and timeline requests per second

• Low Latency: Timeline and tweet loading should be fast (target < 200ms)

• Scalability: Horizontal scaling to support millions of users and growing traffic

• High Availability: 99.9%+ uptime (system should remain available even if some services fail)

• Fault Tolerance: Graceful handling of service or database failures

• Eventual Consistency: Acceptable for timeline feeds (latest tweets may have slight delay)

• Data Durability: No loss of tweets, replies, or user data

• Rate Limiting: Prevent abuse and control traffic from individual users

• Security: Strong authentication, authorization, SSL/TLS, and data privacy

• Performance: Efficient read-heavy (timeline) and write operations

Out of scope–

• Roll own blob storage.

Non-Functional Requirements–

1. Availability >> consistency.

2. Low latency upload and downloads. (as low as possible)

3. Support large files as 50gb

1. Reusable uploads

4. High data integrity. (sync accuracy)

Core Entities–

1. File (raw bytes)

2. File Metadata

3. Users

API–

• POST /files -> Presinged Url

body: File & FileMetadata

• PUT {presigned url}

Body file/chunk

• PATCH /files -> 200

• GET /files/:fileId -> File & FileMetadata

• GET /changes?since={timestamp} -> fileIds[]

High Level Design–

Functional Requirements–

1. Users should be able to input a start location and a destination and get an estimated fare.

2. Users should be able to request a ride based on an estimate.

3. Drivers should be accept/deny a request and navigate to pickup/drop-off

Out of scope—

• Multiple car types

• Rating for drivers and riders

• Schedule a ride in advance

Non-Functional Requirements–

1. Low latency matching < 1 min match or failure

2. Consistency of math. Ride to driver is 1:1

3. Highly available outside the matching.

4. Handle high throughput and surges for peak hours or special events. 100s of thousands of requests within a given region.

Out of scope—

• Resilience and handling system failures

• Monitoring logging, alerting, etc.

• CI/CD pipeline

Core Entities—

• Ride

• Driver

• Rider

• Location

API–

POST /ride/fare-estimate -> Partial<Ride>

{

Source,

destination

}

PATCH /ride/request -> 200

{

rideId

}

POST /location/update

{

lat,

long

}

PATCH /ride/driver/accept

{

rideId,

true/false

}

PATCH /ride/driver/update -> lat/long | null

{

rideId,

status: “pickup | droppedoff”

}

API-

GET /event/:eventId -> Event and venue and performer and ticket[]

GET /search?term={term}&location={lcoation}&type={type}&date={date} -> Partial<Event>[]

POST /booking/reserve

header: JWT | SessionToken

body{

ticketId

}

PUT /booking/confirm

header: JWT | SessionToken

body{

ticketId

paymentEtails(stripe)

}

High Level Design-

Non-Functional Requirements–

1. The system should be highly available. (It should be eventual consistency, and we can tolerate a ½-minute delay). (Availability).

2. Posting and viewing feed should be fast within < 500 ms. (Latency).

3. The system should be able to scale to 2B users. (Scaleability).

4. Users should be able to follow an unlimited number of people. (Scaleability).

Core Entities—

1. User.

2. Follow.

3. Post.

API:

1. POST/posts{"content":{""}} 201 {“postId”://}

2. PUT/users/{id}/followers{“userId”:”userId”} 200

3. GET/feed?pageSize={size}&cursor={timestamp}

Capacity Estimation–

Assuming 2 billion total users and 10% DAU (200 million):

• Posts: 3–5 per DAU/day → 600M–1B posts/day (~7–12K write QPS peak)

• Feed views: 150–250 sessions/DAU/day → 30–50B reads/day (~350–580K read QPS peak)

• Read:write ratio: 40–80:1

• Peak QPS: ~500–800K (mostly reads), with potential 3–5x spikes

• Latency goal: <500 ms p99 for posts and feed loads

• Storage (text posts, 1-year retention): ~150–400 TB (200–400 bytes/post + ×3 replication) High availability with eventual consistency (tolerate up to ~30s delay for new posts in feeds).

Cache Capacity Estimation–

Using push-on-write model with per-user timelines in Redis sorted sets (post IDs + timestamps):

• Stored per user: recent 500–800 posts (covers initial 5–10 scroll pages)

• Size/entry: ~40–60 bytes

• Per-user footprint: ~25–50 KB

• Total for 200M DAU: ~5–10 TB raw

• With ×3 replication, overhead, headroom: 15–30 TB RAM across Redis cluster

• Optimizations:

• Warm cache only for active users (last 30 days)

• LRU eviction + TTL for stale data

• Cache miss fallback: on-the-fly merge from Post DB + Follow indexes

• Separate pull model for celebrity/high-follower accounts to prevent fan-out overload.

High Level Design (HLD)—

Trade-offs & Bottlenecks

• Fan-out on Write (Push Model): Enables fast reads (<500 ms) via precomputed caches but risks write overload for celebrity users (e.g., 1M+ followers → massive queue backlog). Mitigated by hybrid approach (push for normal users, pull for high-follower accounts).

• Eventual Consistency: Tolerates ~30s delay, which simplifies scaling and availability but may cause brief stale feeds (acceptable per NFR).

• Hot Keys/Shards: Popular users or viral posts can overload specific DB shards or cache nodes. Solution: Sharding by user ID, consistent hashing, and separate handling for celebrities.

• Read-Heavy Workload: ~40–80:1 read:write ratio means heavy caching and horizontal scaling of Feed Service + Redis cluster.

• Storage Bottleneck: Text posts scale to hundreds of TB/year; media (images/videos) would require CDN + object storage (e.g., S3-like) and inflate costs significantly.

Future Improvements / Out of Scope

• Personalized Ranking: Add ML-based layer (e.g., engagement prediction) to reorder chronological feed → “For You” style.

• Real-time Updates: Integrate WebSocket / Pub-Sub (Kafka/Redis PubSub) for live push notifications instead of polling.

• Media Support: Store images/videos in CDN, reference URLs in Post entity.

• Search & Discovery: Add full-text search (Elasticsearch) for posts/hashtags.

• Security/Privacy: Rate limiting, abuse detection, private accounts, block/mute features.

Database Choices (Suggested)

• Posts: Cassandra / ScyllaDB (wide-column, high write throughput, time-series friendly) or MySQL sharded by post ID.

• Follows: Graph DB (Neo4j) or relational table with indexes on follower/followee.

• Feed Cache: Redis (sorted sets for timelines) clustered and sharded.

• Queue: Kafka or RabbitMQ for async fan-out to handle spikes.