<Mehraj Hosen/>

The gap between where you are right now and what you want to wear isn’t about money, relationships, or success. It’s all attitude—1% attitude.

Every time you visit a webpage, Chrome quietly runs a check — is this URL malicious?
It doesn't call a server. It doesn't scan a massive database. It uses a Bloom filter. And it gets an answer in microseconds. That's the magic of this data structure. Small. Fast. Brilliant. So what exactly is a Bloom filter?

It's a probabilistic data structure — a fancy way of saying it trades a small chance of being wrong for massive gains in speed and memory. Here's how it works: you have a bit array (just a row of 0s and 1s) and a few hash functions. When you add an item, the hash functions each point to a position in the array and flip those bits to 1. When you query for an item, you check those same positions. All 1s? Probably in the set. Even one 0? Definitely not in the set.

That "definitely not" is the superpower. Why does that matter? Imagine you're Cassandra or HBase, and someone queries for a key that doesn't exist. Without a Bloom filter, you'd do a full disk read — expensive, slow. With one, you check in memory first. If it says "definitely not here," you skip the disk entirely. Gone. Saved.

Medium uses Bloom filters to avoid recommending articles you've already read. Akamai uses them to decide whether a URL is worth caching. Redis has them built in. One data structure, sitting quietly behind some of the biggest systems in the world.
The honest trade-off: Bloom filters can give false positives — they might say "probably yes" when the answer is actually no. But they never give false negatives. If it says no, trust it completely. And the memory savings? For 1 million items, a HashSet needs ~100MB. A Bloom filter? Around 1MB. Same job. 100x less memory. When should you actually use one?

When you need to check membership at scale, can tolerate rare false positives, and can't afford the cost of always hitting disk or a remote database. Spell checkers, fraud detection, network routers, CDN caches — the use cases are everywhere once you start looking.

It's not a replacement for a database. It's the bouncer at the door that turns away the obvious misses before they waste anyone's time. One bit array. A few hash functions. Millions of expensive lookups saved every second. Not bad for a 1970 invention.

#SystemDesign #BloomFilter #DataStructures #SoftwareEngineering #BackendDevelopment #DistributedSystems #DatabaseOptimization #WebPerformance #Programming #TechLeadership #ComputerScience #SoftwareArchitecture

"There are only two hard things in computer science: cache invalidation and naming things." — Phil Karlton

I used to laugh at this quote. Then I spent a week debugging why our users were seeing prices from 3 hours ago. I stopped laughing.

Here's what nobody tells you about caching: the moment you add one, you stop having one source of truth. You now have two—your database and your cache. And keeping them in sync? That's the real job.

Why is it so genuinely hard?

It's not a single problem. It's a cluster of problems wearing a trench coat.

Data changes in unpredictable ways. A user updates their profile. An inventory item sells out. A price gets revised. Your cache doesn't know any of this happened — it just happily keeps serving the old value, convinced it's being helpful.

Then there's timing. Even when you do invalidate the cache, there's a window — however small — where stale data leaks through. In distributed systems with dozens of nodes, that window multiplies.

And then the classic trap hits: you delete a cache entry right as another process is reading it, a third process is writing to the DB, and a fourth is about to repopulate the cache with the old value. Welcome to race conditions.

The strategies engineers actually use:

→ TTL (Time-to-Live): Set an expiry. Simple, predictable. But you're accepting "stale for up to X minutes" as a business decision, not just a tech one.

→ Write-through: Update cache and DB together on every write. Consistent, but adds latency to your writes.

→ Write-behind (write-back): Update cache first, sync to DB asynchronously. Fast writes are risky—if the cache dies before the sync, you lose data.

→ Event-driven invalidation: Publish a change event whenever data updates; subscribers invalidate the relevant cache keys. Elegant at scale, complex to implement correctly.

The lesson I keep coming back to:

Caching is never just a performance decision. It's a consistency decision. Every cache entry is a bet that the data won't change before the TTL expires — or that you'll catch it when it does.

The engineers who get burned by cache bugs aren't the ones who don't understand caching. They're the ones who underestimated how many moving parts stand between "data changed in the DB" and "cache reflects that change."

Phil Karlton said it in the 90s. Thirty years of distributed systems later, it's still true.

If you're building anything at scale, treat cache invalidation with the same seriousness you'd give schema migrations or API versioning. It deserves it.