Why Keethu

Why Keethu Exists

Let's start with a question. Imagine a stock-trading floor. A new price for a popular stock just appeared. It needs to reach 50,000 traders' screens at once, ideally within a blink of an eye. If trader A sees the price one second before trader B, trader A wins the trade. So fairness — and money — depends on everyone getting the news at the same moment.

Now zoom out. The same problem shows up everywhere:

• A multiplayer game where 10,000 players must see each other move on the map.
• A weather app pushing tornado warnings to a million phones in the same county.
• A factory dashboard showing live readings from thousands of sensors.
• A sports app announcing a goal to every fan who's watching.

One central source has news. Many listeners need that news right now. This pattern has a name: publish / subscribe, or "pub/sub" for short. The source publishes. The listeners subscribe. A piece of software in the middle — like Keethu — moves the message from one to the other.

So why is this hard?

It sounds easy. "Just copy the message and send it to everyone." But computers are physical machines with real limits. As the number of subscribers grows from 10, to 100, to 100,000, the obvious approaches start to break in subtle ways. Imagine running a small kitchen: one cook can serve ten diners just fine. Now seat one hundred thousand diners — that same cook, even running at full speed, simply cannot serve everyone in time. You need to rethink the kitchen, the layout, the workflow, even the menu.

This guide is the story of those design decisions for software. Keethu's challenge: deliver a message to 100,000 listeners with a delay no longer than 10 milliseconds (1/100th of a second), on a single ordinary computer. Every choice we explain below exists to make that one sentence true.

The 100,000-Listener Problem

Why this is actually hard — the radio-station analogy

Picture a radio DJ with a single microphone. When she speaks, every listener with a tuned-in radio hears her instantly. The radio works because it broadcasts one signal to the air, and every radio in range pulls that same signal down. The DJ does not call each listener one by one.

Unfortunately, computers connected over the internet cannot broadcast like radio. Each listener is connected to the server by their own private "phone line" (a thing called a TCP connection, which we'll cover later). To send a message to 100,000 listeners, the server has to push the message down each of 100,000 lines, one by one.

Even if each push only takes a millionth of a second (1 microsecond), doing it for 100,000 listeners adds up to one tenth of a second — already ten times slower than our 10-millisecond budget. And we haven't even sent two messages yet.

So Keethu must be cleverer. It needs to push down many lines at the same time, share work across the chips inside the computer, and avoid wasting even a microsecond. To do that, we need to understand how the computer itself works at every layer: the chip, the operating system, the network card, and the programming language. The next six chapters walk through each layer in plain English, building up to how Keethu uses them.

The CPU — the Brain of the Computer Beginner

Everything a computer does — open a web page, play a video, send a chat message — comes down to one chip doing tiny tasks very, very fast. That chip is the CPU, short for "Central Processing Unit". Think of it as a kitchen chef who has been given a recipe and follows it step by step: chop onion, heat pan, add oil, add onion, stir.

A modern CPU follows billions of those tiny steps every second. Each step is called an instruction. A real program — say, sending a stock price to 100,000 listeners — is made of millions of such instructions strung together.

The assembly-line trick (called "pipelining")

If our chef cooked one dish all the way through before starting the next, customers would wait forever. Real kitchens overlap work: while the soup is simmering on the stove, the chef is already chopping vegetables for the next dish. Different stations are busy at the same time.

CPUs do the same thing. Each instruction passes through several stations: read the instruction, figure out what it means, do the work, write down the result. While instruction #1 is being figured out, instruction #2 is already being read; while #2 is being figured out, #3 is being read. The CPU has an assembly line, and at any moment many instructions are in flight at different stages. This is called the pipeline.

Guessing ahead (called "branch prediction")

A good waiter knows the regulars: when Bob walks in, the waiter is already pouring a black coffee before Bob has spoken a word. If Bob really did want coffee, time is saved. If Bob surprises everyone by ordering tea, the coffee gets thrown away — small loss, but rare.

CPUs do the same with if decisions in a program ("if the user is logged in, show their name, otherwise show 'Sign in'"). Instead of waiting to know the answer, the CPU guesses and starts cooking that path. It guesses right about 99 times out of 100. When it guesses wrong, it throws away the wasted work and starts over — like the coffee Bob refused.

Memory shelves (called "caches") Beginner

The CPU is incredibly fast, but the main memory of the computer (called RAM) is comparatively slow — like a librarian fetching a book from a deep storage room. To avoid waiting, the CPU keeps frequently-used scraps of data on increasingly closer "shelves":

• Registers — a tiny set of slots right inside the chef's hand. Holds a handful of numbers. Reading from them takes essentially no time.
• L1 cache — the chef's cutting board. About 1 nanosecond (1 billionth of a second) to grab.
• L2 cache — the nearby counter. About 4 ns.
• L3 cache — the kitchen pantry. About 15 ns.
• RAM — the storage room down the hall. About 70 ns — that's 70 times slower than L1.
• SSD (disk) — the warehouse across town. Tens of thousands of times slower than RAM.

These numbers look small (1 ns vs 70 ns) but multiply them by a million operations per second and the difference adds up to many seconds of wasted time. A program that fits its hot data in L1 finishes faster than a program that constantly runs to RAM, even if both do the "same" work.

Why arrays beat scattered lists

When the chef grabs a recipe card from a shelf, he doesn't pick up just that card — he picks up the whole binder it lives in, because the rest of the recipes are likely needed next. The CPU does the same thing: when it loads one tiny piece of data, it grabs a chunk of 64 neighbouring bytes (called a cache line) and puts the whole chunk on the closer shelf.

This means data stored next to each other is dramatically faster to walk through than data scattered all over RAM. A neatly stacked array is like a binder; a scattered linked list is like recipe cards thrown around the kitchen, each requiring a separate trip.

Two chips, one computer (called "NUMA") Intermediate

Big servers often have two CPU chips sharing one motherboard. It's like a restaurant with two kitchens — east and west — each with its own pantry. Cooks in the east kitchen can grab anything from the east pantry instantly. If they need something from the west pantry, a runner has to carry it across the restaurant. That round-trip wastes time.

The fancy name is NUMA ("Non-Uniform Memory Access"). All it means is: not every shelf is equally close to every chef. Reaching across to the other chip's memory costs about 50–75% more time than reaching your own.

Keethu sidesteps the whole problem by recommending a single-chip computer — one kitchen, one pantry, no runners. When that's not possible, it pins all of Keethu's work to one kitchen and tells the operating system to keep its data on that side too.

Doing eight things at once (called "SIMD") Advanced

Suppose you have to inspect 32 letters and find all the ones with a red stamp. The slow way: pick up each letter, check the stamp, put it down — 32 separate actions. The fast way: lay all 32 letters on a table at once, and use a single sweeping glance to spot every red stamp. One action, 32 results.

Modern CPUs have special instructions called SIMD ("Single Instruction, Multiple Data") that work exactly like that sweeping glance. A single instruction can act on 32 numbers (or 64, with the newest chips) simultaneously. This makes certain kinds of work — scanning text, comparing many values, encoding video — three to five times faster than the obvious one-at-a-time approach.

// Slow way: check one letter at a time
for byte in input { if byte == b'}' { /* found it */ } }

// Fast way (SIMD): check 32 letters at once with a single instruction
// 32× fewer trips through the loop for the same result

Keethu uses SIMD when parsing the small text labels that come with each message — the same technique used by the world's fastest JSON parsers.

The Operating System — the Building Manager Beginner

A computer is a shared building. Many programs (a web browser, a music player, a chat app, our Keethu server) all live in it at the same time. They all want a piece of the CPU's time. They all want some memory. They all want to talk to the network and the disk. Without a referee, they'd step all over each other.

The referee is the operating system, or "OS" for short. On a Mac it's macOS; on a phone it's Android or iOS; on most servers it's Linux. The OS is itself a piece of software, but a special one — it manages everything else. Think of it as a building manager who hands out apartments, schedules elevator usage, and is the only one allowed to call the outside world.

Programs, processes, and threads

Three words that get confused a lot — let's untangle them with the building metaphor.

• A program is the blueprint of an apartment design — a file sitting on disk.
• A process is a built apartment with a tenant living in it. The building manager (OS) gave it a unit number, walls, and its own private space. Two processes can't see into each other's apartments.
• A thread is a roommate inside an apartment. Many threads can live inside the same process, all sharing the same furniture and pantry. They can talk to each other easily, but they have to take turns at the sink.

So one process can run several threads at once — useful when a program wants to do multiple things in parallel. A web browser might have one thread loading a page, another playing a video, another listening for mouse clicks.

The "one roommate per visitor" trap

Here's the trap: each thread costs memory. On Linux, every thread reserves about 8 megabytes of memory for its own private "scratch space" (called a stack). One thread is fine. A hundred threads is fine. Ten thousand threads is 80 gigabytes of memory just for scratch space, before doing any work.

So the obvious design — "one thread for each connected listener" — implodes when 100,000 listeners connect. You'd need 800 GB of memory just to think about sending messages. The building can't even hold that many roommates. This problem has a famous name: the C10K problem (handling ten thousand clients on one machine), first described in 1999.

Asking the building manager for help (called "system calls") Beginner

Apartments are private, but their walls are paper-thin: tenants are not allowed to walk through them. If a program wants to touch the outside world — read a file, send a message over the network, ask for more memory — it has to ring the building manager (the OS) and ask. This phone call is called a system call, or syscall.

Each phone call is short, but not free. It takes about 100–300 nanoseconds for the program to be put on hold, the manager to do the task, and the program to be put back on the line. That sounds like nothing — until you remember Keethu does this a million times a second. At that rate, the phone calls alone can eat 10–30% of our entire time budget.

Waiting for the mail — four ways Intermediate

Servers spend most of their time waiting: for a network message, a database reply, a file. How you wait turns out to be the single biggest design choice for a server. There are four classic ways. Each one is more clever than the last.

Way 1: Stand at the mailbox and stare ("blocking I/O")

The simplest approach. A thread asks "is there mail?" and the answer is "not yet". The thread just stands there and stares. When the mail arrives, the thread reads it and moves on. Easy to write, but the thread is useless during the wait — and you need a thread per mailbox. With 100,000 mailboxes you're back to the C10K trap.

Way 2: Check, leave, come back ("non-blocking I/O")

Now the thread asks "is there mail?", gets "not yet", and immediately leaves to do other things. It comes back later to check again. Better — the thread isn't idle — but now you're making the trip to the mailbox over and over. If you have 10,000 mailboxes, that's 10,000 trips every cycle. Most of those trips will find nothing. Wasteful.

Way 3: Hand the manager a list ("select / poll")

You give the building manager a list of all your mailboxes and say "wake me up when any of them has mail". You sleep. The manager pokes you when something arrives. Much better! But every time you go to sleep, you have to hand the manager the entire 10,000-item list again. The handoff itself becomes the bottleneck.

Way 4: The manager remembers your list ("epoll") Intermediate

How memory really works (called "virtual memory") Intermediate

Each program thinks it has the whole computer's memory to itself. In reality, the OS hands out memory in small chunks (4 kilobytes at a time, called pages) and pretends, on each program's behalf, that those scattered chunks form one long ribbon. It's like a postal system where every tenant believes they have apartments 1, 2, 3, ... in order — but in reality their apartments are scattered across many buildings and the postal system keeps a secret map.

That secret map gets big. With 128 GB of memory, the map has 32 million entries. Looking up the right entry takes time. The CPU keeps a tiny cheat-sheet (called the TLB) of the most recent lookups. When the cheat-sheet hits, great. When it misses, the CPU has to walk through the full map — another delay.

One trick: ask for memory in much bigger chunks (2 megabytes at a time, called huge pages). Now the map has 512× fewer entries and the cheat-sheet almost never misses. Keethu uses huge pages for its large buffers — it's a one-line setting that quietly removes a class of latency spikes.

The Network — How Computers Talk Beginner

When two computers — say, your phone and Netflix's server — send each other information, they need a shared language and a set of rules. The most common set of rules is called TCP (it stands for "Transmission Control Protocol", though nobody but textbooks ever spells it out). Keethu uses TCP to talk to every listener.

Think of TCP as certified mail. When you mail something certified, you get a receipt; if the letter is lost, the post office knows to send it again; and the letters arrive in the order you sent them. TCP works the same way: it guarantees that what you send arrives, complete and in order, even if pieces get lost on the way. (The "plain mail" alternative — fast but with no guarantees — is called UDP, and Keethu doesn't use it.)

The hello handshake

Before any data flows, the two computers exchange three short messages to introduce themselves. It's like a phone call:

Once the three messages are done, the connection is "open" and data can flow freely in both directions. The handshake costs one round trip (about 0.1 milliseconds on a fast local network). Because Keethu listeners connect once and stay connected for hours, this one-time setup cost barely matters.

The mailman who waits to fill his sack (called "Nagle's algorithm")

By default, your computer's network code includes a "helpful" feature called Nagle's algorithm. The idea: if you keep handing the mailman tiny envelopes, he'll wait a moment to see if more envelopes are coming and stuff them all into one sack — fewer trips, less stamp cost. Great for downloading big files. Terrible for real-time messaging, where every envelope is urgent. Nagle can delay a single small message by up to 200 milliseconds (twenty times our entire budget!) just to "be efficient".

Keethu turns this feature off the moment a new listener connects. Every message must leave the door the instant it's ready. This single setting is called TCP_NODELAY.

// Keethu disables Nagle the moment a listener connects
stream.set_nodelay(true)?; // "send immediately, don't wait for more"

The mailbox at the door (called "socket buffers") Intermediate

Each open connection has two little mailboxes managed by the OS — one for incoming messages, one for outgoing. When Keethu wants to send something to a listener, it drops the message into the outgoing mailbox; the OS takes over and actually mails it. When a listener sends something to Keethu, the OS drops it into the incoming mailbox; Keethu picks it up when it's ready.

If a listener is slow to read its mail, the OS's outgoing mailbox to that listener fills up. At that point the OS politely tells Keethu "slow down, this person isn't picking up". This natural slowdown signal has a name: back-pressure. It's the system saying "stop pushing or I'll spill". Keethu has its own layer of back-pressure too, before data even reaches the OS mailbox — so one snail-paced listener never slows the others.

Sending one letter to many people without making 100,000 copies ("zero-copy") Advanced

Imagine you have a typed letter and want to send it to 100,000 people. Photocopying it 100,000 times wastes paper, ink, and time. A smarter approach: keep the original in a binder and give every recipient a pointer — "the letter you want is on page 47 of this binder". Now there's just one letter; 100,000 people are all looking at the same page.

That's exactly what Keethu does internally. When a publisher hands Keethu a message, Keethu allocates the message in memory once. Then, to send it to 100,000 listeners, it hands every listener a tiny pointer — not a copy — to that same memory. This is called zero-copy: the data itself is never duplicated. The memory is automatically freed only when the last listener is done with it (this is called "reference counting" — Chapter 5 explains how it works in Rust).

// One letter in memory…
let letter = Bytes::from("AAPL price update");

// …100,000 listeners, each gets a pointer, not a copy.
for listener in listeners {
    listener.send(letter.clone()); // clone() copies the pointer, not the letter
}
// The actual letter exists ONCE in memory. Freed when the last listener is done.

OS settings that matter (called "kernel tuning") Advanced

Linux ships with conservative defaults that suit a small office computer, not a server delivering a million messages per second. A handful of settings need to be raised before Keethu can do its job. You don't have to memorize these — they live in a config file we set once and forget. Here's what they do in plain English.

The numbers behind these settings exist in Keethu's deployment guide, but the principle is the simple list above.

Speed: the Two Kinds Intermediate

"Fast" can mean two different things

When people say software is "fast", they actually mean one of two things — and these two things often pull in opposite directions.

Latency is how long one thing takes from start to finish. The time from clicking "Send" to the message appearing on the other phone. Measured in milliseconds — thousandths of a second. Low latency means snappy.

Throughput is how many things the system can finish per second. The number of messages a server can push through per second. Measured in messages per second, or bytes per second. High throughput means high capacity.

A bus can carry 50 people across town in 30 minutes — high throughput (people per hour), bad latency (each rider waits 30 minutes). A motorcycle delivers one person in 8 minutes — great latency, but only one person at a time. Most engineering choices in a server are a bus-versus-motorcycle trade-off in disguise.

A small piece of math that always applies

There's one rule of thumb that's surprisingly useful. If X things arrive every second and each one takes Y seconds to process, then on average there will be X × Y things waiting around inside the system at any moment. So if 100,000 messages arrive per second and each takes 10 milliseconds (0.01 sec) to handle, then at any instant there are about 1,000 messages in flight. Sounds obvious, but it tells us how big our internal queues need to be. Make them smaller than that and the system jams up; make them way bigger and you waste memory. (Math people call this "Little's Law"; everyone else calls it common sense.)

Don't trust averages: the percentile story Intermediate

Suppose you measure 100 deliveries. 99 of them take 1 millisecond. One takes 10 full seconds — maybe a momentary network hiccup. What's the "average"? Roughly 100 milliseconds. But that single number is a lie. Most users had a delightful experience; one user thought the system was broken. The average hid that completely.

Real systems describe their speed using percentiles:

Keethu's promise is "p99 under 10 ms". That means: out of every 100 messages, no more than 1 may take longer than 10 ms. Why p99 instead of average? Because at 100,000 listeners, 1% is 1,000 people. If those 1,000 see a slow message every second, they will think the system is broken — and they'd be right to.

Latency spikes — the slow outliers that drag up your p99 — each have their own cause. Below, each card explains a common cause in plain words and how Keethu sidesteps it.

Sharing without taking turns ("lock-free") Advanced

When several workers need to read or change the same piece of data, the classic solution is a lock (also called a mutex). It works like a single bathroom key at an office: whoever has the key uses the bathroom; everyone else waits in line. It's safe, but the line can grow.

For Keethu, the "shared bathroom" is the master list of who's subscribed to what topic. When a new message arrives, the publishing thread needs to read this list. When a new listener connects, the connection thread needs to write to it. With many CPU cores all hitting this list a million times a second, the line for the key becomes the bottleneck.

The fix has two parts:

1. Many small bathrooms instead of one big one. Split the list into 128 separate sub-lists (Keethu uses a library called DashMap). Topic "sports" lives in sub-list 12, "finance" in sub-list 87. Workers updating different topics never bump into each other. Only a coincidence (two workers updating the same topic) ever requires waiting.
2. Special "atomic" CPU instructions for the simplest updates. For counters — like "how many messages have we sent so far" — the CPU has a single instruction that says "add 1 to this number safely, even if other cores are doing the same right now". It takes about 5 nanoseconds instead of the 1,000 ns a full lock would cost.

// Keethu's message counter — no lock, just an atomic increment
let seq = self.seq.fetch_add(1, Ordering::Relaxed); // about 5 ns

The accidental fight — false sharing Advanced

Remember from Chapter 1: when the CPU grabs data, it grabs a 64-byte chunk at a time, not just the one byte you wanted. That's usually a win. But it has a sneaky downside.

Suppose Keethu has two counters that have nothing to do with each other — say, "messages sent" (updated by core A) and "connections open" (updated by core B). If we innocently place those two counters next to each other in memory, they end up in the same 64-byte chunk. Now every time core A updates one counter, the CPU's bookkeeping forces core B to wait. They're stepping on each other for no good reason. This is called false sharing.

The fix is silly-simple: leave empty space between the counters so they land in different chunks. Like seating two strangers at separate tables instead of squashing them onto a bench together.

// Before: two unrelated counters squashed onto the same 64-byte bench
struct Bad { sent: AtomicU64, open: AtomicU64 }

// After: pad them out so each one sits at its own table
struct Good {
    sent: AtomicU64,
    _gap: [u8; 56], // 56 bytes of empty space
    open: AtomicU64,
}

How do you find what's slow? (tools called "profilers") Advanced

You can't fix what you can't measure. Profilers are tools that watch a running program and tell you where it's spending its time. Imagine a stopwatch attached to every step in the kitchen: at the end of the night, you can see "we spent 45% of our time waiting for the oven and 5% chopping vegetables — let's get a faster oven, not sharper knives".

You don't need to know all of these by name, but here's a flavour of what professionals reach for:

• Flamegraph — a colorful chart showing which functions ate the most time. Like a heat map of a busy kitchen.
• perf — Linux's built-in stopwatch. Tells you how often the CPU's cache shelves missed, how many instructions ran, and more.
• strace — lists every phone call (syscall) a program made and how long each took.
• HDR histogram — a special way to record millions of measurements (like "how long each delivery took") and read out exact percentiles like p99, p99.9 with very little memory.

Rust — Why We Picked This Programming Language Intermediate

Programmers write software in many different languages — Java, Python, Go, C++, JavaScript, Rust, and more. They all do the same things in the end, but each has its own personality. We chose Rust for Keethu because of one promise: no surprise pauses.

The "garbage collector" pause

Most popular languages — Java, Go, Python, JavaScript — quietly run a janitor thread in the background. This janitor periodically walks through the program's memory, finds chunks that are no longer in use, and reclaims them. The technical name is garbage collection (GC). It's convenient: the programmer never has to think about freeing memory.

The problem is that the janitor sometimes stops the whole program while it sweeps — typically for 10 to 200 milliseconds. During that pause, every listener of every program connected to that server is just frozen, waiting for the janitor to finish. A 50-millisecond pause means every Keethu listener experiences 50 ms of dead air — five times our entire latency budget. Unacceptable.

Rust's approach: clean as you go

Rust takes a radically different approach. Instead of a janitor, every piece of memory has an owner. The moment the owner goes out of scope (its turn at the kitchen ends), its memory is freed right then. No janitor. No pause. The compiler does all the bookkeeping at compile time, before the program ever runs, so the actual running program has none of this overhead.

This sounds restrictive — and at first it is — but in exchange you get:

• No GC pauses, ever.
• No "use-after-free" bugs (a notorious category of crashes where the program tries to use memory that was already cleaned up).
• Predictable, consistent speed.

For Keethu, where "everyone gets the news in 10 milliseconds" is the entire promise, Rust's no-pause guarantee was the deciding factor.

The waiter trick: async / await Intermediate

Remember Chapter 2's "smart waiter" — the one who takes ten orders at once and serves whichever comes out of the kitchen first? That's exactly what async / await lets a programmer write.

The programmer writes code that looks step-by-step:

async fn handle_listener(connection) {
    let message = connection.read().await;  // pause here if no message yet
    process(message);
    connection.write(reply).await;        // pause here if the network is busy
}

The .await spots are the magic. Wherever you see .await, it means: "I might need to wait here. If I do, please put me down and serve another table, then come back to me when you can." The compiler transforms the function into a state machine — a tiny note that takes a few hundred bytes of memory instead of the 8 megabytes a thread would need. A million such notes fit easily in memory; a million threads do not.

So async / await is just a way for the programmer to describe what should happen, and let the compiler figure out the smart-waiter logic underneath.

Tokio — the smart-waiter staff Advanced

Async / await describes what each waiter should do, but somebody has to actually employ the waiters and assign them tables. In Rust, that "somebody" is a library called Tokio. Keethu runs on top of Tokio.

Tokio hires one waiter per CPU core (so eight cores = eight waiters), each with their own list of tables to look after. When a waiter has nothing to do, instead of standing idle, they peek at a busy waiter's list and quietly take one of their tables. The fancy name for this is work-stealing. It means every CPU stays busy as long as there is any work in the building.

While the waiters work, a separate helper sits at the kitchen door (using epoll from Chapter 2) and waits for any plate to be ready. When something is ready — "table 7's food is up!" — the helper rings the bell and the nearest free waiter goes to deliver it. This is how Tokio handles tens of thousands of pending things on a handful of threads.

One letter, many readers — Rust's "Bytes" Advanced

We saw earlier that Keethu can send one message to 100,000 listeners without making 100,000 copies. The Rust tool that makes this easy is a type called Bytes. It's a piece of memory with a built-in counter: "how many people are currently holding a pointer to me?"

When Keethu wants to share the message with another listener, it just hands them a new pointer and bumps the counter from, say, 47 to 48. When a listener is done, the counter drops by one. When the counter reaches zero — nobody is holding it anymore — the memory is freed automatically. Simple, fast, no garbage collector needed.

// One message in memory
let payload = Bytes::from("new stock price");

// Hand it to 100,000 listeners — each clone just bumps the counter
for listener in listeners {
    listener.send(payload.clone()); // no copying of the message
}
// Total bytes copied: zero. Memory released when the last listener finishes.

The phone book with 128 sections — DashMap Advanced

Keethu needs to look up "who's subscribed to topic X?" instantly, many times a second, from many cores at once. A regular protected list (a "mutex-locked hash map") would force every core to queue up — back to the single-bathroom problem.

Instead, Keethu uses a library called DashMap. Think of it as a phone book that has been pre-cut into 128 separate sections — A's section, B's section, etc. When you want to look up "sports", you compute which section it lives in (let's say section 12) and only lock that section. Someone else looking up "finance" computes section 87, locks 87, doesn't even know section 12 is locked. They work in true parallel.

// Adding a listener to a topic: only that topic's section is locked.
self.topics
    .entry(topic)      // computes which section, locks just that one
    .or_default()
    .push(listener);  // other sections are completely unaffected

Putting It All Together — How Keethu Actually Works Pro

We've walked through chips, operating systems, networks, performance, and Rust. Now it's time to see how Keethu uses all of that, end to end. In this chapter we look at three pieces: the envelope format, the fan-out engine, and the safety valve.

The envelope format ("wire protocol")

When Keethu and a listener talk, they exchange messages over the network. They both need to agree on what a message looks like — where it starts, where it ends, what each part means. This agreement is called a wire protocol (literally: "what travels over the wire").

Most websites use HTTP, which uses text envelopes with lots of headers like "Content-Type: application/json". HTTP is human-readable, easy to debug, and works everywhere — but the envelopes are big (often a few thousand bytes) and reading them is slow because you have to scan for special characters.

Keethu uses its own much smaller envelope, designed for speed:

The fan-out engine — how one message reaches everyone Pro

"Fan-out" is just the picturesque name for "spread one thing to many places", like the blades of a fan spreading air outward. When a stock-price message arrives, Keethu fans it out to every listener subscribed to that ticker.

Inside Keethu, the fan-out happens in five quick steps. Each step uses something we've already learned in this guide:

1. Stamp the message with a number. A single atomic CPU instruction adds 1 to the global counter and gives this message its sequence number — about 5 nanoseconds. (See Chapter 4: lock-free.)
2. Look up who's listening. The DashMap's correct section is unlocked, the listener list comes out — about 20 nanoseconds. (See Chapter 5: DashMap.)
3. Walk the list, listener by listener. Because the list is a tightly packed array, the CPU's automatic cache prefetcher loads the next few listeners while we're working on the current one. (See Chapter 1: cache lines.)
4. Hand each listener a pointer, not a copy. The message body is shared via reference-counted Bytes — no copying. (See Chapter 5: Bytes.)
5. Drop the message in the listener's outbox. A tiny lock-free queue per listener — about 10 nanoseconds. If the queue is full, log a warning and skip; the listener is too slow.

Total cost per listener: about 35 nanoseconds. Times 100,000 listeners: about 3.5 milliseconds. Well within our 10-millisecond budget. The remaining 6.5 milliseconds is the slack for network jitter, OS scheduling, and TCP buffer flushes.

The safety valve — back-pressure Pro

Imagine a fast assembly line dropping boxes onto a slow conveyor belt. If we just keep dropping boxes, the slow belt will overflow — boxes pile up, then crash to the floor. The fix is a safety valve: whenever the belt is full, we stop dropping boxes (and maybe alert someone). This is what software engineers call back-pressure.

Keethu has back-pressure built in at every layer, so a single slow listener can never bring down the whole server.

The key idea: each listener has its own private outbox. If listener #347 is slow, only their outbox fills up — every other listener's outbox keeps draining at full speed. Keethu chooses to drop messages for the slow listener rather than let them slow everybody else down. It's the deliberate trade-off that keeps the system stable under stress.

Where the 10 ms goes — the latency budget Pro

Here's how Keethu spends the 10 ms it has from "message arrives at the server" to "message reaches every listener". Notice how nearly every line is in microseconds or nanoseconds — only the last few items even reach milliseconds.