# Chapter 0: The whole picture

> What parts make up a messaging system, and how does one message get from Ana’s phone to every one of Ben’s devices?

IM Systems in Depth · https://im.liko.page/en/the-whole-picture/

Sending a message looks simple. Ana sends it to a server, and the server passes it to Ben. You can write a working
version in an afternoon, and on the office Wi-Fi it really does work.

The trouble starts outside the office:

- Ana presses Send in a lift and loses signal for a second. Did the message go out? Her app does not know.
- So the app sends it again, and Ben gets the same message twice.
- Ana and Ben write at almost the same moment, and each of them sees the two messages in a different order.
- Ben has not opened the web app on his computer all day. When he does, it must find out which messages it missed,
  without asking each of his hundreds of conversations one by one.
- In a group of 500, one message must reach about 750 devices; in a live room of 1,000,000 viewers, it is pushed a
  million times.
- A million long-lived connections send nothing most of the time, yet each one holds memory. Routers at the mobile
  carrier quietly forget connections that stay silent too long, and one gateway restarting for a deploy sends its
  50,000 phones back in the same second.

Meanwhile, people expect a lot from chat: nothing lost, nothing shown twice, everything in order, and all at once.
A web page that is half a second slow goes unnoticed; a lost message gets a “did you get my message?” right away.

Messaging is hard not because one of these problems is very hard, but because they arrive together and pull on
each other. Retries stop messages from getting lost, and retries create duplicates. Long-lived connections make
chat instant, and they bring a million connections and reconnect storms. Fewer heartbeats save the battery, and
then dead connections go unnoticed. Each fix brings the next problem.

This series, *IM Systems in Depth*, shows step by step how a messaging system grows from the afternoon version into
one that handles all of this. Before we start, let’s look at the whole system once.

## 1. What a messaging system must do

Most backends work by question and answer: the client asks, the server answers. You refresh, and a page comes
back. Messaging works the other way round. Ben does nothing, and the server must **bring** Ana’s message to his
phone. That ability comes from the long-lived connection, so strictly it belongs to the connection layer, not to
messaging itself: the same connection layer can carry live-stream comments, push notifications or device control
(chapter 20). Messaging adds a whole set of guarantees on top of it.

In one sentence: **every message Ana sends must appear on every one of Ben’s devices, exactly once, in the same
order everyone else sees**, even if one of those devices was offline all day.

Every part of that sentence has a cost:

- “Every message”: the network loses packets, so you need acknowledgments (ACKs) and retries.
- “Every one of Ben’s devices”: Ben has a phone and the web app on his computer, and they must agree.
- “Exactly once”: the network can at best guarantee at-least-once delivery, and retries create duplicates, so repeats are dropped by message ID and each message is shown once.
- “In the same order”: no two clocks agree, so timestamps cannot decide the order.
- “Offline all day”: when a device comes back, it must find out what it missed, and fetch it efficiently.

Replace “Ben” with “a group of 500”, or “a live room of 1,000,000”, and the same promise becomes a very different
engineering problem.

## 2. The running example

**The app**: an ordinary social chat app, with 1:1 chats and groups of up to 500, on phones, desktops and the web.
It has no name, so it can’t be mistaken for a real product.

**The people**: Ana and Ben chat with each other, and both are in “the hiking group” (500 members). Ben has a phone
and the web app on his computer.

**The numbers**: the whole series shares one set of assumptions, and every estimate comes from it. For example:
- 10% of users are online at the peak;
- each user sends 40 messages a day;
- each person has 1.5 devices on average;
- a message is about 200 bytes;
- 10% of packets are lost. That is chosen high so failures are easy to see; good networks lose far less.

So one message in the hiking group is stored once, in the group, and pushed to about 500 × 1.5 = 750
devices. By v3 the app has 10,000,000 daily users, with 1,000,000 online at the peak.

These are round numbers chosen for the example, not data from any real company.

## 3. The whole system, and one message’s journey

The diagram below is the system this series builds by the end of the main line (v3), drawn by layer:

- At the top are two devices: Ana, who sends, and Ben, who receives.
- Across the internet are the IM server’s four layers: **connection** (one long-lived connection per device),
  **message** (receiving), **dispatch** (delivering to every device) and **storage**, with shared infrastructure
  at the bottom.
- The column on the left is the **business layer**: accounts, friends and groups, moderation. It sits beside the
  others because messages do not flow through it; the layers ask it when they need to. It is also the part that
  changes most often, and keeping it beside the path keeps the other layers stable.
- The dashed boxes, the business systems (your app’s backend) and OS push (Apple, Google, phone makers), are not
  part of the IM system.

A dotted line runs between the message and dispatch layers. Above it is receiving: the sender waits for the ACK.
Below it is delivering, and from there on everything is asynchronous. The message log is the one hand-off between
the two. A common three-way split you may meet is “access, logic, storage”: the connection layer is the access
layer (often called the gateway), and the message and dispatch layers together are the logic layer.

Look at the whole picture first; point at or tap any layer or card to see what it does and why it is hard. Then
press “Follow this message”: Ana asks the hiking group “Still on Saturday?”, and the diagram lights its eight steps
one at a time. The four steps of the overview on the home page are steps 1–2, 3–4, 5–7 and 8 here.

*[Interactive figure: open the page to use it, https://im.liko.page/en/the-whole-picture/]*

A few words first, since every chapter uses them (all terms are in the [glossary](/en/glossary/)):

- The **message log** is the hand-off between receiving and delivering: a message counts as received once it is in the log. The **message store** keeps the history (one timeline per conversation). A conversation record is one row per person per conversation: pinned, muted, and later how far they have read.
- **Write fan-out**: when a message is sent, write one entry into each member’s inbox; reading means reading your own inbox. **Read fan-out**: keep one copy in the conversation’s timeline, and each reader fetches it from there. From v1 on a message is stored once (read fan-out). That is tied to who builds the conversation list: if a phone builds its conversation list from the messages it receives, it needs one stream of “every message sent to me”, which is an inbox; only if the server builds each person’s conversation list and devices just sync that list can a message be stored once, in its conversation (chapter 10). People with many conversations make every sync expensive; sending only the conversations that changed is chapter 11.
- **Two kinds of number**: the group’s seq (4,811, 4,812, …) sets the order in the group, is used for ordering and dedup, and shows which message is missing; conversation records have their own per-person **record version**, bumped on join, leave or pin, and from chapter 11 on by new messages too. Each device’s **sync cursor** is these two: the record version, and each conversation’s last seq.

This is only a walkthrough, to show where each thing happens. From chapter 1 on, each chapter takes one of these
mechanisms away, and you watch in the simulator what goes wrong.

## 4. Group chat and live rooms: where dispatch parts ways

In both a group and a live room, one message goes to many people, but the two take different paths, and the
difference is all in the delivering part. Note that a live room of 1,000,000 viewers is already another product: its
audience equals this app’s entire peak online count. The figure assumes a busy room sends 100 messages a second, and its viewers are
spread over 20 gateways, 50,000 connections each.

*[Interactive figure: open the page to use it, https://im.liko.page/en/the-whole-picture/]*

In a group, every member matters: every message must arrive and every device must be able to catch up, so dispatch
finds each member’s devices and pushes to each, and an offline one catches up by seq when it returns. In a live room, a million people watch at once, the
screen shows a dozen messages, and a late message is worth nothing. So dispatch does not look people up one by one: it sends one copy to each of those 20 gateways,
each gateway pushes it to the viewers connected to it, and losing some is fine. Chapter 41 builds this design step by step from the numbers.

Real products often keep the two apart as features too. Telegram’s [FAQ](https://telegram.org/faq), for example, says a group holds up to 200,000 members and suits sharing with friends and family or working in small teams, while a channel is a tool for broadcasting to large audiences, with no limit on subscribers. How its servers deliver a message inside is not public.

Live rooms are one example. The last chapters (39–45) change the product in other ways too, and the same decisions flip.

## 5. How the system grows

The diagram above is the end point. The series starts at the beginning: one chat app grows from 1,000 users to
1,000,000 people online. All four layers are there from the start, but at first they are crowded into one program.
Each step of growth forces one of them out on its own and gives it more depth. Switch versions below:

*[Interactive figure: open the page to use it, https://im.liko.page/en/the-whole-picture/]*

## 6. How to read a chapter

From chapter 1 on, each chapter makes one design decision, always in the same five steps listed on the home page:
**the problem**, **watch it fail**, **estimate**, **fix it**, and **the cost and other answers**. At the end of each
chapter the map gains one piece, with a decision card. The simulator is a sandbox. It talks to no real server, so
break it on purpose: raise the packet loss, switch a mechanism off, and see what happens.

## 7. Where to start

- To follow the story: chapter 1, “The smallest chat”.
- For one layer only: tap it in the picture above and see which chapters cover it, or open [the map](/en/map/).
- For every chapter at once: the [chapter map on the home page](/en/#chapters).
- For a word you don’t know: the [glossary](/en/glossary/). Words with a dotted underline show their meaning when you point at or tap them.
