What happens betweenAna’s phone and Ben’s?
Design a messaging system step by step, from one server to a million people online, one decision per chapter. Server and client both, in English and Chinese.
Being checked: the whole series (chapters 0–45 and four side trips) is online as a first draft. The author is checking it chapter by chapter, so text and numbers may change. Simulators and system maps after chapter 14 are still to come.
- delivered
- 0
- resent
- 0
- deduped
- 0
- pulled
- 0
00:00.00 Ana writes it to her local DB: msg_id 7f3a, “sending” (this is the outbox)
A messaging system does two things: receive every message safely, then deliver it to every device, including one that comes back after being offline.
Not drawn: voice and video calls, end-to-end encryption, multi-region failover, retention and compliance, search.
- Send: Ana’s message enters the IM over her connection
- Once stored, reply “sent” (not “delivered”): Ana waits for nothing after this
- Push to Ben’s online devices; offline ones get a notification, only a hint
- Any device that missed something pulls to catch up when it returns
Point at or tap a layer to see its job, why it is hard, and its chapters.
Want every part and every hop? See the engineering diagram in chapter 0
Every chapter goes in this order
- The problemA concrete scene, such as “the same message appeared twice”.
- Watch it failRun the simple design in a simulator on the page.
- EstimateA number you can check on paper explains why it fails.
- Fix itSwitch the fix on in the same simulator and compare.
- The costWhat the fix costs, and what other products choose.
Why this exists
There are many articles about messaging architecture. Most show a final diagram, or one company’s history. Few explain, step by step, why a system ends up the way it does.
Here, one chat app grows from 1,000 users to 1,000,000 people online. The four layers and the business layer beside them start crowded into one program, and each step of growth forces one of them out and deeper.
The last chapters change the product (toB workplace chat, community servers, live rooms, a private messenger, customer service, an IM platform for other businesses) and show the same decisions flipping. There is no single right architecture: the product decides.
Chapter map
The trunk follows one app from v0 to v3; each stop gives its scale and why it has to grow. Then a focus on reliability, and finally the branches change the product. The dot before each chapter is the layer it is mostly about, in the same colours as the diagrams.
connectionmessagedispatchstoragebusinessclientall layersBlue titles: published (first drafts for now)
Opening
See the whole system first.
v0: one server
1,000 users
“We want chat in our app.” Make it work first.
v1: correct on phones
100,000 users
On mobile networks, messages get lost, repeated and out of order.
v2: many people, many devices
100,000 users, several devices each
Product adds a desktop app and several devices per person, and group management (join, leave, roles).
v3: many servers
1,000,000 online
One server cannot hold them, and every deploy disconnects everyone.
Focus: reliability
Once the system is big, the hard part is staying up: knowing messages arrive, and losing nothing through overload, deploys, failing dependencies and a lost data centre.
Branches: other products
The same system for toB, communities, live rooms, customer service, a platform: which decisions flip?
Side trips: on the wireoptional
Optional reading on the protocol.