IM Systems in Depth

What happens betweenAna’s phone and Ben’s?

Design a messaging system step by step, from one server to a million people online, one decision per chapter. Server and client both, in English and Chinese.

Start reading

Being checked: the whole series (chapters 0–45 and four side trips) is online as a first draft. The author is checking it chapter by chapter, so text and numbers may change. Simulators and system maps after chapter 14 are still to come.

v1 in motion: the path a message takes. Four scenes loop; pick a scene to jump to it, or pause and step through.
Anaoutbox (local)◂ nowBenversion + seqsone server, one programConnectionholds connections; user → connectionsMessagededup by msg ID; next seq; insert; ACKDispatchpushes carry seq; latest seqs on returnBusiness (beside)is Ana in this conversation?Storageeach message oncesender+msg IDconv+seqconv.last_seqrecords+versionObject storage+ CDN, upload servicemessageACK + seqpush + seqsync, pullupload addressphoto uploaded in parts of 512 KiB (upload service → object storage)thumbnail and full photo to Ben via the CDNAnaoutbox (local)◂ nowBenversion + seqsConnectionconnections; user → connectionsMessagededup, next seq, insert, ACKDispatchpushes carry seq; latest seqs on returnStorageeach message once; sender+msg ID, conv+seqrecords + version; each conv’s last_seqObject storage + CDNupload service in frontmessageACKpushsyncleft: the photo, uploaded in parts of 512 KiBright: thumbnail and full photo via the CDN
delivered
0
resent
0
deduped
0
pulled
0

00:00.00 Ana writes it to her local DB: msg_id 7f3a, “sending” (this is the outbox)

A messaging system does two things: receive every message safely, then deliver it to every device, including one that comes back after being offline.

Not drawn: voice and video calls, end-to-end encryption, multi-region failover, retention and compliance, search.

App backendbusiness systemsOS pushAPNs, FCMthird partyAnaStill on Saturday?the app’s SDK: outbox, local copy, resendsBenStill on Saturday?✓a phone and the web app, each syncsinternetIM serverBusiness (beside)called, not passedaccounts, authfriends, blockschecked before ACKgroups, membersopen APImessages, callbacksConnectionOne open connection per online device, so we can pushcost: online devices; stateful, a restart reconnects everyoneMessageReceives: no loss, no repeats, in order; replies “sent” once storedcost: messagesDispatchDelivers to every device: push if online, notify if notcost: messages × recipients (group of 500: about 750 pushes)StorageKeeps messages and each user’s chat records, so devices can catch upcost: messages × retention (× recipients where inboxes are written)↑ receive: once stored, Ana gets “sent” and stops waiting↓ deliver: to every deviceaccountsif offlinewakemessage log1234
  1. Send: Ana’s message enters the IM over her connection
  2. Once stored, reply “sent” (not “delivered”): Ana waits for nothing after this
  3. Push to Ben’s online devices; offline ones get a notification, only a hint
  4. Any device that missed something pulls to catch up when it returns

Point at or tap a layer to see its job, why it is hard, and its chapters.

Every chapter goes in this order

  1. The problemA concrete scene, such as “the same message appeared twice”.
  2. Watch it failRun the simple design in a simulator on the page.
  3. EstimateA number you can check on paper explains why it fails.
  4. Fix itSwitch the fix on in the same simulator and compare.
  5. The costWhat the fix costs, and what other products choose.

Why this exists

There are many articles about messaging architecture. Most show a final diagram, or one company’s history. Few explain, step by step, why a system ends up the way it does.

Here, one chat app grows from 1,000 users to 1,000,000 people online. The four layers and the business layer beside them start crowded into one program, and each step of growth forces one of them out and deeper.

The last chapters change the product (toB workplace chat, community servers, live rooms, a private messenger, customer service, an IM platform for other businesses) and show the same decisions flipping. There is no single right architecture: the product decides.

Chapter map

The trunk follows one app from v0 to v3; each stop gives its scale and why it has to grow. Then a focus on reliability, and finally the branches change the product. The dot before each chapter is the layer it is mostly about, in the same colours as the diagrams.

connectionmessagedispatchstoragebusinessclientall layersBlue titles: published (first drafts for now)

  1. Opening

    See the whole system first.

    1. 0The whole picture
  2. v0: one server

    1,000 users

    “We want chat in our app.” Make it work first.

    1. 1The smallest chat
    2. 2From polling to push
  3. v1: correct on phones

    100,000 users

    On mobile networks, messages get lost, repeated and out of order.

    1. 3Acks and retries
    2. 4Duplicates
    3. 5Ordering
    4. 6Offline sync
    5. 7The local database
    6. 8Images and files
    7. 9v1: a reliable channel
  4. v2: many people, many devices

    100,000 users, several devices each

    Product adds a desktop app and several devices per person, and group management (join, leave, roles).

    1. 10Who builds the conversation list
    2. 11Syncing the conversation list
    3. 12Joining and leaving a group
    4. 13Syncing group info and members
    5. 14Read position and unread counts
    6. 15Read receipts
    7. 16Many devices
    8. 17Waking an offline phone
    9. 18Recall and edit
    10. 19v2: groups at 100,000 users
  5. v3: many servers

    1,000,000 online

    One server cannot hold them, and every deploy disconnects everyone.

    1. 20Split the gateway
    2. 21Presence
    3. 22A million connections
    4. 23Heartbeats
    5. 24Reconnect storms
    6. 25Accounts: the app’s users and the IM’s
    7. 26App lifecycle
    8. 27The web client
    9. 28Sharing out the conversations
    10. 29A queue before storage
    11. 30One SDK, many platforms
    12. 31Old apps, new servers
    13. 32v3: a million online
  6. Focus: reliability

    Once the system is big, the hard part is staying up: knowing messages arrive, and losing nothing through overload, deploys, failing dependencies and a lost data centre.

    1. 33Where did my message go?
    2. 34Overload: limits and degradation
    3. 35Deploys without disconnects
    4. 36When a dependency slows down
    5. 37Losing a data centre
    6. 38Drills and postmortems
  7. Branches: other products

    The same system for toB, communities, live rooms, customer service, a platform: which decisions flip?

    1. 39Workplace chat (toB)
    2. 40Community servers
    3. 41Live rooms
    4. 42Private messenger
    5. 43Customer service
    6. 44IM as a platform
    7. 45Same decisions, different products

Side trips: on the wireoptional

Optional reading on the protocol.

  1. W1Framing
  2. W2Encoding
  3. W3TCP or UDP
  4. W4Group encryption