IM Systems in Depth

Chapter 31

Old apps, new servers

The server adds a new kind of message, a poll. Fifty people in the hiking group are still on an app from six months ago. What do they see?

This chapter is a first draft. The author is still checking it, and its text and numbers may change.

Chapter 30 turned the client into one core: fix it once, and every platform gets the fix. But “gets the fix” waits on people upgrading, and people do not all upgrade on the same day.

v3 has 10 million daily users. Suppose one in ten is on a version more than six months old: that is 1 million people. Of the hiking group’s 500, about 50.

This chapter’s question: when the server adds something new, what happens to old apps?

1. A poll message

The server adds a new kind of message: a poll. Cara posts one in the hiking group, “Which mountain on Saturday?”, and it is the newest message in the group.

Side trip W2 already handles half of this: an old app skips fields it does not recognise (a rule of Protobuf’s binary format). But this time it is not an extra field, it is a new type: the old app’s code has no branch for “poll”.

Look at those 50 old apps. Each of the following ways of handling it is common:

  • Drop what you don’t recognise. The parser skips an unknown type and does not store it. But that message holds a seq. Say the poll is number 4,812: the server says the conversation’s latest is 4,812, while the last one on the phone is 4,811. Chapter 6’s sync compares exactly these two numbers; the phone’s is smaller, so it thinks it is behind and fetches what comes after 4,811. What comes back is the poll again, and it is dropped again. This happens on every open, 20 times a day. The unread count is the same: chapter 14 computes “latest seq − read position”, Ana can read at most up to 4,811, and 4,812 − 4,811 is 1, forever.
  • Fail on what you don’t recognise. The parser is too strict: Protobuf’s JSON mapping, for example, rejects unknown fields by default (side trip W2), or the switch on the type has no default. Every sync throws, and the app crashes as soon as it opens, until it is upgraded.
  • Show it as text. A poll has no text field, so the bubble is blank.

One kind is worse: the meaning changes. Chapter 18’s “edit” and “recall” are both new events that point at the original message. An old app that does not recognise them keeps showing the original: an edited message shows no sign of the edit, and a recalled one stays on screen.

2. Compatibility rules belong in the first version

All three problems above can only be fixed inside the old app. But the old app is already installed on 50 people’s phones and can no longer be changed.

So the first lesson: compatibility with the future has to be written into past versions. From v0, every app should keep these rules:

  1. Skip unknown fields (side trip W2).
  2. Store unknown types anyway. Put them in the local databaselocal database本地数据库The phone’s own copy of messages, conversations and the sync cursor, usually SQLite. The screen reads only it, and everything from the network is written into it first; a cursor is committed in the same transaction as the messages it stands for.See the glossary, holding their seq, keep the original bytes, and show the fallback text they carry. With the seqs unbroken, chapters 5 and 6’s gap-filling and sync do not go wrong.
  3. Show them again after an upgrade. Once a new version recognises the type, it parses the original bytes stored on the phone again, with no need to pull from the server (except for messages the server converted for it, section 4).
  4. Recognise “please upgrade”. When the server refuses a login, the old app must be able to show that message and the download link. That error code has to be defined in the first version’s protocol.

3. Every message of a new type carries fallback text

With rule 2, the old app still has to know what to show. So every message of a new type carries a line of text for old apps, such as “[Poll] Which mountain on Saturday? Upgrade to view”.

Matrix, an open chat protocol, makes this a rule: every message must have a plain-text body, and a client that cannot display the message’s type shows this text. Edits too: an edit event’s body is written as “* the changed words”, for clients that do not understand edits. Telegram instead has a placeholder type of its own, messageMediaUnsupported: “Current version of the client does not support this media type.”.

Fallback text only fixes what is shown. After an edit, the old app shows “* the changed words”, but the original message above is still there; a recall is the same, the original stays on the old app’s screen. Fallback text cannot change the original.

4. The server steps in only where fallback is not enough

Where fallback text falls short, only the server can help. For that, the server has to know which version is on the other end.

So when the app connects, its first frame reports its version and capabilities: app version, platform, and the numbers of the features it understands (such as “understands polls”, “understands edits”). About a dozen bytes, once per connection.

Going by that, the server, for this connection:

  • sends only frames the app understands; if the app did not report “understands read receipts”, for example, chapter 15’s receipts are not sent to it;
  • where a new type has an older close relative, converts it into something the old app understands: a new animated sticker, say, becomes an ordinary image, which beats “Upgrade to view”. A converted message is marked “converted”; after the app upgrades, these are pulled from the server again, because what is stored on the phone is not the original.

Telegram does this with layers: each change to the protocol is a new layer, the client reports its layer with invokeWithLayer when it connects, and the server keeps it together with the connection’s other parameters.

5. Anything older is told to upgrade

Some changes old apps can never cope with, a security fix for example. So the server sets a minimum supported version: anything below it is refused at login, with rule 4’s “please upgrade” from section 2.

Each time the minimum goes up, a group of people is locked out. Where to set it depends on how many people are still below it, not on the calendar.

6. The numbers

  • People on old apps: 10 million × 10% = 1 million; 50 in the hiking group.
  • The cost of dropping: each old app pulls the same poll 20 times a day, and its unread count stays at 1.
  • Versions to support: suppose each native platform ships every two weeks and each version is supported for a year: 26 versions × 3 platforms = 78 builds, any of which may connect. The web is the latest version on every refresh; the only old ones are tabs left open without a refresh (chapter 27).
  • Fallback text: about 30 bytes, added only to messages of new types, 15% of a 200-byte message.

7. The cost

  • The protocol carries old things for years. Field numbers are never reused (side trip W2); the server still has to be able to produce old types.
  • Every new type needs fallback text, in several languages.
  • The server has code paths per version. With 78 builds, every server release has to be tested against them (chapter 35’s staged rollout).
  • The minimum version locks people out. How many it locks out is the price of each cut.

8. Other answers

  • Forced upgrades only. Raise the minimum version whenever there is a new type. The simplest; the cost falls on users. More common in workplace chat, where administrators can upgrade everyone at once (chapter 39).
  • The server converts for each version (Telegram’s layers). Old apps need no changes at all; the server carries every conversion.
  • Fallback text only (Matrix’s body). The server converts nothing, and old clients at least see a line of text.
  • IM as a platform (chapter 44). The SDK is embedded in customers’ apps, the platform cannot make anyone upgrade, and how long versions are supported is written into the contract.

9. This chapter’s decision

That completes v3’s twelve chapters. The next chapter walks through v3 from end to end once more: with a million people online, how many machines does each layer need, and what happens when a gateway dies?

Comments

Comments live in GitHub Discussions and need a GitHub account.