# Chapter 14: Read position and unread counts

> Ana read the hiking group on her laptop, yet her phone still shows a red 37 beside it. How should the server keep the number beside each conversation, so that it is exact and the same on every device?

IM Systems in Depth · https://im.liko.page/en/unread/

Ana read the new messages in the hiking group on her laptop and closed it. On her phone, the red dot beside the hiking group still says 37.

That number is the **unread count**: how many messages in the conversation she has not read. It has to be right, and what she read on the laptop has to clear on the phone too.

## 1. First, what does the product need?

Products answer “has she read it?” differently. Ordered by how much extra work the server takes on, there are four tiers:

| Tier | Cases | What the server does |
|---|---|---|
| Up to the client | Read when the chat opens, or only when a message is on screen; show a number, “99+”, or only a dot | Nothing extra: the phone reports “read up to message N” |
| One number is enough | All of a person’s devices agree; messages from before joining don’t count | One read position per person per conversation (section 4) |
| One number plus a little | An exact count, yet system messages and your own should not count (section 5); jumping to an @ while the messages in between stay unread; “mark as unread” (section 6) | Something kept beside the read position |
| Another product | For every message, who has read it and who has not (workplace chat) | Per-message state, another design (chapter 39) |

The trunk product is a social app with groups of up to 500. It chooses: per conversation, every device agrees; opening a chat lands on the first unread message; the count is exact; sending a message does not mean you have read everything above it.

## 2. Who counts: the phone or the server?

| | The phone counts | The phone counts, starting from the server’s rough number | The server counts |
|---|---|---|---|
| How | Count the unread messages in the local database that should count | On each resync, take the server’s rough number (latest seq minus read seq); while online, add one for each new message from others that counts | The server keeps and works out the count, and sends it with the conversation record |
| Good | The phone knows what each message is, so system messages and your own are easy to leave out | Right while online; a simple server | Every device agrees; the count exists while the app is closed |
| Hard | Every unread message has to be on the phone; each device counts on its own | Two ways of counting: the rough number includes system messages, so the count jumps at each resync | The server has to know which messages count (section 5) |
| Example | End-to-end encrypted apps (chapter 42): the server cannot see the content, cannot tell which messages count, and leaves it to the phone | [OpenIM’s SDK](https://github.com/openimsdk/openim-sdk-core/blob/061ac67/internal/conversation_msg/sync.go) | Telegram: each conversation carries an `unread_count` ([dialog](https://core.telegram.org/constructor/dialog)) |

The trunk counts on the server, for two reasons:

- **The phone does not have every message.** Chapter 6 settled that on return the phone pulls only the newest message of each conversation it is behind on. A week without opening the hiking group is 7,000 unread messages; to count them itself, the phone would first download them all, about 200 bytes each, 1.4 MB, just to show one number.
- **The count is needed while the app is closed.** The badge on the app icon can only come from the server, inside an OS push (chapter 17).

## 3. The first idea: a counter

Counting on the server usually starts with a counter: one unread number per person per conversation, plus one for each message, zero when read. It goes wrong in two places:

- **A redone to-do adds twice.** In chapter 11, a message is first written down as one “to-do” row, and a background job then works through it, updating each member’s conversation record. If the job crashes halfway, after the restart it does the whole to-do again, so the members it had already reached are updated a second time. Writing “latest seq = 4,812” twice still leaves 4,812, which is why that was fine. But “unread + 1” done twice turns 5 into 7: one message counted twice.
- **The reset arrives late.** Ana reads on the laptop and closes the lid; the “read” report waits until the laptop is back online. Meanwhile 3 new messages arrive in the hiking group and the counter goes up by 3; when the laptop reconnects, the reset finally lands and the counter goes to 0, though she has seen none of the 3.

A counter knows how many are left, not where she has read to. Once it is wrong, nothing can recompute it; it stays wrong until the next reset.

## 4. One number: the read position

On each of Ana’s conversation records (one row per person per conversation) the server adds a **read position** (read_seq, her read cursor): she has read everything up to this seq. Chapter 11 already keeps the latest seq on the same row, so:

**unread = latest seq − read position**

For now, assume the conversation holds only ordinary messages from other people. Ana’s own messages and system messages take a seq too; section 5 takes them out.

The rules:

1. **The phone reports what it has shown**, at most once a second for an open chat.
2. **It only moves up.** `UPDATE … SET read_seq = N WHERE … AND read_seq < N`. A late or repeated report changes nothing.
3. **When it moves, the record version moves.** The record version is chapter 11’s per-person number that only goes up: whenever one of Ana’s conversation records changes, it gets a new number taken from her row in the users table, and each of her devices pulls only the records above the number it has. So once she reads on the laptop, the phone gets it at its next sync, and the dot clears.
4. **Counting starts at the join.** A new member’s read position is set to the seq of the system message for the join. The messages chapter 12 lets newcomers see (the “last 100 before you joined” setting) all come before it: visible, but not unread.

One more thing to get right: **a read can run ahead of the record.** Ana replies in the group; her phone shows the reply at once and reports “read up to here”, but the to-do writes the latest seq into her record a little later. Say the reply is 4,823: the record still says latest 4,822, the read position is already 4,823, and unread = 4,822 − 4,823 = −1. The fix: the read’s transaction also writes `latest_seq = GREATEST(latest_seq, N)`.

Whether Ana’s own message should move her read position forward is also the next section’s question.

## 5. Second step: count only what counts

This is the price of counting on the server: the phone knows what each message is, the server only knows its seq. Every message has a seq, including two kinds that should not count:

- **System messages**: joins, leaves, a new group name; chapters 12 and 13 gave each a seq.
- **Ana’s own messages**: her reply is not something she has to read.

A shortcut: when you send a message, move your read position to it. [OpenIM’s server](https://github.com/openimsdk/open-im-server/blob/175a7bb/pkg/common/storage/controller/msg_transfer.go) (separate from the SDK in section 2) does this: when it stores a message, it sets the sender’s read seq to that message’s seq. That is fine when Ana reads and then replies in the chat. But if she replies straight from the notification, everything above it that she never saw is marked as read.

So each message carries two more numbers, written in the same transaction that takes its seq (chapter 5):

- **The count so far**: how many countable messages the conversation has, up to and including this one. A system message does not add one; it repeats the previous number. The count is shared by the whole conversation, so Ana’s own messages add one too, and are taken off when working out her unread count.
- **The sender’s running number**: this is the sender’s Nth message in this conversation.

| seq | 4,821 | 4,822 | 4,823 | 4,824 |
|---|---|---|---|---|
| What | message | Cara joined | Ana’s message | message |
| Count so far | 101 | 101 | 102 | 103 |
| Sender’s running number | – | – | 7 | – |

Ana has read up to 4,820; subtracting seqs says 4 unread. The right answer takes four numbers, all kept on her conversation record:

- **the latest count** and **her latest running number**: written by the to-do when it updates her record, only moving up. Here 103 and 7.
- **the count at her read position**: looked up from message 4,820 when she reads. Here 100.
- **her running number at her read position**: looked up from her last message at or before 4,820, with an index on (conversation, sender, seq), one query. Here 6.

> unread = (103 − 100) − (7 − 6) = **2**: messages 4,821 and 4,824.

The server works the count out from these four numbers whenever it sends the record. Section 4’s “read ahead of the record” applies here too: the read’s transaction also raises the latest count to the count at her read position if that is larger, and her latest running number likewise.

Here is one hour of the hiking group. Ana reads on her phone a few times, replies in the chat a few times, and twice replies from the notification; she reads twice on the laptop, and both reports arrive 4 minutes late:

*[Interactive figure: open the page to use it, https://im.liko.page/en/unread/]*

The dashed line is her true unread count, the solid line what the server shows. Minutes when a message or a report is still on its way don’t count; of the 62 minutes left:

- **A counter**: 46 disagree. Redone to-dos count too many; late resets too few.
- **Read position, the last report wins**: this is what it looks like without “only moves up”. The laptop’s old report arrives last and drags the position back; it ends at 7 when the truth is 0.
- **Read position, only up**: the devices agree, but system messages and Ana’s own are counted.
- **Count what counts**, with sending marking the chat as read: the two replies from the notification swallow 3 and 19 messages.
- **Minus her own**: right in every minute.

## 6. Where one number is not enough

**Jumping to an @.** Someone @mentions Ana at 4,900; she has read up to 4,800. She taps the @ button and jumps straight there; the 99 messages in between she has not seen, so the read position must not move to 4,900. An @ is kept as a row of its own (who, which conversation, which message), deleted once seen; the @ badge counts only the rows above the read position, so once the position passes them they no longer count. Telegram does the same: each conversation has an `unread_mentions_count`, the @ button steps through the mentions, and they are cleared once the last one has been read ([mentions](https://core.telegram.org/api/mentions)).

**“Mark as unread”.** The read position cannot go back: the only-up rule forbids it, and another device would report it forward again. Keep a flag instead and clear it on the next read. That read may bring no new messages, so write it as `SET read_seq = GREATEST(read_seq, N), marked_unread = false … WHERE read_seq < N OR marked_unread`: the flag clears and the position still does not go back. Telegram also keeps a separate `unread_mark`.

Ordinary messages are harder: the stretch you skip when jumping to an @ cannot be held in one number. Matrix, an open chat protocol, uses two positions to split a room’s history into three parts: read, possibly partly read, and not yet seen ([read markers](https://spec.matrix.org/latest/client-server-api/#fully-read-markers)). Anything finer means keeping state per message.

## 7. The numbers

- **The phone:** a conversation record grows from chapter 11’s 28 bytes to 39: read position 4, unread count 4, unread @ count 2, “marked unread” 1.
- **Reports:** a report covers every message before it, and a report that does not move the position writes nothing, so there are at most as many reports as messages received. At the v2 peak, 139 messages a second each reach 6.67 people on average (chapter 11): about 927 reports a second.
- **Rows written:** each report changes two rows, Ana’s users row (for the record version) and the conversation record: 1,853 rows a second. With chapter 11’s 1,853, that is 3,707 rows a second, 18.5% of what one storage node can write (20,000 rows a second).
- **Storage:** 8 more bytes per message; at 4 million messages a day, 32 MB.

The figure’s “Another hour” button picks a different hour. Over 200 of them, the counter disagrees in about half the minutes and in 108 of them it ends the hour on a wrong number; this chapter’s design is never wrong.

## 8. Cost

- **Reading is writing.** Each move of the read position bumps the record version, and the other devices sync once more.
- **One more lookup to send, two to read.** Sending takes the sender’s running number (in the same transaction that takes the seq, chapter 5); reading looks up two things.
- **@everyone is expensive.** In a group of 500 it is 500 rows; a flag on each person’s conversation record (“someone mentioned everyone”) saves them.
- **An old report can clear “mark as unread”.** A report of a read made long ago on the laptop arrives late and clears a mark just set.

## 9. Other answers

- **The phone counts, or counts starting from the server’s rough number** (section 2). For apps whose phones download every message, counting on the phone is simplest: no count so far, no running numbers.
- **A counter, reconciled now and then.** Simple; slightly off, corrected from “latest seq − read position” from time to time.
- **Sending marks the chat as read** (section 5). One running number fewer; replying from the notification swallows what was not seen.
- **A dot, not a number.** Nothing to count, subtracting seqs is enough; many apps do this for muted chats.
- **Per-message read state.** A row per person per message: a 500-member group with 1,000 messages a day makes 500,000 rows a day, against 500 read positions. Worth it when a workplace app must know, for every message, who has not read it (chapter 39).
- **Read ranges.** A few ranges per person per conversation, such as [1–40] and [45–50], which remember jumps too.

## 10. This chapter’s decision

*[Interactive figure: open the page to use it, https://im.liko.page/en/unread/]*

**Decision card**

- Problem: The unread count has to be exact and the same on every device. A counter goes wrong when a to-do is redone or a reset arrives late; latest seq minus the read position counts system messages and your own.
- Choice: The server counts: one read position per person per conversation, only moving up, synced to every device with the record version; each message carries the count so far and its sender’s running number, so the server takes off your own messages and sends the exact count with the record.
- Cost: Reading is writing (up to about 927 a second at peak); 8 bytes more per message, 32 MB a day; two lookups per read; @ and “mark as unread” kept separately.
- Revisit when: Whether the other side has read it (chapter 15); the badge of an offline phone (chapter 17); recalling an unread message (chapter 18); per-message read state (chapter 39).
- Other answers: The phone counts (end-to-end encryption); the phone counts starting from the server’s rough number (OpenIM SDK); a counter with reconciliation; sending marks as read; a dot only; per-message state; read ranges.

Ana’s own read position is now clear to the server and every device. But what about the messages she sends? Has Ben read them? How many in the group have? Next: read receipts.
