Joining and leaving a group

Chapters 10 and 11 put the conversations on the server: a message is stored once in its conversation, and whoever wants it fetches it. This chapter asks: who may fetch what?

The hiking group has 500 members and has reached message 4,900. Ben has been in it from the start; he left at 4,816 and came back a week later. Cara joined at 4,837. The rule is the product’s, and the most common one is: you can read the messages from while you were in the group; whether newcomers can scroll back a little further is a setting.

1. A member table that only says “in or out”

The obvious way: a row per member in the member table, and having a row means you are a member. A history read returns everything to members and nothing to anyone else. Many systems do exactly this, and it is fine when the product wants everyone to read all history; the trouble starts when it wants edges.

Cara can scroll back to message 1 the moment she joins; when Ben comes back, he sees the week he was away; and had he not come back, a new phone would get him none of the messages from while he was in. As for “newcomers see the 100 messages before they joined”, the table has nowhere to say it.

Here is the seq line; the two rows show what a history read returns for Ben and for Cara:

Members read everythingthe table says only “in or out”By min_seq / max_seqa row per stay

Ben rejoins a week later Newcomers see the 100 messages before they joined (a setting)

allowed, returnednot allowed, returnedallowed, not returnednot allowed, not returned

Ben sees 64 messages that should be hidden; Cara sees 4,836 messages that should be hidden.

• Members read everything: Cara sees 4,836 extra messages and Ben the 64 from while he was away; untick “Ben rejoins a week later”, and he cannot fetch any of the 4,815 from while he was in.
• By min_seq / max_seq: each gets exactly their own stretches. Tick “Newcomers see the 100 messages before they joined”, and Cara’s start moves back by 100.

The problem: the member table has one switch, “in or out”, and no “from which message to which”; and to record “from which”, a membership change needs a place on the seq line.

2. The fix: a join or a leave takes a seq too

Let membership changes take the message path. Take Ben leaving. As with a message, take the conversation’s next seq, and in the same transaction update the member table and write a system message:

BEGIN;
UPDATE conversations SET last_seq = last_seq + 1 WHERE id = :g RETURNING last_seq;  -- 4,816
UPDATE members SET max_seq = 4815 WHERE group_id = :g AND user_id = :ben;
INSERT INTO messages (conversation_id, seq, sender_id, text) VALUES (:g, 4816, 0, 'Ben left the group');  -- sender 0: system
COMMIT;

Taking the seq locks the conversation’s row (chapter 5), so membership changes and messages fall into one order: 4,815 is the last message before Ben left, 4,817 the first after.

The member table keeps two numbers per person: min_seq, the first message they can see, and max_seq, the last, unset while they are in the group. Ben’s max_seq is 4,815; Cara’s join took 4,837, so her min_seq is 4,837. That stretch is their membership rangemembership range成员区间The stretch of seqs a person can see in a group: min_seq is the first, max_seq the last (unset while they are still in it). Joins and leaves take a seq in the conversation like messages, so who gets each message and how far back each person can read are worked out from this range, whenever they are worked out. The group has a pair too: max_seq is the latest seq, and below min_seq nobody sees anything (chapter 12).See the glossary. There is a row per person per stay, and the rows of people who left are not deleted.

The group has a pair too: its max_seq is the conversation’s last_seq, the one the SQL above takes, and below its min_seq nobody sees anything, for example after the owner clears the history, or when only the last 90 days are kept. What a person can see is where the two ranges overlap.

3. What the range decides

• How far back you can read. History reads return only seqs inside your rows’ ranges. Whether newcomers see earlier messages is a product setting: to show Cara the 100 before she joined, set her min_seq to 4,737. When Ana clears her own history, her min_seq moves just past the latest message, and that holds on every one of her devices.
• Who gets a message. Message N goes to the members whose range holds N, and so does its push. The rows of those who left stay, so computing it later gives the same answer.
• Who may post. The send’s transaction already locks the conversation’s row to take a seq, so it also checks that the sender is still in the group. Ben’s leave and his message both lock that row first: if the leave comes first, his message is refused.
• What devices see. “Ben left the group” is message 4,816 and arrives in order with the others. So it must go to the whole group too: a seq that nobody receives looks like a gap, and devices would fetch it (chapter 5: a jump in seq means fetch what is missing). Ben himself learns from his conversation record (chapter 11).

Telegram’s supergroups work this way: messages are numbered per group, with the same ids for everyone; admins choose whether new members see the history before they joined (channels.togglePreHistoryHidden), and the group’s info gives each person an available_min_id: that message and every earlier one are hidden from them.

4. The numbers

• The member table: a row per person per stay: user ID 8 bytes, min_seq and max_seq 8 bytes together, role 1 byte, about 17 bytes; 500 members, about 8.5 KB.
• What a membership change costs: before, one row; now a message to the whole group: 500 conversation-record updates (chapter 11), plus a push to 500 people.

5. The cost

• A join or a leave goes from one row to one message. In big groups where people come and go a lot, this shows. Adding 30 people at once becomes one message, “Ana added Cara and 29 others”, one seq.
• The membership check moves inside the lock. Every send already checked it, but now does so while holding the conversation’s row; membership changes queue on that row with the messages.
• The rows of those who left stay. Once the to-dos for messages before they left have run, the rows can move to a history table, but not be deleted: fetching what they missed before leaving, or a new device loading their part of the history, still reads them.
• Rejoining adds a row. The old one is left alone; history reads count all the rows, and the stretch in between stays hidden. People who come and go collect rows.

6. Other answers

• Join and leave times. The member table keeps joined_at and left_at, keeps the rows of those who left, and compares with each message’s own time. Many systems do this, and it comes close to this chapter’s design. It slips at the edges: the order in which the times were taken is not the commit order, and server clocks drift, so a message or two around a join or a leave can land on the wrong side, and the send check cannot be ordered against the leave.
• An inbox per person (write fan-out). Telegram’s basic groups work this way: they and private chats share one message box per account (updates), and a message is written into your box only while you are in the group. The edges are exact with no ranges to keep; the cost is chapter 10’s write fan-out.
• Only “in or out”, and accept errors at the edges. For products that do not care about history boundaries, such as channels where everyone reads all history, this is enough.
• Matrix: a membership change is an event in the room’s timeline (m.room.member), and how much history new members see is set by the room’s m.room.history_visibility. The same idea as this chapter.
• Load the members of huge groups on demand. In a group of tens of thousands, the list itself is big; that is chapter 40.

7. This chapter’s decision

The server now knows who is in each group. But Ana’s phone needs a copy too: names, avatars and @mentions all use the member list, and she is in hundreds of groups, the big ones with hundreds of members, so fetching them all every time is out of the question. Next: syncing group info and members.