Chapter 43
Customer service
An online shop puts “Contact us” on its website. At the evening peak, 600 new conversations arrive an hour; 20 agents are online, each handling 4 at once. Ana, not logged in, asks “Do these boots run large?” Who answers her? And what happens when there are not enough people?
This chapter is a first draft. The author is still checking it, and its text and numbers may change.
An online shop that sells outdoor gear has a “Contact us” button on its website and in its app.
At 8 pm, Ana is looking at a pair of hiking boots on the website. She is not logged in. She taps the button and asks: “Do these boots run large?” Right now 20 agents are online. At the peak, 600 new conversations like this arrive every hour.
This chapter’s question: who answers Ana, and what happens when there are not enough people?
1. How it differs from chat between friends
The trunk (chapters 1–32) is chat between friends and in groups. Customer service changes a few things:
- The two sides are not equal. Ana wants to talk to “the shop”, not to one person. Anyone can answer her, as long as they know what was said.
- Customers often have no account: visitors on a website, users of a mini program inside another app.
- One agent handles several customers at once, side by side in a web workbench.
- Work has to be given out. No agent goes looking for a stranger to talk to. Something has to hand Ana to one of them.
- A visit has a start and an end. When it ends, the agent’s place is free for the next customer.
- The numbers are part of the product: how long a customer waits for the first reply (first response time), and how many customers each agent handles an hour.
2. Building it from the trunk’s parts
First, what can we build with only the parts the trunk already has?
Design one: a 1:1 chat with an agent. Ana taps the button, the system picks an agent, say Ben, and opens a chat between the two. In the trunk, a 1:1 chat is defined by its two people: Ana and Ben.
The trouble starts when the agent changes. Ben cannot answer a question about returns and wants to hand Ana to Cara, who handles after-sales. In the trunk, that can only be another 1:1 chat, between Ana and Cara. Cara cannot see what Ana told Ben, so Ana has to say it again, and Ana now has two chats in her list. Next week Ana comes back and gets someone else again; the old messages are in the chat with Ben. The history follows the pair “customer + one agent”, not the customer.
Design two: a group for each customer, with every agent in it. This is like a shared mailbox. The history is all in one conversation, and any agent can see it, so design one’s problem is gone.
Now the trouble is ownership. Each of Ana’s messages goes to 20 agents. At the peak, 100 such groups are open (worked out in section 3), and every agent’s list shows all 100. Nothing says “this one is Cara’s”, so either two agents answer Ana at the same moment, or each thinks someone else will. And when there are not enough people, nothing tells Ana “there are 12 people ahead of you”.
The IM handles both designs easily. Assume 16 messages per conversation, both sides together: 600 conversations an hour is 9,600 messages, 2.7 messages a second, which is 0.019% of v3’s peak of 13,900 a second. A shop 100 times bigger is still only 1.9%. Customer service is not a load problem for the IM.
What is missing is a decision the trunk never makes: who answers. The trunk only sends messages to a conversation’s members. Customer service first has to choose the members.
3. First, how many people
Before choosing people, check whether there are enough. Chapter 36 used Little’s law: the number in the system at once = how many arrive per unit of time × how long each one stays.
- 600 an hour is 10 a minute.
- Assume one conversation holds one of an agent’s “places” for 10 minutes on average. (This is the whole time from accept to close: she handles 4 at once, so each one moves more slowly, and that is already included.)
- Conversations open at once: 10 × 10 = 100.
- 4 places per agent: at least 100 ÷ 4 = 25 agents online.
With only 20 agents there are only 80 places: they finish 80 conversations every 10 minutes, but 100 arrive. The queue grows by 20 every 10 minutes, 120 an hour. Places free up at 8 a minute on average, so someone who arrives after an hour has 120 people ahead and waits 120 ÷ 8 = 15 minutes, and it keeps getting longer. Over the evening, 20 of every 100 customers are more than these people can take.
Are 25 enough? Just enough, on average. But customers do not arrive neatly every 6 seconds, and some chats are long and some short. demo.py simulates two peak hours with random arrivals and random lengths, averaged over 200 runs. The load is the 100 conversations to be handled ÷ the number of places:
| Agents online | Places | Load | Waiting at 60 min | Wait in the second hour: median / 90% |
|---|---|---|---|---|
| 20 | 80 | 125% | 87 | 18.4 / 24.4 min |
| 25 | 100 | 100% | 11 | 1.7 / 3.2 min |
| 28 | 112 | 89% | 1 | 0.0 / 0.4 min |
| 30 | 120 | 83% | 0 | 0.0 / 0.1 min |
With 20 agents, 87 are waiting after one hour, a little fewer than the 120 on paper. The paper sum treats the places as full from the start, so the queue grows from the first minute; in the simulation all 80 places are empty when the peak begins, the 80 people who arrive in the first 8 minutes are picked up at once, and the queue starts a few minutes later.
With 25, at exactly 100% load, the average looks fine, but there is no slack, so once a queue forms nothing pulls it back to empty. In one run (random seed 1), 70 people are waiting at 60 minutes, and in the second hour half of the customers wait 5.3 minutes or more. The load needs room to spare: with 28, almost nobody waits.
Customers do not wait forever either. Assume they leave after waiting 10 minutes. With 20 agents, 19% of customers in the second hour leave before anyone answers, close to the 20% on paper.
Many shops let a bot answer first, for common questions, which cuts the part that needs a person. Assume the bot fully answers 30% of conversations (an assumption). Then 420 an hour need a person, 70 are open at once, and at least 18 agents are needed; with 20, the median wait in the second hour is 0.1 minutes.
How many people work is the shop’s decision; the IM cannot change it. What the IM can do is keep a clear record of the queue, the assignment and the times: who is waiting, for how long, and who should get them next. That is the assignment service below.
4. The fix: the conversation belongs to the customer, with an assignment service beside it
The conversation: one per customer per shop, kept for good. It is an ordinary trunk conversation: it has a seqseq序号(seq)The number of each message in a conversation, assigned in order by the message service that owns it. It sets the order, lets a client see which message is missing, and tells a returning phone which conversations it is behind on.See the glossary, sync and read positions, and nothing from chapters 5–18 changes. Its members are:
- Ana, always;
- whoever is serving her right now: the bot or one agent, a member only while serving her.
Who serves her is decided by a new assignment service. It lives in the business layerbusiness layer业务层The rules: accounts, friends, groups, permissions, moderation. It sits beside the message path: the layers call it when they need to, and messages do not flow through it.See the glossary, not on the message path. Messages still go through the message servers; the assignment service does only three things: it runs the queue, picks people, and keeps time.
Agents are put in and taken out with chapter 12’s join and leave. When Ben is assigned to Ana, Ben joins: this takes a seq, writes a system message “Ben is here to help you”, and gives Ben a membership rangemembership range成员区间The stretch of seqs a person can see in a group: min_seq is the first, max_seq the last (unset while they are still in it). Joins and leaves take a seq in the conversation like messages, so who gets each message and how far back each person can read are worked out from this range, whenever they are worked out. The group has a pair too: max_seq is the latest seq, and below min_seq nobody sees anything (chapter 12).See the glossary row in the member table. Only one thing differs from a group: Ben’s min_seq (the first message he may see) is 1, so he sees everything since Ana’s first visit. Chapter 12 said that how much history a new member sees is a product setting. In customer service the answer is always “all of it”, because the history is the customer’s.
A transfer is a leave plus a join. Ben hands Ana to Cara. In one transaction, Ben leaves (his max_seq, the last message he may see, is set), Cara joins (her min_seq is also 1), one seq is taken, and a message “Transferred to Cara” is written. Cara opens it and sees every word Ana and Ben said. On Ana’s screen it is still the same chat.
Who may message whom is the assignment itself. Chapter 12’s send transaction already locks the conversation’s row and checks that the sender is a member. After Ben leaves, his messages are refused. A customer can reach only the person the system put in the conversation; no separate permission system is needed.
One row per visit. The conversation goes on, but each visit is counted on its own: the seq it started at, the seq it ended at, its state, who is serving, and a few times (queued, assigned, accepted, first reply, closed). A row is about 65 bytes (a conversation ID and an agent ID at 8 bytes each, two seqs at 4 bytes each, five times at 8 bytes each, plus 1 byte of state); assuming 6,000 visits a day (10 times the peak hour), that is about 390 KB a day. Every number the shop manager looks at comes from these times. For example, first response time = first reply − queued.
5. The life of one visit
| State | How it starts | When time runs out |
|---|---|---|
| Bot | Ana sends her first message | The bot cannot answer, or Ana asks for a person → queued |
| Queued | The bot hands her over | Tell Ana her place in line now and then; if she closes the page, she leaves the queue |
| Assigned | The service picks Ben | Ben does not press “accept” within 30 s → next agent |
| With agent | Ben accepts | Ana says nothing for a long time → one reminder, then close |
| Transferred | Ben hands her to Cara | Timed like “assigned” |
| Closed | Either side ends it, or a timeout | Ana sends another message → a new visit starts |
A few notes:
- Every change of state is a system message with its own seq. Ana’s phone and the agent’s workbench receive it through the trunk’s normal sync; no separate notification path is needed.
- The timers belong to the assignment service. It must store every deadline and keep counting after a restart. Otherwise, after a deploy, nobody looks after Ana in the queue.
- Closing deletes nothing. Ben leaves; Ana stays. On her next visit, the new row starts at the next seq, and the agent scrolls up to see the last conversation.
6. Which agent
When Ana reaches the front of the queue, the service picks like this:
- Who is eligible: online, status set to “available” rather than “on a break”, with the skill for this kind of question (for example after-sales), and fewer than 4 open.
- The agent who served her last time, if eligible. He remembers her case, and she does not have to explain again.
- Otherwise the agent with the fewest open; on a tie, the one who has gone longest without a new conversation.
- If nobody with the skill is free, after a short wait (say 30 seconds) widen the search so other teams can take her.
If nobody is eligible, Ana stays in the queue, first come first served, and gets the next free place.
An agent’s workbench is that agent’s conversation list (chapters 10 and 11): the conversations they are a member of. When they leave one, it disappears from the list. The workbench runs in a browser, so chapter 27 applies as it is.
7. Customers without an account
Ana is not logged in. The first time the website opens the chat, it gives the browser a visitor ID and keeps it in the browser. The shop’s server signs a short-lived visitor token for it: the temporary identity chapter 25 gives to guests. Unlike a guest in a live room, this visitor ID has to last and has to map to an IM user ID, because she is a member of the conversation. When she comes back in the same browser, she is the same visitor, with the same conversation.
Later Ana logs in. Now the visitor’s conversation is handed to her account: the visitor leaves, the account joins with min_seq 1. All the old messages are there, and the agents can see she is a returning customer.
If she clears her browser data, or uses another computer, she is a new visitor, and the old messages cannot be linked until she logs in.
8. The cost
- One more service with state. The queue, the deadlines and who served whom must all be stored and survive restarts and deploys. If it stops, messages still flow, but new customers get nobody.
- Every assignment, transfer and close is a message. At 6,000 visits a day and 3 changes each, that is 18,000 system messages, beside 96,000 chat messages.
- One message is refused at the moment of a transfer. Ben finishes a sentence just after the transfer takes effect; it cannot be sent. The workbench should tell him “already transferred to Cara”, not just “failed to send”.
- The rules pull against each other. “The agent from last time” and “the least busy agent” point in opposite directions: an agent with many returning customers stays busier than the others. Give “last time” an expiry.
- A visitor’s history can break. In a new browser and not logged in, she is someone else.
9. Other answers
- The fewest open wins; if nobody is free, wait in the team inbox. Intercom’s balanced assignment: a teammate is eligible if they are a member of the team inbox, active, and below their limit of open conversations. The conversation goes to the one with the fewest open; on a tie, to the one who has gone longest without an assignment. If nobody is eligible, conversations wait in the team inbox; the order is configurable, and in the article’s example those marked as priority go first (Balanced assignment deep dive).
- By spare capacity, skills or in turn. Zendesk’s omnichannel routing can assign by spare capacity, in turn (round robin), or by skills. If an agent does not take the work within a set time, it goes to someone else; on the Professional plan that is 30 seconds. For skills routing it recommends a skills timeout, 30 seconds by default for messaging (Zendesk docs).
- Returning customers go to the agent from last time. LiveChat sends a returning customer to the agent who chatted with them before, if that agent is accepting chats. If the assigned agent does not respond for a while, the chat moves to another agent; on the Starter plan that time is fixed at 3 minutes (LiveChat docs).
- Customer service built on an IM. Tencent Cloud’s Desk is built on its own IM, Tencent Cloud Chat (overview). Its session settings: by default, assign to the agent who has gone longest without a conversation; set how many users one agent handles at once; send queue messages with the user’s place in line; close the conversation when the user does not reply in time; move the user to another agent when an agent does not accept in time; or set a callback URL so your own server does the assigning (session settings).
- Bot and people in the same conversation. Intercom says that when its bot, Fin, hands over, people work from the same inbox, the same customer record and the same conversation thread, with every earlier message attached (Intercom). This chapter’s “the bot is a member too” is the same idea.
- A bot that answers for one account. A Telegram business account can connect a bot that receives and sends messages in the account’s name; the account chooses which chats it covers (existing ones, new ones, contacts or not), and can pause the bot in a single chat to answer there itself (Connected business bots). It is the shape of section 2’s first design: the customer’s one-to-one chat with the owner’s one account, with a bot in front. It covers “the bot answers first”, not “which agent gets the customer”: the other end of the chat is always that one account.
- Merging a visitor on login. Intercom identifies a logged-out visitor with a cookie; once they start a conversation they become a lead. If they log in while the cookie is still there, the lead profile is merged into the user profile automatically, with the conversations of both (Merge lead and user profiles).
- One conversation per visit (a ticket). Each visit opens a new conversation, and the workbench stitches the history together by customer. Each conversation has a clear start and end; the cost is that the customer sees many chats.
- What “first response time” means. In Intercom’s reporting,
time_to_admin_replyruns from the customer’s first reply to a teammate’s first reply, and reports take the median (Build your own reports). Measured from queueing or from assignment, with or without the bot’s part: change the definition and the number changes.
10. This chapter’s decision
Customer service is one shop and many customers. What if many shops and many apps share one IM? Next chapter: IM as a platform.
Comments
Comments live in GitHub Discussions and need a GitHub account.