01 / The Model
Latency
How long you wait for an answer. It depends on the model, the length of the answer, how hard it thinks and how many steps an agent takes.
How you’ll hear it
“Customers are dropping out of the chat.”
“Latency. The answers take too long to start appearing.”
Avoid
Designing a customer-facing flow around a slow, heavy model.
The longer version
Latency is the delay between asking and getting a response. Longer answers, more reasoning and more agent steps all add to it. In a customer-facing service it shapes the experience as much as accuracy does, so it belongs in the design brief, not only the technical specification.