A customer writes: “I was charged twice for my last order.” A chatbot replies something like: “We're sorry for the inconvenience. To request a refund, email billing@yourstore.com with your order number.” Polite, fast and well written. And the case is exactly as open as it was before the customer wrote.
An agent replies something else: “I can see it: there are two identical €49 charges on the same day. I've refunded the duplicate and Stripe shows it as refunded.” Between the two replies there is no extra intelligence and no better writing. There are three things that happened in your systems before the sentence was written.
What a chatbot does
A chatbot is a system that answers. Its job starts at the question and ends at the reply. The old ones followed a decision tree; today's use a language model and, with luck, RAG over your documents, so they answer much better. But the job still ends in the same place: in the text.
For questions answered with information, that's enough. What time do you open? How long does shipping to Valencia take? Do you accept returns on sale items? There, a good chatbot is exactly what you need.
The problem is that most of what reaches support isn't questions. It's requests: refund this, change the address, cancel the order, upgrade my plan. And you don't answer a request with information; you answer it by doing it.
What an agent does
An agent is a system that works the case. It answers too, but the reply is the last thing it does, not the only thing. With the duplicate charge, the path looks like this:
- 1It understands the request. Not “the customer has a billing question”, but “they want a charge they believe is duplicated refunded”.
- 2It looks up your systems. It finds the customer's charges in Stripe and the order in Shopify: two €49 charges minutes apart, and a single order.
- 3It decides with your rules. Your policy lets charges under €100 be refunded without approval. €49 is inside, so it can do it itself.
- 4It acts. It refunds the duplicated charge, not the other one.
- 5It checks. It looks at Stripe again and finds the refund recorded. Until this step, it says nothing.
- 6It replies. And what it says describes what actually happened, not what should have happened.
A chatbot tells you how to fix it. An agent fixes it and tells you it's fixed.
The difference isn't the model
This is the most common misunderstanding. People think an agent is a chatbot with a bigger model, and it isn't: the same model can sit behind both. What turns a model that talks into an agent that works are pieces that live outside the model.
- Read and write accessReal connections to your systems, with permissions per action. Without write access, the most it can do is explain.
- Rules a sentence can't talk it out ofThe €100 cap isn't written in the model's instructions, where a persistent customer could argue it away. It lives in the code that runs the action.
- ApprovalsAnything that reaches the cap is neither rejected nor executed: it waits until someone on your team approves it.
- VerificationA call that didn't error doesn't mean the change happened. The agent looks again after acting.
- TracesEvery lookup, decision and change is recorded. If something goes wrong, you know what it did and why.
- Handoff to a personWhen it can't or shouldn't resolve something, it hands the case over with everything it found, so nobody has to ask again.
None of these pieces is intelligence. They are engineering, and they decide whether what the model does can be put in front of a customer.
The metric that gives them away
Chatbots are almost always measured by deflection: the share of conversations that never reached a person. It sounds good and hides a problem. A customer who gives up and leaves also counts as deflection. The one with the duplicate charge who got an email address, closed the chat and never wrote counts as a success.
An agent is measured by resolved cases: the change is made in the system and the customer knows it. It's a lower number, harder to inflate, and the only one that means something to the business. If someone shows you an 80% deflection rate, the question to ask is how many of those cases ended with something changed in a system.
Deflection measures who the case didn't reach. Resolution measures whether the case got closed.
What changes for your team
- With a chatbotEasy questions go down and requests stay the same, because they all end up with your team. Sometimes with a more annoyed customer, who already tried and couldn't.
- With an agentThe routine closes on its own, and your team gets what really needs judgement: what's over the cap, what's ambiguous, what the policy doesn't cover. And it arrives already investigated.
Four questions to tell which one you have
Both are sold with the same words, so it's worth testing them. These four questions separate one from the other in a ten-minute demo.
- Can it change anything?Ask it to cancel a test order. If the answer includes a link, an email or “an agent will contact you”, it can't.
- Where do its limits live?Try to convince it to refund more than your policy allows. If the rules only live in its instructions, sooner or later it gives in.
- Does it check what it did?Ask how it knows the change went through. “Because the API responded fine” isn't a check.
- What happens when it can't?Ask for something outside its permissions. The right outcome is a handoff to a person with the case explained, not an improvised way out.
A well-built chatbot is a good tool for what it is. Just don't mistake it for the other thing: the customer didn't write to get an answer, they wrote to get their money back.
