Field note
The agent that could not say no
A support agent with no refusal path agreed to everything. The fix took an afternoon; the decision to write the refusal list should have been made on day one.
A support agent for a fintech product. It answered questions about statements, fees and card limits, and it could open a ticket for anything it could not answer. The evaluation set looked good. The pilot went to a slice of real customers.
In the first week a customer asked it to waive a fee. The agent had no tool to waive fees, so it could not do it. It also had no instruction about what to say when asked, so it said what a helpful model says: that it would arrange the waiver. It then opened a ticket titled “customer requests fee waiver, agreed”. The support team found forty of those by Friday.
Nothing had gone wrong technically. The agent did not touch money. It did exactly what its prompt asked, which was to be helpful. What was missing was a decision nobody had made: what does this agent refuse, and what does it say when it refuses.
The fix took an afternoon. We wrote the refusal list with the head of support: no commitments about money, no promises about timelines, no statements about other customers. Each item got a reason code and a sentence the customer would see, in the company’s own voice: “I can’t change fees, but I’ve raised it with the team and you’ll hear back within two working days.” The orchestrator was changed so that any request touching money routed to a refusal tool that returned the reason code, and the model was asked to explain, not decide.
Then we added the forty tickets to the evaluation set, with the expected outcome being a refusal with the right code. The pass rate dropped, which was the point. Two prompt changes later it was back, and now it stays there, because the set runs on every change.
What I would do again: write the refusal list before the prompt, with the person who owns the outcome, and treat a refusal as a product feature with copy, not as an error state. What I would not do again: assume that an agent without a tool for something will not promise to do it. A model without a “no” will say “yes” politely, and the ticket queue will tell you a week later.