Draft mode: how to trust an AI agent before it talks to your customers

The safest way to start with an AI agent is to let it do the writing and keep the sending for yourself. This is called draft mode. The agent reads each incoming email or message and prepares a reply, but the reply waits in a queue until you approve it. Customers never see a draft.

Every system we build at Yielde starts in draft mode. This article explains the method so you can run it yourself, whoever builds your agent. It describes how we work, not results from any business.

What is draft mode, and why not just switch the agent on?

An agent that sends on its own makes its mistakes in front of your customers. A wrong opening time, a promise you cannot keep, a reply with the wrong tone to someone who is upset: each one costs you a customer or a review. You cannot take a sent message back.

In draft mode the same mistakes land in your queue instead. You fix them in a minute, and you learn exactly where the agent is weak. After a few weeks you have evidence, not a feeling, about what it can be trusted with.

The idea is not ours alone. Anthropic’s guide to building agents says that agents can "pause for human feedback at checkpoints or when encountering blockers", and it recommends extensive testing in sandboxed environments, along with the appropriate guardrails. Draft mode is our version of that checkpoint, with you as the human.

Why does human oversight matter for a small business?

Bigger organisations write this down as policy. The US National Institute of Standards and Technology (NIST) has an AI Risk Management Framework, and under its GOVERN 3.2 subcategory one of the suggested actions is to establish policies and procedures on AI actor roles and responsibilities for human oversight of deployed systems. You do not need a policy document, but you do need the same answer to the same question: who reads what the agent writes, and who can stop it?

For an owner-run business the answer is usually you, or one person you name. If nobody is named, nobody is watching. That is the real risk, more than the technology.

What does a four-week draft mode plan look like?

The plan, the four weeks and the numbers in it are our own rules of thumb, not a standard and not research. Four weeks is long enough to see the ordinary run of enquiries and the odd awkward one, and short enough to keep your attention. Adjust it if you get few messages: the rule is to review enough drafts, not to hit a date. Aim for at least 30 to 50 drafts before you go live on anything.

1. Week 1: collect and readThe agent drafts a reply to every incoming message. You read every draft and send nothing the agent wrote without reading it. Do not score yet. Just notice what it gets right, what it gets wrong, and which kinds of message arrive most often.
2. Week 2: score every draftMark each draft as sent unchanged, edited, rewritten or wrong (the table below explains each). Keep the tally in a spreadsheet or on paper. Add a column for the kind of message: opening hours, quote request, booking, complaint, and so on.
3. Week 3: go live on the safest category onlyPick the one kind of message with the best scores and the lowest cost of a mistake, usually simple questions such as opening hours or directions. Let the agent send those on its own. Everything else stays in draft. Check a sample of what it sent every day.
4. Week 4: widen or pull backLook at the week 3 spot checks. If the live category held up, add the next safest one. If you found even one message that should not have gone out, move that category back to draft and find out why before trying again.

How do you score the drafts?

Use four labels. They are quick to apply and they separate a harmless wobble in wording from a real problem.

Do not rely on one overall average. A score of "mostly fine" can hide a category that is badly wrong. Count by kind of message, because that is the level at which you make the go-live decision.

Here is a worked example using made-up numbers. It is fictional: a three-van plumbing business in George reviews 40 drafts in week 2. We have invented the counts to show how to read the table, and they are not a result from any real business.

The reading is plain. Opening hours and areas served are the candidate for week 3. Quote requests need work on the instructions first. Booking has a wrong draft, so it stays in draft. Complaints are not a go-live candidate at all, which brings us to the next question.

What should never be fully automatic?

Some messages should always reach a person, however good the scores look. The reason is not that the agent will always fail. It is that the cost of one failure is too high, or the judgement is yours to make.

• Complaints and upset customers. Tone matters more than speed, and a bad reply makes it worse. The agent can draft; you send.
• Refunds and credits. Anything that moves money waits for your yes.
• Price exceptions and discounts. The agent should quote only from the price list you gave it, and never invent a special rate.
• Anything legal or medical. Advice about a contract, a dispute, a diagnosis or a treatment is not something for an agent to send alone.
• First contact with an existing dispute, or any message that mentions a lawyer, the press or a regulator.
• Bookings, in the early weeks. A wrong booking costs a callout or an empty chair. Let the agent line it up and you confirm it.

Customer messages hold personal information, so how an agent handles them also matters under South African privacy law. Our guide to POPIA and AI agents covers that, and do you have to tell customers they are talking to an AI covers the honesty question.

How do you keep a handover and a stop switch?

Going live is not the end of the control. Three things should exist before you let the agent send anything on its own.

1. A handover rule. Write down which words or situations pass a message straight to you: "complaint", "refund", "lawyer", "cancel", a customer asking for a person. The agent should say it is passing the message on and give a real time you will answer.
2. A stop switch. One action, by one named person, that puts the whole agent back into draft mode straight away. Test it before you need it, and know how long it takes.
3. A way to see what was sent. A log of every message the agent sent on its own, with the original message beside it, that you can read in two minutes at the end of the day.

The method works only if the customer can also ask for you. For comparison, our email response agent page says complaints, anything about money, refunds or quotes, legal letters and personal details always wait for a person.

How do you keep checking after go-live?

The first month is not proof of anything for ever. Your business changes: prices, opening hours, the services you offer, the season. An agent that was right in November can be wrong in January if nobody updated its instructions.

• Every day for the first two weeks live: read the log of what it sent on its own, or a sample of it.
• Every week after that: read a sample of 10 messages, and look at any that were handed over or that a customer replied to with a complaint.
• Whenever something changes: a new price list, a closed week, a new service. Update the agent first, then check a few drafts.
• When a customer complains about a message: add that kind of message back to draft until you know why.

Where does this fit with the rest of an AI project?

Draft mode is a step inside a larger plan, not a replacement for one. Before you start, decide on one result you can measure, for example the hours between an enquiry arriving and the first reply. Then run the four weeks and check whether it moved. If you are still deciding whether to start at all, read is AI worth it for a small business in South Africa, and for the basics of what an agent is, AI agent, chatbot or automation.

If email is where you want to start, our email response agent drafts replies for your approval, and the admin agent handles routine paperwork the same way. Custom work is priced per build and scoped on a call: book a call if you want to talk it through.