I Used AI to Rewrite 25 Customer Replies—The Results Were Uneven

The Task

I write a lot of customer replies.

Most of them are routine. Order confirmations, delivery updates, apology notes, clarification emails. I have templates for the common ones, but the templates get stale. They start to sound robotic. Customers notice.

So a few weeks ago, I decided to test something. I took 25 of my recent customer replies and asked an AI tool to rewrite them. Same message, same facts, but a fresher tone.

I wanted to see two things. First, would the AI drafts be good enough to use? Second, would the results be consistent?

The answer to both: uneven.

The Setup

Tool: One general-purpose AI assistant (free tier)

Date: October 16, 2026

Task: Rewrite 25 customer replies in a fresher, less templated tone

Input I provided: Each original reply, stripped of customer names and order numbers. Plus one instruction about tone.

My baseline: 45 minutes (the time it takes me to rewrite 25 replies by hand)

What I measured: How many AI drafts were usable, how many needed heavy edits, and how many were discarded

The Prompt

Here's what I used for each reply.

"Rewrite this customer reply. Keep all facts the same. Do not add information. Do not change the meaning. Make it sound less like a template and more like a person. Keep it under 100 words. Do not use the phrases 'just checking in,' 'circle back,' or 'I hope this email finds you well.' Here is the reply: [pasted reply]."

Same prompt. 25 replies. I worked through them in batches of five, over three days.

The Results

Here's how the 25 replies broke down.

Usable as-is or with one small edit: 9 replies (36%).

Usable after moderate editing: 10 replies (40%).

Unusable or discarded: 6 replies (24%).

So about three-quarters of the drafts were usable. About one-quarter were not. That's a wide range.

What the Good Ones Looked Like

The best drafts had a few things in common.

They were simple replies. Delivery confirmations. Thank-you notes. Simple acknowledgments. The AI handled these well because there wasn't much context to lose.

They were factual, not emotional. When a reply was just a status update, the AI did fine. It made the language cleaner and slightly less formal. Nothing was lost.

They didn't require relationship context. A short, factual reply doesn't need to know the customer's history. The AI could just rewrite the sentence.

Here's an example of a usable rewrite.

Original: "Thank you for your message. We have received your request and will process it within two business days. Please let us know if you have any questions."

AI rewrite: "Thanks for reaching out. We've got your request and will have it processed within two business days. Let us know if anything comes up."

That's better than the original. It's warmer. It's shorter. It keeps the facts.

What the Bad Ones Looked Like

The worst drafts failed for a different set of reasons.

They invented context. When the original reply referenced an ongoing conversation, the AI sometimes added lines that weren't in the source. "As we discussed last week" appeared twice in one draft. The customer and I had never discussed anything.

They softened complaints too much. When a reply was supposed to be firm—like explaining a late delivery or a missed deadline—the AI made it too apologetic. One draft started with "I completely understand your frustration" when the original had been direct and professional. That tone would have made the situation worse, not better.

They lost important details. When the original reply had two or three specific facts, the AI sometimes dropped one in favor of a smoother sentence. One draft removed a delivery window and replaced it with "soon." That's a factual loss.

They got the tone wrong for the relationship. For a customer I've worked with for years, I can be more casual. For a new customer, I need to be more formal. The AI can't tell the difference. It used the same casual tone for both.

Here's an example of a discarded rewrite.

Original: "We understand the delay has been frustrating. Your order is currently scheduled for delivery on Thursday, and we will confirm once it ships. If anything changes, we'll let you know immediately."

AI rewrite: "I completely understand how frustrating this must be, and I'm so sorry for the inconvenience. We'll get your order to you as soon as we can and keep you posted."

That rewrite lost the specific Thursday delivery window. It also over-apologized. Discarded.

The Numbers

Total time spent: about 50 minutes.

• Reading each draft: 8 minutes

• Light edits on the usable ones: 15 minutes

• Heavy edits on the moderate ones: 18 minutes

• Discarding and rewriting the unusable ones by hand: 9 minutes

My baseline was 45 minutes to rewrite all 25 by hand. The AI version was 50 minutes. So on pure time, the AI version was slightly slower.

But that's not the whole story. The AI version produced better language in about a third of the cases. It also freed me from the blank-page problem, which is worth something. On quality, the AI version was better. On time, it was a wash.

What Still Needed My Attention

Even with the usable drafts, I still did four things manually.

I read every draft against the original. I was looking for invented context, dropped facts, and tone mismatches. Every draft got a read.

I adjusted tone for the relationship. For long-time customers, I made the drafts warmer. For newer ones, I made them more formal.

I verified facts. Anything with a date, a number, or a name got a check against the original.

I discarded what didn't work. Six drafts. I rewrote those by hand. No shortcuts.

The Pattern

Here's what I learned about when AI rewrites work and when they don't.

Works for: Simple, factual replies. Short messages. One-purpose emails. Replies that don't carry emotional weight.

Doesn't work for: Complex replies. Messages with multiple facts. Anything where tone matters. Anything requiring relationship context.

The AI is good at language. It's bad at context. If the reply is mostly language, the AI helps. If the reply is mostly context, the AI gets in the way.

The Verdict

Kept, with a filter. I'll keep using AI to rewrite customer replies. But I'll only use it for the simple ones. For anything with multiple facts, emotional weight, or relationship context, I'll write by hand.

Baseline note: My baseline was 45 minutes. The AI version took 50. So on time, no savings. On quality, better language in about a third of the cases. The tool is worth keeping, but it's not a time-saver for this task. It's a quality tool.

The Limitation

25 replies is a small sample. A different mix of reply types might produce different results. My mix was heavy on simple confirmations and light on complaints. A batch with more complaints might produce worse output.

Also, I used a free tier. Paid versions might handle context better. I haven't tested that.

And I worked in English only. Replies in other languages might behave differently. I don't have data on that.

Test it in real life.