5 rules for a QA form that AI scores fairly

By Uspeech team ·

Your QA form was written for people. When a supervisor reads “handles objections well”, they know what you mean, because they were in the training sessions.

AI only has the words on the page. If the words are clear, it scores every call the way your best supervisor would. If they are vague, two similar calls can end up with different scores.

The good news: you don’t need to rewrite the whole form. Usually it’s enough to add a sentence or two to a few items. Here are the five rules that matter most.

1. Say what full points look like

“Handles objections well” means different things to different people. Spell it out, and say what earns partial points too.

Before: Handles objections well. (20 points)

After: The agent listens to the objection, asks a question to understand it, and offers a solution. 20 points: all three. 10 points: offers a solution without asking. 0 points: ignores the objection.

Simple items like “the agent says their name” don’t need this. Items worth a lot of points do.

2. Say when an item applies, in its own sentence

Conditions in brackets are easy to misread:

Before: Explain the cancellation terms and offer a discount (if the customer wants to cancel).

Does the condition cover both parts, or only the discount? Write it as a separate sentence instead:

After: Explain the cancellation terms and offer a discount. This item only applies when the customer wants to cancel.

3. Keep critical errors for real deal-breakers

A critical error gives the whole call 0%. Use it only for things that can never happen: skipping identity verification, missing a mandatory legal notice, promising something the agent isn’t allowed to promise.

If too many items are critical, half your calls will score zero, and the report stops telling you anything useful.

4. For every “never do this”, say what is still OK

Some forbidden actions look a lot like allowed ones. Asking for a password is forbidden; asking for an ID number is required. Both sound like “the agent asked for a number”. Say which is which:

Never ask for the customer’s password or PIN. Asking for the ID number or date of birth to verify identity is allowed.

5. Decide what happens when the customer hangs up first

Customers often say “great, thanks” and hang up while the agent is still saying goodbye. If your form has closing items (thanking the customer, offering more help), the agent can lose points for something they never had the chance to do.

Add one sentence to each closing item:

Only take points off if the agent said goodbye without doing this.

What you don’t need to write

Uspeech already handles the usual exceptions on its own, so you can skip notes like these:

  • “If the call drops, don’t penalize the steps it never reached.”
  • “Skip this item if the situation didn’t come up.”
  • “Don’t count something the customer says happened on an earlier call.”

Test it on calls you already know

Before using a new form on every call, try it on 20 or 30 calls your team has already scored by hand. Where the AI and your team disagree, the wording of that item is usually the reason. Fix the sentence and try again.

Include a few bad calls, too. If an item never scores zero in your test, you haven’t really tested it.

Quick checklist

  • Important items say what full and partial points look like.
  • Conditions are written as separate sentences.
  • Only real deal-breakers are critical.
  • Every “never do this” says what is still allowed.
  • Closing items say what happens when the customer hangs up first.