Sampling rules and calibration

Route a share of AI-scored conversations to human review, and check that reviewers and the AI score alike.

For: QA leads

Steps

Go to MetriQA → Reviews → Sampling rules and add a rule: channels, score below, auto-failed, risky, AI confidence below, and the share of matches to queue.
After each conversation is scored, the first active rule it matches puts it in the To review queue.
Go to the Calibration tab and select New session. Name it and choose up to 20 AI-scored conversations.
Each reviewer scores every conversation. Other reviewers' scores stay hidden until you submit your own.
Compare the reviewers' average with the AI per conversation and criterion. A gap of 10 points or more is highlighted. Close the session when you are done.

Use calibration after changing a scorecard to check that the AI instructions are clear.
A low-confidence rule (for example below 60%) sends the conversations the AI is unsure about to people.

Still need help? Contact support