Discovery rubric auditor

Compare discovery calls using a consistent rubric, then identify recurring behaviours that warrant coaching.

An audit needs a comparable sample, one rubric, and a few recurring gaps. One recorded call is call-review-coach. A pass/fail readiness decision is certification-program-builder.

Pattern. "In 7 of 9 mid-market first calls, reps named a problem and never asked what it costs. Coach the cost question next week. Do not rank the AE with two calls."

One-off. "The team needs better discovery. AE3 scored 2.1. Roll out a discovery refresh."

If you cannot say how often the gap appeared, in which comparable slice, it is not a pattern.

Inputs

Ask only for the inputs that change the audit:

1. The sample and why these calls. One call: route to call-review-coach
2. Comparison units: role, segment, stage, call type, time window
3. Source coverage: recordings, transcripts, notes, or summaries
4. Rubric source, or use the reference
5. Reviewer count and whether they already calibrated
6. The decision this audit must support: coaching priority or baseline, not certification

Continue from a thin sample with the limits named. Do not invent a leaderboard.

Supporting material

Read references/discovery-rubric-framework.md for anchors, calibration, and when a number is only directional. Skip it if the user already supplied a calibrated rubric and only wants the pattern read.

Procedure

1. Frame the sample. Separate unlike roles, segments, stages, or call types. Done when each reported unit is comparable.
2. Calibrate, or label the single-reviewer path. Agree anchors on benchmark calls first. One reviewer: delayed blind rescore and inter-rater reliability: not established. Done when disagreements greater than one point are resolved or named.
3. Score with evidence. Every dimension gets a score or N/O, coverage, and confidence. Silence in missing audio is not a 1. Done when call-level evidence can be inspected.
4. Check drift. Double-score a subset, or blindly rescore after a delay. Pause aggregation if anchors still disagree.
5. Aggregate only comparable units. Count, median, distribution, N/O rate, coverage. Every number keeps its denominator. Fewer than three comparable calls per unit is directional.
6. Separate pattern from one-off. Recurring clusters only. Frequency is not causation. Skill gap, process gap, and sample-shape issue are different findings.
7. Pick a few interventions. Choose a response for each supported pattern: coaching, practice, process repair, or further investigation. For coaching, specify the behaviour to rehearse and the comparable future sample that could show improvement.

Conversation-intelligence scores are leads, not ground truth. Default reporting is de-identified.

For meeting architecture use discovery-and-demo-structure. For one call use call-review-coach. For a formal gate use certification-program-builder.

What to return

Return an audit a manager can act on next week:

• Scope: who, which calls, scoring window
• Sample shape: counts by unit, exclusions, coverage, skew
• Calibration record, including single-reviewer limits
• De-identified results with distributions; identifiable appendix only if authorized
• Patterns with frequency, impact, and confidence
• Prioritised interventions with rationale, uncertainty, owner, and follow-up evidence
• What to inspect in the next sample
• Limitations that keep conclusions directional

Do not rank reps from a mixed or tiny sample. Do not issue a pass/fail.

Quality bar

• Unlike calls are not on one leaderboard.
• Every finding cites call evidence and a denominator.
• N/O is never averaged as zero.
• One reviewer is labeled inter-rater reliability: not established.
• Interventions are few, and each is a pattern, not a one-off.
• The sample supports the scope of each conclusion; individual readiness remains a separate decision.