Amazon Connect Talent: AI Interviews Match Expert Scoring 89% of the Time

AWS ML Blog · rss · 2026-11-20

AWS details the hiring science behind Amazon Connect Talent, which brings structured behavioral interviews — backed by decades of I-O psychology research — to AI-led interviews at scale via three design principles:

1. Job-related evidence: behaviorally anchored scorecards built from job requirements; the AI interviewer checks each answer for sufficient evidence and asks adaptive follow-ups when details are missing.

2. Interviewing separated from scoring: the interviewer only gathers evidence; a separate process scores competencies 1-5 via a fixed formula with transcript citations, flagging rather than scoring when evidence is insufficient.

3. Fairness safeguards: scoring uses transcribed text only — no video, voice tone, appearance, or identity info.

Independent evaluation results: 100% of job descriptions retained critical information, 91% of competencies traceable, 98% of questions targeted intended competencies; across 36 interviews and 252 rubric signals, agreement with expert scoring hit 89.3% (Cohen's kappa 0.79). Without follow-ups, first-response-only scores ran up to 1.14 points lower on a five-point scale, showing adaptive follow-ups materially capture strengths candidates leave unsaid.

Original post →

More from Apps

Apps channel →