AI Can Tidy Your Interview Notes. It Can't Make the Call

Let AI transcribe your interviews, sort the answers and point out where the evidence is thin. Don't let it decide what the candidate actually did. That one step is the whole job.

Here is the kind of summary that lands in your inbox ten minutes after an interview now: "Led the build-out of the Japan sales team from zero. Strong cross-cultural communicator." Clean, confident, easy to forward. What it doesn't tell you is which half of that sentence the candidate said and which half the candidate did.

These tools are worth using. The question is which part of the work to keep.

Transcription: hand it over, then spot-check

AI transcribes more completely than any note-taker. The only thing you owe it is a second listen on the passages you'll base a decision on.

Transcripts can invent things. In a 2024 study presented at the ACM FAccT conference, Allison Koenecke and colleagues tested OpenAI's Whisper speech-to-text model and found that roughly 1% of transcriptions contained entire hallucinated phrases or sentences. About 38% of those hallucinations included explicit harms, such as made-up violence or false associations. Speakers with longer pauses were more likely to get words they never said.

One percent sounds small. But an interview is an hour, and if the invented sentence lands in the one story that matters, your judgment now rests on something the candidate never said. So relisten to any passage you are going to draw a conclusion from.

Sorting and flagging gaps: AI is more diligent than you

Mapping each answer back to its question and to the scorecard is tedious, and AI does it quickly and evenly. Humans remember the last ten minutes and whichever story was most fun to hear.

The more valuable use is having it flag missing evidence. Follow each story through the four-position interview: the conditions at the time, the trade-offs, the specific actions, and the result with its cost. Success stories usually skip the first and the last. A list of which stories have a happy ending and no conditions or costs is far more useful than a polished paragraph of praise.

It can also separate claims from facts. "I led a team of twenty" is a claim. "I took over a team of seven in April 2022, it was twenty by December, and I hired three of them myself" is starting to look like a fact. AI can highlight the first kind and tell you what to ask next round. It can't tell you what's true. It can tell you where there's no evidence yet.

AI can take the work up to flagging gaps. The judgment step stays with you

Who does what with interview records. Hallucination figure: Koenecke et al. (2024), tested on OpenAI Whisper

1. 1Transcribe: AI, with human spot-checksAbout 1% of transcripts contain invented sentences. Relisten to anything you'll decide on
2. 2Sort by question: AIMap answers to questions and scorecard items. Never rewrite the candidate's words
3. 3Flag gaps: AIWhich sentences are only claims; which of conditions, trade-offs, actions, result and cost is missing
4. 4Judge: a person onlySeparate what was said from what was done. Check vetoes, then the weakest layer, give one of five levels, sign your own name

AI finds where the evidence is missing. Whether the evidence is enough to decide is yours to call.

Why can't AI make the call?

Because the call is turning "he said" into "he did," and AI summaries are built to blur exactly that line.

Summaries round things off. "We opened up the channel pretty fast back then" becomes "Led channel expansion." The interviewer's job is to split "we" into "I" (I covered how in "It was all me": finding what a candidate really did), and the summary quietly runs that step in reverse.

Fluency pulls the same way. A good talker produces a long, tidy transcript with something in every scorecard box. Someone who speaks less and stumbles, but whose details hold together, looks worse on paper. Fluency is a presentation skill. It isn't proof of anything they've done. As I wrote in what I look for in interviews, the thing to collect is events, and adjectives are what AI produces best.

It goes deeper than wording. The method reads people in four layers: core, patterns, equipment and context. Transcripts are mostly equipment, the skills, résumé and projects. Core and patterns show up in how someone tells a failure or what they choose under pressure, often as a pause or a question they steer around. The transcript keeps that. The summary usually drops it.

My five-level fit check doesn't average. It checks vetoes first and then lets the weakest layer set the verdict. Models like to average. Add up item scores and a person with one collapsed layer can land on a respectable total.

Then there's bias, and the textbook case is old news by now. Reuters reported in October 2018 that Amazon had been building a tool since 2014 that rated résumés from one to five stars. By 2015 the company realized it wasn't rating candidates in a gender-neutral way. It had learned from ten years of résumés, most of them from men, so it penalized résumés containing the word "women's" and downgraded graduates of two all-women's colleges. The team was disbanded by early 2017. According to Reuters, recruiters looked at its recommendations but never relied solely on the rankings. If a company with those resources couldn't scrub the bias out, don't assume a general-purpose model summarizing your interviews is neutral.

Last, accountability. A hiring decision has to be explainable to the candidate, and someone has to stand behind it when asked. That someone is the interviewer. A prompt can't sign.

Before you record in Japan

If you interview in Japan, sort out personal data first. What follows is limited to what official sources state.

Under Japan's Act on the Protection of Personal Information, when you acquire personal information you must notify the person of the purpose of use or publish it (Article 21). The simplest way is to tell candidates before you record: that you're recording, what for, and how long you'll keep it.

In June 2023 the Personal Information Protection Commission issued a caution on generative AI. Businesses entering prompts that contain personal information should confirm it's within the stated purpose of use. If personal data goes into a generative AI service without prior consent and is then handled for purposes other than producing the response, that may violate the Act, so check that the provider won't use it for machine learning.

One more thing people miss. Guidelines from the Ministry of Health, Labour and Welfare under Article 5-5 of the Employment Security Act say employers should not, in principle, collect information such as race, ethnicity, family registry domicile or birthplace that could lead to discrimination, or beliefs and creed, or union membership. If a candidate volunteers any of that in small talk, the transcript keeps every word. Delete it and keep it out of the evaluation.

Every AI sentence traces back to the candidate

Every sentence the AI writes in the form "the candidate did X" has to trace back to the candidate's own words, with the conditions and result of that event attached. Anything that doesn't trace back is marked as a claim and goes on the list for the next round.

That way AI saves the time spent organizing, and the judgment stays with the person who signs.

---