Hiring, and the filters · 4.3

Interviews, and what they predict

Interviews, and what they predict. What is actually the case, and how it compares with what is repeated.

A practical software reference for this part of the discussion is Monitask's overview of activity tracking software.

The interview is the most used selection method and the least uniform. Two things called interviews can differ in predictive value by more than any other pair of methods in the literature.

Structured and unstructured

A structured interview asks every candidate the same questions in the same order, scores each answer against a scale defined in advance with anchored descriptions, and has the interviewers score independently before discussing anything.

An unstructured interview is a conversation. It goes where it goes, it covers different ground with each candidate, and it ends with an impression.

The first predicts job performance reasonably well. The second predicts poorly, and the people conducting it are typically confident in it, which is the combination that makes it dangerous rather than merely weak.

What the numbers did recently

For two decades the reference point was a 1998 meta-analysis summarising the predictive validity of selection methods. A reanalysis published in 2022 corrected several of the statistical adjustments used in that literature and produced substantially lower estimates for most methods, and a different ordering.

Structured interviews came out at or near the top. General cognitive ability, long treated as the strongest single predictor, came out considerably lower than the earlier figures implied.

This is worth dwelling on. A widely taught result was revised downward because somebody re-examined the corrections applied to the underlying studies, and the revision is now the better estimate. Nothing about hiring changed; what changed was the arithmetic.

Why the conversation feels better

Because it produces a vivid impression quickly and because the impression arrives with confidence attached. People are poor at knowing how good their own judgement is, and interviewing is a setting with almost no feedback: you never meet the candidates you rejected.

Mechanical combination of scores outperforms holistic judgement across a wide range of prediction tasks, a finding that has held since the nineteen-fifties and is resisted every time it is raised.

The panel discussion problem

Independent scoring before discussion is the part most often dropped, and dropping it removes most of the benefit. Once one person states a view, the others' scores move towards it, and a panel of five becomes one opinion with four endorsements.

Scoring on paper, submitting, and only then discussing costs two minutes and preserves the independence the panel was for.

What to ask

Questions about specific past behaviour, or realistic hypotheticals tied to the actual job, both scored against defined criteria.

Not brainteasers, which several large employers adopted and then abandoned after concluding internally that they predicted nothing. Not questions about weaknesses, which select for practised answers. And not anything about the candidate's life outside work, which selects for similarity to the interviewer.

Work samples

Where a short, realistic piece of the job can be set and scored, it is the strongest thing available and it has a second virtue: it shows the candidate what the work is, which improves their decision as well as yours.

The design constraint is that it must be short, paid if substantial, and genuinely representative rather than a puzzle.

What structure does for fairness

Reduces the room in which similarity bias operates, because the criteria are fixed and the comparison is between scores rather than between impressions.

It does not eliminate it: criteria can encode the same preferences. But a scored process leaves a record that can be audited afterwards, which an impression does not, and that auditability is a large part of the value.

How long it should take

Adding rounds adds cost and adds very little validity beyond the first well-structured interview and one work sample. Processes with five and six stages are common and the marginal stages are mostly reassurance.

They also lose candidates, disproportionately those with other offers, which selects against exactly the people the process was extended to be sure about.

Training the interviewers

Structure only works if the people using it were shown how, and the usual arrangement is that anybody senior enough is presumed able to interview.

A half-day covering question design, anchored scales and independent scoring changes outcomes measurably and is among the cheapest interventions available to any organisation that hires regularly.

What candidates should ask

What the stages are, who scores, and against what. An employer who cannot answer is running an unstructured process, which tells you something about the organisation beyond the hiring.

The candidate experience is a filter too

A slow, opaque or discourteous process loses the candidates with options first. That is a selection effect operating on the employer's behalf in the wrong direction, and it is invisible because the people it removes never explain why.

For broader context, consult CIPD selection guidance.