An interview scorecard that starts better conversations

Before writing interview questions, agree the work the person needs to do. A criterion such as “can explain a product decision using evidence” gives an interviewer something to explore. A label such as “great personality” leaves too much to personal interpretation.

Structured interviews use planned questions and a consistent approach to evaluating answers. The U.S. Office of Personnel Management's overview describes using predetermined questions and shared rating standards. The example below is our own practical worksheet, rather than a validated assessment instrument.

Choose a manageable set of job-related criteria. Divide them between interviews so that different conversations add information. A portfolio discussion might explore decision making and execution; a separate team conversation might explore collaboration through examples of past work. Explain the purpose to the candidate before you begin.

For a product designer, the following worksheet connects each criterion to an interview prompt and a place for evidence. Change the criteria to match the actual role. Don't add a requirement simply because it was on the last team's scorecard.

For each row, record the candidate's example, your observation and any remaining question. Separate what you heard from what you inferred. “Explained two options and the tradeoff between them” is evidence. “Would probably be good in a startup” needs much more explanation.

If you use ratings, define what each rating means before the interviews. Here is one example:

• Needs more evidence: the conversation didn't establish the capability required for the role.
• Partly demonstrated: there is relevant evidence, with a specific gap to explore.
• Demonstrated: the example supports the level of work the role requires.
• Strongly demonstrated: several concrete examples support the capability, including handling meaningful constraints.
• Not observed: the interview didn't cover this criterion well enough to rate it.

“Not observed” is different from a negative rating. If an interviewer ran out of time, the team has a gap in its information. Record that plainly. Don't turn missing evidence into a statement about the candidate.

These anchors are a discussion aid. Tailor them to the role and give interviewers examples of what useful evidence looks like. Avoid treating a numerical average as an automatic hiring or rejection rule.

Ask each interviewer to record their observations before the group starts discussing an overall recommendation. This gives the conversation several starting points instead of letting the first confident opinion become the shared story.

A useful debrief follows the criteria, not the order of interviewers' enthusiasm. Where does the evidence agree? Where are interpretations different? Which unanswered question actually matters for the job? If two interviewers disagree, ask them to explain the examples behind their conclusions.

Make the action at the end explicit. The team might have enough information to decide, need a focused follow-up, or need to revisit an unclear requirement. Don't schedule another broad interview just because the discussion felt inconclusive. Name the missing information first.

Rolefern's interview kits and scorecards give your team a shared place for questions and feedback. Keep the kit close to the candidate profile, and update it when a question repeatedly fails to produce useful information.

Review the process after the role closes. Were interviewers exploring distinct questions? Did the scorecard capture the work that mattered? Were the ratings understandable without the person who wrote them in the room? These are better maintenance questions than whether every form field was filled in.

Keep the final discussion human and grounded in the requirements of the role. The purpose of the scorecard is to make the decision explainable to the team, not to hide it behind a number.