When I was laid off, I interviewed at 22 media organizations. Half of them asked me to submit a proposal of project ideas, how I would run the team, how I would build the team. After a while, this felt a lot like free consulting, and I saw some organizations implement my ideas without hiring me. It was really frustrating.

At this year’s SRCCON, I had the opportunity to facilitate a discussion on how we evaluate design and development skills during the hiring process.

We looked at three main avenues:

• Idea-sharing/proposals: Any form of sharing ideas, strategies and/or plans of what you would do in this position; can be done on your own time and submitted or through a whiteboard exercise

• Problem solving: After being given a problem, you code or design a solution, sometimes in a pair-programming exercise with current members of the team

• Critiques: The interviewer provides a sample of the company’s work and you send an assessment, reviewing either a final product or a GitHub repo

These methods aren’t unreasonable. Work assessment tests are the best predictor of how someone will perform (PDF link) in a job. They can provide consistency across interviewees, since conversations can go in many different directions. You can see how a candidate thinks about challenges and problem-solves, instead of just evaluating the final product. Tests offer the chance for someone to prove their skills when talking about themselves in interviews doesn’t come naturally. As a potential employer, you can see examples of work that are related to your company’s work. And you can see what they can do individually instead of evaluating a portfolio of group projects.

The Problem(s) with Tests

But the way these evaluation methods are being implemented across the journalism industry pose real problems. The candidate may be using different tools and environments than they are used to. The interviewee doesn’t have all the information that the interviewer has; they haven’t been in all the meetings that produced the test, and may lack necessary context. Additionally, tests tend to measure how you perform under pressure more than anything, and how you behave in a completely different environment from your daily work life. Finally, when evaluating how people think, interviewers tend to hire people who solve problems exactly the way they do.

The main problems that came up over and over again in the session were related, and those were the facets we dug into: These evaluation methods force the candidate to complete free work for the company in a high-pressure situation, which creates one-way trust. The candidate has to trust that the company will use his or her work responsibly, whatever that may mean to the people involved.

Structured Interviews: A Strong Alternative

We initially set out to come up with alternative methods for evaluation, and we talked deeply about using structured, narrative interviews to ask how a person works through problems without making them actually do the work. Structured interviews are the second-best way to see how a candidate will perform at your company.

Here were the best practices that came out of the discussion on structured interviews:

• Determine ahead of time which skills sets and personality traits are most important for this position, and craft structured interviews that dig into those specifically.

• For must-have qualities, ask interview questions as part of the application process. This gives the candidate the opportunity to craft a thoughtful response, and the hiring manager can quickly weed out people who don’t make the cut. Wording questions like, “Tell me about a time when you…” is especially helpful here.

• Ask all of the candidates the same questions.

• Create a specific rubric for evaluation and explain what you are looking for to the other people who are involved in the hiring process. They can ask better follow-up questions if they know what you are trying to find out.

• If you have a candidate interviewing with multiple people throughout their visit, have each interviewer dig into a specific part of the job so the interviewee isn’t answering the same questions all day.

• Call a candidate’s references to ask work style and personality questions. Popular question to ask references: “What advice would you give me in managing this person?”

• If you feel like a candidate is not right because of “culture fit,” be sure to question this assumption and dig into why you feel that way. Challenge yourself to hire more diversely.

Interview (and Test) Thoughtfully

But we also came back to evaluation tests, because they are still valuable. Hiring managers appreciate seeing the raw work—what the candidate produces, and how they interact. It’s up to each company to decide how they will implement these tests, ideally in a manner that minimizes the problems and maximizes what the hiring manager needs from the process.

The main takeaway was to be intentional about your hiring process. Too many hires are made in an ad-hoc way that doesn’t compare the candidates in a fair process and likely doesn’t give the hiring manager all of the right information. Participants shared that when they structured their hiring process, they hired higher caliber employees.

Here are the suggestions that came out of our discussion:

• Give candidates a heads up that there is going to be in-person work happening. If possible, give them the homework ahead of time so they can prepare for it. It will give them confidence.

• For in-person work, ask the candidate what tools or supplies they need so they are working with an environment that is comfortable to them.

• Make it clear that the candidate can ask questions throughout the process. This is another opportunity to see how the candidate thinks, and two-way communication elevates a lot of stress.

• Pay candidates as a contract employee for intensive design and coding tests or any strategy documents that are produced. You could have them work on an actual project the company needs done.

• Develop a contract for work produced during interviews so everyone involved is clear on how the ideas and work created will be used. Can the company use the work if the employee is not hired? If the work is used, will the candidate get paid? What’s the copyright situation?

• Consider whether only finalists should complete the task to avoid wasting candidates’ time—but also keep in mind that this may mean you miss out on a candidate that might shine during the test more than a verbal interview. Either way, make sure you use uniform criteria to determine who takes the test.

• Respect candidates’ time. Design a test that won’t take more than 30–60 minutes to complete. You’re not asking the candidate to prove their commitment.

• Make sure you are providing accommodations for candidates so they don’t have to ask. Give a large window of time to complete the task so they can balance with work, family responsibilities and other time commitments. Ask: “How can we make this work with your schedule?”

• Give a test that is not economically actionable for the company so the work cannot be used if the candidate is not hired.

• Design a test where there is no perfect answer and neither the interviewer and interviewee are experts in the topic. If you just finished an investigation into maternal deaths, you likely know way more than the interviewee, and they don’t have enough context to succeed.

Above all—and before you talk to any candidates—think about what you want and what you are asking for from candidates. By communicating your hiring method, you are sending the message that you care about the candidates and respect their time and that candidates are being evaluated fairly in the same way. Being intentional not only gives you a more efficient and fair process, but it’s an incredibly kind thing to do for your candidates. It takes time for you to plan your hiring process, but this is time well spent for your company: the way you hire is part of the company’s brand.

• Share on Twitter
• Share on Facebook