Beyond the Basics: Upgrading Your Interview Strategy for Authentic Responses

In the first two parts of our Interviewing series we took the perspective of the interviewee and discussed ways to set yourself up for success. (Part 1 and Part 2)

Now I’d like to shift gears and talk from the perspective of the interviewer. How do you structure your interview to get the most out of it? If you’re the hiring manager, how do you structure the interview loop so that by the end of the day you feel confident in your hire/no-hire decision?

The answer is simple: signal amplitude.

Signal Amplitude

What the heck am I talking about? Have you ever been in an interview when an interviewer asks you, “So, tell me about yourself.” And then you spend the next 3-5 minutes reciting a prepared spiel, highlighting your job history, accomplishments, etc? Or worse yet, simply work line by line through your CV?

If you ever observed an interview like this from the side, you’d see both the interviewer and interviewee’s eyes glazing over. The recitation is rote, and from the interviewer end, it’s unlikely they’ll hear anything they haven’t heard a hundred times from other candidates. Booooring.

This question doesn’t even work well as a rapport builder because it’s so expected and so noncommittal. It says “Let’s get formalities out of the way before we get to the meat of the interview.” That’s not a good way to establish rapport, and simply shows the lack of preparedness on the part of the interviewer. It yields low signal amplitude. The interviewee is comfortable because they’ve spent hours honing the spiel, and the interviewer is comfortable because they can check out for five minutes and think about hot dogs.

What you want, as an interviewer, is high signal amplitude. In other words, you want to force the interviewee off the beaten path. You want them to think about their answer. You want them to take chances. And that’s not going to happen with generic, prepackaged questions.

Try this one on for size: “Tell me about your biggest mistake that had a tangible business impact.” Obviously, don’t lead with a question like that because it sets up a negative tone for the interview. But it’s worth throwing in once rapport has been established, and the candidate feels relatively comfortable.

Let’s unpack this question a bit. “Your biggest mistake” implies the candidate being asked to tell you a story about their past experience, and one she finds uncomfortable. “That had a tangible business impact” forces the answer away from triviality. It pushes the candidate into revealing something they might not have thought they’d be asked to reveal. Suddenly, they’re thinking about what to say. Should they tell you about that time they caused a site outage? Should they own up to shipping a critical bug? Would that make them look bad in your eyes?

The question also requires a mutual understanding that mistakes happen and are, if anything, a mark of seniority. A candidate that answers by telling you how they broke the build once is clearly far more junior than the candidate that tells you about the time they didn’t plan for all the contingencies in a project and as a result it slipped a month.

Naturally, the next question after this one should be “So what did you learn from that, and what would you do differently now?”

But the uber point in me bringing up this question is that it’s one in a class of questions called…

Behavioral Interviewing

Hopefully, most of you have heard the term by now. It simply means avoiding hypothetical questions and focusing on the candidate’s prior experience. This is why in Part 2 of this series I mentioned that as an interviewee you want to have a prepared library of stories you can use. These stories are meant for this type of behavioral interview questions.

Other such questions might be:
“Tell me about a time you disagreed with a colleague. What happened?”

“Tell me about a time your project was running hot, and you brought it back on track.:”

“Tell me about a time when…”

You get the idea. As an interviewer you want to target specific competencies in your interviewee.

For example, focus on ownership. “Tell me about a time you took ownership of an issue outside of your immediate scope.”

Or, dealing with ambiguity. “Tell me about a time when the requirements for your project were unclear and you had to go in to clarify them. What happened?”

Or, communication. “Tell me about a time when you had trouble being understood or understanding your colleague. How did you approach this?”

And so on. These types of questions lead to higher signal amplitude as they prompt the candidate to tell a story from their past and answer follow-up drill-in questions about that story. That’s gold for an interview.

But it’s also not the only modality for an interview. Here’s another one to try:

Role Playing

I was interviewing a manager once. We’d had a good start where he got the chance to tell me a story focused on ownership, but was clearly getting a little bored, as this modality was one he was clearly used to. So, I decided to switch tacks.

“Let’s do a role play,” I said. “You are my manager. I am your direct, just coming in for a 1:1, clearly upset. Ready?”

His eyes widened a bit. A smile crept across his lips.

“Sure,” he says. “Let’s do it.”

I fell back on the experiences I had acting in a community theater, furrowed my eyebrows and flared my nose.

“What do you do when someone you’re working with is a complete idiot?” I demanded.

He laughed, a little nervously, then started asking probing questions. I continued role playing his direct, seeing how he went about addressing my anger, and how he worked to lead the discussion in a constructive direction. After I ended the roleplay, I asked him a few questions about why he chose the approach he did, and what his thought process was during the interview.

Because the experience was primarily emotional – i.e. I playacted a degree of anger that he had to deal with – it felt far more natural to both of us. (Not because I enjoy acting angry, but because emotions are an intrinsic part of people management, and I would want to avoid any manager uncomfortable with dealing with them.) We both enjoyed the experience and I learned how the candidate approaches an emotionally-charged situation, something that occurs frequently in management.

The Dreaded Whiteboard Question

There’s no doubt that past behavior is a solid predictor of future behavior, and as we’ve discussed, focusing on behavioral-based questions can lead to valuable hiring data. However, for the more technical roles, behavioral-based questions alone aren’t enough to evaluate a candidate’s fit for the role as they are less adapted to ferreting out the candidate’s intellectual ability.

Put more simply, behavioral based questions that focus on technical attributes are easy to hack. “Yes, of course I implemented this super-complex system, and let me regurgitate the speech of how I went about it, which I spent the last 2 weeks memorizing.” A great behavioral interviewer may be able to get at the truth regardless, but, in my experience, companies tend to want to see a coder code before hiring her.

Enter the common “let’s solve a problem on the whiteboard” trope. We’ve talked about this trope from the point of view of the candidate in Parts 1 and 2 of this series. Today I want to focus on how to make this trope more effective from the interviewer’s perspective.

First, let me reiterate that this isn’t an effective way to conduct an interview. I’ve seen employees pass a coding interview and then struggle with ambiguity, communication, and even basic coding! How did they pass the coding part? Lots of practice. Most “common” interview problems asked are, well, common. You can find them in Programming Interviews Exposed, and a ton of other interview preparation books. Naturally, there’s no guarantee that the questions the candidate faces will be from that pool, but there would be some interview loops that draw from it, thereby producing a false-positive result.

And even if you come up with your own, unique problems, there’s a high likelihood that it can be mapped back to an existing question. It will take some smarts to perform that mapping, but chances are not the sort of smarts you’re testing for.

So, how do you get around this issue while still giving the candidate a chance to show off their skill?

Focus on Collaboration

In a real work environment, assuming you have a decent team culture, most problems are solved collaboratively. Several people get together and brainstorm ideas. An engineer might start with an investigation but should then pull in others to validate their initial (half-baked) thinking so they don’t spend too much time running down the wrong track. This is a skill that’s not generally tested well in an interview loop.

What if instead of a standard whiteboard question you asked something that requires this kind of collaboration? A question that’s very ambiguous to start, and can lead down several valid paths?

The interviewer’s gut instinct will be to say “No, that’s not going to work. How will I know if the candidate is going down the wrong path?”

Easy. In my experience, most engineers instinctively recognize the extent to which their colleagues are contributing usefully to the work at hand. Or, in other words, when collaborating with someone, it’s easy to see if they’re pulling their weight or not. Generally, it takes 5-10 minutes, tops. The kind of questions they ask. The kind of ideas they come up with. How they listen to your ideas and then improvise based on them. Did you feel invigorated after the conversation, or drained?

Good engineers recognize quality when they see it, but that takes collaboration. So, focus on collaboration.

For example, a question I like to ask in my technical interviews is one I cribbed from a Google phone screen I once had: “You are running a startup contracted with the California Department of Licensing. They ask you to design a system to manage the state’s car license plates. Go.”

I’ve asked this question to college interns and senior engineers. Obviously, the paths they take through the questions will vary based on seniority, but I’ve always tried to make the experience collaborative. We both kick around ideas for things to explore. We both talk about requirements and scenarios. At the end of the interview, I’m less interested in how much code they got around to writing than by how logically they approached the problem, and how much they contributed to the discussion. I have yet to be disappointed by the result of my hire/no-hire recommendations.

Turn the dynamic

The bottom line is simple: the more you make the interview an interrogation, the less value it will produce. Conversely, the more you can simulate actual work experience, the more of a feel you will get for the candidate’s suitability for your team. Turn the dynamic toward a productive collaboration, and you set the candidate at ease, give them the permission to let their ideas flow smoothly, to take chances. The signal amplitude will increase too, because they’ll have given themselves the permission to think, rather than merely recite pre-learned answers. You will see a difference in post-interview feedback, too. Even the candidates I ultimately didn’t hire spoke positively about their interview experience.

Try it!