Does communication matter in technical interviews? Here's the data

Hey, Aline (founder of interviewing.io) here. This is the 5th post in our Guest Author series.

One of the things I’m most excited about with the Guest Author series is the diversity of opinions it’s bringing to our blog. Technical interviewing and hiring is fraught with controversy, and not everything these posts contain will be in line with my personal opinions or the official opinions of interviewing.io. But that’s what’s great about it. After over a decade in this business, I still don’t think there’s a right way to conduct interviews, and I think hiring is always going to be a bit of a mess because it’s a fundamentally human process. Even if we don’t always agree, I do promise that the content we put forth will be curated, high quality, and written by smart people who are passionate about this space.

If you have strong opinions about interviewing or hiring that you’ve been itching to write about, we’d love to hear from you. Please email me at aline@interviewing.io.

---

Dima Korolev

Dima Korolev finds joy in helping established companies on a large-scale architecture level, previously at PokerStars, now at Miro. His career includes roles at Google (from 2007–2011), along with other large companies and several successful startups. While in school, Dima once earned Coder of the Month and was consistently in the top 100 globally on Topcoder. He particularly enjoys mentoring engineers and finds it emotionally rewarding to help others develop stronger skills and expand their horizons. His love of helping others inspired Dima to become one of the first interviewers on interviewing.io, completing more than 100 interviews to date. You can reach him on LinkedIn, GitHub, Twitter, and his website.

---

The interviewing.io platform has hosted and collected feedback from over 100K technical interviews, split between mock interviews and real ones. It’s generally accepted that to pass a technical interview, you have to not only come up with a solution to the problem (or at least make good headway), but you also have to do a good job of articulating your thoughts, explaining to your interviewer what you’re doing as you’re doing it, and coherently discussing tradeoffs and concepts like time and space complexity.

But how important is communication in technical interviews, really? We looked at the data, and it turns out that talk is cheap. Read on to find out how and why.

The setup

On interviewing.io, engineers can practice technical interviewing anonymously. When an interviewer and an interviewee match on interviewing.io, they join a collaborative coding  environment with voice, text chat, and a whiteboard, and jump right into a technical interview. After each interview, both parties leave feedback, and once they’ve both submitted, each one can see what the other person said and how they were rated.

Here’s the feedback form that interviewers fill out:

As you can see above, there’s one yes/no question: Would you advance this person to the next round? There are also three other questions, each graded on a scale from 1 to 4 (4 is best):

• Technical ability, which we’ll call Code
• Problem-solving ability, which we’ll call Solve
• Communication skill, which we’ll call Communicate

Do the numeric scores predict whether a candidate gets advanced to the next round? You bet! Here are the top buckets, ranked by how often the interviewers say “yes”:

Right away, three pieces of anecdotal evidence suggest that talk is cheap:

1. The only “2” that made it to the list is communication score (see 4-4-2 above).
2. The least valuable “4” coupled with two “3”-s is also the communication score (see 3-3-4 above), with a major drop in hire rate. 
3. And, ironically, 4-3-3, a scenario where the candidate got 3s in both communication and problem solving, has a [negligibly] higher success rate than 4-3-4, a scenario where the candidate got a 4 in communication and a 3 in problem solving!

In fact, the 4-4-2 bucket is the third best one can get, losing only to 4-4-4 and 4-4-3. Interviewers are happy to advance 4-4-2 candidates to the onsite round a whopping 96% of the time. Notably, a candidate is 3X more likely to get rejected with 3-3-4 than with 4-4-2.

So it seems that talk is cheap. But to learn exactly how cheap, we have to go deeper, and this is where data science helps.

Into the data

It’s data science time! Let’s do a thought experiment. Imagine a candidate walks into an interview, impaired by the moral equivalent of 1 point, on either their ability to Code, to Solve, or to Communicate.

To minimize damage, from which ability should our imaginary candidate have that 1 point deducted?

To answer this question, let’s take a look at how much worse this rejection rate would be with a 1-point deduction.

Whoa!

For a good third of all interviews, dropping 1 point from the Code or Solve scores would lead to the rejection rate skyrocketing 6X! Clearly, that last point in Code or Solve is extremely valuable.

At the same time, in all categories, the negative impact of dropping 1 point from the Communicate category is by far the smallest.

The opposite direction is interesting as well. Say our imaginary candidate is fond of Heroes of Might and Magic and was up all night playing. This is not recommended either, but imaginary experiments really help with science. Just before the interview, this candidate meditates, their inner interviewee hero visits a Temple, and their Morale improves by 1. They now have a +1 score point. How should our hero best use it? The rejection rate would now decrease with the score increasing. Hence it’s green in the table.

The TL;DR is that +1 to Code and +1 to Solve have comparable effects, and their effect is always significantly higher than the effect of +1 to Communicate.

The “top 25%” row is not very representative, as the 4s can not improve, and plenty of scores there already are the 4s.

For other rows, boosting Communicate by 1 turns out to barely affect the rejection rate. Boosting Code and/or Solve, on the other hand, can help flip up to 4 of 10 negative outcomes!

Pragmatically speaking

Unfortunately, in real life we don’t have magic potions that +1 your interview performance. But we do have considerable control over selecting which score to improve. Of course, 4-4-4 is the holy grail, but during an interview it is often up to you to decide which score to focus on.

Here is a more realistic example. What if you could trade one score for another?

Again, the table shows the difference in rejection rates, the numbers in red are bad, and the numbers in blue are good. To reduce confusion, the improvements are bolded, and the respective rejection rate percentage points are cited next to those improved numbers. The top row is removed because, as mentioned earlier, it is not representative.

Nothing could be more clear. When in doubt, trade Communicate for Solve or Code. Period.

Conclusions, and why communication still matters

The bottom line: If your objective is to get a “yes,” it is far less damaging to score 1 rating point lower on Communicate, and it is much more beneficial to score 1 rating point higher on Code and/or on Solve. Talk is indeed cheap, and indeed, coding and problem solving is what you need to show in a coding interview.

Keep in mind that, obviously, these insights are valid for coding interviews. In behavioral interviews, the technical skill signal is the least valuable, while problem solving is king. And in systems design interviews, extra points on communication skills, while still the least valuable of three, are not worth exchanging for additional points in tech and problem-solving skills.

It is worth noting that practice coding interviews, while highly useful, are not 100% representative of real-life interviews. A real-life interview, even if explicitly focused on coding, is a mixture of everything.

The more senior the position, the more vital systems design and behavioral skills become. This statement is harder to prove with data, but every interviewer I spoke with agrees—at least within the standard L3 to L6 range, from a junior to staff.

Thus, we can conclude that:

• For L3 and L4, you are better off ignoring the communication score entirely, as long as you are consistently scoring 2+ on it.
• For L5+ interviews, where communication skills are far more important, standard coding practice sessions are not the best way to improve on them. Systems design practice, as well as the behavioral interviews, are there to help.

Good luck, and have fun on interviewing.io!

from Hacker News https://ift.tt/jZJOnYw