[{"_1":2,"_29":-5,"_30":-5},"loaderData",{"_3":4},"routes/__layout/tag.$name",{"_5":6,"_7":8},"tagName","technical interviews","posts",[9],{"_10":11,"_12":13,"_14":15,"_16":17,"_18":19,"_20":21,"_22":23,"_24":-5,"_25":26,"_27":28},"id",1465,"title","You suck at technical interviews","excerpt","Most technical interviews test the wrong things: memorized syntax, whiteboard coding, brand-name employers. Instead, hire for learning ability, intellectual honesty, clear communication, and basic decency. \"Team fit\" is bias in disguise. Homogeneity means you're hiring wrong. Good hiring is hard and slow, but worth it.","body","

You are bad at giving technical interviews. Yes, you. You're looking for the wrong skills, hiring the wrong people, and actively screwing yourself and your company. Without changing anything about your applicant pool, you can hire different people and your company will do better and you will enjoy your job more.\n\n

I realize these are bold claims. In the ten years since I became senior enough to be asked to interview people, I have conducted a great number of technical interviews, been part of a lot of teams at companies big and small, and watched the effect that different types of hires have had on those companies. I'm not claiming to be perfect at hiring -- at various points, I have done nearly all of the things wrong that I'm about to tell you not to do. But here's what I've learned so far.\n\n

You are looking for the wrong things

\n\n

Don't hire for what they already know

\n\n

The primary mistake that people make when interviewing is over-valuing present skills and under-valuing future growth. Don't hire people for what they already know; the pool of people who do exactly the thing you need them to do is much, much smaller than the pool of people who are smart enough to be good at that job.\n\n

But even worse is the way we try to determine whether people have these skills. People ask questions in interviews about obscure syntactical features of programming languages, or details of popular APIs. The famous fizzbuzz test simply asks \"are you aware of the modulo operator?\" If the answer is \"no\" then they are probably a pretty weak candidate, but it provides exactly 1 bit of data. Yet people will spend twenty minutes on it in an interview, a huge waste of limited time.\n\n

Don't hire for what they can remember in an interview room

\n\n

I used to ask people to write code in interviews. This is terrible. Writing code on a whiteboard is an experience so far removed from the real practice of writing code as to be no predictor. Even writing code on a computer, as part of a pair for instance, tests for the wrong ability -- you are asking them to write code under time pressure, with somebody watching. Some of the best engineers I know would melt under those conditions. And if your belief is that writing code under intense time pressure is part of your job, then you should examine whether that's a problem your company has.\n\n

Whiteboard and coding interviews also routinely ask people to re-implement some common solution or low-level data structure. This is the opposite of good engineering. A good engineer will try to maximize productivity by using libraries to solve all but the most novel problems. It's also a poor test: how do you know if somebody is solving the problem or merely remembering somebody else's solution? There is no value to memorizing the details of algorithms you can google in 15 seconds.\n\n

Don't hire for a fancy degree

\n\n

Some people are impressed by academic credentials. Having gone to a good college, or having gone to college at all, are not in my experience good predictors of ability as an engineer. Having a PhD in a relevant field is interesting but also an unreliable predictor: engineers write code and ship software; academics prove theories and write proofs of concept. Somebody smart might be able to do both but it's by no means a given, or even very strongly correlated.\n\n

Don't hire for their previous employers

\n\n

People also over-value brand names on resumes. Don't hire somebody because they worked at a hot company, or a famous one, especially not if that company is big. Variation across teams in big companies is enormous. Just because a company was successful doesn't mean your candidate had anything to do with that. If you are familiar enough with another company's hiring process that you can vouch for it as a good selector of qualified people, you might use that to bump them to the front of an applicant queue, but beyond that, go with what's in front of you.\n\n

Don't hire friends and family

\n\n

And finally, never hire your family and if you can avoid it, don't hire your friends either. Existing relationships create bias and implicit power structures, webs of obligation and loyalty that are at odds with what is best for your company. You will either compromise your friendship or your company, and rather than being forced to pick one or the other, just avoid the conflict entirely.\n\n

Here's what you should be looking for instead

\n\n

Your first stage of interviewing should be attempting to answer two questions:\n\n

Can they do this job? This is not the same as \"can they do this job right now?\" but you need to be confident they can learn how to do the job.
Are they going to get better at this job? The answer has to be yes.

\n\n

Relevant experience is a plus but not a requirement

\n\n

Syntax and API questions are aimed at finding relevant experience but they are bad ways of doing so. Instead, talk about the technology they're going to be working with. Find out how much they know about it. You're not looking to hire or pass on the basis of any individual fact. If they are weak on the skills they need for the job, then find a topic they know a lot about instead, and get them to talk about it, if necessary explaining it to you. You are looking for grasp of complex topics and the ability to clearly communicate about them, which are the two jobs of the working engineer.\n\n

Somebody who is constantly improving is a requirement

\n\n

Much more important than what they know is how they learn it, and how quickly. You are looking for somebody with a track record of learning new skills and applying them successfully. Talk about their career path, and look for evidence of increasing responsibility (which is related to, but not the same as, seniority). Remember that anybody you hire will expect raises every year: somebody who isn't getting better all the time is going to become worse and worse value unless their skills increase in value, too.\n\n

Smart and Gets Things Done™

\n\n

Joel Spolsky's classic essay The Guerilla Guide to Interviewing (and the book that followed it, Smart and Gets Things Done) are some of the smartest things that have ever been written about technical hiring. Smartness is hard to judge, and some of the techniques I'm talking about here should help. But \"gets things done\" is equally important. Have they shipped real software?\n\n

Joel and I don't agree on everything, though. I don't think live coding exercises are particularly valuable. Joel is (or was) also pretty big on understanding pointer math, which is an interesting \"can grasp a complex topic\" question but likewise outside the experience of many excellent engineers, so while it can be a useful thing to try talking about, it shouldn't be an acid test.\n\n

Somebody who can intelligently discuss technology

\n\n

As I mentioned, the two jobs of an engineer are to understand complex concepts, and then communicate them clearly. Somebody who can do just one or the other may have a brilliant career in some other field, but is going to be an inferior engineer. The best programmer in the world can develop incredibly efficient algorithms in record time, but the job of an engineer is to work with a team to achieve something larger, and if you are unwilling or unable to spend time communicating with your colleagues you're only doing half of your job.\n\n

Somebody who knows what they don't know

\n\n

When interviewing I always attempt to find some area of expertise where I definitely know more than my subject. This is not to prove I'm smarter -- see below -- but because it's important to see how somebody reacts when they find themselves out of their technical depth.\n\n

The weakest candidates will try to waffle or make wild guesses. This is a terrible sign, firstly because it never works, and secondly because they thought that it would. In Dunning Kruger terms they are in the bottom quartile, unable to accurately judge their own lack of knowledge. It also means they will try to do this in other situations.\n\n

Strong candidates say \"I don't know\" as soon as they hit their limit, and may start asking questions. The very strongest candidates say \"but if I had to guess\" and then attempt to extrapolate. Those are great because it shows intellectual honesty and a strong desire to figure things out.\n\n

This is a conversation, not an interrogation

\n\n

As I said before, it's a good idea to find an area where you know more than your interview subject. But this is not to prove to them that you're smarter or better: it's to explore the extents of their expertise, to get a sense of the breadth and depth of their knowledge. You will bump into the edges or touch bottom, and that's the point. When you do so it doesn't mean they've failed.\n\n

It's important that they're aware of this contract, too. You want your interviewee to be relaxed and comfortable, because that is the state they'll be in when they're doing their job (if not, your company is awful, please quit). Answers you get from candidates who are stressed or panicky are basically useless. This is true even if they're good answers, because they're not representative answers. Stress and panic are not sustainable states, so you risk hiring someone who only performs when pressured to do so.\n\n

There's always an exception

\n\n

If your candidate has no relevant prior experience, your only option is a more traditional technical interview. This is true of both very junior candidates, and also more experienced people switching from other careers.\n\n

Somebody who has just completed a training course, no matter how intensive or well-regarded, does not know how to be an engineer. They may know how to code, but that's only half the job. Somebody fresh out of college may not even really know how to code beyond academic puzzle-solutions. In my experience, if someone has been coding professionally for less than a year, there's not really been enough time to know if they're good at it.\n\n

Fresh-out-of-college and other junior types are also not going to know anything about how to interact professionally. Not only does this make interviewing them trickier, it's also a terrifying responsibility: if you hire them, the culture and working practices of your company will be what they think of as \"normal\" forever.\n\n

Big companies can accommodate poor communicators

\n\n

The other exception to my rules is if you work at a company large enough that engineers can become deeply specialized. When that happens, you may in fact need the genius programmer who isn't very good at explaining what they're doing. If you're big enough, you can hire a manager whose full-time job is to communicate with that person and then translate back and forth between them and the rest of the company. This works really well but is expensive, so it's not something startups can really afford to do. Past your first 50 engineers, you might consider it.\n\n

The final question: do I want to work with this person?

\n\n

When you're sure your candidate is good enough to do the job, you have another question to answer: do I want to work with this person? There are no specific questions to ask to get this answer; it's more about how they answer the other questions. This makes it a personality test, and that makes it very, very dangerous territory. Personality is subjective, and that means you are inviting bias into the equation. Personal bias, implicit bias, unconscious bias.\n\n

The terrifying possibility of turning away a great candidate because you are biased towards them in some way you don't even know about is why people think giving pure, right-or-wrong technical questions are better. They're not, they're just easier. And they do nothing to protect against bias: when there are 50 syntax questions you ask, it is easy to give hints and passes to the candidates you implicitly prefer and pretend that they were just better. I've caught myself doing this. There is no simple way around this: you have to be aware of your biases, constantly conscious of them, and correct for them, or you will screw your company by hiring inferior people.\n\n

Look for somebody to WORK with

\n\n

The most common way I see startups get the \"do I want to work with them?\" question wrong is by confusing that question with \"do I want to be friends with this person?\"\n\n

Get that assumption out of your head. Those two are not the same. You can have brilliant, productive professional interactions with someone with whom you have absolutely nothing in common with on a personal level, and that's fine. Your company does not need to \"feel like a family\". It's sure as hell not your frat house. You are not picking a drinking buddy (and as an aside, if your company regards drinking as a big part of its culture you probably have deeper problems).\n\n

No Jerks

\n\n

From day one at npm Inc we implemented our No Assholes policy, and I was pleased to read recently that Polyvore (who seem to do brilliantly at maintaining a diverse engineering team) have pretty much the same policy. Avoid the \"genius assholes\", avoid the bitter and cynical, the bullies, the snobs. Don't work with somebody who is going to be mean, unpleasant, or demeaning to their co-workers. There is no level of brilliance and productivity that can compensate for poisoning the morale of your team, and once a team culture is broken it is very hard to fix. Hiring these people, even to get you through a crunch, is not worth what it costs. And if you hire one by mistake, fire them fast, and without hesitation.\n\n

The easiest way to spot that you are hiring a jerk is the phrase \"hire, but not for this team\". That means \"this person has skills, but I don't want to work with them directly\". If you don't, nobody else will, so don't inflict crappy people on other teams.\n\n

But in general jerks are easy to spot. If somebody has a personality flaw so strong and baked-in they can't keep it in check for a couple of hours while being interviewed, it is going to be a huge problem in the regular work environment. Arrogance, rudeness, inattention to detail -- these things turn up quickly, and if you spot them, trust your instinct to avoid them.\n\n

You are NOT hiring for \"team fit\"

\n\n

I have never heard a definition of \"team fit\" that didn't end up sounding like \"let our bias run free\". Phrases like \"looks like they belong here\" are terrifying. More insidious are complaints like \"doesn't like the same social activities we do\". Grow up. Your office is not your frat house, and socializing with your co-workers outside of work is not some crucial test. There is no requirement that you like someone socially as long as you want to work with them.\n\n

And while I'm at it, as an introvert and lifelong non-drinker, may I make a personal plea that you stop incorporating social events into your hiring process? Professional interactions and social ones are not the same. Some people suck at small talk, and are not comfortable at bars. Remember the part about making sure your subject is relaxed and comfortable? It's about them being comfortable, not you.\n\n

How do you reconcile the \"don't hire for team fit\" rule with the \"no jerks\" policy and the \"somebody you want to work with\" requirement? The distinction is subtle, but important: somebody who is good for your team is not necessarily somebody you want to be friends with. It's tempting to look for both, but it's the wrong metric for the success of your company and is ultimately unsustainable.\n\n

Homogeneity is disastrous

\n\n

I'm not going to make an argument that \"diversity is intrinsically good\" for some social purpose. That's a stupid way of looking at it. Lack of diversity is obviously, mathematically, bad for your company. Instead of hiring for \"best for the job\" you have accidentally hired for \"looks like me\". There is no chance that all the best programmers in the world look the same, so a lack of diversity means only bad things. It means \"this team sucks at hiring\", it means \"management and HR are not strong or competent enough to spot and correct this\", and worst of all it means \"this team is not the best people available\".\n\n

But what if I can't find anybody like this?

\n\n

Here I must hand-wave. npm is incredibly lucky on the talent front: people love node and npm, and we get dozens of qualified candidates just by posting a vacancy. My previous startup, awe.sm, was not nearly so popular, but we managed to find good people anyway. It's a matter of taking a long time, and trying every channel: we got great candidates via posts to the Who's Hiring post on Hacker News, via our blog, and once from some coverage on TechCrunch.\n\n

The important thing to remember is that hiring a bad person is more expensive and wastes more time than waiting for a good person. It's tempting to say \"at this point, anybody would do\" but that's never, ever true. The wrong person will not merely fail to do their job, they will make everybody slower at theirs, and unhappy to boot. Don't hire if you're not sure about somebody, and if you hire somebody who's no good, give them as much support and direction as you can afford to get them to turn around, but if you don't see any improvement be ruthless in letting them go.\n\n

This is all very hard

\n\n

This is why you've been giving bad technical interviews all this time: bad interviews are easier to give. They require less thought and creativity and effort on your part. These techniques are tricky to define and tough to follow. But that's what hiring is like. It's really hard, and it always takes much longer than you hope it will. The rules are fuzzy, and there are no acid tests that can be applied. But the payoff to trying harder is a stronger company, better people, a better product, and a happier working life for everyone on the team. Making those things happen is, as a hiring manager, your only job.\n\n

TL;DR

\n\n

many interview techniques test skills that are at best irrelevant to real working life\n
you want somebody who knows enough to do the job right now\n
or somebody smart and motivated enough that they can learn the job quickly\n
you want somebody who keeps getting better at what they do\n
your interview should be a collaborative conversation, not a combative interrogation\n
you also want somebody who you will enjoy working with\n
it's important to separate \"enjoy working with\" from \"enjoy hanging out with\"\n
don't hire assholes, no matter how good they are\n
if your team isn't diverse, your team is worse than it needed to be\n
accept that hiring takes a really long time and is really, really hard\n

","updated",["D",1409528899000],"created",["D",1409048015000],"codename","you_suck_at_technical_interviews","legacy_id","published",["D",1409528899000],"draft",false,"actionData","errors"]

Posts tagged “technical interviews”