> "We’ve seen that most engineers only have the stomach for a limited number of interviews. Investing time in the wrong companies carries a high opportunity cost."<p>I suspect in addition to not having the "stomach" for an unlimited number of interviews, they don't have the vacation days to burn for them. Let's say you are working already and have 10 days a year vacation (pretty standard). With these ridiculous all-day interviews, you have the ability to waste that vacation on a maximum of 10 companies, and that's if you choose not to take any "real" vacation at all during the year.<p>In an environment like today, where every company is ostensibly hiring yet overly picky, a job candidate potentially will waste their entire PTO balance on interviewing with companies serious about interviewing but not serious about hiring.
I'm an "enterprise" programmer because I write in Java... I'm also older than the average age of a start-up employee (late twenties). I've never written a line of Ruby, and I've never written object oriented JavaScript or used Node.js.<p>However, I'm a <i>really</i> good programmer. I just happen to write the majority of my code in Java. If tomorrow we decided to use a new language, I could pick it up in a few days... I have a feeling the "enterprise" thing is just veiled age-ism. Anyone can feel free to correct me if I'm wrong, but at this point, it's literally <i>impossible</i> to learn <i>every</i> new technology. And anyone whose worth their salt should be able to learn a new environment and programming language without too much trouble.<p>I have a feeling that the enterprise thing is just a way for these employers to screen out the older, more experienced programmers.<p>This will probably get this throwaway banned, but Michael O'Church (who I am not) writes a lot about this on his blog. I tend to agree with him.
So... I worked with over 50 engineers prepping for their first SF/Silicon Valley tech interviews. And I worked at Groupon, interviewed at a bunch of YC companies (including one where I took a job).<p>In my experience pretty much nobody is taking Ben Horowitz's advice from <i>The Hard Thing About Hard Things</i>—hire based on strength instead of a perceived lack of weakness. Instead everyone seems to be looking for candidates who fill out all the boxes and have zero thumbs down in interviews. That coupled with extreme risk aversion means long interview cycles, and huge amounts of wasted time for some of the most talented employees and founders on the planet.
> Almost no one passes all their programming interviews. This is true because of randomness in many interview processes (even great people are bad at some things, and an interviewer focusing on this can yield a blocking no).<p>Reading this is somewhat reassuring. I've been over some interviews lately and I had some trouble getting past the tech phone screen.<p>One particular rejection was very frustrating, because I've spend quite some time preparing for the interview. Failed the first one. Got a second chance. And I knew it was over about 5 minutes after it began. "Can you write an algorithm that would sort efficiently this k-sorted array with a complexity strictly inferior to O(n log n)?". Yeah. No. Neither my job nor my hobbies includes sorting almost sorted arrays of integers, and O(n) complexity calculations.<p>It really seems to me that, being hired by a tech company is just completely random. Tech interviews in general are completely random. "Just a numbers game"
This is why I don't really interview my sub-contractors. If I run into someone at a meetup that I think seems smart, I ask them if they'd be interested in picking up a small project. If they accept, I... give them a small project. No resumes. No demeaning interrogation. Straight to work on something real. And I pay them.[0]<p>If they do well, I give them another one. If they don't, I tell them I'm sorry but I don't have any more work for them.<p>Thinking more deeply on the issue, I think I want to know as little about a potential candidate as possible.[1] I don't want to bias myself against them. I've had a lot of bad experiences in my career and I'm sure I'm not perfect at keeping my prejudices to myself. I want to get people into the chair as soon as possible, get them working, and judge from the work. Once I see work getting done, it's really easy to ignore everything else.<p>[0] I don't think I <i>can</i> even make someone do an "interview assignment" for free. I'd be receiving something of value--their labor--but I'd have no accounting of it for taxes and I have no desire to try to figure out how to keep track of something like that.<p>[1] Yeah, I'll say this probably includes whether or not they have any experience with programming. If they can't do the job, we'll find out soon enough. If they can do the job to a satisfactory degree, why do I need know how long they've been doing it before? Give people a chance to surprise you.
I'm not a java programmer (my knee jerk reaction to quickly bang out an MVP would mostly be python), but this negative attitude towards Java seems unfair. There's a lot more to the Java universe than just enterprise Java. There are companies using Java that do cool stuff at large scale, and very reliable. Netflix is heavily Java, and nobody would (at least I wouldn't) argue that their tech is dull. Even if you are looking for pure hip factor, there's things like vert.x, and all these other JVM languages, which are not Java per se, but one of the selling points of those languages is it can utilize tons of libraries in the java ecosystem if needed. The last argument against java would be its verbosity and productivity (lack of), but I'd argue Java has one of the best IDE support among all languages, which helps alleviate the problem significantly.
"Two large YC companies (both with machine learning teams) have told us that they consider interest in ML a negative signal."<p>I wonder why this is?
Since ML/AI are currently "hot" those programmers may be trend followers? Or maybe interest in ML is correlated with being a junior programmer (those that are more senior specialized when ML/AI were not so cool and consequently are in different domains)?
As a "Product Programmer" myself, I find it highly ironic that despite the fact that there is apparently more demand for product-focused developers than technical-focused ones, the interview process for most startups is <i>overwhelmingly</i> technical-focused.<p>If you're looking for product-focused developers, please consider tuning your interview process to evaluate whether or not candidates can build great products, rather than following the herd and grilling them on obscure algorithms and data-structure problems, which has a rather high chance of weeding out the kinds of product-focused developers that you're looking for in the first place.
I've had 10+ interviews in the last month and it's really a shit process. First of all, I'm terrible at technical interviews. There's just something about being in that "we're watching your every move" environment that makes me freeze. I found myself literally reading the same for loop line over and over again, not even trying to comprehend it, but rather zoning out on it as my brain continued to focus on what I thought the interviewer was thinking. This, of course, gets worse as the seconds tick on and I've said and done nothing. I also perform terribly when I'm given a take-home "test" with a time limit. The pressure to finish on time overwhelms my thought processes. Then you have the asinine binary tree questions for a front-end web dev position or the "tell me what's wrong with this horrible, obfuscated code". If I have to work on code like that, I don't want to work there.<p>I had multiple interviews where I was either being video recorded (karat.io) or on a skype call with multiple developers "evaluating" me. This is a toxic environment and I knew in the first few mintutes I didn't want to work there.<p>I have a portfolio full of sites that I've built and I can tell you with confidence that my portfolio didn't matter even a little bit. Not one employer looked at my github account (not that it's impressive, but that says something).<p>The best way to get a job is through your network. A previous freelance client was hiring and pursued me aggressively. Why would they do that if I'm a shit developer as thought by my various interviewers? The experience of "interviewing" with a company that knows my work and shows a genuine enthusiasm for having me on their team is so night-and-day different from every other interview that the decision was easy. They gave me what I asked for and treat me like a valued employee.
The pattern matching issue with Java or C# is interesting, since all these companies generally do want mobile. So I guess the humorous point of advice for that: replace all occurrences of Java on your CV with Android, and C# with WinPhone, and you'll probably bump your chances of getting passed the pattern matching.
> "Second, companies dislike programmers with enterprise backgrounds. Our data shows that companies are less likely to hire programmers coming from Java or C# backgrounds."<p>This I totally understand. Enterprise software is systematically horrible in almost every way: terrible UI/UX, insane degrees of over engineering, high footprint, high cost, and usually at least two to three generations behind on every technological trend. Of those traits I can't stress over-engineering enough... it's epidemic everywhere but most of all in "enterprise" where you'll see insane things like simple web application servers that use gigabytes of RAM and XML schemas that lead to ten-page-long messages to do trivial things. (The use of XML at all is usually a smell unless the domain is very HTML-like such as word processing... and even there extending Markdown would be better in every way.)<p>It doesn't have to be that way. Those facts reflect the management pathologies that exist in large companies and governments where you have a lot of people managing software projects who don't know anything about software... lots of "pointy haired bosses." You also have a lot of dumb requirements in those areas that twist things out of alignment with what users actually want. Startups very often have the luxury of ignoring stupid requirements unless they have to interoperate with legacy stuff.<p>It's so bad that I've actually heard people advise startups to pass on some enterprise <i>revenue</i> if they can afford it (pass on REVENUE!) if it might lead them down an "enterprisey" path, since doing so would in the end result in a systematically inferior product. If the systematic diseases of enterprise are that extreme (and I think they mostly are), then I understand the desire that some startups might have to also systematically avoid developers with enterprise backgrounds.<p>That being said and while I do understand, I think this underestimates the versatility of a good developer. A good developer might have gone into an enterprise job because they need the money but they might be otherwise great at what they do. You can find some great developers by offering to rescue them from enterprise hellholes. Sometimes that alone is all they need to inspire a ton of motivation and loyalty. I mean, you just dragged them out of hades. They're gonna love you.
<p><pre><code> The “Product Programmer” and “Technical Programmer” profiles are identical,
except one is motivated by product design, and the other by solving hard
programming problems.
</code></pre>
This is great.<p>I've struggled with trying to define the difference between the great systems engineers I meet and the engineers who make great products (sometimes at the expense of all elegance under the hood).<p>This sums it up nicely... it's where the focus is. I'm surrounded by engineers focusing on systems programming, but I don't think of myself as similar to them, I've always been end user obsessed and customer focused. It's nice to see this difference acknowledged.
Hi Ammon/Harj, Thank you for that effort and laying it out. Very helpful.<p>Just so the readers don't miss the context: By definition, most companies referred here, I'm guessing, are startups. And startups will definitely want more product-focused engineers, in order to keep moving fast.<p>Interviewing in general, is closer to a date, than it's to a standardized test. The smaller the company is, the more pronounced that characteristic is. For even slightly larger companies, it's a different story.<p>When I was a Director of Engineering at Box, the engineering team was tasked with hiring 25 engineers in a single quarter. For several quarters. When hiring at that scale, it's hard to hire based on personas and elaborate preferences. At that point, process is more important. Anyone that meets a consistent process gets hired. There are always biases at resume selection, but those are only to benefit the later process, and not so much of a preference.<p>[About us: <a href="http://InterviewKickstart.com" rel="nofollow">http://InterviewKickstart.com</a>]
This was a great point: "There’s more demand for product-focused programmers than there is for programmers focused on hard technical problems." A very talented programmer from Dropbox once told me that if I wanted to attract top engineering talent, I needed to be able to show the engineer "a problem that no one else has solved yet." This totally changed the way I wrote my job descriptions and conducted interviews. Led to great outcomes, too.
This is interesting data.<p>I liked what Joel Spolsky said a long time ago. Basically that companies should want two types of engineers. 1. You want a few who are experts in the company's tech stack. 2. The bar for everyone else is just that they're smart and they get things done.<p>I guess the core problem is that we don't have a good objective measure of the latter. (Maybe an IQ test for smarts, if that wasn't political incorrect, but I don't know how you'd show definitively that you "get things done" in an interview.)
This reminds of psychological study about superstition. They studied fishing habits of South American Indian tribes; the tribes that fished in lakes, had consistent and reasonable rules about when/where to go fishing. However, the tribes that fished in the ocean, where you cannot predict the catch, had rather random and superstitious rules about when/where to go. This is due to our brains seeing patterns in randomness.<p>Also, it's perhaps understandable why companies don't want to reveal their preference. If you do it, you open yourself to being gamed. I think it's also better to have unreasonable requirements (looking to give genius programmers some boring product work) than be sorry. Again this happens in absence of actual test (other than wait and see) of who is a good candidate. Females do the same thing when mating.
I had such a disappointing experience with Triplebyte. I take their automated test and everything is hunkydory. After that they gave me a very ambiguous technical interview live-coding session without an interviewer. It seemed like an unreleased feature or a trial a/b test variant. Even now I go back and they've removed every reference to a "programming challenge"!<p>Anyway, the problem involved writing a tree generation/traversal along with a little equation parser and a lot of string parsing (I think, at least). I was really uncomfortable because while it said "we will run this code" I didn't know what their expectations where (In what environment are they going to run the code? Is underscore ok? They said "you can't use built in eval" so do they really want me to write my own eval? Do they care if I look on stackoverflow for string parsing stuff?). I only had an hour for a difficult problem and I spent most of the time wondering what they really wanted and stressing out about little details that wouldn't consume ten seconds of thought on the job.<p>I've interviewed a lot and I'm pretty confident from a lot of experience whiteboarding code or typing up an answer on the spot in interviews, so this was very frustrating and I found it to be disrespectful of my time.
> He solved hard algorithm problems like they were nothing<p>That's mostly practice. When I did webdev I was really shitty at algorithms because there were no algorithms in my daily work. When I did some Baduk AI programming I started being much better at algorithms since I was implementing some custom algorithms myself.<p>If you are interviewing someone for a web dev position, it's kind of ridiculous to screen people by their ability of merging sorted arrays or whatnot.
Well, consider that there are well documented academic studies (that I can't be bothered to look up right now) showing that most hiring processes are no better than throwing darts at a board with regards to retention rate, employee success, and every other measure of success of the HR process. The only thing that is remotely effective are IQ scores, and even then it's only a weak correlation.<p>So regardless of what the companies want, it's near impossible to accurately judge someone in an interview process. Even if you know what you want, it's very difficult to assess how someone is going to perform in a job through an interview process. The main benefit of interviews IMO is to help managers "buy-in" on hiring decisions.
What is the point of interviewing through TripleByte and going through their interview process, if I just have to interview with the YC company again? It's something I would completely avoid. I already know that almost 100% of the companies I send my resume to will respond, so what is the point of adding another set of interviews, which as the article points out, only adds a random level of success.
I would love to see this kind of analysis (i.e. Junior Programmer vs. Child Prodigy vs. Rusty + Experienced) as it applies to hiring women. Are there biases against women with different experience and different "culture fit?" Would be a neat way to apply your data and your company contacts.
Former C# programmer here, in my most recent job hunt I rewrote my resume. I talked more about the value I provided for my company/customer (what the code did) rather than how I wrote the code. I'd say my results improved significantly in terms of getting my foot in the door.
When I look to hire a lawyer, I'm going to evaluate potential candidates by asking them to write statutes from memory in full on a whiteboard under time pressure, because I really want to see how well they think on their feet.<p>As a forward thinking technology executive, I am certain this strategy is correct, because Google does it.
I like the ontology of programmers. Are there any other efforts to do this? Anything data driven or externally validated?<p>Given what a big role personas and demo/psycho-graphics play in marketing, I'm surprised that I haven't heard more about then in the context of hiring and managing.
I apologize for being completely off-topic, but I find it completely ridiculous that we have to scroll past half the page to see the next highest-level comment.<p>Are there any plans for implementing collapsible comment threads?
For a blog article under a data subdomain, there's surprisingly few numbers and quantitative analysis, which is disappointing. And one matrix of survey data as the only visualization.<p>I get that startups do not want to reveal their competitive advantage, but there has to be some give-and-take. Taking an analysis on blind faith alone is frustrating.
>>Ruby or Javascript. (The C# pass rate is actually much lower than the Java pass rate, but the C# numbers are not yet significant by themselves.)<<
This makes me sad. I do not know who the caricature here is, the Enterprise programmer or the YC companies.
Keep all these spurious rejections and arbitrary hiring processes in mind next time pg or BigTech's lobbyists bemoan the desperate shortage of great programmers.
I think the recruiting problem in technology in general is that programmers don't meet and interact with enough different types of programmers in work settings, so they really don't know what they want or what they should be looking for.<p>I'd suggest everyone start doing this instead:<p>For anyone who meets basic criteria / filters, hire them at least part time, and put them to work. If they don't make the cut 3 months later (based on some objective criteria) let them go. Then after you've done all your recruiting like this maybe 1-2 years later evaluate what your team consists of.<p>It's not even a question in my mind. I'm not 99% sure. I'm 100% confident the team will not look like the recruiting team originally envisioned it would look before they started hiring.<p>EDIT: And I'm 100% confident those teams will consist of an overall better quality of engineer, and putting out higher quality products, than they would've ended up with if they stuck with trying to find what they believe to be the right "culture fit" or some concept of what a "prototypical engineer" is / should be.
The discussion about interviews comes down to a fact that if you setup your interview as a test where you position yourself as a chooser you will attract people who need you more than you need them. The best people are going to pass as they have enough options elsewhere.<p>For example I would be very happy to interview for Google 2 years ago and endure the process. Today I could sit with them for an hour and talk about stuff while sipping coffee. I have more opportunities than I can pursue anyway so spending hours solving mazes like some kind of a lab rat is not something I am going to spend my time on.<p>I am not a top expert in what I do but I can write and ship code. People like me (and especially ones better than me) will just not show up for your "process". Either you pursue them or you won't meet them. You will only get people for whom you are so attractive an option that they are willing to donate several hours or a whole day to have a low % shot. Those will not be the best programmers as the idea of paying to get evaluated is not something people with options entertain.
> The company told us they valued process more than raw ability, and he’d not written tests during the interview.<p>Who the hell writes tests during an interview?
I'd really like to know what it is about a resume or in an interview that makes someone seem like more of a 'product programmer'.<p>Is it the particular things they have worked on at other jobs, i.e. their experience, or is it the way they talk about what they want to build?<p>I'd be really interested to hear how I can make myself sound like more of a product programmer.
This is oddly reassuring. Imposter syndrome is practically guaranteed when there is a 99% chance that you only got (or could only get) a good job by a failure of the interview process, since so many companies proclaim to only hire the top 1%.<p>Goes to show that there is a far-greater-than-1% chance that you're legitimately in <i>somebody's</i> top 1%.
When it comes to hiring, I think that the team is more important than the individual. As the beginning of the piece stated, it's not so much about individual characteristics as it is culture/process fit. I think the main takeaway for applicants is not to take interview rejection personally. Even if you do everything "right", you still might not be what they're looking for.<p>Ultimately, I don't think that individual programming ability matters that much. It's extremely rare for the technical talents of one person to turn a company around. Some people can inspire others and turn dysfunctional groups into great teams, and there's never enough of that. But said great teams are ephemeral, like sports dynasties. All a company can do is try to avoid toxic employees and hope that the magic happens.
<i>"I remember the first time I interviewed for a front-end programming position and got asked how to do something in JavaScript on a white board. The specifics are vague, but it’s crystal clear how stupid it made me feel and how little it had to do with the actual job."</i><p>...<p>> <i>"The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size."</i><p><a href="https://signalvnoise.com/posts/3071-why-we-dont-hire-programmers-based-on-puzzles-api-quizzes-math-riddles-or-other-parlor-tricks" rel="nofollow">https://signalvnoise.com/posts/3071-why-we-dont-hire-program...</a>
So someone programming in Java at IBM or Oracle with a mostly academic experience would be all but completely unhireable by YC companies.<p>Incidentally, it was precisely that kind of programmers who built Watson at IBM, possibly the most impressive software of the past decade, which is not only both academically <i>and</i> technically challenging, but also brilliantly packaged and marketed and probably very lucrative to boot.<p>The exact same is true of the Graal team at Oracle, who have made what is probably the biggest breakthrough in compiler technology in the last decade, and might well power many important technologies in the near future (Ruby, server-side JS, R) as well as commercial Oracle products.
I couldn't help noticing that "Strong Junior Programmer" stands out from the rest. It as a category suffers from juxtaposition with "Child Prodigy Programmer". The "Strong" prefix sounds like a meaningless qualifier that only serves to blunt the label of "Junior". I have to guess that the reason companies select out of this category is that the candidates are perhaps not qualified.
I really liked how this analysis was done. Though i dont work in recruitment i have to work with CEOs often and notice similar approaches to hiring for other job roles as well. Way too similar in fact. I'd love to meet a recruitment company that is taking the same approach as TripleByte or perhaps TripleByte will aim to address other fields at some point. Either way its exciting.
Some companies reject people based on green card status, because they don't have the resources to sponsor them. Some companies have specific salary ranges, too. Are those factors considered during your analysis? You seem to be focusing primarily on what companies want, but sometimes what they want and what they can accept are two different things. Thoughts?
"The types of programmers that each company looks for often have little to do with what the company needs or does. Rather, they reflect company culture and the backgrounds of the founders. "<p>THIS. In other words, the good ol' boy system still exists, only under the moniker of "meritocracy."
Employees these days have a paradoxical role to fill. On one hand, the value in an employee is the ability to solve a problem. On the other hand, they have to market themselves as passionate people. The only way to do so is to exclaim interest in interesting subjects which often are directly tied to company concerns.<p>Say a company needs good testers. I would find it very hard to sell myself as a passionate tester. Instead I would say something that's actually interesting to me like AI or machine learning.<p>In order for something to be interesting, it has to be somewhat unknown. How can you be interested in something you have completely mastered and know everything about. It just becomes process for you.
This data needs to be related to the size and stage of the company. That YCombinator companies want UI people with application development experience reflects the YCombinator startup approach - it's all about the cool demo.<p>The requirements may be different in the later-stage companies. But most of the startups never get there. So looking at recruiting goals on a per-company basis from a VC pool will generate a bias towards the skills needed for the cool demo. What Lugg needs are people such as the article suggests. What Uber needs at its current scale is quite different. Uber will hire far more people than Lugg, but it's weighted the same in this model.
There is a subtle danger in only selecting candidate (people) whom think, act of look like you... the venture may end, not for a lack of talent or capability, but for a lack of ideas or questioning the normative, cultural convention.<p>Hire tortoises, hedgehogs and hares, introvert and extroverts, peacemakers and activists ... so long as there is respect, civility and productivity, because it's the "pulling" away from the center of gravity and institutional momentum that leads to exploring other great opportunities of venture successes that weren't originally founders' core product.
I am growing less and less negative about recruiters. They can be a bit pushy, but I think that recruiters can learn how to modulate and not exaggerate. If a company wants good candidates, it will have to go to them, because the good candidates will not come to them. Good candidates are usually already too busy to bother. That leaves the company no other choice than to appoint a recruiter and try to take the initiative in contacting the good candidates anyway.
So companies want people that can build and ship products, and hire for them based on their ability to solve tricky CS problems on a whiteboard?<p>Something seems amiss here.
Well if people really want product-motivated engineers, I'm going into the video games field. Also becoming a game designer.<p>I guess the ideal place for the technically motivated is in large companies like Google/Microsoft or in open source. Should we get a masters and find something more suited to our interests? I'm not sure what the answer actually is.
There is a good reason why startups prefer Product Programmers:<p>If you focus on product, then you have strong feedback loop in your design and development cycle: you regularly see how users interact with your product and improve.<p>If your focus is on technology, then there is almost no real feedback and you are likely to optimize something not very valuable.
"interest in ML as a negative signal"<p>Why, though? Is it because a candidate would be too interested in other things than focusing on mundane web development? Passionate about programming doesn't square well with passionate about ML, because the latter is not so much about programming?
> "Reading bios of founders and applying to companies where the CTO shares your background is probably an effective job-search strategy"<p>My takeaway is basically that in an organization of sufficient size founders should have little or nothing to do with technical hiring.
> We do our interviews without looking at resumes (in order to find great people who look bad on paper)<p>They don't ask for a resume, but the first thing that comes up in the interview is "where have you worked in the past, and what did you do there?"
It feels like this article contains a huge amount of bias. To sum it up in a sentence... of course 'founders' all want 'product programmers'!<p>Ask only tenured, hands-on CTOs, and I bet that table would come out much different.
I am <i>very</i> interested in which of these companies are the outliers. What is the one company in the table that expressed a dislike for product-focused programmers? Which are the few that like academic programmers?
It's what your data "show." Not what your data "shows." I am always skeptical of the analytical competence of someone who uses incorrect language to describe their own work.
> "To that end, we’ve spent the last two months doing detailed interviews with CTOs and lead recruiters at the top 25 Y Combinator companies."<p>How were you determining which YC companies were top?
I would be interested in A/B comparison of company responses according to immigration status (i.e. whether or not a candidate needs visa sponsorship). Any data available?
In this thread people stress how it is important to learn new things.
And then how showing interest in a new thing (Machine Learning) is a red flag.
Go figure.
Pattern here is you are better if you start up instead of looking for a job. I've seen many and many mediocre engineers becoming employers just to spare themselves rejection.
The biggest issue in interviewing is the bad inexperienced interviewers. It's very very hard to be a good effective interviewer and in my experience even otherwise very smart programmers are not great at this. Here's few things that would help to find better match and eliminate lot of false negatives:<p>1. Pre-filtering should be based on candidate's public contributions at places like GitHub, StackOverflow, Wikipedia etc. Pre-conceived notions of someone being C# or academic is just simply bad.<p>2. Don't ask question that you didn't had to solve doing your job. It's absolutely the hardest part for interviewer to come up with great questions that distill the problem they had solved on the job in to something that can be described in 10 mins and worked on in about hour. Most interviewers are unable to do this and fall back to puzzles stolen from websites or coworkers like them. If you are not smart enough to form the great question using actual problems in your job than you have no business being an interviewer.<p>3. Don't do 45 minute interviews. That time is too short. Candidate has already taken a day off, there is no reason why you should limit each interview to 45 mins and make candidate rush. Countless great candidate get eliminated because they fall short of 15 mins, do longer initial chit-chat or just take one wrong turn.<p>4. Strongly discourage interviewers to look for exactly the right answers. Again hard to do than said. Most candidates that sail through problem have very likely practiced similar problem to death. The bad interviewer than penalize candidates who hadn't practiced that same problem VS who luckily happened to do so. Ultimately you are required to eliminate 80%-99% of candidates and this comparison is so handy that interviewers fall for it despite of knowing it.<p>5. Good interviewers knows the trickiest part of the problem and are willing to help out candidate without penalties. Bad interviewers considers any hints or help as sign of weakness and are mentally subtracting points. If you are an interviewer who thinks that your job is to give problem, sit back and watch the show then you are wasting everyone's time. Good interviews are lively discussion, two way conversation and in fact a collaboration.<p>6. Bad interviewers ask seemingly easy problem but that has chance of making a big tricky mistake or omission. Interview turns in to sport spectacle to see if gladiator candidate was able to duck the fire from the dragon that was behind him. Good interviewers make sure problems are real problem and not a competition to set up traps that everyone easily falls in to.<p>It's kind of bizarre that most companies claim that they have huge headcount to fill and they don't find talent while their "rejects" have already been having great jobs at great companies with nice history of career growth and compensation raises. The real problem is bad inexperienced interviewers who have been molded to ask same puzzles they had been asked and have been trained to have expectation for candidate to magically arrive at correct answer with error free compact code under 30 minutes. It's utter nonsense. The result is that most candidates now keep working on puzzles for months which has little relevance to actual problems in job and not even a remote indicator of candidate's passion or ability to take initiative or collaborate or ship something in real production consistently.