Why we don't hire programmers based on puzzles and tricks

I remember when I was prepping for my Microsoft interview. There was so much lore online as to the types of questions they'd ask and how to prepare. I even found 'How Would You Move Mount Fuji?' in the library and started reading through it. It was an absolute waste of time as I, fortunately, learned weeks before the interview. As an example, one of the questions I was asked was 'how would you write the C function strcmp() without using any libraries?'<p>My gripe these days is that places I've interviewed with now that I have a few years of experience are still concerned about big-O notation and a lot of the more academic questions. I feel like I shouldn't have to go back to my college notes to prep for an interview for a job that I'd really be using my current experience for. I think it comes down to other developers not spending time really thinking about the quality of their interview questions. They just do a search for 'programming interview questions' and call it a day.<p>My most recent interview was with Amazon. I was asked the dumb big-O runtime stuff as well as how I'd count all of the stars in the universe. It pissed me off, but I did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form.

This is something to think about.<p>The interview process is so badly gamed that these days its kind of a ceremony. In a recent interview, I gave a candidate a practical problem. He was allowed to use the sed man page, and then I gave him a file and search/replace problem. Nothing much! The candidate fails! Instead he asks me if there is going to be a algorithm/data structure interview. I told him, I don't know but as far as I was concerned I was only going to test him on practical everyday problems.<p>It was almost like I had committed a sin. He was like, every company has those standard one's so why was I doing like a practical test. These days all candidates do is go through most common puzzles/algorithms/data structure stuff. Its actually quite easy to get access to those. Just spend some time on interview forums and you can practically game any interview, in any company any where around the world today.<p>My approach is straight simple. Throw a manual/documentation at the person. Give him a problem common enough to test a practical everyday situation. Check if he can pass the the test of reading the documentation and figuring out the solution to the problem. Or give him/her their favorite IDE/text editor + documentation + problem and let them solve it.<p>If they can do it, or even get the approach right they pass the test.<p>Else if all they are going to do is vomit some mathematical theory from college text books, which isn't even remotely relevant to our everyday work- I'm hardly interested in hiring such candidates.

Damn right. Even worse than whiteboarding random algorithms are logic puzzles.<p>I can <i>sort of</i> see how implementing my own hash table fits into a dev job, but I why manhole covers are round, or light bulbs in a room has absolutely <i>nothing</i> to do with anything relevant in our field.<p>People who use logic puzzles as proxies for <i>anything</i> in an interview have my everlasting scorn.

Interviewers ask puzzles and give you tricks because they want to see your problem solving process. Obviously, anyone who hires simply off a litmus test of such questions is not doing their job, but that doesn't preclude the use of such brain teasers/problems at all.<p>Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing. There isn't a canned response, you get to see them work under pressure, you experience live problem solving and you can understand how they move through their work better. All of these factors are just as important as competency when hiring.<p>A great programmer doesn't "fizzle" in front of a difficult interview question. He/she may or may not get the correct answer, but as long as they have the requisite problem solving skills and communicate with their interviewer well they will come out fine. I'd say that showing <i>how you fail</i> can be quite beneficial when trying to get a job, especially if you happen to fail gracefully :)<p>It seems like 37signals is just echoing platitudes to gain karma/publicity at the expense of cheapening the discussion.

I'm a self-taught programmer. These stories about programming puzzles in interviews always make me nervous to look for a job.<p>My first job wasn't a programming job, but I did a lot of programming because the need arose and I was the only one capable of doing it.<p>I always felt too inexperienced to look for a real programming job. A friend pushed me to interview for the position I'm in right now (full time web developer). They didn't make me do any whiteboarding or puzzles. What I did do is give them a code sample and we discussed how what I could have done better and why.<p>I got the job and I'm doing quite well, despite my lack of a CS background.<p>Since I don't have a formal CS background, and most of my development experience is web development, I have basically no experience dealing with complex algorithms, but I've found I can solve a hard problem with enough effort (Google, co-workers, etc).<p>For web developers, wouldn't more appropriate questions relate to "please design a database for a website that needs to track the following", or "What would your process be for designing and implementing a REST API that would...". These would allow the developer to show you their thought process of working through a large problem without there being a necessarily right or wrong answer.

Exactly. I never ask those questions. I ask practical problems that we solve every day here. Some are simplified of course.<p>Also I don't try to be confrontational, I take the "let's solve this together". That often puts them at ease and makes for a better experience and a more productive interview.<p>Now it is still an interview and the selection process could be brutal but this way I get the most _relevant_ knowledge and information about the applicant in the shortest amount of time.

Sigh. This is the issue that will just never die.<p>Let's just summarize the points:<p>- Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here);<p>- Side projects, even open source projects, can be of questionable "real world" value;<p>- I've personally encountered many "programmers" who can talk a good game but can't code a for loop;<p>- tests like the FizzBuzz test [1] are, in my experience, remarkably effective as an early <i>negative filter</i>. This is an important point. If someone blows you away at FizzBuzz, it doesn't mean they're an awesome engineer. But if they can't do it, it almost certainly means they <i>aren't</i>. The idea here is to spend the most time with candidates who might potentially work out and wasting as little time as possible on those that probably won't;<p>- the problem with these kinds of whiteboard coding problems is that the tendency is for interviewers to think the problem needs to be "hard". It doesn't. In making it too hard (IMHO) you risk destroying the value of your filter;<p>- pop-quizzes of obscure language features, the kind that might appear in certification exams, are a waste of time. I have no argument with that;<p>- whiteboarding code by itself is not a great filter. It should be used in conjunction with a multi-faceted interviewing approach that involves testing fundamentals, the ability to construct a relatively simple algorithm, the issues of working on a team and on a production code base and systems design.<p>- the problem with simply talking about "real world" code, as the author suggests, is you're no longer finding a good engineer, you're finding someone you like, someone who thinks like you. This falls under the umbrella of cultural fit, which is of course important, but don't mistake that for engineering skill.<p>- I think we can all agree that "logic" puzzles like "how would you move Mount Fuji?" or "if you shrunk to 1cm in size and dropped in a blender, what would you do?" are stupid.<p>- testing "back of the envelope" estimation however can be useful. I mean things like "how much storage is required to store satellite images for Google Maps?" The idea isn't to get an accurate answer. It's to see what assumptions the candidate states and, based ont hose assumptions, to come up with a reasonable ballpark number.<p>The problem here is that there are many engineers who can't comprehend the possibility that there is someone being paid to be a programmer who can't code. But I assure you this is the case. It's shocking but true. Simple coding tests largely filter these people out so if you're offended by such simple tests, just do it and move on. I assure you there's a reason why they exist.<p>One final prediction: There's some guy here on HN who always posts the exact same huge comment on any hiring thread. I'm sure it'll pop up any moment now.<p>EDIT: let me add a point about trial periods and take home assignments.<p>Both of these are guaranteed recipes for mediocrity. Truly outstanding candidates need to justify the time investment for either option and very few companies have the kind of gravitas that would justify it.<p>Anything written without supervision will be of questionable provenance at best.<p>As for whether or not someone will work out in your organization, bringing them in for a day is (IMHO) of questionable value. Many engineers are introverts. I include myself in this. It's <i>incredibly</i> awkward as is to be in a new company or even a new team in the same company. I question the value of any such assessment over what you learn in 1-4 hours of interviewing.<p>[1]: <a href="http://www.codinghorror.com/blog/2007/02/why-cant-programmers-program.html" rel="nofollow">http://www.codinghorror.com/blog/2007/02/why-cant-programmer...</a>

Eight minutes, 21 points, zero specifics.<p>I don't understand the 37signals love sometimes.

The question I always like to ask and reserve a good chunk of time for is along the lines of "What non-technical hobby or interest do you have?" and then "If you had all the technical resources you could dream of, how would you now bring something new to your hobby?" and once we've gone through that the real question is:<p>"Given that when you're duck hunting the key is to shoot ahead of where the duck is, and that a lot of the suggestions have been the next iteration applied to the hobby/pastime... what's the third or fourth iteration?".<p>Basically I want to see the excitement and passion about technology, their pastime, and how they think, how they identify problems, solve them in numerous ways, go beyond just today's solution. And I get wrapped up in it too, I also get excited by some of the stuff that comes up.<p>With a great candidate... the interview ends with us both inspired and a thousand new thoughts floating around the room.<p>The vast majority of candidates cannot leap beyond the first iteration, and some struggle to even see how anything within their hobby could ever change.

About a year ago I travelled to the south west (of UK) for a job - A full day of interviews, product ideas, I nailed the whole lot. Apart from the obligatory coding test. I have 15 years in this game and I have coded OSS or commercial code every day or week in Python, the language of the test, since 96.<p>It was a codility.com test - I had not actually been expecting it but, kerpow. I rewrote my own abs() because I forgot such a thing existed, the problem itself is still blanked from my brain - and every passing minute it got worse.<p>After an hour and a half, the interviewer took pity on me and drove me to the station.<p>I actually sat on the train took the codility demo test. 100%, 3 minutes. Signed up, took more. I could do them. Just impossible to know what went wrong but it went badly wrong.<p>Ultimately it worked out well. I would have had to live weekdays down there and my marriage probably would not have survived it. Now I work 15 minutes walk away and give my kids breakfast each day after we sit on the sofa and watch Mister Maker.<p>For me that test was a wake up call.<p>Firstly, I run my own business - being at the mercy of one boss is rubbish.<p>Secondly, these tests are rubbish for deciding who to hire - but they are OK for programmers as a form of continuing education.<p>Thirdly, the exercises at the end of each chapter of SICP are much much better form of education.<p>Forthly, I was once asked how a large corporate IT shop should handle a reduction in workforce. I said march everyone into a room and those who cant code FizzBuzz get a pink slip<p>Even given my experiences above - I still think that is the best option. Sometimes a guy just dies on that day. Its unlucky. But sometimes it works out for the best.<p>edit: clarity

I do tests for prospective candidates, but my approach is slightly different.<p>After the interview I email them a small unfinished program. I then give them a set of instructions on what I would like them to do to finish it off, and they can then do it at home on their own dev setup; with the internet and any other resource they would normally use for development.<p>I give a coding style guide, and in the areas where some specific technique is required I make that explicit.<p>The app has a few bugs in it, and some of the techniques asked for are slightly leftfield.<p>I instruct them that if at any point they have any questions then feel free to email me - and they won't be marked down for asking questions.<p>In fact I mark candidates up if they ask questions.<p>The task shouldn't take more than 30-45 mins for a decent coder.<p>I find this to be a really valuable approach to seeing how a coder works. It shows how they work with others' code, it shows they can follow instruction, it shows they can communicate when unsure rather than hiding away and it gives a good guide to their style of coding.<p>It's a far less confrontational approach, and gives the candidate the best chance of showing what they can do.<p>In the interview I tend to be much more interested in the person, because I'd rather have a coder who is slightly less capable that I can train, than a coder who is amazing at everything but won't fit in. I'll still discuss past projects and dig deeper where necessary, but I like to find out how they influenced the outcomes of projects rather than get too deep on algorithms and the like.<p>So far it's worked very well :)

I think in addition to the measures he says, if you're hiring for a web position, it's good to make sure they understand how the stuff they're using functions. If they don't have a good grasp of how http works / url parameters / that kind of thing, then they can have some nice looking code which seems to work, but has faulty assumptions that can be security and bug nightmares down the road.<p>Of course, you could always take smart people and train them - but seriously, who does that anymore?

The best interview I ever had, I was given a problem to solve with a system, a whiteboard and 20 minutes.<p>I sketched out a quick ERD, and explained the high level architecture of the system. No code was written -- it was assumed that I could implement said system if I could design it.<p>The worst job interview grilled me on low level algorithms (I'm not a CS grad) and wanted me to sketch out a quick sort. Of course, I can describe a quick sort now as a result of said interview, but since it was all C# I doubt we were going to have to implement our own sort algorithms.<p>Another terrible interviewer asked me about projects I did in college. I finished grad school 3 years ago, and have 8 years professional experience...

I've been on both sides of the hiring process - and most recently was going through the interview process again as I sought out my next opportunity. One thing that stood out to me was the way companies would approach the white boarding or coding challenges. While I agree with the articles issue with coding up a specific question (especially when it is yet another fizz-buzz - even if it has a twist to it) - the puzzle style interview can provide some kind of insight in to the thought process behind someones work. One company in particular really stood out by having a great white boarding exercise that not only was entertaining, but offered a decent amount of a challenge that it really made them a frontrunner in my final decision making process. A good coding interview can be beneficial not only to the hiring company, but can also give decent insight in to how the company works for the interviewee - and that is something that should be equally important if you are trying to attract talent.

At ITA Software, we found in-person quizzing to be a lot less useful an indicator than looking at puzzle (or other) code someone had written "offline".<p>We also found puzzles to be a useful talent magnet. If you're Google (or perhaps 37Signals) you don't need a talent magnet. If you're a small company nobody's heard of, one can be quite helpful.

This topic keeps coming around and it always makes me wonder why people have such a hard time hiring good programmers.<p>I say because the best places I've worked at already have a "system" in place for their front-end people. You can still be a marginal developer and do well if you follow the system. It takes a few months, but once you know how they write their code and the standards they use, it's pretty easy to write sustainable, reusable code.<p>This also makes it considerable easier to think and work outside the box. You can take a few of the better programmers, tell them about a new framework or approach you want to use on a project and let them loose. If it's a success, you merge these new ideas into the existing ecosystem. Simply repeat as necessary.<p>The industry really puts a lot of hard work into finding and hiring good candidates, but it's not the programmers who are important. It's the system you're plugging them into which matters.

When I came to Google for my onsite interview, in the group of people scheduled for that day there was this guy with an alpha-nerd attitude including a Rubik cube that he kept playing effortlessly -- he could completely scramble the thing and then solve it in seconds almost without looking at it. I was never able to solve more than two faces of Rubik in my life :) fortunately no stupid puzzles at all in the interviews (at least not in mine), I got in. The Rubik guy, never saw again. FWIW.

Key word here was "front-end". Algorithms aren't usually featured as often there. Before we break out the broad brushes, lets not paint all programming jobs as requiring the same skills.<p>[1] Eric Lippert summed this up quite nicely:<p>&#62;&#62; Does not having knowledge in Data Structures really affect one's career in programming?<p>&#62; Well it certainly will prevent you from getting a job on my team. But like I said before, programming is a huge field. There are lots of kinds of computer programming that don't require knowledge of data structures.<p>[1] <a href="http://programmers.stackexchange.com/a/102123" rel="nofollow">http://programmers.stackexchange.com/a/102123</a>

Over time I've made interview challenge problems progressively easier, as I've found that the trickier ones don't give any more information than the easier ones. My challenge now basically amounts to 'read this doubly nested for loop and tell me what it does', equivalent in difficulty to fizzbuzz.<p>If you can't solve that, even in the pressure of an interview, then I'm sorry, but you have no business working as a programmer, and I'm offended that you even applied.<p>And yes, people flunk it all the time.

The last interview I gave for a senior dev I tried making up some custom discussion questions to give after the basic tell me about yourself stuff. The questions boiled down to:<p>1. Junior developer code review where a save method included a call to a list method "because it's more efficient" than making a separate ajax call. Explain to the junior developer why that's not necessarily a good idea.<p>2. UI wants a tree view of categories where each category has a name and a parent category. The number of categories can be arbitrarily large. Discuss problems you might run into using a database agnositic approach (e.g. Hibernate) and what solutions you'd propose.<p>3. Prototyping a new app, two potential clients in the same domain have widely different ideas of how those domains should be represented (i.e. obviously similar domain class names, very different properties). Discuss ways of modeling domain classes such that each client sees the data in their preferred format.<p>All in all, I think the interview went pretty well because the discussion questions led to discussing all sorts of related topics that gave me a good feel for the types of problems the candidate had run into and if he had success in dealing with them.<p>It was great giving an interview without arcane things like explain the yield keyword in C# and how it might be used.

As Cletus wrote in his detailed comment, "this is the issue that will just never die." We discuss company hiring procedures here on Hacker News over and over and over, because most of us have applied for a job at least once in our life, and those of us who are building up a start-up have to think about whom we would hire.<p>The point in the submitted blog post by 37 Signals, "The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size," basically says that you should do a work-sample test to hire a programmer. And that's what research says. The more a hiring manager understands what a worker will do on the job, and the better the manager appreciates what a new hire may grow into doing after a few years on the job, the better the manager can devise a work-sample test to identify a good worker. There is a research base for this. Work-sample tests aren't perfect hiring procedures either--nothing is foolproof, and every hiring procedure goes wrong with both "false positives" and "false negatives"--but you can improve your odds of hiring a good worker by using a work-sample test routinely whenever you are hiring. As a job applicant, you can select better rather than worse bosses and co-workers by aiming most of your applications at companies that explicitly use work-sample tests as part of the hiring process.<p>With the help of several Hacker News participants, I have written a FAQ about company hiring procedures, revised over a few months of edits to fit the typical recurrent HN discussion of this issue. See<p><a href="http://news.ycombinator.com/item?id=5227923" rel="nofollow">http://news.ycombinator.com/item?id=5227923</a><p>for the latest posting of that, with full references to the research literature and legal cases about hiring workers in today's world. Feel free to contact me through my HN profile<p><a href="http://news.ycombinator.com/user?id=tokenadult" rel="nofollow">http://news.ycombinator.com/user?id=tokenadult</a><p>if you have suggestions for improving that FAQ before I post it to my personal website.<p>P.S. Even before I saw the prediction at the end of Cletus's comment, I planned to make the prediction untrue.<p>EDIT TO RESPOND TO FIRST REPLY ABOUT PUZZLE QUESTIONS: Yes, if a company in the United States insists on using puzzle questions as a hiring procedure, and justifies using those puzzle questions by saying that they want to see which applicants are "good problem-solvers," or "able to think on their feet," a rejected job applicant just might be able to subject the company to a very expensive lawsuit based on employment discrimination, unless the company has prepared beforehand a validation study showing that those puzzle questions have a bona-fide relationship to work requirements. I would not advise a company to take that risk, especially when the legally safer alternative of doing a straight-up work-sample test is available. The law is different in other countries, and as a reply in this thread points out, in the EU it is generally legal to use straight-up IQ-type tests in hiring processes, although those are underused by private companies in Europe, according to the sources in my FAQ post.<p>ANOTHER EDIT, TO LINK TO AN UNBELIEVABLE ANECDOTE ABOUT HIRING:<p>In August 2012, I heard a story from a hiring manager of programmers about the hiring procedure he uses as an initial screen for applicants who have degrees in computer science: "Write a loop that displays the numbers 1 to 100." That sounds awfully easy, even to me, but he says that the great majority of his applicants with accredited CS degrees fail that screening test. My earlier telling of the full anecdote<p><a href="http://news.ycombinator.com/item?id=4603414" rel="nofollow">http://news.ycombinator.com/item?id=4603414</a><p>and his<p><a href="http://news.ycombinator.com/item?id=4919749" rel="nofollow">http://news.ycombinator.com/item?id=4919749</a><p>seem pretty nearly unbelievable in what they imply about how clueless many CS grads are, and yet I think the anecdote is a true description of reality. Cletus too mentions in this thread, with considerable agreement from reply comments, that many people hired as programmers cannot actually program.

The best questions to ask are the ones that pertain to actual work.<p>Think of a problem you had before and solved or are attempting to solve. Ask the person being interviewed how they'd handle the problem. Do they have good process, intuition, and problem solving skills? Are they clear about communicating their intentions and explaining their decisions?<p>This isn't a trick - it's what the person is going to be doing there, in and out, every day.

I consider myself a proficient developer, however, in situations where I'm required to solve an obscure puzzle while someone is staring at me, generally makes me uncomfortable...no matter how easy the question. I interviewed for an internship position at Microsoft, where I was required to write some bizarre code on graph paper while the interviewer literally stood over my shoulder. Horribly uncomfortable, I barely made any progress, and was subsequently denied a position (obviously). After the interview, I went home and solved the question in a matter of minutes, wrote the code, optimized, and fully tested it. While I choked during the interview, I saw fellow classmates fly off to Redmond...classmates that I <i>knew</i> weren't better developers than I was, simply because I had worked with many of them on projects several times. That's when I knew this whole puzzle interview stuff was nonsense.

As someone that just went through a whole slew of interviews I would say I agree with this to some extent. I'm a frontend developer and while CS principals are important, asking questions that are not practical to everyday development, are counter-productive and do not give you a view into how that person tackles a real problem.<p>Things like "write a function that performs merge sort", are bullshit because you would never do this in real life, largely because languages have sort methods that are already extremely performant. Plus a lot of languages will already be using some variant of merge sort and exposing it as a "sort" method.<p>If you are given a problem like "create a tabbed news component" or you're asked to do a small project (nothing longer than an hour), I think you can still get a good understanding of how people solve problems. This is less stressful, less "gotcha" mentality, and a fair approach.

Shameless plug: I've been working on this : <a href="http://initialround.com" rel="nofollow">http://initialround.com</a> to allow applicants to complete an interview at home and in a browser, to eliminate the pressure of the "quizzing cage". I'd be interested to hear people's take on this approach.

<i>[I] got asked how to do something in JavaScript on a white board. The specifics are vague, but it’s crystal clear how stupid it made me feel and how little it had to do with the actual job.</i><p>So did I. (Well, it was C then, but same idea.) The prospect of scrawling code on an inapplicable medium while people stared at every move is, of course, irritating.<p>Instead of going to the whiteboard, I pulled out my notebook computer, fired up a code editor, and told the several guys present to continue the interview _while_ I worked on the challenge. Time was saved by multitasking, I did the work in a sensible normal manner, everyone was comfortable with the situation, and they were happy with the resulting code.<p>I got the job. And turned it down.

"Clever architecture increases possibility. Clever code increases complexity." - @edw519 the other day - <a href="https://twitter.com/edw519/status/301393887174483968" rel="nofollow">https://twitter.com/edw519/status/301393887174483968</a><p>Testing with excessively clever tests and puzzles will attract a mindset of creating more complex code and than needed. Architecture a problem in a way that solve the problem elegantly but remains open and flexible is far more a tougher challenge than an algorithmic challenge.<p>There's a fine line between testing a hire to make sure someone's behind the steering wheel vs. communicating that you want things solved in interesting ways.

I've been interviewing pretty much non-stop since last April. I feel like I've become somewhat of an expert when it comes to interviewing processes.<p>First off, as I've become a better programmer, I feel this fact is best shown in my opinions. When I first started out, if you had asked me my opinion of PHP vs Python, I would not have been able to say anything coherent. Now that I've worked with both technologies, my opinion is much more coherent.<p>Instead of asking puzzle questions, ask candidates their opinion. "What is your opinion of Mongodb?" I've spent enough time in the trenches with Mongo to have quite an opinion of that technology. Same with Postgres, Django, Javascript, Coffeescript, etc.<p>I think hands down, the worse type of interview questions are the ones where you're told to solve a problem, but <i>you can only solve the problem in a certain way</i>.<p>Its become a interview idiom for me. One recently was to reverse the words in a string. In comes "This is a string", out comes "string a is This". No problem. In python this can be done with one line:<p>the_string.split()[::-1]<p>It takes me 5 seconds to write that out. The interviewer telle me "good job", then he asks me to do it again, but this time to do it without using any standard library functions such as split or reverse.<p>At this point, I can almost with 100% certainty that this company will be a terrible place to work. It wouldn't piss me off it it wasn't for the fact that like 90% of all interviews I do end up like this.<p>I've through about this a lot, and what I've come up with for an explanation as to why I can;t do these types of questions is because my algorithm writing process is very subconscious. When I'm writing code and I come to a problem that requires a lot of thinking, I usually stop, do something else and let the problem float around for a bit in my subconscious. I got my best ideas when I'm in the shower, on my bike riding around time, even while reading a news article. When people are watching me (especially strange people I don't even know) I don't do my best thinking. At this point I'd do anything to be able to think in front of people. Because its keeping me from being able to get jobs.

To change it up a little bit, has anyone here been asked simple debugging questions? I've interviewed at numerous places and I've never been asked the question why a certain piece of code is breaking or how to fix it. The questions could be designed to be self-contained and just complex enough to get a feel for how the candidate thinks. My consulting job usually takes me to places where a project is in trouble or failing. It would seem that quickly understanding broken foreign code would be a huge asset. The closest question I've been asked is: "Here's a Singleton implementation in Java. What can go wrong?"

Basically, most of the questions I see asked in interviews are contained in Levitin's algorithms book. The greatest potential for someone to excel at those trick interviews is to be fresh out of college.<p>I walked out of my first interviews, because 'I do not like programming games.' The first was a 4d matrix puzzle with more mathematical/theory roots than programming/logic.<p>My third interview started with the above quote and I was hired on the spot -- I believe they took it as a rebellion of every fresh out of college kid wanting to program video games.

I have a long-winded blog post about this very subject awaiting some polish.<p>I've done my fair share of interviews. I still don't like them. I've narrowed it down to basically two things:<p>1. There is a lot of highly subjective, mystical advice about what "works," in a technical interview and what doesn't. Big problems, small problems, trivia, puzzles, white-boards, no white-boards... hardly any of it is thoroughly studied and built on solid theory. It's all conjecture and personal bias.<p>2. How the candidate is feeling and where their head is at on the day of the interview <i>matters</i>. I can go on about cache alignments, bus errors, segmentation faults, C++'s lack of order in evaluating function arguments, why template class declarations must exist in the same compilation unit as the definition, why CLOS is amazing and the computation model of recursion is brilliant (or at least why I think so). Yet on a bad day I can (and have) botched perfectly trivial, benign problems.<p>However at the end of the day you need some sort of process. I just think that the current practices are not good enough. It seems to me that companies are probably turning away perfectly fine candidates without realizing it. There's pros and cons to every approach I suppose. But I still think trivial problems are dumb.

I agree wholeheartedly here. I know many developers (myself included) who would fail a FizzBuzz test or any instance where they're asked to write code on a whiteboard and yet can produce some really clean, efficient and above all well-written code. How about instead of asking people to write on a whiteboard or do Computer Science 101 tests, how about you give them a computer, a problem and ask them to solve it?<p>Don't forget it's not always how smart they are, it's their work ethic and whether or not they'll be a good fit for your team. Personalities that meld well with already hired employees are an essential must, no point hiring an introvert for a position when the team are extremely social people who have Friday afternoon beers.<p>Here is a tip: hiring a developer? Ask them to write a blog application, a task management system or a real world problem they will encounter if hired by your company not some ridiculous test that could stop you hiring a great developer. I consider myself to be pretty good, but I don't have a CS degree, I suck at mathematics and can only do simple math, yet I am able to produce exceptionally great code especially in instances where deadlines are tight and sometimes unreasonable.

This is a complicated issue, but in my opinion the google/facebook style of interview is the one that gives you less false positives, even though it gives tons of false negatives. In other words, very few people who can do it are bad coders (false positives), but many good coders can't do it (false negatives).<p>So, as companies like Google and Facebook have so many applications, they're much more concerned about false positives than false negatives.

Just my 2 cents on this issue - writing code on a whiteboard isn't always about the code you write.<p>It's not about the actual code you produce in the interview, it's about talking through your thought process, your ability to reason while under pressure, and your attitude of whether or not "this is beneath me - wah!".<p>I've been in several interviews where I had to write code. I made some mistakes, some things I plain out didn't know (for my first job, I had no clue how to write any JavaScript, and I explained that it was just something I didn't know how to do, but that I'd be willing to learn it as required). I still got hired from all of them. So, if you think coding tests during interviews are about producing flawless code, they're not. They're to see how you think - and that is profoundly important as a developer.<p>I would also argue that a developer's existing skills are the least important factor when deciding whether or not to hire someone. Every developer works as part of a team and has to collaborate and communicate with others in that team. If you want to hire a developer, you need to see how well they communicate complex technical problems to non-technical people.

I used to give interview candidates a little background lecture on our problem domain then ask them for input on whatever I was working on that week. E.g. If you were trying to parallelize this code, how would you structure it? I never asked fizzbuzz style coding questions, because I think they're stupid and I probably couldn't code a linked list on a white board (I rarely code on whiteboards you see).

Most eye-opening thing about programmer interviewing I've read: <a href="https://sites.google.com/site/steveyegge2/five-essential-phone-screen-questions" rel="nofollow">https://sites.google.com/site/steveyegge2/five-essential-pho...</a><p>I probe for weakness in 5-6 different areas of computer science in the phone screen - keeps me from bringing in weak candidates.<p>Then in the on-site, I give them my Macbook and ask them to solve a few simple problems with some flat files I conjured in a language of their choice that's installed on that machine (C/C++/Java/Python/Ruby/PHP/Perl/shell/etc). You get an idea of how they solve problems, think about solutions, fix bugs, avoid common errors, and what they look up on the web (I let them use a browser to lookup apis and so forth - it's a good way to weed out "cut and paste" programmers). The people who do well finish quickly and we chat about the company with the remaining time - the people who do poorly use almost all the time on the first problem, make a bunch of mistakes, and usually don't understand the proper data structure for the job.

Employer told me "We do hardcore stuff!" come on really hardcore?! I would really love to post their apps here but I'm not that kind of person.

If I can have a word on FizzBuzz?<p>There are a few programmers that have a proclivity towards "clean, perfect" solutions, but FizzBuzz is ugly.<p>Making FizzBuzz work at all is just stupidly simple; but the moment you ask yourself about the little details, you notice that you either have to make an if tree, a bunch of if-elseif's, or some other hinky thing that ruins the "purity" of this simple little program.<p>Alot of people take the perspective that if you go<p><pre><code> if(i % 15) print "fizzbuzz"; else if(i % 5) print "fizz"; //or was it buzz? else if(i % 3) print "buzz"; //or was it fizz? else print i; </code></pre> then you've found an optimal solution and you can wash your hands of it, until you think to yourself: "but an if tree would use less comparisons! Should I be readable? Or take the efficient route? But if I'm thinking about efficiency. wouldn't a lookup be faster and simpler?"<p>I suspect when presenting a problem like this, the interviewer filters out two kinds of people: the people who "can't code", and the people who are obsessive.

I was invited to a company for a game development position. One part of the interview consisted of a fairly challenging mathematical/algorithmic problem. It wasn't impossible, but it's certainly not something I can do in ten minutes at a white board with some people taking notes about my every movement.<p>The other part was being shown intentionally messy and broken code. Not code that'd fail to compile (although it sure looked like it would), but something thrown together in a completely insane manner that I couldn't possibly make sense of in 15 minutes. Much of it boiled down to, "What would happen if you call this function and use this variable before you actually define them and then do x, y and z?", and the way this language did it was quite different from other languages. I mean, yes, I did learn a lot about the language from that interview, but I'd really hope that's not the kind of code they'd be throwing at me or expecting me to write at their company.

When I interview others, I use a couple of basic whiteboard show-me pseudocode questions (how would you reverse a string, how does a linked list work, give me an example of a callback). It seems to me that if they know their stuff, they can answer those. If someone can't get through any of them, or at least have a valient stab at it, chances are they don't have the background I want.<p>By the same token, I've been interviewed myself where I've been given weird shit questions like others have mentioned; not the blender one, but shit along that line. That's nothing I personally do without some serious contemplation, not to be achieved in 5 minutes in an interview. Anyone who does so likely knew the answer ahead of time or is extremely puzzle-oriented. I think the real test is to examine the puzzle-solving processes of the candidate, but as a rule I don't think much of those kinds of questions.

Ok... I was under the impression that 37Signals hired mainly remote workers. Could this account for the lack of a white board?<p>Personally, I use a whiteboard when hiring. It's not the deciding factor... but it's a part of the bigger picture. As part of a small start up I need to know right then and there if they know what I need them to know.

Coming up with good questions to ask in an interview is an effort that the hiring team needs to invest in. This involves a top down reflection of the job along with a vision for the job. It requires to ask him/her self before hand "What sort of tasks is this hire going to be doing" , " What language skills does he need" , " Does he need to reinvent a sorting algorithm to get this job " .<p>Well this rarely happens , to start of folks (generally) tasked with the hiring process are not the right "vision" folks. They are highly task oriented individuals aka MS Project specialists.<p>A real test would have to encompass among many things. - Do they are ask the right questions ? - Are they capable of picking up new languages, ideas , technologies while defending or questioning changes in an articulate yet respectable demeanor. - how they handle things when they get to a wall.

That's what I was discussing with my friend a few days ago. He had an interview with Google last month and it took him a month or two to prepare for that. he did not pass the interview, but he got an interview with Amazon this month. He was busy with other things and he could not study for the interview. He mentioned that he cannot remember most of the things he prepared and read for the last interview. I am forgetful too, and I guess 90% of people forget details about algorithms and how to answer a puzzle very fast if they don't use it everyday.<p>So the question is what is the value of a skill or knowledge that fades away after a month or two? I know people who got hired by Google or other big companies that forgot all the information they acquired before the interview. Cause most of it is useless for the actual job. Many logic questions or puzzles are well known and people tend to know the answer before the interview.<p>If I had to interview someone; I would assign him a project from the company that is part the of the job he is going to do, leave him alone in a room with a computer for a time period I expect someone to finish that specific project. Finally, I come back to see the out come.<p>It is good for the applicant, as there is no pressure on him (except some time constraints maybe which is not a big deal), I do not put him on the spot and he does not have to deal with his boss while answering to a question. He does not have to dig unrelated information before the interview, and he can show off his programming skills if he has any.<p>It is good for the company, as I do not have to spend a day interviewing and prepare for that, I will know right away from the code if the guy is a good fit or not, I can see his programming skills, I know ahead of time how long in average it takes for an internal person to do the job, so I can measure his performance relatively. I do not even have to evaluate the code in front of him, I can invite multiple candidates at the same time, put them in separate rooms. Get the results, evaluate them and invite them for the second round if I liked their code.

Well the good news is this problem should be solved soon since we spend so much time focused on it.<p>I don't care for puzzles. Just conversations. I skim resumes looking for glimpses into personality more than looking for skill sets. I have a guy that went to Yale on my team. I didn't even know that until he mentioned it the other day, yet it was on his resume. I missed it because I don't care where you went. I don't care how fast you can solve a puzzle. It's put up or shut up and show me what you've built when you come here. If you can do that in an interview there's a great chance I'll hire you. The last two hires whipped out apps they'd built and then we went over how they accomplished it. To me that's more valuable than any other process, having a portfolio or sample ready when you show up.

I was interviewing with Microsoft last semester for a SE internship. At the end of the interview with a senior engineer, we were just chatting about various topics. This is one topic that we talked about. His response was something along these lines:<p>"Of course we do not write code on whiteboard everyday. Of course we do not write code to detect if a graph is acyclic (my interview question) on a daily basis. Of course we do not put our engineers on the spot with these types of questions. However, when the time comes, may it be only once in 10 years, we know that we have hired the right people because these questions show your thinking abilities - not if you know the answer or not."<p>Btw, I didn't get an offer, but I'm going to Amazon where I had very similar interview process as Microsoft.

Ok, this is true story and has nothing to do with my credibility. Few days ago glenwood systems, a core product developing company conducted a recruitement drive in my college. There was me, a coding geek with all my passion for one thing, and there was him, who is a puzzle freak. They selected him past the first two rounds full of puzzles and math, finally rejected him knowing his programming incompetence. And they missed someone who really could help. me. There are a ton of books out there teaching you how to crack interview puzzles and programming riddles. There's no book telling you to get your hands dirty with projects and code. The author is right. Hire the one who did things, not the one who learnt to crack puzzles.

Not employed people for years, but I never put much value on ultimate skills. I used to think long term. So, give me the the right human being, with a reasonable level of knowledge, and very quickly they will be more than good enough. A little more time and for me, they become pretty much perfect.<p>I want the right personality. The ultimate skills will follow. So, I just chat to candidates and employ those who I feel I and my people can work with. I never left it so late that we needed instant skills. To me that would be poor management.<p>Prior to that, I have tested and done many of the things people cite here, but most of the time I got awful employees who turned out to be good at doing interviews.<p>Yeah, a bit hippy dippy, but it worked for me.

I won't present a résumé, a github link, a linked in profile, or do nonsense tests. I always send a full web-application I built for code review, do a technical demonstration and provide sample work on their codebase. Any other route is a waste of time.

Programming is about problem solving. What you want in a programmer is someone who knows how to approach solving a problem in a structured repeatable manner.<p>What getting someone to solve small problems like FizzBuzz in the interview gives you an idea of how then approach the problem. Can they decompose the problem into smaller chunks ? Can they then refine their solution into a better solution ? Do they know when to stop refining before the solution becomes difficult to understand ? These are the skills you want to asses from a potential programmer.<p>Programming languages, platforms, frameworks come and go but if you don't know how to approach solving a problem then that knowledge is ultimately worthless.

Some of the best hiring strategy I've came across is indeed "test driving". Take a challenging problem your team has, slim it down and ask candidate to do some quick prototype during a weekend. No interviews, puzzles or whiteboarding :). If you were going to fly down a candidate on-site for a day of interviews instead, it's going to take same time anyway considering logistical effort on candidate's part.<p>Test-driving, however, is unfortunately neither scalable nor "leak proof". As soon as one candidate gives away your test drive question, the next candidate could easily cheat away inflating their apparent awesomeness compared to other better candidates.<p>However it would be incorrect to write down all whiteboarding interviews as "evil". Like any other interviewing techniques, it really depends on how you do it. A good whiteboarding question that allows candidate to use CS fundamentals to solve fairly non-trivial problem that you also needed to solve for your current work is a great question. There are 3 major reason why this doesn't work as expected at some large companies:<p>1. Interviewers can't think of good whiteboarding question and fall back to commonly asked popular puzzles. These interviewers are also often the ones who have one "favorite" question that they would ask everyone. This is absolutely #1 problem why whiteboarding interviews devolves in to secret puzzle marathon. The best questions that interviewer could ask are actually the ones that they needed to solve themselves as part of their work recently. This keeps questions relevant and refreshed regularly. It also allows to compare candidate with themselves and follow the philosophy of hiring people smarter than you. It's however hard because interviewers themselves needs to continue doing something interesting regularly.<p>2. Interviewers provide feedback to each other during the loop. It's not coincidence that if 1st interviewer gives "hire", all rest tend to do same at companies where there is no clear policy of not communicating feedback before end of the interview.<p>3. Interviewers don't follow up candidate response by extending question, building on to next level perhaps open ended scenario, asking auxiliary details such as test cases, complexity etc.

Eh, being able to understand stacks and queues, big-O, etc and problem solving are important things, but otherwise agree that software construction skills are important as well as understanding tradeoffs bottlenecks, networking etc.

Why do we always feel the need to upvote posts like these as if we don't discuss them every 2 weeks?<p>Next up in the topic cycle: Discrimination in tech. Why working more than 40 hours a week is killing your productivity.

Aaand HN has just written a 10-hour book on the subject of hiring.

At my old company we'd tell interviewees to bring a laptop loaded with there usual dev tools. After a couple of typical interview questions they were then instructed to create a small program using whatever language/tools they'd like. They had "unlimited" time and sat at an empty cubicle. Candidates also has had full access to the Internet. It was amazing how several people could not even complete the simplest program without needing extensive assistance.

I wish this were the case more. I graduated with and electrical engineering degree, but over the years I've spent the bulk of my time coding up DB, big data stats, ML, etc. to help me do my job. Trying to explain I'm more than capable while I falter with simple puzzle based question is an impossible gauntlet that has kept me from even considering a career shift. (Granted I'm a much better analog designer than I am a java code.)

I think the skills you need to throw together a nice RoR app are different from other positions.<p>I wouldn't want on critical infrastructure code alongside someone who can't reason through the performance implications of their code on the fly. You can waste a hell of a lot of time diagnosing and fixing performance problems because someone built an entire module around an inappropriate data structure.

It's one thing to ask people to right syntactically correct code. It's another thing to ask them to write pseudocode in whatever language they please and understanding their process.<p>At the end of the day, as Hal Abelson said, computer science is not about computers, or science. It's about being able to formalize process.

Kent Beck tweeted a very fitting comment: 'Programming contest problems shouldn't be algorithms, they should be like "set up continuous deployment of a multi-region Django app on AWS"'<p><a href="https://twitter.com/KentBeck/status/299528659746840576" rel="nofollow">https://twitter.com/KentBeck/status/299528659746840576</a>

Comments are closed on the article and I wanted to thank David.<p>I'm one of those who freeze up like a deer in headlights when asked to do a quiz. But have a degree in CS, have made several enterprise level packages by myself, and have been programming for 31 years -- so am fairly confident that I can program.

Really liked the text at the bottom:<p>(If you need help posting a comment, feel free to use any of these samples: “You make todo lists, you don’t need real software engineers”, “Math is actually really important, you know!”, “Google is worth one gajillion dollars and they use quizzes, so there!”)

All great and well spending 20-40 hours trying someone out to see if they are a good fit, but this is only practical once you are down to a very short list of final candidates.<p>Nothing is said about how you actually get to this final list and that of course is where the real challenge lies.

Because no one will hire a writer based on his knowledge of long words.)<p>Excellence in the grammar is also irrelevant.

On Coding Portfolios: <a href="http://programmers.stackexchange.com/questions/102490/requiring-programming-portfolios-in-interviews-recruiting" rel="nofollow">http://programmers.stackexchange.com/questions/102490/requir...</a>

is there any well known data/studies on selection methods for programmers?<p>I can understand the reasons why not to use algo problems as a filter, but I would have thought simpler forms of intelligence testing would still be quite useful.

What is this continuous reference to FizzBuzz from American developers?<p>Being born in the other side of the Atlantic if it wasn't for HN, I would never heard of FizzBuzz in my 14 years of software development.

Agreed - but I guess if you're 37signals you just attract the right people and you don't have to worry at all. But if you're not then maybe some of these riddles are just good heuristics.

I like this: <a href="http://cdnnew.theinspiration.com/wp-content/uploads/tattoo_QR.jpg" rel="nofollow">http://cdnnew.theinspiration.com/wp-content/uploads/tattoo_Q...</a>

It's too easy to prepare for the common ones. I have a list of them which I share with my students before they graduate. Sometimes we work through them for fun.

Couldn't agree more. Every time I've agreed to a "programming test" I find myself wonder why I agreed to it in the first place.

Congratulations to 37signals for adopting a hiring practice that most companies have been using for more than a decade.

These puzzles and IQ tests are as pervasive as they are irrelevant. Nowadays, you even need to pass one to get some government jobs. Here's the one to pass to get a job in the EU agencies:<p><a href="https://europa.eu/epso/application/passport/quiz.cfm?lang=EN&#38;comp_id=1&#38;quizid=10&#38;f_sub=+OK" rel="nofollow">https://europa.eu/epso/application/passport/quiz.cfm?lang=EN...</a>

One of my career goals is to get a programming job without having to show that I can add a leaf to a binary tree through recursion (again).<p>I don't mean this as flippantly as it might sound. I used to think it was kind of fun and exciting to be at the whiteboard, thinking on my feat, dealing with various curveball data structures and algorithms questions. I certainly always read up on this stuff prior to interviews, and from what I've heard, I'd be advised to do it again if I get an interview at google or something.<p>But the last time I did this left me depressed. I realized that I've done so much of my work in obscurity that people are still asking me about binary trees. I want to be very clear that <i>I don't blame them</i>! I also started to realize that I have a lot more to offer than my ability to traverse various tree structures, and that the interviewers seem unaware of this. But it's up to me to show them, not on them to just take me at my word.<p>The last time I went through a massive, all day technical interview, I didn't get the job, partly because I didn't review my data structures and algorithms book (again), but probably also because I came off as cranky and irritated. I was busy that week, probably should have put it off.<p>Just earlier that week, I had taken several clients from pharma and semiconductor companies through some beta testing of our software, refining the model, getting feedback on use, optimizing the code base so it could give answers more quickly. The company I was interviewing for created supply planning, forecasting, and inventory management software for various large businesses. As a math major with an MS in Industrial Engineering and a background writing software, it was right up my alley.<p>My interviewers showed no interest in anything other than my ability to code or a couple branches of math. In fact, some of them seemed so inexperienced that I'm not sure they were aware that this kind of experience even exists to be asked about (Oh, you work with clients? Our project managers do that). At lunch, one guy asked me how to swap two integers without creating a third integer. This is after a full morning of technical grilling on math and programming, with a lengthy afternoon to follow.<p>Fortunately, I'm now in a job where I am encouraged, not just allowed, to contribute to open source, do presentations at conferences, try to build communities, and so forth. I would not consider any job that did not have this element (and now that I have a job like this, I'm not looking anyway). But if I were to look around, I would vastly prefer to be hired for my known and widely used code base than for my ability to detect cycles in linked lists, implement a hashing function, or print out all possible permutations of a string (with no duplicates!).<p>I'm not there yet, but that's my career goal.

Thats good. So we don't need anonymous to crack your system.. :)

I interviewed at several companies about a year ago, and one had a practice that really stood out. I was impressed by how organized it was, how well it seemed to be a good test of a job candidate, and how it gave me many opportunities to show my qualifications. It was a whole day, but they flew me out and put me up in a nice hotel nearby.<p>The day was broken up into sessions of about 1 hour each (maybe a bit more). In each session it was me and 3-4 other people, but the people rotated, so I got to meet lots of engineers, a PM or two, the hiring manager, etc., and they all got to meet me. A few people showed up more than once.<p>The first session was basically an introduction. I think it was shorter than the others. No quiz-style questions, but stuff about my background, etc., and opportunities for me to ask questions also.<p>Next I did a presentation at the whiteboard describing some project I'd worked on. They had told me this would be a part of the day, so I could have prepared (though I didn't). I talked about my recent startup, describing some of the data model and machine learning tricks I'd developed. They asked lots of questions, some to clarify, others about my reasoning, how it would scale, if I'd considered technology X, etc.<p>In the next session was me at the whiteboard again, but here they described a new feature they'd like to add to their site, and my job was to sketch out a rough solution. So partially this was about my ability to ask clarification questions, and also about my big-picture design skills. Again they asked lots of questions, and once we even got into a specific SQL query (where I opted to use an EXISTS with a correlated sub-query). So this session seemed a lot like the first, but more impromptu on my part and probably more practiced on theirs.<p>Then they took the whole team out to lunch at a nice Thai restaurant (~15 devs plus the director of engineering and the PM). This part was relaxing and fun. Of course this was still part of the interview, and wouldn't be relaxing for everyone, but still it was a good way to feel out personality fit for both sides. Also there were too many people for me to be the focus of attention, so that was a nice break.<p>Finally there was one afternoon session where they gave me a laptop with a toy Rails app, and had me add a feature (change a relationship from one-to-one to one-to-many, or something like that). There were two devs watching me code, and I had to do the whole MVC thing: write a migration, tweak the controller, edit the view. I got to talk through the changes as I did them, so they knew my reasoning. It made me smile to slip in a few fancy vim moves; I don't know if they noticed. But it made me realize this was also a test about some basic mechanics, too. They let me choose my editor/IDE, so it was a chance for them to watch whether I knew my tools.<p>Then there was a final session for follow-up questions etc. This was fairly short I believe.<p>All along the way I had lots of opportunities to ask questions of my own, which I appreciated.<p>There were no puzzles, no trivia quizzes. Personally I'd add a FizzBuzz test to the initial phone screen (like Han Solo "They hardly asked me any questions."), but otherwise I plan to completely rip off this template the next time I hire someone myself. Maybe it can help some of you!

interview != quiz

&#62;I remember the first time I interviewed for a front-end programming position and got asked how to do something in JavaScript on a white board<p>&#62;how little it had to do with the actual job.<p>That doesn't make any sense. How is solving a problem with javascript irrelevant to the job of a front end developer? That's what they do.

I'd never work for a place that gave me a fucking puzzle to solve. I'd probably smack the interviewer upside the head for wasting my time.<p>The best way to do it is pay the person per task as an independent contractor until you're sure a full time position is what you're both looking for. If they're helpful to you then you keep calling them and if not everybody keeps their dignity, no big deal. Don't humiliate potential employees right off the bat with stupid games.

I'm going to avoid the cultural flamewar around whether companies <i>should</i> do this.<p>I don't mind it. I've interviewed for enough jobs and had enough acceptances and rejections to learn something: it's not a big deal. If you don't know an API function, you don't know it. The likelihood that you won't get the job because you didn't know that one question is unlikely.<p>In fact, part of the evaluation is whether you can handle <i>not</i> knowing the answer with grace. The fail-outs are the ones who use their cool or seem to think the question is stupid, not the ones who get it wrong.<p>Once you realize that you don't need to get all of the answers to pass an interview, it gets easier.

No need to read. The author of this post has clearly missed the point.

I hope someone from Goldman Sucks reads this.

I have heard google is a big violator of this?