Why 37signals Doesn't Hire Programmers Based on Brainteasers

I helped start the practice at ITA Software (acquired by Google last year) of using puzzles as part of the hiring process. We ultimately even put our puzzles in the Boston subway as recruiting ads. My assessment, after more than a decade of doing it:<p>- Good puzzles are actually a talent attractor; many smart people found out about our company via our puzzles.<p>- Good puzzles are ones that scale well; i.e., where the basic problem is pretty easy and can be solved by a decent programmer in a few hours, but with harder variants that can take much longer.<p>- "Take-home" puzzles (as opposed to in-person whiteboard tests) weed people out who don't really like to program and/or can't finish things; this is a useful filter. If someone complains that "they shouldn't have to spend three hours writing code to get an interview," that itself is a pretty good counter-indicator. (Yes, we are all busy. But you're talking about starting a relationship with the company that may last 10+ years. You can do three hours of prep work for your interview. And you have to code a fun puzzle in the language of your choice, not some subroutine in a 30-year-old COBOL banking system.)<p>- It seems that in-person whiteboard or locked-in-a-room tests are pretty poor indicators of success. Many good programmers put in this situation significantly underperform their true abilities.<p>- It's not a great idea to evaluate someone <i>purely</i> on the basis of puzzle-solving ability.<p>- Many one-liner puzzles are bad indicators, because you either need to "know the trick" or have memorized the answer. Many, but not all, Microsoft and Google interview questions I hear about -- I've never interviewed at either place myself -- sound like they fall into this category. (E.g., from a recent article I read about Google: "You're reduced to the size of a nickel and thrown into a blender. How do you get out?" I don't care if you're clever enough to answer that; I just want you to able to write enormous amounts of high-quality code for me.)<p>Im my experience, the best way to find out if someone is a good programmer is to talk to him/her at length about something he/she has built. Can he/she talk in detail about how it worked? What challenges were involved? What tradeoffs were made? And, above all, was there passion behind the work and these choices?<p>A good interview for me was one where the candidate came in having written or contributed to a large system -- for fun or work -- and was excited to tell me about it. It didn't really matter whether it was relevant to the work we did (airfare search). Anecdotally, it seemed like people who really loved to code generally worked out pretty well. People with great-looking resumes that didn't love to code -- but maybe loved other things, like arguing over which language, operating system, architecture, or business strategy was better -- usually didn't produce much.<p>If the first thing you want to do when you wake up in the morning is code something, you're probably going to do well as a coder on a hard-core software team. Otherwise, not. Everything I personally did on programmer-hiring strategy was a proxy for figuring out whether someone was like that or not.

So people who handle these stressful situations well and solve these problems under pressure only represent one type of developer - which Google has in spades - the kind who does very well in a structured environment, got great grades in their undergrad program, loves to worry over interesting and difficult problems, etc. The kind of people who don't do well in these situations (like myself) are often valuable in other ways. Some people cave under pressure, and even more people have very low self-esteems when it comes to programming and intelligence in general. Yet these same people can be highly valuable and productive given enough time to make mistakes and solve problems in their own way. These types of people may be better at seeing the bigger picture and are less likely to over-engineer a solution or fall into the not-invented-here trap. I think it's interesting that Google finds a lot of bright engineers using these high-pressure hiring tactics, but that they often turn to wholesale talent acquisitions when it comes to building products like Google Plus. My guess is the vast majority of the talent Google has gotten through acquisitions would have failed miserably going through Google's traditional hiring process.

Hiring developers based on brain teasers is like hiring journalists based on how well they do at scrabble.

My first interview out of college was for a position as a Mac developer for a company that made a PC email product (for Novell's LAN message exchanger -- this was in 1990). I'd done several business-focused Mac applications while in college, including a really complex EDI app, and I thought they'd be interested in seeing them as part of the interview. I even brought selected portions of source code.<p>Nope! Instead they gave me an hourlong verbal interview, which went great, took me to lunch with the team, which went great, then sat me down with a lead developer, who had printed out some of the winning entries from the obfuscated c contest, and asked me to read them and tell him what each program would do when compiled. That did not go so great. I didn't get the job.<p>That experience remains burned into my brain, and as a manager I won't give puzzles to an interviewee. I'll ask to see code and expect them to speak intelligently about it &#38; the choices they made, but I won't ask them to write code (or, compile code in their head) on the spot.<p>(Postscript to my story -- that company never shipped a Mac email client. I believe they interview-filtered themselves out of the entire Mac market.)

After considerable reflection upon this point, I tend to fall on the side of those that say that a really _simple_ problem, like FizzBuzz, is quite valuable as a negative filter. It saves one valuable time in dealing with people that, as it turns out, can't code at all, literally, but are incredibly good bullshitters. There are those. But that's about as far as I think whiteboard code quiz shows should go.<p>It took me a long time to appreciate the utility of something like FizzBuzz in terms of how Pareto-optimal it is.<p>I think it finally clicked when I was getting interviewed for my US citizenship by USCIS in 2008. In the preliminary stages of the application process, the officials had gotten us all psyched up about how there is going to be a US civics test and an "English test". We were given a 100-question booklet of Civics 101 and a lot of anticipatory anxiety.<p>So, imagine my surprise when the interview came around and with it, the storied, fabled English proficiency test. You know what it was? Listening to one simple sentence read aloud and transcribing it accurately. It wasn't even a hard sentence, it was something on the order of "The man walked his dog across the yard and over the turnpike". Really? Really?! I was flabbergasted. Come on, I could pass this in like 4 languages. One sentence? That's all?<p>After thinking about it, I realised there was more utility to it than I had appreciated, though. Yeah, I've been speaking English for 18 years, but there were people in the room who literally did not. They could not pass this test. And for someone who really cannot speak, write or understand English much, this test would be genuinely challenging or impossible to pass. At the same time, 90% of people who can pass it probably are sufficiently proficient in English to function in this society. I'm not arguing that it's a particularly insightful, perspicacious or revelatory test, but the point is that it accomplishes more than you think, from a statistical perspective, even though anyone who is even marginally literate would dismiss it as ridiculously easy to the point of absurdity.<p>So it goes with "a priori" programming tests, I think.

Funny, every time I read "puzzle" or "brainteaser", I think, "I don't know why manhole covers are round," or "Who gives a shit that it took 7.612 seconds for my program to identify the 78,498 prime numbers less that 1,000,000." (Yes I know, get a life.)<p>But every time I read "FizzBuzz", I drop everything and go back to refactor my latest version, aspiring to get it down to one conditional.

I don't enjoy brainteasers. If an interview question begins, "Four people and a goat need to cross a river...", I cringe.<p>But when I'm interviewing, I <i>do</i> ask people for a pointer to their open source projects. And if they don't have open source projects, I give them a workstation and ask them to write code. I've seen too many self-proclaimed programmers who claim, "I spent the last 2 years writing Python," who can't sum a list of 10 numbers without using Google. (Hiring can break my heart.)<p>One way or another, you've got to see code. Real code is best, but it's better to ask people about toy data structures than it is to hire 3 team members who can't write FizzBuzz.

To DHH and team: thank you for using reasonableness and sanity in your hiring practices. Wish this was more the norm, but indeed it seems to be the exception.<p>Trying to evaluate programmatic skills is surely a challenge, but I've found a method that seems to work very well -- simple walkthroughs. I'll do at least two walkthroughs of code, asking a candidate to explain to me what's happening using two different artifacts:<p>* Some code they've written that solves some problem<p>* Some code we've written that solved some problem<p>This has proven to be quite effective in rooting out those who look great on paper and talk a great game, but fall apart when put into action. On the flip side, I've yet to find someone who could explain their code in sufficient detail that then proved incapable once in the job.

Here's the main thing that fucks this all up: there are a very limited set of brainteasers out there.<p>This means that if someone gets your brainteaser right in an interview, there's, say, a 90% chance that they've heard a question just like it before (or one with a similar "trick"). The false positive rate is too high. Many people know the answers to brainteasers <i>far</i> beyond their ability.<p>If you were to ask a completely new brainteaser, as difficult as the canonical ones, but requiring a different trick, you would get a very low success rate.<p>Side note: there's too much criticism on HN about "what does this question have to do with my day-to-day job". This is a bullshit point. The point of a brainteaser is to test problem solving ability, creativity, etc. It may not do a very good job of that, but just because you won't be "reversing sentences in place" in your job doesn't mean that the question is invalid. Think about what the question is trying to test.

The best interview is a pair programming session around some 1-2 hour problem. It takes longer, and it may come later in the interview process, but it's a high bandwidth way of assessing someone.<p>If they don't have a strong open source background (not everyone does), earlier in the process it helps to ask a couple very simple programming tasks, such as writing a function to reverse a string in place or the infamous fizzbuzz. It will weed out people with great resumes but poor skill.

Don't know about the rest of HN but I found it strangely reassuring that even someone as talented as DHH can end up in front of a white board during an interview and feel stupid. I've been there...

"The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written"<p>This a thousand times over. Go through their github (or whatever) profile, look at their commits, see how they write tests, etc. It doesn't take a huge amount of time, if you are any good at what you do, you'll quickly be able to filter out a failure (like, he doesn't have code that you can browse).

&#62; The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size.<p>I agree with the latter. This is how I used to hire programmers, one month on a freelance contract first with a single project. The importance of the code itself is also overrated IMHO. It's just one tiny part of someone you add to your team, what about creativity, taking initiative, never giving up, being a team player, etc. Each of them are equally important to plain coding skills.

I think the best interview I had was at Mochi Media. What each interviewer did was pick some programming headache that gave them trouble in the past couple weeks and present the problem to me to see how I would solve it. It was probably the hardest interview I ever had but I think it's great way to assess wether a candidate is able to solve problems they see.

I would hope "you just broke the user experience for 50 million people, there are 30 internal teams screaming your name" is higher pressure and stakes than an onsite interview loop. You can make or break your career in that situation too.<p>I had to deal with a crisis this week that boiled down to a poorly optimized SQL query or two. Of course, the manner in which they failed meant our DB started disk swapping and hurting our availability.<p>Contract to hire is something we do but it is rare. An initial try out period doesn't do us much good since the learning curve is long.<p>I don't trust take home assignments unless their solutions can't be googled. And it's a lot of work to create and evaluate problems of that nature. You want them just long enough, not a huge time sink for you or the candidate. You want them to demand a range of solutions so there are many ways to fail and succeed.<p>I agree we should not be doing riddles/puzzles. For me the most straightforward way to gauge a candidate's ability in an hour is to ask them to program on the spot and see how they act. Yes, it's a game and the candidate can learn to beat the system and oversell themselves. But the vast majority of people I don't want to hire do not go to these lengths.

I don't get the puzzle aversion. Sure, you can stare at someone's code as much as you want to but I don't think you can infer a lot about how their thought processes worked when they were writing it (a proper use of version control helps here but not much). You only see the outcome. You may say - I don't care how they think, I want the job done. Well, that would make sense if they were contracting on an self-contained project. But usually you hire someone to join a team, where how the process works is more important than the actual outcome. Especially whether or not the process is flexible and fits your workflow. So puzzles are good for that, especially that it's not about getting the right answer but approaching the problem. The reason they're abstract, which I agree sometimes crosses a line, is that leaving out the technical details of a problem makes sure that both candidates familliar with the technical details and those that are not have equal chances of doing well.<p>The problem with this post is that it puts "puzzles", "API quizzes" and "math riddles" into one category. I can see how one can be offended by an API quiz such as "what does X do in Y?", especially if they have a track record in Y or could just look it up on the Web if they didn't know. But math riddles? Puzzles? I'd actually be grateful that I don't have to deal with technical catches and can purely focus on a solution.<p>One thing to consider is that recruiting, especially at a larger scale, is about minimizing false positives rather than maximizing the collective value. Sure, someone good at puzzles might turn out to be an average engineer but a bad one? I don't think so. Whereas, I can see how someone with a huge pile of code on GitHub can get stuck in a critical situation where if they write unit tests or how they design their APIs is meaningless.

I suppose the type of development 37 signals will do is more concerned with product design and less to do with fast algorithms etc.<p>I haven't really used their products but in terms of technical sophistication they probably aren't much advanced beyond CRUD type web apps (I may be wrong here).<p>If you wanted to hire a games engine developer or someone to design algorithms for google then brainteaser problems might be more relevant.

Interview and hiring techniques are a perennial favourite on HN. I just have a couple of comments (disclaimer: I work for Google).<p>I personally detest abstract puzzles like "why are manhole covers round?" or (worse) "How do you move Mount Fuji?" AFAIK these types of questions are not and have never been used in engineering hiring at Google (sales, etc I know nothing about). WSJ and other sources have made this (AFAIK inaccurate) claim. It's possible individual interviewers do this.<p>My own philosophy is that a <i>simple</i> whiteboard coding problem is an incredibly useful litmus test. I don't mean anything incredibly complicated either. If you can't write a function that gives me a row of Pascall's triangle, that's a problem. Note: I don't expect you to know what Pascall's Triangle is, merely implement it when explained.<p>Some of you may think such a test is a waste of time but I assure you it is not. It's astonishing how many people can't do this (and even more astonishing how many that do make it through resume and even phone screens).<p>The key here is "simple". One problem is that interviewers fall into the trap of thinking these problems are too simple and make the problems increasingly harder. Worse, they may get in a situation of asking someone a problem that is either something they know (because they've covered it before) or they don't (where it's not something you can simply nut out). This is a useless indicator in my experience.<p>Just because you can create a function to return a row of Pascal's triangle doesn't mean you're a great programmer but if you can't, it almost certainly means you aren't. It's a negative signal filter, nothing more, but an incredibly quick and useful one.<p>API pop quizzes I'm not a big fan of either, generally speaking, but on the other hand if you've used a language sufficiently long, basic questions about commonly used libraries aren't unreasonable either.<p>Interviewing is a numbers game. Your goal is to come up with a process that balances time spent by the interviewer with finding a suitable candidate. Note that I didn't say accurately assessing each and every candidate. It's sufficient for the employer to simply find one qualified candidate even if you falsely determine someone who is qualified isn't.<p>A certain error rate is to be expected. It reaches a point where the time investment to reduce it outweighs the benefit of higher accuracy.<p>One final note: I detest (both as an interviewer and an interviewee) any kind of "take home" test (other comments have mentioned this).<p>As an interviewer, it's extra work to assess, people cheat, good candidates won't bother and so on.<p>As an interviewee, I'm not going to spend hours on you without speaking to someone first to find out how likely it is that you're a good match for me and the likelihood that you believe I'm a good match for you.<p>I would argue that a take home test is significantly more effort than a simple coding problem that doesn't have a significant increase in accuracy.<p>The one exception to this is automated testing where the problems themselves are relatively interesting. Some of the Facebook Puzzles fell into this category (and some don't). People may do those anyway for kicks. But again, there's nothing stopping someone going to Github and copying and pasting someone else's solution so what value is the filter, really?<p>EDIT: clarified my Pascall's Triangle comment. This isn't a trivia problem. I don't expect people to know what it is, merely implement it when explained.

Whenever I see a firm use brainteasers during the interview process, I just wonder why they don't simply recruit the best Sudoku players. That seems about as valid a way to judge software development skills.

I work for a web development agency, which develops projects for some of the largest companies. We generally work on App Engine (python).<p>At the day of my interview, they didn't really ask me any brain teasers.<p>They asked if I knew App Engine, I showed them my open source project for App Engine.<p>They asked if I knew django-nonrel, again I showed them another open source project of mine, that used django-nonrel.<p>That was it.<p>Since the day I was hired, I did plenty of good work. Now I'm entrusted with a global CMS project which is being used by hundreds people and viewed by millions.<p>If they had asked me brainteasers, I would probably have failed most of them, as I don't really practise with those kind of problems. But I like building web apps. And that's what I am doing right now.<p>If your company earns money by solving brainteasers, then you should ask them during interview.

Companies like Google hire hundreds (thousands?) of developers per year, hiring policies at 37Signals are bound to be different from those at Google.<p>If you have to weed out thousands of candidates, trying brainteasers to estimate their problem-solving capabilities doesn't seem to be a bad strategy. If a candidate is not familiar with a particular problem, his/her solution may be suboptimal, but the process of getting to that (suboptimal) solution may be a good indicator of the candidate's problem solving skills.<p>Obviously, syntax-checking a candidate's on-paper Javascript code isn't going to help anyone.

I would submit that the biggest reason for this view and the reason many people hold it is that 37signals as well as most of the programming jobs are CRUD jobs where non-algorithmic skills are probably more important than algorithmic skills and IQ is also less important. Brainteasers are an IQ test where you are not concerned about false negatives.

I'd feel uneasy if asked to show some of my code.<p>Obviously i cannot show something i'm currently working on, and everytime I look back at some code I wrote six moths ago I find myself wondering "did I <i>really</i> write this piece of crap?!". And it's always been this way - at least since i've been six months into programming. I hope it is a good sign.

I knew an engineer who used the 'why are manhole covers round' question in every interview. He was always shocked at how 'quickly' the interviewee figured out the 'answer' of "so it doesn't fall down the hole". These new kids - they're smart. As far as I know, he never figured out that 90% of people have already heard that one.

Read the comments, and I've got to say: I'm not one of you guys. I don't write code, I build products (otherwise known as software), that human beings use.<p>I looked up Pascal's triangle in Wikipedia, so now I know what it is, and I could write the code tonight to handle the shit out of it, but I'd probably mess it up on a white board when commanded to perform like a monkey in front of guys in T-shirts.<p>I own and run a software company that I started myself. I built the first 5 successful products (not code, products - there's a difference [kind of a big one too]) myself, and I don't give a crap about Pascal's triangle. I'd probably fail every coding test on the planet, but when it comes to delivering finished, polished, working products, that 1000s of human beings will actually use without picking up a phone and calling customer support, I've got that covered.<p>Stay classy with your 'code' guys.

I was rejected in a recent interview (2 months later I'm waiting to 'hear back' means rejection) where I wasn't competent at writing algorithms to do various tasks such as implement an insertion sort based on this image: <a href="http://en.wikipedia.org/wiki/File:Insertion_sort_animation.gif" rel="nofollow">http://en.wikipedia.org/wiki/File:Insertion_sort_animation.g...</a><p>My confusion was the job was looking for a generalist who had experience with lots of different technologies. Specifically, working with node.js and socket.io polling methods, API design, working with some python framework and backbone.js.<p>I'm honestly not sure what the lesson there was. Am I a bad developer? Do I navigate interviews poorly? Does my breath stink?

I'm convinced that interviews are a complete crapshoot for both sides. My first job interview went like this.<p>1. 45mins late for the interview.<p>2. Quizzed a chain of brilliant engineers. Couldn't answer a singe technical question they asked. Lots of "Sorry my experience was more with X, not Y".<p>3. Flatly said "You guys are looking for someone else, I don't have the skills you're asking for".<p>-&#62;Hired because I was honest. Learned quick. Promoted every year.

Unfortunately a large number of us work for employers with strict IP agreements and we cannot just show a potential employer some code or work on a small project for them so that they may "try [us] out". Even working on open source projects can be tricky.

I haven't had to really be interviewed in 15yrs, but the last one I had was probably the best. I walked into the office of the guy who would be my manager, he introduced himself then immediately turned to the whiteboard and started writing and drawing while saying "This is what I'm about to start working on, we need to figure out how to do it.". We spent the next 3 hours working out a solution together.<p>I ended up not accepting their offer, mainly because of the things I learned in the interview. But I still feel like that was a good way for both sides to evaluate each other.

I thought they hire them based on their choice of personal computers (sorry, couldn't resist)?<p><a href="http://david.heinemeierhansson.com/arc/000433.html" rel="nofollow">http://david.heinemeierhansson.com/arc/000433.html</a>

Puzzles and brainteasers seem rather off-putting to me in an interview environment. It's like the interviewer is saying, "I can solve this, are you as smart as I am?", which isn't the type of relationship I want with coworkers. I prefer to steer interviews into a more collaborative conversation, to see if I would like working with the person.<p>Anyone who expects me to spend 3 or more hours or so on some take home test just to interview with them, or thinks that the only people worth hiring are those that "the first thing you want to do when you wake up in the morning is code something" aren't people I want to work for. I love programming, but at the end of the work day I'm going to go home and engage in other aspects of life. I wouldn't want a boss that had different expectations.

I saw outright that I'm not an algo guy, you already have those types of people -- namely the guy interviewing me. But they dont know how to scale systems or know overall product. If they want to look at code even I call bad, I can point them at my github account :)

I made a post on HN a couple of weeks ago basically saying the same thing. It can be rather frustrating to be asked questions with seemingly no relevance to the actual position.<p>I understand what they're going for though - the companies that do it correctly are looking at the candidates thought process through a challenging situation.<p>While I think it's important to figure this out, I believe it's more important to know how well the candidate can do the actual job, that of sitting down and writing quality software. Brainteasers and writing modified sorting algorithms won't get this (unless the company is in the sort algorithm business I suppose) but having the candidate write code which could actually be used in the job would.

I remember reading Heroku does similar. Using applicants in a project works well.<p>I have always tried to push my potential employers to use me on a small real life project. I know it's time consuming (and you end up working for em for free) but it works both ways. I get to know what kind of work environment they have and they get to know me in a real deal. I find it much more comfortable because the puzzles always seem to be a hit and go. The solution may click or it may not.<p>I know it's a start but if this catches in the industry I expect employers to even pay for your 30-40 hours interview.

I don't understand why people are forced to write on whiteboard. Some get very stressed and lose their ability to think, some can't multitask (paint on whiteboard, think and talk to the interviewer at the same time). Just give them the task, give them a computer, give them 30 minutes of privacy and then let them explain the complete, working and tested solution - you get better results as this situation is more like regular coder's daily routine.

There's nothing more infuriating and insulting to me than being asked to write some lame string sorting function on a pad of paper during an interview. I've been burned more than once because of pop quizzes like these before... lots of missed opportunities because some HR person thinks that writing dinky code no one actually uses in a real project is more important than my portfolio of projects that I've designed, created, and maintained.

Been considering the following hiring technique, interested in thoughts/opinions:<p>We accept applicants of any experience level. To apply, find an open source project on github, get a patch accepted to it. It has to be of useful scope with a unit test, object-oriented code only. Send us the diff, a brief work history, a reference and why you want to work for us.<p>If we like it, come work with us for a day, paid at market rates, and we can both assess whether it's a good fit.

Google doesn't ask these questions! It's a fiction that is probably perpetuated because the topic is SEO linkbait.<p>Again: Google doesn't ask these questions!

I'm always amused by interview questions.<p><pre><code> Primarily because I do terribly at white board problems thanks to disgraphia(terrible handwritting), and secondily because while the questions might give some indicator of whether or not you can code hello world, or a binary tree depth first search algorithem they do a terrible job of testing your actual domain knowledge. For instance, when interviewing for a SDET position i've never been asked a single question on implementing a DI framework, designing custom asserts, designing a test object/mock object, etc. When interviewing for a web development position for high volume sites i've never been asked about cache invalidation, reduccing http requests, designing an ajax event system to reduce unnecsesary calls. Etc. I have been asked a bunch of IQ tests masquerading as mid-low level programming questions, and random trivia questions on language features.</code></pre>

Puzzle solving ability is a reliable indicator only if the candidate hasnt specifically prepared for them. Many Indian IT companies in the mid-late 90's conducted exams with difficult programming puzzles in them. This was great for a while, but soon those who wrote these exams told everyone else and the puzzle pool dried up. Future groups scored really well due to being better prepared against a known pool of puzzles. These days most IT companies have moved on to SAT/GRE style analytical problems.<p>Imagine your luck if you are at a Google interview and already know the Pascal triangle. You can just put up an act pretending to analyze various aspects before unveiling your grand solution.<p>If companies are merely using analytical ability as a filter, a SAT/GRE style exam will do better because the problem pool is much larger making it less vulnerable to preparation.

We can all agree that there are different kinds of programming. There are some really hard problems out there that need to be solved. There's also the kind of programming that's a matter of getting users to interact with data. When it comes to web application development (my background), I think puzzles aren't the right way to approach the interview. To be honest, it shouldn't be the job of the company to find out what the interviewee does and doesn't know. It's the duty of the applicant to control the interview, come in with their laptop blazing, and start walking through building a web application from scratch or to show off something they are building, talk about the decisions that were made through the process and why.

I've been in a couple of these interviews before and I've also had to try to interview programmers I wanted to work on projects for me ... an interesting approach that I haven't had the opportunity to test exhaustively is to start out like the computer SAT exams.<p>Let me explain.<p>On the SATs, they ask you a simple question, and if you get that right, they step up the difficulty and on an on until they discover you're a genius. If you get the answer wrong, they ask you another question of similar difficulty to the last question you got right and on and on.<p>Now with regards to interviewing, I like to start from the basics ... I'll start out with simple questions about languages they've used, so that they're comfortable. Syntax isn't <i>majorly</i> important to me, you can google it ... however I take note of how comfortable the interviewee is with the syntax of the languages they're using (especially if its one we use). So while it won't count against them, it can count in their favor.<p>From there I'll ask them more difficult questions. How would you implement this one thing? We have this one problem we're dealing with, how would you solve it? If they can't do it, I'll try another and another, giving them multiple chances to show that they can hang. All the while I'm poking and prodding to find out how they think about stuff ... and what makes them tick.<p>And then I'll go to the really hard computer sciencey stuff (which bores me to tears to be totally honest. sorry.) ... just to see how proficient they are with it. Obviously a person who eats algorithms and datastructures for dinner in addition to acing all the other stuff is someone I'd love to have if they work well with us on a small project we'll have them do.<p>I do the same thing with programs they've written, side projects they've done etc. To summarize, I'm just trying to find out where they are on a continuous scale from "total newb" to "could write os x from scratch if they needed to"<p>Basically, I think brain teasers CAN be useful, but I think we should use them carefully. Value the way someone thinks over just them just getting answers correct.<p>my 2c

I personally adore riddles and algorithmic puzzles, but I too have had to confront the reality that they simply don't provide much information about the candidate's actual coding talent. Instead, where I'm working, we combine a few different strategies to gauge an applicant's overall potential:<p>1. Nobody gets an onsite interview without solving a coding problem first. These are designed to take a couple of hours for a competent programmer. For each applicant, we send the challenge most functionally relevant to the position they're interviewing for. Despite the negative comments elsewhere about "take-home" problems, I think they expose a lot of information about a person's coding style and thought process that whiteboard code does not. For instance, we recently received a solution to a simple graph-related question that was technically correct and usually fast, but contained at least five nested layers of loops and took forever on a certain class of edge cases. The candidate obviously understood the problem, but seemed oblivious to the fragility of his solution. You can't grade for style or really pound the edge cases in an onsite interview, because candidates usually have their hands full enough finding any solution.<p>2. At least two of the interviewers spend the majority of the time on big-picture, conceptual, and software design questions.<p>3. I split my interviews between puzzles (the 17-minute bridge, the 200m building with 150m of rope, etc) and algorithmic questions. However, I don't care about whether the candidate answers my riddles correctly or not; I'm only observing behavior and process: in other words, whether they remain calm despite uncertainty, and whether they ask relevant questions that reveal more information about the problem and make observations that narrow the possible space of solutions. (For example, in the bridge question, realizing that the slowest people should cross together is as good as a solution to me.) I do care about their responses to the algorithm questions (e.g., find a subrange of an array that sums to 0), but mainly to determine whether they have a reasonable idea of which data structures to use, and how effectively they detect problems with the rough solutions that go up on the board at first.

Every Time i see these posts on here, I get a sad face. I am a SQL Server DBA and most of these puzzles don't translate well to SQL. That doesn't stop me from trying!

I really don't know what s right way to interview, but I do know that you are actually hiring a team not set of individuals. You need to have balanced team (means to be smarter than you on each of the important fronts) to deliver a successful product.<p>So I guess it might be ok to hire somebody based on brainteaser if you need a sharp, quick thinking person which will be a great asset during brainstorming sessions. I don't know.

It's worth noting that 37signals believes in hiring by requiring candidates work on a 20-40 hour project before making the hiring decision. There approach would be completely unacceptable to a large number of developers, it's only because of their strong reputation and remote hiring that people are even willing to consider it.

I once interviewed for an architect position for one of the "web scale" companies and it was a algo fest all through. All the while I remembered the dev manager (whom I'd have reported to) saying that they were looking for an architect for a new project and the key challenge was learning all the existing components they have, picking the right ones for the job (or proposing to build new ones as required) and then - most importantly - getting it sold to a bevy of other architects and stakeholders.<p>This was right up my alley as I currently lead 3 teams and we regularly have to deal with architecture and design issues on a large application with tons of other teams involved; and I was therefore pretty stoked about the job -except that the algorithm wall stood between me and it. Not having ever done an algorithms course formally(although I read the books out of interest) I went down in flames; at least I think I did - because I'm yet to hear from them. Wanting closure I even contacted them and asked point blank if I was rejected, but I was assured that I WASNT rejected and that they'd "get back to me".<p>The process itself was largely the standard "how would you write &#60;algo x&#62; in java", which I did not do very well. One interviewer asked me to implement a Queue using two stacks and maintained that "there's a better way" despite my multiple attempts. I finally asked him if this was an actual real world problem that they faced in their company to which his answer was a glib "No, I was just testing your problem solving ability"<p>To be fair, however, some of the other interviewers were better.<p>One actually glanced through my resume, picked on one of my open source projects and grilled me on what I'd do given a hypothetical implementation constraint on a particular piece of the architecture. In hind sight it looked like he was trying to get at my algorithmic problem solving ability while connecting to something I'd built; but at that moment I was so hung up on the <i>problem</i> that that my software solved than how the software itself was built that I ended up frustrating him with my pre-thought solutions. He kept asking for "from scratch" solutions to which most of my responses were "but it already exists in X, so I'll just use it".<p>The best interviewer was a senior guy who was totally OK with my lack of algo knowledge; and very quickly switched to designing a solution to a web scale problem. He asked me for one solution, picked out the flaws in the solution using Big-O notation and asked me to refine it. This went on until we reached the asymptote that they had actually implemented in production. Along the way, he gave me more of the solution than I did to him, but I ended up with a better appreciation of the whys and wherefores of the problem; and more importantly, problems of the kind.<p>My general take away was that companies that use algorithm tests make the following assumption:<p><pre><code> algorithm solving ability =&#62; general problem solving ability </code></pre> ...and this is obviously true for companies where scale is a fundamental issue, ie , where the scale of the problem has not been addressed by <i>any existing solution</i>.<p>However, the problem with using this definition to measure candidates are:<p>1. Your company problems may be solvable using existing solutions. Then the real litmus test for your candidates is how well he can assess and assemble existing components/frameworks/libraries to realize the solution. This is where attitude matters more than anything else; as does prior work and interest in such work. "What was the best project in your career and why" might be the best question to ask such a candidate.<p>2. Your interviewers may never connected the LHS of this equation to the RHS. This is where, IMO, questions like "queue using 2 stacks" or "manhole covers being round" come. The interview becomes cargo-cult worship at the Algorithm/Puzzle altar.<p>So my suggestion to the companies that still have algorithm-based interview processes is: when presenting such a problem in an interview at least tie it to a real-world scenario that you had to deal with in your work place so its not a test of how much practice I've had with Cormen and Skiena (yup read the yegge post).<p>Speaking of which: Am I the only one who finds all algorithm books as being too prescriptive? They all seem to dwell on the laundry list of data structures followed by the laundry list of algorithms; leaving the reader to figure out the "what to use for what" and the "why" all by himself? I'd pay for an algorithms book in the style of Polya's "How to solve it". Skiena's war stories are an attempt in that direction, but a full frontal attack on A&#38;DS would be great.<p>Aftermath: I left that interview with two takeaways:<p>1) I had to shore up my knowledge of algorithms and data structures simply because it was the "basic tools of the trade" and there <i>ARE</i> still problems around that need you to dig that deep in the industry (thanks to the final interviewer)<p>2) Large organizations have "how to assemble software using existing pieces" issues in their software development, not "build from scratch" issues; but developer(turned interviewer) hubris will always skew towards testing for the ability to build (proxied by algorithm skills).<p>Either way - 37 signals notwithstanding - it makes sense to catch up on your algo skills :)<p>Oh, and please, please, please lets do away with the Hollywood principle when it comes to developer job rejections. If I'm rejected, tell me. I can take it.

There are so many different views here about interviewing that it can get quite dizzying.<p>1.) 37 signals view is that there is no point in asking programming puzzles or brain teasers; this goes for white board questions as well. Their view is that, it's just better to go over real code the person has written, and if they seem like a good fit, then why not try them out.<p>2.) Cletus's view is that you should ask a "simple white-boarding problem", that would weed out in theory weed out the "bad programmers", with a certain error margin.<p>3.) Dmbaggett's views is that solid programming puzzles (one's that are usually are take home, or one's that you do at home) are good for attracting programmers that will fit (the role of a computer scientist at ITA). However cheap whiteboard problem's aren't necessarily the best for finding great programmers.<p>4.) edw519 commented a while back, that written code sample done at the time of interview is best, as it provides rooms for some discussion material that will gauge if the interviewee is a great fit for the role.<p>[1] <a href="http://news.ycombinator.com/item?id=834513" rel="nofollow">http://news.ycombinator.com/item?id=834513</a><p>4.)Another article on hacker news, "I Won't Hire You", got pretty much down voted for it's view, that you have to be greater than great to work at Golem Technologies.<p>[2] <a href="https://news.ycombinator.com/item?id=3428567" rel="nofollow">https://news.ycombinator.com/item?id=3428567</a><p>5.)There was one other article on hacker news; the article mentioned how in the end programming interviews puzzle, didn't matter and the programmers that they hired and did the best, ended up not being the most productive programmers.<p>6.)An article about hacking the current interview system. Apparently the top commentator timr, really thought that you should grill the interviewee to find out if they are hacking the system.<p><a href="http://news.ycombinator.com/item?id=3370341" rel="nofollow">http://news.ycombinator.com/item?id=3370341</a><p>The list can go on and on.<p>So in the end what is the best algorithm for interviewing a programmer? Are you hiring out of fear, that you need X to do a Y job, but you don't want X to destroy your company? Are you hiring to find a rock star programmer, who can take your company to the next level? Are you hiring a programmer to do a specific well defined job? Are you hiring to fill your ego, that you have the power over someone?<p>It seems very hostile to get a programming job. It seems that some interviewers want you to come in and contribute from day one, which I have never seen quite happen. Why are some interviewers so harsh and dicks with the interviewees?<p>If a programmer fails your interview, does it mean they are truly stupid?<p>Is your interview really a true gauge of IQ? Does your ego play into the interviewing process as one HN commenter wrote<p>"The biggest problem with anything like this is the idea that 'here is some test of inherent intelligence - I am far better than you so you are inherently unable to do this thing' which is just the biggest barrier to actually trying to do something - if you think you inherently suck or at least are simply mediocre, your motivation to do that thing is severely reduced."<p>The general feeling I get is that as interviewers, we are trying to put down and ridicule, interviewees for whatever reason did not pass our test. Maybe in accordance with our company's guidelines or our own ego's.<p>Instead of seeing the glass half full with interviewee's potential we are seeing it half empty, as their ability will never be good enough.<p>I think with determination and practice anyone can be "great" programmer. It's sad to see many interviewer's don't realize this, cause "they don't have time."

Why isn't anyone testing candidates' ability on reading and working with existing code? I do that all the time with interviewees.<p>Why I do that? The obvious being the potential hire is required to work within a large code base (often code not written by yourself) and having the ability to read and understand quickly what a piece of code does is a great skill that not many great coders have.

I thought it was because 37signals didn't want their brainteasers splattered across reddit &#38; ycombinator.<p>Today I Learned ....