We looked at how a thousand college students performed in technical interviews

I&#x27;m leaning towards blaming this on the interviewing process.<p>Last year I spent some time interviewing job candidates and what I learned from this is that the recruitment process in IT, in general, is broken.<p>A typical candidate is first faced with some kind of task to weed out the lower 20% of applicants. After that comes the proper interview during which the candidate is asked various questions like what was their major or how is the abbreviation &quot;SOLID&quot; expanded.<p>In my experience, none of this correlates highly with future performance and this may be the reason why regardless of the school they attended those students performed roughly the same.<p>Time and time again the one thing that seperates the best from the rest is their ability to perform code reviews. I have yet to find somebody who&#x27;s a poor developer, but a great reviewer.

&gt; What this means is that top-tier students are achieving the same results as those in no-name schools<p>This is an misleading conclusion that ignores a HUGE selection bias. I doubt top MIT CS students, for example, would feel the need to practice coding interviews on interviewing.io

&gt; <i>What this means is that top-tier students are achieving the same results as those in no-name schools.</i><p>When NYU and Arizona State are &quot;Top 50&quot; while Michigan State and Vanderbilt are classified as &quot;no-name schools&quot;, I question the meaningfulness of this blog&#x27;s baseline and what it means for an interview to be technical.

After running over a hundred interviews within the last few years, this doesn&#x27;t surprise me at all. Interviewing is a skill separate from school, and if you train for it, you can become good at it, regardless of your pedigree.<p>I actually think studying for the SATs is a better comparison - it&#x27;s a test that doesn&#x27;t exactly translate to real-world performance, but has huge bearing on prospects, and often the best advice for acing both (tech interviews and the SATs) is to simply do as many practice tests&#x2F;problems as possible in the areas you&#x27;re weakest in.

People have forgotten how to train their core social and psychological skills like resilience and confidence. Unfortunately, these are not things we are born with or can code our way out of. These are things we have to train very hard at similar to how martial artists or athletes approach their training. The thing most programmers and knowledge workers in general are missing is true belief in their own skills (you have been doing this how long? come on snap out of it) and the resilience that comes from rejection after rejection, and failure after failure. After a certain point in your experience you should come to understand that raw skills are not really the end all, be all. If you are not the guy then you are not the guy no matter how many degrees you have. Sometimes the guy with the PhD from MIT is not the right guy because he doesn&#x27;t connect with the rest of the team, or the vision, or whatever, simple as that.<p>Also, if you&#x27;re testing only for specific skill sets rather than aptitude and interest then you already failed as a hiring manager.

&gt; Indeed, statistical significance testing revealed no difference between students of any tier when it came to interview performance.<p>Ah, the classic confusion between not finding enough evidence to prove there is a difference, and finding enough evidence to prove there is no difference.<p>The juniors from elite schools in particular have fewer 1s and 2s and more 3s and 4s than the other juniors. Really, you found statistical evidence that they aren&#x27;t from different distributions? I&#x27;d love to see it.

I think this is a bit specious, at least from anecdotal evidence. I know bunches of people at my school (Top 50) that have interviewed at companies that ask leetcode-style questions yet very few get offers or even make the 2nd round or onsite. If you look at LinkedIn you&#x27;ll see the vast majority work at Cisco, IBM, SAS, Amazon etc and not Google, Jane Street, or Facebook.<p>The quality of the cohort and, to a lesser extent, the quality of their DS&amp;A classes matters quite a bit.

Maybe what they&#x27;ve proven is that technical interviews are not adding much information to the hiring process. This pretty much matches with my experience that quality of interview is at best only very loosely correlated with actual ability as a software engineer. Much more relevant is direct experience working with them. School quality is a strong signal.

Charts don’t have the same scale. As the result, it’s hard to visually compare these - for example, for juniors, the shapes are similar for score 4, but top tier school students are above 20 mark, while the rest are below.<p>Interviewing is expensive, especially if you hire the wrong person, and if a given population tends to cluster around higher scores, that what you’d pick first.

I started wondering why CS interviews aren&#x27;t more based around reputation and experience, like other jobs.<p>I&#x27;ve seen a lot of bad devs get hired into places with supposedly high bars, or devs being let go and then ending up at Google next. While some places often let go of good devs, I&#x27;m talking of cases where I believe it was justified.

We all know that interviews are terrible. Wouldn&#x27;t <i>that</i> be the problem? Then students from top-tier universities achieving the same interview score as student from lower-tier universities is meaningless. The interviews fail to capture what matters.<p>The best jobs I had and also jobs I performed <i>the best</i> I got through credentials and experience, not scribbling on a whiteboard.<p>If you went to top 10 school in the world, you worked <i>hard</i> (at least it used to be this way). I don&#x27;t need to look at specifics of what you did but I know you needed grit, good work-ethic, personal time sacrifice, etc to get through. Unless you&#x27;re super smart. In either case it adds to your personal brand.<p>I think that&#x27;s what perpetuates the system. Given unlimited time I&#x27;d be happy to give everyone a shot. But time is precious so stick to what you know? I know how things are in the Uni I went to. If a CV landed on my desk form someone who did the same course, I&#x27;d put them on top of my list.

This tells us nothing about the talent or intelligence of the candidates because these tests are bullshit.<p>I wish companies like this didn&#x27;t exist to enable these types of interviews.

This really makes no sense. Even if every school had identical curriculum and teachers, students who manage to get into an MIT or Stanford have to have work ethic and prior experience that places them well above the average CS student. The fact that these students have already been vetted by a respected school is one reason why I think companies prioritize them.<p>I am all for big data analysis disproving common conceptions, but this feels off.

Just goes to confirm my own experience with grads that the difference between schools is largely cohort, not quality of students.

The graphs are super confusing. Without reading the story it&#x27;s hard to figure out if the point is their sameness or if you should keep trying to figure out the deltas.<p>Given that sameness was the point - just make a single graph with 4 colored lines.

There is one problem with conclusions, they assume that interview score is the same as real world success of a candidate. It is not so, interview result correlates with future success but there is no strict determinism. So companies prefer students that successful in interview AND come from good school, rather than just rely on interview results. It allows them to maximize probablilty of the finding a good employee.

I suppose a bit of methods nitpicking is in order. Wouldn&#x27;t it make a lot more sense to use equivalence testing if one wants to write a statement like &quot;Indeed, statistical significance testing revealed no difference between students of any tier when it came to interview performance.&quot; AFAIK statistical significance testing cannot reveal this at all.<p>Of course there is no real explanation of the method that was used besides the fact that it was some sort of &quot;statistical significance testing&quot;. Equivalence testing makes more sense to me if one wants to essentially say that MIT is Aspirin but Ohio State is a generic drug that is similar enough to work just as well (for a reasonable definition of similar enough).

On the other hand, they discount self selection bias, and the bias of &quot;qualifying&quot;.<p>Plus the charts don&#x27;t even show up, probably due to load.<p>edit: And the bias of the interviewers who use their platform, who may not be able to attract top tier talent, and are happy to get even barely competent people... It&#x27;s impossible to say since the charts don&#x27;t show up and we know nothing about the methodology and motivations of the different actors.

There is huge selection bias. The talent that already perform well in technical interviews most likely spend less time practicing (and using interviewing.io)

The charts were moderately cool when one of them loaded the first time I opened the page, but they&#x27;re not really superior to static images, especially when they don&#x27;t load at all. And when they are elements that your whole article hinges on, well...

Oh come off it, for an ounce of credibility can they maybe keep the scale of their graphs the same? And the link to the &quot;statistical significance testing&quot; is nothing of the sort. I&#x27;m calling bullshit.

I failed my first 2 tech interviews and Google and Snapchat leaving me feeling cheated by the dumbass whiteboard. Whiteboard tests are tailor-made for the inside the book thinkers. The ones that stop learning when the chapter ends and soon forget when the next subject begins. Computers are the thing that gets you to the thing. You want people who can see that the software they are working has a broader impact beyond 1s and 0s.<p>PS. 5 Years later, I&#x27;m now the CT0 of my company managing 50 people. So srs, not srs Googs you missed out =P

As far as I understand, the interviews they conduct are preliminary interviews, and questions on such interviews are much easier than questions on on-site interviews. So, basically they test for simple CS&#x2F;programming tasks which any person who studies CS should be able to perform.

A serious question- is there a syllabus of interview questions ? That is something that says the most important subject areas are X, and this puzzle covers X<p>(I am not sure how one decides if finding shortest substrings is important or not ... perhaps most common usage?)

I would love to see more recruiting at state schools and community colleges etc from tech companies. There are a lot of hard working people who could be great to work with but didn&#x27;t have the family background they needed to be noticed.

The graphs&#x2F;images do not show for me. Chrome 64.0.3282.140. Win 10

Looks like the server is overloaded... images in middle of the article aren&#x27;t loading for me.

Garbage in, garbage out.<p>What is used in divvying up a <i>self-selecting</i> cohort of <i>undergrads</i> performing on technical interviews in 2018 are the US News &amp; World report rankings for <i>graduate schools</i> composed in 2014 &quot;based on a survey of academics at peer institutions&quot; (<a href="https:&#x2F;&#x2F;www.usnews.com&#x2F;best-graduate-schools&#x2F;top-science-schools&#x2F;computer-science-rankings" rel="nofollow">https:&#x2F;&#x2F;www.usnews.com&#x2F;best-graduate-schools&#x2F;top-science-sch...</a> … and I don&#x27;t think the lack of that link in the article is an accident).<p>Having a selective grad school doesn&#x27;t mean much if anything for the standards and teaching in that school&#x27;s undergrad program, especially for state schools with giant undergrad populations and relatively small grad programs.<p>For example, Illinois&#x27; grad school is selective and highly thought of by professors and thus it&#x27;s treated as an &quot;elite&quot; school and students of UCSD&#x27;s undergrad CS program are classified as &quot;top 15&quot;.<p>Regardless, my alternate hypothesis would be &quot;students of a specific level of confidence in their ability to pass a technical interview use interview.io for a limited period of time in which it provides value to them … that population of students receives a certain distribution of scores&quot;.

One data point: I was at Georgia Tech for undergrad, now at Caltech for a CS PhD. Both are good schools, but Caltech is definitely considered more &quot;elite&quot;. As a Caltech student, I definitely attract attention that I just didn&#x27;t get at a Georgia Tech student.

People who bother to practice for coding interviews (using the same website) and successfully land an interview do similarly well on coding interviews... That&#x27;s a lot of selection bias.