Lessons from 3,000 technical interviews

The author draws a hard distinction between Udacity&#x2F;Coursera MOOCs (good) and traditional master&#x27;s degrees (bad). I&#x27;ll interject that with Georgia Tech&#x27;s Online Master&#x27;s in Computer Science program [0], which is delivered via Udacity and insanely cheap [1], you can get the best of both! (Their &quot;Computability, Complexity and Algorithms&quot; class is one of the top Udacity courses cited in the article.)<p>Keep in mind that a traditional degree program does have a huge advantage over a strict MOOC: accountability. It sounds good to say that anybody can go push themselves through one of these courses. Try pushing yourself through ten, and actually writing all the papers and implementing all the code, while working full time and having a family. That grade looming at the end of the semester really does wonders for your motivation. Plus you can get help from live professors and TAs, and the Piazza forums for OMSCS are full of smart, curious students who love talking about the subject at hand. There&#x27;s a richness to the degree experience that I don&#x27;t think you get with scattered classes.<p>(Obvious disclaimer: I&#x27;m a current OMSCS student)<p>[0] <a href="http:&#x2F;&#x2F;omscs.gatech.edu" rel="nofollow">http:&#x2F;&#x2F;omscs.gatech.edu</a> [1] <a href="https:&#x2F;&#x2F;www.omscs.gatech.edu&#x2F;program-info&#x2F;cost-payment-schedule" rel="nofollow">https:&#x2F;&#x2F;www.omscs.gatech.edu&#x2F;program-info&#x2F;cost-payment-sched...</a>

&quot;Whether passing an algorithmic technical phone screen means you’re a great engineer is another matter entirely and hopefully the subject of a future post.&quot;<p>This sentence plus the inverse correlation between experience and &quot;interview performance&quot; shown there. Makes a big smell about how biased are those interviews to themselves and not to real technical interviews.<p>From the data it looks like the questions asked using that service are the ones you might learn in university and after many years not using them, that knowledge fades away because you&#x27;re not using it.<p>This is reinforced by MOOCs being the 101 of the subject they&#x27;re dealing with. It would be interesting to see if there are trivia questions from 101 courses.<p>The most obvious bias is in the clickbait title. Those 3K interviews are in a specific platform, meaning they&#x27;re done in a specific way.<p>So after checking their results it seems that interviews done using that service benefit people with fresh university or 101 lessons knowledge.<p>What worries me more is the lack of improvement and perhaps the moral superiority of ending the article with a &quot;these findings have done nothing to change interviewing.io’s core mission&quot;. It feels like the entire statistics game shown there was to feed back what they already knew.

Thanks for writing this Aline. As a recruiter for almost 20 years, I wish I had access to all my data and then the time to compile it, and anecdotally I&#x27;d expect the finding about MOOCs would be similar.<p>The most selective of my hiring clients over the years tended to stress intellectual curiosity as a leading criterion and factor in their hiring decisions, as they felt that trait had led to better outcomes (good hires) over the years. MOOCs are still a relatively recent development and new option for the intellectually curious, but it&#x27;s not much different than asking someone about the books on their reading list.<p>Unfortunately, demonstrating intellectual curiosity often takes up personal time, so someone with heavy personal time obligations and a non-challenging day job is at a significant disadvantage. One could assume that those who have the time to take MOOCs also have time to study the types of interview questions likely favored by the types of companies represented in this study.<p>Thanks for continuing to share your data for the benefit of others.

I am perplexed why anyone would think that interview performances has any interesting statistical relevance. Much more interesting would be how successful the candidate was after receiving a job at the company.

Interesting article! Some minor statistical pet peeves:<p>1. Setting non-significant bars to 0 seems fishy. Leaving them and putting confidence intervals on everything would let them speak for themselves.<p>2. Calling something effect size is ambiguous. That&#x27;s like saying you measured distance in units (and the wiki article on effect size linked makes clear there are a billion measures of effect size).<p>I&#x27;m guessing their measure of effect size were the beta coefficients in a multiple regression?

Interesting bit on the MS degree. I followed the link, and I&#x27;m not quite as surprised that the correlation is poor, or even negative, given the way the data was collected and analyzed.<p>Absolutely agree that some MS degrees are pretty much less rigorous cash cows by now, that allow students to skip the fundamentals such as data structures, operating systems, and compilers.<p>However, many CS MS degrees actually do require this as a background, to the point where some programs have emerged to prepare non-CS majors for MS degrees, kind of like those post-bac premed programs. It&#x27;s hard to believe that those MS degrees, which require a decent GPA in those core courses, along with high GRE scores (sorry, but we are talking about interviewing skill, which may be more related to exam taking ability than job performance), wouldn&#x27;t result in a similar profile to people with CS degrees from top schools.<p>This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading.<p>That&#x27;s an aside, though. The value may very well be in the prep for these degrees (ie., the post-bac CS coursework required for admissions to a reputable MS program). If you can get that through online courses (udacity or coursera) through genuinely rigorous self-study? Yeah, that might do it, for far less money. I&#x27;ve audited a few of them, and they&#x27;re the real deal, that&#x27;s the real coursework there.

Interviewing is a funny thing.<p>I remember when I graduated from a &quot;Top School&quot; and interviewed at &quot;hot startups&quot; from the valley. I aced a lot of the interviews - why? Because I had just taken classes on LinkedLists, Binary Trees, HashMaps, etc... So when they asked me to whiteboard a &quot;shortest path algorithm&quot; it was just rehashing what I did in school.<p>Years later, looking back, I fail to see the relevance in most of the technical questions. In fact, if I had to do those questions over again today I would probably fail miserably. Yet, I have been in the industry for a while now and have worked with countless more technologies and have accomplished far more than my younger self.<p>Just because someone performs well in a technical interview doesn&#x27;t mean they will do a good job. That is the data that really matters. I&#x27;ve interviewed hundreds of candidates as a hiring manager for some big startups, and from my experience technical interviews are not a great indicator of success.<p>I&#x27;m saying this coming from someone who has gone to a &quot;Top School&quot; and done multiple Coursera&#x2F;Udacity&#x2F;etc classes.<p>Yes, someone might be able to whiteboard a random forest or write a merge sort, but do they know how to engineer a system? Can the candidate:<p>&gt; Communicate well with others in a group?<p>&gt; Solve unique technical problems?<p>&gt; Research and learn new technologies effectively?<p>&gt; Understand how to push back to product owners if there&#x27;s scope creep?<p>etc...<p>These are all things that are not really analyzed in many technical interviews.<p>As I&#x27;m reading this analysis all I can think of is that it is pretty useless - if not dangerous for the industry.<p>What I&#x27;ve found is that it is critically important that someone knows how to code at some basic level. But their ability to code and explain algorithms on the fly, while probably relevant in academia&#x2F;research, is such a minor part of the day-to-day of a programmer - At least from my experience.

The takeaway from this is that those who do best are those with:<p>- the wealthiest&#x2F;most financially supportive parents&#x2F;relatives<p>- upbringings that are conducive to academic success<p>- the most free time<p>as those are the ones who, by a large margin, attend top schools, work at top companies, and have time to spend on self-learning. Another data point of confirmation of a well-studied idea.<p>Assortative mating: <a href="http:&#x2F;&#x2F;www.economist.com&#x2F;news&#x2F;briefing&#x2F;21640316-children-rich-and-powerful-are-increasingly-well-suited-earning-wealth-and-power" rel="nofollow">http:&#x2F;&#x2F;www.economist.com&#x2F;news&#x2F;briefing&#x2F;21640316-children-ric...</a><p>Few poor at rich schools even all these years later: <a href="https:&#x2F;&#x2F;www.nytimes.com&#x2F;2014&#x2F;08&#x2F;26&#x2F;education&#x2F;despite-promises-little-progress-in-drawing-poor-to-elite-colleges.html?_r=0" rel="nofollow">https:&#x2F;&#x2F;www.nytimes.com&#x2F;2014&#x2F;08&#x2F;26&#x2F;education&#x2F;despite-promise...</a><p>Why people care about elite schools: <a href="https:&#x2F;&#x2F;medium.com&#x2F;@spencer_th0mas&#x2F;why-america-cares-about-elite-schools-411798386ba7#.yream7cae" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@spencer_th0mas&#x2F;why-america-cares-about-e...</a>

Not to harp on the &quot;technical interviews are disconnected from actual work!&quot; angle too much, but I&#x27;m reminded of a comment from a thread about the creator of Homebrew failing a Google interview. Someone pointed out that it goes to show that it&#x27;s possible to create widely-used software without an intimate knowledge of CS. I wonder if that&#x27;s a disconcerting fact for some employers to grapple with.

Until recently I worked at a startup as Machine Learning Engineer&#x2F;Data Scientist. There I got some experience interviewing people and looking at their resumes. In my experience, which is very limited compared to this post, people who put an MOOC on their resume are usually less qualified compared to people who don&#x27;t.<p>There is nothing wrong with MOOCs, but they are almost always beginner-level. If you put them on your resume it kindof implies you don&#x27;t have a lot of experience beyond that. Putting the Coursera Machine Learning course on your resume would be the equivalent of putting Java 101 on your resume for a Software Engineer.<p>I would recommend anyone to put projects on your CV instead. Even if you don&#x27;t have a lot of work experience, just put side-projects and school projects on there.

Interesting and surprising, especially the experience thing. I think I am a significantly better engineer than earlier in my career, so I assumed experience would count for a fair bit. Then again I have inherited projects from experienced guys who make crap high level architecture decisions and the code is way more difficult to work with than it ought to be.<p>But then this article seems to be measuring interview performance, not actual ability on the job. So is any of it actually relevant at all?

I wonder if &quot;took courses&quot; could be a stand in for &quot;prepared heavily&quot;. It seems like people with all the other attributes might think they didn&#x27;t need to study. People without them might think they did and took courses to &quot;catch up&quot;. In my experience, preparation is the key driver of performance in these types of interviews.<p>It seems reasonable that a person who took a MOOC might have prepared in other ways as well while people who didn&#x27;t probably didn&#x27;t prepare much at all (since watching a few Algo lectures seems the most accessible refresher.)

Top school is probably serving as a proxy for intelligence in this analysis...a well known predictor of both interview and actual job performance.

Unless I missed it in the article the data is all about passing the interview not acutally seeing if any of these things correlate to the employees working out in the 1,3,5 year time spans.<p>With this data you&#x27;re just biasing towards people who interview well, which, I don&#x27;t think you actually care about.<p>Well I mean I guess you do if you&#x27;re a recruiter (if you&#x27;re a moral recuiter you care about both), but not really if you&#x27;re an employer.

I think you are seeing the effect of people who have decided for themselves to pursue lifelong learning. The Udacity&#x2F;Coursera thing just clusters these people in a way that you notice them in the stats. But remember that statistics do lie. You need to dig into the reality behind the numbers, and question whether you are measuring all the right indicators.<p>My experience comes from several decades developing software and from time to time, hiring people. The people that worked out best, either as colleagues or hires, always seemed to be learning new things and were ahead of the curve trying out new techniques or tools before they became popular.<p>If you understand how a tool&#x2F;technique becomes popular as the mass of software developers wrestle with new problems and finally find a way to master them, then it makes sense that constant learning makes some people stand out of the crowd. They happen to be the first ones to learn the new tool&#x2F;technique and if they do not introduce it to their development team, then when management does make the decision to introduce it, the folks who know how to drive it have a chance to excel and appear to be rocket scientists.

Searched the article and the comments here for &quot;Pluralsight&quot;, with zero hits. So what makes Udacity&#x2F;Coursea preferable? TLDR, I&#x27;m asking this as Pluralsight was a significant contributor to my landing my latest role after redundancies.<p>The long version: I recently landed a role after some time off, having changed from mainly back end Php&#x2F;Coldfusion to C# in the last year. I was able to make the switch in my last role. For me, moving to C# was a big transition; as well as guidance from a (fantastic) mentor, I used Pluralsight to learn C#, asp.net and DDD - e.g. from Jon Skeet, Scott Allen and Julie Lerman, to mention but a few.<p>Being completely burnt-out on the old stacks, I was set on making my next role a C# one. I&#x27;ve come to love what Microsoft are doing with Core, open sourcing etc, as well as the strictly typed C# language and ability to use NCrunch with live unit tests. So I signed up for a year after relinquishing my corp subscription, kept doing their courses, and found the training material highly accessible with great quality content. Each interview was a learning process, when I didn&#x27;t know something from a test, I&#x27;d go and study it so that I&#x27;d be better prepared for the next role. One of these was the study of data structures and basic computer algorithms, where I was lacking. I might not have had years of experience, but the experience I had was mostly best practice.<p>During my search, I typically got great feedback on the fact that I was doing Pluralsight courses, and it was a significant factor in being hired for the new role - it showed cultural fit, in addition to passing their tech tests (which happened to involve structures). My company had interviewed a <i>lot</i> of candidates, struggling to find the right talent. Just possessing technical skills is one thing, having the right attitude towards learning is another.<p>At any rate, I&#x27;ll keep using Pluralsight to raise my proficiency in my new stack - even as an old timer, I am having a newfound level of enthusiasm towards my whole profession which I haven&#x27;t felt since I coded in assembly on the good old Amigas. I would be interested in knowing why Coursera &#x2F; Udacity might be better or more accepted in the marketplace though.

It&#x27;s rather shocking how much effect Udacity&#x2F;Coursera had on interview performance - more than graduating from a top school or being employed at a top company:<p>&quot;...only 3 attributes emerged as statistically significant: top school, top company, and classes on Udacity&#x2F;Coursera.&quot;

The master&#x27;s in CS can be useful if:<p>1. You have an undergrad degree in liberal arts 2. You pay as little tuition as possible 3. You take no time off and continue to work FT<p>These apply to me -- my undergrad was in English, I paid 6k total (27% of the 21k total cost) and went to school at night over 4 years while my career continued to progress.<p>Most of the people in my program couldn&#x27;t write a FOR loop if their life depended on it, they viewed it (incorrectly) as a jobs program while the school needed the $$ to keep the dept afloat, so I&#x27;m not surprised they fared poorly in technical interviews.<p>But that doesn&#x27;t mean the degree isn&#x27;t useful. If you&#x27;re already a programmer, it helps get your foot in the door at many places. HR managers&#x2F;recruiters feel more confident forwarding on your résumé, they can&#x27;t parse your GitHub repos.<p>The degree is icing on the cake, it&#x27;s not going to magically turn you into the Cinderella of Programming if you have no real-world experience. I got my master&#x27;s with a QA and a paralegal and today? They&#x27;re still a QA and a paralegal.<p>That being said, timed technical interviews are almost universally asinine, IMHO. When in real life do you have 10 minutes to figure out a problem? Or are prevented from Googling the answer? The measure of successful programmers is how efficient and professional they are in problem solving, not how much useless information they can keep in their head.<p>Things I&#x27;ve never had to do in &#x27;real&#x27; life: -Never had to split a linked list given a pivot value -Never had to reverse a string or a red&#x2F;black tree -Never written my own implementation for Breadth First Search<p>etc etc<p>Personally I&#x27;d rather see take-home assignments that roughly approximate the type of work you&#x27;d do, which in my career has been churning out new features or applications. Does knowing the time-complexity of radix sort vs heap sort really have a material impact on your effectiveness as a programmer? No.

It should be noted that these technical interviews are biased to a particular style, so the data only really is of relevance for these types of interviews.

On the master&#x27;s front, I went down a slightly unusual path. I enrolled in a master&#x27;s program in music technology at NYU [1]. I already had a master&#x27;s in engineering from Princeton [2], but after time away from the software world, I wanted to retool for a return to engineering, but with a focus on applications that actually mattered to me.<p>It turned out to be a very expensive, but very fulfilling decision, and it paved a route for a very successful past four years.<p>Compared to my first master&#x27;s, it was less theoretical and much more project-based. In that sense, it was fantastic preparation for career work, because every semester, I had to conceptualize and ship 4-5 different projects in all sorts of subject areas. The value of that shouldn&#x27;t be underestimated. It also directly led me to cofounding a startup that had a brief lifetime, but effectively converted me to a full-stack engineer.<p>Today, I don&#x27;t use much of the subject matter I learned in my day-to-day, but I draw on the creativity, problem-solving skills, and work patterns every day.<p>My Princeton program was great too, but I thought I&#x27;d share about the NYU program, as that was the more outside-the-box choice. There&#x27;s something special to be said for a master&#x27;s degree, when it&#x27;s interdisciplinary and let&#x27;s you focus on the intersection of engineering skills and subject matter expertise.<p>[1] <a href="http:&#x2F;&#x2F;steinhardt.nyu.edu&#x2F;music&#x2F;technology" rel="nofollow">http:&#x2F;&#x2F;steinhardt.nyu.edu&#x2F;music&#x2F;technology</a><p>[2] <a href="http:&#x2F;&#x2F;ee.princeton.edu&#x2F;graduate&#x2F;meng-program" rel="nofollow">http:&#x2F;&#x2F;ee.princeton.edu&#x2F;graduate&#x2F;meng-program</a>

yes, this is absolutely startling:<p><i>For people who attended top schools, completing Udacity or Coursera courses didn’t appear to matter. (...) Moreover, interviewees who attended top schools performed significantly worse than interviewees who had not attended top schools but HAD taken a Udacity or Coursera course.</i><p>Possible explanation might be that people going through regular degree typically spread themselves thin over many subjects (digital electronics, compiler design, OS theory, networking etc) while MOOC folks sharply focuses on exactly the things for interviews (i.e. popular algorithms). Its like interval training for one specific purpose vs long regime for fully rounded health. The problem here is not academic system but how we measure performance in interviews. I highly doubt if results would be same if interviewers started asking questions from all these different subjects instead of just cute algorithm puzzles.

We really need a further correlation between people who pass the interviews and job performance a year later. I do a lot of interviewing at my current job and we have found no strong correlation at all between CS skills and actual ability to &quot;get things done&quot;.<p>We toned down the CS type questions since they tend to take too long. We still ask a few basic tree and string manipulation questions to weed out the people who have no idea how to program and get insight into how the person thinks.<p>I still feel at the end of the day we could flip a coin on accepting an interview candidate once they have shown basic competency and have the same results.<p>I have been telling candidates that a public github repo with a nice commit history carries much more weight with me then a CS degree since we have been burned so many times before.

<p><pre><code> If you know me, or even if you’ve read some of my writing, you know that, in the past, I’ve been quite loudly opposed to the concept of pedigree as a useful hiring signal. With that in mind, I feel like I owe clearly acknowledge, up front, that we found this time runs counter to my stance. </code></pre> Did the interviewers have access to the applicant&#x27;s resume? If so, to what extent do these results simply reflect the interviewers&#x27; bias for top schools and famous companies?

While I do think that interviewing is broken, I would love to see the raw data with this. For example, did Udacity courses have other related traits associated with them, IE: did these candidates that also have a certain number of years of experience, degree etc.? 3000 is a small sample size and I&#x27;m wondering if there is some sampling bias here.

Something I wonder is how the participants in these interviews were selected from the general population of job candidates. Painting with a broad brush, the best workers might not even be candidates, because they&#x27;ve already been hired. And the best candidates might be the least likely to seek coding interview practice.

I conducts lots of tech interviews for SWE positions, and as everybody&#x27;s boning up on algorithmic trivia, I&#x27;ve learned that I can get a stronger hiring signal by asking <i>simpler</i> questions that people with an aptitude for programming will succeed on and people with an aptitude for memorizing the implementations of algorithms will not.<p>(Simple example: given two closed intervals [a..b] and [c..d], how do you compare the four values to determine whether or not the intervals overlap? You may laugh, but it defeats about 50% of candidates in the first minute of an interview because they just don&#x27;t understand simple relationships and Boolean expressions.)

I thought the most interesting finding was that completing Udacity or Coursera courses on programming&#x2F;algorithms (for non-top school graduates) was highly predictable of strong interviewing performance.

Interestingly, they suggest that if you attend a to school, the effect of Udacity is negligible. I&#x27;d argue that Udacity is this filling in gaps of a poor education.

I have been through an interview process many times; I have never been asked a technical question &#x2F; asked to do on blackboard or on computer problem solving.<p>I guess that that form of interviewing is simply not common in my neck of the woods (Denmark).<p>I am curious to what sort of questions &#x2F; tasks are actually given to the interviewee?<p>And are they in any way biased towards more textbook&#x2F;academic ones? (I.e. &quot;implement bubble sort&quot; rather than &quot;create a blue button&quot;).

One of the reasons I got hired by Airbnb is that I took MOOCs, but I also believe that most of my knowledge comes from reading books and that&#x27;s a thing I didn&#x27;t put on my CV. So, even if showing interest in learning opens for you a huge amount of opportunities, I think you actually have to go deeper than just enrolling on a couple of MOOCs.

I feel like things are operating according to the following pattern:<p><pre><code> 1. Go to college: a. spend many semesters in lectures all of which tangentially brush upon the final exam based on the whims of the lecturer. b. cram for final exam last minute panic to crunch memory according to advice on content which was brushed upon during lectures. 2. Interview for job: a. cram for interview by going to coursera to crunch memory according to interview memes based on the whims of the interviewer. b. spend the rest of term of employment exercising skills which tend to be tangentially brushed upon during both interview and schooling while the majority of actual tasks are often googled and stack-overflowed into place based on arbitrary design decisions and politicized stack choices. 3. Results: a. good interviewees have learned appropriate memes to reassure interviewers. b. good students have learned obligatory cruft to reassure professors. c. actual necessities are tangential to many or most entry barriers. </code></pre> How accurate is this?

It was a very good reading, but I wonder how interviewing performance relates to job (&quot;real&quot;) performance?

When you are interviewing for a specialist post (and most posts are specialist to some degree) you are looking for evidence that the candidate can do that particular job. Therefore a course that indicates that they have the particular skills required is highly desirable!

I&#x27;m not surprised that MOOCs are a big factor, people like me who have left school years ago have forgotten how to write a BFS, we need something to brush up those knowledge.<p>If you run statistics against using sites like careercup, you may find that being the top factor.

Huh, I hadn&#x27;t bothered to list MOOCs on my resume since I didn&#x27;t think employers would be interested, maybe data like this will make employers more interested in the courses, which would probably get more people to shell out for the certificates.

Basically a &quot;bad&quot; programmer that can&#x27;t write maintainable code that prepares for technical interviews by brushing up on algorithms and whiteboard style questions will do better than a very good programmer with lots of years experience.

<i>&quot;We got this data from looking at interviewees’ LinkedIn profiles.&quot;</i><p>Verification of completion and award id? There are a lot of individuals who will add a degree regardless of attendance, completion or award. Who validates the assertions?

Very unsurprising for me. You are measuring your ability to solve algorithm puzzles. Most engineers don&#x27;t actually do many algorithm puzzles in day-to-day work, especially the types of algorithms that interviews tend to focus on like sorting and dynamic programming. So &quot;years of experience&quot; is not measuring experience in what you&#x27;re actually being tested on. On the other hand, you do exactly those types of things in many CS classes, and in Coursera classes, algorithms are exactly what you practice. So it makes sense it correlates.<p>Top company is a predictor for the obvious reason - it&#x27;s selection bias for people who already passed those interviews at the company. You&#x27;re not good at the interview because you worked at the company, you work for the company because you&#x27;re good at the interview.<p>Master&#x27;s degrees seem like largely for international students needing visas, career switchers, etc so not surprised they are not a strong predictor. And if anything the course material moves past the intro data structures stuff the whiteboard interviews tend to test.<p>The only huge surprise for me here is that Coursera is a stronger predictor than top company and top school. I would have predicted top company &gt; top school &gt; Coursera.<p>The post that I would be much more interested in is correlating performance reviews to interview performance. That gets suggested as a possible future post.

Sad to see so much detail paid to the data, and so little to the setup of the experiment itself.<p>It shouldn&#x27;t be surprising that an online technical screen favors candidates who&#x27;ve participated in a MOOC, but is blind, say, to years of experience. A screen like this is timed-performance-at-a-distance, which resembles MOOC participation. The full spectrum of qualities that comprise a Good Hire might incorporate the other signals from the post, but this type of interview won&#x27;t test them.<p>(I&#x27;ll be the first to admit I&#x27;m biased against performative coding in engineering interviews. Tech screens like this are often necessary, though, so they have their place.)

Was there any kind of statistical correction applied when the data were partitioned into MOOC + top school vs MOOC + no top school?

Slightly Off-topic, but does anyone have an invite for this platform? I have been trying to get one since ages.

That graph is about the best evidence I&#x27;ve seen that interviews are garbage. Years of experience doesn&#x27;t matter at all? Coursera courses are the best thing ever?

This is a very poorly done analysis. At a minimum she needs to define top school&#x2F;top company. Also I&#x27;d like to see the confidence intervals around the effect sizes. In addition, looking up MOOC information from LinkedIn may result in a lot of false negatives. (She doesn&#x27;t mention if MOOC courses in non-CS subjects count.) Did all the interviewees have CS degrees? What about the Masters degrees, is she including non-CS ones? Is the sample of interviewees representative or there&#x27;s any selection bias that we should be aware of?<p>A study which doesn&#x27;t answer so many basic methodological questions is garbage.

&quot;I’m excited to see that what mattered way more than pedigree was the actions people took to better themselves.&quot;<p>so a degree from a top school is not earned (nor are admissions i guess), but rather conferred at birth? i beg to differ.<p>the commentary on the &quot;disutility&quot; masters degrees is even worse.

Is this guy a paid shill for academic friends trying to boost enrollments and overcome the disillusionment of the younger people who realize too much emphasis is placed on academics and not enough on practical application?<p>The world needs more vocational schools and trade schools and technical schools than it does colleges and universities.