Can a simple algebra test predict programming aptitude?

Why do you care about it in the first place? Also what is your definition of good programmer? Just make a function? Make a class? Make a big app?<p>IMO you don&#x27;t need to learn too much math before programming to be a good programmer. Basic operations are good enough, if you need more you learn as you go, you&#x27;ll never know what you need. Lazy learning, like lazy variable loading.<p>The test you are making are more like problem solving which is closer to what programming is and not math. Of course basic math (+, -, &#x2F;, *), but what it really matters is knowing what operations to do. The operation itself is the easier part.<p>A better test would be to make a chose between which approach to take to solve a problem and explain the rational behind. That is what will define a good programmer.<p>Also what better indicator you want than programming itself? You&#x27;ll never make a choice based on this.<p>Also one thing is to figure out how to solve &quot;here are 3 consecutive integers with a sum of 69. What are they?&quot; than solving &quot;here are X consecutive integers with a sum of Y. What are they?&quot;. Same in &quot;Adriana’s age is 1&#x2F;3rd of her dad’s age. If her dad is 36 years old, how old is Adriana?&quot; vs &quot;W’s age is X of Y’s age. If Y is Z years old, how old is W?&quot;. The complexity increases a lot.

For an article about math, they seem surprisingly weak at differentiating between causation and correlation.<p><i>It’s fairly common for incoming Computer Science majors to ask the question, “Why do I have to learn all this math if I just want to learn to program?” The correlation above suggests a possible answer: The ability to understand basic mathematics is likely correlated with the ability to “think algorithmically,” which is well-known to be a foundational skill for expert programmers.</i><p>If it&#x27;s just a weak (as the chart suggests) correlation, it doesn&#x27;t mean learning one will make you better at the other. And if you do assume causality - it can go either way. Perhaps learning math makes you better at programming. Perhaps learning programming makes you better at math.<p>My 2 cents... It&#x27;s more complicated because there are other things involved. Perhaps it&#x27;s the nerd gene that makes people who like computers also more likely to play D&amp;D and be in band. (I was a card carrying member) Does one of these 5 variables cause the other, or are they all part of the same thing?

If there is one thing that has always bothered me, is people evaluating student&#x27;s prowess in &quot;Computer Science&quot; (which is not programming, shame on you) by having them learn your normal run-of-the-mill programming language&#x2F;paradigm like Javascript, Python, PHP and whatnot and then claim they are not apt to program because they cannot solve problems or handle variables, mutability, etc etc.<p>There are A LOT of different programming&#x2F;automation paradigms in the world, lots of different languages, declarative, functional, OOP, imperative, logic, etc etc, maybe some of these students just can&#x27;t get the convoluted mutable and unsafe structure of a Python or Java program but they might grok and appreciate the safety and simplicity of ML&#x2F;Haskell or the flexibility of Lisp, why would you turn them down in such a way or force them through a path they don&#x27;t really need to go through?<p>During my first year of Bachelor Computer Science at University we had introduction to programming in Scheme. It was a mind-expanding experience to me, I already knew how to program in C, Java, C++ and C# with a bit of PHP&#x2F;Javascript back then and I was much ahead most of my peers but still I sat at the computer, trying to earn this alien language to me, and luckily I was humble enough to force myself through it and adapt my mind to a different paradigm. I know for certain that a lot of my friends and colleagues who already knew how to program struggled hard on that exam and some had to re-take it the following year (Which was unfortunately changed to C++ and they managed to pass without troubles) simply because they didn&#x27;t think Scheme was a &quot;real&quot; programming language or useful at all. It was just too different from their previous experience.<p>What&#x27;s even more interesting, I know some people from that class who had never touched a programming language before and weren&#x27;t particularly strong at math. Those people are the ones I recall enjoying the course the most, they found it the easiest among all the other courses we had and passed it with excellent grades. Simply because their mind was apparently better wired for such paradigm and they had no preconceptions or prejudices that prevented them from learning it properly.<p>So my bottom line is, what makes you think that some people might not just have a &quot;differently wired&quot; brain that makes them think more easily with a different non-imperative paradigm for programming?

The scatter plots don&#x27;t show that strong a correlation.<p>From the test questions, this isn&#x27;t an &quot;algebra test&quot;. It&#x27;s a word problem decoding test. That&#x27;s appropriate to programming, where you have to go from an informal specification of the problem to a formal one.

Does a &quot;simple algebra test&quot; correlate with IQ? Does &quot;programming aptitude&quot; (whatever the hell that is meant to be) correlate with IQ? The headline would then follow.<p>I know this is a socially unacceptable opinion but I think those correlations exist.

This article is built on a fundamentally invalid premise:<p>&gt; 30-60% of CS college majors have failed their Introduction to Computer Science course because they simply could not learn to program.<p>This &quot;You either have it or you don&#x27;t&quot; mindset is strikingly elitist. I simply don&#x27;t buy the idea that there are people out there who absolutely cannot learn to program. I mean, put someone on a desert island and tell them they can&#x27;t leave until they can write a program that uses a linked list, and I&#x27;m pretty sure most people would be able to get off the island eventually.<p>Of course, that doesn&#x27;t mean that some people can learn to code more easily than others. But I&#x27;d be willing to posit that the vast majority of people could learn to code given enough time and the proper instruction.

I am not great at higher level Math (calculus) and I&#x27;m good with algebra but I do not think I&#x27;d consider those good predictors. I feel like what allows me to program is strong spatial reasoning. Doing a quick google search I found this: <a href="http://www.quora.com/Whats-the-nature-of-the-relationship-if-any-between-high-spatial-reasoning-skills-and-programming-aptitude" rel="nofollow">http:&#x2F;&#x2F;www.quora.com&#x2F;Whats-the-nature-of-the-relationship-if...</a><p>Probably just anecdotal but curious to know what others feel about spatial reasoning? I think specially with object oriented programming it is helpful to be able to visualize all of the &#x27;actors&#x27; in your head and how they relate&#x2F;intertwine.

What? Where&#x27;s the correlation? The data looks like a random scatter plot to me. It reminds me of this Physics report from ages ago...<p><a href="http://pages.cs.wisc.edu/~kovar/hall.html" rel="nofollow">http:&#x2F;&#x2F;pages.cs.wisc.edu&#x2F;~kovar&#x2F;hall.html</a>

An R^2 value of 0.33 does not even come close implying linear correlation.

Basic algerbra is used all the time in programming. It&#x27;s part of a programmer&#x27;s fundamental skill set. Of course, somebody who already has that particular skill will definitely have the advantage in a short programming course. It&#x27;s like starting a 100m sprint with a 3 second head start. It says nothing, however, about who will be the better programmer over the long term. So I don&#x27;t think their test is a good predictor of programming aptitude.

I&#x27;ll reply to a specific comment already made as a subcomment here, to comment about the larger issues that have come up in several top-level replies in this thread.<p><i>I don&#x27;t think such a hypothesis is awkward but I doubt high IQ as an excellent predictor for an individuals performance in tasks that are not IQ tests. As a layman when it comes to psychology I think IQ measures some things but the way the brain works, no real world task a person does is exactly like those IQ tests.</i><p>The interesting blog post submitted here is talking about the bread and butter of &quot;industrial and organizational psychology,&quot; namely about how to select individuals for a training program. There are three generations of published research on this topic already, and there is a huge amount of ongoing research on this topic, because organizations all over the world want to figure out how to select successful applicants when there are more applicants than places in school or work programs.<p>The short answer is that there is a HUGE body of research to show that the single best hiring process you can use for hiring a worker, if you want to get a worker who will perform well on the job, is to use an IQ test for hiring.[1] The long answer is that some other applicant characteristics matter too, of course, but the single best thing to look at in a job applicant is &quot;general mental ability.&quot; Work-sample tests are also very good for hiring for specific jobs, and are grossly underused in hiring in the United States.<p>To the point of the interesting submitted blog post, one always has to be empirical about these issues. The people running the bootcamp so far have found data that suggests that the algebra test they have tried is a bit more revelatory than the IQ test they tried, and less expensive besides. One response to that might be to suggest a test like the Wonderlic test (an inexpensive IQ test designed for company hiring procedures) but in the end, results matter. If empirically at this bootcamp, the algebra test works better than some other selection procedure, it doesn&#x27;t even really matter why it works, just that it serves the bootcamp&#x27;s purpose of identifying successful students from among multiple applicants. The data set is still small. I am very glad that the blog post includes a scatterplot of the data. More bivariate data should be shown that way, in blog posts on dozens of topics.<p>[1] My FAQ post on company hiring procedures, which I am still revising to put on my personal website after composing it for Hacker News, provides references for this and some commentary on legal issues in hiring.<p><a href="https://news.ycombinator.com/item?id=4613413#up_4613543" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=4613413#up_4613543</a>

I can&#x27;t dispute the author&#x27;s statistics, after all, they&#x27;re just a function of the data. But looking at the first graph, it seems like a generous interpretation of &quot;predict.&quot; The anticipated signal-to-noise for any individual student looks like it would be pretty poor.<p>One other thought occurs to me. Would the same algebra test produce similar correlations with courses such as English and History, when adjusted for the differences in overall pass rates for those subjects?<p>It seems odd that something as interesting as programming isn&#x27;t introduced until college. That seems like accepting students as music majors, who have never played music before. If I had my druthers, computation and simplistic programming would be part of the mainstream K-12 curriculum.

&gt;It’s fairly common for incoming Computer Science majors to ask the question, “Why do I have to learn all this math if I just want to learn to program?”<p>Well, there&#x27;s the reason you have a high drop-out rate in CS. People don&#x27;t know what it is! Computer Science is not a vocational program. Computer Science is not computer programming.<p>If someone had talked to these kids asking “Why do I have to learn all this math if I just want to learn to program?” and told them &quot;You don&#x27;t have to learn all this math if you just want to learn to program&quot; before they became CS majors, then they might not have become a CS major in the first place.

Most programming jobs involve a lot of algebra. You have variables, you do arithmetic to those variables, you apply functions. It&#x27;s not a great surprise to me that if you are good at high school algebra, you can probably learn how to program.<p>You might not be a great programmer, but you&#x27;ll be able to do it.

Lot of people in here offering cheap opinions, while mgirdley&#x27;s team has done the actual work. They&#x27;re not saying that the kinda of algebra test they&#x27;re proposing is foolpoof, perfect, subjectivity-free, or even the best way of testing for programming aptitude.<p>What they _are_ saying is that the test is an efficient, effective way to do it, with better ROI than IQ tests.<p>It doesn&#x27;t actually matter _why_ that&#x27;s the case, though Animats&#x27; suggestion seems plausible: &quot;From the test questions, this isn&#x27;t an &quot;algebra test&quot;. It&#x27;s a word problem decoding test. That&#x27;s appropriate to programming, where you have to go from an informal specification of the problem to a formal one.&quot;

I think that many people have difficulty learning to code in college because many of them aren&#x27;t that enthusiastic about it. At least anecdotally, many people at my school are in computer science because that&#x27;s where the zeitgeist is. If it was the eighties they&#x27;d be getting ready for law school.<p>I think that people would do much better in intro CS classes if we could get rid of some of the hype around programming and tech in general, and treat it like other courses. That way, far more of the students would be genuinely interested in the subject.

I&#x27;m very good at Algebraic math. But I don&#x27;t think an algebraic test will show programming aptitude. While algebraic and calculus analysis are a form of problem solving that works incredibly well. It is by no means the fastest, or most eloquent way of solving a problem. Its just a single tool that a problem solver can use.<p>The main thing I fall back on is Heron&#x27;s Problem. The classic algebraic&#x2F;analytical solution is to find the local minimum of a function. Which works, this method isn&#x27;t flashy yet completely functional. It requires no insight, just mentally vomiting something you learned in high school&#x2F;college.<p>The geometric solution is far simpler, and offers insights into the foundation of trigonometry. Its really so simple and eloquent you feel like an brutish idiot doing a long form analysis.<p>I offer this because tools are merely tools, and math supplies many tools. You wouldn&#x27;t hire a finish carpenter just based on his&#x2F;her ability to swing a sledge hammer would you? I mean s&#x2F;he will have to swing a hammer, but will it have to be a sledge hammer? Will they have to use a nail gun? Screw driver? I hope they are proficient in all of these tools, and since I&#x27;m outsourcing my time to them, I hope they know which tool to use at which time.

The correlation doesn&#x27;t seem particularly strong, but the dependent variable here isn&#x27;t right. It&#x27;s a stack ranking that&#x27;s forced to a normal distribution. A normal distribution might be appropriate if they randomly selected US citizens to join the program, but they don&#x27;t. Students self-select to apply and they reject the bottom 66% of applicants. The resulting talent profile is far from being normally distributed.

You can correlate most anything with most anything. I&#x27;m not saying don&#x27;t try, just to don&#x27;t give correlations much credence in and of themselves. Thinking globally, what are the odds a coincidence doesn&#x27;t happen? And when you exclude outliers, you may be excluding the most profound subjects, just as when you include non-outliers, the subjects may not be able to generalize outside the system of measurement.

&gt; Despite hours of studying and tutoring, most of these underperforming students struggle with, and many ultimately give up on, programming as a career<p>They should follow the lead of those who figure out earlier they&#x27;re not cut out for programming: embellish the CV with fake study, and go straight into the workforce as a programmer. Or they could get transferred into programming from some user department pitching to bring a &quot;valuable user perspective&quot; to IT. Or grease up some IT manager after-hours who&#x27;ll bring them in as contractor with &quot;special skills not readily available in the labor market&quot; to bypass the usual HR checks and aptitude filters. Or if they&#x27;re not of the same ethnicity as the HR personnel, send in a double to sit the aptitude test for them, knowing HR staff don&#x27;t check applicant ID&#x27;s too closely because they know no-one&#x27;s going to make it through to an IT interview if they do. Most HR staff and IT managers just want the staff count up, they don&#x27;t care whether they&#x27;re productive or detrimental to the projects. If they have staff they have people to blame.

So I guess the real underlying message they are trying to say is, if a student can&#x27;t pass an algebra test, they probably can&#x27;t learn to program, so don&#x27;t even bother trying to teach it to them. A more generalized form of the hypothesis is that: most people can&#x27;t learn how to program.<p>Both forms of the hypothesis are pretty easy to refute and have already been refuted. See <a href="http://code.org" rel="nofollow">http:&#x2F;&#x2F;code.org</a> and <a href="http://madewithcode.com" rel="nofollow">http:&#x2F;&#x2F;madewithcode.com</a> for example. I just taught 2nd graders programming yesterday who don&#x27;t even know how to divide, let alone any algebra.<p>And of course, there&#x27;s always the radical idea that maybe you could actually teach them some algebra skills in the context of programming tasks. Just like some calculus students may need some remedial instruction and support.<p>Luckily, there&#x27;s a whole field of research with several articles on this very topic. The field is called computer science education. See SIGCSE, for example: <a href="http://www.sigcse.org/" rel="nofollow">http:&#x2F;&#x2F;www.sigcse.org&#x2F;</a><p>In fact, the field has already researched and debated this issue before, as well. There was a controversial article called &quot;The camel has two humps&quot; in 2006 that made the same claim as this post that a simple aptitude test could predict whether someone could learn how to program or not. The article (which was never even officially published) was later retracted: <a href="http://retractionwatch.com/2014/07/18/the-camel-doesnt-have-two-humps-programming-aptitude-test-canned-for-overzealous-conclusion/" rel="nofollow">http:&#x2F;&#x2F;retractionwatch.com&#x2F;2014&#x2F;07&#x2F;18&#x2F;the-camel-doesnt-have-...</a>

An older study suggests :<p><i>In both genders,performance on Mathematics was found to be the best predictor in programming ability,followed by performance on a spatial test.</i><p><a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.110.9842&amp;rep=rep1&amp;type=pdf" rel="nofollow">http:&#x2F;&#x2F;citeseerx.ist.psu.edu&#x2F;viewdoc&#x2F;download?doi=10.1.1.110...</a>

Surely the confidence bands in the plot aren&#x27;t 95% confidence intervals...<p><a href="http://uhlbcmcbx4a4vt1d.zippykid.netdna-cdn.com/wp-content/uploads/2014/12/898x640xPredictProgrammingAptitude1.png.pagespeed.ic.KzRMOyK9kJ.png" rel="nofollow">http:&#x2F;&#x2F;uhlbcmcbx4a4vt1d.zippykid.netdna-cdn.com&#x2F;wp-content&#x2F;u...</a>

R-Squared of 0.33?? Not a great fit, considering the sample size.

How is programming aptitude measured and defined?

I&#x27;d be interested to see more questions from the test, but from the examples given I&#x27;d say s&#x2F;algebra test&#x2F;problem solving test&#x2F; is in order.<p>In that light, the findings become less surprising---still interesting though, since we can test math-word-problem-solving ability quite easily with pen and paper. Could this mean my KhanAcademy math achievements are a better hiring signal than my github?<p>In terms of hiring, it certainly makes more sense for candidates to pass a 1h test rather than expect them to &quot;prove themselves&quot; by solving a week-long coding challenge...

<a href="http://en.wikipedia.org/wiki/Betteridge%27s_law_of_headlines" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Betteridge%27s_law_of_headlines</a>

<i>we’ve possibly found a simple method for sorting out the ninja programmers from the less capable programmers</i><p>Can we please stop calling them ninja&#x2F;rockstar&#x2F;etc programmers and simply use &quot;professional programmers&quot; or &quot;master developers&quot;? This language is cute when you&#x27;re 16, I guess, but personally I don&#x27;t want to be compared to a middle-ages asian hitman for hire.

Aptitude tests really annoy me. At least for me, when I was told that I was bad at math, it discouraged me and I didn&#x27;t pursue it. Later when I was all grown up and decided to go back to school, I really applied myself and it turns out I was pretty good at physics and math, despite my aptitude when I was a teen.

I&#x27;ll only comment on the opening line and the research it points to about the failure rate in intro to Computer Science. The author of the codeup piece immediately jumps to the conclusion that it&#x27;s an inability on the part of the failing students to be able to learn to program (a detail that the study linked doesn&#x27;t comment on). However, my first reaction is that if the failure rate is higher than we want, it means we either (1) aren&#x27;t teaching the class well; or (2) the students are coming in without proper preparation.<p>Many students coming into an intro to CS class are already programming whizzes, and then there are others who have no experience with programming at all. So, already the instructor is in an impossible situation. Do you bore the promising kids with the major head start, risking losing future good students? Or do you ramp the class up to their speed and risk losing the kids who didn&#x27;t come in already knowing the material? Obviously, most departments and instructors will, wittingly or not, choose the latter.

All of this is predicated on the idea that programming is solving sexy math problems.<p>Reality is that most programming is taking bits out of one bucket, combining them, and dropping them into some other bucket. It&#x27;s the digital equivalent of shovel work.

&quot;Programming&quot;?<p>What kind of &quot;programming&quot;?<p>What &quot;ability&quot;?<p>Ability to write space-O(my_god * n) sorting algorithms? Ability to duct-tape queries to an ill-documented SOAP webservice and upsert the return into a Your-Boss-Normal-Form database schema? Ability to brutally optimize the runtime of a really hairy numerical analysis algorithm? Ability to design a centralized data pipeline architecture and lead 12 hackers during the implementation and migration? Ability to find the off-the-shelf OSS project that solves the first 50% of the problem instead?<p>I was watching the news the other day on TV, which is something I almost never do, and there was a segment about the educative value of tablets, smartphones and laptops for teenagers and children. The value of the freely available information on the internet was discussed, along with the quality of free and... not free educative and scientific applications, and the ill effects of such devices on people&#x27;s attention span, and... That&#x27;s it.<p>Example devices included many, many Apple products, a few off-the-shelf Android devices. A Windows machine running MS Word.<p>Same ideology regarding technology in education: locked-down workstations with user-friendly applications, internet access which blocks every port except for 80 and 443.<p>A relentless and sustained effort to erase everything but the topmost layer of the IT stack. Locked-down devices. People come to me asking me to remove viruses, to speed up their old XP machines fraught with annoyware. No Windows license, don&#x27;t want to pay for one, don&#x27;t want Linux. Computers run on magic, right? How can you know that programming stuff and not be able to remove all that <i>bad magic</i> from my computer. Computer Science students who can&#x27;t find the slash on their keyboards. People who see computers as a monolithic entity rather than a brittle but transparent stack of conceptually distinct layers. The new generation of systems administrators, can&#x27;t scp a tar at the other end of the lab.<p>A teacher spending three hours explaining red-black trees to a class scrolling down 9gag; students, diploma in hand, who aren&#x27;t entirely sure what&#x27;s the difference between an array and a hash, who are not sure of what happens when one puts an array in another array, in PHP, after 120 hours spent theoretically writing PHP.<p>Computer Science students convinced that Linux is 100% not worth learning, in any way, because nobody uses it.<p>This is how the knowledge economy dies.

Appreciate the idea behind the article which may have value, but the data types seem ordinal? 1.0-4.0? is that trying to mirror US GPA system. Normalized based on what? p-values are an archaic only useful for MBAs and Med students.

Don&#x27;t have the study handy, but measures of working memory such as the OSPAN were found to predict proficiency in a programming course even better than an algebra test that participants also took.

It would be interesting to administer these algebra tests to junior&#x2F;mid&#x2F;senior level programmers currently in the field.

One thing I&#x27;m a little uncertain of is the ideology behind these kinds of things, it seems like the primary purpose here is to keep people out of the field.<p>Also I think those are some pretty weak quantitative results, and good exhibits for argument against p-values.

Is the complete algebra test available somewhere?

no

No.

I really don&#x27;t want to sound rude, but this sounds like a classic case of correlation not causality, and I could see the consequences really hurting some people.<p>I am REALLY terrible at algebra (to the extent that I am borderline LD in math), and I have completed a degree in CS and hold a steady job as a software engineer. And I am definitely not the only one. Ask a room full of developers how many are bad at math, and I guarantee the results will surprise you. On the other hand, I know a plenty of folks I went through school with that were math majors so good at algebra they could do full page derivations piss drunk without at hitch. And yet, they would take an intro java course and be totally lost. If they didn&#x27;t get a java intro class, how do you think they would have done with something as algorithmic as an assembly language, a functional language, or understanding the nuts and bolts like Turing machines and automata?<p>Algorithmic thinking is definitely an important part of programming, but it is just that, only a part. And quite a few types of development don&#x27;t emphasize that sort of thinking. I could perhaps see this being a PARTIAL solution for areas where one would be doing a lot of functional programming, but the vast majority of the job market these days is still OOP.<p>This could really quash the job prospects of programmers who are perfectly capable of writing quality code but are poor with algebra.