Don't be fooled by the October 2017 date of this article. There should be a 2014 in the title, since this appears to be a re-print of a prior study:<p><a href="http://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf" rel="nofollow">http://web.cs.ucdavis.edu/~filkov/papers/lang_github.pdf</a>
Although this is interesting, we should perhaps not hold a high degree of confidence in the results, because the methodology relies on the content of commit messages and the number of commits (if I read this correctly).<p>- The content of commit messages varies widely in expressiveness and meaningfulness, from 'Fix bug' to a detailed explanation. This confounds the classification of a commit.<p>- The number of commits can be very misleading depending on the committer's workflow. Some committers merge topic branches to include all their intermediate work in progress commits, which could overrepresent commits flagged as errors. Other committers rebase their topic branches into fewer, or even single, commits before merging. Or, some commits may fix multiple defects.<p>This kind of analysis is conceptually a worthy endeavor; it would be more meaningful if the metrics it employed were more strongly correlated with the attributes it was trying to analyze.
Figure 2 suggests another possible bias favoring functional, managed languages: a lot of errors for C/C++ are related to concurrency and performance. But those are mostly non-bugs for other languages, since when concurrency or performance are a requirement then most of those studied languages would not be considered anyway.<p>It seems similar to the paradox that makes the best medicine appear to have a lower survival rate just because it's given to most serious patients.
Uh, where is it shown how "software quality" or "code quality" is measured or determined? Can anyone provide a succinct definition of <i>quality</i> which the paper uses?<p>As best I can tell, they use commit messages to identify bugfixes, and later they jump to "defective commits". Presumably the bugfix commit is not the defective commit. There is no explanation I can find that shows how they arrive at a defective commit from a bugfix commit.<p>This specific methodology seems rife with weakness, all of which should be explained clearly and admitted up front.
curl is an extremely successful tool/library, I would consider it high quality without knowing the ratio of Normal patches to bug fixes. Skyrim is well known to crash and corrupt save games but regarded as one of the greatest RPGs ever made. the game programmers obviously did a lot of things right to produce such a hit. I'm not saying you need to be a worldwide success to write quality code, just that low bug count doesn't always mean high quality and the other way around. quality is measured by user experience
I’m mildly surprised by how well Clojure performs here. It isn’t statically typed yet fares much better than Haskell/Scala! From my experience Clojure is also a joy to write, sometimes even more than Haskell.
I think it's a good starting point to look at a large number of open source projects in the wild. The individual differences in skill, size, etc. average out between them. It's important to establish whether any statistically significant trends exist before anything further can be discussed meaningfully.<p>If we see empirical evidence that projects written in certain types of languages consistently perform better in a particular area, such as reduction in defects, we can then make a hypothesis as to why that is.<p>For example, if there was statistical evidence to indicate that using Haskell reduces defects, a hypothesis could be made that the the Haskell type system plays a role here. That hypothesis could then be further tested, and that would tell us whether it's correct or not.<p>However, this is pretty much the opposite of what happens in discussions about features such as static typing. People state that static typing has benefits and then try to fit the evidence to fit that claim. Even the authors of this study fall into this trap. They read into the preconceived notions that are not supported by the data in their results. The differences they found are so small that it's reasonable to say that the impact of the language is negligible.
> For example, in languages like Perl, JavaScript, and CoffeeScript adding a string to a number is permissible (e.g., "5" + 2 yields "52"). The same operation yields 7 in Php. Such an operation is not permitted in languages such as Java and Python as they do not allow implicit conversion.<p>Regarding Perl, the quoted statement is wrong:<p><pre><code> $ perl -E 'say "5" + 2'
7
</code></pre>
Furthermore, this is not an implicit conversion. The + operator is an <i>explicit</i> numeric conversion. Here's a more detailed description:<p><a href="https://codespeaks.blogspot.ca/2007/09/ruby-and-python-overload-operator-for.html" rel="nofollow">https://codespeaks.blogspot.ca/2007/09/ruby-and-python-overl...</a>
Fascinating study, but I think a lot of the conclusions in this study are self-evident. For example:<p>"However when compared to the average, as a group, languages that do not allow implicit type conversion are less error-prone while those that do are more error-prone."<p>A lot of the conclusions are along these lines: languages with explicit type conversion have less [type conversion] errors. Well, of course...<p>Still worth a read though, and makes a strong case for functional, statically-typed languages.
Summary :<p>> The data indicates that functional languages are better than procedural languages; it suggests that disallowing implicit type conversion is better than allowing it; that static typing is better than dynamic; and that managed memory usage is better than unmanaged. Further, that the defect proneness of languages in general is not associated with software domains. Additionally, languages are more related to individual bug categories than bugs overall.
This article is dated October 2017 and claims to be the first large-scale evidentiary study. But I have definitely seen either this exact study or another one nearly identical, also using GitHub and also having similar results for the languages, and that was at least 1 year ago. So perhaps this article is a re-print of a prior study?
Interesting that the social element isn't mentioned.<p>Smarter programmers are likely to be able to get their head around the strict requirements of functional languages and they are the ones using the languages at the moment.<p>Java, on the other hand, is pretty much the COBOL of this generation.
There are several points made about typing in another related post here yesterday:<p><a href="https://news.ycombinator.com/item?id=15378800" rel="nofollow">https://news.ycombinator.com/item?id=15378800</a>