Finding Critical Open Source Projects

Top 10:<p>- Python: salt, core (<a href="https:&#x2F;&#x2F;github.com&#x2F;home-assistant&#x2F;core" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;home-assistant&#x2F;core</a>), pandas, scikit-learn, numpy, airflow, erpnext, matplotlib, pytest &amp; pip<p>- Rust: servo, cargo, rust-clippy, tokio, rust-analyzer, tock, tikv, alacritty, libc &amp; substrate<p>- JS: node, react-native, react, gatsby, three.js, bootstrap, material-ui, odoo, next.js &amp; Rocket.Chat<p>- Java: elasticsearch, flink, spring-boot, hadoop, netty, jenkins, beam, bazel, alluxio &amp; pmd<p>- C++: tensorflow, ceph, pytorch, bitcoin, electron, Marlin, Cataclysm-DDA, llvm-project, rocksdb &amp; QGIS<p>- C: git, linux, linux, php-src, openssl, systemd, curl, u-boot, qemu &amp; mbed-os

As others have mentioned, while this may seem like a good idea, the results are often bizarre, and it&#x27;s not hard to see why - the metrics and algorithm are here: <a href="https:&#x2F;&#x2F;github.com&#x2F;ossf&#x2F;criticality_score#criticality-score" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ossf&#x2F;criticality_score#criticality-score</a>.<p>That algorithm seems unnecessarily complicated and includes somewhat dubious metrics when, in my mind, the only thing that really &quot;counts&quot; when it comes to &quot;criticality&quot; are &quot;how many other things use&#x2F;depend on me&quot;, and there are much easier ways to determine this:<p>1. For languages with a common package repo, how many other packages depend on me? With NPM, for example, it&#x27;s pretty easy to figure out how many other packages depend on a given package.<p>2. For &quot;top level&quot; projects, look at downloads.<p>My guess is just looking at either (or both) of those metrics would give you better results.

List of the top 200 is at :<p><a href="https:&#x2F;&#x2F;commondatastorage.googleapis.com&#x2F;ossf-criticality-score&#x2F;index.html" rel="nofollow">https:&#x2F;&#x2F;commondatastorage.googleapis.com&#x2F;ossf-criticality-sc...</a><p>(from the repo)<p>So gnucash is #15 . At #75 is gcc .<p>This seems like a great idea, but perhaps some refinements are needed.

There are some issues with the criticality score (<a href="https:&#x2F;&#x2F;github.com&#x2F;ossf&#x2F;criticality_score" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ossf&#x2F;criticality_score</a>).<p>For example, it counts the number of issues as determining criticality. If there is any kind of money attached to this score (now or in the future), obviously this is going to encourage people to introduce more bugs into their projects.

Sounds interesting in theory, but the results are somewhat bizarre. For example, their published list of the top 200 &quot;most critical&quot; open-source C projects includes Urbit and Stellar, and includes micropython but does <i>not</i> include CPython.

I like this idea, which pops up here and there occasionally, but this particular &quot;criticality score&quot; appears to measure popularity, rather than criticality. For one, it doesn&#x27;t take into account whether there are common alternatives to a given project&#x2F;package. Of course data like that is hard to gather automatically and from GitHub, but possibly that&#x27;s just not a great source, and&#x2F;or could be combined with others -- such as operating systems&#x27; repositories, which would contain information not just about alternatives, but also about packages required to run a minimal system, and to build it.

It&#x27;s nice that they&#x27;re automating this even though the results seem currently a bit random. Fedora has a concept of Critical Path packages (<a href="https:&#x2F;&#x2F;fedoraproject.org&#x2F;wiki&#x2F;Critical_path_package" rel="nofollow">https:&#x2F;&#x2F;fedoraproject.org&#x2F;wiki&#x2F;Critical_path_package</a>). The concept is related to various activities, eg. a package is critical if is is needed when installing a graphical desktop. But as far as I know packages are picked manually and therefore we probably miss packages or over&#x2F;underestimate criticality.

The link is to the home page of the Google Open Source Blog, and not this specific blog post. It should be updated to:<p><a href="https:&#x2F;&#x2F;opensource.googleblog.com&#x2F;2020&#x2F;12&#x2F;finding-critical-open-source-projects.html" rel="nofollow">https:&#x2F;&#x2F;opensource.googleblog.com&#x2F;2020&#x2F;12&#x2F;finding-critical-o...</a>

Neither the blog post nor project README directly mentions whether the 0.0 to 1.0 score goes from most-critical to least, or least-critical to more.<p>It appears 1.0 is most-critical, if you dig into ranking lists.

Have you heard of the WWII bullet holes problem that Wald solved? This project seems to fall into the trap that Wald’s colleagues fell into. The more popular a project the more critical it becomes when that’s not actually the case. You want to know where the bullet holes you don’t see are because that’s the thing that’s going to take you out. All the bullet holes you’re seeing and surviving are not great but survivable.<p>Critical to me means “what’s the thing I <i>have</i> to pay attention to or I’ll suffer consequences”. Having a dependency on a non-popular crate is definitely an increasing score of criticality for that dependency. I may have misread but the fatal error in the metric to me is that popularity of a project increases its criticality when it should decrease. A popular project with lots of contributors means it has lots of stake holders already and a way to successfully manage that. What you want is the small independent projects that popular projects depend on. In other words, what’s the smallest, hardest to notice malicious change I can make in the supply chain of software development to enact the most disruptive change?

Lots of things to check out here, and it&#x27;s limited only to projects that are on github, but unsurprisingly it looks like the top 5 are:<p>- Node (0.984)<p>- Tensorflow (0.969)<p>- Git (0.945)<p>- Linux (0.936)<p>- PHP (0.919)

I don&#x27;t understand how gnucash is more critical than Arduino, wine or nginx. Is gnucash very widely used?

Strange list, PMD so high, quarkus also, but e.g. jackson is lower then those both.<p>elasticsearch at the top? flink? It is first time I hear about it and I work in Java shops for quite some time.

See also: <a href="https:&#x2F;&#x2F;www.fsf.org&#x2F;campaigns&#x2F;priority-projects" rel="nofollow">https:&#x2F;&#x2F;www.fsf.org&#x2F;campaigns&#x2F;priority-projects</a>.

This url links to the top-level blog - for the specific post indicated by the title it needs to be <a href="https:&#x2F;&#x2F;opensource.googleblog.com&#x2F;2020&#x2F;12&#x2F;finding-critical-open-source-projects.html" rel="nofollow">https:&#x2F;&#x2F;opensource.googleblog.com&#x2F;2020&#x2F;12&#x2F;finding-critical-o...</a>

Very interesting and I hope this leads to actual recompense&#x2F;compensation for people &quot;thanklessly maintaining&quot; OSS. Not sure how it possibly fits into Google&#x27;s newfound profit-at-all-costs motive, but the gesture is an important first step towards some semblance of justice.

Seems like a good direction. My main hope is that if metrics like these become more accepted and&#x2F;or important, that they find a way to avoid running afoul of Goodhart&#x27;s law.

btw, just one thing, having seen this slashdot comment: <a href="https:&#x2F;&#x2F;news.slashdot.org&#x2F;comments.pl?sid=17816088&amp;cid=60823564" rel="nofollow">https:&#x2F;&#x2F;news.slashdot.org&#x2F;comments.pl?sid=17816088&amp;cid=60823...</a><p>what i strongly recommend that google do - instead of doing this work which is, as you&#x27;ve probably noticed from the comments, well-meaning but highly likely to be biased - is:<p><pre><code> *make a large donation to NLnet* </code></pre> don&#x27;t tell them what to do with the money: leave that up to them. NLnet is extremely good at ensuring that money is used effectively, by requiring that they come up with a project plan involving milestones. they <i>do not</i> just &quot;dump money at the developer&quot;, which has a known high historically-backed probability of causing more harm than good: they require the milestones to be completed, 100%, before the money is paid.<p>note, here: they do not use algorithms to assess a project: they use people. they also assess the proposal against the usefulness of achieving key objectives.