TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Operation Rosehub – patching thousands of open-source projects

727 点作者 fhoffa大约 8 年前

28 条评论

fhoffa大约 8 年前
This is one of the most impactful projects I&#x27;ve seen built using the GitHub source on BigQuery dataset (since we published it).<p>If you want to see other use cases - I&#x27;ve collected plenty of other stories from multiple parties at:<p>- <a href="https:&#x2F;&#x2F;medium.com&#x2F;google-cloud&#x2F;github-on-bigquery-analyze-all-the-code-b3576fd2b150" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;google-cloud&#x2F;github-on-bigquery-analyze-a...</a><p>Disclosure: I&#x27;m Felipe Hoffa and I work for Google Cloud (<a href="https:&#x2F;&#x2F;twitter.com&#x2F;felipehoffa" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;felipehoffa</a>)
评论 #13771225 未加载
jayfk大约 8 年前
I&#x27;ve built something like this for Python projects.<p>You add your repo and a bot is constantly checking for insecure and&#x2F;or outdated packages and sends you a pull request if you need to update.<p>It&#x27;s free for open source projects at <a href="https:&#x2F;&#x2F;pyup.io" rel="nofollow">https:&#x2F;&#x2F;pyup.io</a>
评论 #13770296 未加载
评论 #13771208 未加载
评论 #13774253 未加载
评论 #13772400 未加载
评论 #13770650 未加载
评论 #13774376 未加载
rrggrr大约 8 年前
So many questions... What does this say about Google&#x27;s hiring, about its employee&#x27;s values, about values across the tech community? I can remember a time when managements would have shut this down, when employees would have said, &quot;not my problem&quot;, when entire industries would have buried their heads in the sand.<p>Is it the lack of liability and regulation that clears the way for this kind of corporate citizenship? Is it cultural?
评论 #13770660 未加载
评论 #13770492 未加载
评论 #13774380 未加载
评论 #13770718 未加载
评论 #13771991 未加载
评论 #13770561 未加载
评论 #13771280 未加载
vog大约 8 年前
I like the &quot;bank teller&quot; analogy used in the article.<p><i>&gt; it would be like hiring a bank teller who was trained to hand over all the money in the vault if asked to do so politely, and then entrusting that teller with the key. The only thing that would keep a bank safe in such a circumstance is that most people wouldn’t consider asking such a question.</i><p>This does not only work for deserialization issues.<p>It is a great analogy for a huge class of IT security issues!<p>Maybe we should use that one when communicating with the media. This this works much better than the usual burglary analogy. I like how it points out that this is about stupid and&#x2F;or malicious behaviour (code), where the attacker (hacker) just needs curiosity, and may find this out even by accident. The attacker did not have to break something, and did not damage anything, to get into something. In particular, this makes clear that this is caused by irresponsibile behaviour of the organization and&#x2F;or other entities to whom they delegate trust.<p>Even for more complicated scenarios, I like the bank teller analogy more than the classic burglary analogy. In that case, the attacker observes multuple bank tellers, and notices e.g. that if you ask the first teller for form A and put in certain words, another bank teller will accept it and give you a stamped form B, which you can show to a third teller in another branch office who will look a bit confused, but finally accept it and hand over all money to you.<p>We need to get over blaming the messengers[1], buying zerodays and declaring cyberwar. What we really need to do is to finally make our[2] computer systems secure and trustworthy, at least up to a certain minimum-level of sanity: no exec, no injection (i.e. typing&#x2F;tagging), no overflows (i.e. static analysis), input validation, testing, fuzzing, you name it.<p>And this cannot work by just adding more and more complex security measures outside, but more importantly simplifying and cleaning up inside. Although rewriting software from scratch is very risky, radical refactoring is not! And every good software engineering course tells you how to do it correctly.<p>[1] security researchers, but also &quot;amateur&quot; hackers, or just someone running into it by accident because the security issue became so large it finally <i>had</i> to be noticed by someone.<p>[2] in the sense of: everyones!
评论 #13772596 未加载
tombh大约 8 年前
Is <a href="https:&#x2F;&#x2F;libraries.io" rel="nofollow">https:&#x2F;&#x2F;libraries.io</a> not a more comprehensive and community-focused response to the same problem?<p>libraries.io did make it to the front page a few months ago, but I think its underlying vision might not have been driven home from just glancing at its home page. It supports 33 package managers (not just Java, though I&#x27;m sure Rosehub doesn&#x27;t just do that either) and Github&#x2F;Gitlab&#x2F;Bitbucket, not just Github. And it provides both email notifications and auto PRs.<p>But that&#x27;s just the overlap with Rosehub. On top of that it offers the means to discover libraries based on a Dependency Rank (think Page Rank but using dependencies instead of hyperlinks). Which in turn allows it to surface projects with a high &quot;Bus Factor&quot; -- projects maintained by few committers, but depended on by many (so they&#x27;d be more affected by said committers getting run over by a bus). AND it mines the licenses for a project, notifying if any of the dependent licenses are incompatible with the parent license. What&#x27;s more it&#x27;s a non-profit organisation receiving enough funding to employ 2 full time devs.<p>I think libraries.io is Rosehub and more, to quote the about page;<p><pre><code> Our goal is to raise the quality of all software, by raising the quality and frequency of contributions to free and open source software; the services, frameworks, plugins and tools we collectively refer to as libraries. </code></pre> To take the liberty of extrapolating from the libraries.io vision: open source security isn&#x27;t just about fixing patches, but about supporting the environment, people, conditions and tools that contribute to open source software.
评论 #13771594 未加载
saurik大约 8 年前
I am extremely sad that this turns into an argument <i>for</i> making certain that all source code in the world is at least indirectly accessible specifically via GitHub (at which point people will find it there and expect the developers to respond and generally track everything going on there, even projects which are much happier using more open tools); like: it isn&#x27;t sufficient that your code is &quot;open&quot;, it actively has to be part of the unified GitHub empire.
评论 #13771483 未加载
orf大约 8 年前
In their query they do:<p><pre><code> FROM (SELECT id,content FROM (SELECT id,content FROM [bigquery-public-data:github_repos.contents] WHERE NOT binary) WHERE content CONTAINS &#x27;commons-collections&lt;&#x27;) </code></pre> Why the subquery? Why not WHERE NOT binary AND content CONTAINS...? is this a bigquery thing?
评论 #13770341 未加载
评论 #13770246 未加载
评论 #13772915 未加载
评论 #13770373 未加载
评论 #13770240 未加载
tlrobinson大约 8 年前
Wow. I wonder how much a query that searches the content of all of Github costs (if you&#x27;re not Google). This page says the dataset is 3TB+ <a href="https:&#x2F;&#x2F;cloud.google.com&#x2F;bigquery&#x2F;public-data&#x2F;github" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;bigquery&#x2F;public-data&#x2F;github</a> and presumably most of that is content.
评论 #13770935 未加载
评论 #13770866 未加载
评论 #13770929 未加载
cypherpunks01大约 8 年前
Nice! That&#x27;s some good citizenry.<p>Interesting fact: Justine was the founder of occupywallst.org, which was the highest-trafficked publisher&#x2F;web hub for the Occupy Wall Street movement before she worked for Google.
markcerqueira大约 8 年前
&quot;Patches were sent to many projects, avoiding threats to public security for years to come.&quot;<p>Are these pull requests that the project would still need to approve&#x2F;merge or were they just pushed in?
评论 #13770116 未加载
评论 #13770287 未加载
评论 #13770120 未加载
luhn大约 8 年前
As scary as Google&#x27;s massive size and power is, it&#x27;s pretty awesome that they&#x27;re incentivized to do things like this to help the internet because they <i>are</i> the internet.
mrgrowth大约 8 年前
I read so many of these kinds of articles out of curiosity and rarely understand them.<p>Thank you for adding in the part about the bank teller.<p>For reference: &quot;it would be like hiring a bank teller who was trained to hand over all the money in the vault if asked to do so politely, and then entrusting that teller with the key.&quot;
joelthelion大约 8 年前
&gt; But unlike big businesses, open source projects don’t have people on staff<p>To read that from Google is frankly disappointing. While this is true of many open-source projects, it doesn&#x27;t have to be that way. Red Hat (and Google!) are brilliant proofs of this.
评论 #13772493 未加载
bla2大约 8 年前
Really cool, kudos to people helping with this. I wonder if this could have been done in a way that non-Googlers could have pitched in too, given that this is for a public good -- but it&#x27;s tricky with security issues.
hokkos大约 8 年前
How does it work for transitive depandancies ? If you use a package that use a vulnerable Apache common? Does a pr is sent to update the package when it is updated?
tropo大约 8 年前
If I understand it right, this bug involves code pulling in old buggy libraries, sometimes indirectly via other libraries. It seems that there is a reference to a specific bad version, not the actual inclusion of cut-and-paste code.<p>Eh, why not just get rid of the bad version? Alternately, release a bug-fixed copy with the same version number.<p>Any breakage is a case of &quot;oh well, you&#x27;re safe now&quot;. Leaving the security hole is probably worse breakage.
评论 #13771462 未加载
评论 #13775308 未加载
make3大约 8 年前
I wish you could do the same thing with mental illness.. massively send pull request to correct everyone&#x27;s bad brain code.. &lt;sorry&gt;
评论 #13790235 未加载
评论 #13771248 未加载
mirekrusin大约 8 年前
It&#x27;s interesting that this type of initiative, which is admirable, will spike up some java &quot;popularity&quot; metrics on GitHub.
hawski大约 8 年前
I was thinking about doing something similar with bigquery and github data to search for uses of strncpy in C code. But I am not that good with the query language and also bigquery didn’t support multiple users properly (this adds friction).<p>I still think it’s a good idea. It would be even better to search for a few C pitfalls more, but strncpy is probably the easiest to search for.
ploxiln大约 8 年前
I think this is one good concrete example of why the npm style of private dependencies for each lib is not the greatest thing ever, while the non-recursive style in python (or C) is overall more manageable (if you are actually managing your dependencies instead of ignoring them).
codelion大约 8 年前
we have been doing thus for a while now : <a href="https:&#x2F;&#x2F;www.sourceclear.com&#x2F;blog&#x2F;millions-of-program-builds-vulnerable-to-man-in-the-middle-attacks&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.sourceclear.com&#x2F;blog&#x2F;millions-of-program-builds-...</a>
评论 #13770451 未加载
11928311大约 8 年前
So, Google does ... something and is showered with praise.<p>Thousands of volunteers work in the saltmines and get nothing.<p>Business as usual. Myths like &quot;Google sponsored Python!!!&quot; propagate when they do nothing at all.<p>Disgusting.
Dem0stheneS大约 8 年前
That&#x27;s outstanding news. Hats off to the volunteers doing the work on this.
rburhum大约 8 年前
Mad thank-yous to Google for this!
lvlds大约 8 年前
Awesome! Contgrats to the team!
snambi大约 8 年前
What is in it for google?
评论 #13774525 未加载
评论 #13771202 未加载
muzster大约 8 年前
Operation Rosebud
lolive大约 8 年前
Wouldn&#x27;t a graph database be a more suitable tool for that kind of task?
评论 #13774545 未加载