TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Fully Countering Trusting Trust Through Diverse Double-Compiling (2009)

88 pointsby micaekedover 8 years ago

8 comments

dwheelerover 8 years ago
jacques_chester: You&#x27;re absolutely right that recreating things bit-for-bit identical can require some real elbow grease. I had to overcome bugs in the Tiny C compiler (tcc) and gcc to make them work, as described in the paper. Date&#x2F;time stamps can create problems too.<p>But all these problems are quite doable. Nobody claims that gcc is small, yet I managed to get that working. Compiler makers can follow a few guidelines to make it much easier, see: <a href="http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;dissertation&#x2F;html&#x2F;wheeler-trusting-trust-ddc.html#4.Guidelines%20for%20Compiler%20Suppliers" rel="nofollow">http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;dissertation&#x2F;html&#x2F;whe...</a> Check out the graph at the Debian Reproducible Builds project at <a href="https:&#x2F;&#x2F;wiki.debian.org&#x2F;ReproducibleBuilds" rel="nofollow">https:&#x2F;&#x2F;wiki.debian.org&#x2F;ReproducibleBuilds</a> - yes, some packages don&#x27;t reproduce bit-for-bit, but they&#x27;ve made a tremendous amount of progress and managed it for most packages.<p>You can see some related information at this page: <a href="http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;</a> including a video of me discussing the paper.
评论 #12667447 未加载
rurbanover 8 years ago
Can we please add a (2009) to the title?<p>This is a classic paper on reproducible builds, everybody is working on since. Better overview: <a href="http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.dwheeler.com&#x2F;trusting-trust&#x2F;</a><p>Older discussion, 7 years ago:<p>* <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1104338" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1104338</a>
jacques_chesterover 8 years ago
The hard part -- sometimes <i>really</i> hard -- is the &quot;bit-for-bit identical&quot; requirement.<p>Lots of builds are <i>recreatable</i> but not <i>reproducible</i> (there is probably a better term of art here). You can go back to a point in time and build the version of the software as it was, but you are not guaranteed to get a bit-for-bit clone. (See <a href="https:&#x2F;&#x2F;reproducible-builds.org" rel="nofollow">https:&#x2F;&#x2F;reproducible-builds.org</a> for a thorough discussion)<p>The problem is that there are lots of uncontrolled inputs to a build that are due to sourcecode or compiler changes. Most famously there are timestamps and random numbers, which mess up all sorts of hashing-based approaches.<p>These can even be non-obvious. Just the other day I and a colleague were investigating the (small but unsettling) possibility that an old buildpack had been replaced maliciously. We compared the historical hash to the file: different. We rebuilt the historical buildpack with trusted inputs: still different.<p>Then we unzipped both versions and diff&#x27;d the directories: identical.<p>What had thrown our hashes off was that zipfiles, by default, include timestamps. We have a build that is recreatable but not reproducible.<p>Speaking of builds, we are able to reproducibly build some binaries but not others. Off the top of my head our most high-profile non-reproducible build is NodeJS. Some other binaries (Ruby and Python, in my not-at-all-complete recollection) are fully reproducible.<p>This difficulty with fully reproducing makes it hard to provide a fully trustworthy chain of custody. A company which uses Cloud Foundry have in actual fact stood up an independent copy of our build pipelines inside their own secure network, so that they can be completely autarkic for the build steps leading to a complete buildpack. This doesn&#x27;t defend against malicious source, but it defends against malicious builds.<p>Disclosure: I work for Pivotal, the majority donor of engineering to Cloud Foundry. As you&#x27;ve probably guessed, I&#x27;m currently a fulltime contributor on the buildpacks team.
评论 #12667454 未加载
评论 #12667471 未加载
评论 #12667495 未加载
评论 #12667543 未加载
评论 #12667706 未加载
评论 #12670035 未加载
dwheelerover 8 years ago
I&#x27;m the author. Ask me anything about it.
评论 #12667552 未加载
评论 #12667684 未加载
评论 #12667473 未加载
lambyover 8 years ago
Hi all, I do a lot of work on Reproducible Builds within Debian (AMA...!).<p>Just wanted to mention we are now having regular IRC meetings:<p><a href="https:&#x2F;&#x2F;lists.reproducible-builds.org&#x2F;pipermail&#x2F;rb-general&#x2F;2016-October&#x2F;000071.html" rel="nofollow">https:&#x2F;&#x2F;lists.reproducible-builds.org&#x2F;pipermail&#x2F;rb-general&#x2F;2...</a>
contingenciesover 8 years ago
It strikes me that the use of diverse systems to reinforce assumptions of trust within a given subsystem is an architectural paradigm not limited to compilers. The key problems are implementation feature-set or edge-case differences and overhead (real time and maintenance&#x2F;up-front development). In fact, it would be ideal with multiple client versions&#x2F;implementations on any service (particularly distributed or financial) and indeed I have done this in the past. Not sure if this paradigm has a name... anyone? I suppose you could just say consensus-based hedging.
评论 #12668268 未加载
bitwizeover 8 years ago
I still can&#x27;t see this paper referenced without thinking of HISSATSU! DOUBLE COMPILE!: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=FHkFzRZdlV4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=FHkFzRZdlV4</a>
nickpsecurityover 8 years ago
This again. A perfect example of solving the wrong problems in a clever way. To his credit, Wheeler at least gives credit to the brilliant engineer (Karger) who invented the attack, points out it took 10 years before that knowledge reached anyone via Thompson (recurring problem in high-security), and did the reference essays on the two solutions to the actual problem (high-assurance FLOSS &amp; SCM&#x27;s). That&#x27;s what you&#x27;re better off reading.<p>Here&#x27;s a quick enumeration of the problems in case people wonder why I gripe about this and reproducible builds fad:<p>1. What the compiler does needs to be fully specified and correct to ensure security.<p>2. The implementation of it in the language should conform to that spec or simply be correct itself.<p>3. No backdoors are in the compiler, the compilation process, etc. This must be easy to show.<p>4. The optimizations used don&#x27;t break security&#x2F;correctness.<p>5. The compiler can parse malicious input without code injection resulting.<p>6. The compilation of the compiler itself follows all of the above.<p>7. The resulting binary that everyone has is the same one matching the source with same correct <i>or malicious</i> function but no malicious stuff added that&#x27;s not in the source code already. This equivalence is what everyone in mainstream is focusing on. I already made an exception for Wheeler himself given he did this <i>and</i> root cause work.<p>8. The resulting binary will then be used on systems developed without mitigating problems above to compile other apps not mitigating problems above.<p>So, that&#x27;s a big pile of problems. The Thompson attack, countering the Thompson attack, or reproducible builds collectively address the tiniest problem vs all the problems people actually encounter with compilers and compiler distribution. There&#x27;s teams working on the latter that have produced nice solutions to a bunch of them. VLISP, FLINT, the assembly-to-LISP-to-HLL project &amp; CakeML-to-ASM come to mind. There&#x27;s commercial products, like CompCert, available as well. Very little by mainstream in FOSS or proprietary.<p>The &quot;easy&quot; approach to solve most of the real problem is a certifying compiler in a safe language bootstrapped on a simple, local one whose source is distributed via secure SCM. In this case, you do not have a reproducible build in vast majority of cases since you&#x27;ve verified source itself and have a verifying compiler to ASM. You&#x27;ll even benefit from no binary where your compiler can optimize the source for your machine or even add extra security to it (a la Softbound+CETS). Alternatively, you can get the binary that everyone can check via signatures on the secure SCM. You can even do reproducible builds on top of my scheme for the added assurance you get in reproducing bugs or correctness of specific compilations. Core assurance... 80&#x2F;20 rule... comes from doing a compiler that&#x27;s correct-by-construction much as possible, easy for humans to review for backdoors, and on secure repo &amp; distribution system.<p>Meanwhile, the big problems are ignored and these little, tactical solutions to smaller problems keep getting lots of attention. Same thing that happen between Karger and Thompson time frame for Karger et al&#x27;s other recommendations for building secure systems. We saw where that went in terms of the baseline of INFOSEC we had for decades. ;)<p>Note: I can provide links on request to definitive works on subversion, SCM, compiler correctness, whatever. I think the summary in this comment should be clear. Hopefully.<p>Note 2: Anyone that doubts I&#x27;m right can try an empirical approach of looking at bugs, vulnerabilities and compromises published for both GCC and things compiled with it. Look for number of times they said, &quot;We were owned by the damned Thompson attack. If only we countered it with diverse, double compilation or reproducible builds.&quot; Compare that to failures in other areas on my list. How unimportant this stuff is vs higher-priority criteria should be self-evident at that point. And empirically proven.
评论 #12667665 未加载
评论 #12667903 未加载