Why FPGA manufacturers keep their bitstream formats secret

It is not the bitstream per say that you want, but the full description of the Device Database (defining every programmable switch in the FPGA).<p>Somebody asked why are FPGA EDA tools so large. Well, this is the number one reason.<p>So, at the end of the day, the real reason why FPGA companies don&#x27;t open source their bitstream (and as I said, the actual database) is simply because it will be a major undertaking for them to document in any way that will actually make it possible for the community to use. An FPGA is NOT a processors so it not as easy to document as documenting an instruction set.<p>So, very hard to do, and not enough of a business justification to do so (combined with old school management that don&#x27;t really understand the value of open source). That is it.<p>BTW, it will actually be relatively doable to document the basic Logic Cell, but the problem is that in today&#x27;s modern FPAGs, the logic portion is a relatively small portion (when considering complexity) compared to the very complex I&#x2F;O interfaces.<p>I think the best you can hope for (and what I believe both X and A are moving towards) is a more flexible tool flow, and heavy use of technologies like Partial Reconfiguration, which should allow you to build lots of tools and simply use the FPGA tools (mostly P&amp;R and Timing Analysis) as &quot;smaller&quot; black boxes, while allowing open source or third party to build higher level system integration tools (which IMO, is what is more needed today).

So, who will set up a transparent FPGA manufacturing company with old tech that will then prove all these companies wrong by capturing the market?<p>FPGA manufacturers keep their bitstreams secret because there is no real demand for openness. The customers are happy with the product, they are <i>applying</i> the chips and they work. Outside of the tinkerer world there exists a vast realm of industry where access to the bitstream format would be a nice-to-have but not a must. If it were then a manufacturer would surely take the lead in this and capture the marketshare of the others.<p>For tinkerers and open source proponents this is of course a less-than-ideal situation, we&#x27;d like to see that bitstream documented because then we can make tools operate on them and that generate them. But for the vast majority of the customers (some exceptions do exist) this is not a big issue.<p>The most heard complaint is that the toolchains are cumbersome, slow and too large, not that the bitstreams aren&#x27;t open (funny though: those toolchains would be fixable <i>if</i> the bistreams were open...).

Bitstream formats are probably the worst-kept secret in the industry. FPGAs are inherently very regular in structure, so figuring out which bit goes where is pretty trivial. Try to disclose what you find in public, however, and they&#x27;ll quickly send their lawyers after you...<p>From what I know, this is the only public effort that&#x27;s remained up so far:<p><a href="http:&#x2F;&#x2F;www.clifford.at&#x2F;icestorm&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.clifford.at&#x2F;icestorm&#x2F;</a>

I don&#x27;t know if they realize this, but imo all they cause it having less people working with FPGA&#x27;s. As a result it&#x27;s not a tool people will reach to when it could be used.

<p><pre><code> &quot;Customers will damage their FPGAs with invalid bitstreams, and blame us for selling unreliable products.&quot; This used to be true when FPGAs had internal tristate resources (so you could drive the same net from competing sources); this is no longer the case for modern devices. </code></pre> I agree that this is a bogus argument -- nobody would blame the chip vendor for damage caused by a third-party bitstream -- but I don&#x27;t see what it has to do with tristate logic. The only thing keeping me from shorting out two (or two thousand) ordinary CMOS drivers with an assign statement is the toolchain, isn&#x27;t it?

Here&#x27;s my official prediction: there&#x27;s going to be a sea change in the FPGA market in the next five to ten years. The reason FPGAs are so complex is that more than half of the die space is given over to the routing structures. This means that to be even in the ballpark of ASIC performance they have to have a complicated mish-mash of logic tiles and all sorts of other special-purpose tiles (like arm cores etc.).<p>The die space issue is an artifact of using SRAM to store the configuration (likewise using flash is also too big and cumbersome), not to mention the power usage. However, the new non-volatile memories coming up (like memristors or nano-ram) are absolutely minuscule and will have a dramatic effect on the competitiveness of FPGAs. All the sudden you can have an extremely regular structure, with no specialized tiles, with performance comparable to ASICs. In fact, there&#x27;s an argument that they will outperform ASICs because as we start getting into quantum effects because of die feature shrink, being able to function despite a relatively high degree of errors (which FPGAs can route around with minimal performance loss) will become a huge advantage.<p>Here&#x27;s a paper that talks about it:<p><a href="https:&#x2F;&#x2F;www.ece.ucsb.edu&#x2F;~strukov&#x2F;papers&#x2F;2006&#x2F;Worldcomp2006.pdf" rel="nofollow">https:&#x2F;&#x2F;www.ece.ucsb.edu&#x2F;~strukov&#x2F;papers&#x2F;2006&#x2F;Worldcomp2006....</a> [PDF]<p>The very regular structure of the iCE40 is quoted as a reason for its choice as a reverse-engineering target - this helps a great deal with the tooling issue. If the above prediction is true, then FPGAs can potentially be: extremely regular, on par with ASIC performance, and potentially cheaper to produce because of economies of scale. These factors will, I believe, change the dynamics of the market to a point where an open source FPGA is a viable option for crowd-sourcing.

I always assumed it was because, when you make an interface&#x2F;API public (by documenting it and offering it to your customers) you have a professional obligation to keep it reasonably stable. People get understandably upset when APIs for things like twitter and facebook keep changing under them, making things that worked before stop working.<p>Making the bitstream format a public API would make it harder for them to change&#x2F;update&#x2F;improve it without making themselves look like assholes for breaking third party software.

So then if none of those things are true, wht do FPGA manufacturers keep their bitstream formulas secret?

We really need a fully free FPGA toolchain. I am particularly interested in liberating the Xilinx CSG324 chip that was chosen for the Novena. I have heard that it is possible to fry the chip with a bad bitstream, so this baby step[0] is all I&#x27;ve managed to accomplish. The fpgatools project is messy, but the only project I know of that has figured out the bitstream format for a closely related Xilinx FPGA model. I even wrote some Scheme code that could produce the exact same bitstream as the example fpgatools program using a little domain-specific language... if only it could run on my FPGA.<p>Others note that the bitstream format is only a small piece of the puzzle, and while I agree, I don&#x27;t yet care about the HDLs and all of the tools built on top. Just figuring out the bitstream format for more FPGAs would be a huge win and would enable a free toolchain to be begin to be written.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;davexunit&#x2F;fpgatools&#x2F;commit&#x2F;06e95c379cefd9c8e21dad44747109e9095cc5b5" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;davexunit&#x2F;fpgatools&#x2F;commit&#x2F;06e95c379cefd9...</a>

Screw the formats: build your own. There&#x27;s all kinds of bright people doing hardware development in college with top EDA tools and cheap prototyping through eg MOSIS. One already did a FPGA architecture on 45nm:<p><a href="http:&#x2F;&#x2F;www.eecs.berkeley.edu&#x2F;Pubs&#x2F;TechRpts&#x2F;2014&#x2F;EECS-2014-43.pdf" rel="nofollow">http:&#x2F;&#x2F;www.eecs.berkeley.edu&#x2F;Pubs&#x2F;TechRpts&#x2F;2014&#x2F;EECS-2014-43...</a><p>Anyone wanting to see more projects, info on what developing open HW will take w&#x2F; potential paths, and so on can follow my links here:<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10468898" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10468898</a><p>Anyway, we have most of what we need in terms of tooling. It&#x27;s just going to be a huge loss getting the initial I.P. built and ASIC-proven on each process node. Best route is academics w&#x2F; professional support each doing a piece of key I.P., using grants + discounts to build it, and then public domaining it. So, who knows a near billionaire who wants to change the HW world for the better? And who can fight patent suits?

To summarize, there exists at least one person who wants to write their own tools, or at least use open source tools.<p>For at least this one person, downloading closed source software from the chip manufacturer is not satisfactory.<p>The comments also add that these downloads can be on the order of 8-20GB.<p>Has anyone ever wondered what is in those large binaries?<p>Does someone think the larger size somehow offers potentially more IP protection?<p>Does the FPGA utilities world have anything like teensy_loader_cli? It&#x27;s about 28k. Not limited to MS Windows. Works with both BSD and Linux.

It&#x27;s simple economics --- lock in customers to buy your chips. Just like razor blades and razors.

how far are people in reversing the bitstreams? I&#x27;m thinking maybe I should choose an FPGA and try to raise money on kickstarter and spend one year on it.

If they want to keep the bitstream format closed for some deranged IP reason - ok, fine. Just document all the cells (they do it to an extend anyway) and provide a readable format for a placed and routed layout.<p>This way the entire toolchain can be open source, keeping only the bitstream packer closed. Lawyers are happy, users are happy - win-win.