TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Why I find IEEE 754 frustrating

140 pointsby plesnerover 10 years ago

10 comments

StefanKarpinskiover 10 years ago
Good essay. The dynamic scoping &#x2F; mutable global state issue, especially with respect to rounding modes, is a total nightmare. We&#x27;ve been trying our hardest to provide great support for rounding modes in Julia and it just doesn&#x27;t work well. The trouble is that there are functions that need normal rounding regardless of the current rounding mode while other operations can safely respect the current rounding mode. There are still other functions for which neither is appropriate. A software-implemented sin function, for example, may completely break if used with an unexpected rounding mode, so you don&#x27;t want that. But you also don&#x27;t want to just completely ignore the rounding mode either. What you really want is a sin function that returns a correct result, rounded according to the rounding mode. But that means you really need five variants of the sin function and you need to dynamically dispatch between them based on the current rounding mode – i.e. global mutable state – which is, of course, quite bad for performance, not to mention the explosion of complexity. I&#x27;ve come to believe that the rounding mode concept as designed in IEEE 754 was only ever feasible to use when you were writing your own assembly code.<p>The essay is not quite right about the motivation for rounding modes – they are not intended to support interval arithmetic. When we got the chance to ask him about it, Kahan was quite negative about interval arithmetic, noting that intervals tend to grow exponentially large as your computation progresses. Instead, the motivation he suggested was that you could run a computation in all the rounding modes, and if the results in all of them were reasonably close, you could be fairly certain that the result was accurate. Of course, at the higher level the essay is exactly right: the fact that this motivation isn&#x27;t explained in the IEEE 754 spec is precisely what allowed this misunderstanding about the motivation to occur at all.
评论 #8578928 未加载
评论 #8579061 未加载
评论 #8579771 未加载
评论 #8578478 未加载
评论 #8578701 未加载
评论 #8578736 未加载
jwsover 10 years ago
When confronted by closed standards with open drafts I generally just implement the last draft. I might buy the closed spec to check, but my code comments and documentation will all reference the draft since that is what people can read.<p>You might tuck a copy of these into your personal library in case IEEE purges them.<p>IEEE754 base document: <a href="http://www.validlab.com/754R/standards/754.pdf" rel="nofollow">http:&#x2F;&#x2F;www.validlab.com&#x2F;754R&#x2F;standards&#x2F;754.pdf</a><p>IEEE754r draft: <a href="http://www.validlab.com/754R/drafts/archive/2006-10-04.pdf" rel="nofollow">http:&#x2F;&#x2F;www.validlab.com&#x2F;754R&#x2F;drafts&#x2F;archive&#x2F;2006-10-04.pdf</a><p>Perhaps someone who has seen both versions can comment on how close these are to the closed versions.
评论 #8578443 未加载
评论 #8579140 未加载
ekimekimover 10 years ago
Python&#x27;s Decimal module (though not its floats, for some reason) has, IMO, a pretty good implementation of these features.<p><a href="https://docs.python.org/2/library/decimal.html#context-objects" rel="nofollow">https:&#x2F;&#x2F;docs.python.org&#x2F;2&#x2F;library&#x2F;decimal.html#context-objec...</a><p>Basically, it encapsulates attributes and status flags into a thread-local &quot;context&quot; which you get&#x2F;set through normal function calls. There&#x27;s also the helpful &quot;with&quot; syntax which allows you to say &quot;run this code block (and anything it calls, etc) with this context instead of the current one, then restore the current one on exit&quot;.<p>A sibling comment talked about a sin() example where you want to use an explicit rounding mode for your calculations, then apply the global rounding mode to the result. Under this paradigm it would look something like:<p><pre><code> with MySpecialContext(settings, etc) as ctx: calculations... check status flags in ctx get result round result # this uses the parent context return</code></pre>
colomonover 10 years ago
Looking at the broader issues here, I know I&#x27;ve had the same sort of problems with the ISO 10303 (STEP) standard [1]. Overall, it consists of dozens of $100+ books, most of which amount to little more than a long list of descriptions of the classes that can be used to transmit CAD data. Everything is in turgid bureaucratese. I&#x27;ve seen nowhere in the standard with any sort of high level description of how those classes are intended to be used, no motivation for why things are the way they are. There are some recommended practices documents, but they mostly seem to cover fringe areas like how to handle colors rather than core areas like the preferred approach for handling CAD geometric data.<p>It just seems so odd to spend so much effort to develop a public standard, then make it expensive and hard to use. Doesn&#x27;t that defeat the entire point of having a standard?<p>[1] <a href="http://en.wikipedia.org/wiki/ISO_10303" rel="nofollow">http:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;ISO_10303</a>
stephencanonover 10 years ago
This essay raises a lot of concerns about global state, especially with regard to rounding and flags (or exceptions). That&#x27;s a common misconception, but nothing in IEEE-754 requires that this state be global. In the C language bindings, for example, dynamic rounding mode and status flags have thread scope.<p>In fact, dynamic rounding modes are not required <i>at all</i> by IEEE-754 (2008). The revised standard requires that languages provide a means to specify static rounding at &quot;block&quot; scope, which is a language-defined syntactic unit.<p>&gt; (4.1) the implementation shall provide language-defined means, such as compiler directives, to specify a constant value for the attribute parameter for all standard operations in a block; the scope of the attribute value is the block with which it is associated.<p>&gt; (2.1.7) block: A language-defined syntactic unit for which a user can specify attributes.<p>You can take &quot;block&quot; to mean whatever makes sense for your language: it could be a single arithmetic operation[1] or it could be the whole program (though it&#x27;s more useful if it isn&#x27;t). It is recommended, but not required, that languages provide a means to access &quot;dynamic&quot; rounding modes as well, which correspond roughly to what most people think of when they think of IEEE-754 rounding modes as widely implemented, but again a huge amount of flexibility is left to the languages to choose exactly what scope and precedence rules make sense for their language.<p>[1] efficient hardware support for such fine-grained static rounding is still somewhat lacking in the commodity CPU world. On GPUs and some other compute devices, it is quite natural (and &quot;dynamic&quot; rounding is sometimes quite a hassle). AVX-512 will bring support for per-instruction static rounding to mainstream CPUs.<p>When we look at flags, the situation is much the same. Languages completely specify the scope of flags. There is no requirement of mutable global state. For example:<p>&gt; (7.1) Language standards should specify defaults in the absence of any explicit user specification, governing ... whether flags raised in invoked functions raise flags in invoking functions.<p>Like with rounding, current commodity CPUs make it <i>easier</i> to provide flags with thread scope, but IEEE-754 does not require it. Commodity hardware works the way it does because mainstream languages work that way. If a different model makes sense for your language, do that.<p>Finally, the concern about &quot;exceptions&quot; is entirely misplaced. &quot;Exception&quot; in IEEE-754 simply means &quot;an event that occurs when an operation on some particular operands has no outcome suitable for every reasonable application,&quot; which is a rather different meaning than the way &quot;exception&quot; is understood in colloquial PL usage. Under default exception handling, which is all that IEEE-754 requires implementations to support, all that needs to happen in the case of an exception is for the implementation to raise the corresponding flag, the scope of which is (as previously discussed) up to the language to specify.<p>I would encourage you to direct questions like these about the spec to committee members. If you work for a big company, a few members probably work with you. If you don&#x27;t, most committee members are happy to answer questions, even from people they don&#x27;t know.<p>The concerns about access to the spec itself and to the minutes are well-placed, and definitely something that the committee is aware of. (But mostly out of the committee&#x27;s hands; it&#x27;s up to IEEE to set pricing. Send them your comments!)
评论 #8579051 未加载
评论 #8579821 未加载
sunfishover 10 years ago
The mutable global state problem is hard. And, its implications are woven throughout the spec. For example, pow(0, nan) is 0, not nan, because that&#x27;s slightly more convenient in some cases, and it&#x27;s assumed that you can always check the Invalid flag to see if any nans were produced and swallowed.<p>At the same time though, it&#x27;s not designed that way accidentally or in ignorance of the problems it creates. IEEE-754 knew that programming languages wouldn&#x27;t be very happy about global state, and chose to keep it because they believed it was still the best approach. In many other areas, IEEE-754 pushed against people who said it would be too hard to implement, and in retrospect they ended up being right in many cases. It&#x27;s tempting to wonder if global mutable state really was too much of a tradeoff though, in retrospect.
评论 #8578652 未加载
emiliobumacharover 10 years ago
Very actionable suggestions. Hopefully some of the ieee folks are reading.
dosshellover 10 years ago
Can&#x27;t we make an opensource description of the IEEE754 which resolves the issues? While the spec doesn&#x27;t describe the same thing it should be no legal problems - right?
评论 #8579942 未加载
Sanddancerover 10 years ago
This is a rather poor rant. There&#x27;s pretty much an easy answer to everything he describes that&#x27;s been done in the academic world, and in other programming languages for decades, and almost certainly in his own programming language for similar problems.<p><pre><code> Can I copy it into my own spec or would that be copyright infringement? Can I write something myself that means the same? Or do I have to link to the spec and require my users to pay to read it? </code></pre> Giving a summary is allowed even in the most draconian of interpretations of copyright. More loosely, pulling a quote of a few lines is standard for any sort of academic exercise; essays on novels certainly don&#x27;t have you pull out the novel when they want to quote a bit of text.<p><pre><code> What if your language doesn’t have exceptions? What does it mean to provide a status flag? </code></pre> You certainly have places elsewhere in your language for handling errors. How do you handle integer divide by zero? How do you handle timeouts when doing networking? How do you handle disk full errors? I hope you don&#x27;t go stomping along after an error in those cases.<p><pre><code> This may have made sense at the time of the original spec in 1985 but it’s mutable global state – something that has long been recognized as problematic and which languages are increasingly moving away from. </code></pre> This is because statelessness is a leaky abstraction. Your machine&#x27;s saving to disk, it&#x27;s allocating memory for other things, it&#x27;s pulling in stuff and shooting stuff over the network. Floating point has lots of knobs that you need to turn because there are various rounding rules for various problem domains. You&#x27;ve almost certainly got a type system for your language, perhaps your answer is to create, or have a way to create, a floatRoundUp type for when a user needs to go beyond your default rules. In a pure functional definition, I think one way to consider the problem is to think of a floating point type as the actual number and the state registers. If you don&#x27;t care, then just use the number part, if the state register matters, put those in too. Saving flags means that you can get repeatability. With a given set of flags, you&#x27;ll always get the same answer.<p><pre><code> Program block? Attribute specification? Nothing else in my language works this way. How do I even apply this in practice? If attributes are lexically scoped then that would mean rounding modes only apply to code textually within the scope – if you call out to a function its operations will be unaffected. If on the other hand attributes are dynamically scoped then it works with the functions you call – but dynamically scoped state of this flavor is really uncommon these days, most likely nothing else in your language behaves this way. </code></pre> There are two ways of handling this problem. Decide for the user, or expose the functionality needed to provide scoped floating point. The most &quot;pure&quot; way sounds like using the aforementioned type system to give defaults, but have a way for the user to tweak the rules as needed. You&#x27;re the designer, one of your jobs is to make these sorts of decisions for the user. Things like your i&#x2F;o library probably have similar problems, I don&#x27;t see why floating point is all that much different.
评论 #8578965 未加载
GFK_of_xmaspastover 10 years ago
$88 is not a lot of money for a professional software person, especially if you can get your employer to pay for it.