This seems to keep coming up, and I see confusion in the comments. There is a standard: IEEE 754-2008. There are <i>additional</i> things people add like approximate reciprocals and approximate sqrt. But if you don't use those, and you don't make an association error, you get consistent results.<p>The question here with association for summation is what you want to match. OP chose to match the <i>scalar</i> for-loop equivalent. You can just as easily make an 8-wide or 16-wide "virtual vector" and use that instead.<p>I suspect that an 8-wide virtual vector is the right <i>default</i> for people currently, since systems since Haswell support it, all recent AMD, and if you're using vectorization, you can afford to pay some overhead on Arm with a double-width virtual vector. You don't often gain enough from AVX512 to make the default 16-wide, but if you wanted to focus on Skylake+ (really Cascadelake+) or Genoa+ systems, it would be a fine choice.
Exact floating point accumulating is more or less solved with xsum [1] -- would it work in this context?<p>[1] <a href="https://gitlab.com/radfordneal/xsum" rel="nofollow">https://gitlab.com/radfordneal/xsum</a>
Invariance w floating point arithmetic seems like a fool's errand. If the numbers one is working with are roughly on the same order of magnitude than I would consider integer / fixed point instead. You get the same results in this case (as long as you are careful).
see also streflop (2006)<p><a href="https://nicolas.brodu.net/en/programmation/streflop/" rel="nofollow">https://nicolas.brodu.net/en/programmation/streflop/</a>
I'm still wondering if there could exist an alternative world where efficient addition over decimal numbers that we developers use on a day to day basis is associative. Is that even possible or is there perhaps some fundamental limit that forces us to trade associativity for performance?<p>It seems to me that non associative floating point operations force us into a local maximum. The operation itself might be efficient on modern machines, but could it be preventing us from applying other important high level optimizations to our programs due to its lack of associativity? A richer algebraic structure should always be amenable to a richer set of potential optimizations.<p>---<p>I've asked a question that is very much related to that topic on the programming language subreddit:<p>"Could numerical operations be optimized by using algebraic properties that are not present in floating point operations but in numbers that have infinite precision?"<p><a href="https://www.reddit.com/r/ProgrammingLanguages/comments/145kps7/could_numerical_operations_be_optimized_by_using/" rel="nofollow">https://www.reddit.com/r/ProgrammingLanguages/comments/145kp...</a><p>The responses there might be interesting to some people here.