Should small Rust structs be passed by-copy or by-borrow? (2019)

A potential lesson here (i.e. I am applying confirmation bias to retroactively view this article as justification for a strongly held opinion, lol):<p>Unless you are gonna benchmark something, for details like this you should pretty much always just trust the damn compiler and write the code in the most maintainable way.<p>This comes up in code review a LOT at my work:<p>- &quot;you can write this simpler with XYZ&quot;<p>- &quot;but that will be slower because it&#x27;s a copy&#x2F;a function call&#x2F;an indirect branch&#x2F;a channel send&#x2F;a shared memory access&#x2F;some other combination of assumptions about what the compiler will generate and what is slow on a CPU&quot;<p>I always ask them to either prove it or write the simple thing. If the code in question isn&#x27;t hot enough to bother benchmarking it, the performance benefits probably aren&#x27;t worth it _even if they exist_.

I don’t feel like this gave a satisfactory answer the question. Since everything was inlined, the argument passing convention made no difference in the micro benchmarks. But what happens when it does not inline? Then you would actually be testing by-borrow be by-copy instead of how good rust is at optimizing.

This is one advantage of Ada, where parameters are abstractly declared as &quot;in&quot; or &quot;in out&quot; or &quot;out&quot;. The compiler can then decide how to best implement it for that specific size and architecture.

This is one of those questions where you really, honestly, do need to look at a very low level.<p>Back in the ancient days, I worked at IBM doing benchmarking for an OS project that was never released. We were using PPC601 Sandalfoots (Sandalfeet?) as dev machines. A perennial fight was devs writing their own memcpy using <i>dst++ = </i>src++ loops rather than the one in the library, which was written by one of my coworkers and consisted of 3 pages of assembly that used at least 18 registers.<p>The simple loop was something like X cycles&#x2F;byte, while the library version was P + (Q cycles&#x2F;byte) but the difference was such that the crossover point was about 8 bytes. So, scraping out the simple memcpy implementations from the code was about a weekly thing for me.<p>At this point, we discovered that our C compiler would pass structs by value (This was the early-ish days of ANSI C and was a surprise to some of my older coworkers.) and benchmarked <i>that</i>.<p>And discovered that its copy code was <i>worse</i> than the simple <i>dst++ = </i>src++ loops. By about a factor of 4. (The simple loop would be optimized to work with word-sized ints, while the compiler was generating code that copied each byte individually.)<p>If you are doing something where this matters, something like VTune is very important. So is the ability to convince people who do stupid things to stop doing the stupid things.

I always prefer by-borrow. That&#x27;s because in the future this struct may become non-copy and that means some unnecessary refactoring. My thinking is a bit like &quot;don&#x27;t take ownership if not needed&quot; - the &quot;not needed&quot; part is the most important thing. Don&#x27;t require things that are not needed.

&gt; Blech! Having to explicitly borrow temporary values is super gross.<p>I don’t think you ever have to write code like this. Implement your math traits in terms for both value and reference types like the standard library does.<p>Go down to Trait Implementations for scalar types, for instance i32 [1]<p>impl Add&lt;&amp;i32&gt; for &amp;i32<p>impl Add&lt;&amp;i32&gt; for i32<p>impl Add&lt;i32&gt; for &amp;i32<p>impl Add&lt;i32&gt; for i32<p>Once you do that your ergonomics should be exactly the same as with built in scalar types.<p>[1] <a href="https:&#x2F;&#x2F;doc.rust-lang.org&#x2F;std&#x2F;primitive.i32.html" rel="nofollow">https:&#x2F;&#x2F;doc.rust-lang.org&#x2F;std&#x2F;primitive.i32.html</a>

Oh neat, that’s my blog. My old posts don’t resurface on HN that often.<p>Lots of criticism of my methodology in the comments here. That’s fine. That post was more of a self nerd snipe that went way deeper than I expected.<p>I hoped that my post would lead to a more definitive answer from some actual experts in the field. Unfortunately that never happened, afaik. Bummer.

It&#x27;s compiled, so, without any investigation at all, I would have been disappointed if there were any significant difference in the code emitted in these cases. I would expect the compiler to do the efficient thing based on usage rather than the particular syntax. I may have too much faith in the compiler.

I&#x27;d be interested to know what the benchmarks of the two rust solutions are when inlining is disabled so we can get an idea of the different performance characteristics of each function call even if it&#x27;s not a very realistic scenario.<p>The other question I have is which style should you use when writing a library? It&#x27;s obviously not possible to benchmark all the software that will call your library but you still want to consider readability, performance as well as other factors such as common convention.

I would go with the version that gives the clean user interface (that is, by copy in this case). <i>If</i> it turns out that the other version is significantly more performant <i>and</i> this additional performance is critical for the end users consider adding the by-borrow option.<p>The clarity of the code using a particular library is such an big (but often under-appreciated) benefit that I would heavily lean in this direction when considering interface options. My 2c.

I just went through all of this when building a raytracer.<p>* Sprinkling &amp; around everything in math expressions does make them ugly. Maybe rust needs an asBorrow or similar?<p>* If you inline everything then the speed is the same.<p>* Link time optimizations are also an easy win.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;mcallahan&#x2F;lightray">https:&#x2F;&#x2F;github.com&#x2F;mcallahan&#x2F;lightray</a>

The benchmarks lack the standard deviation, so the results may well be equivalent. Don&#x27;t roll your own micro-benchmark runners.<p>References may get optimized to copies where possible and sound (i.e. blittable and const), a common heuristic involves the size of a cache line (64b on most modern ISAs, including x86_64).<p>Using a Vector4 would have pushed the structure size beyond the 64b heuristic. You would also need to disable inlining for the measured methods.

Note that this is from 2019, so it&#x27;s probably worth re-benchmarking to see if anything has changed in the interim. Can we get the year added to the title?

For this code, the compiler inlined the call. So there should be no difference between pass by copy or pass by reference, which is what was measured. Where it could matter is when the code isn’t inlined. But with small structs it might not matter all that much.<p>It does sometimes matter though. One optimization I’ve seen in a few places is to box the error type, so that a result doesn’t copy the (usually empty) error by value on the stack. That actually makes a small performance difference, on the order of about 5-10%.

Folks, processors continue to give smaller and smaller gains every year. Something has to give. If you have critical path code that absolutely must max out the core, then this type of analysis (as pedantic as it is) is useful in the long run.

It&#x27;s not like you can do arithmetic with references, so maybe the ergonomics of by-value vs. by-reference shouldn&#x27;t really be that different.<p>The cost of by-value lies in memory copies, while the cost of by-reference lies in dereferencing pointers where the values are needed, which might mean many more memory reads are needed than with by-value (depends on what you&#x27;re doing). So it&#x27;s just hard to tell which will do better in general -- there&#x27;s no answer to that.<p>For a library, maybe providing by-value and by-reference interfaces should be good (except that will bloat the library). For everything else just use by-value as it has the best ergonomics.

I did the test on my computer:<p>Rust - By-Copy: 14124, By-Borrow: 8150<p>C++ - By-Copy: 12160, By-Ref: 11423<p>P.S. Just built it using LLVM under CLion IDE and the results are:<p><pre><code> G:\temp\cpp\rust-cpp-bench\cpp\cmake\cmake-build- release\fts_cmake_cpp_bench.exe Totals: Overlaps: 220384338 By-Copy: 4397 By-Ref: 4396 Delta: -0.0227428% Process finished with exit code 0</code></pre>

This actually bothers me. I think the rust performance here is praise worthy. What bothers me is that we piled complexity over complexity at the hardware and compiler levels, and ended up in a situation where you got no way to get a reasonable understanding of how low level code will perform. Nowadays the main reason to program in a &quot;low level&quot; language is that you know that on average the compiler will be able to do a better job because the language doesn&#x27;t have abstractions that map poorly to the hardware model. But for much of it you can forget about &quot;I know what the hardware is going to do&quot;

I&#x27;m late to this discussion, sorry.<p>But at the risk of loss of respect, I&#x27;ll wait for Rust2ShinyNewLanguage to solve this.<p>All I know is I hope I&#x27;m smart enough to understand ShinyNewLanguage&#x27;s compiler. Or maybe even build it.<p>I&#x27;ve got several projects that could use some additional Boxes of structures, and borrow instead of move, and maybe a few more complex reference counting mechanics.<p>Rust forced me to understand what that meant. That&#x27;s good for building a better engineer.<p>But it&#x27;s not fun to work with.<p>I hope the next experience is better. Sorry Rustaceans.

There is no single answer to this question because it&#x27;s going to depend completely on call patterns further up. Especially in regards to how much of the rest of the running program&#x27;s data fits in L1 cache, and <i>most especially</i> in regards to what&#x27;s going on in terms of concurrency.<p>The benchmark made here could completely fall apart once more threads are added.<p>Modern computer architectures are non-uniform in terms of any kind of memory accesses. The same logical operations can have extremely varied costs depending on how the whole program flow goes.

My first thought was &quot;now what is the calling convention for float parameters again? they are passed in registers right? the compiler can probably arrange so they don&#x27;t have to actually be copied&quot; and then I realized it will probably even inline it.<p>Anyway, assuming it&#x27;s not inlined I would guess pass-by-copy, maybe with an occasional exception in code with heavy register pressure.<p>Edit: Actually since it&#x27;s a structure, the calling convention is to memory allocate it and pass a pointer, doh. So it should actually compile the same.

Anyone know why seemingly knowledgeable people (like the person who wrote this article) don&#x27;t use micro benchmarking frameworks when they run these tests?<p>Also, whenever you do one of these, please post the full source with it. There&#x27;s no reason to leave your readers in the dark, wondering what could be going on, which is exactly what I&#x27;m doing now, because there&#x27;s almost no excuse for c++ to be slower in a task than rust--it&#x27;s just a matter of how much work you need to put in to make it get there.

The general usability impact matters slightly less than it looks here, in part because the `do_math` with references in the article has two extra &amp;s, and in part because methods autoreference when called like x.f().<p>Performance-wise, if you&#x27;re likely to touch every element in a type anyway, err on the side of copies. They are going to have to end up in registers eventually anyway, so you might as well let the caller find out the best way to put them there.

The Rust-test implements the traits Add, Sub, Mul by value. This makes the few references less important in the total test. The ergonomics argument is motivated by using these traits. Otherwise, references would have had the same ergonomics.<p>But also, the struct is 3x32 bits, and Rust auto-implements the Copy-trait for it. It is barely larger than u64, which is the size of the reference.<p>But life is only simpler when Copy and Clone can be auto-implemented.

I covet Ada’s feature where you just specify if a parameter is in, out, or inout; the compiler’s figures out whether to copy or pass a pointer.

I haven&#x27;t benchmarked that, but in Rust `ScalarPair`s (i.e., structs who have up to to scalars) are passed in two registers, while bigger structs are always passed by pointer. Therefore, passing bigger structs by move will require the compiler to copy them, while with references it is not required to, so references may be faster in that case.

I understand that this is an example for the purposes of answering the given question, but when actually doing things with 3D vertices one should be thinking in terms of structures of arrays. As someone said here already: good generals worry about strategy and great generals worry about logistics.

I&#x27;m surprised he tested MSVC and Clang, and not GCC which usually generates faster code than those two.

You are comparing two completely different compilers; I wouldn&#x27;t worry all that much about the difference between rust and C++. If you do want to compare them directly, why not use LLVM for C++ as well? That will highlight any language-specific differences.

Discussed at the time:<p><i>Should small Rust structs be passed by-copy or by-borrow?</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=20798033" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=20798033</a> - Aug 2019 (107 comments)

This is not really surprising in such a case. The Rust compiler is pretty good at optimizing out uneeded copies. Here it does see that the copied value is not used after the function call, so it should simply not emit the copies in the final assembly.

Minor nit: many of the differences in the article aren&#x27;t really specific to the Rust vs C++, but rather differences between llvm vs whatever compiler backend is used by msvc.

Interesting! Of note, My `Vec3` and `Quaternion` types (f32 and f64) have `Copy` APIs, but I&#x27;ve wondered about this since their inception.

I don&#x27;t know much about this, but by-copy sounds nice. I&#x27;d rather own stuff and be happy.

Should have also tried pass-by-move .

A more direct comparison would have been a r-value reference.

So c++ is less complicated than rust in some cases?

This is one of the problems I have with writing rust code. You have to think about so many mundane details that you barely have time left to think about more important and more interesting things.

I guess by-copy bc I’m cool

It is a problem of statistics and depends on internals of underlying operating system. I’m not sure you really need that sort of optimisation