This is known as the "small string optimisation" in C++, so you can see a similar implementation in Clangs libc++[1].<p>One interesting corollary is that <i>moving</i> short strings in an implementation that does this could actually be ever so slightly (negligibly) slower than moving long ones (since byte copies are slower than word copies). But generally, this is a free lunch optimisation and can save you hundreds of megs of memory when writing programs dealing with millions of short strings.<p>[1] <a href="http://llvm.org/svn/llvm-project/libcxx/trunk/include/string" rel="nofollow">http://llvm.org/svn/llvm-project/libcxx/trunk/include/string</a> - search for "union"
<a href="http://www.slideshare.net/nirusuma/what-lies-beneath-the-beautiful-code" rel="nofollow">http://www.slideshare.net/nirusuma/what-lies-beneath-the-bea...</a> (from march 2012) also discusses this.<p>Also (pedantic):<p><pre><code> #define RSTRING_EMBED_LEN_MAX ((int)((sizeof(VALUE)*3)/sizeof(char)-1))
</code></pre>
sizeof(char) is always 1, so that division is superfluous.
There's some discussion at <a href="https://news.ycombinator.com/item?id=3425164" rel="nofollow">https://news.ycombinator.com/item?id=3425164</a> , including some interesting technical/benchmarky comments.
Title: "Never create Ruby strings longer than 23 characters"<p>Conclusion: "Don’t worry! I don’t think you should refactor all your code to be sure you have strings of length 23 or less."
Wouldn't it be better to use this declaration though:<p><pre><code> struct RString {
struct RBasic basic;
union {
struct {
long len;
char *ptr;
union {
long capa;
VALUE shared;
} aux;
} heap;
char ary[];
} as;
};
/* apologies if I messed up the syntax here */
#define RSTRING_EMBED_LEN_MAX (sizeof(((RString*)(0))->as) - 1)
</code></pre>
Then you can even use the padding the compiler added, if any, plus you can add more things to heap and the embed length will grow automatically.
For anyone interested, he points to an older translation of the Ruby Hacking Guide, there is a pretty much complete translation at<p><a href="http://ruby-hacking-guide.github.com" rel="nofollow">http://ruby-hacking-guide.github.com</a>
I suppose the thing to do is analyse your app for the average string length, and just recompile your Ruby with that. Would be even better of it was a command line parameter.
Extremely interesting. But I cannot quite understand why RSTRING_EMBED_LEN_MAX is calculated that way.<p>VALUE seems to be unsigned int defined via "typedef uintptr_t VALUE;" and "typedef unsigned __int64 uintptr_t;"<p>But why is it calculated like that I don't get. Anyone can explain?
I wonder why they didn't make cut-off optimization points at 33?<p>When programmers don't know in advance how long name/email/input/whatever field is going to be - they just use the magic "power of two" length :)<p>So 32 (or 33) in this case would be more reasonable.
Reminds me of this Mr Show sketch :) <a href="https://www.youtube.com/watch?v=RkP_OGDCLY0" rel="nofollow">https://www.youtube.com/watch?v=RkP_OGDCLY0</a>
This all sounds rather terrible for Ruby, doesn't it? It isn't so much that the short string is faster (though I'm left unclear whether it itself is on the stack/heap, though given the GC nature of Ruby and practical considerations of the language, it must be the heap), but rather that the cost of the short string is <i>also</i> added to the long string in the heap (assumed) allocation of the RString (which becomes larger and thus more difficult to malloc).<p>If this is intended to sit on the stack, which I find highly unlikely (especially given the timings that seem to be the delta between one malloc and two, and would be much more significant if it were a stack allocation versus a heap allocation. This is not comparable to small string optimizations for the stack in C++), maybe. But otherwise it seems like a poorly considered hack.<p>The string type could as easily have been dynamically allocated based upon the length of the string, where the ptr by default points inside that same allocated block. If the string is expanded it can then be realloced and the string alloced somewhere else. No waste, a single allocation, etc.