In case you're curious about exploitability for code execution... the bug is that this array (Kmax = 16):<p><pre><code> static Bigint *freelist[Kmax+1];
</code></pre>
which is indexed by the log2 of the size of the bigint, is not checked:<p><pre><code> static Bigint *
Balloc(int k)
{
[...]
if ((rv = freelist[k]) != 0) {
freelist[k] = rv->next;
}
</code></pre>
If it's 0, then the allocation is done safely.<p>Where does Balloc get called? First with an estimate of the size required for the whole thing, from two locations, then with steadily increasing values starting from 1, from the mult function. With the first allocation we can theoretically perform a complicated operation (a good thing when ASLR is involved) to any location in ruby's bss section after freelist, but every increase requires doubling the size of the input string, and 16 already requires a 300k string, so going more than a few notches forward is impractical. With the second, I think k=16 is guaranteed to get a hit, because the next variable after freelist in the code is:<p><pre><code> static Bigint *p5s;
</code></pre>
On my system, and probably on all, this duly shows up immediately afterwards in the binary, and p5s is set before the allocation, so it's not null. The code ends up "allocating" p5s and copying 64k of arbitrary data into p5s->x, which comes from the 'private_mem' static array, also in the bss section. Although this is certainly dangerous, on the 3 systems I tried (OS X, 64-bit Linux, 32-bit Linux), there was much less than 64k of memory in the segment after the array and something read-only immediately followed, so it was guaranteed to crash before anything interesting could happen. It could be possible to exploit anyway if another C thread were accessing the data (not sure if mri ever does this), but the timing would be extremely difficult.<p>So I suspect that on most systems, this is purely a DOS, even though it involves an overwrite of arbitrary data. However, I could definitely be missing something, as the code is fairly complicated.