Finding a CPU Design Bug in the Xbox 360

I&#x27;m now wondering if I have enough material to do an interesting writeup for my time as a CPU bug-hunter in verification.<p>The client (a now vanished startup) had a small 8-bit CPU design which they wanted validation for, using the technique of executing random sequences of instructions and comparing the result against an emulator. We wrote the emulator independently from their architecture description. Given that most instructions were a single byte plus arguments and most of those were valid, the test coverage was pretty thorough. All looked fine until I added support for interrupts, at which point we discovered that an interrupt during a branch would not return to the correct point in execution.<p>Verifying security properties of processors is <i>really hard</i>; you can go looking for specific types of failure, but I&#x27;m not aware of a general way of proving no information leakage.

Article Excerpt: &quot;So a speculatively-executed xdcbt was identical to a real xdcbt!&quot;<p>I&#x27;ve never thought about it until I saw the above line in the article, and that thought went something like this:<p>&quot;Assembler instructions which might never have the conditions met for their execution during a program&#x27;s runtime might be speculatively executed nonetheless, and this, depending on the nature of the instruction executed, might have huge ramifications up to and including program incorrectness and even program failure.&quot;<p>In other words, your absolutely 100% deterministic Turing machine (or programs that you write on it that you deem to be 100% deterministic) -- may not be quite so deterministic when viewed against these understandings...<p>It adds a whole new dimension to what must be thought about when writing assembler code...<p>Anyway, it&#x27;s a really great article!

Awesome story! I&#x27;m curious though why the branch predictor was running the xdcbt instruction if &quot;The symptoms were identical. Except that the game was no longer using the xdcbt instruction&quot;.<p>Was the game no longer &quot;using the xdcbt instruction&quot;, but the branch predictor caused issues, because they put a jmp instruction in front of it instead of removing the instruction entirely?

Would it be fair to say that having any sequence of bytes in memory which looks like the <i>xdcbt</i> instruction (even if those bytes are just data) is unsafe? Given that a stale entry in the branch prediction table might end up pointing at those bytes.

This issue that xdcbt is supposed to solve happens at a completely different level. Performing a full disk to tape backup would slow systems down because the entire disk would be copied through the OS buffer cache, evicting data used by other processes.<p>The UNIX fix for this was to use a raw device or O_DIRECT to bypass the buffer cache.<p>Maybe Intel&#x27;s new cache partitioning feature offers a similar fix, see:<p><a href="https:&#x2F;&#x2F;lwn.net&#x2F;Articles&#x2F;694800&#x2F;" rel="nofollow">https:&#x2F;&#x2F;lwn.net&#x2F;Articles&#x2F;694800&#x2F;</a><p>Actually in the comments someone mentions using cache partitioning for security. Maybe the threads used by jit code could be placed in their cache partition to avoid some of Spectre.

Correct me if I&#x27;m wrong, but won&#x27;t Meltdown&#x2F;Spectre bug allow people to jailbreak pretty much any os&#x2F;device that has CPU that &quot;supports&quot; these bugs? This could potentially open a lot of currently closed devices to people.

I have seen <i>so many</i> bugs and crashes created exactly because of this kind of thinking:<p>&gt; [insert X here] was no longer guaranteed, but hey, we’re video game programmers, we know what we’re doing, it will be fine.

Reading about the architecture of that chip (3 core, PowerPC) I am amazed at how smooth GTA V could run on it.

Where are the L1 caches?

Great read!

Would gcc&#x27;s &quot;__builtin_expect&quot; construct help in these cases?

Very well written. Thanks for sharing!

<i>Scratches PowerPC off the list of trustworthy CPUs</i><p>Sooo, out you go, G5. Any suggestions what to use for online banking? A 486 can&#x27;t handle all the jQuery and tracking scripts.