Logit Prisms: Decomposing Transformer Outputs for Mechanistic Interpretability

Off topic: can anyone tell me what the author has used to convert their paper into this beautiful webpage? It's apparently a trend with ML papers to have their own website.

If you&#x27;re reading this, you may be interested in my other work on &quot;Exploring Llama-3 MLP Neurons&quot; as well.<p><a href="https:&#x2F;&#x2F;neuralblog.github.io&#x2F;llama3-neurons&#x2F;" rel="nofollow">https:&#x2F;&#x2F;neuralblog.github.io&#x2F;llama3-neurons&#x2F;</a>

Tangent, the screen reader recording is cool. The main issue for me is that it doesn’t know where I am on the page, so it just starts from the beginning and I need to guess the position of the relevant audio based on how far down on the page I am. It would be cool to have some sort of bookmarks that match both the text and the audio, so that I can skip both to the same section at the same time

Okay so what stood out to me here is the math work: namely, if we can identify the parts of the model which are doing the math... Can that be &quot;augmented&quot; to generalise it? Could you take over the math neurons and either optimise them by hand or plug in a software &quot;implant&quot; to do the work properly?