Maccarone: AI-managed code blocks in Python

It&#x27;s so awesome to learn a new term like &quot;macaronic language&quot;.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Macaronic_language" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Macaronic_language</a><p>I finally have the right term to describe the warning signs from 1960&#x27;s-era mainframes that coined &quot;blinkenlichten&quot;.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Blinkenlights" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Blinkenlights</a><p>There should be a German term for this, but &quot;gefälschter deutscher&quot; doesn&#x27;t quite capture it.

&gt; What prevents my program from behaving differently after each preprocessing run?<p>&gt;The strength of your faith in GPT-4.<p>I got a chuckle out of that

In theory, an AI that wrote proofs for their code (ala coq) could be used to validate preconditions specified by the developer, right?

Very cool project. How reliable are you finding your prompts? They look like good choices based on my experience prompting GPT-3.5 and 4 for code editing.<p>FYI, I think my open source tool aider would work out of the box to serve this use case. You would just run:<p><pre><code> aider file.py —msg “implement the comments” </code></pre> Of course aider works with any popular language, not just python. And it can do a lot of other coding tasks. It&#x27;s like pair programming with an AI.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;paul-gauthier&#x2F;aider">https:&#x2F;&#x2F;github.com&#x2F;paul-gauthier&#x2F;aider</a>

Isn&#x27;t this how copilot &#x27;just&#x27; works, except with comments? What&#x27;s the advantage over copilot?

Copilot currently keeps in context the file you are editing. Cross file support is coming but not here.(<a href="https:&#x2F;&#x2F;githubnext.com&#x2F;projects&#x2F;copilot-view&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;githubnext.com&#x2F;projects&#x2F;copilot-view&#x2F;</a>). But it would be very very useful.<p>One logical concept that&#x27;s also been noodling in my brain was to construct a DFA(Deterministic finite automata) from the code seen in all the files and then offer the n-1 tokens to the language model and constrain the nth token&#x27;s selection from the valid ones. I recall someone did this for things that produce DFAs that are fairly small in size(like JSON) and that essentially produced 100% valid JSON without hallucinations(It could be garbage JSON).<p>So for example if I had a `class ABC` then typing `abc.` could produce: 1. all the methods on it that were valid and 2. had arguments from the surrounding code informed by the LLM.

It&#x27;s like in-painting, but for code :)

I love it.<p>I&#x27;d like to make something more constrained. Instead of a fully-general programming language, let the LLM configure data-flows between pre-defined modules, field mappings, or presentations.<p>Then, hopefully, we could let the end-user more directly edit the prompt.

Would python decorators be better for something like this?<p>I always get squeamish when I see magic comments

Nice, it&#x27;s like cog (<a href="https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;cog&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;pypi.org&#x2F;project&#x2F;cog&#x2F;</a>), but automatic.<p>It could replace template rendering in the long run.

Assuming you&#x27;re using source control properly and read the diff before running it, I guess this is one way to make sure that a comment matches the code? If the bot changes it, maybe your comment wasn&#x27;t clear enough?

How hard would it be to use code llama instead? <a href="https:&#x2F;&#x2F;ai.meta.com&#x2F;blog&#x2F;code-llama-large-language-model-coding&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;ai.meta.com&#x2F;blog&#x2F;code-llama-large-language-model-cod...</a>

Not sure I see the benefit over standard ai integration into an editor, what am I missing?

I tried implementing something like this over the summer but couldn&#x27;t make progress with the 20-30 minimum response time for each OpenAI-generated block. From the demo video it looks like this runs pretty fast -- or does it?

The name is foretelling of the end result: a flying spaghetti monster.

Looks amazing. Would you ever consider using Claude as well?<p>I prefer to use Claude for code generation if using a newer framework or language (the 2021 cutoff with gpt-4 is unfortunate)

I&#x27;m usually just copy pasting my entire file into chatgpt and ask it for help - it re-writes the whole damn thing, no need to have managed sections.

Seems likely that dev work moves towards this sort of thing - boilerplate being AI managed.<p>It&#x27;ll be hell to debug an ever shifting codebase though

This is really cool.<p>Practically, how often does this lead to new errors from the AI managed codeblocks when you update code elsewhere?

similar to this? <a href="https:&#x2F;&#x2F;www.askmarvin.ai&#x2F;welcome&#x2F;quickstart&#x2F;#ai-functions" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.askmarvin.ai&#x2F;welcome&#x2F;quickstart&#x2F;#ai-functions</a>

What happens if you start editing the code in a block?

This answer in the FAQ is wonderful:<p><pre><code> What prevents my program from behaving differently after each preprocessing run? - The strength of your faith in GPT-4.</code></pre>

&quot;Hallucination isn&#x27;t a real problem, people will always scrutinize the generated code!&quot;<p>Sigh...

There are so many languages with awesome type systems which can help guide AI to generate better code — and yet, these experiments always choose Python.