Using ChatGPT to generate a GPT project end-to-end

Every post that claimed using ChatGPT to achieve non-trivial tasks turned out to have non-trivial human intervention.<p>&gt; (from the original article) In fact, I found it better to let ChatGPT generate a toy-ish version of the code first, then let it add things to it step-by-step. This resulted in much better output than, say, asking ChatGPT to generate production-quality code with all features in the first go. This also gave me a way to break down my requirements and feed them one at a time - as I was also acting as a code-reviewer for the generated output, and so this method was also easier for me to work with.<p>It takes a human who really knows the area to instruct ChatGPT and review the output, point out silly mistakes in the generated non-sense, and start next iteration. This kind of curated posts always cut off the most part of the conversations and the failed attempts, and then concatenate successful attempts with outputs of quality. Sure, it will be helpful as a super-IntelliSense. But not as helpful as the post suggested.<p>I&#x27;ve tried to do something like in the post, but I was quickly bored with waiting output, reviewing, all the iterations. One important aspect about programming is that reading code may not be easier than writing code. And in my case, it&#x27;s more painful.

This is not really the same, but may be interesting to some: I subscribe to ChatGPT plus for a month to check out GPT-4. The rate limits were cumbersome though and it can be easy to waste a prompt, so I started to bootstrap:<p>I would explain my problem to 3.5 and ask it to suggest comprehensive prompts to use with 4 to maximize my limited quota. It worked very well.<p>In the long years to come the most advance AIs may become so far removed from us that the best intermediaries will be their less advanced brethren.

Cool -- I did something similar with the goal: Imagine and simulate an instrument that doesn&#x27;t exist and ended up with this -- it even created the assets or prompts for other AIs to make assets where it couldn&#x27;t, including the model<p><a href="https:&#x2F;&#x2F;pwillia7.github.io&#x2F;echosculpt3&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pwillia7.github.io&#x2F;echosculpt3&#x2F;</a>

I had chat gpt 3.5 build a small web app for me too. I have since been building some tooling for this sort of GPT-assisted programming.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;paul-gauthier&#x2F;easy-chat">https:&#x2F;&#x2F;github.com&#x2F;paul-gauthier&#x2F;easy-chat</a>

It&#x27;s good at writing new code, with sufficient prompting. But the big open question as of now for engineering orgs is - can it edit existing code like developers, just by instructions. Is there any hands on experience anyone has on copilot-x?

I directed chatGPT to code <a href="https:&#x2F;&#x2F;chrome.google.com&#x2F;webstore&#x2F;detail&#x2F;franz-ai-text-rewriter&#x2F;lclcncbdhfpolppomniofbkiaaebaefg?hl=en" rel="nofollow">https:&#x2F;&#x2F;chrome.google.com&#x2F;webstore&#x2F;detail&#x2F;franz-ai-text-rewr...</a> a chrome extension which rewrites any content in place on any website.<p>not a single line of code was written by me by hand.<p>but damn many prompts.

I&#x27;ve also made six small apps completely coded by ChatGPT (with GitHub Copilot contributing a bit as well). Here are the two largest:<p>PlaylistGPT (<a href="https:&#x2F;&#x2F;github.com&#x2F;savbell&#x2F;playlist-gpt">https:&#x2F;&#x2F;github.com&#x2F;savbell&#x2F;playlist-gpt</a>): A fun little web app that allows you to ask questions about your Spotify playlists and receive answers from Python code generated by OpenAI&#x27;s models. I even added a feature where if the code written by GPT runs into errors, it can send the code and the error back to the model and ask it to fix it. It actually can debug itself quite often! One of the most impressive things for me was how it was able to model the UI after the Spotify app with little more than me asking it to do exactly that.<p>WhisperWriter (<a href="https:&#x2F;&#x2F;github.com&#x2F;savbell&#x2F;whisper-writer">https:&#x2F;&#x2F;github.com&#x2F;savbell&#x2F;whisper-writer</a>): A small speech-to-text app that uses OpenAI&#x27;s Whisper API to auto-transcribe recordings from a user&#x27;s microphone. It waits for a keyboard shortcut to be pressed, then records from the user&#x27;s microphone until it detects a pause in their speech, and then types out the Whisper transcription to the active window. It only took me two hours to get a working prototype up and running, with additions such as graphic indicators taking a few more hours to implement.<p>I created the first for fun and the second to help me overcome a disability that impacts my ability to use a keyboard. I now use WhisperWriter literally every day (I&#x27;m even typing part of this comment with it), and I used it to prompt ChatGPT to write the code for a few additional personal projects that improve my quality-of-life in small ways. If people are interested, I may write up more about the prompting and pair programming process, since I definitely learned a lot as I worked through these, including some similar lessons to the article!<p>Personally, I am super excited about the possibilities these AI technologies open up for people like me, who may be facing small challenges that could be easily solved with a tiny app written in a few hours tailored specifically to their problem. I had been struggling to use my desktop computer because the Windows Dictation tool was very broken for me, but now I feel like I can use it to my full capacity again because I can type with WhisperWriter. Coding now takes a minimal amount of keyboard use thanks to these AI coding assistants -- and I am super grateful for that!

As someone who always wanted to work on multiple projects but was lacking the time and manpower. GPT has truly made that possible. Looking forward to GPT5.

You don’t mention it explicitly, but I assume you’ve manually copied and pasted all the code, as well as the various patches with updates? In my experience, that quickly makes new suggestions from ChatGPT go out of sync with the actual state of the code. Did you occasionally start the conversation over and pasted in all the code you currently had, or did this not turn out to be an issue for you?

Combine TDD and Self debugging into a workflow and you almost have a new paradigm of software development where entire applications can be developed with a series of prompts. Software programmers have finally programmed themselves out of jobs! It&#x27;s kind of poetic justice that LLMs trained on open source code is replacing us.<p>We should have never listened to Richard Stallman. &#x2F;s

Good preview of the near future of software dev, but I’m also wondering of the trend of companies forbids use of AI generated code do to copyright ambiguity. I suppose pressure to reduce cost as and move faster will overcome that concern.

Great article! It reminds me a lot of becoming a manager, especially this line<p>&gt; Finally, it was also tiring. Imagine being reduced to giving only instructions and doing code review. Reading and understanding code is tiring!

I liked this metaphor:<p>&gt;It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful.

I really appreciate the well documented, well commented code.<p>It&#x27;s usually hard to find energy to do this on personal projects, if nothing else that aspect makes GBT coding worth it to me<p>Especially in this domain, most of the pytorch projects are impossible to read with the lack of comments and obscure variable names

When you actually work with a high level engineer they can do a lot automouusly and can cut through ambiguous instructions based on experience, but they also require interactions that clarify important decision points and there are many. Gpt-x is miles away from this outcome

What value does a developer deliver when their entire process is done by an LLM?<p>Is there no desire for a creative process?

Is anyone letting an LLM code and run its code by itself, then iteratively fix any bugs in it without human intervention until it e.g. passes some black box tests?<p>Would it be possible to significantly improve an LLM using such unsupervised sessions?

&gt;It was more like handholding a fresh grad who had absorbed all of human knowledge but needed someone to tie various parts of that knowledge to create something useful.<p>10 INT but 0 WIS, that&#x27;s a good mental model for LLMs.

I&#x27;ve found the best way to pair program with ChatGPT is with GPT4 API through a VSCode extension by @jakear [1] that uses the Notebook interface. Instead of setting a language for each cell, you set roles like &quot;system&quot;, &quot;user&quot;, or &quot;assistant&quot; and when you run a cell it sends the cells as chat messages.<p>A huge benefit of this format is that you can delete cells, edit the responses from GPT4 to incorporate changes from future queries, and even rearrange or add mock assistant messages to prime the conversation. As ChatGPT suggests changes, I incorporate them into the main code cells and replace the old queries&#x2F;feedback with new queries feedback. Since the old changes are incorporated into the parent cells, it loses track a lot less and I can also touch it up to use the right file paths, APIs, etc when it messes up.<p>You can go a step further and monitor the llm file with inotify and extract assistant messages, infer the file path from the responses, and automatically write them to file as you update the notebook. That eliminates the back and forth copy pasting.<p>It&#x27;d be nice to extend that interface to include Jupyter notebook cells so we can use ChatGPT to generate notebook cells that can be parsed and executed in the interface directly.<p>Edit to add another tip: I use a variation of the below system prompt for working on larger sessions. Each user message begins with a file path and contains a code block with the contents of the file. After each user message containing a file, I manually add an assistant message that just says &quot;continue&quot;, which allows adding several files at different paths. The last user message, the one I actually execute, contains the &lt;request&gt; tokens and the description of the modifications I want in the code. I incorporate the suggested changes into the messages then rinse and repeat. Prompt (sadly I forgot to record where I found it):<p><pre><code> You are a Rust AI programming assistant. The user will send you the relevant code over several requests. Please reply &quot;continue&quot; until you receive a message from the user starting with the tokens &quot;&lt;request&gt;&quot;. Upon receiving a message from the user starting with the tokens &quot;&lt;request&gt;&quot; please carry out the request with reference to the code that the user previously sent. Assume the user is a senior software engineer who needs minimal instruction. Limit your commentary as much as possible. Under ideal circumstances, your response should just be code with no commentary. In some cases, commentary may be necessary: for example, to correct a faulty assumption of the user or to indicate into which file the code should be placed. </code></pre> [1] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35605159" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=35605159</a>

I just asked ChatGPT (v4) to write an app (a trivial one), and it refused:<p><i>As an AI language model, I&#x27;m unable to write an entire app for you.</i><p>I guess they&#x27;ve shut down that capability...

&gt; Also ChatGPT is bad at dealing with abstractions beyond 2 layers.<p>I wonder if this will be true in 5 years.

Why the random forgetfulness?

The technological singularity is approaching fast.<p>It&#x27;s pretty clear that, given terminal access and an appropriate outer loop, GPT models can iteratively create new GPT models (either by writing and executing Pyhton code, or later versions trained on LLM weights may even be able to output new weights directly). If the inner workings of the loop are sufficiently obfuscated (in the code-based version), it wouldn&#x27;t necessarily be clear to us what had changed, and the model weights&#x2F;architecture on their own are not interpretable. That&#x27;s very close to a singularity definition (machines that self-improve faster than what our understanding can keep up with), the only open question is whether the new versions would actually be improvements. But that sounds solvable too (except that it wouldn&#x27;t be easy for us to tell for sure).