I see a lot of replies here talking about asking GPT questions in a domain an individual is an expert in, and being disappointed in the results. I think this is missing the point a bit. The impressive part of GPT isn’t that it’s an expert in anything, but that it’s better than an average human on many topics. For example - someone brought up GPT not comprehending homomorphic encryption fully. Well, I have a MS in CS and have been professionally developing software for 15 years, and I’ve even READ articles on homomorphic encryption multiple times, but I can’t articulate what it is very well. I just remember it’s inefficient and infeasible at this time.<p>The thing that is mind blowing to me is that I can talk with GPT about basically any topic and it can help get me in the right direction, I can talk with it to get clarity on anything that’s confusing, and it’s like a way to collaborate on thought without a human. That’s mindblowing to me!
ChatGPT4 blew my mind when I asked it some specific details about my PhD thesis. It was a relative simple question, but you needed to know some domain knowledge to give the correct answer. ChatGPT not only gave a wrong answer, it completely made up a story was beyond repair. It is concerning (blew my mind) how confident ChatGPT can provide a completely wrong answer.
I recently used it (in combination with copilot) to help me implement weighted, blended, order-independent transparency for a network visualization project I'm working on... in just one long evening.<p>Only to realize in the process that all I really needed was to modify the shader program's depth checking behavior. Even just disabling it altogether looks fine and makes everything way faster. Never in my wildest imagination a year ago would I even dream of implementing WBOIT in one night and then just leaving it in a branch for a one line change.<p>In the process I realized a little trick on my own, using what I learned, that modifying the depth values in the fragment shader could allow selective alpha-blending/occlusion. Basically the idea was to put stuff I wanted to blend order-independently all at an equal z position and then set the depth function to "less than or equal" instead of "less than". I used the occluding geometry's depth buffer as a source to modify the translucent stuff's depth buffer in the fragment shader (with a slight offset to get it behind the occluding stuff).<p>There's absolutely no way I'd have had the guts to try all that in one night without the robot tools.
I've been automating some of our team's most menial tasks and since we're in a corporate environment we don't have control over what is and isn't installed on our servers, so using bash is the sane default choice.<p>None of the scripts it produced are incredible or whatever (the tasks being more menial than lohic-based), but considering the intricacies of bash and how often I just faceplant into them as an experienced sysadmin, if feel as though the ability to just say "write a script that compares a list of agents against every manager and outputs the ones not found anywhere" and have it output basically a working script first try is impressive.
As a side project I've been writing a tool that drives GPT4 for coding purposes. It finds FIXME comments and equips the LLM with a set of high level shell commands it can use to explore the codebase and edit the files. This works well and I find it superior to the (often semi-broken) chat UI that OpenAI provide. There's no need for copy paste, and it can be given instructions like<p><pre><code> // FIXME: Implement this class paying attention to the unit tests
</code></pre>
at which point it will go look up the unit test class, read it, understand what the code needs to do and proceed to edit the necessary algorithms into the code. The tool is a mix of response parsing/execution and prompt building, so it can do builds/tests/fix cycles. At the end you get a git branch with the work it's done.<p>By this point I'm kinda developing an intuition for what GPT4 can and can't do, but whilst pushing against what it <i>can</i> do I still often find myself being impressed. Maybe not "mind blown" anymore because it's amazing how quickly you can get used to this stuff, but still. Not only does it do an excellent job of figuring out what to do when the instructions are clear enough, but it also has commands for adding library dependencies and will use relevant open source libs to make its job easier.<p>The hard part about using this sort of tool is that you can quickly become bottlenecked on figuring out what it is you actually want. It's easy to run ahead of yourself and end up with a mess. This sort of very rapid "painting with code" feel is a bit new; probably the slowness of typing allows your subconscious to think ahead a bit when doing programming normally. Also, I'm still teaching it how to explore the codebase efficiently so things that require a lot of context aren't quite there yet. I've got a whole plan for how to make that work better though.<p>I expect at some point soon some big company or some $100M "seed round" startup will come along and do this better, but I didn't want to wait.
I asked it to draft a legal document based on some bullet points and it created it, in a different language, in keeping with the legal norms of that country. I sent it to a local lawyer for review and they only came back with very minor things.<p>Did the reverse with some regulatory documents too, asking it to summarise in vernacular then asking specific questions to better wrap my head around the docs. I wouldn't trust it fully for this of course, more like a supplementary learning aid.<p>I keep hitting limits with context size though. Eg. When coding it becomes cumbersome once your project hits a certain complexity. You have to carefully construct prompts to get useful outputs. Being able to add the entire source code into a prompt to have more contextual responses would really increase utility.
That i'm finally breaking my "just google it" habits for technical stuff.<p>Had it write non blocking micro-controller code for a project (about 75% of the output was put into production) it helped making variable names and general structure. The first output was bad because it blocked the button read while "breathing" and led but then I told it: "non blocking code please".
Honestly, it hasn't. Last night I was trying to use it/Bing to write a simple API request based on a JSON schema, and it failed miserably. Nothing super complicated either, it was just a poor fit. Both GPT3 and GPT4 repeatedly generated nonworking code, which surprised (and frustrated) me.
I think people are most surprised when using it outside of their own domain.<p>As an example: I recently showed my mother GPT-4. I asked her what she wanted to see, and having little knowledge of the tool's capabilities, she asked me to surprise her. Apparently I had done this before (with GPT-3), as the poem I generated about Petoskey stones was met with little fanfair: "I've seen it generate poems... what else can it do?"<p>At that point I showed her a Python file and asked GPT to translate it to Typescript. Needless to say she's not a poet or a programmer, but the fact it could do something like that, entirely outside of her domain, was mind-blowing.<p>For me, I'm surprised how little I use Google...
There are a few types of RF receiver architecture and I was asking ChatGPT4 to provide the codes in Mermaid for direct conversion receiver architecture since it does not has support for image generation capability. The provided codes contained some errors when copied/pasted to Mermaid. Then I feedback the errors back to ChatGPT4 it then apologized and duely provided the correct codes that Mermaid can draw the receiver diagram. After that the diagram is fed into Krogi and draw.io for further processing resulting in the direct conversion diagram with editable boxes and texts that is ready to be used or modified with draw.io.
It blew my mind when I gave it a TypeScript function with multiple nested awaits, all tangled up into a chain of multiple `map` calls. A comment I wrote above it was “TODO refactor this monstrosity.” It was clever enough to split it into a main function and multiple utils. And it added JSDocs. The result passed my existing tests immediately.
It’s extremely under utilized.<p>Everyone could be producing way more than they currently are.<p>Everyone could be generating 1 app every 3 hours.<p>Everyone could be writing entire novels.<p>Everyone could be writing meaningful Wikipedia articles.<p>DIY guides should be 100x larger .
This week between ChatGPT and Phind I asked an AI four questions. Two were about SMT-LIB, one was about homomorphic encryption, and one was about software-defined networking.<p>Both ChatGPT and Phind insisted on giving me example SMT problems and then incorrect solutions to them. The HE answer was so vague as to be useless, and I don’t actually know enough about SDN to verify the last one but it <i>seemed</i> useful.<p>Overall it didn’t give me a lot of confidence in the two systems.
I was at the beach and used it to write a streaming C++ web server integrating whisper.cpp. I should opensource it eventually, but was like totally amazing since I used to write C++ all the time but it's been like 15 years and was like well i'll use c++20 and sure enough even with a few drinks and a lot of sun the code came together really nicely.
I was testing the pixie observability tool for an internal PoC. It has only Go and Python client libraries. I asked Bing AI, how to the call the API in Java. It made up an fake Java client library and gave instructions for adding maven dependency. When prompted further, it gave some sample code as well.
When 4 perfectly followed a complex prompt that I had spent hours trying (unsuccessfully) to get 3.5-turbo to follow even somewhat acceptably. Both models continue to impress me with their ability to tell me how to tune my prompts to get better results more consistently.
Idk about “blow my mind” but I was surprised that it helped me figure out “the song at sporting events that goes “boom boom, ch”.”<p>Maybe that’s an easy thing to know, but I expected to not get a good answer.