Sorry, but a new prompt for GPT-4 is not a paper

If you do enough measurements on that new prompt then I don&#x27;t see why this shouldn&#x27;t be a paper. People overestimate the value of &quot;grand developments&quot;, and underestimate the value of actually knowing - in this case actually knowing how well something works, even if it is as simple as a prompt.<p>Compare with drug trials: Adderall only differs from regular amphetamine in the relative concentration of enantiomers, and the entire value of the drug is in the measurements.

I feel this has nothing at all to do with LLMs and more to do with academic incentives in general. Focusing on quality over quantity won&#x27;t advance your career. Publishing lots of new papers will, as long as they meet the minimum threshold to be accepted into whatever journal or conference you are aiming for. Having one good paper won&#x27;t increase your h-score, three mediocre papers might.<p>Doubly so when there&#x27;s a new breakthrough, where one of your low-effort papers might end up being the first saying something obvious that ends up being really important. Because then everyone will end up quoting your paper in perpetuity.

Being dismissive about this tweet or agreeing with the author is one thing. Not realizing that the absolute minimum of a scientific paper can be much lower than a new prompt for GPT-4 is what everyone should be aware of.

It is a _paper_, but it&#x27;s not science, since GTP-4 is closed source and thus not reproducible in a lab.<p>If OpenAI disappears tomorrow, papers are GTP-4 will likely be of little to no value, which is another tell of a non-scientific exploration.<p>(note: not all explorations are scientific, and that is great! Science is just one of many tools for exploring lived reality.)

While I think the twitter post author is being a bit of an ass, they’re sort of right about the overvaluing we’ve put on simply better prompts. I wrote an opinionated GitHub gist about this exact issue:<p><a href="https:&#x2F;&#x2F;gist.github.com&#x2F;Hellisotherpeople&#x2F;45c619ee22aac6865ca4bb328eb58faf" rel="nofollow noreferrer">https:&#x2F;&#x2F;gist.github.com&#x2F;Hellisotherpeople&#x2F;45c619ee22aac6865c...</a><p>I do the whole NLP publishing thing and I’ve hesitated to “write a paper” about applying techniques already known and used everywhere in the stable diffusion community to NLP models. That said, the AI community loves to pretend like they discovered something, such as a recent paper purporting to be the first to do “concept slider” Lora’s, despite these existing for months before that work was published on Civit.ai. The authors of course didn’t cite those already existing models.<p>Everyone chasing citations and clout hard right now because these professors and researchers realize that they only have 5-10 years before AI eats their jobs and most other white collar jobs. I don’t blame them. I want my mortgage paid off before I’m automated away!

The current scientific research apparatus is more about being first than about being correct or thorough. A paper that gets out early means more citations, and many of the faculty sit on the editorial boards, and are able to suggest&#x2F;enforce specific citations during the review process. Academics aren&#x27;t fully to blame for this, it&#x27;s just how the incentives are set up in the system. Tenure and promotions are increasingly based on h-index; a measure of impact based largely on the number of citations.

To bring some data to a sour grapes fight: <a href="https:&#x2F;&#x2F;paperswithcode.com&#x2F;sota&#x2F;code-generation-on-humaneval" rel="nofollow noreferrer">https:&#x2F;&#x2F;paperswithcode.com&#x2F;sota&#x2F;code-generation-on-humaneval</a><p>For code generation, GPT4 is getting beat by the small prompt library LATS wrapped around GPT3.5. Given the recent release of MagicCoder &#x2F; Instruct-OSS, that means a small prompt library + a small 7B model you can self-host beats the much fancier GPT4.<p>Similar to when simple NNs destroyed a decade of Bayesian modeling theses &amp; research programs, it&#x27;s frustrating for folks going other paths. But it doesn&#x27;t make the work &#x27;wrong&#x27;.

Reminds me of what a real programmer is: <a href="https:&#x2F;&#x2F;sac.edu&#x2F;AcademicProgs&#x2F;Business&#x2F;ComputerScience&#x2F;Pages&#x2F;Hester_James&#x2F;Real%20Programmer.htm" rel="nofollow noreferrer">https:&#x2F;&#x2F;sac.edu&#x2F;AcademicProgs&#x2F;Business&#x2F;ComputerScience&#x2F;Pages...</a>

Can a person just go upload anything on arxiv or is there a review process around these things?<p>What I am really asking is &quot;what makes something a paper and not a blogpost&quot;?

How to add prompt knowledge into research? By having papers about it.<p>Shouldn&#x27;t be the tooling around it good enough that a few prompt papers don&#x27;t overload the system?

Nah, this is just an early example of many &quot;this is too easy it doesn&#x27;t count&quot; defensive human arguments against AI.<p>Parallel to the &quot;you use copilot so your code quality is terrible and you don&#x27;t really even understand it so it&#x27;s not maintainable&quot; human coping we are familiar with.<p>If there is any shred of truth to these defenses, it is temporary and will be shown false by future, more powerful AI models.<p>Consider the theoretical prompt that allows one of these models to rapidly improve itself into an AGI. Surely you&#x27;d want to read that paper right?

Developing prompts for these models isn&#x27;t a science yet. It does seem to meet most of the criteria for an art though.<p>We recognize some outputs as high quality, and others as low quality, but often can&#x27;t articulate the exact reason why. It seems that some people are able to reliably produce high quality results, indicating there is some kind of skill involved. More precisely, the quality of an individual artist&#x27;s last output is positively correlated with the quality of their next output. A kind of imprecise &quot;shop talk&quot; has emerged, self describing as &quot;prompt engineering&quot;, which resembles the conversations artists in other mediums have.<p>For people in tech this will seem most similar to graphic designers. They produce much nicer looking interfaces than lay people can. We often can&#x27;t explain why, but recognize it to be the case. And graphic designers have their own set of jargon, which is useful to them, but is not scientific.<p>&quot;Prompt artist&quot; is a better term than &quot;prompt engineer&quot;.

Should this be a paper?<p><a href="https:&#x2F;&#x2F;not-just-memorization.github.io&#x2F;extracting-training-data-from-chatgpt.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;not-just-memorization.github.io&#x2F;extracting-training-...</a><p>There is supporting analysis and measurement, but the essence is a single type of prompt, and DeepMind is a heavyweight lab I think it’s fair to say.<p>Moreover there’s evidence people independently reported this result months beforehand on Reddit based on casual observation.

Studying the way LLMs behave to different prompts (or different ways of fine-tuning for a set of prompts) is valuable science.<p>Some of the most interesting papers published this year (&quot;Automatic Multi-Step Reasoning and Tool-Use&quot;) compare prompt strategies across a variety of tasks. The results are fascinating, findings are applicable and invite further research in the area of &quot;prompt selection&quot; or &quot;tool selection.&quot;

Attacking the participants in a systemic shift is 100% useless as it doesn&#x27;t target the culprit.<p>In programming we have a similar phenomenon, that StackOverflow-driven (and I guess now GPT-driven) juniors have overtaken the industry and displaced serious talent. Because sufficient amounts of quantity always beats quality, even if the end result is inferior, this is caused by market dynamics, which operate on much cruder parameters than the sophisticated analysis of an individual noticing everything around them becoming &quot;enshittified&quot;.<p>SO-driven juniors are cheap, plentiful, and easily replaceable. And a business that values less expense and less risk therefore prefers them, because it has no way to measure the quality of the final product with simple metrics.<p>The same mechanism is driving AI replacing our jobs currently, the avalanche of garbage papers by academics. This is entropy for you. We see it everywhere in modern society down to the food we eat. Quality goes away, replaced by cheap to produce and long shelf life.<p>If we don&#x27;t fundamentally alter what the system sees as ACCEPTABLE, and VALUABLE, this process will inevitably continue until our world is completely unrecognizable. And to fundamentally alter the system, we need an impulse that aligns us as a society, startles us into action, all together (or at least significant majority of us). But it seems we&#x27;re currently in &quot;slowly boiled frog mode&quot;.

Why not? A paper is not necessarily scientific nor a breakthrough. In my view, a paper is written and documented communication that&#x27;s usually approved by peers in the field. Also a blunt observation in nature can be noteworthy. However, we don&#x27;t see such papers anymore as these fields have matured. Just go back in the history of your field and you will find trivial papers.

Real science is reserved for those with real expertise! As the self-anointed gatekeeper of real science I decree that other peoples’ work fails to meet the minimum standard I have set for real science! Mind you not the work other actors in the scientific community publish and accept among their peers - they are not real scientists and their work is trivial. For shame!

The art and science of building these models is not disputed, but I think that the scientific value of prompts is tightly linked to reproducibility.<p>If you’ve developed a new prompt for a model whose weights you can directly access, then this prompt could have scientific value because its utility will not diminish over time or be erased with a new model update. I’m even generally of the view that a closed API endpoint whose expiration date is years into the future could have some value (but much less so). But simply finding a prompt for something like ChatGPT is not useful for science because we don’t even have certainty about which model it’s executing against.<p>Note that some of the best uses of these models and prompting have nothing to do with academics; this is a comment focused on the idea about writing academic papers about prompts.

I can maybe understand the frustration from a “scientific” perspective, but for a lot of these “one prompt papers” - you still need someone to sit down and do the analysis and comparisons. Very few papers focus only on GPT&#x2F;ChatGPT.<p>Additionally, it gives people other ideas to try for themselves. And some of this stuff might be useful to someone in a specific scenario.<p>It’s not glamorous research or even future-proof seeing as how certain prompts can be surgically removed or blocked by the owner of the model, but I don’t think it warrants telling people not to do it.

If a new prompt enables a new task or enhances performance on a task then it absolutely should be published.<p>Back in the day would compiler optimizations be not worthy of publishing?

It&#x27;s hard to draw these lines, because you will certainly filter out a lot of bad (i.e. useless, low contribution to any field) papers, but you might <i>also</i> filter out some really important papers. Research being basic or something anyone could&#x27;ve done doesn&#x27;t count againt its potential importance, just the expected value of importance I guess.<p>I&#x27;d rather we had a few too many bad papers than a few too few great papers.

Arxiv is like the MENSA of the tech world<p>The similarity being that it’s ego masquerading as academic.<p>Most things shared from there should have just been a blog post.<p>The last year has showed that AI&#x2F;ML research and use did not need academic gatekeeping by PhDs and yet many in that scene keep trying self infatuating things with the lowest utility.

Behind all this is a valid question. How does one evaluate prompts and LLMs? As gipeties (custom gpts) become more popular millions of hours will be wasted by ones that have been built badly. Without some sort of automated quality control, gipeties will become a victim of their own success.

What&#x27;s the difference between a paper on a new prompt and a paper discussing a new domain-specific model, e.g. heart failure risk? If they analyze the problem and solution equally, they both seem useful. It&#x27;s not like most other ML papers share their weights or datasets.

This reminds me of how there was a boom in half-baked studies around COVID, e.g. modelling this or that aspect of the pandemic, or around mask wearing.<p>I imagine that most of these will simply have had little to no impact, and will only serve to bolster the publication list of those who wrote them.

ChatGPT droppings have to be at least as relevant and newsworthy as these findings [1]<p><pre><code> [1] https:&#x2F;&#x2F;www.wbur.org&#x2F;news&#x2F;2022&#x2F;07&#x2F;27&#x2F;harvard-shorenstein-research-january-6-insurrection-president</code></pre>

Tired: Asking participants to sign an ethics pledge at the top of a tax return makes them more honest.<p>Wired: Asking an LLM to write out their steps first makes them more accurate.<p>They seem equally interesting to me, but one is a lot easier to replicate, and the other is easier to lie about.

It depends I guess.<p>If you solve a problem that had been around for a while and LLMs offer a new way of approaching it, then it can definitely become a paper.<p>Of cause one has to verify in sophisticated experiments, that this approach is stable.

I am sorry but what can ChatGPT do that a couple of minutes of googling couldn’t solved? Write half hearted essays that all contain the same phrase?

Yes these papers are optimizing for social media hype, I.e what is the quickest and easiest path to make noise on social media ?

Sorry, but if they can get away with it, they&#x27;ll release it as a paper.<p>It&#x27;s not like most papers are much above that anyway...

Times are changing. Human researchers will dedicate more and more time towards getting language models to work in desired ways rather than doing the research themselves. Language models will largely be the ones making &quot;research&quot; discoveries. Both should be considered valid research IMO.

Doesn&#x27;t academia incentivise quantity over quality anyways?

A new prompt is not a paper, but you can prompt it for a paper.

Anyone caught doing this should be kicked out of the industry. Period. You&#x27;re scaming those funding your &quot;research&quot;, you are misleading readers, and are producing low quality content wasting everyone&#x27;s time.

Excuse me? Step by step wasn&#x27;t paper-worthy? Hard disagree.<p>LLM research is currently in its infancy, because they are no older than a few years old. And a research field in its infancy is bound to have a few noteworthy &quot;no sh*t, Sherlock&quot; papers that would be obvious from hindsight.<p>The fact is, LLMs are a higher-order construct in machine learning, much like a fish is higher-order than a simple cellular colony. Lower-order ML constructs do not demonstrate emergent capabilities like step by step, stream of consciousness thinking, and so on.<p>Academics should be less jaded and approach the field with beginner&#x27;s eyes. Because we are all beginners here.

Still beats most psychology papers

Academia needs to get over itself, can&#x27;t wait to see how amazing this tech is going to get when the next generation who decide never to bother with those stuffy and navel gazing institutions becomes the driving force behind it.<p>Looking forward to &quot;I made this cool thing, here&#x27;s the code&#x2F;library you can use&quot; rather than the papers&#x2F;gatekeeping&#x2F;ego stroking&#x2F;&quot;muh PhD&quot;.<p>Think if Google had built an AI team around the former rather than the latter, they wouldn&#x27;t have risked the future of their entire company and squandered their decade head start.