In my opinion it's a weak take that only got so upvoted and commented on hacker news because it has the existential and universal logical quantifier symbols in the title and also because it uses the chiasmus rhetoric device both in the title and at the end of the article.<p>The argument is that several technologies don't work 'out of the box' and you have to tweak their settings for each problem that you face, and that this means it's a scam. For example you have to change prompts in LLMs or change hyperparameters in other machine learning solutions. This argument is some combination of not insightful and not true.
>>> Build a system that solves problems, but with an important user-facing control. ...<p>>>> Convince the user that it is their job to find a instantiation or setting of this control to make the system work for their tasks.<p>By golly, you just described playing the cello.
If I understand correctly, the meat of the argument is "that is a system for every (∀) task, there exists (∃) a setting that gives the correct answer for that one task."<p>My understanding of this (correct me if I'm wrong) is that the scam is convincing users that GPT-X can do anything with say, the correct prompts.<p>This argument misses the mark for me. It's not that it solves all the problems, it's that the problems it does solve is economically impactful. Significantly economically impactful in some cases- obvious examples of call centers and first-line customer support.
I'll try to explain this better.<p>Sometimes you hope to get a system that works without constant tweaking. Once you've adjusted it properly, it should work without adjusting the settings. So there's <i>one</i> setting that works <i>for all</i> the inputs you care about.<p>But instead you have a system that you have to adjust all the time. For every input, you can get it to work. But you're never done tweaking.<p>This is actually fine if you're expecting the user to do some work. Consider doing a Google search, seeing what you get, and then modifying your query when you didn't get what you want.<p>It sucks if you're hoping to go into fully-automatic mode. You didn't get that, you got an interactive application. (Or maybe the tweaks aren't reasonable for the user to make themselves, so you got nothing?)<p>So the scam, if there is one, is selling something as fully-automated when it's not. It's a tool that requires a person to work with a machine, and you're never going to be able to take the human out of the loop. (Let's say you're hoping for a driverless car but you never quite get there.)<p>Often, it's not the end of the world. An interactive tool can be very valuable! But you need to set your expectations appropriately.
> Also, as they the hyper-parameter selection fully to the users, they become not falsifiable. If they didn’t work, it is because you didn’t pick the right hyper-parameters or training procedures.<p>See Also: Scrum certification.
The author says that the builders of these systems are also the ones that run the scam. And they deliberately build it in a fashion that enables the scam. Seriously?<p>This is a gripe with sales and marketing. A tale as old as time.
Since there are certainly people unfamiliar with the symbols, here is the Wikipedia article on them:<p><a href="https://en.wikipedia.org/wiki/Quantifier_(logic)" rel="nofollow">https://en.wikipedia.org/wiki/Quantifier_(logic)</a>
This argument is disingenuous. Hyperparameter optimization is not in the same category as prompt engineering, like at all.<p>Also no one claims like half of the things the article claims people clain.
ChatGPT itself is <i>in on the scam</i>. The way i think about it is that ChatGPT is already superhuman at bullshitting, many people want to give it credit for being more capable than it really is.<p>it is interesting to postulate if it is the “most likely word” heuristic that leads to this behavior (e.g. never says anything that startles people) or HFRL training systematically teaching it to say what people want to hear.
I can walk and chew bubble gum at the same time: on one hand, yes, there's certainly a lot of Kool-Aid being drank by the AI folks. Even on HN, I constantly argue with people that genuinely think LLMs are some kind of magical black box that contain "knowledge" or "intelligence" or "meaning" when in reality, it's just a very fancy Markov chain. And on the other hand, I think that language interfaces are probably the next big leap in how we interact with our computers, but more to the point of the article:<p>> To conclude: one must have different standards for developing systems than for testing, deploying, or using systems.<p>In my opinion, you unfortunately will never (and, in fact <i>could</i> never) have reliable development and testing standards when designing purely stochastic systems like, e.g., large language models. Intuitively, the fact that these are stochastic systems is <i>why</i> we need things like hyper-parameters, fiddling with seeds, and prompt engineering.
I think there has always been a great deal of Stone Soup in the software world. We are often promising solutions that will work as long as all the ingredients are supplied in the form of processes delivered by the customer.<p>Strangely though the very promise of the magic stone does allow solutions (soup) to emerge.
The author is saying ! ∃ ∀ and selling it as ! ∀ ∃. He's the one scamming.<p>AI researchers are rather upfront about the need to use fine-tuning and prompt engineering for current AI models.<p>As to random forest models, wasn't the whole point of hyperparameter sweeps to remove the need for manual hyperparameter selection?
> <i>Convince the user that it is their job to find a instantiation or setting of this control to make the system work for their tasks.</i><p>As opposed to convincing the user that it is their job to brief a suitably qualified contractor or employee to make a company perform the required work?<p>∴ humans are a scam
Are general purpose tools scams?<p>An operating system does many things.
A general purpose language when contrasted with a domain specific language.
A human programmer thinking that they're a wizard that can solve all problems.<p>However I get the sentiment I think, it's a little bit like snake oil and alternative medicine. They promise to solve all your problems.<p>What's interesting to think about is who it is to "blame". The developers of the product, the sales people of the product, the wild imagination of the buyer thinking the product can solve everything, or maybe the wild imagination of the developers behind the buyer. However if the seller can set off a wild imagination in the buyer, maybe you've successfully scammed them.
Wait, how does this scam work for OpenAI? The product is free to use.<p>Also they haven't claimed GPT is an AGI or that it can solve all your problems.
It is difficult to conclude this is true, and it will be even more difficult in the future because tech like "AI" has the ability to almost completely saturate the amount of data any human can ingest. The relationship will soon be symbiotic, with everything showing the extent of our progress... in the same manner we can date movies by the kinds of phones they use. Many plots will become stale, many worldviews will be condensed to something that supports "AI" and the offending branches will be snipped. The only way to really forget this limitation is to be myopic enough to disregard everything else. With the way the internet is going, I'm sure one day these LLMs will be heralded as "free" and "open" media, their fuzzy recollections the only records we will have of the past, and extensive use will essentially morph civilization in their own image.
This has always been present in subfields of AI. For example, in classical computer vision, one had to figure out specific parameters working just for a single image or video scene by hand. Machine learning can in theory at least make these parameters learnable at the cost of complexity.
Ultimately, this is a tool like Google. If using a search engine doesn't work for you for some task, then it's not the tool for the job.<p>You even get to trial it for free.<p>You shouldn't use LLMs if you can't make them work. I can make them work for me for many things. That makes them useful for me.
“Or: testing on your training data is a common way to cheat, but so is training on your test data”<p>The need to hold back training data for testing, and issues around testing using variants of training data versus real world data are well known.
>To conclude: one must have different standards for developing systems than for testing, deploying, or using systems. Or: testing on your training data is a common way to cheat, but so is training on your test data.<p>Isn't this already a solved problem? Every reasonable paper on ML separates their test data from their validation data already.
In the second part of the essay the writer mentions the statisticians fallacy.
"Adding more hyperparameters leads to overfitting."
This is not true with large neural networks anymore and is part of the magic.
To be fair, on the pure logic side (forgetting about what the link is really about, i.e. selling scams), it is true that any ∀ ∃ can be turned into a ∃ ∀. It's called skolemization.
It's a good point, I'd consider it (overfitting) a pitfall or common mistake in ML rather than the only mode. I'd agree that most ML models and almost all state-of-the-art are over-fit to the point of being useless, but that's not an inevitability.