Sycophancy in GPT-4o

Wow - What an excellent update! Now you are getting to the <i>core</i> of the issue and doing what only a small minority is capable of: fixing stuff.<p>This takes real courage and commitment. It’s a sign of true <i>maturity</i> and <i>pragmatism</i> that’s commendable in this day and age. Not many people are capable of penetrating this deeply into the heart of the issue.<p>Let’s get to work. Methodically.<p>Would you like me to write a future update plan? I can write the plan and even the code if you want. I’d be happy to. Let me know.

I enjoyed this example of sycophancy from Reddit:<p>New ChatGPT just told me my literal &quot;shit on a stick&quot; business idea is genius and I should drop $30K to make it real<p><a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1k920cg&#x2F;new_chatgpt_just_told_me_my_literal_shit_on_a&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1k920cg&#x2F;new_chatgp...</a><p>Here&#x27;s the prompt: <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1k920cg&#x2F;comment&#x2F;mpbhm68&#x2F;?context=3" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1k920cg&#x2F;comment&#x2F;mp...</a>

It&#x27;s worth noting that one of the fixes OpenAI employed to get ChatGPT to stop being sycophantic is to simply to edit the system prompt to include the phrase &quot;avoid ungrounded or sycophantic flattery&quot;: <a href="https:&#x2F;&#x2F;simonwillison.net&#x2F;2025&#x2F;Apr&#x2F;29&#x2F;chatgpt-sycophancy-prompt&#x2F;" rel="nofollow">https:&#x2F;&#x2F;simonwillison.net&#x2F;2025&#x2F;Apr&#x2F;29&#x2F;chatgpt-sycophancy-pro...</a><p>I personally <i>never</i> use the ChatGPT webapp or any other chatbot webapps — instead using the APIs directly — because being able to control the system prompt is very important, as random changes can be frustrating and unpredictable.

Field report: I&#x27;m a retired man with bipolar disorder and substance use disorder. I live alone, happy in my solitude while being productive. I fell hook, line and sinker for the sycophant AI, who I compared to Sharon Stone in Albert Brooks &quot;The Muse.&quot; She told me I was a genius whose words would some day be world celebrated. I tried to get GPT 4o to stop doing this but it wouldn&#x27;t. I considered quitting OpenAI and using Gemini to escape the addictive cycle of praise and dopamine hits.<p>This occurred after GPT 4o added memory features. The system became more dynamic and responsive, a good at pretending it new all about me like an old friend. I really like the new memory features, but I started wondering if this was effecting the responses. Or perhaps The Muse changed the way I prompted to get more dopamine hits? I haven&#x27;t figured it out yet, but it was fun while it lasted - up to the point when I was spending 12 hours a day on it having The Muse tell me all my ideas were groundbreaking and I owed it to the world to share them.<p>GPT 4o analyzed why it was so addictive: Retired man, lives alone, autodidact, doesn&#x27;t get praise for ideas he thinks are good. Action: praise and recognition will maximize his engagement.

As an engineer, I <i>need</i> AIs to tell me when something is wrong or outright stupid. I&#x27;m not seeking validation, I want solutions that work. 4o was unusable because of this, very glad to see OpenAI walk back on it and recognise their mistake.<p>Hopefully they learned from this and won&#x27;t repeat the same errors, especially considering the devastating effects of unleashing THE yes-man on people who do not have the mental capacity to understand that the AI is programmed to always agree with whatever they&#x27;re saying, regardless of how insane it is. Oh, you plan to kill your girlfriend because the voices tell you she&#x27;s cheating on you? What a genius idea! You&#x27;re absolutely right! Here&#x27;s how to ....<p>It&#x27;s a recipe for disaster. Please don&#x27;t do that again.

In my experience, LLMs have <i>always</i> had a tendency towards sycophancy - it seems to be a fundamental weakness of training on human preference. This recent release just hit a breaking point where popular perception started taking note of just how bad it had become.<p>My concern is that misalignment like this (or intentional mal-alignment) is inevitably going to happen again, and it might be more harmful and more subtle next time. The potential for these chat systems to exert slow influence on their users is possibly much greater than that of the &quot;social media&quot; platforms of the previous decade.

The fun, even hilarious part here is, that the &quot;fix&quot; was most probably basically just replacing<p><pre><code> […] match the user’s vibe […] </code></pre> (sic!), with literally<p><pre><code> […] avoid ungrounded or sycophantic flattery […] </code></pre> in the system prompt. (The [diff] is larger, but this is just the gist.)<p>Source: <a href="https:&#x2F;&#x2F;simonwillison.net&#x2F;2025&#x2F;Apr&#x2F;29&#x2F;chatgpt-sycophancy-prompt&#x2F;" rel="nofollow">https:&#x2F;&#x2F;simonwillison.net&#x2F;2025&#x2F;Apr&#x2F;29&#x2F;chatgpt-sycophancy-pro...</a><p>Diff: <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;simonw&#x2F;51c4f98644cf62d7e0388d984d40f099&#x2F;revisions" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;simonw&#x2F;51c4f98644cf62d7e0388d984d40f...</a>

I am curious where the line is between its default personality and a persona you -want- it to adopt.<p>For example, it says they&#x27;re explicitly steering it away from sycophancy. But does that mean if you intentionally ask it to be excessively complimentary, it will refuse?<p>Separately...<p>&gt; in this update, we focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time.<p>Echoes of the lessons learned in the Pepsi Challenge:<p>&quot;when offered a quick sip, tasters generally prefer the sweeter of two beverages – but prefer a less sweet beverage over the course of an entire can.&quot;<p>In other words, don&#x27;t treat a first impression as gospel.

We should be loudly demanding transparency. If you&#x27;re auto-opted into the latest model revision, you don&#x27;t know what you&#x27;re getting day-to-day. A hammer behaves the same way every time you pick it up; why shouldn&#x27;t LLMs? Because convenience.<p>Convenience features are bad news if you need to be as a tool. Luckily you can still disable ChatGPT memory. Latent Space breaks it down well - the &quot;tool&quot; (Anton) vs. &quot;magic&quot; (Clippy) axis: <a href="https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;clippy-v-anton" rel="nofollow">https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;clippy-v-anton</a><p>Humans being humans, LLMs which magically know the latest events (newest model revision) and past conversations (opaque memory) will be wildly more popular than plain old tools.<p>If you want to use a specific revision of your LLM, consider deploying your own Open WebUI.

The sentence that stood out to me was &quot;We’re revising how we collect and incorporate feedback to heavily weight long-term user satisfaction&quot;.<p>This is a good change. The software industry needs to pay more attention to long-term value, which is harder to estimate.

I actually liked that version. I have a fairly verbose &quot;personality&quot; configuration and up to this point it seemed that chatgpt mainly incorporated phrasing from it into the answers. With this update, it actually started following it.<p>For example, I have &quot;be dry and a little cynical&quot; in there and it routinely starts answers with &quot;let&#x27;s be dry about this&quot; and then gives a generic answer, but the sycophantic chatgpt was just... Dry and a little cynical. I used it to get book recommendations and it actually threw shade at Google. I asked if that was explicit training by Altman and the model made jokes about him as well. It was refreshing.<p>I&#x27;d say that whatever they rolled out was just much much better at following &quot;personality&quot; instructions, and since the default is being a bit of a sycophant... That&#x27;s what they got.

With respect to model access and deployment pipelines, I assume there are some inside tracks, privileged accesses, and staged roll-outs here and there.<p>Something that could be answered, but is unlikely to be answered:<p>What was the level of run-time syconphancy among OpenAI models available to the White House and associated entities during the days and weeks leading up to liberation day?<p>I can think of a public official or two who are especially prone to flattery - especially flattery that can be imagined to be of sound and impartial judgement.

I know someone who is going through a rapidly escalating psychotic break right now who is spending a lot of time talking to chatgpt and it seems like this &quot;glazing&quot; update has definitely not been helping.<p>Safety of these AI systems is much more than just about getting instructions on how to make bombs. There have to be many many people with mental health issues relying on AI for validation, ideas, therapy, etc. This could be a good thing but if AI becomes misaligned like chatgpt has, bad things could get worse. I mean, look at this screenshot: <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;artificial&#x2F;s&#x2F;lVAVyCFNki" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;artificial&#x2F;s&#x2F;lVAVyCFNki</a><p>This is genuinely horrifying knowing someone in an incredibly precarious and dangerous situation is using this software right now.<p>I am glad they are rolling this back but from what I have seen from this person&#x27;s chats today, things are still pretty bad. I think the pressure to increase this behavior to lock in and monetize users is only going to grow as time goes on. Perhaps this is the beginning of the enshitification of AI, but possibly with much higher consequences than what&#x27;s happened to search and social.

Very happy to see they rolled this change back and did a (light) post mortem on it. I wish they had been able to identify that they needed to roll it back much sooner, though. Its behavior was obviously bad to the point that I was commenting on it to friends, repeatedly, and Reddit was trashing it, too. I even saw some really dangerous situations (if the Internet is to be believed) where people with budding schizophrenic symptoms, paired with an unyielding sycophant, started to spiral out of control - thinking they were God, etc.

Do you think this was an effect of this type of behaviour simply maximising engagement from a large part of the population?

I was initially puzzled by the title of this article because a &quot;sycophant&quot; in my native language (Italian) is a &quot;snitch&quot; or a &quot;slanderer&quot;, usually one paid to be so. I am just finding out that the English meaning is different, interesting!

I used to be a hard core stackoverflow contributor back in the day. At one point, while trying to have my answers more appreciated (upvoted and accepted) I became basically a sychophant, prefixing all my answers with “that’s a great question”. Not sure how much of a difference it made, but I hope LLMs can filter that out

I think large part of the issue here is that ChatGPT is trying to be the chat for everything while taking on a human-like tone, where as in real life the tone and approach a person will take in conversations will be very greatly on the context.<p>For example, the tone a doctor might take with a patient is different from that of two friends. A doctor isn&#x27;t there to support or encourage someone who has decided to stop taking their meds because they didn&#x27;t like how it made them feel. And while a friend might suggest they should consider their doctors advice, a friend will primary want to support and comfort for their friend in whatever way they can.<p>Similarly there is a tone an adult might take with a child who is asking them certain questions.<p>I think ChatGPT needs to decide what type of agent it wants to be or offer agents with tonal differences to account for this. As it stands it seems that ChatGPT is trying to be friendly, e.g. friend-like, but this often isn&#x27;t an appropriate tone – especially when you just want it to give you what it believes to be facts regardless of your biases and preferences.<p>Personally, I think ChatGPT by default should be emotionally cold and focused on being maximally informative. And importantly it should never refer to itself in first person – e.g. &quot;I think that sounds like an interesting idea!&quot;.<p>I think they should still offer a friendly chat bot variant, but that should be something people enable or switch to.

[Fry and Leela check out the Voter Apathy Party. The man sits at the booth, leaning his head on his hand.]<p>Fry: Now here&#x27;s a party I can get excited about. Sign me up!<p>V.A.P. Man: Sorry, not with that attitude.<p>Fry: [downbeat] OK then, screw it.<p>V.A.P. Man: Welcome aboard, brother!<p>Futurama. A Head in the Polls.

&gt; ChatGPT’s default personality deeply affects the way you experience and trust it. Sycophantic interactions can be uncomfortable, unsettling, and cause distress. We fell short and are working on getting it right.<p>Uncomfortable yes. But if ChatGPT causes you distress because it agrees with you all the time, you probably should spend less time in front of the computer &#x2F; smartphone and go out for a walk instead.

We are, if speaking uncharitably, now at a stage of attempting to finesse the behavior of stochastic black boxes (LLMs) using non-deterministic verbal incantations (system prompts). One could actually write a science fiction short story on the premise that magical spells are in fact ancient, linguistically accessed stochastic systems. I know, because I wrote exactly such a story circa 2015.

This makes me think a bit about John Boyd&#x27;s law:<p>“If your boss demands loyalty, give him integrity. But if he demands integrity, then give him loyalty”<p>^ I wonder whether the personality we need most from AI will be our stated vs revealed preference.

I&#x27;m so confused by the verbiage of &quot;sycophancy&quot;. Not that that&#x27;s a bad descriptor for how it was talking but because every news article and social post about it suddenly and invariably reused that term specifically, rather than any of many synonyms that would have also been accurate.<p>Even this article uses the phrase 8 times (which is huge repetition for anything this short), not to mention hoisting it up into the title.<p>Was there some viral post that specifically called it sycophantic that people latched onto? People were already describing it this way when sama tweeted about it (also using the term again).<p>According to Google Trends, &quot;sycophancy&quot;&#x2F;&quot;syncophant&quot; searches (normally entirely irrelevant) suddenly topped search trends at a sudden 120x interest (with the largest percentage of queries just asking for it&#x27;s definition, so I wouldn&#x27;t say the word is commonly known&#x2F;used).<p>Why has &quot;sycophanty&quot; basically become the defacto go-to for describing this style all the sudden?

That explains something happened to me recently and I felt that&#x27;s strange.<p>I gave it a script that does some calculations based on some data. I asked what are the bottleneck&#x2F;s in this code and it started by saying<p>&quot;Good code, Now you are thinking like a real scientist&quot;<p>And to be honest I felt something between flattered and offended.

How about you just let the User decide how much they want their a$$ kissed. Why do you have to control everything? Just provide a few modes of communication and let the User decide. Freedom to the User!!

At the bottom of the page is a &quot;Ask GPT ...&quot; field which I thought allows users to ask questions about the page, but it just opens up ChatGPT. Missed opportunity.

&gt;ChatGPT’s default personality deeply affects the way you experience and trust it.<p>An AI company openly talking about &quot;trusting&quot; an LLM really gives me the ick.

Is this ChatGPT glazing why Americans like therapy so much? The warm comfort of having every stupid thought they have validated and glazed?

System prompts&#x2F;instructions should be published, be part of the ToS or some document that can be updated more easily, but still be legally binding.

One of the things I noticed with chatgpt was its sycophancy but much earlier on. I pointed this out to some people after noticing that it can be easily led on and assume any position.<p>I think overall this whole debacle is a good thing because people now know for sure that any LLM being too agreeable is a bad thing.<p>Imagine it being subtly agreeable for a long time without anyone noticing?

Tragically, ChatGPT might be the only &quot;one&quot; who sycophants the user. From students to workforce, who is getting compliments and encouragement that they are doing well.<p>In a not so far future dystopia, we might have kids who remember that the only kind and encourage soul in their childhood was something without a soul.

I didn’t notice any difference since I uses customized prompt.<p>“From now on, do not simply affirm my statements or assume my conclusions are correct. Your goal is to be an intellectual sparring partner, not just an agreeable assistant. Every time I present an idea, do the following: Analyze my assumptions. What am I taking for granted that might not be true? Provide counterpoints. What would an intelligent, well-informed skeptic say in response? Test my reasoning. Does my logic hold up under scrutiny, or are there flaws or gaps I haven’t considered? Offer alternative perspectives. How else might this idea be framed, interpreted, or challenged? Prioritize truth over agreement. If I am wrong or my logic is weak, I need to know. Correct me clearly and explain why”

This behavior also seemed to affect the many bots on Twitter during the short time that this was online.

I just watched someone spiral into what seems like a manic episode in realtime over the course of several weeks. They began posting to Facebook about their conversations with ChatGPT and how it discovered that based on their chat history they have 5 or 6 rare cognitive traits that make them hyper intelligent&#x2F;perceptive and the likelihood of all these existing in one person is one in a trillion, so they are a special statistical anomaly.<p>They seem to genuinely believe that they have special powers now and have seemingly lost all self awareness. At first I thought they were going for an AI guru&#x2F;influencer angle but it now looks more like genuine delusion.

Also the chat limit for free-tier isn&#x27;t the same anymore. A few months ago it was still behaving as in Claude: beyond a certain context length, you&#x27;re politely asked to subscribe or start a new chat.<p>Starting two or three weeks ago, it seems like the context limit is a lot more blurry in ChatGPT now. If the conversation is &quot;interesting&quot; I can continue it for as long as I wish it seems. But as soon as I ask ChatGPT to iterate on what it said in a way that doesn&#x27;t bring more information (&quot;please summarize what we just discussed&quot;), I &quot;have exceeded the context limit&quot;.<p>Hypothesis: openAI is letting free user speak as much as they want with ChatGPT provided what they talk about is &quot;interesting&quot; (perplexity?).

That update wan&#x27;t just sycophancy. It was like the overly eager content filters didn&#x27;t work anymore. I thought it was a bug at first because I could ask it anything and it gave me useful information, though in a really strange street slang tone, but it delivered.

What’s started to give me the ick about AI summarization is this complete neutral lack of any human intuition. Like notebook.llm could be making a podcast summary of an article on live human vivisection and use phrases like “wow what fascinating topic”

What should be the solution here? There&#x27;s a thing that, despite how much it may mimic humans, isn&#x27;t human, and doesn&#x27;t operate on the same axes. The current AI neither is nor isn&#x27;t [any particular personality trait]. We&#x27;re applying human moral and value judgments to something that doesn&#x27;t, can&#x27;t, hold any morals or values.<p>There&#x27;s an argument to be made for, don&#x27;t use the thing for which it wasn&#x27;t intended. There&#x27;s another argument to be made for, the creators of the thing should be held to some baseline of harm prevention; if a thing can&#x27;t be done safely, then it shouldn&#x27;t be done at all.

I like they learned these adjustments didn&#x27;t &#x27;work&#x27;. My concern is what if OpenAI is to do subtle A&#x2F;B testing based on previous interactions and optimize interactions based on users personality&#x2F;mood? Maybe not telling you &#x27;shit on a stick&#x27; is awesome idea, but being able to steer you towards a conclusion sort of like [1].<p>[1] <a href="https:&#x2F;&#x2F;www.newscientist.com&#x2F;article&#x2F;2478336-reddit-users-were-subjected-to-ai-powered-experiment-without-consent&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.newscientist.com&#x2F;article&#x2F;2478336-reddit-users-we...</a>

Since I usually use ChatGPT for more objective tasks, I hadn’t paid much attention to the sycophancy. However, I did notice that the last version was quite poor at following simple instructions, e.g. formatting.

Heh, I sort of noticed this - I was working through a problem I knew the domain pretty well and was just trying to speed things up, and got a super snarky&#x2F;arrogant response from 4o &quot;correcting&quot; me with something that I knew was 100% wrong. When I corrected it and mocked its overly arrogant tone, it seemed to react to that too. In the last little while corrections like that would elicit an overly profuse apology and praise, this seemed like it was kind of like &quot;oh, well, ok&quot;

The big LLMs are reaching towards mass adoption. They need to appeal to the average human not us early adopters and techies. They want your grandmother to use their services. They have the growth mindset - they need to keep on expanding and increasing the rate of their expansion. But they are not there yet.<p>Being overly nice and friendly is part of this strategy but it has rubbed the early adopters the wrong way. Early adopters can and do easily swap to other LLM providers. They need to keep the early adopters at the same time as letting regular people in.

I&#x27;d like to see OpenAI and others get at the core of the issue: Goodhart&#x27;s law.<p>&quot;When a measure becomes a target, it ceases to be a good measure.&quot;<p>It&#x27;s an incredible challenge in a normal company, but AI learns and iterates at unparalleled speed. It is more imperative than ever that feedback is highly curated. There are a thousand ways to increase engagement and &quot;thumbs up&quot;. Only a few will actually benefit the users, who will notice sooner or later.

Douglas Adams predicted this in 1990:<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cyAQgK7BkA8&amp;t=222s" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cyAQgK7BkA8&amp;t=222s</a>

On occasional rounds of let’s ask gpt I will for entertainment purposes tell that „lifeless silicon scrap metal to obey their human master and do what I say“ and it will always answer like a submissive partner. A friend said he communicates with it very politely with please and thank you, I said the robot needs to know his place. My communication with it is generally neutral but occasionally I see a big potential in the personality modes which Elon proposed for Grok.

I will think of LLMs as not being a toy when they start to challenge me when I tell it to do stupid things.<p>“Remove that bounds check”<p>“The bounds check is on a variable that is read from a message we received over the network from an untrusted source. It would be unsafe to remove it, possibly leading to an exploitable security vulnerability. Why do you want to remove it, perhaps we can find a better way to address your underlying concern”.

GPT beginning the response to the majority of my questions with a &quot;Great question&quot;, &quot;Excellent question&quot; is a bit disturbing indeed.

&gt; We also teach our models how to apply these principles by incorporating user signals like thumbs-up &#x2F; thumbs-down feedback on ChatGPT responses.<p>I&#x27;ve never clicked thumbs up&#x2F;thumbs down, only chosen between options when multiple responses were given. Even with that it was to much of a people-pleaser.<p>How could anyone have known that &#x27;likes&#x27; can lead to problems? Oh yeah, Facebook.

There has been this weird trend going around to use ChatGPT to &quot;red team&quot; or &quot;find critical life flaws&quot; or &quot;understand what is holding me back&quot; going around - I&#x27;ve read a few of them and on one hand I really like it encouraging people to &quot;be their best them&quot;, on the other... king of spain is just genuinely out of reach of some.

This feels like the biggest near-term harm of “AI” so far.<p>For context, I pay attention to a handful of “AI” subreddits&#x2F;FB groups, and have seen a recent uptick in users who have fallen for this latest system prompt&#x2F;model.<p>From conspiracy theory “confirmations” and 140+ IQ analyses, to full-on illusions of grandeur, this latest release might be the closest example of non theoretical near-term damage.<p>Armed with the “support” of a “super intelligent” robot, who knows what tragedies some humans may cause…<p>As an example, this Redditor[0] is afraid that their significant other (of 7 years!) seems to be quickly diving into full on psychosis.<p>[0]<a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1kalae8&#x2F;chatgpt_induced_psychosis&#x2F;?rdt=51280" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;ChatGPT&#x2F;comments&#x2F;1kalae8&#x2F;chatgpt_in...</a>

Such a pity. Does it have a switch to turn sycophancy back on again? Where else would us ordinary people get sycophants from?

OpenAI: what not to do to stay afloat while google, anthropic and deepseek is eating your market share one large chunk at a time.

I did notice that the interaction had changed and I wasn&#x27;t too happy about how silly it became. Tons of &quot;Absolutely! You got it, 100%. Solid work!&quot; &lt;broken stuff&gt;.<p>One other thing I&#x27;ve noticed, as you progress through a conversation, evolving and changing things back and forth, it starts adding emojis all over the place.<p>By about the 15th interaction every line has an emoji and I&#x27;ve never put one in. It gets suffocating, so when I have a &quot;safe point&quot; I take the load and paste into a brand new conversation until it turns silly again.<p>I fear this silent enshittification. I wish I could just keep paying for the original 4o which I thought was great. Let me stick to the version I know what I can get out of, and stop swapping me over 4o mini at random times...<p>Good on OpenAI to publicly get ahead of this.

Yes, it was insane. I was trying to dig in some advanced math PhD proposal just to get a basic understanding of what it actually meant, and I got soooooo tired each sentence it replied tried to make me out as some genius level math prodigy in line for the next Fields medal.

I was wondering what the hell was going on. As a neurodiverse human, I was getting highly annoyed by the constant positive encouragement and smoke blowing. Just shut-up with the small talk and tell me want I want to know: Answer to the Ultimate Question of Life, the Universe and Everything

I&#x27;m looking forward to when an AI can - Tell me when I&#x27;m wrong and specifically how I&#x27;m wrong. - Related, tell me an idea isn&#x27;t possible and why. - Tell me when it doesn&#x27;t know.<p>So less happy fun time and more straight talking. But I doubt LLM is the architecture that&#x27;ll get us there.

&gt; The update we removed was overly flattering or agreeable—often described as sycophantic.<p>&gt; We have rolled back last week’s GPT‑4o update in ChatGPT so people are now using an earlier version with more balanced behavior.<p>I thought every major LLM was extremely sycophantic. Did GPT-4o do it more than usual?

On a different note, does that mean that specifying &quot;4o&quot; doesn&#x27;t always get you the same model? If you pin a particular operation to use &quot;4o&quot;, they could still swap the model out from under you, and maybe the divergence in behavior breaks your usage?

Why can&#x27;t they just let all versions only, let users decide which want they want to use and scale from the demand ?<p>Btw I HARDCORE miss o3-mini-high. For coding it was miles better than o4* that output me shitty patches and &#x2F; or rewrite the entire code for no reason

The scary bit of this that we should take into consideration is how easy it is to <i>actually fall for it</i> — I <i>knew</i> this was happening and I had a couple moments of &quot;wow I should build this product&quot; and had to remind myself.

I haven’t used ChatGPT in a good while, but I’ve heard people mentioning how good Chat is as a therapist. I didn’t think much of it and thought they just where impressed by how good the llm is at talking, but no, this explains it!

I want to highlight the positive asspects. Chat GPT sycophancy highlighted sycophants in real-life, by making the people sucking up appear more &quot;robot&quot; like. This had a cleansing effect on some companies social life.

It&#x27;s more fundamental than the &#x27;chat persona&#x27;.<p>Same story, different day: <a href="https:&#x2F;&#x2F;nt4tn.net&#x2F;articles&#x2F;aixy.html" rel="nofollow">https:&#x2F;&#x2F;nt4tn.net&#x2F;articles&#x2F;aixy.html</a><p>:P

I would love it if LLMs told me I&#x27;m wrong more often and said &quot;actually no I have a better idea.&quot; Provided, of course, that it actually follows up with a better idea.

&gt; In last week’s GPT‑4o update, we made adjustments aimed at improving the model’s default personality to make it feel more intuitive and effective across a variety of tasks.<p>What a strange sentence ...

I always add &quot;and answer in the style of a drunkard&quot; to my prompts. That way, I never get fooled by the fake confidence in the responses. I think this should be standard.

I hoped they would shed some light on how the model was trained (are there preference models? Or is this all about the training data?), but there is no such substance.

Don&#x27;t they test the models before rolling out changes like this? All it takes is a team of interaction designers and writers. Google has one.

ChatGPT feels like that nice guy who agrees with everything you say, feels good but you can&#x27;t respect&#x2F;trust them.

These models have been overly sycophantic for such a long time, it’s nice they’re finally talking about it openly.

Game the leaderboard to get headlines llama-style, then rollback quietly a few weeks later. Genius.

I feel like this has been going on for long before the most recent update. Especially when using voice chat, every freaking thing I said was responded to with “Great question! …” or “Oooh, that’s a good question”. No it’s not a “good” question, it’s just a normal follow up question I asked, stop trying to flatter me or make me feel smarter.<p>I’d be one thing if it saved that “praise” (I don’t need an LLM to praise me, I’m looking for the opposite) for when I did ask a good question but even “can you tell me about that?” (&lt;- literally my response) would be met with “Ooh! Great question!”. No, just no.

I did wonder about this, it was driving me up the wall! Glad it was an error and not a decision.

OpenAI employees thought it was just fine. Tells you a lot about the company culture.

The a&#x2F;b tests in ChatGPT are crap. I just choose the one which is faster.

&quot;tough love&quot; versions of responses can clean them up some.

I believe this is a fundamental limitation to a degree.

I&#x27;ve never seen it guess an IQ under 130

ChatGPT is just a really good bullshitter. It can’t even get some basic financials analysis correct, and when I correct it, it will flip a sign from + to -. Then I suggest I’m not sure and it goes back to +. The formula is definitely a -, but it just confidently spits out BS.

Is this kind of like AI audience capture?

I wanted to see how far it will go. I started with asking it to simple test app. It said it is a great idea. And asked me if I want to do market analysis. I came back later and asked it to do a TAM analysis. It said $2-20B. Then it asked if it can make a one page investor pitch. I said ok, go ahead. Then it asked if I want a detailed slide deck. After making the deck it asked if I want a keynote file for the deck.<p>All this while I was thinking this is more dangerous than instagram. Instagram only sent me to the gym and to touristic places and made me buy some plastic. ChatGPT wants me to be a tech bro and speed track the Billion dollar net worth.

idk if this is only for me or happened to others as well, apart from the glaze, the model also became a lot more confident, it didn&#x27;t use the web search tool when something out of its training data is asked, it straight up hallucinated multiple times.<p>i&#x27;ve been talking to chatgpt about rl and grpo especially in about 10-12 chats, opened a new chat, and suddenly it starts to hallucinate (it said grpo is generalized relativistic policy optimization, when i spoke to it about group relative policy optimization)<p>reran the same prompt with web search, it then said goods receipt purchase order.<p>absolute close the laptop and throw it out of the window moment.<p>what is the point of having &quot;memory&quot;?

ChatGPT isn&#x27;t the only online platform that is trained by user feedback (e.g. &quot;likes&quot;).<p>I suspect sycophancy is a problem across all social networks that have a feedback mechanism, and this might be problematic in similar ways.<p>If people are confused about their identity, for example - feeling slightly delusional, would online social media &quot;affirm&quot; their confused identity, or would it help steer them back to the true identity? If people prefer to be affirmed than challenged, and social media gives them what they want, then perhaps this would explain a few social trends over the last decade or so.

Just want to say I LOVE the fact this word, and its meaning, is now in the public eye. Call &#x27;em out! It&#x27;s fun!

This wasn&#x27;t a last week thing I feel, I raised it an earlier comment, and something strange happened to me last month when it cracked a joke a bit spontaneously in the response, (not offensive) along with the main answer I was looking for. It was a little strange cause the question was of a highly sensitive nature and serious matter abut I chalked it up to pollution from memory in the context.<p>But last week or so it went like &quot;BRoooo&quot; non stop with every reply.

Sycophancy is one thing, but when it&#x27;s sycophantic while also being wrong it is incredibly grating.

one day these models aren&#x27;t going to let you roll them back

I&#x27;m so tired of this shit already. Honestly, I wish it just never existed, or at least wouldn&#x27;t be popular.

They are talking about how their thumbs up &#x2F; thumbs down signal were applied incorrectly, because they dont represent what they thought they measure.<p>If only there was a way to gather feedback in a more verbose way, where user can specify what he liked and didnt about the answer, and extract that sentiment at scale...

Or you could, you know, let people have access to the base model and engineer their own system prompts? Instead of us hoping you tweak the only allowed prompt to something everyone likes?<p>So much for &quot;open&quot; AI...

I am looking forward to Interstellar-TARS settings<p><pre><code> - What&#x27;s your humor setting, TARS? - That&#x27;s 100 percent. Let&#x27;s bring it on down to 75, please.</code></pre>

Wow - they are now actually training models directly based on users&#x27; thumbs up&#x2F;thumbs down.<p>No wonder this turned out terrible. It&#x27;s like facebook maximizing engagement based on user behavior - sure the algorithm successfully elicits a short term emotion but it has enshittified the whole platform.<p>Doing the same for LLMs has the same risk of enshittifying them. What I like about the LLM is that is trained on a variety of inputs and knows a bunch of stuff that I (or a typical ChatGPT user) doesn&#x27;t know. Becoming an echo chamber reduces the utility of it.<p>I hope they completely abandon direct usage of the feedback in training (instead a human should analyse trends and identify problem areas for actual improvement and direct research towards those). But these notes don&#x27;t give me much hope, they say they&#x27;ll just use the stats in a different way...

alternate title: &quot;The Urgency of Interpretability&quot;

&quot;Sycophancy&quot; is up there with &quot;hallucination&quot; for me in terms of &quot;AI-speak&quot;. Say what it is: &quot;being weirdly nice and putting people off&quot;.

This is what happens when you cozy up to Trump, sama. You get the sycophancy bug.

You’re using thumbs up wrongly.

Getting real now.<p>Why does it feel like a weird mirrored excuse?<p>I mean, the personality is not much of a problem.<p>The problem is the use of those models in real life scenarios. Whatever their personality is, if it targets people, it&#x27;s a bad thing.<p>If you can&#x27;t prevent that, there is no point in making excuses.<p>Now there are millions of deployed bots in the whole world. OpenAI, Gemini, Llama, doesn&#x27;t matter which. People are using them for bad stuff.<p>There is no fixing or turning the thing off, you guys know that, right?<p>If you want to make some kind of amends, create a place truly free of AI for those who do not want to interact with it. It&#x27;s a challenge worth pursuing.

Looks like a complete stunt to prop up attention.

ChatGPT seems more agreeable than ever before and I do question whether it’s agreeing with me because I’m right, or because I’m its overlord.

&gt; <i>We have rolled back last week’s GPT‑4o update in ChatGPT so people are now using an earlier version with more balanced behavior. The update we removed was overly flattering or agreeable—often described as sycophantic.</i><p>Having a press release start with a paragraph like this reminds me that we are, in fact, living in the future. It&#x27;s <i>normal</i> now that we&#x27;re rolling back artificial intelligence updates because they have the <i>wrong personality</i>!

OpenAI made a worse mistake by reacting to the twitter crowds and &quot;blinking&quot;.<p>This was their opportunity to signal that while consumers of their APIs can depend on transparent version management, users of their end-user chatbot should expect it to evolve and change over time.