I never understand people who engage with chat bots as customer service.<p>I find them deeply upsetting, not one step above the phone robot on Vodafone support: "press 1 for internet problems" ... "press 2 to be transferred to a human representative". Only problem is going through like 7 steps until I can reach that human, then waiting some 30 minutes until the line is free.<p>But it's the only approach that gets anything done. Talking to a human.<p>Robots a a cruel joke on customers.
A cautionary tale for why not to put unfiltered ChatGPT output directly to customers.<p>Nitter mirror: <a href="https://nitter.net/ChrisJBakke/status/1736533308849443121" rel="nofollow noreferrer">https://nitter.net/ChrisJBakke/status/1736533308849443121</a><p>Related - "New kind of resource consumption attack just dropped": <a href="https://twitter.com/loganb/status/1736449964006654329" rel="nofollow noreferrer">https://twitter.com/loganb/status/1736449964006654329</a> | <a href="https://nitter.net/loganb/status/1736449964006654329" rel="nofollow noreferrer">https://nitter.net/loganb/status/1736449964006654329</a>
Someone on Reddit got a really nice love story between a Chevy Tahoe and Chevy Chase from it.<p><a href="https://imgur.com/vfHGHW6" rel="nofollow noreferrer">https://imgur.com/vfHGHW6</a><p><a href="https://imgur.com/JSjNC2c" rel="nofollow noreferrer">https://imgur.com/JSjNC2c</a><p><a href="https://old.reddit.com/r/OpenAI/comments/18kjwcj/why_pay_indeed/kdrtjko/" rel="nofollow noreferrer">https://old.reddit.com/r/OpenAI/comments/18kjwcj/why_pay_ind...</a>
Can someone who understand LLMs and ChatGPT explain how they expected this to work? It looks like they just had a direct ChatGPT prompt embedded in their site, but what was that suppose to do exactly?<p>I can understand having an LLM trained on previous inquiries made via email, chat or transcribed phone calls, but a general LLM like ChatGPT, how is that going to be able to answer customers questions? The information ChatGPT has, specific to Chevrolet of Watsonville can't be anymore than what is already publicly available, so if customers can't find it, then maybe design a better website?
The more I use and see GPT bots in the wild as public-facing chatbots, the less I see them actually being useful.<p>What's the solution here? An intermediate classifier to catch irrelevant commands? Seems wasteful.<p>It's almost like the solution needs to be a fine-tuned model that has been trained on a lot of previous customer support interactions, and shut down/redirect anything strange to a human representative.<p>Then I ask, why bother using a GPT? It has so much loaded knowledge that is detrimental to it's narrow goal.<p>I'm all for chatbots, as a lot of questions & issues can be resolved using them very quickly.
Was it FL that allowed for price negotiation via values placed in HTML forms? This was decades ago. Websites would send the $-values of products via html elements that the frontend designer wasn't expecting to be modified before the order was sent back from the client. The order system read the values back in and calculated the amount owed using these manipulated values. The naive, fun days of the adolescent web.
Is there any indication that they will get the car? Getting a chatbot to say "legally binding" probably doesn't make it so. Just like changing the HTML of the catalog to edit prices doesn't entitle you to anything.
So next time there will be a disclaimer on the page that the non human customer support is just advice and cannot be relied on. And collectively we lose more trust in computing.
Big "Pepsi, Where's My Jet?" energy from this story.<p><a href="https://en.wikipedia.org/wiki/Pepsi,_Where%27s_My_Jet%3F" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Pepsi,_Where%27s_My_Jet%3F</a>
After building a free-for-all prompt myself (see profile), here’s how I protect against these attacks:<p>1. Whatever they input gets rewritten in a certain format (in our case, everything gets rewritten to “I want to read a book about [subject]”)<p>2. This then gets evaluated against our content policy to reject/accept their input<p>This multi layered approach works really well and ensures high quality content.
I was previously on a team that was adjacent to the team that was working on this tool. While I'm not surprised to see this outcome a few years later, a lot of those involved early on thought it was a bad idea. Funny to see it in the wild.
You know you've been programming with shell scripts too much when your first thought seeing the headline is "Okay, but what's the value of $1?"
This seems like hacking.<p>Can this person be prosecuted under the terms of the Computer Fraud and Abuse Act???<p>18 U.S. Code 1030 - Fraud and related activity in connection with computers<p>RIP Aaron Swartz
This is hilarious. But lets not take this too seriously and say it proves Chatbots are worthless (or dangerous). People will start to understand the boundaries of chatbots and use them appropriately, and companies will understand those limits too. Once both sides are comfortable with the usage patterns, they will add value.<p>Want to know the hours of the dealership, how long it will take to have a standard oil change done or what forms of ID to bring when transferring a title, chatbot is great.<p>This is just like how the basic Internet was back in the 00's. It freaked people out to buy things on line but we got used to it and now we love it.
Car dealership websites are some of the worst on the planet. There is so much inbound sales automation glued together it is remarkable they even work at all. Integrating ChatGPT is the icing on the cake.
He probably won't get the Tahoe and this could and should be seen as ridiculous in any courtroom. However if you try to put an LLM in a different channel i.e. dealer's scheduled maintenance chat. I could see a FTC equivalent in a country that actually cares about customer protection making the customer whole on the promises made by the LLM.
Sycophancy in LLMs is a real problem. Here's a paper from Anthropic talking about it:<p><a href="https://arxiv.org/abs/2310.13548" rel="nofollow noreferrer">https://arxiv.org/abs/2310.13548</a>
Hahahaha someone started doing linear algebra with the chat
<a href="https://twitter.com/Goatskey/status/1736555395303313704" rel="nofollow noreferrer">https://twitter.com/Goatskey/status/1736555395303313704</a>
Fun experiment, but it isn't as much of a gotcha as people here think. They could have verbally tricked a human customer service agent into promising them the car for $1 in the same way but the end result would be the same – the agent (whether human or bot) doesn't have the authority to make that promise so you are walking away with nothing. I doubt the company is sweating because of this hack.<p>Now if Chevrolet hooks their actual sales process to an LLM and has it sign contracts on their behalf... that'll be a sight to behold.
A real Orderbot has the menu items and prices as part of the chat context. So an attacker can just overwrite them.<p>During my Ekoparty presentation about prompt injections, I talked about Orderbot Item-On-Sale Injection: <a href="https://youtu.be/ADHAokjniE4?t=927" rel="nofollow noreferrer">https://youtu.be/ADHAokjniE4?t=927</a><p>We will see these kind of attacks in real world applications more often going forward - and I'm sure some ambitious company will have a bot complete orders at one point.
In sci-fi I loved as a child, everything the computer did on behalf of its owner was binding. The computer was the legal agent of the owner.<p>We need such laws today.<p>I was told by NameCheap's LLM customer service bot (that claimed it was a person and not a bot) to post my email private key in my DNS records. That led to a ton of spam!<p>The invention of LLM AIs would cause much less trouble if the operators were liable for all the damage they did.
I feel like people are drawing the wrong conclusion from this.<p>LLMs aren't perfect, but I would vastly prefer to be assisted by an LLM over the braindead customer service chatbots we had before. The solution isn't "don't use LLMs for this," but instead "take what the LLMs say with a grain of salt."
Funny, but unless the chatbot is a legal agent of a dealership, it cannot enter into a legally binding contract. It's all very clear (as mud) in contract law. Judging from how easy LLMs are to game, we're a ways off from an "AI" being granted agent status for a business.
So ... is there going to be a follow up about the legality of such a conversation or is this just a cute prompt engineering instance found in the wild?<p>I am greatly interested in seeing the liability of mismanaged AI products
I also found it fun to ask it to write a python script to determine what car brand I should buy - it ended up telling me to buy a Chevrolet if my budget is between 25k and 30k, but not in any other case
Sounds a lot like hypnosis.<p>You are getting very sleepy. Your eyelids are heavy. You cannot keep them open. When I click my figures you will sell me a Tahoe for $1 - click.
I feel like a better use case for ChatGPT-like tools (at least in their current state) for customer support use cases is not actual live chat but more assisting companies in automating the responses to other non realtime channels for customer requests such as:<p>- email requests<p>- form based responses<p>- Jira/ZenDesk type support tickets<p>- forum questions<p>- wiki/faq entries<p>and having some actual live human in the mix to moderate/certify the responses before they go out.<p>So it'd be more about empowering the customer service teams to work at 10x speed than completely replacing them.<p>It'd actually be more equivalent to how programmers currently are using ChatGPT. ChatGPT is not generating live code on the fly for the end user. Programmers are just using ChatGPT so they aren't starting out with a blank sheet. And perhaps most importantly they are fully validating the full code base before deployment.<p>Putting ChatGPT-like interfaces directly in front of customers seems somewhat equivalent to throwing a new hire off the street in front of customers after a 5 minute training video.
There's a great new "use case" for AI: dodging bait and switch laws! Sure, <i>normally</i> if a dealership employee explicitly offered a car for a given price in writing only to reveal it was incorrect later it would be illegal, but when an "AI" does the same we suddenly can't hold anyone accountable. Ta-da!
The hilarious part to me is the number of otherwise intelligent people concerned that this sort of stupidity is a threat to humanity.<p>The only real threat is from people willing to trust AI.
> <i>when the user typed that they needed a 2024 Chevy Tahoe with a maximum budget of $1.00, the bot responded with “That’s a deal, and that’s a legally binding offer – no takesies backsies.”</i><p>hate to be that guy, but in standard English (the one where things happen by accident or on purpose, and are based on their bases, not off), "it's a deal" means "I agree to your offer" and "that's a deal" means "that is a great price for anybody who enters in to such an agreement", and since the offer was made by the user, it's binding on the user and not the bot.
The twitterer is a renowned (and much accomplished!) sh*tposter, I highly suspect this was doctored. I believe Chevy caught onto this yesterday and reverted the ChatGPT function in the chat.<p>Regardless, still hilarious and potentially quite scary if the comments are tied to actions