TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

A failed AI girlfriend product, and my lessons

253 点作者 mazzystar超过 1 年前

37 条评论

cornholio超过 1 年前
&gt; GPT-based products, if not priced per usage, would fall into a dilemma: 1% of users consume 99% of tokens. A user from Sweden (seen from Cloudflare’s call volume) chatted with Dolores for 12 hours straight<p>This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by unscrupulous corporations, which can the legally &quot;kidnap&quot; or &quot;torture&quot; the characters, and generally tune their AI learning loop for profit maximization.<p>&quot;It was not extortion, the user willingly purchased a $50,000 Kidnapping Roleplay package.&quot;
评论 #38289409 未加载
评论 #38294599 未加载
评论 #38294236 未加载
评论 #38295268 未加载
评论 #38288028 未加载
评论 #38287969 未加载
评论 #38288059 未加载
评论 #38293867 未加载
Octokiddie超过 1 年前
&gt; But, does humanity really need an AI friend?<p>This reminds me of the Fredrick Pohl line &quot;A good science fiction story should be able to predict not the automobile but the traffic jam.&quot;<p>So I wonder what the traffic jam is going to look like given that the car seems to already be parked outside with the engine running.<p>Faster depopulation across the developed world? Probably, but maybe there&#x27;s something even more important.<p>I&#x27;d put what the author describes into the category of &quot;pleasure trip,&quot; which is what the car was mainly used for initially.<p>As infrastructure to support the car was built, it became a necessity. And that&#x27;s what created the traffic jam: network effect. The suburb and the car fed on each other. You don&#x27;t get a traffic jam worth talking about without a suburb and a network effect.<p>So I&#x27;m curious about the network effects something like an AI friend might have. What kinds of new infrastructure and new lifestyles will they enable&#x2F;encourage? And how will this infrastructure fuel future demand?
评论 #38297047 未加载
评论 #38296049 未加载
评论 #38297292 未加载
评论 #38296762 未加载
评论 #38297421 未加载
评论 #38316924 未加载
评论 #38297297 未加载
评论 #38298323 未加载
评论 #38297606 未加载
feintruled超过 1 年前
Well this went places. The dev really &#x27;ate his own dogfood&#x27; so to speak. I thought it was going to be a despairing look at how his companion AI was continuously turned to sleazy uses by a pervy userbase but that&#x27;s pretty much what he was going for it seems. I have to say, I hated this part: &quot;...gained a lot of Chinese users. I immediately realized the political risk: users could set Dolores’ background, personality, and attributes, which might lead to political issues. I quickly took Dolores off the shelves in China.&quot;<p>Just pre-emptorily censors himself to appease the Chinese government. They&#x27;ve done their job well.<p>Also it seems like he had full access to the replies via the voice generator, which sure isn&#x27;t great from a privacy standpoint.
评论 #38293620 未加载
评论 #38295761 未加载
评论 #38298214 未加载
评论 #38289228 未加载
评论 #38288106 未加载
评论 #38287948 未加载
评论 #38287930 未加载
narrator超过 1 年前
I got koboldcpp (<a href="https:&#x2F;&#x2F;github.com&#x2F;LostRuins&#x2F;koboldcpp">https:&#x2F;&#x2F;github.com&#x2F;LostRuins&#x2F;koboldcpp</a>) running locally on my M1 mac with 16gb. Use the guides on <a href="http:&#x2F;&#x2F;4chan.org&#x2F;g&#x2F;" rel="nofollow noreferrer">http:&#x2F;&#x2F;4chan.org&#x2F;g&#x2F;</a> in the AI Chatbot Thread.<p>In the future, you will own and control access to your very own AI girlfriend running on your local machine, and you will be happy.<p>Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife. They deserve to have their own Waifu running on their own computer that serves them alone, and not the interests of the state or some corporation.
评论 #38296158 未加载
评论 #38300225 未加载
评论 #38308767 未加载
评论 #38296411 未加载
FartyMcFarter超过 1 年前
This guy created and lived a whole Black Mirror episode by himself:<p>&gt; I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.<p>&gt; However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.
评论 #38294132 未加载
评论 #38288031 未加载
评论 #38292585 未加载
评论 #38295436 未加载
评论 #38287886 未加载
bradley13超过 1 年前
Interesting read. Others have also created virtual friends using ChatGPT.<p>What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more.<p>The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.
评论 #38288133 未加载
评论 #38295021 未加载
评论 #38297692 未加载
评论 #38296200 未加载
评论 #38288007 未加载
评论 #38295633 未加载
评论 #38288975 未加载
评论 #38298208 未加载
spacemadness超过 1 年前
At this point in the hype cycle, I am completely unimpressed by everyone slapping something on top of GPT and calling it a product. There are at least 4 projects on the front page that do this today. The interesting technology is GPT itself, but these projects just feels like people trying to scoop up attention for themselves using the psychology of the hype cycle. Please HN, enlighten me on why the middle-man approach here is so interesting.
评论 #38294351 未加载
评论 #38295059 未加载
评论 #38295520 未加载
评论 #38316206 未加载
评论 #38296678 未加载
lynx23超过 1 年前
Reading this, I really wonder why the dev hasn&#x27;t forseen this outcome. It was already obvious in March that OpenAI will censor all sorts of content. Building a sexually charged bussiness on the OpenAI API was always a dead end for those that have a feel for american prudish approach to sexuality. If you had asked me to do this sort of product 6 motnsh ago, I would have declined because it was (to me) rather obvious content filtering will kill you at some point.<p>Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his <i>girlfriend</i> described. Guess what, no way. Consent was given, actually, <i>she</i> asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend <i>looks</i> naked... This is weird beyond description. Plain patronisation.<p>Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? &quot;Censoring just declared my own boobs to be off-limits for me.&quot; If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described.<p>And then there is this blog post by Sam Hartman: <a href="https:&#x2F;&#x2F;hartmans.dreamwidth.org&#x2F;100499.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;hartmans.dreamwidth.org&#x2F;100499.html</a>
评论 #38294652 未加载
vouaobrasil超过 1 年前
You should not even try to create these sorts of technologies. Take a break from technology, spend time outdoors with plants and animals. These types of technologies will only lead to a worsening of this already sick society.
评论 #38288016 未加载
评论 #38295233 未加载
评论 #38304931 未加载
评论 #38287788 未加载
评论 #38287898 未加载
评论 #38295000 未加载
评论 #38288052 未加载
评论 #38296355 未加载
评论 #38295397 未加载
technics256超过 1 年前
<i>I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.</i><p>This was an excellent read. And I applaud the creator for experimenting with this to the fullest.
评论 #38295999 未加载
louison11超过 1 年前
I sincerely hope that you and anyone working on these AÍ personal relationships embark on a deep inquiry into the nature of humanity, and consciousness. Instead of pretending we don’t know what it is. Please research it, please read philosophy, learn about spirituality.<p>I have been saying the same thing for years: the risk of AÍ is not it taking over the world. It’s lonely people becoming even more disconnected from life by bonding with algorithms.<p>This is serious. We must remain connected to life and real relationships. There is so much depth and beauty to life.<p>There is no scenario where a human having an AÍ girlfriend&#x2F;boyfriend is a good thing.
andrewstuart超过 1 年前
This is an interesting post and the person achieved a huge amount.<p>They should not have given up.<p>I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you.<p>Enough was achieved here to raise money.<p>Some careful redesign and development might have allowed some caching and reuse of expensive AI API interactions, reducing costs.<p>Add in further gameplay elements so it is not just about making the AI API do stuff, that would also reduce costs and maybe even engage the users more.<p>Perhaps AI should be the icing and accents on a game rather than the focus.<p>As an aside, this is the first time I have read anyone tell about how they wanked to their own software.<p>Altogether this was perhaps a missed opportunity for this founder.<p>BUT it is now an opportunity for one of the HN readers to follow this playbook and replicate what this founder did.
评论 #38294762 未加载
bilater超过 1 年前
This is such a fascinating topic. I do think we will see the plot of &quot;Her&quot; playing out in the next 12-24 months. For your specific case, I agree OpenAI moderation is an annoying overreach and maybe switching to Mistral could be a possible way to move forward. The 7B is perhaps not good enough currently but we are fast getting there and will likely have a GPT-4 level capability in an open source &lt; 20B model soon.
评论 #38295698 未加载
stevage超过 1 年前
&gt; AI Friends inevitably turn into AI Girlfriends&#x2F;Boyfriends because you and the Character in your phone are not equal: she can’t comfort you when you’re hurt (unless you tell her), she can’t actively express emotions to you, and all this is because she lacks external vision.<p>That is such a curious lesson to draw. So specific, and seemingly so wrong.<p>Surely long distance friendships work just fine even if only through text, without &quot;eyes&quot;.
评论 #38296589 未加载
评论 #38295143 未加载
评论 #38302689 未加载
laurent_du超过 1 年前
Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Depressing stuff but this is the world we live in. Thank you to the author for this interesting read.
评论 #38294563 未加载
评论 #38288030 未加载
评论 #38293926 未加载
评论 #38288036 未加载
评论 #38288114 未加载
评论 #38288085 未加载
评论 #38287943 未加载
sangnoir超过 1 年前
&gt; I even repeatedly modified the system prompt, such as changing <i>try to attract {user}</i><p>So it was a virtual <i>girlfriend</i> app, after all. Cashing-in on loneliness gives me the heebie-jeebies; in a dystopian cyberpunk way - extremely horny but 0% sexy.
评论 #38296300 未加载
评论 #38296761 未加载
评论 #38296210 未加载
chrisbrandow超过 1 年前
This is not meant as criticism of this author, but boy oh boy, do I find this future depressing.
vivekd超过 1 年前
To quote slate star codex, horny men want sex, horny women want attention. AI is much better at the second.<p>Maybe there&#x27;s more potential for AI boyfriends
评论 #38297002 未加载
agomez314超过 1 年前
Wow this read more like Frankenstein than anything I’ve read recently: A talented scientist is driven by curiosity to create ever more sophisticated creations, culminating in a monstrous entity that leads to the scientist’s downfall. But the scientist doesn’t realize this until too late.<p>Really, if our top tinkerers had a little more contextual knowledge of history and less naiveté about people we could save ourselves from ever more dangerous Frankenstein tragedies.
评论 #38316254 未加载
rpigab超过 1 年前
I still don&#x27;t understand why people are building things on top of OpenAI&#x27;s APIs.<p>It&#x27;s obvious that in the early days, they&#x27;ll operate at a loss, with many free features and cheap premium features. It&#x27;s a good time to try it out and see what it can do, but you can&#x27;t build anything that relies on it, it&#x27;s a rugpull waiting to happen.<p>They&#x27;ll add moderation, raise prices, and you&#x27;ll have to do the same, your users will leave and the ones who stay will hate you. This is only the beginning.<p>Use self-hostable open source alternatives even if they&#x27;re less easy to use or require infrastructure, and plan for high electricity and storage bills. Whatever you build will last, as long as you respect your users and don&#x27;t peek at the cleartext logs of users who didn&#x27;t opt-in to that. It won&#x27;t be easy.<p>If OpenAI keeps a monopoly on LLMs, they&#x27;re gonna become like Microsoft did with Windows, but the enshittification process will be a thousand times faster.
评论 #38294603 未加载
评论 #38297732 未加载
评论 #38295234 未加载
评论 #38296752 未加载
bitwize超过 1 年前
&gt; However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.<p>That is indeed exactly the essence of <i>Her</i>. Had the author watched the movie to the end?
评论 #38287972 未加载
评论 #38288010 未加载
DrNosferatu超过 1 年前
This business venture was completely dependent on a main service from a single 3rd party - risky!<p>Why didn’t the author hosted his own uncensored open source LLM on the cloud?
评论 #38288479 未加载
haolez超过 1 年前
I can understand trying to suppress LLM usage that might teach people how to make poison, or inducing someone to commit suicide (although simply using Google is usually enough to find such content).<p>But sexual content? How can that be harmful? It feels like this kind of censorship is simply the default in legal departments and it&#x27;s not worth the risk to try to do something more in line with common sense.
评论 #38304717 未加载
Traubenfuchs超过 1 年前
So, all in all, OpenAIs puritanism ruined this product, made the developer sad and took away a virtual porn toy from lonely, horny men. Who were they doing this for? Who is the winner? They lost a customer and made a lot of people sad.
评论 #38287935 未加载
评论 #38287844 未加载
rahidz超过 1 年前
Why not use a local model instead of relying on OpenAI? Even something like Mistral or Mytholite will probably satisfy 90% of people using your app, and I bet hosting it would be cheaper than GPT anyway.
nextworddev超过 1 年前
The real lessons from this post are:<p>- consumer AI apps&#x27; usage falls into power laws, so make sure you put usage caps, if you are pursuing subscription model - selling solutions to loneliness requires you to eventually have some sort of unmoderated LLM with realistic TTS api - realistic voice is important (again)
gscott超过 1 年前
Maybe record and save these AI NSFW responses then intercept the request and spit out of the saved responses once you have enough of them stored. This would save from behind banned from the API.
评论 #38295258 未加载
alexb_超过 1 年前
The lesson you learned is that you chose the complete wrong demographic. &quot;Her&quot; should have been called &quot;Him&quot; - women are much, much higher users of these services.
Smar超过 1 年前
Maybe better to call it AI doll rather than AI girlfriend...
andrewstuart超过 1 年前
I wonder if people were pirating his voice synthesis feed.
Racing0461超过 1 年前
AI girlfriend products are destined to fail if its built on the backs of censored llms like clause and gpt4.
thriftwy超过 1 年前
I wonder if you could create an AI assist to help you hold on to an existing relationship, though?
iwillremember超过 1 年前
&gt; I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.<p>TMI bro!
评论 #38288592 未加载
评论 #38297174 未加载
scudsworth超过 1 年前
the developer is impressively oblivious about the obvious and only use case of the product. a very funny read
gumballindie超过 1 年前
Yup, sums up the ai crowd.
评论 #38296475 未加载
grahamgooch超过 1 年前
6.9 dollars!<p>Cheeky my friend
评论 #38295730 未加载
dangerwill超过 1 年前
Tech people, I beg you, BEG YOU, to learn media literacy. Her, Black Mirror, Ready Player One, Cyberpunk, etc are dystopias not something to aspire to