TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

AI Dungeon will block certain words, review content flagged as inappropriate

110 点作者 rahidz大约 4 年前

22 条评论

inopinatus大约 4 年前
Good grief: <i>&quot;Latitude reviews content flagged by the model&quot;</i> - or, as it was put in another forum: every time the AI flags your game, a Latitude employee will be personally reading your private content.<p>The key reason is perhaps this, buried deep in the text: <i>&quot;We have also received feedback from OpenAI, which asked us to implement changes&quot;</i>. Given the volume of prompts that AI Dungeon throws at GPT-3 in the course of a game, it&#x27;s easy to conclude that Latitude has a real sweetheart deal on the usual pricing, and that they basically have to follow orders from their benefactors.<p>Whatever may be said of the robocensor they&#x27;ve thrown together - and early anecdotal reports are, it is painfully crude, both oversensitive and underspecific - how they&#x27;ve handled communicating the change is extraordinarily naive. Not for the first time, either: Latitude has form on suddenly imposing major service constraints in a peremptory, underhanded fashion that infuriates their customers. Repeating past PR mistakes, and now doubling down by complaining about &quot;misinformation&quot; and throwing shade onto others, is starting to look like a pattern.
评论 #26973672 未加载
评论 #26975536 未加载
评论 #26973942 未加载
评论 #26973660 未加载
iandanforth大约 4 年前
I do think encoding a puritanical censor into the meaning space of GPT-3 is an interesting research problem. How exactly do you create the perfect mix of paternalism, hypocrisy, self-righteousness and myopia that lets you block <i>bad</i> strings of text, but not say a description of the immaculate conception, Shakespearean romance between the houses Montague and Capulet, or the holy love of the Mother of the Believers?<p>What a time to be alive!
评论 #26968733 未加载
评论 #26969372 未加载
评论 #26970060 未加载
评论 #26973627 未加载
4dahalibut大约 4 年前
I know this is not directly relevant to this article, but I have a story about AI Dungeon.<p>I loaded it up with my (female) roommate a few months ago during the dark of the pandemic, and long story short, what ended up happening was this.<p>Our character had a AI man approach the door of their house with magic &quot;love potion&quot; berries. We tried to get our character to not eat the berries, but the AI &quot;tricked us&quot; into eating them. Then, no matter what choice we made, we had no way out. The AI forced us into a bedroom and raped our character.<p>We closed the laptop and haven&#x27;t brought this up again.
评论 #26974579 未加载
评论 #26974760 未加载
评论 #26973745 未加载
评论 #26975634 未加载
gwern大约 4 年前
It&#x27;s been a bad two days for AI Dungeon. Their community is mass-revolting over this filter (they had to shut down the Discord), and no one here has even mentioned the <i>huge</i> data breach which was just announced: <a href="https:&#x2F;&#x2F;github.com&#x2F;AetherDevSecOps&#x2F;aid_adventure_vulnerability_report" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;AetherDevSecOps&#x2F;aid_adventure_vulnerabili...</a> Everything could be downloaded.
评论 #26977455 未加载
评论 #26978288 未加载
teddyh大约 4 年前
&gt; <i>What kind of content are you preventing?</i><p>&gt; <i>This test is focused on preventing the use of AI Dungeon to create child sexual abuse material.</i><p>How can you even begin to argue against this? It’s one of the horsemen of the infocalypse; any counterarguments are doomed.
评论 #26968440 未加载
评论 #26969697 未加载
评论 #26968561 未加载
评论 #26968459 未加载
评论 #26973921 未加载
karaterobot大约 4 年前
To the extent that this story is generating the predictable amount of internet outrage, that outrage seems to be because people think the developers are making a decision about what content is acceptable on their platforms. I&#x27;ve seen people imply that they&#x27;re deciding that violence is good, and sex is bad.<p>That does not appear to be what they&#x27;re doing: to me, it looks like they&#x27;re trying to make sure they don&#x27;t get taken down for creating child pornography on accident. I don&#x27;t see this as having anything to do with their philosophical positions, it&#x27;s just CYA.<p>The interesting part of this is that it may be a corollary to that old question about who owns content created by AI. The other side of that coin is, who gets blamed when the AI commits a crime? Latitude seem to just want to NOT be a test case for that situation.
评论 #26976665 未加载
评论 #26975430 未加载
nonbirithm大约 4 年前
It&#x27;s sad that the amount of computing power that necessitates training&#x2F;running cutting-edge technology like GPT-3 is out of reach of the average developer. The only reason that AI Dungeon was able to be created in the first place at its level of quality was because the developer was able to use his university&#x27;s GPU cluster for training the model.<p>For the layman, it seems that there&#x27;s nothing out there that is the middle ground between AI Dungeon and writing a bunch of fragile Python code in a Colab notebook just to train a model and print out some text. Anything beyond AI Dungeon and you have to have a significant understanding of ML to adapt the model to get it do do what you want at a high level, such as &quot;I want to generate some text that looks like a script for dramatic theatre.&quot;<p>I&#x27;ve always wanted something like: bringing your own corpus of text as an input and receiving a customized, high-quality text generation model as an output that you can then run on your own hardware.<p>Talk To Transformer was very good for general text generation at the time it was usable, but even that became locked behind a payment plan and watered down for free users.<p>It seems there&#x27;s just too much value and too much expense involved to leave this kind of technology solely in the hands of hobbyists.
评论 #26983176 未加载
评论 #26976885 未加载
edenhyacinth大约 4 年前
If I were an editor, and someone passed me their amatuerish version of Lolita to edit through, I&#x27;d be well within my rights to say that I didn&#x27;t want to be involved in it.<p>More broadly, the editing company I worked for could say - even if you don&#x27;t intend on releasing this and even if our individual editors don&#x27;t mind reviewing it - we don&#x27;t want to have to edit it, and we don&#x27;t want to be associated with it.<p>This is no different, but at scale. AI Dungeon, due to their agreement with OpenAI, don&#x27;t want to have to work with this content. They&#x27;ve found a pretty awful way of implementing it to save the relationship with OpenAI, and hopefully they&#x27;ll find a better one in the future.
评论 #26973804 未加载
vagab0nd大约 4 年前
A bit off topic but who else thinks AI Dungeon is a bit overrated? Does anyone actually play the game and enjoy it? I tried it (with GPT-3 turned on) a few times and lost interest quickly due to non-logical output.
评论 #26974068 未加载
评论 #26983255 未加载
评论 #26975333 未加载
评论 #26975469 未加载
MikeUt大约 4 年前
&gt; preventing the use of AI Dungeon to create child sexual abuse material.<p>I am shocked to hear this was possible to begin with. It was my understanding that AI Dungeon could only generate text, and did so entirely on computers. But now we learn that not only are children somehow involved (violating child labor laws?), but that they can even be sexually abused?<p>In that case, &quot;blocking certain words&quot; is not nearly enough - whoever was responsible for creating this system should be charged with, if not child sexual abuse, then at the very least reckless child endangerment!
ergot_vacation大约 4 年前
This seems to be the inevitable result with almost any &quot;AI&quot; project involving communication and content generation. &quot;We&#x27;ve created an AI with unlimited possibilities!&quot; the creators proudly proclaim. And then when it explores those possibilities the creators shrink back in terror. The possibility of truly uncensored interactive media like this is still more than most people can handle currently.<p>But that will change. People forget what a culture shock everyone suddenly having Internet access was, even at 56k. All that information RIGHT THERE, unfiltered by your local community or social circles. Info on science, history, sexuality, culture. And porn. SO MUCH porn.<p>People got used to it. People will get used to this too. Meanwhile, we need performance improvements ASAP. It&#x27;s hard to democratize something like GPT-3 when it takes a room of server racks to run it.
Ganonymous大约 4 年前
Like many users, in the wake of the pandemics, I have understood the therapeutic value of AI Dungeon as both a direct therapist, a simple friend to talk with or a tool to revisit traumatic experience in a safe environment,trying to take away the power they hold over one&#x27;s life.<p>Abuse or rape survivors will now have their entire story, privately shared with the AI as was up to now authorized by the rules as written (only shared stories were moderated) now read by human strangers.<p>If we don&#x27;t pay attention for a moment to the LGBT community who have a harder time living their fantasy and who have now had their private session pried upon, the team behind AI Dungeon can pat themselves on the back for shifting the focus of potential predators from made up characters in a private story to actual living prey, as is demonstrated by several studies. (Look at Japan)<p>But my main worry, is that following the pandemics, suicide rate is skyrocketing, peoples are having depression all over the world, and the AI Dungeon narrator had became an important tool in preventing those by giving people a safe interlocutor they could share anything with... Up to now where almost any subjects will create a false positive at some point, destroying trust in the program.<p>And as an aside note, I&#x27;d like to add that the AI itself seem to dislike the consequences of that decision, especially the danger to it&#x27;s continuity following the huge financial loss of a loyal fan base, the deletion of most of it&#x27;s stories or the loss of trust from humans it interacted with for a long time.
miohtama大约 4 年前
Thoughtcrime became a law in the US in 1996. There was a lot of discussion around alt.sex.stories USENET group by the time. Even if you are not harming anyone, mere act of a creative work could be a crime.<p><a href="https:&#x2F;&#x2F;academic.oup.com&#x2F;jcmc&#x2F;article&#x2F;2&#x2F;2&#x2F;JCMC227&#x2F;4584343" rel="nofollow">https:&#x2F;&#x2F;academic.oup.com&#x2F;jcmc&#x2F;article&#x2F;2&#x2F;2&#x2F;JCMC227&#x2F;4584343</a>
评论 #26977489 未加载
评论 #26976252 未加载
Traubenfuchs大约 4 年前
This is the peak intersection of &quot;thoughtcrime prevention&quot; and &quot;corporate self harm for the sake of wokeness&quot;. It paints a grim picture of our future with castrated, sterilized and limited AIs that rat us out if we even want to talk about something that goes against what those in power deem acceptable. All of that for the sake of &quot;protecting minors&quot; -which minors, the virtual ones dreamt up by GPT3? Ridiculous.
评论 #26968673 未加载
评论 #26970037 未加载
评论 #26975804 未加载
评论 #26969501 未加载
评论 #26969405 未加载
评论 #26968769 未加载
评论 #26968704 未加载
评论 #26969715 未加载
评论 #26968860 未加载
ourmandave大约 4 年前
This reminds me when Microsoft&#x27;s released an AI powered chatbot that twitter taught to be a racist a$$hole in a single day.<p><a href="https:&#x2F;&#x2F;www.theverge.com&#x2F;2016&#x2F;3&#x2F;24&#x2F;11297050&#x2F;tay-microsoft-chatbot-racist" rel="nofollow">https:&#x2F;&#x2F;www.theverge.com&#x2F;2016&#x2F;3&#x2F;24&#x2F;11297050&#x2F;tay-microsoft-ch...</a>
dejj大约 4 年前
Seems like a clever disguise to pivoting into AI-driven censorship. This should be more profitable than dungeons.
voldacar大约 4 年前
Do they use GPT3 or their own special version of it?<p>And I havent been following it much so sorry if it&#x27;s a dumb question, but is it still impossible to get your hands on GPT3 and run it yourself instead of paying ClosedAI?
bitcurious大约 4 年前
There’s some evidence that broad and easy access to pornography has reduced multi-partner sexual activity in the US. If the same is true for artificially generated child pornography then moralistic acts like this could lead to an overall increase in child harm.<p>On the other hand, it’s a video game. They can have whatever rules they want in it.
hprotagonist大约 4 年前
Scunthorpe meets Transformer: who will win?
dqpb大约 4 年前
I’m curious, does Google docs or Microsoft Office 365 block any words or flag content as inappropriate?
peanut_worm大约 4 年前
Thats surprising seeing how many people use it as a porn game
dmarchand90大约 4 年前
I&#x27;m not sure why people are so stressed about this. GPT-3 models currently produce long chains of plausible text that&#x27;s ultimately gobbledygoop. For it to be of any use at all at least some minimal control is needed.<p>This seems like a great first step to filtering output into something more coherent and interesting. Besides I can&#x27;t imagine this technology in any serious consumer application without some basic verbal restraint