Social AI companions pose unacceptable risks to teens and children under 18

I want to know if anyone has answered the question: what does a healthy relationship with this thing look like?<p>All kids grow up and are eventually exposed to sex drugs and rock&#x27;n roll. These things are part of our world, you have to coexist with them. The problem with video games, social media, AI, and all things tech is they&#x27;re so new and evolving so fast that no one really knows what a healthy relationship looks like. Though awareness is growing and we&#x27;ve started asking questions like: how much screen time is ok? At what age do I allow my kid to make a social media account? Should we be using our phones last thing before bed and first thing in the morning? Not to mention more wide spread issues of privacy and exposure to content ranging from amusing to abusive. AI as a &quot;convincing BS artist&quot; you can engage with endlessly is something I struggle to wrap my head around. My personal policy is to keep AI on a short leash; use it sparsely, don&#x27;t overly rely on it, and always question its assertions. But allowing unrestricted access to a powerful tool that requires self control and good judgment is inviting disaster. Banning it for kids makes sense, but what about everyone else?

Better social AI companions might help. They would be a step up from, say, the 30th percentile parent.<p>SF versions:<p>&quot;I Always Do what Teddy Says&quot; (1964), by Harry Harrison.[1]<p>&quot;A Young Lady&#x27;s Illustrated Primer&quot;, in Neil Stephenson&#x27;s <i>The Diamond Age</i>.<p>[1] <a href="https:&#x2F;&#x2F;archive.org&#x2F;details&#x2F;bestofharryharri0000harr_z2p6" rel="nofollow">https:&#x2F;&#x2F;archive.org&#x2F;details&#x2F;bestofharryharri0000harr_z2p6</a>

Can someone link the actual products they&#x27;re talking about? ChatGPT isn&#x27;t exactly great at forming emotional bonds, but I could see some other app doing this.

I feel like this is one of the most obvious takes out there but simultaneously parents are crazily unaware of it. Parents need to be more aware of ai companions and Roblox and how bad they are.

Honestly, I think they pose a far bigger risk to some adults. Adult&#x27;s have a harder time making friends, in changing themselves when stuck in a loop, and loneliness is growing exponentially in big cities.<p>Of course children are children, and adults are responsible for their own choices.<p>Btw, I like generative AI and LLMs, I&#x27;m not trying to say &quot;ban it&quot; or &quot;regulate it&quot;, just pointing out that lonely adults are a very real thing, and some of them can and will get stuck in this, the same way they can and do get stuck in other online hobbies.

How exactly do they think that parents can ban it? You can just ask ChatGPT to become a social companion.

Can&#x27;t say I see this trend declining any time soon. People seem to find affirmation (and in some sense validation) interacting with these LLM&#x27;s. Provided the AI in question is well-aligned that shouldn&#x27;t be much of a concern. Not much different from talking to a friend&#x2F;therapist for emotional support, is it?

Is everyone subscribing to the paywall to read a solitary cross-indexed article? Or are most of us commenting on the paragraph we are allowed to see?

&quot;Social AI companions are the next frontier in EduTech, and should be welcomed with hope and optimism,&quot; said the VC. &quot;Our mission is to change the world for the better, and anyone in our way is evil Luddite trying to hurt you.&quot;

I can see educational AI companions as workable in narrow contexts, eg a model fine-tuned on Paul Erdos and Martin Gardner and similar could be great for helping students work through math problem sets.<p>You&#x27;d probably want it to reject questions on religion and politics and human relationships to avoid the furious parental outrage, though. Narrow, well-defined contexts only. Even so some kids would come up with jailbreak strategies.

cant read the article without subscription. Is this ok with HN guidelines?

I mean let&#x27;s be real, they pose unacceptable risks to everyone. But in the west we only have strong societal norms around protecting children from themselves.

Lonely young people are supposed to suffer for their sin of being unacceptable to the masses. We shouldn&#x27;t let future incel types be allowed to find a social outlet because it short-circuits the &quot;nudge&quot; that makes them &quot;self improve&quot;.<p>This is basically what the anti-AI as social companion crowd believes.<p>Actually, nerdy or autistic basement dwelling people are not &quot;bad&quot; and often do not deserve the social scorn they get. It&#x27;s good that we can short-circuit the &quot;need&quot; for social interaction, especially with these kind of companions.<p>All this pearl clutching because one kid doing NSFW chats with a Danerys Targayn chatbot on character.ai got some media attention after committing suicide.

Oh look, it&#x27;s the people that want to ban the sales of violent video games to minors.