OpenAI deems its voice cloning tool too risky for general release

pretty convinced at this point these announcements are more about building buzz for openai then any real concern for humanity

Given how good the newly released available-weight VoiceCraft[1] project seem to be, I would say the cat is already out of the bag.<p>People will soon have to learn that they should not trust voiced content. In regards to scam prevention, the key learning will to validate the security of the channel and never trust a voice alone.<p>If someone you know calls you from an established channel (their usual account &#x2F; phone etc), it&#x27;s probably safe. But any unauthenticated channel communication (unknown account etc) should be treated as suspicious.<p>[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;jasonppy&#x2F;VoiceCraft">https:&#x2F;&#x2F;github.com&#x2F;jasonppy&#x2F;VoiceCraft</a>

It is an Easter miracle, whom amongst us has not prayed for 500+ words on some software that may or may not exist but certainly has zero impact on anything in the world?

Why the hell did they build it then?

lol this marketing technique is getting very old. <a href="https:&#x2F;&#x2F;github.com&#x2F;coqui-ai&#x2F;TTS">https:&#x2F;&#x2F;github.com&#x2F;coqui-ai&#x2F;TTS</a> is already amazing and open source.

Yeah I also have an AGI already, but I won&#x27;t release it, it&#x27;s too dangerous for you lot.

I had a job about a month or two ago on Upwork which was supposedly going to be about a character.ai clone or something similar. After discussing it for weeks, on the last day they decided it wasn&#x27;t about personas in general and absolutely had to be impersonation of Trump and Biden.<p>I told them it was a very questionable idea. But I needed the money to be able to make rent. So I took the job hoping that they were still interested in other types of characters and would be reasonable.<p>I had a basic demo with Trump&#x27;s voice up in about a week. I showed them the first demo and had a phone call suggesting that this might actually be illegal pretty soon.<p>They were sure there was no problem and said they had spoken to lawyers.<p>The next day, Eleven Labs sent out an email explaining that some voices were getting banned. The attempt to clone Biden&#x27;s was blocked by Eleven Labs.<p>They had structured the contract as a single fixed fee, although what we had discussed was weekly payments. At the last minute without discussion they had thrown in a talking head requirement.<p>So I received the first 451 HTTP status code I have ever seen when requesting that D-ID do a lip sync animation of Trump&#x27;s head.<p>I ended up using a different video model from replicate that was not a great lip sync but very engaging if you squint. I did not do Biden. Did about three weeks of work and collected around $630 after Upwork took their fee.<p>I&#x27;m proud to say I did make my rent payment though since I had another project at the same time which was a low hourly but still helped.

Potentially unpopular opinion- this may have been done in part with the objective of enforcing regulations or setting up AI laws. Given OpenAI’s position that comes across as regulatory capture.<p>Wiki article on the topic: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Regulatory_capture" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Regulatory_capture</a>

Election year for ~4.21% of the world population… Or that’s the part they are keeping in mind with this announcement anyway.

[dupe]<p>Some more discussion on official post: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39866493">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=39866493</a>

Can we not already basically do that? I swear I’ve seen pretty convincing AI text-to-speech that could be used for nefarious purposes.

I usually would&#x27;ve commented something along the lines of how &quot;open&quot; their AI is, but to be fair, this is a pretty understandable move. Sadly, it&#x27;s only a matter of time before someone else will offer this feature to the public. So &quot;open&quot;AI is only slowing down the process.

Remember when GPT-2 was deemed &quot;too dangerous to release&quot;? Spreading FUD appears to be OpenAI&#x27;s viral growth strategy.<p><a href="https:&#x2F;&#x2F;slate.com&#x2F;technology&#x2F;2019&#x2F;02&#x2F;openai-gpt2-text-generating-algorithm-ai-dangerous.html" rel="nofollow">https:&#x2F;&#x2F;slate.com&#x2F;technology&#x2F;2019&#x2F;02&#x2F;openai-gpt2-text-genera...</a>

This is not interesting anymore. I am too busy looking at the spicy xz backdoor

build up for a mole leaking the code or a cyberattack originating in eastern europe or russia or a combination of both, thus including corruption and creating the need for stronger oversight of people working on AI by some clandestine agency and of course extreme regulation of open source AI and people capable of writing that kind of code, which would include extreme regulation of access to the training data and so on ... the interest groups in power censored the training data, took openai under their wing, and the fun continues. sorry, couldn&#x27;t resist.

My own speculation is it’s too crap &#x2F; embarrassing to release.

Are they trying to get competitors regulated?

How very Open of them

then why develop it in the first place?