Once the generation times get to real-time, I dunno what's going to happen. I follow the voice acting community a lot and this is a big existential threat level of worry in that community.<p>But I also see some positives for the narrative voice field, but at the expense of actual actors.<p>The latest sequel to a favorite audio book series has the professional narrator pronouncing different character names and town names entirely different than the previous 7 books.<p>In another book the narrator completely changed the character voices in the sequel compared to the first.<p>The positives for listeners is eventually we can guarantee that voices are completely consistent between narrations. An editor will soon be able to describe the emotions of a character and nuance how the AI performs certain scenes.<p>It's really sad but I think the end of human audio storytelling is coming to and end quite rapidly.
I think he's right about this—things are going to get weird and ugly and people aren't prepared for what's coming. Here's my suggestion for safeguarding your sanity in the years ahead: find the people and the blogs you love, embrace RSS, block or avoid everything else, read old books, stop listening to podcasts, revert to email as the primary channel for occasional "social" correspondence, abandon all side projects that require additional screen time, go outside.<p>This is the way.
Those fake AI voices are perfect tools for phishing campaigns.<p>I wonder if ChatGPT combined with a voice generator can mount successful phishing campaigns.<p>It looks like Cyberpunk future we never asked is already here.
I tried ElevenLabs and it is truly amazing. You don't need much audio to train it and the final result can be incredible. I shared some of the snippets with friends and relatives and after the intitial scare (same as John) we agreed that the outcome wouldn't be different than a few years ago...<p>We had impersonators for decades, so what have stopped political parties to hire an impersonator to create fake audios? The IA in this case is going to make it more accesible, but in political parties you don't want thousands of fake audios (they would lose credibility), you need only one.<p>Also, photoshopping photos would have a similar effect, and that technique has been available for years as well.
I've used a tool to make myself sound like a girl, it is freakily realistic and uncanny hearing what it produces. Chinese love scam rings are going to be all over this in a year, coupled with image/video generation
> I don’t think the general population is prepared for this<p>i dont know how many videos it will take of trump and biden shit talking eachother on xbox live but it looks like several on youtube are near 1m views
Would a Roger Stone or Steve Bannon type fake a Russian collusion dossier and then run it through the media for literally years?<p>What are the "types" that would do something like this?<p><a href="https://www.bbc.com/news/world-us-canada-59168626" rel="nofollow">https://www.bbc.com/news/world-us-canada-59168626</a><p><a href="https://www.theguardian.com/us-news/2022/oct/11/russian-analyst-igor-danchenko-steele-dossier-sources" rel="nofollow">https://www.theguardian.com/us-news/2022/oct/11/russian-anal...</a><p><a href="https://www.nytimes.com/2022/10/18/us/politics/igor-danchenko-russia-acquittal-trump.html" rel="nofollow">https://www.nytimes.com/2022/10/18/us/politics/igor-danchenk...</a><p><a href="https://www.npr.org/2021/11/12/1055030223/the-fbi-arrests-a-key-contributor-to-efforts-trying-to-link-trump-with-russia" rel="nofollow">https://www.npr.org/2021/11/12/1055030223/the-fbi-arrests-a-...</a>
It is scary for sure. We were heading towards a brave new world even before chatGPT, where what's real was no longer to be taken for granted and you couldn't trust your eyes and ears. ChatGPT has just hastened things. I'm sure a new market will emerge for authentication services that verify the speaker or the person in the video similar to how we had twitter verified checkmark.<p>On the flip side I can't wait for someone to build a product where I record a few conversations with my parents while they are alive, and then when they're gone, through chatGPT + vocalfakes, I can have a parent forever. Sure, I'll know it's not the real thing, but when you really miss them, it certainly can make the pain a little less.
Meh. We achieve a state of complete systemic disbelief, and alternate trust mechanisms develop.<p>[Clip of Abe Lincoln in RayBans saying "Don't believe everything you see and hear on the Internet" goes here]<p>Even the stuff that wasn't deepfaked was mostly bollocks anyway.
Was anyone else not particularly impressed by this? I haven't listened to much Steve Jobs stuff but it sounded just stilted enough for me to think something was up. I'm not sure if it was because I came into it with a skeptical mindset because of the article and context around it. It also may be do to the fact that I have been using screen reading software for about 30 years. The only people who I've heard more then synthetic speech may be my close family and I'm not sure about that. Is there anywhere that offers a test where you have to determine what is generated and what is not with random clips lacking background info?
On the flip side, if these deepfakes are so simple to generate, they will surely bombard us constantly throughout the day - radio, TV commercials, youtube videos, podcasts.<p>And people quickly become desensitized to that kind of thing. It could be the case that, after some initial "ramping up" period, these deepfakes are so cheap and abundant that no one falls for them.<p>When was the last time you got a call from a number you didn't recognize, and you actually picked it up? If you're like me, probably not in a long time, because you're aware it's almost certainly some scam/robo call.
If you know Casey Liss from ATP or his other podcasts, James Thompson made a deepfake of him with permission and it is extremely good.<p><a href="https://mastodon.social/@jamesthomson/110062947060928918" rel="nofollow">https://mastodon.social/@jamesthomson/110062947060928918</a><p>I had no idea things had gone this far.
It is difficult to predict what will happen, but nefarious use of something that powerful is a given.<p>Generated images and audio clips are still pretty easy to detect, we're not there yet, but close.<p>What I worry about is not the obvious usage and risks, but the second and third order effects, those we can't predict.<p>I will be interesting.
This feels so over-the-top. "A recording of Joe Biden forgetting his own name or what year it is, or Kamala Harris claiming to be running an abortion clinic?" Give me a break, you could already have done this with a high-quality voice impersonator -- cost is no real concern at that level anyway.<p>I think this is only really a risk at the low end -- people scamming others with fake references from quasi-celebrities. Not great, but overall feels pretty minor of a concern. We already allow the carriers to scam everyone in the US by allowing anyone to call your cell phone and try to tell you you're behind on your insurance, or that your computer has a virus. There's plenty of scams out there, if we really thought this was a problem, we'd care more about the existing ones.
When this topic has come up with family and friends, folks often say that they aren't worried (yet) because while a human can be fooled, they can't yet fool readily available forensic tools, and perhaps can't ever.<p>I can't speak to the veracity of that claim, but as the post points out, the past several years has shown us that it doesn't matter, not in the least.<p>The author goes on to say how it feels inevitable that we'll see a Bannon or Stone type use this technology to create fake scandals.<p>I'm more worried about the grass roots efforts. Crowdsourced conspiracies like QAnon. Now they'll have more capable tools to radicalize people.
Everyone's worried about people using this to generate things that people haven't said (The "grab em" comment by Trump) but inversely to this is that you can claim any REAL content about someone was AI generated by someone else.
I've already been seeing issues with AI art being used to fake Trump's arrest, people are genuinely confused and it even took me a second to realise I was being had
> It’s all fun and games in these demos, but this is inevitably going to be put to use by ratfuckers to create fake scandals in political campaigns...And it feels inevitable that a Roger Stone or Steve Bannon type will use this technology to commission, say, a recording of Joe Biden forgetting his own name...<p>The road to hell is paved with good intentions, and all politicians are going to use this for their political campaigns, not just one specific side. Both.<p>Let's not veer off the wider point and believe that only one side will use it for bad things. All politicians are liars, and no matter what side they are on, if it benefits their agenda to influence the electorate to gain power, they will use it; even if it is used for spreading lies or false and misleading claims.
This is one of those things where Balaji is ahead of the curve - the way to guarantee metadata (e.g. who the speaker is) is to do it cryptographically on-chain.<p><a href="https://twitter.com/balajis/status/1583495595737481217" rel="nofollow">https://twitter.com/balajis/status/1583495595737481217</a>