I'm getting a couple of emails daily from bots that are clearly part of fake companies with fake websites that list fake employees, and trying to sell me fake services that I'll never get if I was to pay for.<p>I'm not surprised by this. It's par for the course for the times we live in, and I'm well aware that many people might really believe this stuff.<p>My only question is why the garbage factory hasn't gone into overdrive faster, and how long will it take until there's so much garbage online that the Internet and all forms of digital communication essentially break?<p>Its not like we haven't had spam and scams for decades (still waiting on my Nigerian prince). But generative AI allows it to be so much more varied and targeted and in greater volume.
> My only question is why the garbage factory hasn't gone into overdrive faster, and how long will it take until there's so much garbage online that the Internet and all forms of digital communication essentially break?<p>"Why" is filters and price. Spam filters, CAPCHAs, tarpits, shadow-banning — there's been inauthentic behaviour since "telegram" was a thing sent from an office and printed on paper, rather than an app, and this has led to an arms race from all the people who don't want that. When I was at university (early 2000s), I vaguely recall they claimed that for every real email on our systems, we had about 100 spam messages, most of which were silently deleted and didn't even get as far as our spam folder.<p>"How long until" depends on when the price of a fake that's good enough to fool your automatic filters, is low enough that the users are getting so much rubbish that they can't be bothered to use the thing any more — given how many ads most people seem willing to tolerate, this may be as high as 1 in 3 messages being fake.<p>It's an arms race, so the exact date is not predictable.
1) it's expensive to have AI output millions of emails, assuming it's tailored somehow and not just a template
2) AI is already in place to detect spam, so it's AI vs AI and it's probably more expensive on the attacker side.
I feel like there is a market effect where people gravitate away from fake content. There are a lot of subreddits where people post text stories. Ten years ago I think these were more authentic, but today they are overwhelmed with fake stories that people recognize them as fake so growth of those areas stops.<p>I think winners of online content will be platforms that can keep slop from clogging your recommendations. Perhaps requiring accounts to register with a SIM card would be cost effective enough from keeping farmers from making thousands of account. I think if a couple fake TikToks enter your feed people will find it a novelty, but if people are feeling constantly duped the platform will suffer.<p>Of all my AI concerns this is pretty far down on my list, though I'm nervous for the weird future.