I feel that Google search has been slowly dying for many years now.<p>Where in the past I could find documentation, forum-posts, wikis - concise information - these days I find SEO-optimised marketing fluff, link-farms, clickbait, long-form articles with thousands of words that ultimately say nothing, political tirades that don't make sense for any non-U.S.-citizen, and ads - tons and tons of ads.<p>It would seem that it was Google ads and Google search's own algorithm - which directly led to this state of affairs.<p>And that's the main reason why asking some LLM for answers is so much more attractive than using search: the AI cuts through all that crap, and extracts the actual information. If it's not hallucinating, that is.<p>The web is no longer for humans or for finding information. It's for bots, crawlers, for the Google algorithm and for marketing.<p>Humans simply can no longer digest it - we need AI to generate something more concise, more to the point, more digestible. Without the marketing, fluff and bait and filler.
The funny thing is the site which should have been removed like the of Stackoverflow spam clones, or sites like Canva and Pinterest that make thousands of similar looking pages with slightly different heading are still allowed and rank on Google.<p>Also hate the top 10 pages whenever you search the best something, like best domain name registrar. I don't want to read a spam blog post with affiliate links, I wish google would show me the actual domain registrars instead (like chatgpt does when i ask it). Google has been gamed so badly and they have been doing nothing about it just because the spam blog posts contains their ads.<p>There are some tricks I learned on HN to use uBlock origin to filter these spam sites but Google really needs to fix this. There is only so much an adblocker can do to fix search. And right now all the useful content is getting blocked while the spam content is not only allowed but ranking on top of everything.
This is Google's inability to innovate.<p>Google stated goal was the "organize the world's information". Nevertheless, they didn't come up with Wikipedia, the highest quality curated human-readable information repository. They also didn't come up with ChatGPT, arguably the first LLM good enough that can perform non trivial tasks of data recall and organization.<p>Google had early success with search. Then decided to cement their lead by just throwing money at "all the smartest people"[1] so that their competitors would have difficulty hiring. This put Google in the situation where it's better to spend its resources fighting to keep the advantage that they have (by controlling the Internet infrastructure; android, chrome, email, etc) than innovating.<p>Google is dead man walking, unless something very substantial changes.<p>[1] "all the smartest people", read, "academically successful but rule-obeying, unimaginative, and risk-averse people".
Doing planetary science, I used to be in the habit of searching for image IDs on google. Usually particular orbital images of the moon which had been mentioned in a paper. There are of course specific archives to search for this stuff, but google used to give surprisingly good results. Now I get literally zero results when I search an image ID. What changed? Did they remove their index of the Planetary Data System? Did they remove the indexes of planetary science papers?
Google's death is deeper than just the corpus, their ranking algorithm is also deeply degrading.<p>A week ago, a search for < James Palmer manchester foreign policy > wouldn't return <a href="https://foreignpolicy.com/2017/05/23/i-love-manchester-but-please-stop-celebrating-my-hometown/" rel="nofollow noreferrer">https://foreignpolicy.com/2017/05/23/i-love-manchester-but-p...</a> on the first page while < James Palmer manchester site:foreignpolicy.com > would [1] [2]. This appears to be fixed as of today but this kind of kindergarten level search intent not being fulfilled would have been unthinkable for Google even just a few years ago.<p>[1] <a href="https://twitter.com/BeijingPalmer/status/1670904508191322112" rel="nofollow noreferrer">https://twitter.com/BeijingPalmer/status/1670904508191322112</a><p>[2] <a href="https://twitter.com/shalmanese/status/1670973880637493249" rel="nofollow noreferrer">https://twitter.com/shalmanese/status/1670973880637493249</a>
This is the Internet version of the tragedy of the commons: <a href="https://en.wikipedia.org/wiki/Tragedy_of_the_commons" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Tragedy_of_the_commons</a><p>So many people and groups want to profit with as little effort as possible that the commons (freely available data, open APIs, FLOSS software, etc.) is being overrun with little to no regard for the long term effects.<p>Data can be copied endlessly, but it has to be generated the first time and updated or else it becomes stale and its usefulness decays. If no one is willing or able to generate or update it, there is nothing good and the signal-to-noise ratio falls off sharply. Everything is noise and it sucks.
Many years ago, Google was very useful without Reddit. There is a reason why they are such a behemoth: they actually used to create very usable tools and services. People wanted to use them, because their products were good. Search, Gmail, Maps, Translate (etc)... all of them are (were) gems.<p>Don't know if they let search rot or they broke it intentionally. But it got broken regardless of Reddit. Reddit has a lot of info I guess, but it is far from being the only website on the internet. However seemingly google stopped indexing more than the top 150-200 websites on the internet (and even those results are often lacking the searched words).
> What if Wikipedia started charging or restricting API access?<p>Wikipedia has a downloadable data dump that would cost almost nothing to serve to Google, and it has an organisational mandate to make that data available. If they decide to charge for access to that I'm sure Google can afford it. Let's not throw around completely unrealistic hypotheticals.
The true meta problem here is the age old question of how to fund services, including search, on the web. Google, Twitter, Reddit, Gfycat are all immediate variants of the same problem, which does not have a technological solution. The naive idea that all these things can be free, especially when promoted by those in an ecosystem as propped up as it is by ad revenue, is ridiculous.<p>Someone has to pay for these things to be developed and operated and typically when they do they get to call the shots, which leads to the modern UX disaster.<p>At least now the VC situation means we have a lot less product-dumping-by-any-other-name intended to destroy the market for legitimate participants.
Google problem now is not Reddit nor Twitter. Google converted in just a brand with really awful results and full of AdWords campaigns where it is not clear what marketers are paying for. It seems like they need to return to the original Brin and Page paper [1]. Google continue to do amazing things (e.g. AlphaZero, Project Zero) may be in the same way as Xerox Parc did amazing things in the past but nothing really new cames from the organization. BTW Google Workspace (e.g. Sheets, Docs, etc) is a good competitor in the office space. Good business execution, the search in Google Drive is awful.<p>[1] <a href="https://research.google/pubs/pub334/" rel="nofollow noreferrer">https://research.google/pubs/pub334/</a>
Unlike the author, I'm not very worried about Wikipedia. Yes, they ask for donations, but that's not a sign that they're dying; donations are their main revenue source. That might be unusual among the web's top sites, but it's not in the grander scheme of things, i.e. there are lots of charities and nonprofits.<p>I didn't know that LLM training sets used Wikipedia. I would have thought that the CC-BY-SA license (on all text) would keep them away. It's not like Chatbot-4000 can cite all the Wikipedia articles used to train it, nor is Chatbot-4000 going to license all its output under a CC-BY-SA license (as required by the license terms).
Google's Search is dying because of Thousands of self-inflicted cuts.
A couple of years ago there were about 3 or clearly marked ads on the first page, followed by generally useful results.
Right now, I don't have the patience to scroll past the ads which are almost indistinguishable from useful links.
The cut that hurt the deepest for me is when negated search terms stopped working. You used to be able to get subsets of data, finding things without a homonym, but now the algorithm just gives you the most common results and the most paid ads.
I am starting to wonder more if Google search died or if the web did. You read a lot of comments about how in the past you could find a lot more information, but the reality is that no one is posting that information outside of the walled gardens. It's no wonder Google search sucks, if they don't have direct access to these walled gardens the information is locked behind the door...
They need to stop trying to so heavily curate what users are allowed to see in their searches. Since 2020, Google's attempts to operate as the internet's ideological censor have had a noticeable negative impact on the quality of their search results.
This post doesnt seem frontpage worthy, but I'll join in on the google bashing whenever I have a chance.<p>The other day I tried to google search 4chan, imgur and reddit came up before 4chan. heck 4chan wasnt even on the first page...<p>On a similar note, they are trying their best to hide wikipedia, now I have to specifically mention wiki most of the time.<p>I have the best website for a specific thing, I do well in quite a few SEO searches. However, the generic term, (which I'm still the best in), I am nowhere to be found. Blogspam and lower quality advice are all over the first 2 pages. Heck, there is actually terrible advice in the first 2 pages, like dis/misinformation.<p>The other day I was searching something I knew existed, I knew the page, and google would not give me the page. Ended up typing in the website.<p>Whatever is going on at google is in the major red flag territory. We need to perma fork android, we need to get off chromimum, find an alternative to Pixel. We are near the end times of google, they are going to be AOL taking advantage of old people and those who refuse to change their ways.
What changed?<p>Just an asumption here. But could it be the fault of the web rather than Google's?<p>Could it be that the web itself is being so flooded with SEO-ed crap content that even Google can't sort through it all?
I would really love to read a deep dive on why Google Search quality has declined so much. There's a lot of anecdotes but I haven't read much that sounds particularly authoritative.
In the TPUv4 paper it sounded like Google search moved over to some really crude AI embedding vector stuff. If you search "book of master system" it gives only results for "book of genesis" because it mixes up the biblical book of Genesis with the Sega Master System.<p>Searching any programming terms, like camel-case variable names from a system library without exact quotes, just gives random results about Katy Perry and stuff now.
Is the article's author worrying about Google specifically or search generally?<p>Because Google specifically has been degrading from self inflicted cuts for years.<p>Agreed that reddit going dark and machine generated "content" will only make it worse, but perhaps he should have talked about search generally then?
What is the motivation of anyone to make a website at this point? I don't see it.<p>I've not seen very much discussion about the issues these LLMs are going to cause, this article is sort of talking about it - twitter and wikipedia could restrict their content, but - what about the millions of webmasters who ultimately created what the web is today?<p>What is the motivation of anyone to really create content any more, apart from for actual social discussions on social media? LLMs will steal the content, and no one will read it.<p>I really don't know what this will bring in the future. Will online content fizzle out in 2-3-5 years and fresh content will just disappear? Will LLM only be good up to 2025 data?<p>This is a massive problem that changes the web as we know it, yet I don't see any discussion of this. Mark Andreeson bought it up for a sentence on lex Friedman, but that was it. He didn't know the answer.
Google has sucked as a company from the very beginning. Not necessarily the technology, but the founders and the culture.<p>How many of you remember back in 2005 or so, when the first lawsuit for delisting was brought against Google by a small business owner? The Google attorney showed up with a complete history of this person's searches and proceeded to demean him in court.<p>Probably not many of you, because Google buried the story. Today it can't be found (I can't anyway). It was written about in a highly critical book of Google a few years later.<p>Think Reddit's API story is bad? Google did the same thing back in 2010-2012, slowly tightening the clamps on anyone monitoring them to put them out of business. My company was saved by a call from a famous billionaire to Matt Cuts, after which our little problem promptly disappeared.
completely tangential, but this thread triggered a wave of nostalgia reminiscing how good the few years around 2010 were (or rather seemed to be through rose-tinted glasses) in tech. Among all else it was an era when it really felt that google did just great on almost all fronts, search included. Web in general was lot less app- and mobile-centric because those things were still pretty new things comparatively.
For the years I worked at Google I always wondered when they would update the “organize the worlds information” mission. The reason was simple: information eventually will reorganize itself around everyone trying to make their own money. Not money for Google. It felt like a mission that would only decrease in urgency not bc they are solving it but the world (internet) would get better at it over time
in the 90's one search engine after another went this way. Each new engine worked for a while then advertisers started to game it and it gradually became useless.<p>It actually impressive that google has managed to avoid this flaw for so long.<p>The only thing I am not sure of is have advertisers finally found ways to game the system or is google doing it to themselves having forgotten why we all started using it in the first place.<p>I reckon that chat GPT will end up like the Douglas Adams HHGTG dystopian elevators at some point too :)
> Google will lose results, site by site<p>The author is overlooking that Google can cut a deal with Twitter and Reddit and everybody else. That's for mutual benefit. It keeps driving traffic to them which is great PR and it keeps Google relevant.
I use DuckDuckGo now almost exclusively for search. It's the default in all my browsers. A few years ago Google still did better for some searches but now I don't feel that to be the case.
>> Large models are trained on public data scraped via API. Content-heavy sites are most likely to be disrupted (why post on StackOverflow?) by models trained on their own data. Naturally, they want to restrict access and either (1) sell the data or (2) train their own models. This restriction prevents (or complicates) Google’ automatic scraping of the data for Search (and probably for training models, too).<p>This is going to be the interesting part to me - one HUGE usage scenario for AI would be doing automated searches and distilling results from the web - if sites start preventing that then AI innovation could be stifled - OR we'll see a scenario where the average person is very limited with that their AI subscription can access. We could have a situation where e.g. Fandango prevents AI from searching its site to help people plan their movie outing, but instead has their own model that they'll charge for access. We could have models talking to each other, deals made between the owners of the models for access, and the average citizen uses a search engine optimized for monetization, that may have data that's months out of date but they can pay extra for the model that provides current data.
LLMs cause quite the paradigm shift.<p>Before, there was a very strong if not existential incentive to get indexed by scrapers, in particular search engines. Now the situation is flipped where there's a strong incentive to put up ever higher walls, in particular for LLM scrapers.<p>The difference is obvious: LLMs don't give back. They just rob your life's work and monetize it locally. Why would a platform or its users open up to that?
>What if Wikipedia started charging or restricting API access?<p>Wikipedia does charge Google.<p><a href="https://timesofindia.indiatimes.com/business/international-business/google-agrees-to-pay-for-wikipedia-content/articleshow/92400157.cms" rel="nofollow noreferrer">https://timesofindia.indiatimes.com/business/international-b...</a>
In the very early days of google, you could search for a simple problem, and the result would be on the search page. No point in clicking. I guess a few people clicked the other answers, so they rose to the top.<p>So now it is very rare to get the answer on the search page, because it makes no money for google.
While that's very true, I actually like searxng for that. And thanks to Googles slow death and the (it feels like that at least) Google's slow death, we'll see more of that and improvement to existing projects in the future, imho.
I think the article is mostly incorrect. The deep web has been way larger than the open Internet for a very long time. I see other problems with Google search like it does not work as well as it did some years ago.
>Large models are trained on public data scraped via API. Content-heavy sites are most likely to be disrupted (why post on StackOverflow?) by models trained on their own data. Naturally, they want to restrict access and either (1) sell the data or (2) train their own models. This restriction prevents (or complicates) Google’ automatic scraping of the data for Search (and probably for training models, too).<p>I've seen the other side of the coin, far too frequently. Many websites allow Googlebot to index for free, but once a person clicks on Google serach results, they ask for money.
Since Google collects a lot of personal data from each user, search results for the same query are different and personalized. This seems like a perfect solution - beside the tracking.
For example, if someone searches for ‘python’, Google can use the context to show results about either a snake or a programming language. But what happens if someone wants to learn to program in Python as first programming language and has previously searched for a frog and visited the zoo last week?
In my opinion, the information that Google collects is useless and doesn’t enhance the search results.
Y’all know there are other search engines, right? Yandex is becoming increasingly more useful for politically incorrect content and DMCA violating content. Bing has always been the best for porn. DuckDuckGo is great for many things as well.<p>SEO becomes a lot more difficult to pull off when there isn’t one search algorithm used by 99% of users.
If large websites like Reddit, Twitter, Wikipedia ban Google Search, maybe they die, too along with Google search.<p>Which means we return to late '90s and early 2000s situation when there were no behemoths hoarding data and knowledge and there were many specialized websites. I am happy with checking tech news on HN, checking stuff about cars on a car website, read about photography on a photography website and so on, instead of searching countless Facebook groups and subreddits.
From an DTC advertiser’s perspective, Google’s made so many hostile decisions of late: selling remnant space through Performance Max, ditching UA for GA4 without porting data, releasing Bard… Just failure after failure while the end consumer search experience just goes down the tubes.<p>Lots of smart people making very bad products that get in the way of other smart people making good products. It’s shocking to look at the wreckage of their decades of poor decision-making.
Reports of Google's death seem greatly exaggerated. The still have 95% share on mobile and 84% on desktop, the latter because Microsoft can make Windows try to force Bing upon you. Google probably have like 99% of the profits in search.<p>The idea of sites making their data inaccessible is not new. Experts Exchange is still out there siloing the answers but Stack Overflow has all the market share though being open. Google will be fine.
Seems like a lot of confirmation bias on this thread, but I've found google search to be fine for most of the search queries I've used. The real threat seems to be chatGPT which has been a great timesaver for information that needs to be customized
What google is no doubt trying to do right now as their highest priority is train their AI bot to sell products to you as part of its responses. I guarantee they are putting 100 times as much effort into that as they are into AI safety.
What if an ISP decides to block ads and spam at DNS level? That theoretically can clean up the Internet a bit.<p>It would be good not only for the customers, but also for the ISP since it has less junk to travel through its network.
Already everyone and their brother rate limits everything except googlebot (or everything except googlebot and bing)-- it's already a big part of what creates a search lock-in for google.
Over 50% of searches don't end up to being a click to a site which means less and less traffic is gained from Google who still monetizes the data they have scraped and the answer they give to users who search.<p>For content creators this has meant they need to find more and most importantly more agressive ways to monetize their content since less and less traffic means you need to show more ads to keep the money flowing. If google lowered the amount of answers it actually gives and would lead people to click on the sites more there would be less ads and users would be happier as well and there would be less paywalls as well.