I've received a fair number of these, dating back to late last year and continuing to this day. I'm not sure when Panda came in, but I wouldn't say they've increased or decreased since they started.<p>The sites I run are all based around user generated content, and the links are all genuine instances of people sharing information and linking in the process. None of it is backlinks provided to benefit some third party, and we've never participated in link swapping or anything like that.<p>We think the link removal requests are dodgy.<p>Suspicion is that by and large the requests do not come from the companies actually associated with the linked site. And that when challenged those senders of the request have then squirmed, apologised and claimed the request was sent by accident.<p>Example: <a href="http://pastebin.com/P9tsWL0x" rel="nofollow">http://pastebin.com/P9tsWL0x</a><p>Basically: I believe that a fair number of these requests are from SEO companies attempting to get competitor sites a lower pagerank so that their properties fare better.<p>Only a minority of requests seem to come from the companies linked, and in part I wonder whether other SEOs are cargo-culting the phenomena by copying it without understanding it.<p>I forwarded an example to Matt Cutts a while ago thinking that this whole area feels spammy and dodgy, but I understand he's busy and must get a lot of mail.<p>I've not removed a single link as a result of these bizarre notices.
Gad, I am so sick of SEO. Not Jacques article, but SEO in general.<p>Google owns the game. They run the game on a computer. Ergo, if you want people naturally coming from Google, you must do things its computer likes.<p>Only they won't tell you that. Instead, they'll offer platitudes like "write good content and the users will come" when we all know you could write great content until the cows come home and if nobody links to you, you ain't getting no traffic.<p>And I think it's unfair to call all these guys leeches, miscreants, or whatever. I don't like a lot of the things they do, but I also respect the fact that I live in a first world country. I have a good way of living. If I were terribly impoverished and only had to spam a lot to feed my family? I'd do it. We assume everybody else on the web lives the same lives that we do. We also are getting this quasi-religious thing going on where Google must return what I want at the top of the search results. If it does not, somebody has sinned. I'm not drinking that cool-aid.<p>I'm with Jacques on the solution: a new protocol and the elimination of single-points-of-failure. This thing where Google keeps updating it's algorithm and tens of thousands of people keep gaming the system has to stop. It's not healthy behavior either for Google or for the spammers. And it's destroying the web.<p>Sidebar: you know, if you think about it, with all the walled gardens and vendors refusing common protocols and such, the web itself is under attack from multiple angles.
> <i>In the scientific world there are no spammers and there is no direct commercial advantage to creating a lot of nonsense paper that cite your own paper, also there is some oversight in the world of science and the people there have a reasonably high level of integrity.</i><p>Um...<i>what</i>? If it were anyone but the OP, who always writes with a lot of thoughtfulness and insight, I would've assumed the graf above is satire. Academic discovery and citation is very much being gamed; the only reason why we don't notice it more is because the academics don't have the same tools and infrastructure that web spammers do and, also, the world of academic research is not something the average person outside of academia closely parses.
This is a nice description of the hell of modern WWW.<p>But things are worse than that! With few exceptions (Stack Exchange and Wikipedia are notable) most searches will return sites that have been SEOd.<p>My Google search for [spectacles cases] returns these two sites on the first page:<p>(<a href="http://www.spectaclecases.co.uk/" rel="nofollow">http://www.spectaclecases.co.uk/</a>)<p>(<a href="http://www.aglassescase.co.uk/" rel="nofollow">http://www.aglassescase.co.uk/</a>)<p>> <i>Welcome to SpectacleCases.co.uk. You will find a wide selection of Glasses Cases / Spectacle Cases / Sunglass Cases.</i><p>> <i>Made from leather, fabric, metal or plastic finished to a very high quality. Hard and soft cases for spectacles, glasses and sunglasses. We also have a good selection of cheap glasses cases which offer great protection for your glasses.</i><p>> <i>Welcome to AGlassesCase.co.uk. The one stop shop for Glasses Cases, Spectacle Cases and Sunglass Cases. We also sell a number of Glasses Cloths</i><p>> <i>Made from a range of quality materials including leather, fabric, metal and plastic all finished to a very high standard. We sell hard and soft cases for spectacles, glasses and sunglasses. We also have a good selection of cheap glasses cases which offer great protection for your glasses.</i><p>These two different sites are the same company.<p>Maybe they're a great place to buy spectacles cases from, but it's vaguely upsetting that Google can create freakin' awesome stuff (A self driving car! It is actually wonderful and futuristic) yet can't fix this stuff. Obviously, Google are not to blame, and really the problem is with sleazy SEO and odd behaviours by vendors.
> In the scientific world there are no spammers and there is no direct commercial advantage to creating a lot of nonsense paper that cite your own paper<p>Yet. Google's on it though.<p>>> The launch of Google Scholar Citations and Google Scholar Metrics may provoke a revolution in the research evaluation field as it places within every researchers reach tools that allow bibliometric measuring. In order to alert the research community over how easily one can manipulate the data and bibliometric indicators offered by Google s products we present an experiment in which we manipulate the Google Citations profiles of a research group through the creation of false documents that cite their documents, and consequently, the journals in which they have published modifying their H index. For this purpose we created six documents authored by a faked author and we uploaded them to a researcher s personal website under the University of Granadas domain. The result of the experiment meant an increase of 774 citations in 129 papers (six citations per paper) increasing the authors and journals H index. We analyse the malicious effect this type of practices can cause to Google Scholar Citations and Google Scholar Metrics. Finally, we conclude with several deliberations over the effects these malpractices may have and the lack of control tools these tools offer.<p><a href="http://arxiv.org/abs/1212.0638" rel="nofollow">http://arxiv.org/abs/1212.0638</a>
"Destruction of the Web" is a bit hyperbolic. I tried the SEO game for the interest of my clients, I've come to realize that it's a better long term strategy to simply focus on good, relevant content and let nature take it's course. When you're publishing content targeted at a certain audience, spammy visits to your website just increase your bandwidth cost without much benefit.<p>Granted there is inherent benefit in coming in on top of a google search, but time and time again, I've seen good content naturally dominate and stay on top. Occasionally some black had spammer comes out on top.<p>And truth of the matter is, I think it's google/<insert search engine here>'s job to figure out a way to discern good content from spam.<p>Google Dominated early on, b/c they were able to parse thru a lot of the garbage and find you what you were looking for.
There will always be spammers. And if this whole arms race thing is true, I think the spammers are likely to hit a ceiling before Google or another search engine is.<p>As a programmer, my initial instinct was to write code to counter what Google's algorithm would expect. Then, I thought, if I'm going to put in that much work, why the hell not build a better search engine myself?<p>I believe that if Google doesn't do a great job of getting the best possible results, not only do they face a treat from other giants like bing, but also from crafty programmers who may be writing black hat seo crap now and have the epiphany to try to build a better search engine (yes... I know... I think PG has a real point with the search engine thing)
I'm "relatively" new to the web. I started using it around 2004. To me, the only way to browse the web goes through Google. I don't think there's a single day I spent in front of the computer without me hitting Google at one point. I even use Google search when I'm specifically targeting Wikipedia or StackOverflow.<p>For the people who got introduced to the web before me, how was "web browsing" done in the earlier decade of the Web? I'm assuming Google is not the first search engine available, but I'm pretty sure search engines were not the only way to go around.<p>I understand the concept behind the web, "a globe spanning network of computers linked by hyperlinks pointing to useful information". But was it as simple as that? You only had access to addresses you knew or links available on these pages? Where did you go to find interesting websites or how would you look for specific information (like, for instance, how would you research the working internals of a car engine for a school project?)
What about something like a disavow.txt file that site owners could use to list domains or URLs with unwanted inbound links. It would be similar to the Google Disavow Links tool, but more open and standardized.<p>We could write a simple spec around it. I'd see it being similar to robots.txt in form and function... easy for a human to write, easy for a search engine to parse, easy to generate programmatically if you need to scale it up.<p>Also, it avoids the black-hat SEO problem since only folks with access to the site could control the content of the disavow.txt file.<p>Thoughts?
Google will allow you to "disavow" links from spammy sites that are pointing to yours, so there's no real reason to ask people to remove them.<p>Another problem that seems to be on the rise is corporate shills. In the past these were easy enough to spot, they were people who would go to blogs/forums and spam them with blatant attempts at promotion.<p>Now they seem to be getting much better, they will come to a site like HN and read the comments that people are posting and will write their own comments in a similar style, not embed links and fly under the radar as legitimate users.<p>There are marketplaces for selling aged accounts for these purposes. This makes me very skeptical of any type of product recommendation from a website user.
"Destruction of the Web"? This is really going to destroy the web? Though it is an issue, I hardly think that this is going to destroy anything. May inconvenience some things, cause confusion about when to link (and when to unlink) in the short term until an equilibrium is reached and the next SEO change comes along. But I think that this title itself is Link Bait (something else that might be "destroying the web" by this lose definition of destruction).
Similar article and discussion from a few months ago:<p>A painful tale of SEO, spam and Google's role in all<p><a href="http://scriptogr.am/slaven/post/a-painful-tale-of-seo-spam-and-googles-role-in-all" rel="nofollow">http://scriptogr.am/slaven/post/a-painful-tale-of-seo-spam-a...</a><p><a href="https://news.ycombinator.com/item?id=5296005" rel="nofollow">https://news.ycombinator.com/item?id=5296005</a>
Just like I don't put all my API eggs in one big corporation's basket like the so-called "ecosystem" platform of Apple, Microsoft, et al.<p>I don't put all of my search eggs in the google basket.<p>I think that google search was great for about a decade. In the past couple of years, I am starting to have serious doubts about Google's search results. No I don't want the result tailored to me, or whatever IP address that I am searching from. And, I suspect there are lots of sites that are relevant that are not showing up. I also suspect that someone can do search better than Google.<p>There are other dominant and not so dominant search engines, please use them ...<p><a href="https://duckduckgo.com/" rel="nofollow">https://duckduckgo.com/</a>
<a href="http://www.yandex.com/" rel="nofollow">http://www.yandex.com/</a>
<a href="http://www.baidu.com/" rel="nofollow">http://www.baidu.com/</a>
<a href="http://www.bing.com/" rel="nofollow">http://www.bing.com/</a>
<a href="http://gigablast.com/" rel="nofollow">http://gigablast.com/</a>
I've never understood the concept of black-hat SEO.<p>If your site isn't worthwhile enough to be on the front page of relevant search results, why would you pay somebody to set up link farms and comment spam <i>et al</i> that <i>might</i> get your site up the rankings; instead of just paying Google to put your site as an ad when people run the relevant searches?<p>Either way it's going to cost you money, but one is guaranteed to work & have no negative repercussions; the other isn't. Is it just about price, or do the SEO guys have really good marketing, or what?
Good article, but the author takes a somewhat generous view of academic citations. There are spammers in academia -- e.g. editors and reviewers who block rivals from publishing or who demand citations of their own work in revisions.
Destruction just seems like such a harsh word.<p>Any time you measure something, you impact what you measure. Without being pedantic about the Physics, look around. If you measure when people get in and our of work, they will optimize around it, and other things may suffer. If you measure defects in code, people will optimize it.<p>But back to the web... Search by definition is a measurement or prediction of usefulness. It will definitely impact what gets searched. But that collateral damage can be minimized. And it some cases it can improve the web experience, though I wouldn't bank on it. I certainly don't view the Web as a pristine wilderness being destroyed.
Sorry, but for me this article is a bit BS, because what the author describes as the destruction of the web is for me the healthy and eternal balance between good and evil, all over the society.<p>One one side, systems like Google, facebook... compete to be the best spam filters, on the other side, cheaters try to fight back by using more advanced spam techniques.<p>At the end of the day, the Web keeps improving and tons of new applications flourish. It also makes it harder to spammers who can then decide to go white hat.<p>I must admit the quality of Google's SERPs is going down since Panda, especially for long tail keyword phrases, but look, Hacker News is one good example of alternative to Google and Facebook, and the web is not just about Google...<p>I also believe black hats are the best friends of Google, because they push the level of anti-spam higher, where newer or smaller search engines can't compete because of its lack of history.
The way to solve this problem that's been discussed before is to require authenticated identity in the protocol layer, so every packet can be traced to a real, live person.<p>But then that breeds new problems:<p>1) The NSA might like the ability to tie every packet to a person, but privacy and anonymity are generally good things.<p>2) Just because you have a reliable way to track/measure "real" reputation/authority/trust, will that stop people from abusing it? Did the offline version of this stop Paula Deen from building an empire on unhealthy eating only to later reveal her own diet gave her diabetes? No.<p>Human nature is driving a fair bit of this stuff, and has nothing to do with Google, the web, protocols, or spam. We always try to eliminate the "flawed human" from systems, and it never works.<p>3) The fact that reputation/authority/trust is unreliable might actually be a feature not a bug. For one thing, it allows some dude with no social capital to get a toehold and get his stuff in front of users. Generally Google allows this to happen, and if the content sucks, it falls away. I don't mind a bit of spam if it's the price for more diversity and opportunity for people outside the "lucky sperm club" to rise.<p>Overall, I don't buy this "destruction of the web" stuff. If anything, Google has made both the web AND search are way better today. It's possible that Google's anti-spam strategies will hit a point of diminishing marginal return, and the spammers will catch up and the balance will swing in their direction again, but so far that's not been the trend.<p>I think the equilibrium we're seeing is that Google allows a very small amount of spam tactics spam to work for a while, but they use other signals that keeps that stuff from getting major traffic (e.g. eHow). So gaming the system can get you in search rankings, but if you suck, you won't stay there.<p>But more importantly, I don't want to trade off privacy and anonymity to eliminate what amounts to a very small amount of spam.
Chris Coyier talked about it last year:
<a href="http://chriscoyier.net/2012/08/17/sweet-spammer-justice/" rel="nofollow">http://chriscoyier.net/2012/08/17/sweet-spammer-justice/</a>
><i>I believe this will have to come in the form of a reboot, a protocol designed from the ground up to combat these issues and a way to search the web that makes it infeasible for a single party to control such a large volume of traffic</i><p>For all intents and purposes, Google <i>is</i> the Internet lobby that has the ear of legislators. They're not gonna stand for any kind of protocol reboot that would destroy their fundamental and wonderfully lucrative business model.
What Google seems to ultimately be doing with its Pandas and other attempts at stopping search spam is TEACHING businesses that the only safe way forward is great content and white hat practices. Sure, you might temporarily get some advantages with search spam but come next Panda and you might be totally screwed. Better play it safe and play nice with Google.
So if I refer my link in this discussion will that be spamming or is it allowed to be referred as I am trying to let people know some SEO guidelines which I feel like sharing here.<p><a href="http://webmasterfacts.com/google-webmaster-guidelines-seo/" rel="nofollow">http://webmasterfacts.com/google-webmaster-guidelines-seo/</a><p>But since the original Google page already explains much, I think there should be no provision of building pages with "verified" keywords as well. Now that official T&C directly states that one cannot take control of the content but one can sure take control of the spam words. What if I had an added service to the link above to some SEO company?<p>Would that have been ethical link building strategy? I don't know much about how things run around the Google cubicles but I know one thing for sure - No one is going to tell that for the next 100 years.<p>@Matt: Keep the fight on
@rest: Keep the questions coming.<p>Good Luck.
That was a long, windy read for … no clear argument and a ton of begged questions. Spammers have been trying to get backlinks since the 90s – no evidence that this has suddenly become unmanageable. There's a flat claim that nobody makes legitimate backlinks any more, which is both completely unsupported and transparently wrong.<p>“Destruction of the Web” is a bold title for what appears to be a minor kerfluffle affecting only web marketing types trying to scam search engines. If you produce decent content or something else useful, you can ignore it and carry on: back links will take care of themselves as they have for the last couple decades.<p>Put another way: the link-bait title got me to click but the anemic post left me less likely to come back and uninterested in sharing. Fix that if you want better search engine rankings.
When I started getting emails from companies asking for the spam-links from my sites to be removed I assumed it was a <i>good</i> sign. If they're cleaning up their spam-links then presumably Google have found a way to make those spam-links not pay.<p>Hopefully fewer spam-links and robot-comments, no?
One option would be to "avow" links rather than "disavow". Start with the assumption that all links are valueless and untrustworthy and only count those which the webmaster has marked as good.
At the cost of result quality, one could exploit social networks as a possible solution: promote the pages that your neighbors visit for an extended period of time (and thus deem to be of high value) by a factor corresponding to said neighbor's distance from you. If the queries are slightly randomized such that their order differs per person and per session, then new possible results could tested.