The Great Search Engine Hypocrisy

The clearest, most consistent opinion voiced by the major search engines to webmasters is this…

It is a fairly simple proposition that has been applied to the vast majority of grey and black hat search technologies: cloaking, ip-delivery, doorway pages, and keyword stuffing. Essentially every type of search spam, even link spam tangentially, relates to this premise.

So let’s look at the latest incarnations of search engine hypocrisy waged by both Google and Yahoo.

1. The NoFollow Tag.

Originally, webmasters were held accountable for each and every outbound link. The industry even developed a name for the unfavorable sites which webmasters should avoid linking: “bad neighborhoods”. While websites were a target for spammers trying to promote “bad neighborhood” sites (as they still are today, despite growing nofollow use), the principle remained in tact: show the bots what you show your users.

The NoFollow tag grossly violated this long held principle: empowering the webmaster to tag content that will be openly displayed to users without regard, but flagged for the bots as untrustworthy. Rather than preventing sites which display untrustworthy/unsavory links from being promoted via high rankings, these sites are now exempted by using a tag that the unsuspecting site visitor never sees. There is little reason for a user to believe that the results in Google will lead to a page with trustworthy links.

Subsequently, what has occurred is exactly what the search marketing industry had predicted. Massive user-generated sites have begun using NoFollow well beyond its original intents, marking all external links using the rel=nofollow tag.

2. The NoContent Tag.

Yahoo has recently proposed a NoContent Tag. It is the logical extension of NoFollow. If we are going to allow webmasters to present links in a different way to the bots than to users, why not allow them to do the same with content?

Some have responded that this is not cloaking because “Yahoo will still see your entire page”. However, “Yahoo! will not use the terms contained in that section as information for finding the page or for the abstract of that page in search results.” And even though Yahoo! explains that it is not cloaking, there really is no other word for it.

In actuality, it is cloaking in reverse. Rather than showing the bots extra content, webmasters show the users extra content. While perhaps not to the level of sneaky redirecting practices and IP Delivery, the conclusion is straight-forward.

Webmasters using NoFollow and Robots-NoContent are sending conflicting messages about the trustworthiness of their content to the bots and to the users. NoFollow, and soon NoContent, is being abused as a way for webmasters to get away with showing untrustworthy content.

The Solution.

The crazy thing about it all is that solutions to link spam do exist which do not violate this principle. Rather than promoting or developing this type of technology, the Search Engines create quick fixes that hurt the User. A simple combination of the tools below is more than enough.

1. Akismet
2. LinkSleeve
3. Captchas
4. Admin Moderation
5. Community Moderation

We use Akismet + Linksleeve + Admin Moderation. Effective as of this morning, TheGoogleCache dropped the nofollow tag from all of its comments. Why should we use it? We hand-approve every comment that makes it onto our pages. If we don’t trust the link, we won’t post the comment.