Evidence-Based Search Engine Optimization

Over the last several years, our industry has produced myriad theories of how firms or consultants should go about SEO, each with their own set of shortcomings. Through these flawed systems, a new overarching theory of how our industry should behave has budded. First, let’s identify some of those flawed systems.

1. Ethical SEO: A do no harm type of solution that strictly adheres to the guidelines of major search engines.
   • Greatly inhibits the ability of a site to perform in the most competitive industries
   • Follows a subjective set of ideals (whose ethics is right? mine is)
   • Forces SEO’s to react drastically to position changes (such as Google coming out against reciprocal links or paid links)
2. Performance-Based SEO: An elusive goal which attempts to create a pay-for-performance scheme.
   • Encourages consultants to use unscrupulous, dangerous tactics
   • Discourages consultants from using long-term solutions which can take longer to implement
   • Difficult to choose appropriate metrics: conversions, traffic, or rankings? Each has their own shortcoming.
   • Ignores that SEO is a competition between sites and that performance cannot be guaranteed.

To What Should We Aspire?

1. A system that first-and-foremost serves the client interest
2. Our techniques should be experimentally and statistically verifiable.
3. Our techniques, not necessarily our tools, should be open.

Where We Are

What has come about, although not yet cohesively, is a push towards what I would like to call evidence-based search engine optimization or ebSEO. This style takes its name from the growing evidence-based medicine movement which seeks to “apply more uniformly the standards of evidence gained from the scientific method to certain aspects of medical practice“.

Unfortunately, we are still in the most nascent stage of ebSEO. Some of our best work to be produced, while extremely valuable, is still based heavily on talking-head opinions and their own research, rather than clear, finite experimentation with set hypotheses. For example, in my opinion the most valuable piece of annual research, SEOMoz’s search ranking factors is simply a compendium of expert opinions. While it will continue to be valuable, its recommendations are not ground directly in the scientific method. Even the more robust research that we see, such as Fortune Interactive’s study on the importance of certain ranking factors is based on a purely observational/statistical method which, while arguably still part of the body of techniques known as the scientific method, should be replicated in an experimental format to determine their accuracy and ability to be generalized across the spectrum of keyphrases.

Making the Transition

The first step in finally turning the corner towards ebSEO is that we need to develop a culture of data. I am not saying that we need to get to the level of every person understanding the ins-and-outs of matrix decomposition and latent semantic indexing. This is a culture of pseudo-academic jargon that bears little-to-no impact on how we actually go about the business of SEO day-to-day. On the contrary, I think that we have jumped the gun to begin looking at these emerging technologies when we still have not fully answered more simple questions. We don’t need keyword popularity and simple Keyword Effectiveness Indexes, we need comprehensive data. When a customer asks you whether or not they need to target a particular keyword, we need to stop giving off-the-cuff answers, we need to politiely respond “let me cruch the numbers on that.”

The second step is using experimentation to determine how to use that data. Google’s webmaster tools and sitemaps program make it very easy to perform ranking experimentation. We can control every variable that may impact a site’s ranking, vary them 1 at a time, and test them in Google using nonsensical words. From that point, we can use the data above to make accurate models and predictions. Some examples are here.

The final step is making this data open. If results like these are not made available to the general public, we continue to run into the problem of clients being unable to verify whether or not the work done on their behalf is truly evidence-based.

How We Make Our Money

If we openly distribute our methodologies, if we band together to determine solutions that anyone can follow, how do we maintain a commercial, profitable industry? More importantly, how do we distribute these methodologies without endangering their effectiveness?

The true value that an SEO will offer his or her client is not simply the knowledge of how to manipulate the search results – it is the ability to turn that knowledge into real results using the tools of the trade effectively and efficiently. There is hardly a single industry whose techniques have managed to evade publication on the internet. Yet, you still hire the plumber, still eat dinner out, and don’t change your own oil.

Some Experiements to Be Run: Any Takers?

1. More Important, Anchor Text Link or Title Tag? Get 1 site indexed with the nonsensical keyword in title tag using Google Sitemaps. Get 2nd site indexed using Google Sitemaps and use anchor-text link to point to 3rd site. Determine whether 1st site or 3rd site ranks higher.
2. Importance of Keyword Location in Title? Get 3 sites indexed in Google via Google Sitemaps, each with nonsensical word in title, but in different locations (“huygj word word”, “word huygj word”, “word word huygk”). See which ranks highest.
3. Many, many more… there are, obviously, hundreds of experiments to be run.