New Research: Keyword domains still work, despite what Google says

Keyword domain names were once thought of as a golden ticket to top rankings, but the general consensus in the SEO community seems to be that their power has diminished in the wake of Google’s Exact Match Domain update.

I have a long history with keyword domains, having been an early promoter of the marketing value of a good domain name in my previous role as CEO of domain marketplace Sedo.com. So I decided keyword domains would be the perfect way to kick off CanIRank’s new series of original research examining the impact and interplay of various ranking factors. My goal with this series is to use the unique capabilities of CanIRank’s software to provide some hard data on ranking factors that goes beyond the typical “X is correlated with high rankings, but Y is not” analysis which in my opinion is based upon an overly simplistic view of modern ranking algorithms.

 

CONTENTS

 

OBJECTIVE

What, if any, role do keyword domains play in SEO today? Is it still worth spending tens of thousands of dollars for a category-defining domain name if it’s not an automatic ticket the front of the line?

Specifically, we want to examine:

  • Are keyword domain names still beneficial?
  • How much do they help?
  • In what ways do they help?
  • What is that help worth, in dollar terms?

In order to adequately answer these questions, I’ll be introducing a new approach to SEO Ranking Factor analysis which uses CanIRank’s competitive analysis software to place ranking factors within their real-world context rather than examining each factor in isolation as previous studies have done.

 

HISTORICAL BACKGROUND

In the olden days (which in the SEO industry means only a few years back), all it took was a few pages of content and a decent link or two and keyword domain names would practically leap to the top of the search engine rankings. Google and Bing loved them alike; not surprisingly SEOs bought up exact-match domains by the truckload to take advantage of their ranking magic.

Then, in 2011 Google’s Matt Cutts began to hint that keyword domains may be working a little too well, and in late 2012 Google announced the Exact Match Domain update specifically targeting low-quality sites built on keyword domain names.

Although hard evidence on the impact of Google’s updates was limited to simplistic correlation studies and anecdotal reports, consensus amongst SEO experts swung quickly to the belief that keyword domains, or at least exact-match keyword domains, were no longer a significant benefit to search engine rankings.

This study is an attempt to elucidate the many shades of gray that exist between “yes, they do work”, and “no, they don’t work”.

 

A NEW APPROACH TO RANKING FACTOR ANALYSIS

Previous studies examining the impact of a single ranking factor generally look at the correlation between high rankings and that single factor. For example, do pages with more links rank higher than pages with fewer links? Do websites with domains matching a keyword rank higher for that keyword?

While correlations are certainly interesting, they imply a linear relationship between individual factors and search engine rankings. An apt metaphor for this world may be rankings of professional sports teams. More is always better! Just add up all your “points” on the various ranking factors and rank according to the final score!

However, modern machine learning like that which powers search engines today is much more complex. Sure, there may be points. But in my opinion they’re more likely to come from a series of intertwining classifiers trying to determine e.g., what type of site we’re dealing with, how quality the page is, and how authoritative the site is in that topic area, and whether or not the page is spam. Each individual factor is simply another piece to the puzzle. An apt metaphor is a blind man trying to identify an elephant: How large is it? Does it have a tail? Does it have a trunk?

In this more complicated non-linear world, each individual ranking factor only means something when put in context with other factors: more links are better, unless they’re all from low-quality sites with the anchor text “cheap payday loans”. Thousands of auto-generated thin content pages are bad, unless it’s a shopping keyword, in which case it’s probably exactly what the user is looking for: a major ecommerce retailer.

CanIRank’s unique ability to measure the relative impact of different ranking factors gives us an opportunity to study the impact of keyword domains in a much more nuanced manner than simple correlation studies.

 

STUDY METHODOLOGY

Using CanIRank’s SEO Competitive Analysis software, I collected data on 200 ranking factors for just over 10,000 URLs ranking in the top 10 Google results for a randomly selected pool of commercially-relevant keywords.

The data collected by CanIRank’s software mostly falls into one of 7 areas, which we call the Major Ranking Factors (since we’re data geeks and hence not very creative):

  • Page Relevancy: relevancy of the specific page to the keyword
  • Website Relevancy: relevancy of the entire website to the keyword
  • Page External Relevancy: relevancy and strength of links to the specific URL
  • Website External Relevancy: relevancy and strength of links to the entire website
  • Page Strength: trust and authority of the specific URL
  • Website Strength: trust and authority of the entire website
  • Social: popularity of the page on social networks such as Facebook, Twitter, Google+, LinkedIn, etc.

Examples of some of the data points we collect are:

  • keyword usage in title, body, headings, img alt tags, and other HTML tags
  • related term usage in each of the above areas
  • content length
  • # indexed pages
  • relevancy of indexed pages to keyword topic
  • relevancy of home page
  • relevancy of domain name
  • # external and internal links to page and website
  • MozRank, MozTrust
  • Page Authority and Website Authority
  • Facebook shares/ likes/ comments, tweets, +1s
  • Others that we’re holding back to protect CanIRank’s secret sauce ;)

We then divided the URLs into those using keyword domains (which we defined as root domains exactly matching the keyword or mostly matching the keyword), and brandable domains (which included everything else, although not all of them are necessarily brand names). For example, the keyword “tahoe lodging” included TahoeLodging.com and TahoeMountainLodging.com as keyword domains, and TripAdvisor.com, Expedia.com, Kayak.com, and others as brandable domains.

 

KEY FINDINGS

Using each of these data points, we can put together a fairly nuanced picture of how keyword domains perform differently than non-keyword domains.

  • Keyword domains rank on average 11% higher than brandable domains
  • Brandable domains needed an average of 40,000 more links to hit #1, and 35,000 more links to reach the Top 10
  • Brandable domains needed 69% higher Domain Authority and 22% higher Page Authority to rank in the Top 10
  • Keyword domains were able to hit #1 with half as much content, and only using the keyword half as frequently
  • Looking at all Ranking Factors, it’s clear that keyword domains rank more easily due to higher Website Relevancy and Website Ext Relevancy scores

Let’s examine each of these in detail…

 

average-rank

Keyword domains rank on average 11% higher than brandable domains

This represents the naive approach to answering the question of whether or not keyword domains are still beneficial. We simply looked at the average rank of domains that contain the keyword vs. domains that do not contain a keyword, and found that keyword domains rank a bit over 1 place higher. Of course you’re smart enough to know that a result like that doesn’t actually tell us much: maybe all of the websites using keyword domains hired better SEOs, or were more aggressively targeting that keyword. Corrrelation doesn’t equal causation, so to understand whether that higher average rank is attributable to the use of keyword domains or not, we need to dig further into the results to understand how the characteristics of top ranking sites built on keyword domains differ from top ranking sites built on brandable domains.

 

website-linkslinks-page

Brandable domains needed an average of 40,000 more links to hit #1, and 35,000 more links to reach the Top 10

One of the largest differences between high-ranking keyword domains and high-ranking brandable domains is that the keyword domains achieved their rankings with much fewer links than the brandable domains. The average keyword domain in the top ten had tens of thousands fewer links from other websites. While that may seem extreme, in many ways this result is predictable: the high-ranking brandable domains include megasites like Amazon, Wikipedia, eBay, and About.com that rank for everything under the sun, while many keyword domains are smaller websites focused on a specific topic. Note that this pattern was consistent whether we looked at the # of links to the website (~35000 more for brandable domains) or the ranking page (~1900 more for brandable).

 

website-authoritypage-authority

Brandable domains needed 69% higher Domain Authority and 22% higher Page Authority to rank in the Top 10

Since most of you know that raw number of links is a notoriously unreliable metric (50,000 blog comment links does not a high ranking make!), we also looked at Moz’s excellent Domain Authority and Page Authority metrics. Here again, the pattern was consistent: in order to hit the #1 ranking, a brandable domain needed a 81% higher Website Authority than a keyword domain, and a 69% higher Website Authority to reach the top 10. Stated otherwise, the average Website Authority of keyword domains in the top 10 was 41, while brandable domains had an average Website Authority of 70. For Page Authority, keyword domains in the top 10 averaged 40, while brandable domains averaged 49 (22% higher).

 

content-lengthkeyword-usage

Keyword domains were able to hit #1 with half as much content, and only using the keyword half as frequently

So we know that keyword domains have an easier time with on-page factors, but what about on-page factors like keyword usage and content quality? While these factors are more difficult to compare quantitatively (higher keyword usage is associated with relevancy, but at the extremes it’s also associated with spam), it does appear that here too keyword domains have an advantage over their brandable counterparts. Brandable domains in the top 10 had over twice as much text content on their ranking pages, and they used the target keyword and variations nearly twice as often. This is surprising given that keyword domains are almost by definition highly-focused on the keyword topic, and seems to be one of strongest indicators of a relevancy boost attributable to the keyword domain.

 

ranking-factor-comparison

Looking at all Ranking Factors, it’s clear that keyword domains rank more easily due to higher Website Relevancy and Website Ext Relevancy scores

So what exactly is going on here? Can we really say that keyword domains benefit from an algorithm boost, or are these patterns simply indicative of the varying nature of websites built upon keyword domains and brandable domains? As with any good scientists, in order to move beyond mere correlation we need to see if we can find some underlying causes. CanIRank’s Major Ranking Factor scores provide one way to do just that. We see that keyword domains consistently score much higher than brandable domains on factors related to Website Relevancy and Website External Relevancy, about the same on Page Relevancy and Page External Relevancy, and much lower on factors related to Page Strength and Website Strength. Translated into plain English, this says that keyword domains rank higher because they are more likely to be highly focused on the keyword topic, and they’re more likely to have people link to their website using anchor text relevant to the keyword. These positives enable them to rank higher despite generally having many fewer quality links to their ranking page and overall website.

 

CONCLUSIONS

Not a boost, just a different type of website

Despite Google’s EMD update and other changes, keyword domains still do provide significant advantages to SEO campaigns. While a simplistic look at correlations or average rankings implies marginal benefit, a more detailed analysis of underlying ranking factors reveals that keyword domains make achieving high rankings easier in way that is very tangible and understandable to anyone who has even lightly dabbled in SEO: they require fewer links.

However, rather than being a function of some mysterious “boost” that Google can dial up or down at will, it appears that keyword domains attain this benefit through their inherent qualities. Websites built on keyword domains naturally attract more keyword-relevant anchor text, one of the oldest but still most influential ranking factors, and something that happens only rarely for brand sites. In addition, websites built on keyword domains tend by their nature to be highly focused on the keyword topic, and at least some of this relevancy may be attributable to the domain name. In many ways, small, focused keyword websites are a crucial part of Google search result pages that seem to always feature the same half dozen big brands. They provide necessary domain diversity, and it’s hard to argue that an entire website focused on a topic doesn’t provide better information than a single page of content written by a non-specialist on say, CNN or the New York Times.

How to determine if a keyword domain is right for your next SEO campaign

The value of being able to rank with fewer links varies by industry and the nature of your business. CanIRank’s software can show you what it took for existing rankers to earn their top spot, as well as specific insights into the types of links, anchor text, and on-page targeting that’s currently working well for any given keyword. From there, it’s up to you to extrapolate what it would take to close that gap: what is the cost of each new piece of content you produce, and how many links do they generate on average? How effective are your PR campaigns, affiliate programs, brand building, and social media marketing?

Compare these promotional costs to the investment required to purchase a keyword domain name to see which path is right for you. My personal opinion: for new websites, new product areas, or any new topical content that diverges significantly from your established content, using a keyword domain to launch a satellite site provides a fast-track to SEO success, and the ROI is almost always exceptional. (And you can usually transfer those rankings back to the main brand site later through judicious use of 301 redirects or other strategies.)

As Google increasingly cracks down on aggressive link building tactics, it’s also worth considering the risk reduction benefit of being able to rank well with fewer links. Many “old standby” link building tactics such as directories, blog comments, article marketing, blog networks, forum profiles, and to some extent link buying and guest posting, have become less effective, more expensive, or downright damaging. That trend shows no sign of relenting anytime soon.

Future Research

Most discussion of keyword domain names centers around exact match domain names. I specifically focused this study on a broader definition of keyword domain names to increase our sample size and because the mechanism by which keyword domain names work their magic doesn’t seem to require an exact match of the keyword. However, there may be additional benefits that accrue to exact match domain names which this study doesn’t begin to capture. It would also be interesting to examine the impact of domain extension (especially in light of new gTLDs), hyphens, and extraneous additions like “online” or “HQ”. (My hypothesis is that the quality of domain does matter, since TahoeLodging.com is more likely to attract clicks and links than a less credible domain like Tahoe-Lodging-24.biz.)

As this is intended to be the first post in a series focused on the impact of different ranking factors, I welcome any feedback or suggestions as to how we can make these studies more useful for you in the future. I’m also looking for requests as to which ranking factors you’d like to see examined in more detail.

 

Google Poisons the Well: Expiring Domain Auctions Now Come With Extra Risk

are expired domains bad for SEO

Is it now dangerous to drink from the well of expiring domain auctions?

In medieval times desperate defending armies would sometimes dump diseased corpses into wells as they retreated. Rank with bubonic plague or tuberculosis, the well water became undrinkable, crippling the advancing army by depriving them of safe water sources. Never mind that this kind of scorched earth tactic also hurt the defenders’ own countrymen, depriving entire villages of their sole source of clean water: the end justifies the means, right?

Now it seems that Google has dipped into the history books with the latest tactic in their long-running war on manipulative SEO. Early evidence suggests that purchasing domains from expired domain auctions may have been one of the signals used to detect and deindex Private Blog Network (PBN) sites in a recent mass penalty. In other words, if you build a website on a domain name purchased through GoDaddy Auctions, NameJet, or other expiring domain name auctions, you’ll now need to be extra careful to avoid having your site classified as a PBN site and penalized or even completely deindexed.

That’s right: the well of expiring domain names has just been poisoned, depriving us all of one of the best sources for quality domain names at affordable “wholesale” prices. Anyone wishing to avoid the risk of being classified as a PBN site is now left with only two options: pay a premium to buy a domain on the secondary market, or register an inferior available domain name — for example, perhaps one of the new gTLD domain names in which Google was a heavy investor?

How does Google detect PBN websites?

Modern machine learning algorithms such as those deployed by the search engines often rely upon a combination of signals to classify or evaluate websites. For example, we might build a classifier to identify animals like this:

Animal Classifier
Color: gray gray
Trunk: yes no
Tail: yes yes
Size: Huge Tiny
Result: Elephant Mouse

Note that not all gray, tailed animals are elephants, just as not all low-quality sites built on expired domains are part of a Private Blog Network: some are just crappy websites, indicative of incompetence perhaps, but not nefarious intentions. If we were building a classifier to detect Private Blog Network websites, we might look for a number of signals that these sites have in common:

  • built on expired domains recently purchased at auction
  • templated site design
  • minimal, low-quality content
  • shared ownership or footprint with many similar sites
  • low cost web hosting
  • no quality inbound links or social signals
  • no brand signals
  • characteristic outbound linking patterns

In other words, purchasing an expired domain is not in itself enough to get your site penalized, and it’s unlikely to be so in the future. However, it is very possible that it is one of the signals used by Google. Early reports from folks who had their PBNs deindexed seem to suggest that this is the case:

Greg from NoHatDigital shared that all of their sites built on expired domains purchased in auctions, including those that were fully-developed sites and not PBN sites, were deindexed:

nohatdigital

Many commenters at NoHatDigital shared similar experiences:

comment thread discussing impact of expired domain auctions on reindexing

Finally, some commenters at NichePursuits indicated that their deindexed sites were also purchased through GoDaddy auctions:

nichepursuits-commenters

 

A War on Web Spam Domainers

While most people talk about Google’s war on web spam and its unintentional collateral damage to small businesses and hobbyist publishers, the search behemoth hasn’t been generous to the domain industry in recent years either. First, they pulled the rug out from under the domain parking industry with massive, unexplained revenue declines of 70-80%, even as PPC in general continued to grow. At the same time, their increasing algorithmic preference for big brands has hampered demand for new domains as it becomes more and more difficult for smaller websites to compete, forcing them to turn to social media and author platforms to make their voice heard. More recently the Exact-Match Domain update slashed the value of keyword domain names. Now, the latest moves to turn expired domain names into a spam signal threaten to destroy one of the last healthy areas of the domain industry, the expiring domain marketplaces.

Moving Forward: Better Deals If You Can Avoid Becoming a False Positive

Though it’s too early to see the impact this change will have on the expiring domain name market, my opinion is that we will see reduced demand and lower prices for expiring domain names with strong backlink profiles. If this turns out to be just one in a series of Google attacks on expiring domains, we could see SEO buyers disappear from expiring auctions completely. On the other hand, that change is unlikely to result in lower prices for quality premium domain names, as those two segments of the market rarely overlap (SEOs usually purchase low-quality domains with strong backlink profiles, and are priced out on quality domains with high resale value.)

I would still be willing to take the risk of building a new website on an expired domain name, as long as I knew I could avoid being falsely classified as a PBN site. Some ways to avoid being labeled a false positive might include:

  • Use unique whois info that doesn’t overlap in any area with your domain portfolio or other websites
  • Have only a small number of high-quality websites in your Google Webmaster Tools, Google Analytics, AdSense, etc.
  • Add high-quality original content with rich-media, images, many outbound links, etc.
  • Build a brand: about page, social media profiles, strong inbound links, press, brand mentions, etc.
  • Use a quality custom site design
  • Go easy on affiliate links and AdSense

 

 

Optimizing SEO ROI: A Not-So-Simple Formula to Uncover High-Value, Low-Difficulty Keywords

Today we launched a new feature that I’ve been wanting to see in SEO software for nearly 10 years. You probably already know that CanIRank offers an extremely accurate keyword difficulty tool, and we provide you with an estimate of a keyword’s value in the form of search volume, advertiser cost per click, and how much traffic you stand to gain by improving your ranking. But the final step— comparing ranking difficulty to value potential to decide which keywords offer the biggest bang for the buck— has always been a manual process reliant upon more than a little experience and gut feeling. Until now.

Today, we closed the loop.

CanIRank can now automatically calculate for you which keywords offer the best balance between ranking difficulty and value potential, specifically for your website. Here’s a sneak peek:

maximizing the ROI of your SEO activities

The High ROI tab on CanIRank’s Keyword Ideas page identifies which keywords offer the best balance between traffic value (search volume and ad value) and ranking difficulty (Ranking Probability or RP) specifically for your website.

How does it work?

First, CanIRank’s assessment of keyword difficulty relies upon our own proprietary metric called Ranking Probability (the RP column in the screenshot above).  We put over 2 years of research into perfecting Ranking Probability. For any given keyword, it takes into account the top ranking websites’ overall strength, what topics they’re considered authoritative in, their link structure, keyword relevancy and other on-page features, anchor text relevancy, and many other factors (around 200 in all). We then analyze how typical pages on your website compare for all of these factors to assess whether a page on your website would be competitive if it were targeting that specific keyword. The result is a much more accurate and nuanced concept of keyword difficulty which doesn’t just say “hard” or “easy”: it gauges how far a respective website would need to go to become competitive. With refinement, we could stop talking about keyword difficulty and start talking about the much more useful “Cost-to-rank“: how much time and money one would need to put into an SEO campaign to make a website competitive for a given keyword.

For example, SEOBook.com and QuickenLoans.com are both hugely authoritative websites with a tremendous amount of ranking power. But SEOBook would have a lot of work to do if they wanted to rank for “home loan”, while “SEO” would be a huge challenge for QuickenLoans. Both keywords have comparable difficulty, but each website would have a very different cost-to-rank. CanIRank’s Ranking Probability captures this difference while traditional keyword difficulty tools do not. So that’s the first half of our calculation.

keyword difficulty tool

Cost-to-rank, as estimated by CanIRank’s Ranking Probability score, is much more actionable than traditional “Keyword Difficulty”

Next, we look at a keyword’s value potential. Keywords that are frequently searched have more traffic-generating potential, but as anyone who’s ever tried to make money off a top ranking for “Paris Hilton” (200,000+ searches) will tell you, not all clicks are valued equally. Assuming that a keyword is relevant to your business, advertiser Cost Per Click serves as an excellent proxy for a keyword’s commercial value. Terms that are likely to generate sales– like “mesothelioma attorney” ($167 CPC) or “serviced office London” ($43 CPC)– have a high Cost Per Click because advertisers (other businesses like yours) know they can make money from people searching those terms.

Finally, we compare Ranking Probability to Keyword Value to come up with a Keyword ROI. In doing this calculation, we have to consider that Ranking Probability is not linear: a $1,000 Keyword Value is exactly twice as valuable as a $500 keyword, but a 40% Ranking Probability is a lot more than twice as difficult as an 80% Ranking Probability (i.e., much more than double the Cost-to-rank).

Don’t forget that ROI numbers will vary depending upon the website– a newer website that lacks the Website Strength to compete for anything but long-tail keywords may have their highest ROI keywords in the 40s, while a big brand website will likely find opportunities with an ROI over 100.

How to use Keyword ROI in your SEO & Content Marketing Campaigns

Let’s take a look at some of CanIRank’s keyword suggestions for my favorite example site, YogaDVDs.net:

Some keywords score well because they're "low-hanging fruit", others because they're very high value.

Some keywords score well because they’re “low-hanging fruit”, others because they’re very high value.

Note that keywords have a High ROI for different reasons. Some, like “baby yoga DVD” are low-hanging fruit: keywords that should be very easy for this site to rank for, and thus are worth targeting even if they don’t deliver a ton of traffic. Others, like “pregnancy yoga dvd”, are a bit more competitive, but could be worth the extra difficulty. YogaDVDs.net is a small site that’s doing a lot right, so it has some high Ranking Probability scores within its niche. Don’t be discouraged if your highest ROI keyword opportunities score lower: as your traffic grows and your site improves, you’ll become competitive for more and more valuable terms.

Once you’ve found a keyword that looks to be worth targeting, click on the Ranking Probability score to check out the full Report and better understand how your site compares to the competition. Then check the Issues section of the Report to see what you would need to improve in order to be competitive. If the changes seem doable and consistent with your overall marketing plans,  click “Build an Action Plan” to have CanIRank generate instructions for how you can fix your site’s Issues and improve your competitiveness. That Action Plan then becomes the basis of your promotional campaign once you’ve built a page targeting the keyword. If there’s a lot of work to be done, you can pass on the Action Plan to a freelancer or consultant to complete for you.

SEO Task Manager

Once you’ve chosen a keyword, get a customized Action Plan telling you how to close the gaps with your competition.

Conclusion

Rather than using our gut or simply hoping that the keywords we’re targeting will end up being worth the effort, CanIRank lets us quantify both input (cost-to-rank) and output (traffic value), optimizing our keyword selection and giving our SEO campaigns the best possible shot at achieving a high return on our time and marketing spend.

We hope that this feature will help you improve the productivity and profitability of your SEO and content marketing, bringing those activities one step closer to the kind of accountability available for other marketing channels like PPC or online advertising. While it may seem like this kind of detailed computer analysis and calculation is a bit of overkill for something as messy as SEO, keep in mind that some of the world’s most valuable companies were built on tiny 3% efficiency advantages enabled by new technology (such as Walmart’s innovative electronic inventory control system in the 1970s). Automated ROI estimation won’t replace the role of experience and human judgment in keyword research anytime soon, but we do hope that it will enable you to be at least a little more efficient and productive than you were before. And the cumulative benefits of smarter keyword choices and modest time savings, compounded over hundreds of analyzed keywords, will go a long way towards putting you ahead of your competition.

Happy ranking!

PS: If you’re new to Keyword Research for SEO and would benefit from a walkthrough of the entire Keyword Research process with CanIRank, you may find this tutorial video helpful:

What is the Best Keyword Difficulty Metric?

Keyword research is arguably the most important step in the SEO process. It’s the perfect antidote to Google’s recent algorithm updates making it increasingly expensive, complicated, and risky to attempt to “punch above your weight” by manipulating your way to the top of the search engine results: just find keywords where your site is already competitive. Or at least, where the competition is weak enough that ranking won’t take every trick in the SEO book.

As one of the the fundamental principles underlying CanIRank is that less SEO is better SEO, one of our goals was to create the single best measure of keyword difficulty, so that users can tell at a glance whether or not their site is competitive for a given keyword.  The result is the CanIRank Ranking Probability score, the first measure of keyword difficulty that assess keyword difficulty relative to your website.

keyword difficulty score

CanIRank’s Ranking Probability is more actionable than traditional Keyword Difficulty scores.

Other tools will tell you that “home loan” and “SEO” both have about the same keyword difficulty (85-87%). CanIRank will tell you that QuickenLoans.com scores a 100% Ranking Probability for “home loan” but only 77% for “SEO”, while SEOBook.com has a 95% Ranking Probability for “SEO”, but only a 72% chance of competing for “home loan”. As a site owner, getting keyword difficulty scores relative to your website is both more accurate and more actionable.

Not only are accurate Ranking Probability predictions important for helping to guide you in making the right decisions, but they’re also a reflection of our understanding of search engine ranking factors. When CanIRank makes accurate predictions, it suggests our ranking models do a good job capturing the impact of various ranking factors. This understanding of ranking factors is what enables us to then draw meaningful comparisons between websites, highlight areas in need of improvement, and recommend Actions that will have the greatest impact on your site’s ranking.

So we’ve invested a lot of time, money, and research into creating what we feel are the most sophisticated ranking models in the SEO universe.

accuracy of seo metrics

How well do they work?

The short answer is that while no predictions are perfect (just ask the weather man), CanIRank is over twice as accurate as other SEO tools, and can be used to explain a majority (but not all) of the ranking differences between websites.

More specifically, as a measure of how competitive a given web page will be in the Google search results, CanIRank’s Ranking Probability score is:

Before I dive into the analysis, I’d like to emphasize that all of the above are excellent and useful SEO tools, and this post is in no way an attempt to disparage or put down competing offerings. While CanIRank comes away looking pretty good in this study, it’s far from an exhaustive analysis, and all of the tools have their own strengths and weaknesses which I hope you’ll explore to determine what best meets your needs.

The Experiment

I based this experiment on the notion that an ideal metric for a page’s competitiveness would have a perfect rank order correlation to Google search results. That is, for any given keyword the #1 result in Google should always have a higher competitiveness score than the #2 result, which should have a higher score than the #3 result, and so on. Perfectly ranking 40 search results in the correct order would require that a metric encapsulates many of the same factors that Google considers, and also weights them similarly. In other words, it’s quite a rigorous standard for a competitive analysis tool, especially as for practical purposes it’s not as important that we can correctly order a #3 result before a #4 result as it is that we can tell when a URL is likely to rank #4 and when it is likely to be #40 (or not ranking at all).

russian dolls in order

Can SEO metrics order results correctly?

To measure how closely popular SEO metrics come to this ideal, I first pulled the top 40 Google results for 25 keywords randomly selected from a batch of 2 million keywords used in PPC advertisements. For each of the resulting 1,000 URLs, I collected the CanIRank Ranking Probability score, SEOMoz Page Authority, Google PageRank, MajesticSEO CitationFlow, and SerpIQ Competition Index. I then threw out all the URLs where SEOMoz didn’t have any data, leaving just under 700 results.

For each keyword, I then ranked the URLs according to each metric, and calculated the correlation between the rankings predicted by the metric, and the actual ranking in the Google results. Statistics geeks will recognize this as Spearman’s Rank Order correlation coefficient. A coefficient of 1 suggests the rank order of the two variables is perfectly correlated, while a coefficient of 0 means that there is no correlation.

Here’s an example of what the keyword table looks like for one of the keywords, “bankruptcy NJ” :

table of google results ranked by seo metrics

You may recognize this methodology as similar to past studies conducted by SEOMoz to determine the correlation between various web page attributes and high Google rankings, in which they’ve uncovered correlations ranging from 0 for keywords in an H1 tag to 0.03 for content length to 0.25 for # of linking root domains (the highest correlation).  You may be relieved to discover that, in this data sample at least, all of the SEO metrics outperformed nearly all of the raw page attributes. Woo-hoo! Perhaps there’s something to this SEO stuff after all.

The Results

correlation with Google search results

In order from lowest mean correlation to highest, let’s examine the suitability of each individual metric in more detail. Please keep in mind that this is an apples to oranges comparison as each of these metrics is designed to capture something slightly different, and as far as I know only CanIRank is even attempting to capture all of the factors that go into determining a URL’s competitiveness.

SerpIQ – Competition Index – 0.17

serpiq-logoSerpIQ’s Competition Index is “designed to give you a quick snapshot of the average difficulty of a Search Engine Rankings Page (SERP). The score is calculated by looking at the main ranking factors for a site, mainly: domain age, page speed, social factors, on page seo, backlink amount and quality, and PageRank.” Unfortunately I wasn’t able to get a Competition Index score for URLs beyond the top 10 results for each keyword, so SerpIQ’s correlation of 0.17 was calculated based upon the top 10 results only.

MajesticSEO – CitationFlow – 0.19

majestic_largeCitationFlow is Majestic’s version of Google PageRank, described as “a number predicting how influential a URL might be based on how many sites link to it.” As with PageRank, CitationFlow gives more weight to links from pages that themselves have many inbound links. As a predictor of Google rankings, CitationFlow was about as useful as PageRank, with a mean correlation of 0.19.

Google – PageRank – 0.21

logo4wGoogle’s PageRank, measured on a 0 to 10 scale and intended to capture the relative importance of a web page, was once the only game in town as far as metrics go. Unfortunately it’s become much less reliable in recent years, and most SEOs don’t put much weight in it anymore. However, as a predictor of Google rankings, PageRank scored the third-highest correlation, suggesting that it does indeed still seem to play a role. I used RankSmart’s handy bulk PageRank checker to collect the data for this study.

Moz – Page Authority – 0.27

Moz-logoMoz (previously known as SEOMoz) describes their Page Authority metric as “Moz’s calculated metric for how well a given webpage is likely to rank in Google.com’s search results”. There are some pretty smart folks up in Seattle working for Moz, and it shows in the excellent performance of their Page Authority metric, which had the second-highest correlation with Google search results. This is especially impressive, considering that (as far as I know, at least) Page Authority is not intended to capture any relevancy factors. In their own studies, Moz has calculated Page Authority correlations as high as 0.38; I’m not sure why the result is so much lower in my study.

CanIRank – Ranking Probability – 0.57

logoCanIRank’s Ranking Probability scores had the highest correlation to Google search results, with a mean rank order correlation coefficient of 0.57. Of course, this is to be expected as Ranking Probability is really the only measure intended to capture the complete picture of search engine ranking factors. Although 0.57 represents a fairly strong relationship between the variables (by comparison the correlation between IQ and academic grades is around 0.5), I think we can still do much better, and there are a number of improvements in the works which I think will push this number even higher.

Conclusion

With a rank order correlation coefficient of 0.57, CanIRank’s accuracy is high enough for us to feel comfortable claiming that Ranking Probability scores can be useful in assessing whether or not a web page is competitive for a given keyword. Of course no one outside of Google knows what factors are included in their algorithm, and they have access to all kinds of data (such as Chrome, Google analytics, Android, SERP click-tracking, etc) that other companies do not have. Our goal is not to replicate their algorithm, but simply to find out whether a ranking model built upon publicly available information retrieval concepts would correlate strongly enough to draw useful conclusions. We think this study demonstrates that this is possible, and with further work we hope CanIRank will be able to provide even more useful guidance.

 

How Much is a #1 Ranking Worth to Your Business?

Estimating the Traffic Value of a potential target keyword is a critical step in a comprehensive keyword research process. If you know how much value a top ranking will bring to your business, you can better determine how much time and money it would be worth investing in SEO. You can even calculate the ROI of each keyword you’re considering, and choose to target keywords that will bring the most additional revenue for the least cost. That’s the kind of accountable marketing that even your CFO will love!

The Traffic Value of a keyword is a function of a) the value to your business of a visitor who searched for that keyword (Earnings Per Click), and b) the number of clicks you receive from that keyword (# Clicks). Here’s a simple formula:

Traffic Value = Earnings Per Click * Num Clicks

 

In SEO the number of clicks is determined by the total number of people searching for that keyword (# Searches) times the percentage of people who click on your search engine listing (Click-Through Rate), so the formula becomes:

Traffic Value = Earnings Per Click * Num Searches * CTR

Calculating the Traffic Value of a Keyword – Example

As an example, let’s assume that 500 people per month search for the keyword “Yoga DVD reviews”. The owners of YogaDVDs.net have calculated that each visitor to their website looking for “Yoga DVD reviews” is worth $0.50 (more on how they determined this below).  On average 40% of searchers click on the #1 ranking. The Traffic Value of “Yoga DVD reviews” can be calculated as:

Traffic Value = $0.50 * 500 * 40% = $100/ month

$1200 a year may not sound like a lot, but in practice the improvements you make to gain a #1 ranking for a competitive keyword like “Yoga DVD reviews” inherently results in top rankings for lots of other less competitive related keywords, like “best Yoga DVDs”, or “Yoga DVD ratings”. I call these “Satellite Rankings” because they are effortlessly pulled in by the “gravity” of the much larger keyword. In my experience Satellite Rankings can easily lead to a doubling or tripling of the Traffic Value of a keyword.  In some cases the combined value of primary + Satellite Rankings ends up being 10 times the value of the primary keyword alone!

Let’s look in more detail at how to estimate each component of the Traffic Value formula:

Earnings Per Click

There are two ways to estimate your Earnings Per Click: the Best Way, and the Easy Way. The “Best Way” is to calculate your Earnings Per Click based upon your own sales statistics. For example, if you had 1,000 visitors from the keyword “Yoga DVD reviews”, 5% of those customers went on to make a purchase, and your average profit per new customer is $10, you earned $500 on those 1,000 clicks, so your Earnings Per Click is $0.50.

Earnings = 1,000 visitors * 5% conversion * $10 profit = $500
Earnings Per Click = $500 / 1,000 clicks = $0.50

But how are you supposed to know those numbers if you’re not yet ranking for the keyword? Well, there are a couple of options. First, you can buy some advertisements on the keyword through a pay-per-click ad program like Google AdWords. Track how many clicks you receive from that keyword and how many sales those visitors make, and you can easily calculate your Earnings Per Click. Another option is to estimate your Earnings Per Click based upon the performance of a similar keyword.

If the above sounds like too much work, the “Easy Way” to calculate your Earnings Per Click is to base it upon the average Cost Per Click for that keyword on Google AdWords. The basis for this is that many advertisers are willing to bid right up to the value of a click, so in essence you are piggybacking on the work the advertisers have already done calculating their respective Earnings Per Click. To get the Google AdWords Earnings Per Click, search your keyword on Google’s Keyword Planner (a free Google AdWords account is required to use the Keyword Planner).

Cost Per Click from Google Keyword Planner

The Google AdWords Keyword Planner tool provides estimates for monthly search volume and average Cost Per Click.

 

Number of Searches

Google’s Keyword Planner is also your best source for data on the number of searches a keyword receives. Unfortunately, the numbers that Google provides can be somewhat unreliable, so if you’re interested in a more precise estimate the best method again is to take out an ad on the keyword via Google AdWords (keep in mind that the organic search listings receive 5 times as many clicks as the paid ads!).

Click-Through Rate

The Click-Through Rate (CTR) on your search listing will depend upon your ranking in the search results, and the attractiveness of your listing relative to the other results. It should come as no surprise that the top rankings receive the majority of clicks. Studies have shown that the #1 ranking receives from 50-60% of the clicks, #2 is clicked about 12-15% of the time, #3 about 9% of the time, and the remaining results go down from there. Only 1 in 10 searchers click on a listing beyond the first page.

distribution of clicks on search results

Most of the clicks go to the top few rankings.

While these figures are for the average search result, sometimes there are factors which cause CTRs to deviate from the norm. For example, brand searches like “Puma shoes” or navigational searches like “IRS Form 1040″ are likely to have nearly all the clicks concentrated on a single result (the Puma home page and the official IRS page for the 1040 form, in these cases). Well-known brands generally attract a lot of clicks regardless of where they’re ranked. For example, people searching for “HDTV reviews” are more likely to click on a listing for CNET or Consumer Reports even if they’re ranked #3 or #4, if the #1 result is an unknown brand like Brads-hdtv-reviews.biz.

To estimate your CTR, consider how attractive your result snippet is relative to the existing results. If there are much bigger companies than yours included and they do a good job addressing the needs of the searcher, count on receiving less than 50-60% of the total search volume even if you do reach the #1 position.

Maximizing the ROI of your Content Marketing

Now that you know how to calculate the Traffic Value of a keyword, your SEO and Content Marketing efforts can be optimized just like any other marketing channel: generate maximum sales with minimal spend. So how do you calculate the cost of ranking for a keyword? That’s where CanIRank’s Ranking Probability scores come in. Higher Ranking Probability scores equate to a lower Ranking Cost. In general, a Ranking Probability score of 90% or above means that your costs will be limited to content production and following your usual content promotion process. A Ranking Probability score around 80% means you’ll also need to dedicate a few hours a week to completing SEO tasks for a month or two. Scores around 70% or lower usually mean that you’re looking at several months work of regular SEO work, or spending heavily on more aggressive promotion.

With a little practice you’ll start to figure out what cost a given Ranking Probability score equates to for your particular website. Then it’s simply a matter of comparing each keyword’s Ranking Cost with its Traffic Value and you are on your way to making much better decisions in your content marketing.

How to Get the Most from CanIRank

Used correctly, CanIRank isn’t just a way to help you find new keywords to target; it’s an entirely new approach to growing search engine traffic that will help you dramatically improve the ROI of your content marketing efforts. CanIRank is about helping you spend less time and money on traditional SEO activities like link-building and content promotion. Less SEO means less risk; so there’s no more living in fear of the latest Google algorithm updates or penalties. Most importantly, we hope CanIRank will enable you to spend more time doing what you do best — growing your business and delighting your customers — and less time trying to appease search engine robots.

Don't worry too much about pleasing robots.

Don’t become too focused on pleasing robots.

In order to achieve these goals, every piece of promotional content needs to earn its keep. There’s no point spending your valuable time writing a terrific blog post that no one will ever read. So how can we make sure that the website content you create gets noticed by both search engines and readers?

The answer requires flipping the traditional SEO process on its head: rather than creating a page and then trying to find ways to push it to the top, we will use CanIRank to ask the question: if I created a page about this topic on my website, what would it take to rank? By spending a few minutes more on upfront research, we can save hours of work later. No more “publish and pray”; instead CanIRank injects much-needed transparency and accountability into the content marketing process. For every piece of content you create you’ll have a reasonable idea of how much time and money it will take to rank well for that topic, and can adapt your content strategy accordingly.

A “New School SEO” Process with CanIRank:

Sound good? Great! Enough theory, let’s see how a New School SEO / Content Marketing process enabled by CanIRank might look in practice:

  1. Brainstorm keyword possibilities

  2. Narrow down keyword options to only those where our site is competitive

  3. Figure out what it would take to rank

  4. Generate an Action Plan for every keyword we want to target

  5. Create the content and follow the Action Plan until we rank!

Step 1: Brainstorm Keyword Possibilities

Brainstorm keyword ideas with Google Keyword Planner

Content Marketing works best if you’re not fixated on ranking well for just one or two keywords. Your first step should be to brainstorm as many different target keywords as you can think of. What types of questions do your potential customers have? What related information might they look for before making a purchase? Potential customers rarely use the “correct” vernacular: ask people outside of your industry what words they might use when searching for a business like yours.

If you run PPC ad campaigns (such as Google AdWords or Bing Ads), you can use the keywords from your paid search ad campaigns to inform your content marketing (organic or unpaid search) efforts. In fact, one of the best free tools for coming up with new keyword ideas is the Google Keyword Planner Tool. Enter a seed keyword or a URL and Google will suggest hundreds of related keywords, including the number of times that keyword is searched each month, an estimate of the cost per click (CPC), and an indicator of how competitive those ad placements are.

Step 2: Narrow Down the Keyword Possibilities

Use Ranking Probability score to narrow down your keyword possibilities

Hopefully you now have a list of Keyword Possibilities with dozens of topics that you might want to write about. Your next step is use CanIRank to narrow down the list to only those keywords where your website is currently competitive. (Don’t worry, as your website improves in authority, you’ll be coming back to the rest of those topics later.)

On CanIRank’s New Report page, enter a few of the keywords you’re interested in, and the URLs of a few pages on your website. Select “A page like” from the URL dropdown menu so that CanIRank knows you’re trying to evaluate the ranking potential of a similar page as if it were targeting that keyword.  Make sure you use URLs that are representative of the page you’ll be using to target this keyword. For example, if I’m researching a keyword for a new catalog page, I’ll include the URL of a few existing catalog pages. For a new blog post topic, I’ll enter the URLs for 3 or 4 of my existing blog posts. A good trick is to use one successful post, one average post, and one unpopular  post, so that CanIRank will return a range of Ranking Probability values representing “best case”, “typical case”, and “worst case” scenarios.

Try to find keywords with a Ranking Probability score in the 80s or 90s. These are the keywords where you’re most likely to be able to rank without having to spend a lot of time or money on promotion.

Step 3: Find out what it would take to rank

Use competitive analysis table to figure out what it would take to rank

Your list of Keyword Possibilities should now be down to the 3 or 4 keywords with the highest Ranking Probability score (hopefully 80% or better). Before you make a final selection, take a closer look at the Competitive Analysis table and “What do you need to improve…?” sections of the Report so that you can get a better idea of what really stands between you and a top ranking.

Are the existing rankers aggressively targeting this keyword by including it (or a variation) in their page title and content? (You’ll need more than good Page Relevancy to rank!) Is your Website Strength lower than most of the other rankers? (Choose a different keyword or be prepared for a long, slow process of building trust!) Do several rankers have low Page External Relevancy and/ or Page Strength scores? (That could be the weakness you need, since those scores are much easier to boost than Website Strength!)

Under the “What do you need to improve to rank better?” section, take note of any Ranking Factors where your URL scores below the minimum for all Rankers. Will you be able to easily increase that score to become more competitive? Click on the “Issues” button to get more specific details: are the Issues limited to one or two areas of weakness that you’ll easily be able to address?

Step 4: Generate an Action Plan

Action Plans are designed to address your site's weaknesses.

The deep dive into your website’s strengths and weaknesses vis à vis the competition should have enabled you to choose a target keyword with a relatively straightforward path to ranking. Now you’re ready for the fun part: taking action! Successful content marketing has two aspects:

  1. Thoughtful, creative content creation
  2. Effective content promotion

CanIRank is designed to help you with the second aspect, guiding you towards just the right kind of content promotion necessary to help your content rank well in the search engines and attract a steady stream of free traffic. These promotional activities are encapsulated in CanIRank’s Action Plan. Action Plans are a sequence of SEO and online marketing tasks designed to address the specific needs of your page, so each Action Plan focuses on a single URL competing for one keyword.

To generate a new Action Plan, click on the “Build Action Plan” button next to the URL that is most representative of the page you’ll be using to target this keyword. CanIRank will create the new Action Plan and take you to the “My Actions” page, where you’ll find your first few Actions. Each day you complete (or skip) Actions, CanIRank will generate a new batch of Actions focused on addressing the most urgent needs of your website (from the SEO perspective, at least). We recommend setting aside a little bit of time every day to complete an Action (or 2 or 3…), since small consistent effort will lead to much better results than sporadic spurts of heroic effort (which is not only less effective, it can sometimes lead to search engine penalties!).

Step 5: Create the Content and Promote Away!

Complete Actions and watch your rankings climb!

Creating a thoughtful and creative piece of content is perhaps the most important step of the entire process. The better job you do creating content that your readers find genuinely useful, interesting, and engaging, the easier you will find the content promotion activities to be. Spend some time putting yourself in the shoes of your customers and thinking about what they would find genuinely helpful. Time and money invested in quality writing, images, and web design acts like a lever to improve the effectiveness of your entire website, making it easier to attract links, keep visitors engaged, and convert browsers into paying customers.

After publishing your dazzling content and spending some time (perhaps 30 minutes/ day for a month) working through promotional Actions, if you’re still not ranking, you may want to try creating a new Report for the same keyword, this time selecting “Existing page” and entering the URL of the new page you’ve created. This Report will show you how your new page compares to the hypothetical model we analyzed previously, and may identify new Issues that you’ll want to build an Action Plan to address.

If you don’t see results right away, take heart: sometimes search engines take time to recognize improvements (especially in smaller websites), but as long as you keep doing the right things, eventually the rewards will catch up to your good efforts!