Josh Pigford, the creator of The Apple Blog, has posted an extremely well-written rant of sorts regarding the major flaw with Alexa website traffic statistics. Alexa is a serviced owned by Amazon that provides website traffic statistics based on information collected from web surfers with the Alexa toolbar installed. The problem is that the Alexa toolbar only runs on Internet Explorer on Windows machines, effectively voiding every single alternative browser and Mac user.
To give you an idea of how little the Alexa toolbar is used, IE6+/Windows users must physically go out onto the interweb, download and install the toolbar themselves. This is not some “oh it came bundled with Windows, how cool” deal. In addition, many pieces of spyware prevention software actually flag Alexa as spyware, as it tracks your browsing habits and reports back to the mothership (no not that mothership.. or that one). As a side note, people have tried to spread a Firefox extension that hooks up to Alexa, but Firefox users are all too smart for that ruse.
Why is this important? As Josh noted, many businesses are utilizing Alexa stats for things like advertisement services. This ends up hurting webmasters that own sites with a large chunk of Firefox, *nix, OS X, etcetera users.
Businesses like Text Link Ads and ReviewMe place a very large amount of weight on Alexa rankings. Obviously they need some way to rank sites, but the problem here is that the tool they are using is largely skewed towards Windows IE users when not every category of site has that demographic as the large majority.
This whole talk about Alexa started out when I posted something in the 9rules member forums regarding blogger John Chow’s Alexa traffic stats compared to mine. John mentioned in a recent post that he received 266,641 page views from 127,614 unique visitors during the month of February. For February, I received 227,852 page views from 136,064 unique visitors. Does the Alexa graph below show that?
Alexa shows some wildly inaccurate information, but to Alexa’s knowledge that is correct. John Chow’s readers seem to be heavy PC users as John runs a PC-centric tech hardware site as well. It can only be assumed that he has more Windows and IE readers than I, a Steve Jobs-idolizing college student, do. Since Alexa bases everything off of data from IE users toting their hated toolbar, this chart leaves most of my readership out in the proverbial, unmonitored cold.
Moral of the story – take Alexa stats with a grain of salt if even that. Josh Pigford pointed out that there are alternatives to Alexa (Compete, comScore, Hitwise), but the problem is getting companies already utilizing Alexa information to stop and change what they’re doing.
Before I go, I’ll leave you with a Grade A rant on the subject written by the knowledgeable-in-all-things-random Kyle Neath:
We have far too tight a resolution of website statistics. As owner of a server, we can analyze down to the request. Unfortunately there’s a lot of noise out there (spam bots, search engine spiders, people typing in the wrong domain, etc). However, we still feel we have a good grasp on how many “people” visit our site, even if that definition of a person is based on an IP address.
Realize that no other media has this. Magazines come next in terms of resolution, but even they can’t count the number of companies that subscribe that then pass the magazine on to 50 other people.
Media like Television can’t even try to grasp this resolution. How many people watched LOST last night? Well.. we don’t have a hit for every person that watched it. We can’t even count how many people downloaded it via bittorrent. Suddenly the game gets more complex. The reality is that we only have good resolution for very high profile TV — The top 100 shows or so. Trying to measure how many people watched Sci-Fi’s B-movie of the month is extremely difficult.
This is where we come back to Alexa. To me Alexa is a lot like measuring TV stats. We’re taking the best possible source data we have, and trying to apply it to Websites. This data when compared to TV is excellent. We can get a really good handle on the top 100 sites — hell, even top 1000, maybe more. But then people apply this Alexa information to sites in the 10,000’s. Whoa… we’re starting to introduce a LOT of assumptions, and a lot less good data. Extrapolations are frequent, and data can get skewed easily.
Now people take that alexa-view of their rank 20,000 site and compare it to their fine-grain stats by their webserver. Well, of course it’s going to be off. It’s not even comparable. When you apply that fine-grain resolution view to the extrapolated-view of alexa, it’s going to look extremely inaccurate — and it is.
Really, this whole web site “statistics” game is just a bunch of guessing and assumptions. Remember that it all starts off with the assumption of what a visitor is. If I changed my assumption of a visitor to a unique IP per 10 hours, I could more than double my traffic. Think about it.
As a webmaster, domain owner, blogger, etc., what’s your stance on this issue? I believe that if companies can’t be easily cajoled to stop using Alexa statistics, Alexa should at least hire someone to fix their problems.
PaulStamatiou.com runs on the Thesis Theme for WordPress
Thesis is the search engine optimized WordPress theme of choice for serious online publishers. If you’re a blogger who doesn’t understand a lot of PHP, Thesis will give a ton of functionality without having to alter any code. For the advanced, Thesis has incredible customization possibilities via Thesis hooks.
With so many design options, you can use the template over and over and never have it look like the same site. The theme is robust and flexible enough not only to accommodate a site like PaulStamatiou.com, but also to enable the site to run far more efficiently than it ever has before.
Stumble This
{ 18 trackbacks }
{ 38 comments… read them below or add one }
I agree with you Paul.
To add to your list of Alexa quality related posts, Alexa Toolbar and the Problem of Experiment Design, a worthy read (…You would have seen this is you subscribed to the web dev resources :P).
Thanks for your insight into what really seems to be a problem with the ratings paradigm. I guess perception is the name of the game even if it is based on an illusion. Alexa has created the perception that what they are providing is more meaningful than it actually is. If enough bloggers jump on this, tho, and help to change that perception, maybe something can be accomplished to change that perception.
I’ve never believe in Alexa (Alexaholic) stats. Completely innaccurate results. This should already be a given knowing that Windows users [only] must physically install a toolbar in order for traffic to be tracked. That alone already should have raised eyebrows. That means I could setup a few different machines all loading the same page using the Alexa bar.
Take Alexa with a grain of salt? How about ignore Alexa completely?
Great post Paul, I had no idea how Alexa gathered its data up until reading this article… I just never bothered to look into it.
If everything you’re saying is true, it bothers me that Alexa doesn’t prominently display a disclaimer with it’s traffic rankings that will let people (read: advertising executives) know that the statistics are incomplete. I’m curious to find out how popular the Alexa toolbar actually is, but the fact that the stats leave out a very large percentage of users is a huge downside.
For sites that have a large percentage of Mac or Firefox users (of which mine is one of those sites), where does that leave them in terms of competitive advantages? Are advertisers going to be more likely to advertise on their sites, or on sites with a higher Alexa ranking? Without gathering all relevant data, my bet is that advertisers will choose the site with the higher Alexa ranking.
So where does that leave things? Smart advertising executives should be able to find out that Alexa’s statistics are incomplete… but who’s responsibility is it to point those facts out?
Nice post. I’m going to point people to your post when I try to make this point to people in the future. It’s funny – I had _just_ mentioned this to Noah at MyMint the other day in passing. Good to see you take a moment to cover it.
That’s certainly true, but advertisers keep an eye on Alexa Score before making decision. I remember, there was a Firefox Addon called SearchStatus, which simulated these Alexa data.
I’m not sure how it does it, but its quite interesting to have a look over it:
http://www.quirk.biz/searchstatus/
Yeah I mentioned that in my sidenote in the 2nd paragraph… how many people do you know that run that extension?
Agreed, but there isn’t much we can do about it, unless with educate advertisers that Alexa has no damn importance.
This post is awfully good, lets hope that advertiser have a visit to this post
What’s the problem with “Alexa is only available on Windows” – that’s 90% of the market. Isn’t it?
Markus: Paul is a mac guy! Don’t you know?
Markus – the problem is that people with a large mac/linux following don’t get represented fully.
From what I’ve seen/heard, Compete isn’t any more acurate than Alexa. comScore and Hitwise are much better alternatives (although you have to pay some money to use them). I think that’s why a lot of folks rely on Alexa… it’s free.
Unfortunately, in this situation, I think you’re getting what you pay for.
Thanks for hijacking my rant. ;)
hence the linkback. =)
Haha…I wasn’t linkbacks were worth much of anything.
I don’t monitor with Alexa at all, but I loved the Cat.
Web metrics that only consider clicks, time spent, and search or various attempts to combine those three will always be flawed. The bottom line is that none of those statistics tell you about the PEOPLE who are using the site. The only way to measure your audience is by figuring out who they are. Alexa can’t do that. IPRO has a new ad exchange called SPOTSITE that’s free and you can look at IAB verified research on thousands of sites to figure out who is really interested, not whether or not they got there by mistake and left the window open.
I don’t understand why there is so much misinformation about this subject. In my opinion, Alexa traffic stats are actually very reliable for sites ranked inside 10,000.
Contrary to popular belief, Alexa statistics are reported on any computers using the Firefox plugin SearchStatus (this includes Macs).
Also contrary to popular belief, Alexa statistics are skewed toward web developers, not grannies using IE. A disproportionately high percentage of the web developer population uses the SearchStatus plugin (and therefore reports traffic data to Alexa) when compared to the web surfing population as a whole.
While you shouldn’t quote Alexa rankings as ultimate truth, Alexa data is useful for comparing website to website, especially when those websites are in the same market (ie: Digg.com vs. Reddit.com).
@J. Bryan Scott – How many people use that ffox extension?? Who is going to go out of their way to help Alexa? I’ve never heard of anyone, even web developers, using that plugin and I just asked my whole aim buddy list of web developers.
I’m glad you asked. Here is my supporting evidence:
http://www.alexa.com/data/details/traffic_details?site0=digg.com/&site1=slashdot.org&site2=reddit.com&site3=flickr.com&site4=facebook.com&y=r&z=3&h=300&w=500&range=1y&size=Medium&url=http://digg.com/
All 4 sites with a disproportionately tech-heavy audience (for purposes of the discussion, lets say web developers) jumped right after the Apr demarcation on the graph. But facebook.com didn’t.
What accounts for the jump in tech-heavy sites but no jump in Facebook?
The SearchStatus plugin for Firefox got Dugg (and other major press) during the middle of April. Many people who follow these tech sites installed the plugin. Facebook does not have a tech-heavy audience, and its users didn’t install the plugin (at least not in disproportionate numbers). Knowing this, the statistical implications behind the strange jump in April follow intuitively.
It’s not a matter of going out of your way to help Alexa. I use the SearchStatus plugin because it is a way for me to quickly see a ballpark range of traffic a domain gets. I find the tool genuinely helpful and informative, especially when comparing sites with similar audiences. I run it (and Firefox) on all my machines.
“quickly see a ballpark range of traffic a domain gets.”
don’t most ppl use the google toolbar w/ PR rank for that kind of info? not necessarily traffic but PR is kinda similar.
PageRank is similar, but it’s more of a credibility ranking since it’s based on inbound links. Even if people don’t link to you, they might still visit your site every day and find the information very useful.
I don’t think I have linked to Facebook more than a few times, but I probably put 100 pageviews on their site every day (but less all the time thanks to their beautifully-implemented AJAX).
I have never linked to you, but I read most of your blog posts, and sometimes I will open up my web browser to read them (or comment). I don’t benefit your PageRank, but I do help your Alexa rank.
Now for some web-economic analysis: To sum this up, my proportionate marginal contribution to Facebook’s PageRank is much less than my proportional marginal contribution to Facebook’s Alexa ranking. I think this is true of most of Facebook’s audience, barring only those who have true capacity to impact PageRank such as press sources and big name bloggers.
If you are an advertiser, you want lots of eyeballs (and better yet, clicks) on your ads a lot of the time. Which do you care more about: the benefit of a few, or the benefit of many (or at least, more) ?
You are comparing apples to oranges because you are not in the same niche.
If you compare apples to apples, blogs in the same niche, and then compare how they stack up on each service, you end up with a completely different outlook.
Oh I have already done that
http://andybeard.eu/2007/02/compete-toolbar-bug-with-google-reader-compete-vs-alexa-stats.html
Also TLA do not place a large weight on Alexa, their largest weight is on Bloglines subscriptions (40%) which puts your blog at a huge agvantage over someone like John Chow, many of whose subscribers wouldn’t have even heard of Bloglines.
I hate the fact that TLA relies so much on Alexa…. thank god for Google PR coming in to the picture…
I haven’t used Alexa for a long time. I remember trying to use it about four years ago for research purposes, but I always used to wonder about the reliability of the figures even then.
Alexa is not just for IE. I have the SearchStatus extension installed for a few weeks. I just found out that I also contribute to Alexa’s user base.
However, either Alexa or SearchStatus is not working properly, in my opinion. I am developing/running a relatively low-traffic site (Alexa: 657,000, Quantcast: 2,030,000). I check the stat of the site through out the day (via the admin site in the same domain). I am currently living outside US. From Google Analytics and Quantcast scripts, I know that most of the traffic are NOT from the country where I am in. But Alexa currently shows that most of the traffic to my site IS from the country that I am!
I am not sure if the culprit is the SearchStatus or Alexa. Since then I have removed SearchStatus extension and block the cookies from Alexa (not sure if it does anything).
What do you guys think?
As I mentioned in my blog, Alexa is NOT completely worthless. Even steaming piles of turd can be utilized as fertilizer :) It has value if you are going to SELL your site or sell advertising. Sure, refering to Alexa stats is like a Penis Enlargement site referring to dubious anatomic measurements. The bottomline is people give it credence so…. there’s definitely sales and marketing mileage off Alexa. Does it yield helpful info to grow your site? No. Does it increase site sales and advertising prices? Yes.
You appeared to have ignored the last sentence by Kyle:
…If I changed my assumption of a visitor to a unique IP per 10 hours, I
could more than double my traffic. Think about it…
What was your assumption when you counted your visitors? What was John Chow’s? It is 99% likely that you two are using very different methods to count your users. If you had used the exact same method to count your pageviews and visitors his traffic might actually be a lot higher than yours, as reported by Alexa, and in fact, I’d bet it is. Your critique of Alexa is based on a HUGE assumption and is probably bogus.
@geoff – What does Alexa consider a unique visitor?
I’m kinda late to this discussion, but I’ve about decided that Alexa ranking is a bit like those cheesy late-night infomercials that hawk those “available for a limited time” Nascar plates (with the 24-karat gold rim, of course) in that what makes those “collectable” is that someone declared them to be so.
In other words, Alexa rank (along with Google PageRank, IMO) is a self-propagating – almost viral – hoax. Alexa-originated hits are mostly important because, well, Alexa says they are. And since so many other bloggers have bought into the it, well, it must be true.
well,in a way,alexa is one of the better way to judge the billions of websites out there.so alexa is still the preferred ranking system for me
Yeah I totally agree with you .. sometime I just don’t understand why with more traffic it goes down.. I don’t think it’s accurate but some advertisers still use it to judge whether your blog has good enough exposure or not using alexa..
Agreed, but I still get annoyed with it regardless of how useless its results are to me.
I completely agree with you here. Alexa ratings are generally unreliable.
Hello,
Just wanted to add an information. The official addon for Alexa on Firefox is called Sparky
I don’t fully rely on Alexa rankings, but nobody can deny it reflects in some way a website’s popularity. It is not the global ranking that interests me, it is by country ranking.
Regards
Alexa is not a great provider in website traffic statistics, because it only gets its statistics from users with the alexa toolbar, that is true, but still heaps of webmasters check it out before purchasing any site, it is reassuring to them.
Informative tutorial. thanks for sharing. Advertisers must read..