Avvo's Top Legal Blogs list (TLB) — a ranking of the top legal blogs (blawgs) — is a textbook example of smart marketing. It provides interesting information and attracts inbound links (especially from the featured blogs).
I applaud the list in the abstract, but deplore its implementation. It strikes me as a lazy effort designed purely for marketing purposes with little to no regard for accuracy — strange for a company whose mission revolves around providing actionable information about lawyers.
Avvo's TLB has two major problems — sample pool and data source. A list by its very nature depends entirely on these two components. In other words, TLB is unreliable. It's also a slap in the face to legal bloggers, most of whom frankly need a slap in the face, just not of this variety (I'd like to see legal bloggers debate each other more often and also criticize lists like TLB).
Avvo's Sample (Kiddie) Pool
Many a blogger has no doubt wondered why their blog does not appear on TLB considering that as of today 15 blogs on the TLB have such a miniscule audience that scant data exists. And then they see the reason:
"[W]e could not include every blog we would have liked to (no subdomains or folders on non-blog sites), but we are working to change that. In the mean time [sic], if you wish to inquire about adding your blog to the list, please send an email to ..."
As you can see, Avvo admits that it does not include any blogs which exist as part of larger sites, eliminating many of the top blogs in one fell swoop.
But what about a blog such as Jonathan Turley, which won the ABA's Blawg 100 contest earlier this year in the Professors category? It resides at its own domain name and has an Alexa ranking of 158,350 (see below) yet is absent from Avvo's TLB.
It seems like Avvo put together an initial list and has since relied on bloggers to let them know about the existence of their blog.
What's the point of a list that omits such a large chunk of the legal blogosphere? The result is a kiddie pool of data a mile wide and six inches deep.
Alexa = Garbage In, Garbage Out
I suspect if you got Jeff Bezos drunk, he would express deep pride in and enthusiasm about all of Amazon's services — except Alexa, which he would admit is useless.
Avvo uses Alexa as its data source for TLB.
Alexa collects data on Web site usage from people who install its toolbar in their Web browser.
Remember your college statistics class? No? Well, here's a quick refresher. You can rely on statistical data about a group in only two situations:
- When everyone in the group provides data (100% participation).
- When you obtain data from a random sample of the group that is representative of the whole group (a small margin of error applies to the results).
Data collected in any other manner might be entertaining, but it won't be reliable. Like Alexa.
Not every Internet user has the Alexa toolbar installed so participation is not 100%. And Alexa does not engage in random sampling. Instead, anyone can install the toolbar (self-selection).
Don't believe me? Google executive Matt Cutts conducted a very simple experiment to demonstrate Alexa's flawed data.
Using Alexa to gauge the popularity of legal blogs is like using a group of lawyers hanging out in a trendy lounge to determine the country's best-selling beer. Your survey might conclude that it's Peak Organic Pale Ale when in fact the answer is Budweiser.
How Avvo Can Fix TLB and Make It Useful
I would not criticize TLB without offering a solution. In fact, many possible solutions exist, including:
- Forget about statistical rankings and instead provide editorial rankings. Avvo could take it a step further and invite people to vote for their favorite blogs from the pool it selected in a variety of categories. The only problem with this solution is that it already exists — the ABA Blawg 100.
- Drop Alexa as a data source and instead use Compete or a similar service. Like Alexa, Compete obtains data on Web usage from people who have installed a toolbar. Unlike Alexa, Compete controls the distribution of its toolbar so that it doesn't suffer from self-selection problems. While the data is not as reliable as that from a true random sample, it's about as good as the Nielsen system that networks and their advertisers use for TV ratings.
- Require legal blogs that wish to participate in TLB to install SiteMeter or some other free javascript-based Web analytics tool on their blogs and share the data with Avvo for ranking purposes. This method would provide the most reliable rankings thanks to data from 100% of the group — legal blogs that opt to participate. In this scenario, legal blogs absent from the list would have only themselves to blame.
Each of these methods would require Avvo to invest more resources into TLB. Because TLB seems to have little to no relation to Avvo's core business, Avvo may not want to make a significant investment. That's perfectly understandable, but Avvo cannot have it both ways. In its current incarnation, TLB does not, as Avvo claims, enable you to "[k]now who has the most popular legal blog [sic] based on objective, third-party data."
About TechnoEditorials
A TechnoEditorial is the vehicle through which we opine and provide tips of interest to lawyers, law office administrators, and others in the legal profession. We link to each new TechnoEditorial and much more in our BlawgWorld newsletter, which is free. Please subscribe now.
