{{ searchResult.published_at | date:'d MMMM yyyy' }}

Loading ...
Loading ...

Enter a search term such as “mobile analytics” or browse our content using the filters above.


That’s not only a poor Scrabble score but we also couldn’t find any results matching “”.
Check your spelling or try broadening your search.


Sorry about this, there is a problem with our search at the moment.
Please try again later.

Here at Econsultancy we’re big on data, and we believe that the best way for publishers to monetise their online audience is to properly evaluate visitors, just like the smarter retailers do. We’ve created a tool to show you how we’re trying to do this.

Called ‘Project Arachnid’, the tool is essentially a real-time visualisation of our website traffic. You can see new visitors arriving at the different sections of our website, represented by dots.

Project Arachnid

The visualisation is limited to the top-level pages of our website, as it would look too cluttered were we to index every single page. That’s not to say we don’t have the data for visitor activity on these deeper pages.

The different colours and sizes of the dot represent value. We use blue to indicate a relatively low value visitor, based on what we know about them. In some cases this isn’t very much, but we can see where on the site they are, in which part of the world they live, and the referral source, among other things.

A bright red dot indicates that they’re a high value visitor. This could be because they’re looking at one of our bigger ticket items, such as in-house training, or it could be because we know a lot more about them. Purple lives somewhere inbetween red and blue, and as such conveys a medium value. Note that the colours are relative... let Arachnid play out for a minute or so to better understand how it works.

A white circle around the dot means that they’re logged in, which allows us to see their user data. This includes things like subscription status, purchase history, content contributions, and so on. It provides a much richer view of a particular user (although we've obscured some of the data to make Arachnid available for public consumption). We allocated values and multipliers to each visit, as you can see by clicking on one of the dots...

The values come from a blend of data from Google Analytics, and our in-house e-commerce and CRM databases. There's more detail on the Arachnid page. You’ll see that the average visitor to the blog is worth a mere £0.09, whereas somebody looking at the ‘Why Join’ page is worth £13.60. Multipliers kick in based on recency and frequency metrics, among other things. 

This sort of data is invaluable for marketing automation, which we’re going to be rolling out over the next year or so, and proactive customer service. We will be able to set triggers and alerts, based on real-time user activity. For example, we could instigate a live chat session, for high value visitors who are lingering on a particular page, and who may need help. Or we could send an email or training brochure to somebody who has been looking at our training courses. 

There’s a lot of scope to do smart things, based on real-time data. We remain bullish about prospects for publishers, so long as they think like retailers. It’s something that we’re trying to do, to make the most of our web traffic.

The project was the brainchild of CEO Ashley Friedlein and largely engineered by Chief Architect Tom Stuart. It allowed us to play around with HTML5, using the Canvas and WebSocket APIs. We recommend that you use Chrome or Safari to see Arachnid in action, as some other browsers do not yet properly support these new technologies.

Anyway, do check out Project Arachnid and let us know what you think. 

Chris Lake

Published 25 August, 2011 by Chris Lake

Chris Lake is CEO at EmpiricalProof, and former Director of Content at Econsultancy. Follow him on Twitter, Google+ or connect via Linkedin.

582 more posts from this author

Comments (14)

Save or Cancel
Will Grant

Will Grant, Web technology professional at Bitcala.com

Nice job guys - this is a great use of some HTML5 features.

Getting excited just thinking about the possibilities...

almost 6 years ago


Tim Leighton-Boyce, Analyst at CxFocus

Wow. I'm not so much excited about the visualisation, although that is very compelling, as I am about the calculations involved.

I'm particularly interested by the concept of using GA's $Index metric in this way. That seems to have disappeared from their v5 interface, and I'm hoping that's only a temporary thing. Your use of it here demonstrates a very practical benefit.

almost 6 years ago

Ashley Friedlein

Ashley Friedlein, Founder, Econsultancy & President, Centaur Marketing at Econsultancy, Centaur MarketingStaff


Yes, we did want to help show what is possible with HTML5. What I really wanted to do was to use WebGL (http://en.wikipedia.org/wiki/WebGL) to make it 3D as well so you 'fly' through and around it. And I also wanted to demo it using a touch interface. And, actually, I'd like to make it do things using voice commands (Google provide APIs for this) possibly with an 'assistant' like the Nabaztag 'internet-enabled rabbit'.

Basically I want it to be like a real life Minority Report / Ironman experience because it is eminently possible.

At the very least we'll have this on a giant touch-enabled screen in our new offices ;)

almost 6 years ago

Ashley Friedlein

Ashley Friedlein, Founder, Econsultancy & President, Centaur Marketing at Econsultancy, Centaur MarketingStaff


Yes, I quite agree that it is annoying that Google have removed $Index in their latest interface as we have found it useful.

Various of us are campaigning behind the scenes to get it brought back... watch this space.

almost 6 years ago

Jon Vlahhos

Jon Vlahhos, Manager at Coming

Outstanding. The possibilities.

almost 6 years ago

Matt Clarke

Matt Clarke, Ecommerce Director at B2B


almost 6 years ago

Seema Kumari

Seema Kumari, Head of Digital Marketing at Hearst Magazines

Very impressive...Love it

almost 6 years ago

James Gurd

James Gurd, Owner at Digital JugglerSmall Business Multi-user

Hi Chris,

Now that's a rather neat tool. I just opened up a new tab for a specific section of the site and then saw myself coming sliding in to the Arachnid visualisation. Apparently my content contributions are worth £801 - cash is fine.....

Next time I'm in I would quite like to get a some more info on how you're using it with marketing automation as that's got huge potential.


almost 6 years ago

Ashley Friedlein

Ashley Friedlein, Founder, Econsultancy & President, Centaur Marketing at Econsultancy, Centaur MarketingStaff


If you reload the Arachnid visualisation and revisit then your value should have gone up from your latest comment here... ;)

Ah, yes, Marketing Automation. We have big plans for that next year. But the dirty little secret is that whilst Arachnid might look quite impressive the hard part (which we haven't yet figured out) is what exactly to *do* with it (apart from put it in our office on a giant plasma screen)! We have plenty of ideas, of course, but nothing yet by way of a case study in how to turn something like this into something more valuable than a whizzy showcase....

almost 6 years ago


Traffic Parthenon

Understanding where your traffic is coming from, what their primary interests are and how they're are evolving is imperative for good customer retention as well as acquisition of new traffic sources.

This tool looks incredible, giving real-time visual updates of what's occurring. I'm really excited to see how this project will evolve.

almost 6 years ago



very impressive, when can this be expected commercially ?

almost 6 years ago


Justin Volpato, Founder at Rival Web Design

This is really fun to watch, I want to come up with a version for our ecommerce sites now.

almost 6 years ago


Antonio Meic, CTO at Kolektiva web d.o.o.

Awsome :)

almost 6 years ago

Peter Bordes

Peter Bordes, CEO & Founder at oneQube

very smart and great use of data visualization...

over 5 years ago

Save or Cancel

Enjoying this article?

Get more just like this, delivered to your inbox.

Keep up to date with the latest analysis, inspiration and learning from the Econsultancy blog with our free Daily Pulse newsletter. Each weekday, you ll receive a hand-picked digest of the latest and greatest articles, as well as snippets of new market data, best practice guides and trends research.