Lies, Damned Lies, Klout Scores and Vanity Metrics
‘Just because everything can be measured, doesn’t mean that you should measure everything’
During the summer the football club I support (Arsenal) offloaded one of their squad players, a young Brazilian player called Denilson. This was a move that resonated well with most supporters who had grown tired of his inability to exert any influence on a match. He played in a central midfield role, essentially as a defensive midfielder, but most of his passes proved ineffective for an essentially attacking team.
Despite that, his OPTA stats scored quite highly. (For those non-football (soccer) fan readers, OPTA are the number one organisation measuring and compiling data and analytics around sports.) In fact, a couple of years ago he was the highest ranked defensive midfield player in the Premiership…according to the stats…yet his ability to influence the outcome of a game was almost zero. The actions that contributed to his score – blocks, interceptions, passes etc – were many, yet they had little overall impact.
If he was a social media user his Klout score would have been very high – yet most people connected to him would say he had little influence.
Continuing with the football theme, the other week I watched the live game between Fulham and Tottenham. The match stats show that Fulham had the majority share of possession, that they had 26 goal attempts against only 8 by Tottenham, and 11 corners against only 1. From those stats you may assume that Fulham won the game. But they didn’t. Tottenham scored from 3 of their 8 chances whilst Fulham scored from only 1 of their 26.
If soccer teams had Klout scores then Fulham would have had a much higher score that day.
The quote at the start of this blog was taken from a presentation by Tom Farrell of Paddy Power that was given at a conference – Future Digital Strategies – that I chaired last week. Unlike my usual manor of recruitment and HR this was a chance to mix with digital marketers working for companies ranging from Expedia to Disney, Virgin Atlantic to Facebook. As chairman I was able to ask questions, and I asked most of the speakers about influence…how they identify influencers.
For commercial businesses the key influencers mainly used to be journalists and editors – trade press, local and national press and broadcast media – but in the new social media landscape the position is less clear. Everyone I asked was looking to identify bloggers and tweeters who had reach and impact…and they all seemed to use measures such as Klout and Peerindex as a starting point. Not as the final decision maker but as an indicator of who to investigate further.
Now there’s been a lot written in the blogosphere, on Twitter and Facebook too, about these vanity metrics and ‘influence measuring’ analytics and most of it has been negative. It is not my intention to use this blog to question their validity or credibility, their accuracy or otherwise…but really to look at our need to measure things, and why these tools are relevant.
After all is their basic premise that different to what we already know?
Ask any 14 year old how they can get to do cool things and they’ll probably tell you it’s by hanging out with the cool kids. If you want lots of friends and parties…hang out with people who’ve got lots of friends because they’ll tend to get invited to lots of parties.
And if you want to be noticed have some interesting things to say…be entertaining and informative and people may listen.
Groups tend to form around noisy people who know stuff…or sound as if they know stuff.
So far so obvious…hardly ground breaking wisdom, but it seems to irk many.
It’s not as if the ‘thought leaders’ around social media didn’t have this figured out ages ago.
About 15 months ago a tweet from one of the medium’s undisputed thought leaders appeared in my timeline asking a music question about a specific band. I knew the answer and tweeted straight back. About 10 minutes later he tweeted his thanks to four people – not including me.
I was curious. I did a quick search and saw that I was one of about 20 people that had answered him…yet he acknowledged only four of us. It wasn’t in chronological order as a couple of the people he thanked had answered well after others…and my response appeared to have been one of the first. The thing that set the four out from the rest was their number of followers and volume of tweets…he thanked those with the largest followings, who happened to be the ones who tweeted the most.
On a similar theme, back in June I ran some ‘influencers’ through Tweetstats. I used the HR Examiner lists for top online influencers in HR and recruitment and there were 15 names (some people appear on both lists) – I used Tweetstats to try and measure conversation. I was looking for the % of their tweets that were replies to another tweeter as opposed to a broadcast statement, a blog link or a quote.
A few weeks earlier I had seen Scott Stratten present and he had recommended that @ replies should be around 70% for engagement and conversation. Only one of the 15 I checked had a reply ratio in excess of 50%…most actually struggled to get over 30%. The most damning stat of all was that the number one online influencer in recruitment that month (I know the positions change each month) had a reply ratio of less than 1%!
That’s right…pure broadcast, no conversation. I sent him a couple of messages…needless to say, I didn’t get a reply!
But still we look for ways to measure things.
From sport to economics, we look for a way to make sense of what’s happening by measuring and comparing statistics. Some methods may be less reliable than others – in my two football examples neither Denilson nor Fulham got the outcome that the figures may indicate – but it doesn’t stop us analysing and measuring.
And we’re still making sense of this new social media landscape.
What is important to one business will be different from another. We have no clear definition of online impact or influence…nor is there uniformity about what businesses may want from engaging with ‘influencers’.
Some could just want a message spread as far as possible whilst others could be looking for a peer recommendation from someone who has impact and can inspire action.
And there’s probably a dozen algorithms yet to be created that will measure these things in a multitude of ways.
So no, current measures aren’t perfect.
They may not recognise, for example, that someone like Gary Franklin runs a group (FIRM) with 4000 members, all of whom are influenced by him. And they don’t recognise that my day job may well give me ‘klout’ with many more people than just my twitter following – after all, in the old order of print media it was magazine circulation that was important, not how many people actually read a specific journalist.
Nor do they recognise offline activity that feeds back into online, say a TV appearance (yep, I’ve had one of those) or conference/unconference presentations and chairing.
Its early days though and I’m sure there will eventually be both a definition and a measure of influence that is universally accepted. And until then measures like Klout and Peerindex will be around because most people want to know these things!
And businesses still want to know who their key influencers are…