Data and Analytics

Talk tonight - "The Keys to the City of Knowledge" by Conrad Wolfram at Policy Exchange, calling for Computable Data. Once one got over the "Leader of company specialising in computable data search argues for more computable data to search" shocker ( ) there were some interest points made.

Firstly, he argued that computable data is about presenting the "data behind the data", for what he terms "citizen computation" - or "trying to get the answers, not other people's answers". Now this I agree with, as we note that Gresham's Law is increasingly applying to online data, ie bad data is driving out good. I also buy his argument that discussions about decisions is "where the sliders are on data models, not simple Black/White answers" that pass for public and media debate. The issue I have with this though is the level of maths capability required of everyone - and in the UK at any rate, maths and the whole STEM area is relatively lowly valued among the university educated (and relatively lowly paid in the UK), never mind teaching much higher levels of maths skills to les autres. Wolfram argues that Maths needs to be taught differently in schools, and that Computable Data now is like computers in the Assembler days, and we need need to get to "Mac" layer of computable data fast.

But that was all by the by, what really did interest me in the talk were three other points he made:

1. The Value Chain of Knowledge - here are some notes I made:

- Base data becomes less valuable, ability to analyse data grows in importance.
- Curation of good data will rise in value
- Bandwidth no longer = $ so everyone now pumps a lot out, so the pressure is on curation not creation (data, data everywhere)

Thus A Guiding Rule: Compositional knowledge >> dead information


2. What Data is most likely to emerge first? Mainly data that is either publically funding, or largely yours, eg:

Publicly funded R&D - should be computable
You! (Facebook etc) - whether you like it or not, it will be available
Governments - still lots of info locked up, and they are still biggest users of own data.

Areas that are most likely to be early data sources are:

- Health - biggest gainer due to diagnosis improvement, which is inefficient & labour intensive. Sensor based medicine is coming. Also data on relative hospital performance (Tripadvisor for your bypass op, as Susan Calman may have put it)
- Social security - reduced costs of service, also easy to spot fraud from data if looked at in enough data
- Economic data - publically funded
- Educational data - eg school performance (the underlying data, not simple tables), University R&D
- Transport - for co-ordination

He points out it is necessary to unpack simple metric data, (exam results, school league tables) as they are both easy to game and not hugely informative. Work off computable data, not metrics
Corporate information - a lot is missing ( very private)


3. National Productivity - the Computable country needs a computable knowledge economy, which requires:

- government computable data layer
- government funded entities must make data available
- smart disclosure of legally reported data
- maths education

There was a rather fascinating angle on this, during the question phase. Essentially the discussion had moved to getting corporate data out from behind the firewall (he argues for a VRM-like ownership of your own data) but the point was made that privatisation is bad for Big Infrastructure and some other areas, so maybe a Computable Country should re-nationalise some areas as the data creates more value in a public entity than in a private one*.

Now THAT is an interesting argument, if one started to calculate the value of the chained data....the economics of Open Data may move from "Interesting" to "Critical"

(*Somehow I don't think the Policy Exchange would have intended this, being a right wing think tank )

I attended the WebScience Trust event yesterday on Data Observatories, a very "Motley Crew" (As Dame Wendy Hall put it) of people who are active in the space of Web data analytics etc. It was a good session as it had represention from various academics studying the area from Oxford, Warwick, UCL, Cardiff Universities etc, a number of non-profits, research groups, and a few companies operating in the area (like us). I took copious notes which I am too lazy to write up but as a very good summary of the day is over here on the JISC blog why repeat the good work of others.

Apart from being fascinating to see how many ways so many other people are attacking this emerging and extremely varied area, my main takeaways of the day were:

1. The impact of the Internet of Things - Prof David de Roere of the Oxford e-Research Centre gave an interesting talk on current activity in the WebScience Trust, but one slide - a good old 2 x 2 so beloved of consultants like us (see above) - on the structure of the work caught my eye. After some discussion with David, I have helpfully (ahem) added another aspect we are looking at after eating too much Raspberry Pi, and (with apologies to David) have amended his slide above. It also allows us to re-introduce a long forgotten Broadstuff character, Mrs Fridge...

2. Who shall watch the watchers - a general issue around the binary nature of this emrgent technology. On the one hand is it's fascination and utility in so many spheres, and yet on the other its risks (privacy, snooping etc) was a theme throughout the day. There was a presentation by Edelman and the University of Southampton where they showed realtime network nail-ups for people re-tweeting on Twitter so you could see who were orginators, influencers, amplifiers etc. Another member of the audience asked the question that given one could do this sort of analysis, and given that sock-puppeting/astroturfing is rife, is this technology being used to to actively direct sock puppet activity. Now we've been pondering exactly this since last year, when we were asked if there were ways of looking at social media sites and see if "people" bad mouthing someone on these sites were sockpuppets or real, and thus were the sentiments genuine or contrived by competitors.

3. Shocked they were shocked - Switch Concepts Ltd told us that ‘Google has a file the size of an encyclopedia on everyone in this room', and went through what they can collect and how. That's not what shocked me, what shocked me was that quite a few people in the room - experts in this space all - did not know this. If some of these people don't know realise (i) what the big boys do and (ii) the scale they do it at, god help regulators, politicians and the like - never mind Joe Public!

One person suggested we needed to derive a "3 Laws of Robotics" for web data collection and analysis companies. Amen to that!

Also, it was interesting to see not just the mix of hard scientists and "soft" scientists, but the segue of hard scientists doing soft science, soft scientists doing hard science, etc - a Motley Crew indeed....

The xkcd cartoon above says it all, re: my earlier post on the increasing trend towards "fact free analysis" in The Daily Meedja.

Interesting article in HBR over here, implying that "big set" data analysis reaches limitations to its effectiveness fairly fast:

Firstly, remember the Netflix competition to improve their algorithm:

Five years ago, the company launched a competition to improve on the Cinematch algorithm it had developed over many years. It released a record-large (for 2007) dataset, with about 480,000 anonymized users, 17,770 movies, and user/movie ratings ranging from 1 to 5 (stars). Before the competition, the error of Netflix's own algorithm was about 0.95 (using a root-mean-square error, or RMSE, measure), meaning that its predictions tended to be off by almost a full "star." The Netflix Prize of $1 million would go to the first algorithm to reduce that error by just 10%, to about 0.86.

In just two weeks, several teams had beaten the Netflix algorithm, although by very small amounts, but after that, progress was surprisingly slow. It took about three years before the BellKor's Pragmatic Chaos team managed to win the prize with a score of 0.8567 RMSE. The winning algorithm was a very complex ensemble of many different approaches — so complex that it was never implemented by Netflix.

I recall the guys at a UK Netflix lookalike, LoveFilm, telling me that about 5 factors got the 80/20 prediction, so there was clearly a massive falling off in effectiveness as data analysis complexity increased.

But that is predicting intention behavior of demand, so what about retention - is this any easier, after all, one should have bucketloads of data and llots of historical nous dealing with ones's customers? It would appear not:

A study [pdf here] that Brij Masand and I [Gregory Piatetsky-Shapiro] conducted would suggest the answer is no. We looked at some 30 different churn-modeling efforts in banking and telecom, and surprisingly, although the efforts used different data and different modeling algorithms, they had very similar lift curves. The lists of top 1% likely defectors had a typical lift of around 9-11. Lists of top 10% defectors all had a lift of about 3-4. Very similar lift curves have been reported in other work. (See here and here.) All this suggests a limiting factor to prediction accuracy for consumer behavior such as churn.

(Lift is the ratio of actual churn vs the churners in the "big data" analysis, so if a "Big Data" algorithm predicts a list of customers that has 20% of actual churners in it, vs an averagel churn of 2%, that is a "Lift" of (20/2) = 10. That still means the list is 80% wrong though.

And how about predicting Ad effectiveness?

The average CTR% [Click Through Rate] for display ads has been reported as low as 0.1-0.2%. Behavioral and targeted advertising have been able to improve on that significantly, with researchers reporting up to seven-fold improvements. But note that a seven-fold improvement from 0.2% amounts to 1.4% — meaning that today's best targeted advertising is ignored 98.6% of the time.

(Actually, 0.1% sounds high to me, I'd think it was almost an order of magnitude lower nowadays)

Interestingly the article predicts Big Data will help more in the emerging services:

Big data analytics can improve predictions, but the biggest effects of big data will be in creating wholly new areas. Google, for example, can be considered one of the first successes of big data; the fact of its growth suggests how much value can be produced. While analytics may be a small part of its overall code, Google's ability to target ads based on queries is responsible for over 95% of its revenue. Social networks, too, will rely on big data to grow and prosper. The success of Facebook, Twitter, and LinkedIn social networks depends on their scale, and big data tools and analytics will be required for them to keep growing.

Google's reducing profits may be a sign that it's advantage is coming to an end, which - if the view here is right about dimiinishing returns - does not augur well going forward. Also, as they warn:

if you're counting on it to make people much more predictable, you're expecting too much.

Quite. And yet, and yet...one more tweak...

Also, bear in mind there are some big impacts in pivotal areas. A small change in a competitive area like say churn can have tremendous impact, especially if one is in a zero sum game (eg mature mobile phone markets), and played over multiple cycles. For example, assume 2 companies with equal market share, both with 20% churn. A very simple simulation will show if one player can get a sustained reduction of 1% of that 20% churn - ie to 19.8% in monthly customer retention over say 36 cycles (3 years) will give that player 53.5% vs the others' reduced 46.5% share - a shift of 7% points of market share - not a bad structural change in any saturated market, in fact shifts like that can drive competitors out.....

The answer, as always, is to accurately understand the costs vs the benefits.

SAI:

Facebook is in talks to buy Opera, the company behind the Opera web browser, PocketLint reports. Opera has both a mobile web browsing app and a desktop browser, and it's an alternative to Internet Explorer, Mozilla's Firefox browser, Google's Chrome browser and the Apple Safari browser. Opera says it has more than 270 million users on its browser. Additional sources told The Next Web that it's looking for buyers and currently has a hiring freeze.

Its an interesting strategy for 2 reasons:

(i) Facebook is already doing every sort of datamining it can think of (and probably that its Genetic Algorithms can think of), and apart from your credit card the next best way of knowing all about your intentions is your browser. That is whay, by the way, thers is no way I'll use Google Chrome (though it seems my privacy concerns are not a worry of the Great Masses out there, with Chrome now having the largest installed base).

(ii) One big factor in The Great IPO Tank was the worry about it's inability to monetise Mobile, and Opera is the smartphone browser of default - so that's an insdtant window on the habits of millions of (relatively wealthy, early adopting) mobile users

I will immediately disconnect Opera from my Smartphone if Facebook get their hooks on it. And if they issue a credit card....

Interesting post from Nic Brisbourne, summarising a GigaOm article on Big Data case studies. Like Nic I am dubious about some areas so applied a classic 2x2 analysis (see diagram above) to parse where I thought value might lie. His summary is below, my comments are in italics:

I was interested when GigaOM put up a post this morning titled 10 ways big data changes everything. I read through the ten ‘case studies’, and summarised them below. I’ve put my opinion on the trend in italics after a summary of the GigaOM case study. There was, in my opinion, a lot of fluff in the examples they chose, and of the ten there were only two that really stood out to me as areas with the depth and breadth to be home to multiple successful startups, and they were business intelligence applications of big data and virtual assistants.

1. Mining social media and other music sites to predict the next Lady Gaga – GigaOM highlights Next Big Sound as a key company in this market. Will this idea work? So far Next Big Sound has ‘two undisclosed major record labels’ as customers for this service, making it too early to tell. (There was a fascinating book from a few years back called The Predictors, where one of the case studies was the conflict between the analytics vs the culture of the "personal nature" of the similar movie industry)

2. Finding opportunities for energy savings by comparing consumption patterns – observing yourself is powerful in any context and making consumers aware of how the different things they do effect their energy consumption is no different. Comparison with other others is one way to drive behaviour change, but simple displays showing realtime energy consumption may have more impact. (I think there is an opportunity for any business with zillions of fungible datapoints, but the value per is quite low)

3. Virtual assistants – software tools that help us with the more routine tasks we have to deal with. Siri is perhaps the most famous example, but there are many others. GigaOM chose to highlight a datacentre management assistant technology called Autopilot from a German company called Arago. Virtual assistants combine analysis of large volumes of data with artificial intelligence and will, I think, have a huge impact on many aspects of life over the next ten years. (The overall value in this market is potentially huge as it encompasses every type of business activity, and it is doing a relatively high value task - aggregating and simplifying)

4. Data fuelled recommendations – GigaOM gives the example of Foursquare which is making recommendations based on the data it gets from its users checking in and leaving tips and comments. Recommendations get more powerful when they have more data and I think the future will see recommendations based on data from multiple sources – Foursquare, Facebook, Twitter, and anything else that is getting traction with consumers. Ultimately I see this as a subset of the 3. Virtual assistants. (I agree with Nic, these are niche applications, and risk being one trick ponies in a market where real value comes from multi-factor aggregation - but it is a very large market they can impact so I'd say its my 3rd "interesting" sector)

5. Tracking disease epidemics – GigaOM tracks the story of how Twitter tracked a recent outbreak of cholera in Haiti and how that can be of use to aid agencies. There are numerous examples now of social media being used to good effect in crisis situations which is great. I’m not sure if it really counts as big data though. (Niche application of "Business Intelligence tools in my view, and fairly small as a market I suspect)

6. Business intelligence – GigaOM describes how a startup called Parse.ly is helping publishers analyse publishing data to see what content is driving traffic and predict what might work in the future. I think business intelligence is most obvious near term application for big data and that the most obvious short term startup opportunity is technologies and services for this market. Unsurprisingly this is where most of today’s hot big data companies are playing.

7. Mining cellphone billing records – GigaOM describes a number of use cases for this data including helping with malaria education in Kenya. I see cellphone billing records as one more dataset that can be minded to good effect. It is no different in principle to social media data and ultimately the successful services will be the ones that are able to draw value from multiple datasets. (This in my view is like utility analysis - huge amounts of data but relatively low value per unit)

8. Using data to predict and create video hits – Netflix is the case study this time. I am sure that by mining user data Netflix will be able to better guess how popular new video content will be, but it isn’t clear from the GigaOM article how much better it will be than traditional human based analyses which focus on success of similar content in the past.(See my notes on music - but this is a bigger market)

9. Touch screen interfaces for interacting with big data – touchscreens offer new possibilities for manipulating large datasets with novel graph and chart interfaces. I’m not so sure about this one. Funky charts have value in presentation situations, but I think that most of the insight from big data will be gleaned by analysts who work by looking at different cuts of the data and zomming in and zooming out.(I see this as the UI end of any of these systems, I don't think that Big Data will drive me to buy a second tablet - but it may drive replacement of many industrial terminals and PC screens, but IMO it is just upgrade/cannibalisation of the screen market)

10. Hospitals using big data to improve efficiency – GigaOM describes how electronic patient records have delivered some improvements, but that there is a way to go. There is huge opportunity for IT enabled cost savings in healthcare delivery. The challenges are more organisational and bureaucratic than technical though and the opportunity for startups is to build services highly tailored to the healthcare market in which they work rather than exploiting the latest big data technologies.(I see this as a niche application of Business Intelligence/VA/Utility monitoring, but as Nic says the issue here is data privacy)

Interesting its now suddenly so popular, when we built the data analytic MDE in 2007, theer was very little interest in deep analystics of the social media datastream, now there are (I heard at the FT Conference) "50 starving London startups looking for funding for Big Data".

Another point made by Nic is that:

One interesting thought which occurred to me whilst compiling this list is that it will likely get much easier for companies to realise the latent value in their data. Over the years we’ve seen a lot of plans from companies that put some value on data generated as a by-product of their main business and only a few of them succeeded in monetising that value.

With many years experience in this game i'd say the potential is there, but often the culture, mindset and busines models are too differemt/difficult for the company to easily assimilate them. Still, there is nothing like a spinoff.....