Tuesday, July 7. 2015
The Augmented Tennis view - today on a screen, tomorrow in your retina?
Last Thursday I was lucky enough to visit IBM’s Big Data Crunching Operation behind (amd underneath) the annual Wimbledon Tennis tournament, courtesy of fellow -Tuttler Andrew Grill of IBM (who says nepotism never pays) and also courtesy of the knowledgeable enthusiastic (despite me being the Nth tour guest of the week) Sam Seddon who escorted me through the databunkers.
Now I am far from the first Old Tuttler blogger that Andrew has invited, my colleague Davit Terrar went last year and Neville Hobson went on the Tuesday last week. Thus the “what happens behind the Datascenes at Wimbledon ” is already more than well covered, just look at David’s blog post from last year - to quote:
Sam [Seddon] has a team of around 200 supporting Wimbledon using an impressive array of terminals and technology on site, supplemented by an enormous amount of Cloud compute power from data centres in Amsterdam and the USA. They are providing a service to the All England Lawn Tennis Club to help make Wimbledon the premier sporting event, but in doing so they are serving the audience at the ground, fans around the World, the radio and TV broadcasters of the event, the event sponsors, the Club itself, and even the players directly
David did a good analysis of the overall IBM operations last year, and had some cracking photos - so what can I add to this? Probably the best is to talk about it from the point of view of things I know about in some detail – in this case, high end datacentre infrastructure, real time big data & analytics, and system automation (in this case, Watson)
High End Datacentre Infrastructure
Firstly, from the point of view of datacentre infrastructure, this is a high pressure operation – real time, high visibility, peaky data flow. High embarrassment if things go wrong. Also, the web based systems get quite a few attacks 24/7. And as well as running all the tennis data feeds discussed below, IBM also run the ticket checking and security operations of the whole site.
Secondly, from an Operations & Logistics point of view this is further complicated as the whole setup is also completely transient – for 2 weeks a year the Tournament Tennis circus descends on Wimbledon. The entire IBM system rolls in, the porta-datacentre and its operatives are wheeled into the empty bunkers from locations elsewhere in the world (along with all the TV broadcast trucks, meedja personalities, commentators in their glass boxes, tennis line callers, buckets of strawberries and flagons of Pimms etc etc) - and then two weeks later the whole panjandrum disappears and the whole site is emptied and mothballed again for another year.
This is non trivial stuff, kudos to IBM for making it so seamless!
Data Flowchart of all the IBM Wimbledon dataflows - camera had too much Pimms clearly
Big Data & Real Time Analytics
From a “big data fan” viewpoint there are 3 main data handling operations (see flowchart above, and weep) - in summary::
Every shot played by every player is captured, not just physically but with rich metadata – tennis experts analyse every shot and record what it was, why it was played, did it work/fail. Interestingly IBM captures data from every major tournament so you can pick up a rich data picture of every player and the permutations of their matched with others.
Added Tennis Metadata
Not only player match stats, but their historical metadata and also Wimblestats are collected – for example, veteran Leighton Hewitt hit his 1500th ace the day I was there, and they can play comparison games with stats back to the 1870's
Social Media Analysis
The 3rd operation is the monitoring of Twitter feeds off the Gnip firehose. To my readers much of this is familiar ground – find, scrape, focus, process, analyse, use output to inform and focus further coverage etc. (David covered it last year if you’d like more detail) However, there were a few counter-intuitive things I learned:
- Court Prowess and Twitter are loosely linked – Rafa Nadal was the most talked about all player week, despite being an early exit
I was a bit taken aback by the last one, but I guess when you think about it, in media there is a general trend that where porn blazes a trail, advertising soon follows
To a data modelling/simulation wonk like me this was all fascinating, and I hit Sam with all sorts of “what ifs” but he reminded me that It's all about understanding the business of tennis - that you have to look at who the customers are, and what they want. I may want to run the Moneyball Crunch of all crunches on the tennis world and create virtual avatars so I can play Bjorn Borg against Rafa Nadal, but realistically the data is required by:
'You know my methods, Watson.'
….said Sherlock Holmes in The Crooked Man (he never said “Elementary, my Dear Watson), and to me this is what is starting to set apart what IBM is doing vs other big iron data operations.
For those who don’t know about Watson, it is a very powerful Machine-Learning system initially focused on natural language processing. It’s key role is Question Answering, so underlying the language processing is are powerful deduction chain algorithms. In short, if you can feed it information and deduction methods, it can form its own hypotheses and reasoning chains This makes it useful for quiz shows (it won at Jeopardy in 2011), medical diagnoses, and – well, tennis among other things.
Right now it is being taught the basics of Tennis geekdom (What is love? – answer – a score of zero), being force fed Wimbledon tennis history (to draw parallels etc) and is starting to look at the social media chatterflow. But if you look at the data processing going on at Wimbledon, there are a LOT of workflows, methods and deduction chains that are ideal for a natural learning machine learning system.
Data for Watson - feed me, Seymour....
So when afterwards the IBM people asked me what my main thoughts were from the tour, I told them that in my view, many of their 200 or so operatives here will disappear in the next 5 - 10 years as Watson and other AIs start to take over the rote data processing going here, and then the less rote, and then the really high value add..
And yes, it felt traitorous to say that, sipping Pimms and watching the bright young meedja things gambolling on the grass, but at the end of the day the automation of dataflow & analytics is just the next step in an ongoing process.
Wimbledon is just part of a trend I see everywhere right now, wherever previously unstructured data is being digitised. First comes the rudimentary digitisation of hitherto unstructured data (often from manual or semi manual processes ), then comes the early analytics which creates huge value from the low hanging datafruit, and increasingly we now are starting to see the application of automation, both in data capture (IoT) and analytic reasoning (MI/AI). The future will come one automated data feed and workflow method at a time.
Capturing courtside data, play by play. How many years to full automation?
Automating the Tennis Tournament Production Factory?
Wimbledon makes 70% + of it’s money from selling Media rights for the two weeks of the tournament (and as more and more courts get their own datafeeds this revenue will only increase).
And as all those assets, that span acres of prime London real estate – the main courts, the techie bunkers and media circus rings, the player palaces, the buildings for the huge catering and logistics operations – all largely lie fallow for 50 weeks of the year. Wimbledon will always need to maximise income and minimise costs from those 2 weeks a year.
Thus they will inevitably be pushed into the economics of ongoing automation of Tennis Tournament production. The need to both reduce operating costs and produce ever more value-add from the datafeeds is probably inevitable. The competition for media $ will only intensify, other tournament operations will up their value add year by year. So that means more and more AI for the data handling and analytics tasks, to produce more value add for Wimbledon and its customers.
Unless, unless…..the unquantifiable human input can be shown to make the difference between an experience that will delight, versus one that is artificially inserted.
Maybe automation will have it’s limitations, we may not like a smart Watson, but instead a foil Warson to our Holmes - Quoth Mr Holmes:
A confederate who foresees your conclusions and course of action is always dangerous, but one to whom each development comes as a perpetual surprise, and to whom the future is always a closed book, is indeed an ideal helpmate.
We shall see. I’d place my money on Smart Watson though....
Monday, July 6. 2015
From the "Well, we would have told you so if you'd asked" Dept:
For those who are surprised and shocked by the popping of the Chinese stock market bubble, can we point you to the Broadstuff Bubble-O-Meter which says that, when taxi drivers give you stock tips and private citizens gamble their savings it really is the endgame of the bubble, and the next step is "pop"
Addition from the "we didn't tell you this though":
And right to the end, those with financial interests in the bubbletime, and their mouthpieces, will insists that there is no bubble.
This is a public service announcement
Wednesday, June 3. 2015
As I can’t get alone to InfoSec15 this year, I thought I would get a flavour of what’s going on by pointing our experimental text mining tool in that direction. See the graph below. Bruce Schneier and John McAfee seem to have the influence they deserve, followed by BSOD (is Windows 10 here already? ) and “trashier tearaway”, which seems to be a reference to an article in The Register about the perils of allowing BYO-IoT-D (Bring Your Own Internet of Things Device!) into secure networks. also, full marks to whoever thought up “malvertising”!
Hope to get along next year and find out what’s going on in person.
(Apologies for my earlier, half-formed draft of this post – now deleted – just proves I really am too old to multitask!)
Friday, May 15. 2015
Andrew McAfee in the FT explaining to the FT audience what has long been blindingly obvious to most everyone else who is living through it - if you automate away the high value jobs and all thats left is low value low skill jobs, average wages and living standards fall, and inequality rises.
He uses the though experiment of a 2 industry economy, one with hi tech, high wage manufacturing and low tech, low wage dog grooming. Over time the high tech labour gets displaced by automation and more people become dog groomers, so average wages in the population fall, those left in manufacturing get richer, inequality widens etc etc. He then observes, as if its a massive revelation, that:
This example is obviously trivial and contrived, but it does include some trends that we see in the actual economy — lots of jobs growth in the low-productivity service sector, sluggish overall demand growth and lots of automation in manufacturing. It also helps explain why corporate profits are so high these days: the factories, after all, have maintained their revenues while halving their payrolls.
No shit, Sherlock!
I guess what's really sad is you probably do have to use a 2 industry model-as-parable to explain it to FT reading, degree educated, often economist types. Piketty tried it the big data way but to little discernable impact to date. They analyse the trees to death, but all too often cannot see the woods.
Now, why not do a little sociohistorical analysis too, and look at what happens when unemployment and wages fall and inequality rises, and a large section of a population is discontented and feels it has no future. Guess what - the economic environment stops being like the one we have at present.
Here's a hint - start with France, 1789, and continue reading about "transformational change", mob revolt style ....the Deus is ex the Machine Age.
Thursday, May 14. 2015
Article from The Economist echoes something I've been thinking increasingly - there's not a lot of new stuff coming out of guruland these days. I thought it was just me getting old and cynical, as increasingly everything "New" just seemed to be a rehash of something from 2 decades ago or more. I had started assumed New Gurus (Gurii?) were mining old, out of print books and rehashing old whines into new booklets.
The Economist has noticed the same, and is looking for reasons why - they think that:
Ah, so its a generational fin-de-siecle then.The Economist also argues that its becoming harder to mint new Gurus now as (i) the world is more interested in Big Data than Big Ideas, and (ii) there are a million Mini-Gurus blooming in the blogosphere. Also companies are increasingly trying to be their own Thought Leader.
To sum up, they note that the Guru industry itself is not exactly behaving like one of these fast moving, always changing, disruptive modern industries that the Gurus bang on about:
...considering the resources that are devoted to thinking about management, it is remarkable how much virgin territory remains. There are still no Chinese management gurus to challenge the leadership of the ageing Indian establishment. There are still no serious books on what the internet economy means for the boundaries of the firm or markets for talent. The guru industry seems ripe for disruptive innovation.
I did read a study several years back, some Swedish researchers looked at what makes a Guru - and one of their major conclusions was that a Guru is selected and anointed by kingmakers, just being smart and original isn't enough. In the old days these kingmakers were usually publishing houses who would then market the cr*p out of said Guru's tome/s . But, as with the music industry, that media model is largely gone - there just isn't the ROI to hugely promote modern Gurus. There is an analogy with the Music industry, where being an original talent also was never enough, the game was always about getting the A&R - and so the oldies in Guruland and Musicland keep a high mindshare as the total cumulative spend on their PR dwarfs the trifling modern amounts spent so the Old Guard have the bg billings, the best new talent gets as far as it can without the Old Skool PR $ into specific verticals, and the Long Tail's output is largely irrelelevant (or gets mined by the better known).
Wednesday, May 6. 2015
Do you remember all the predictions that "email will die" and that social tools will replace it with "ambient intimacy" or similar?
Turns out that not only is it very unlikely email will ever die (see my argument re Riepl's Law on that score) but is also highly likely to grow, not shrink.
The reason for this is due to
(i) The sheer volume of comms on the new systems has just reproduced email's problem - ANY system, email included, is effective when the volume of messages is manageable, and overload looks the same on all of them. Except email has better timestamping, search and storage as its been around longer.
(ii) But email as the new killer app is die to the proliferation of all the social tools delivering said Ambient Intimacy, and thus fragmentation of the medium. There are now so many of them, that the Ambience is broken up into a veritable Tower of Babel of different protocols. The requirement is increasingly for one system to find them, rule them all, and in the darkness (of the communications server farm) bind them.
And guess which system is the most ubiquitous, flexible, integrateable, multi-capable to do this?
So that inbox of yours that overfloweth - the net effect of all these new social tools will just be to make it worse.
You heard it here first...
(by the way, you can subscribe to get Broadstuff via email....)
Thursday, April 30. 2015
Everywhere you turn there is this focus on the "Attention Economy", defined by Wikipedia as follows:
..content has grown increasingly abundant and immediately available, attention becomes the limiting factor in the consumption of information. Attention economics applies insights from other areas of economic theory to enable content consumers, producers, and intermediaries to better mediate and manage the flow of information in light of the scarcity of consumer attention
Which is all very well, but what has really occurred is an arms race between various service providers to divert your attention to their new new thing, rather than any others, and certainly not to the (slightly dull) thing you probably really should be doing.
I call this the Distraction Economy, and its really an anti-economic effect as:
Taking these in turn:
Accrete no or very little Value
There is an infinite array of diversions seeking to distract you. The internet is always wrong, there is always one more interesting tweet-link to read, one more comment for someone's Facebook wall, one more go on the game du jour. And what are you losing - they are all free, right? Well that is the billion dollar question. What is the value of your time spent on these distraction? There are two ways of measuring this:
Problem is that, for the average joe (or josephine) in the New Digital Economy the Conversion Value is near zero, as is the Conversion Rate, so this is an essentially zero-value accretion. So the +ve value is near zero
However, there is a negative opportunity cost - chances are most distractions reduce your own time and effectiveness actually spent on valuable tasks - i.e. unless you have zero valuable tasks to do, paying attention to the "attention economy"'s products is going to cost you (see part 3 below)
First came the idea of the Great User Experience, then when that arms race was tailing off came Gamification, now that is reaching its limits there is Addictification - the best and brightest minds of a generation are being used to try and ensure that people will pay attention to non essentials. Neuroscience, behavioural science, mathematics and a fistfull of 'ologies are being used to try and make this or that piece of digital bubblegum register in people's attention-span and grab that 15 minutes of fame.
As noted above, time being diverted is very unlikely to be accretive in itself, and is highly likely value destructive as:
(i) Less time being spent on valuable tasks - that has to have an opportunity cost impact in the medium and long term
(ii) It has been shown, in study after study, that distracting yourself from a task reduces your efficiency in performing that task, plus also imposes a "setup" and "teardown" time penalty as your brain switches over from task A to task B. The worrying thin about the distraction economy is that it feels like you are more effective, when in fact the hard metrics show the opposite.
What's the answer?
Firstly, it is to recognise that most of the bright shiny products of the "attention economy" are not there to make you money, they are there to make money out of you.
Secondly, to recognise there is an entire industry out there trying to make these grab your attention, with a miniscule industry building antidotes.
So the only real solution is to time-limit the channels - turn off ambient alerts, make a time to do emails/twitter/facebook etc, and keep to it.
Wednesday, April 15. 2015
TS Eliot wrote that the world ends not with a bang, but a whimper. It is our observation that the worlds of most monopolies effectively end with regulation of some sort or other, either breaking them up , forcing access, or regulating away super-profits (or some combination). And now, after several years of rumblings, the EU has finally started to look at Google. Today Margrethe Vestager, the European Union competition commissioner, announced an investigation into Google's practices for favouring its own shopping sites in guiding searches - NYT:
Google has been accused of these things for quite a few years, and there have been rumblings of EU action for some time so the only question in our minds is "why now". The NYT notes that:
“The decision by the commission to position itself as the lead competition authority for the digital age may trigger anger among some U.S. politicians
It also allows the EU, currently embattled by internal accusations of being sclerotic, unrepresentative, comatose etc etc to actually look like it is taking a lead in Doing Something in an area of considerable concern to EU technology firms.
The EU is also considering looking at Google's practice with Android:
The European Commission also said on Wednesday that it was stepping up a separate investigation into whether phone makers that agree to use Android — and that also want Google applications like YouTube — face contractual requirements to place those applications and other Google-branded applications in prominent positions on a mobile device.
If these are proven true, it will be clear that Google, a noisy advocate of net netrality on others' platforms, does not practice net neutrality on it's own.
Throubles usually come in threes, they say - any bets on an EU investigation into tax avoidance practices by US multinationals in Europe soon?
Tuesday, April 14. 2015
Fascinating chart in HBR (above) on the relative change in valuation of Brands and Customers since the emerge of Digital monitoring and social media as it becomes easier to measure customer behaviour and brand impact, and to know which half of advertising works:
And the results...
Oh dear - the demise of Brand Marketing will no doubt fill millions with gloom......
Monday, April 13. 2015
Went to Chinwag's Fintech event last Thursday to catch up on what's going on, well in Fintech really, looked like an interesting variety of speakers. Notes follow below:
To kick it off, Cass Professor Gianvito Lanzolla set the secene with a discussion of the current structure of technology change in Finance, he looked at the drivers of change
1. Tech drivers in the near future
Dave Birch of Consult Hyperion followed up with his view on what people are doing today
Tech in banking is a "William Gibson" time (aka future is here, just unevenly distributed). The Meme of the Moment is around "mobile", but what that seems to mean in practice is contactless mobile devices - which is hardly disruptive. Dave introduced the concept of "Chromewash" e.g. - Apple Pay (today's darling) runs on "yesterday's rails", i.e. merely a continuance of what has come before but overlaid with New New hype.
He looked at revenues from European banks - c 50% comes from Interest, Transactions c 30 %, Fees c 10%, Other c 10%. The raft of Fintech innovators are already in most obvious spaces of stuff banks do, so profit pools most at risk are the large ones - loan areas (Interest revenue) and Transaction disaggregation - he pointed Braintree as an example of a startup and to an Anthony Jenkins FT article last week, noting that the pressure is on to end universal banking:
We shall see....Dave also made the point that you can't dissociate Tech pressures from regulatory pressures (not new news - cf Carlota Perez et al), Regulation drives tech to huge degree eg regulations forcing direct API access to customer accounts means that banks can't compete on regulated API anyomre, so must compete on other "added value" (to whom areas:
- non regulated
His view is that a major area of disruption will be Identity, it's a mess and identity theft is out of control, opens field to players from other industries. Endgame is Amazonisation of banks, banks are no longer where we store money, they are where you store identity.
The Panel session had the above people plus Jonathan Kramer, Sales Director of per-to-peer lender Zopa and Mutaz Qubbaj, CEO & Co-founder of Squirrel, moderated by Ben Rooney, co-Editor in Chief at startup Informilo, ex WSJ - who quipped that Chair is journo who has just started up - quip "you know you're in a bubble when journalists leave good jobs to form startups"
The main Panel Discussion issue was "where are the opportunities in the near future? These broke down into:
Working with banks
- KYC, CTF, AML (and other incremental improvement of all TLA technologies....)
Competing with banks
- where banks do things inefficiently eg consumer and SME lending
Banks are hugely vertically and horizontally integrated, tend to ossify and can be outmanouvred. But banks always have 1st option on trust and first mover advantage - though are everywhere slow and reluctantly trusted, much less liked (even less than RyanAir apparently).
Then came the issue of Innovation vs. Regulation:
Regulation not bad per se, but is usually poorly done/enforced (and typically regulating the stable where the horse has already bolted).
There is very little startup activity that is better (from a major's point of view) than can be gained than by playing a regulatory arbitrage game that only big boys can play ("Too big to fail" being the zenith of this issue).
Any truly radical new, small play, if it gets too big gets regulated (ie Zopa can scale, but how big does it get before regulation comes in). Zopa said it now wants peer to peer lending to be regulated (to be safe from high risk-adopting new entrants who would force tighter regulation, I presume). Another example, Funding Circle who make loans to small companies, is not regulated yet - but what happens if this market grows - there is always going to be a rotten apple/major incident and brings in tighter refulation. (Ditto Crowdfunding.... )
And finally - The obligatory Bitcoin question!
I think Dave Birch had this one down best - bitcoin itself is not particularly interesting, but blockchain is interesting as it allows trading without settlements. This is not just for currency, but for all sorts of 3rd party transactions like managing dishwasher guarantees, and also has a strong intersection with IoT
Anyway, to a large extent it showed that change will probably be slower than the enthusiasts think, as startups have a hard time plus negative inducements in growing too large and/or too fast, and regulation can often be used by incumbents to protect positions to a very high degree.
My "Note to self" was to look again at mobile payments in Africa and what else is being done on these platforms, as there are no legacy systems and typically that is where "next gen" disruptive technologies prove themselves.
(Page 1 of 277, totaling 2764 entries) » next page
More Broad Stuff
Poll of the Week
Will Augmented reality just be a flash in the pan?
Creative Commons Licence
Original content in this work is licensed under a Creative Commons License