Tuesday, July 21. 2015
Ashley Madison, an online site which in essence is a market for meeting others interested in infidelity* (as opposed to dating sites where people pretend to be single) has had its data hacked. Including the data of the credit cards used, as it is a pay-for site - Grauniad:
Apart from a certain feeling of schadenfreude for c 37 million very worried people, to me this is more a sign that yet again, on t'Internet, sex sets the trends that others will soon follow. As porn showed us how video and credt card paymenst would work, this shows how personal data hacking and trading will operate. As the Grauniad notes, at the root it is:
the mass scale of the hack attack has to be recognised for what it is: a gross invasion of privacy.
Anyone who thinks this sort of hack will be limited to sex sites is extremely naive....as human data's monetisation increases, so its value will increase. If Information is the new oil, it will be mined.
*I guess to show our hipster street cred I should call it an Uber-for-Adultery
Monday, July 20. 2015
...is in my view a seminal post from fellow UK blogger Nic Brisbourne, Managing Partner at Forward Partners. Nic makes (in my opinion) 3 very perceptive points in his post (here is my expurgation for the attention deficited Broadstuff audience , the whole article is over here on The Equity Kicker and more nuanced):
As Nic says, the new opportunity for retail, therefore, is to solve the paradox of choice. Also As he notes, recommendation engines, Social Media recommendation etc are just the start. Now Nic's company has invested in some startups (it's what they do after all). Here in Broadstuff Towers we don't have a clue who will win this race, you can imagine a number of possible axes of success, but I think this is a very useful way to think about the "Future of Retail"
Friday, July 17. 2015
The IoT is 70% Hackable, says HP. Only 70%? It's hardly even growed up yet! PIcture Source
My colleague David Short has long been concerned about the hacking potential on "digital cars", as all intelligent devices in a car typically share the same databus, and as they become more wired to "the grid" the probability of hacking will grow, whereas so far the probability of manufacturers to take preventative measures is at best static. (And, as David often says, "preventative measures" will only deter the casual hackers, taking care of the real professionals requires considerable redesign of any such simple system)
What does this mean - well, anything from me being able to lock your car and demanding ransom money be paid to an anonymous bank account to deliberate sabotage.
But this is just the tip of the iceberg, now that the IoT is officially on top of the Hype Curve, it means that any hacker worth their salt will be sharpening their code, and the truth is - (having been doing IoT stuff for nigh on 30 years, since long before IoT was called IoT), there has been absolutely minimal interest from the promoters in either mentioning or preventing the possibility of hacking, mainly as (i) it tarnishes the hype, (ii) it makes the seamless simplicity of those IoT solutions a lot less so and (iii) it costs money - theirs to shore it up or yours if they don't.
And we are not talking about good old datahacking and datascraping, the thing about the IoT is it controls devices, so hacking them not only gets the data, it lets the hacker control those devices. At its simplest it is swiping cycles like a botnet does, at its most malign its causing dangerous malfunctions, at its mainstream use cases its probably going to be used to facilitate crimes we already know and love - theft, extortion, etc. And not all teh bad guys will be bad guys - ther are quite a few corporates and quasi governmental agencies who see a benficial paycheck from controlling more of what YOU! used to do for free.
Now I watch the Economist with interest, as its my "mainstreaming" litmus test - as in when something is mentioned in The Economist, it means its about to get mainstream publicity. And today they let the cat out the bag....Dr Graham Steele of Cryptosense, quoted below, sounds just like David:
...many of the firms making these newly connected widgets have little experience with the arcane world of computer security. He describes talking to a big European maker of car components last year. “These guys are mechanical engineers by training,” he says. “They were saying, ‘suddenly we have to become security developers, cryptography experts and so on, and we have no experience of how to do all that’.
As The Economist notes, this is the InnocentNet of the 90's, replayed:
the biggest difficulty is that, for now, companies have few incentives to take security seriously. As was the case with the internet in the 1990s, most of these threats are still on the horizon. This means getting security wrong has—for the moment—no impact on a firm’s reputation or its profits. That too will change, says Dr Anderson, at least in those industries where the consequences of a breach are serious.
One difference is the 'Net has generated a huge community of hackers, they were a rare breed 20 years ago, now not at all, and the IoT is the biggest unlocked toybox you an imagine. Now you can be sure thet critical devices worth a lot of money will have some attention thrown at them, but for the rest...
So what to do - well, I don't mean to be a killjoy - there is a typical trend for new technologies liek this - first comes the hype, then the arbitrage, then the cowboys, then regulation to clean up Dodge City, then it settles down. our advice. For practical purposes for now, we are not even in Cowboy phase so best is to be a "laggard early adopter"and let others experience the delights of being first into the Internet of Hacked Things for a good few years, but if you must adopt these New New Things:
(i) Don't sign up to any current metering or "smart" devices - none of them have any anti-hacking capability yet. If the control button is not yours and yours only, caveat emptor!
(In fact, never mind "intelligent wearables" - the other hype curve No 1 contender, I reckon the high value market is in cybercloaking wearables)
Tuesday, July 7. 2015
The Augmented Tennis view - today on a screen, tomorrow in your retina?
Last Thursday I was lucky enough to visit IBM’s Big Data Crunching Operation behind (and underneath) the annual Wimbledon Tennis tournament, courtesy of fellow Tuttler Andrew Grill of IBM (who says nepotism never pays) and also courtesy of the knowledgeable and enthusiastic (despite me being the Nth tour guest of the week) Sam Seddon who escorted me through the databunkers.
Now I am far from the first Old Tuttler blogger that Andrew has invited, my colleague David Terrar went last year and Neville Hobson went on the Tuesday last week (here is his post). Thus the “what happens behind the Datascenes at Wimbledon” is already more than well covered, just look at David’s blog post from last year - to quote:
Sam [Seddon] has a team of around 200 supporting Wimbledon using an impressive array of terminals and technology on site, supplemented by an enormous amount of Cloud compute power from data centres in Amsterdam and the USA. They are providing a service to the All England Lawn Tennis Club to help make Wimbledon the premier sporting event, but in doing so they are serving the audience at the ground, fans around the World, the radio and TV broadcasters of the event, the event sponsors, the Club itself, and even the players directly
David did a good analysis of the overall IBM operations last year, and had some cracking photos - so what can I add to this? Probably the best is to talk about it from the point of view of things I know about in some detail – in this case, high end datacentre infrastructure, real time big data & analytics, and system automation (in this case, Watson)
High End Datacentre Infrastructure
Firstly, from the point of view of datacentre infrastructure, this is a high pressure operation – real time, high visibility, peaky data flow. High embarrassment if things go wrong. Also, the web based systems get quite a few attacks 24/7. And as well as running all the tennis data feeds discussed below, IBM also run the ticket checking and security operations of the whole site.
Secondly, from an Operations & Logistics point of view this is further complicated as the whole setup is also completely transient – for 2 weeks a year the Tournament Tennis circus descends on Wimbledon. The entire IBM system rolls in, the porta-datacentre and its operatives are wheeled into the empty bunkers from locations elsewhere in the world (along with all the TV broadcast trucks, meedja personalities, commentators in their glass boxes, tennis line callers, buckets of strawberries and flagons of Pimms etc etc) - and then two weeks later the whole panjandrum disappears and the whole site is emptied and mothballed again for another year.
This is non trivial stuff, kudos to IBM for making it so seamless!
Data Flowchart of all the IBM Wimbledon dataflows - camera had too much Pimms clearly
Big Data & Real Time Analytics
From a “big data fan” viewpoint there are 3 main data handling operations (see flowchart above, and weep) - in summary::
Every shot played by every player is captured, not just physically but with rich metadata – tennis experts analyse every shot and record what it was, why it was played, did it work/fail. Interestingly IBM captures data from every major tournament so you can pick up a rich data picture of every player and the permutations of their matched with others.
Added Tennis Metadata
Not only player match stats, but their historical metadata and also Wimblestats are collected – for example, veteran Leighton Hewitt hit his 1500th ace the day I was there, and they can play comparison games with stats back to the 1870's. Much of this is to add rich data to the various media outfeeds - website, TV, Radio, Social Media.
Social Media Analysis
The 3rd operation is the monitoring of Twitter feeds off the Gnip firehose. To my readers much of this is familiar ground – find, scrape, focus, process, analyse, use output to inform and focus further coverage etc. (David covered it last year if you’d like more detail) However, there were a few counter-intuitive things I learned:
- Court Prowess and Twitter are loosely linked – Rafa Nadal was the most talked about all player week, despite being an early exit
I was a bit taken aback by the last one, but I guess when you think about it, in media there is a general trend that where porn blazes a trail, advertising soon follows
To a data modelling/simulation wonk like me this was all fascinating, and I hit Sam with all sorts of “what ifs” but he reminded me that It's all about understanding the business of tennis - that you have to look at who the customers are, and what they want. I may want to run the Moneyball Crunch of all crunches on the tennis world and create virtual avatars so I can play Bjorn Borg against Rafa Nadal, but realistically the data is required by:
'You know my methods, Watson.'
….said Sherlock Holmes in The Crooked Man (he never said “Elementary, my Dear Watson by the way ), and to me this is what is starting to set apart what IBM is doing vs other big iron data operations.
For those who don’t know about Watson, it is a very powerful Machine-Learning system initially focused on natural language processing. It’s key role is Question Answering, so underlying the language processing is are powerful deduction chain algorithms. In short, if you can feed it information and deduction methods, it can form its own hypotheses and reasoning chains This makes it useful for quiz shows (it won at Jeopardy in 2011), medical diagnoses, and – well, tennis among other things.
Right now it is being taught the basics of Tennis geekdom (What is love? – answer – a score of zero), being force fed Wimbledon tennis history (to draw parallels etc) and is starting to look at the social media chatterflow. But if you look at the data processing going on at Wimbledon, there are a LOT of workflows, methods and deduction chains that are ideal for a natural learning machine learning system.
Data for Watson - feed me, Seymour....
So when afterwards the IBM people asked me what my main thoughts were from the tour, I told them that in my view, many of their 200 or so operatives here will disappear in the next 5 - 10 years as Watson and other AIs start to take over the rote data processing going on there, and then the less rote, and then the really high value add..
And yes, it felt traitorous to say that, sipping Pimms and watching the bright young meedja things gambolling on the grass, but at the end of the day the automation of dataflow & analytics is just the next step in an ongoing process of transformation.
Wimbledon is just part of a trend I see everywhere right now, wherever previously unstructured data is being digitised. First comes the rudimentary digitisation of hitherto unstructured data (often from manual or semi manual processes ), then comes the early analytics which creates huge value from the low hanging datafruit, and increasingly we now are starting to see the application of automation, both in data capture (IoT) and analytic reasoning (MI/AI). The future will come one automated data feed and workflow method at a time.
Capturing courtside data, play by play. How many years to full automation?
Automating the Tennis Tournament Production Factory?
Wimbledon makes 70% + of it’s money from selling Media rights for the two weeks of the tournament (and as more and more courts get their own datafeeds this revenue will only increase).
And as all those assets, that span acres of prime London real estate – the main courts, the techie bunkers and media circus rings, the player palaces, the buildings for the huge catering and logistics operations – all largely lie fallow for 50 weeks of the year, Wimbledon will always need to maximise income and minimise costs from those 2 weeks a year.
Thus they will inevitably be pushed into the economics of ongoing automation of Tennis Tournament production. The need to both reduce operating costs and produce ever more value-add from the datafeeds is probably inevitable. The competition for media $ will only intensify, other tournament operations will up their value add year by year. So that means more and more AI for the data handling and analytics tasks, to produce more value add for Wimbledon and its customers.
Unless, unless…..the unquantifiable human input can be shown to make the difference between an experience that will delight, versus one that is artificially inserted.
Maybe automation will have it’s limitations, we may not like a smart Watson, but instead prefer a nice-but-slightly-dim Watson to our Holmes - quoth Mr Holmes:
A confederate who foresees your conclusions and course of action is always dangerous, but one to whom each development comes as a perpetual surprise, and to whom the future is always a closed book, is indeed an ideal helpmate.
We shall see. I’d place my money on Smart Watson though....
Monday, July 6. 2015
From the "Well, we would have told you so if you'd asked" Dept:
For those who are surprised and shocked by the popping of the Chinese stock market bubble, can we point you to the Broadstuff Bubble-O-Meter which says that, when taxi drivers give you stock tips and private citizens gamble their savings it really is the endgame of the bubble, and the next step is "pop"
Addition from the "we didn't tell you this though":
And right to the end, those with financial interests in the bubbletime, and their mouthpieces, will insists that there is no bubble.
This is a public service announcement
Wednesday, June 3. 2015
As I can’t get alone to InfoSec15 this year, I thought I would get a flavour of what’s going on by pointing our experimental text mining tool in that direction. See the graph below. Bruce Schneier and John McAfee seem to have the influence they deserve, followed by BSOD (is Windows 10 here already? ) and “trashier tearaway”, which seems to be a reference to an article in The Register about the perils of allowing BYO-IoT-D (Bring Your Own Internet of Things Device!) into secure networks. also, full marks to whoever thought up “malvertising”!
Hope to get along next year and find out what’s going on in person.
(Apologies for my earlier, half-formed draft of this post – now deleted – just proves I really am too old to multitask!)
Friday, May 15. 2015
Andrew McAfee in the FT explaining to the FT audience what has long been blindingly obvious to most everyone else who is living through it - if you automate away the high value jobs and all thats left is low value low skill jobs, average wages and living standards fall, and inequality rises.
He uses the though experiment of a 2 industry economy, one with hi tech, high wage manufacturing and low tech, low wage dog grooming. Over time the high tech labour gets displaced by automation and more people become dog groomers, so average wages in the population fall, those left in manufacturing get richer, inequality widens etc etc. He then observes, as if its a massive revelation, that:
This example is obviously trivial and contrived, but it does include some trends that we see in the actual economy — lots of jobs growth in the low-productivity service sector, sluggish overall demand growth and lots of automation in manufacturing. It also helps explain why corporate profits are so high these days: the factories, after all, have maintained their revenues while halving their payrolls.
No shit, Sherlock!
I guess what's really sad is you probably do have to use a 2 industry model-as-parable to explain it to FT reading, degree educated, often economist types. Piketty tried it the big data way but to little discernable impact to date. They analyse the trees to death, but all too often cannot see the woods.
Now, why not do a little sociohistorical analysis too, and look at what happens when unemployment and wages fall and inequality rises, and a large section of a population is discontented and feels it has no future. Guess what - the economic environment stops being like the one we have at present.
Here's a hint - start with France, 1789, and continue reading about "transformational change", mob revolt style ....the Deus is ex the Machine Age.
Thursday, May 14. 2015
Article from The Economist echoes something I've been thinking increasingly - there's not a lot of new stuff coming out of guruland these days. I thought it was just me getting old and cynical, as increasingly everything "New" just seemed to be a rehash of something from 2 decades ago or more. I had started assumed New Gurus (Gurii?) were mining old, out of print books and rehashing old whines into new booklets.
The Economist has noticed the same, and is looking for reasons why - they think that:
Ah, so its a generational fin-de-siecle then.The Economist also argues that its becoming harder to mint new Gurus now as (i) the world is more interested in Big Data than Big Ideas, and (ii) there are a million Mini-Gurus blooming in the blogosphere. Also companies are increasingly trying to be their own Thought Leader.
To sum up, they note that the Guru industry itself is not exactly behaving like one of these fast moving, always changing, disruptive modern industries that the Gurus bang on about:
...considering the resources that are devoted to thinking about management, it is remarkable how much virgin territory remains. There are still no Chinese management gurus to challenge the leadership of the ageing Indian establishment. There are still no serious books on what the internet economy means for the boundaries of the firm or markets for talent. The guru industry seems ripe for disruptive innovation.
I did read a study several years back, some Swedish researchers looked at what makes a Guru - and one of their major conclusions was that a Guru is selected and anointed by kingmakers, just being smart and original isn't enough. In the old days these kingmakers were usually publishing houses who would then market the cr*p out of said Guru's tome/s . But, as with the music industry, that media model is largely gone - there just isn't the ROI to hugely promote modern Gurus. There is an analogy with the Music industry, where being an original talent also was never enough, the game was always about getting the A&R - and so the oldies in Guruland and Musicland keep a high mindshare as the total cumulative spend on their PR dwarfs the trifling modern amounts spent so the Old Guard have the bg billings, the best new talent gets as far as it can without the Old Skool PR $ into specific verticals, and the Long Tail's output is largely irrelelevant (or gets mined by the better known).
Wednesday, May 6. 2015
Do you remember all the predictions that "email will die" and that social tools will replace it with "ambient intimacy" or similar?
Turns out that not only is it very unlikely email will ever die (see my argument re Riepl's Law on that score) but is also highly likely to grow, not shrink.
The reason for this is due to
(i) The sheer volume of comms on the new systems has just reproduced email's problem - ANY system, email included, is effective when the volume of messages is manageable, and overload looks the same on all of them. Except email has better timestamping, search and storage as its been around longer.
(ii) But email as the new killer app is die to the proliferation of all the social tools delivering said Ambient Intimacy, and thus fragmentation of the medium. There are now so many of them, that the Ambience is broken up into a veritable Tower of Babel of different protocols. The requirement is increasingly for one system to find them, rule them all, and in the darkness (of the communications server farm) bind them.
And guess which system is the most ubiquitous, flexible, integrateable, multi-capable to do this?
So that inbox of yours that overfloweth - the net effect of all these new social tools will just be to make it worse.
You heard it here first...
(by the way, you can subscribe to get Broadstuff via email....)
Thursday, April 30. 2015
Everywhere you turn there is this focus on the "Attention Economy", defined by Wikipedia as follows:
..content has grown increasingly abundant and immediately available, attention becomes the limiting factor in the consumption of information. Attention economics applies insights from other areas of economic theory to enable content consumers, producers, and intermediaries to better mediate and manage the flow of information in light of the scarcity of consumer attention
Which is all very well, but what has really occurred is an arms race between various service providers to divert your attention to their new new thing, rather than any others, and certainly not to the (slightly dull) thing you probably really should be doing.
I call this the Distraction Economy, and its really an anti-economic effect as:
Taking these in turn:
Accrete no or very little Value
There is an infinite array of diversions seeking to distract you. The internet is always wrong, there is always one more interesting tweet-link to read, one more comment for someone's Facebook wall, one more go on the game du jour. And what are you losing - they are all free, right? Well that is the billion dollar question. What is the value of your time spent on these distraction? There are two ways of measuring this:
Problem is that, for the average joe (or josephine) in the New Digital Economy the Conversion Value is near zero, as is the Conversion Rate, so this is an essentially zero-value accretion. So the +ve value is near zero
However, there is a negative opportunity cost - chances are most distractions reduce your own time and effectiveness actually spent on valuable tasks - i.e. unless you have zero valuable tasks to do, paying attention to the "attention economy"'s products is going to cost you (see part 3 below)
First came the idea of the Great User Experience, then when that arms race was tailing off came Gamification, now that is reaching its limits there is Addictification - the best and brightest minds of a generation are being used to try and ensure that people will pay attention to non essentials. Neuroscience, behavioural science, mathematics and a fistfull of 'ologies are being used to try and make this or that piece of digital bubblegum register in people's attention-span and grab that 15 minutes of fame.
As noted above, time being diverted is very unlikely to be accretive in itself, and is highly likely value destructive as:
(i) Less time being spent on valuable tasks - that has to have an opportunity cost impact in the medium and long term
(ii) It has been shown, in study after study, that distracting yourself from a task reduces your efficiency in performing that task, plus also imposes a "setup" and "teardown" time penalty as your brain switches over from task A to task B. The worrying thin about the distraction economy is that it feels like you are more effective, when in fact the hard metrics show the opposite.
What's the answer?
Firstly, it is to recognise that most of the bright shiny products of the "attention economy" are not there to make you money, they are there to make money out of you.
Secondly, to recognise there is an entire industry out there trying to make these grab your attention, with a miniscule industry building antidotes.
So the only real solution is to time-limit the channels - turn off ambient alerts, make a time to do emails/twitter/facebook etc, and keep to it.
(Page 1 of 277, totaling 2767 entries) » next page
More Broad Stuff
Poll of the Week
Will Augmented reality just be a flash in the pan?
Creative Commons Licence
Original content in this work is licensed under a Creative Commons License