Entries by Alan Patrick

They say predictions are dangerous, especially about the future. For better or for worse we tracked Brexit, the US Election, and the UK 2017 election and got them right despite the polls. We also tracked the French one and got the right guy, but underestimated the support level - see over here)

Well, we have been tracking the German election using social media since February, and 6.8 million tweets later we have some predictions on Friday we made an attempt to predict the German election using our own data analytics systems that analyse social media, and the results for the 7 main parties were were (assuming the 7 main parties are c 95% of all votes:

AfD Election outcome = 15% (- but a wide range variance of c 12 - 18%)

CDU 25%

CSU 8% (CDU + CSU = 33%)

FDP 7%

Grüne 7%

Linke 9%

SPD 27%

The range is typically +/- 2% (except the AfD, as discussed below)

The usual caveats

Note that German Twitter usage is relatively low as a % of population, so (as was true in UK and US in the low penetration years) it may still be left leaning if it is still mainly early adopters, so our predicted SPD and Linke figures may be a little high by a few %. Ditto, the FDP figure could be under-represented.

But the real question mark is over the AfD. Why is our system showing a most probable result so much higher than the polls, and saying there is a wider range. (Incidentally, it shows there is even a remote chance it could be greater than 20% so could there be another upset?) The issue is the sheer volume of support (see blob picture above) yet lower possibility of votes (it's quite far to the left of the CDU and SPD on a log scale), i.e. about 1/2 the impact.

Or are we just wrong re the AfD - the polls are saying 11%?

(Update - many adjusted to 12% on Saturday after we submitted this)

In brief, when it comes to these "alternative" party outcomes the system has seen the "right" picture in all previous elections, and we are betting it will do again. Our ingoing assumption is the the "Shy voter of unpopular party" effect is in play, because it has been before in similar situations.

(And it ain't about bots...bot activity has been very low (and besides our system is quite good at ignoring them)

The rest of the discussion about what we are seeing and thoughts about why are over here on our Dataswarm site

(Update) As of Sunday evening exit polls it's been none too shabby though it seems the SPD has undershot at the benefit of the smaller parties and the AfD is c 13-14%, not 15% as we predicted, but higher than poll predictions. We shall see by tomorrow

So the UK General Election results are now known except for 1 out of 650 seats, Tories have 318 seats but needed 326 for a majority of 1. The Tory eventual vote share in 2017 was 43%, Labour was 40%, a 7% difference an puts the Tories 8 seats below. In 2015 it was Tories 36.9%, Labour 30%, a 23% difference.

Our algorithms had it spot on, predicting a hung Parliament, lower and median were hung parliaments with the upper bound a slightly smaller Tory majority. So we got it, right?

Well, yes, the algorithms behaved very well, but the humans (mainly me) didn’t.

The reason was the worry over social media biassing too liberal/labour, as it did in 2010 and 2105. Now we knew it wasn’t as biassed as 2015, given the larger demographic now on it, but we (I) thought there would be a bias, which we put at somewhere between 3 and 8%, so our adjusted eventual range meant the median was a small Tory win, the high end was a bit better than they did, the lower was a hung parliament. The predictions were very tight - we were talking c 20 - 30 seat spread over 650 seats, but the Tory win was on the bottom limit, and we actually called the median as a small Tory win

So - algorithms got it (there was nearly no social media bias in 2017) and the humans over-compensated for the bias.

(To be fair, this was still a range from hung to “slightly better than last time”, with a tiny solution range and it was a damn sight better than nearly all the polls were able to do, and way better than the pundits)

Another thing we'd note is that the narrative played out in the polls, punditry and press about the election bore little relationship to what we were seeing, and as our systems were nearly spot on in their prediction and most of the above were way off, I am inclined to believe our system a lot more (It also makes me very cynical about the ability and motives of said polls, pundits and press). To summarise, we saw the following:

- There was no "huge Tory lead", it was a chimera, they were fooling themselves.
- In week 1 the LibDems made most of the running but quickly fell away as the main parties got going
- After a week or so Labour started to close the gap on the Tories.
- Labour's Manifesto was a step change. We saw no sharp impact from the Tory Manifesto, except from about then on Labour started to gain on them faster. They were quite close to each other so may all be wrapped up together..
- The only topics to "break the surface" as very influential were Brexit and the NHS, Scottish referendum a distant 3rd. All others were in the noise.
- From about 2 weeks before election day the end outcome had emerged on our system, if you go back to our post 1 week before the election you can see the result graphically and it hardly changed.

In short – fire the human, promote the algorithms!

We have turned our analytic engine onto the UK election, looking at social media to predict it. It has worked for Brexit and Trump, got both French phases right (but underestimated Macron). It says that the Tories will win, with about the same majority as now, with a margin of error somewhere between a small loss (largest party but not enough to form a majority) and a c 20 - 25 seat lead).

(Update - the results are now known, and it was the lower bound of our prediction. In fact, the algorithms had it spot on, the error was me adjusting for possible social media bias - see below - which was non-existent in fact.)

You can see more detail of the prediction here on the DataSwarm site

However, the hard bit is predicting the bias of Social Media - in 2015 Social Media thought Labour would win, but the Tories squeaked in, there was quite a strong bias towards Labour on social media . We don't know where social media is now, we believe there is less of a bias as there are more people on it ,so that means it is more representative of the demos. In our view it is worth several additional % points which gives an outcome somewhere between a hung vote and several tens of seats lead, on average about the same.

Tomorrow, all will be revealed and we will no doubt be tweaking the predictive algorithms again.

This is a summary of the more detailed post on what we are seeing on the DataSwarm Analytics blog. In essence we are seeing Labour's rise continue, and they are now very close to the Tories (or more accurately, the "Theresa Party" as that's what their strategy has been till very recently. Here's what we've been seeing over the election weeks:

The Liberal Democrats came out the blocks early, with a clear story, and garnered a disproportionately large social media share initially. As expected, the other opposition parties took a while to get going, but we were surprised that the Tories – who called the election so one would expect them to hit the ground running, did not.

In fact, we have seen the opposite for the Tories. They seemed slow to get going, and never really took off. We also never saw this “huge lead” they were supposed to have that convinced them to call the election (we have some views on this, based on bubblevision but will wait till after the election to talk about that), and support has ebbed away from them from the outset.

Labour have been doing well, picking up support. We see two major drivers of this:

- The Labour Manifesto was very popular with a lot of people, not just traditional Labour voters
- Jeremy Corbyn is simply not as bad as the British media and his opponents have repeatedly painted him, so – as with Trump – he wins merely by being better than the very low expectations people had. In this way I think the anti-Corbyn brigade have shot themselves in the foot.

The other parties are quite small or regional, they won’t shift much so we are not really watching them.

Some caveats though - we have been here before, Social Media always underestimates Tory support as its key demographics are largely missing, and the "Shy Tory" (talk liberal in public, vote conservative in private ) effect.

So, who do we think will win? Well, there was a debate last night which is showing signs of being a "tipping point" for Labour but its early days and the last weekend will be a major influence we think. We will give our final predictions early next week - stay tuned.

Le Pen couldn't win now, right? (Above - 538's prediction of Trump's chances of success the night before the US election)

Well, using our analytical systems we predicted that it would be Macron & Le Pen last Friday, and this Monday it is clear that this has come to pass.

But it has been an interesting experience, for 2 main reasons:

1. Our system is based on social media data analysis. But France has a lower social media penetration than the US and UK so what happens on social media is not as good a representation of the overall picture as the US and UK. Thus, our results needed quite a lot of interpetation and normalisation. One day there will be algorithms that can do this easily, but today it's still largely about pattern spotting, and knowing what's happened elsewhere and/or before (eg UK Liberal Democrats were going to win "bigly" according to Twitter in 2010, but they came distant third - because early social media adopters trend strongly liberal. We saw this in France too with Hamon & Melenchon)

2. Macron is a "Mule" in the Asimovian sense - he came out of nowhere (he had no presence or even party a year ago) and he broke any models one can build from looking at long term patterns (Trump and the Brexit leavers are similar forces breaking today's politics, and one ignoree their impact at great risk). We built a "Euromodel" for elections to test against based on recemnt trends in UK, Holland, France & Germany, and the main hypotheses were:

- The centre-right absorbs more of the far-right thinking, squeezing them, while the more centre-oriented followers stay with the rightward moving centre-right due to the of lack of a better home to go to.

- The center left collapses, its voters move both right and left

The latter has happened again (as it did in Holland), but the Macron-mule effect meant that the former didn't - Macron has managed to replace the Centre-Right and take some of the rightwards moving Centre-Left vote without shifting that far Right (yet).

The big picture hasn't changed - our model said the 2nd round would go to the Far Right and Centre-Right candidates, and the Centre candidate should still win eventually, in theory. Also mapping a two horse race is simpler than a multiple horse one, even if it is very close. But "Macron the Mule" is not the original Centre-Right and still "anything could happen", so 538's predictions today that Le Pen is over are surprising given their experience in the US (the graphic at the top of the page is their prediction the day before the US election voting). While they are making a sensible prediction sensible based on the data so far, they are in our view premature as the 2 horse race has hardly started and the data has not really started to tell us what will really happen - we will need to take all the lessons of the first round into the second.

Above - our system busy predicting the German election - still far too early to tell very much, but this slide is showing examples of the Centre Right CDU moving rightwards.

We had already set our systems onto watching the French and German elections, and will have something to say about the French Round One very soon. And now, courtesy of the UK Government, we have a UK election to track again too!

As you may recall, we have set our analytic engines onto election watching for Brexit last year which we got right, and it managed to predict the Trump election result publically (see here). .

Our current Euromodel election hypothesis, which was built after an analysis of the Dutch election, predicts that (in broad terms):

1, The main Centre-Right party moves considerably more Right, adopting quite a few of the Far-Right policies and narratives. We're seeing this in France, and starting to in Germany - see above chart. Arguably Theresa May's post-Brexit Tories have already done this. This ploy may lose some of its more centrist supporters (to who though?) but it prevents the Far Right from taking far more right wing supporters.

2. The Centre-Left is decimated, (This already happened to the Lib Dems, and to a lesser extent Labour in 2015 - so arguably it's happened already - but the reasons were different, Coalition Blues and Scottish Nationalism respectively). But in the Dutch election their "Deplorables" - traditional Labour working class voters in UK terms - went both left and right. Arguably Corbyn's Labour is already "more left" so will catch the Left-jumpers, but will there be a desertion of "Right-jumpers, and if so, where to?

We assume the Scots will stay SNP unless a major shift in Labour towards Remain occurs, which is risky for them (see "Deplorables" above).

Anyway, using this as the initial hypothesis for building a system dynamic model (although these days its called "machine learning", and if the hype carries on it will be "AI" by next year) it is then theoretically* possible to calibrate the social media monitoring we can do to look at cause and effect and the deltas, and refine the model and thus predict the outcome of an election.

We shall see....

*We've done it already for US and Brexit, but there is quite a lot of judgement required for interpretation, and some luck - so "training" the system (and training ourselves to read it) is very important.

Was asked how we do this at Broadstuff Towers, in the light of our extraordinarily good prediction record ....so in true Listicle BS style, here are 10 "Red Flags" (to use the emotive form of "pointers"). The more of these it scores, the more the chance of hype, bullshit and eventual shocking, painful collapse:

Some of the red flags are more traditional analysis based - "Type 1" - about the core technology, economics or regulatory frameworks:

(i) It uses the hype tech du jour (aka AI or IoT today) - hype tech is typically nowhere near the promised capability of the Hype, so is a good sign of impending failure

(ii) It requires "this time it will be different" economics or business models or human behaviour or regulation

(iii) Or similarly, the previous "real life" incarnation works nothing like this new thing (eg Uber is going to revolutionise the economics of the previously wafer thin margin taxi industry)

(iv) It mangles some fundamental law of physics or engineering (you'd be surprised how many do...drone delivery for eg)

(v) It goes against/arbitrages some regulatory or legal principle which you know will eventually be slapped on it.

However you can also discern quite a lot by "Type 2" red flags - analysing the activities of the commentariat, whose job it is to raise hype. (as noted above, hype is a good sign of an impending failure) - these are typical signs of this process:

(vi) It's a prediction from Planet Mobile (over 11 years of writing Broadstuff, we've found Mobile predictions are always the most, er, optimistic)

(vii) Most Silicon Valley Tech journalists think its great (the SV wolfpack too often operates somewhere between the poles of groupthink and shilling)

(viii) It's by a journo who writes paeans about SV companies (or if its already a book, run for the hills).

(ix) It's about the latest idea of a Person/Co who is currently a SV darling. Bonus flag if is a company is preparing for IPO

(x) it's pushed by a large Tech Co but is out of their usual sphere of competence.

As an aside, most Tech media, especially free to reader, is optimistic in nature. Also most Tech journalists are not STEM trained so don't always know what's happening "under the hood" - so caveat reador. Right now nothing beats the Type 1 BS around AI stuff, except maybe the Type 2 BS around the sexier Unicorns.

As you may recall, we have set our analytic engines onto election watching for Brexit last year (see here), which it got, and it managed to predict the Trump election result (see here). We have now set it onto watching the French and German elections, and will have something to say about the French one soon.

Tracking the data flow is one thing, but to make systems predictive it is necessary to build systemic mathematical models that can dynamically adjust as new data comes in (this is the basis of machine learning as well) and for that it's worth doing a quick analysis of the Dutch election to see if some form of model is emerging. Our analysis is that the following occurred in Holland:

1, The main Centre-Right party moved considerably more Right (see diagram above), adopting quite a few of the Far-Right policies and narratives. It lost some of its more centrist supporters but prevented the Far Right from robbing its more right wing supporters.

2. The Centre-Left was decimated, We suspect that what happened in the UK and US has played out here too, in that the "white blue collars" went left and right - ie the pattern of the white blue collar class deserting their traditional party affiliation holds true here as it did in UK and US. It's where they went that differs from the US and UK, in that there was quite a shift to more Left parties like the Greens (arguably if Bernie Sanders had stayed in the US race US as a 3rd, it could have looked more like this).

3. We have a hypothesis (awaiting more detail) that the white blue collars are not in such a bad position economically in the EU as in the UK/US (better training, better working conditions and welfare), so are not as desperate/willing to embrace the populist option as a last hope - yet (see * below).

At any rate the above is a good start for modelling how Germany and France will play out, in that we assume the main Right Wing parties will adopt more Far Right clothing, and move considerably to the right to ensure they don't leak support there. (To an extent this is arguably what the Tories are doing, betting that their more centre-ist voters won't go to LibDems or Labour)

What this will mean is that a shift to the Right will be of similar size to the US/UK across the EU, just the "traditional" parties have learned from UK/US to embrace not reject the Far Right policies, to keep themselves in power. But the policies will shift to the Right so the effect is similar.

Anyway, if one uses that as an initial dynamic flow system model, it is then possible to calibrate the social media monitoring to look at cause and effect and the deltas, and refine the model.

An aside - one of the main reasons polls got it "wrong" in the US and the UK was an unwillingness of mainstream groups to believe the incumbent side could lose (see here). In Holland it was the opposite before the election, a common assumption was the Far Right would do way better than they did - but within hours of the outcome they were saying it was "back to normal/Far Right was defeated". This is very wrong too, as noted above. It will be interesting to see if the mainstream media/polls behave in the same way in France and Germany.

*A note - The summer had not yet come at the time of the Dutch election, and won't really have started by the French one - but it will have ended by the German election and if there has been a repeat of the migrant flows of recent years, one can hypothesize it will be much harder stemming the voter flow to the the Far Right than in Holland.

Business Insider:

Facebook is closing hundreds of its Oculus VR pop-ups in Best Buys after some stores went days without a single demo

Well, that was quick - Broadstuff predictions for Tech 2017, No. 11, Dec 31 2016:

11. AR/VR - Useful bits of AR will become integrated into Mobile and Wearable devices over time, VR will be a niche pursuit until (if) price points come down hugely and even then its not likely to expand much farther than the gaming aficionado market. Resist all blandishments that this is the future, it won't be.

Hate to say "We told ya", but.....

After our systems predicted Trump's win, we were asked a number of times about the impact of Fake News (and Bots, Russian Hacking etc - we will cover those in separate posts) and here is a summary of some of the useful research we looked at:

Stanford/ NYU Research

Firstly, research by Hunt Allcott of NYU and Matthew Gentzkow of Stanford, published by Stanford University looked at the sources and takeup of Fake News. They defined “fake news” as "news stories that have no factual basis but are presented as facts". By news stories they meant stories that originated in social media or the news media, i.e. excluded false statements originated by political candidates or major political figures. They also excluded websites well-known to be satirical, such as the Onion.

Firstly, they found that in the US elections, people mainly got their news by from sources other than websites and social media (see pie chart below, left). But online media (websites and social media) was where most Fake News was disseminated. They also looked at how Fake News was disseminated on the online media (below, right) and the majority was transmited via social media with a significant minority going direct (to websites or their feeds) or finding it in search results, This contrasts hugely with how top news was disseminated, mainly via older channels but online the major source was via direct access and then search.



They also looked at how people reacted to Fake News, ve Mainstream media news, and also inserted Placebo news (stories they made up) to test reactions. The chart below shows how people reacted:



The Figure presents the share of headlines that survey respondents that recall seeing (blue bar) vs. recall seeing and also believing (red bar). They averaged responses across all the headlines within four categories of headlines they presented - "Big" true stories; Smaller true stories; Fake stories and Placebo stories that they had made up headlines for. In short they found that 15 percent of people reported seeing the Fake stories, and 8 percent reported seeing and believing them (about 55%). But the chart also shows a number of other interesting tendencies:

• Rates of both seeing and believing are much higher for true than fake stories


• They are higher for the “Big True” headlines (the major headlines leading up to the election) than for the “Small True” headlines (the more minor fact-checked headlines that were gathered from Snopes and PolitiFact).


• Placebo fake news articles, which never actually circulated, are approximately equally likely to be recalled and believed as the Fake news articles that did actually circulate.


• This false recall rate is similar for Fake and Placebo articles, this suggests that the raw responses significantly overstate the circulation of Fake news articles, and that the true circulation of Fake news articles was quite low

The last test they did was to model what impact Fake News would have had to make to shift opinion in the most closely fought wards to ensure the Democrats won. For Clinton to have won the election, Trump’s margin of victory would have to decrease by ~ 0.51% of the voting age population, which would have shifted Michigan, Pennsylvania, and Wisconsin into Clinton wins and deliver the Electoral College. Thus, the core question was whether fake news could have increased Trump’s margin of error by more than 0.51 percent of the voting age population. The table below summarise the outcome of their model. In summary, the column on the far right looks at how many times more effective the Fake News would have had to be compared to TV advertising to have had to have shifted the vote. For example, on line 1 a Fake News story as it performed in reality was would have had to be 37 time more effective to shift opinion. If recall was 7% of all stories, it would have had to be 27 times more effective. Line 8 sows that if Fake News shares were 20x greater it would still have to have been 13 times more effective



Their overall conclusion was that Fake News was very unlikely to have had a major effect:

Social media were not the most important source of electionnews, and even the most widely circulated fake news stories were seen by only a small fraction of Americans. For fake news to have changed the outcome of the election, a single fake news storywould need to have convinced about 0.7 percent of Clinton voters and non-voters who saw it toshift their votes to Trump, a persuasion rate equivalent to seeing 36 television campaign ads

• We suspect the Stanford study underplays impact on people, our empirical observation is that many people believe what they want to believe no matter how untrustworthy the source, and will go to great mental gymnastics to justify their belief even when its shown to be totally false, so that IPSOS figure of 75% may well be closer then the c 55% (8% of 15%) of the Stanford study.


• Also, the Stanford model is generic and averaged, if (and it's a big if) Cambridge Analytica was able to pinpoint just the people it needed to persuade, in just the wards it needed to persuade them, actual impact could be higher

In other words this may underplay the total impact of Fake News, but even so the model is still showing it has to be a LOT more effective to actually swing the votes. Our view is its a marginal contributor, but in a 50/50 split election (which in effect this was) even small margins can be effective, especially if used in conjunction with a number of other small nudges.

Also, the Stanford model's definition of "Fake News" is very strict - we believe there is far more "False" news - news that bends the truth, or is economical with it - in circulation, and that acts in a similar manner. A lot of this sort of news is meat and drink to more "respected" media as well (and it is they that are leading the complaints against "Fake" news).

At any rate, expect more use of Fake News in future campaigning, and in attempts to persuade in general.

We have looked at how our systems can counter this, and believe we have some solutions