Subscribe: broadstuff
http://broadstuff.com/feeds/index.rss1
Added By: Feedage Forager Feedage Grade B rated
Language: English
Tags:
bitcoin  centre  election  elections  major  media  party  polls  predict  russian  social media  social  system  trump  voters 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: broadstuff

broadstuff



The weblog of multi-media consultancy Broadsight www.broadsight.com



 



Twitter is for bullet points, not essays

2017-11-14T06:51:27Z

So, Twitter has expanded the 140 character limit to 280 for all users. I've used it for a week or so like this, and overall what I've found is:

- It's useful as you don't have to artificially concatenate twts, and you can be clearer in what you mean
- If you stick to around 140 characters and stick to the point, it flows as well as the original.

But...
- Some users (too many) put no new, useful information in the extra 50%, the signal-to-noise ratio is effectively halved
- A pageful of 280 character twts is half as much information as one of 140 character twts, and given too many are full of woffle it can be even worse.
- A 280 character tweet needs some formatting functions to be easy to read.

Now no doubt Twitter could put in basic editing functions, but this seems to be adding complexity to a system where the major attraction is speed and simplicity.

IMO Twitter is for fast-to-read bullet point information, not micro-essays, and especially not micro-essays by the verbose.



Dulce et decorum est, pro Patria mori....but not such a good plan if you don't

2017-11-11T11:02:00Z

(image)

Lest we never forget, not all unknown soldiers are dead

(Rogers cartoon, Pittsburgh Post-Gazette)



Adversarial Perturbation - fooling AI Image Processing, one Pixel at a time

2017-10-31T21:49:01Z

(image)
Cat registering as Guacamole on Google Inception V3


Covered in boingboing and LabSix- what happens when people start to spoof image processing algorithms..Can you make a cat look like guacamole?

Three researchers from Kyushu University have published a paper describing a means of reliably fooling AI-based image classifiers with a single well-placed pixel.

It's part of a wider field of "adversarial perturbation" to disrupt machine-learning models; it's a field that started with some modest achievements, but has been gaining ground ever since.

But the Kyushu paper goes further than any of the research I've seen so far. The researchers use 1, 3 or 5 well-placed pixels to fool a majority of machine-classification of images, without having any access to the training data used to produce the model (a "black box" attack).

Must admit I'm intrigued and a bit heartened, there may be a way out of the continual CCTV snooping world that is emerging after all. Research paper is over here .



Fake News about Fake News, or How do I hire those Russians?

2017-10-31T13:10:51Z

Latest news on the Russians on Facebook brouhaha from Recode At Facebook, roughly 126 million [or about half the voting population] users in the United States may have seen posts, stories or other content created by Russian government-backed trolls around Election Day, according to a source familiar with the company’s forthcoming testimony to Congress.... Google, which previously had not commented on its internal investigation, will break its silence: In a forthcoming blog post, the search giant confirmed that it discovered about $4,700 worth of search-and-display ads with dubious Russian ties. It also reported 18 YouTube channels associated with the Kremlin’s disinformation efforts, as well as a number of Gmail addresses that “were used to open accounts on other platforms.” And Twitter will tell Congress that it found more than 27,000 accounts tied to a known Russian-sponsored organization called the Internet Research Agency: There is a bit more from NBC: Facebook says in the testimony that while some 29 million Americans directly received material from 80,000 posts by 120 fake Russian-backed pages in their own news feeds, those posts were “shared, liked and followed by people on Facebook, and, as a result, three times more people may have been exposed to a story that originated from the Russian operation. Beforehand, Facebook had said they had found about $100,000 of Russian spend on the platform over the election. To put these numbers into context, the Clinton campaign spent about $1.4 billion on media during this period, and that is without the free publicity from nearly every US media organ being mainly on the Democrat side. The Russian media pieces would have been swamped by this spend and volume, never mind all the rest of the pieces of media on the average social media timeline. The Trump campaign spent c $1bn itself, not on Russians. A BBC article says Facebook estimates that: "just one in 23,000 or so messages shared on the network were from the Russians". That's a ratio of 23,000/1 (about 0.004%), and if you compare the ratio between the Russian spend of c $100k vs the c $2.5 bn spent by Trump & Clinton (25,000/1) then they are no more effective in getting their message transmitted than any other piece of media on average. If the Russian spend is underestimated then they are in fact quite poor at writing attractive content, unless of course the volume is also underestimated in which case it approaches this average again. . The election was won on c 55,000 voters voting for Trump in marginal wards, of a total of c 130m voters. about 0.04% of all voters. So for the Russians to have "stolen the election" with 0.004% of all Ads, their Ads must be 10x as persuasive as all others just to be have an equal chance. Or they have to be extremely targeted. But from the above, it seems the Russians reached c 50% of the potential voting population or about 130m people , so doing the paper-napkin maths at the most optimistic (The 130m messages seen went only to the "right 50% of the 130m who actually voted) so 23,000/1 is now 11,500/1, about 0.02%, so they are 2x more targeted and thus have to be only 5x as persuasive. So in order to believe the Russians "stole the election" you either have to believe that either: (i) They are extraordinarily persuasive copywriters and great targeters to boot, and can do wonders on a shoestring, beating the best US Ad agencies that Democrat money could buy (they bought the best) or (ii) Facebook (and the others) are massively underestimating both their spend and % of all messaging, but it has to be at least 10x more just to get parity on reaching those 0.04% of marginal voters who turned it for Trump, never mind persuading them Or alternatively, you have to believe they were a pretty marginal force in the election (albeit with intent). Incidentally, we used our data analytics technology to predict Trump's win (and a few other recent elections too) correctly, and we did it [...]



Czech Election - Our predictive model correct(ish) again

2017-10-30T22:22:00Z

Although we didn't track the Czech Election with the full system (unlike these 5), we have mapped it to our OECD political predictive model which we built after the Norwegian and Dutch elections. It states that:

1, The main Centre-Right party moves considerably more Right, adopting quite a few of the Far-Right policies and narratives. This ploy may lose some of its more centrist supporters (to who though?) but it prevents the Far Right from taking far more right wing supporters. If it doesn't (Germany) then the Far Right takes a larger share

2. The Centre-Left is decimated, their various "Deplorables" classes - traditional working class voters - go both left and right. This leads to a rise in the Far-Left numbers. The hard to predict move is the Centre left voters who move rightwards, as there is no natural home for them at the moment, and they seem to scatter depending on local conditions.


So far it's The Czech Election mapped to this, with one "known" exception we've seen before (the "French Disconnection"):

- Parties closer to EU liberal establishment values were left massively depleted. The ruling centre-left Social Democrats (CSSD), saw its share of the vote tumble to become the sixth-largest party.

- Far-right and anti-establishment groups made gains in the election. Far Right SPD support was up to 10.6%, and The Pirates will make their debut in parliament with 10.8% seats. The centre-right Civic Democrats second with 11% each.

- But, as with Macron in France. a populist centreist party, ANO (Yes) collected a share of almost 30%. Our model predicts the Centre left moves right and left, but in most other countries there has not been an easy centre-right choice - except France with a populist centre-ist, and now the Czech Republic.


And as elsewhere, more parties with a wider range on the polltical spectrum are in the parliament, making it harder to put together larger majority coalition.

Resulting setup is:

ANO (Yes): 29.6% +11%
Civic Democratic Party: 11.3% +4%
The Czech Pirate Party: 10.8% +8%
Freedom and Free Democracy party (SPD) : 10.6% New Entrant +10.6%
Communist Party of Bohemia and Moravia (KSCM): 7.8% -7%
Social Democrats (CSSD): 7.3% -13%
KDU-CSR (Christian Democrats) 5.8% -1%
TOP-09 (Liberal) 5.3% -7%


More is certainly to come, as the EU still refuses to shift it's position. As mathematical historian Peter Turchin points out "the governing elites of the EU behave as though they all believe these disintegrative trends that I and others have written about are just a "blip". The more they believe that, we predict the less it will be.




How much does it cost to mine 1 bitcoin - $1,200!

2017-10-29T14:51:24Z

We always believed Bitcoin was overpriced, now it appears that the cost of mining one Bitcoin is very, very high in energy terms. From ZeroHedge - "James Stafford, editor of Oilprice.com not only does the math, but explains the energy-driven geographic arbitrage currently driving bitcoin mining".

The bitcoin boom is well and truly underway, and investors are constantly looking for new ways to gain an advantage in this space The best way to do this, it seems, is by cutting the energy costs of mining this precious commodity. The bitcoin mining industry consumes 22.5 TWh of energy annually, which amounts to 13,239,916 barrels of oil equivalent.

With 12.5 bitcoins being mined every 10 minutes, that means the average energy cost of one bitcoin would equate to 20 barrels of oil equivalent.

At c $59 a barrel at the moment that's about $1,200 to produce each bitcoin

With Bitcoins currently valued at about 100 barrels of oil, there is still clearly a profit - but two things are not clear:

1. So, who is paying this bill? The yearly cost of the energy necessary to mine Bitcoin determines its economics. Cheap electricity is exactly what made China the Bitcoin mining king, but the economics are now moving to naturally cold areas like Iceland. But why pay for mining yourself? We read of increasing attempts to spoof your personal computers to mine bitcoins.

2. Where are the Green/Climate lobby in all this - they are usually after Energy intransigence like a very noisy rash, but have so far been very quiet on this, but if you search the Google there is nary a peep. Maybe they know that making a fuss will piss off a lot of their normal support and funding base?



Austrian Election - will it run to trend? (Yes it will)

2017-10-15T09:15:00Z

We have not been monitoring the Austrian election, nor predicting it formally (and correctly) as we have done for the major elections from Brexit onwards. However we did predict that most OECD countries are moving to a specific model, viz our Euromodel (initially built after the Dutch Election in March. based on what we saw in Brexit and the US Election): In essence, it is a move away from the status quo, via a movement to the poles - right and left. How it plays out depends on the country, so far in the northern European and US it has mainly been a major shift to the right and a smaller one to the left. It may well be the reverse for Mediterranean countries, but in northern Europe the model predicts: 1, The main Centre-Right party moves considerably more Right, adopting quite a few of the Far-Right policies and narratives. This ploy may lose some of its more centrist supporters (to who though?) but it prevents the Far Right from taking far more right wing supporters. If it doesn't (Germany) then the Far Right takes a larger share 2. The Centre-Left is decimated, their various "Deplorables" classes - traditional working class voters - go both left and right. This leads to a rise in the Far-Left numbers. The hard to predict move is the Centre left voters who move rightwards, as there is no natural home for them at the moment, and they seem to scatter depending on local conditions. Thus this is what we expect to see in Austria, we shall be keeping track. Update - as of c 5pm UK time Sunday, the following early results are being reported by by Austrian broadcasters: People’s Party 30.2 % (Center Right) Freedom Party 26.8 % (Right) Social Democrats 26.3% (Center - Left) Update - Monday c 5pm - Final results are People’s Party 31.4 % (Center Right) Freedom Party 27.4 % (Right) Social Democrats 26.6% (Center - Left) The Right Wing Freedom Party has made gains, and the Centre-Right People' Party has moved more right in order to counter them making even greater gains. (Stipulation 1 of the Euromodel). This is also lowest the Social Democrats have polled ever (though it is hardly changed from the last election), but they are now the 3rd party and it appears they will not be a part of the governing coalition for the first time ever and that represents a major drop in their influence (Stipulation 2 of the Euromodel) though they have been saved further ignominy by Austria's Greens (our model says a shift to the far left is the main flow, Austria has confounded this a bit by having a Far left implosion) There are the usual protests of "dirty campaigning" and "fake news", and this is a recurring refrain across every election since Brexit. The reality is however, as even former US President Obama has alluded, that there is a major sociopolitical trend happening across the OECD. There are also headlines of a "shock result" whichh by now is very odd, as we have seen it in elections in country after country over the last year or so, and there will be more to follow unless some major structural changes are made in the EU and even more broadly in the OECD. Big picture, Democratic OECD countries seem to reaching c 50% of the population who believe the current system is not serving them, and these are voting against the status quo and driving major changes country by country. How and why they do it depends on the country, though there are a lot of common traits and issues. To mitigate this trend however requires these countries and their greater institutions like the EU to make fairly major changes to their status quo structures, which so far we see little sign of - so these "shock" votes will probably continue. [...]



Predicting 6 elections correctly using data analytics, despite the polls and pundits - some lessons

2017-10-14T15:38:42Z

Above - system map of the US Election dataswarm in process. Trump is in front - this was at a time the mainstream US media and polling was saying Clinton was far in front, and we then knew an upset was very likely coming. Using our systems we have managed to predict 5 elections (strictly speaking 6 as the French one is 2 rounds) correctly over the last year, 3 very much against all the polling predictions (Brexit, US, UK 2017) and one that still produced "shock" results (Germany). We went public on 5 of them, before election day in each case. (The bullet points below link to blog posts on our DataSwarm site of each of the individual election prediction summaries) Predicted Trump win – we went public with our prediction 3 days before the election. The polls said Hillary was a shoo-in, but the system saw a close race and a dynamic support base that Trump had built up from the beginning and plumbed for him. Predicted UK election – went public 2 days before. The system predicted a hung parliament, we got one. Polls were predicting a major Tory win. Predicted French Election (both rounds) – went public 2 days before both. Got both right. System underestimated scale of Macron win in second round, but in our review we learned how to more accurately predict future voter flow from the system's memetic linkages - major learning point bonus! Predicted German Election – went public 2 days before. Predicted AfD surge and CDU crash better than any polls. Brexit was an initial side project, done with a bit of social media sampling and system dynamic modelling, but as that worked and we got it right - despite all the poll opinions being completely the opposite - we decided to carry on with other elections as as we found that a "known" outcome at a point in time allowed us to calibrate the algorithms quite well. We find that with commercial work measuring aspects like "sentiment" or "influence" and similar can give rather nebulous results. Measuring the true picture is not always simple. How do these really work, and how do they relate to what we can measure? Using known outcomes like election would really help calibrate these metrics, and we could show our clients how our system definitely was working correctly. Besides, if we got a few elections right we might open up a new area of business! Each election taught us something new about the way the algorithms should work, and what needed tweaking or expanding, and we learned some new insights about how human behaviour in reality maps to the metrics you can "see" with social media analytics. We also learned quite a bit about how media works in persuading people, and how "fake news" et al operates. We also proved to ourselves that the system was good in 2 English speaking cultures (US and UK), and could also operate in French & German language and culture - and could crunch tens of millions of units of social media data. We also had quite a few insights about what is happening at a population level politically, and our views differ quite a lot from a lot of the "conventional wisdom" we saw during and after the elections. These are some of the high level takes on each election: Brexit, 2016 We did not put the main system on this, but used a series of small data samples over the election period, and then built a fairly simple system dynamic model to predict outcomes of a 2-horse election race. Turned out it worked, and against all the conventional wisdom and poll predictions to boot. The Remain camp (the favourites) seemed to go in with a static strategy and refused to shift it when it was clear it was losing. In essence, Remain's arguments, exaggerated by "Project Fear" style messaging, were increasingly being perceived by neutrals to exaggerate the risks and that led to an increasing resistance to their message and it gave a foothold to the pro-Leave media t[...]



German Election Predictions

2017-09-22T22:50:00Z

(image)
Dataswarm memetic analysis of German election as of Friday 22nd (2 days before election)


They say predictions are dangerous, especially about the future. For better or for worse we tracked Brexit, the US Election, and the UK 2017 election and got them right despite the polls. We also tracked the French one and got the right guy, but underestimated the support level - see over here)

Well, we have been tracking the German election using social media since February, and 6.8 million tweets later we have some predictions on Friday we made an attempt to predict the German election using our own data analytics systems that analyse social media, and the results for the 7 main parties were were (assuming the 7 main parties are c 95% of all votes:

AfD Election outcome = 15% (- but a wide range variance of c 12 - 18%)

CDU 25%

CSU 8% (CDU + CSU = 33%)

FDP 7%

Grüne 7%

Linke 9%

SPD 27%

The range is typically +/- 2% (except the AfD, as discussed below)

The usual caveats

Note that German Twitter usage is relatively low as a % of population, so (as was true in UK and US in the low penetration years) it may still be left leaning if it is still mainly early adopters, so our predicted SPD and Linke figures may be a little high by a few %. Ditto, the FDP figure could be under-represented.

But the real question mark is over the AfD. Why is our system showing a most probable result so much higher than the polls, and saying there is a wider range. (Incidentally, it shows there is even a remote chance it could be greater than 20% so could there be another upset?) The issue is the sheer volume of support (see blob picture above) yet lower possibility of votes (it's quite far to the left of the CDU and SPD on a log scale), i.e. about 1/2 the impact.

Or are we just wrong re the AfD - the polls are saying 11%?

(Update - many adjusted to 12% on Saturday after we submitted this)

In brief, when it comes to these "alternative" party outcomes the system has seen the "right" picture in all previous elections, and we are betting it will do again. Our ingoing assumption is the the "Shy voter of unpopular party" effect is in play, because it has been before in similar situations.

(And it ain't about bots...bot activity has been very low (and besides our system is quite good at ignoring them)

The rest of the discussion about what we are seeing and thoughts about why are over here on our Dataswarm site

(Update) As of Sunday evening exit polls it's been none too shabby though it seems the SPD has undershot at the benefit of the smaller parties and the AfD is c 13-14%, not 15% as we predicted, but higher than poll predictions. We shall see by tomorrow




UK Election Prediction - Promote the Algorithms, fire the Humans

2017-06-09T16:51:06Z

(image)
The endgame analytics in diagram form, evening of June 7th – as you can see, the Tories & Labour are very close according to the system and it pretty much got it right on the button.



So the UK General Election results are now known except for 1 out of 650 seats, Tories have 318 seats but needed 326 for a majority of 1. The Tory eventual vote share in 2017 was 43%, Labour was 40%, a 7% difference an puts the Tories 8 seats below. In 2015 it was Tories 36.9%, Labour 30%, a 23% difference.

Our algorithms had it spot on, predicting a hung Parliament, lower and median were hung parliaments with the upper bound a slightly smaller Tory majority. So we got it, right?

Well, yes, the algorithms behaved very well, but the humans (mainly me) didn’t.

The reason was the worry over social media biassing too liberal/labour, as it did in 2010 and 2105. Now we knew it wasn’t as biassed as 2015, given the larger demographic now on it, but we (I) thought there would be a bias, which we put at somewhere between 3 and 8%, so our adjusted eventual range meant the median was a small Tory win, the high end was a bit better than they did, the lower was a hung parliament. The predictions were very tight - we were talking c 20 - 30 seat spread over 650 seats, but the Tory win was on the bottom limit, and we actually called the median as a small Tory win

So - algorithms got it (there was nearly no social media bias in 2017) and the humans over-compensated for the bias.

(To be fair, this was still a range from hung to “slightly better than last time”, with a tiny solution range and it was a damn sight better than nearly all the polls were able to do, and way better than the pundits)

Another thing we'd note is that the narrative played out in the polls, punditry and press about the election bore little relationship to what we were seeing, and as our systems were nearly spot on in their prediction and most of the above were way off, I am inclined to believe our system a lot more (It also makes me very cynical about the ability and motives of said polls, pundits and press). To summarise, we saw the following:

- There was no "huge Tory lead", it was a chimera, they were fooling themselves.
- In week 1 the LibDems made most of the running but quickly fell away as the main parties got going
- After a week or so Labour started to close the gap on the Tories.
- Labour's Manifesto was a step change. We saw no sharp impact from the Tory Manifesto, except from about then on Labour started to gain on them faster. They were quite close to each other so may all be wrapped up together..
- The only topics to "break the surface" as very influential were Brexit and the NHS, Scottish referendum a distant 3rd. All others were in the noise.
- From about 2 weeks before election day the end outcome had emerged on our system, if you go back to our post 1 week before the election you can see the result graphically and it hardly changed.

In short – fire the human, promote the algorithms!