The automation of hate speech

The dissemination of hate speech on Twitter is an obvious cause for concern. Until recently it seems to have relied mainly on the efforts of bigoted individuals and – although there is a lot of it – the need for manual posting has set some limits on how many offensive tweets there might be.

But now something far more dangerous is happening. Automation means that robot Twitter accounts can be used to propagate hate speech on a massive – and virtually limitless – scale.

This is currently happening in the Middle East where thousands of robot accounts are posting an endless barrage of tweets directed against Shia Muslims. Although this has so far escaped the attention of the world's media, it's a development that should concern everyone – Muslims or not – because the same techniques can be used to attack any social or religious group.

Many of the tweets purport to have been posted by people in Saudi Arabia (though they often use the #bahrain hashtag) and basically they aim to promote Shia-phobia among Sunni Muslims. Marc Owen Jones, who made the discovery and first wrote about it on his blog last week has since examined 5,456 tweets posted from suspect accounts on June 26. These were not retweets but individual tweets containing cut-and-paste text. Of the 5,456 there were only 12 original tweets – all the rest merely repeated one of these 12 messages.

All of them, Marc Owen Jones says, contained "derogatory and sectarian" terms, repeatedly associating Shia Muslims with violence and terrorism.

Further research by Marc Owen Jones has shown that tweets from the sectarian robots dominate discussion of Bahrain on Twitter. During one 12-hour period that he monitored, they accounted for more than half the tweets posted under the #bahrain hashtag, and the effect of this is to drown out critical discussion of the regime's behaviour towards the kingdom's Shia population.

The fact that these tweets are posted in Arabic probably makes it more difficult for Twitter to monitor them, and shutting down accounts seems to have little effect. Last week, Twitter suspended some 1,800 suspect accounts but that left hundreds of others untouched and whoever is behind the campaign has continued to create new ones.

In a blog post last week I looked at some of these accounts and demonstrated that they do not belong to real people. Among other things, they were using profile photos grabbed – apparently at random – from the internet.

Of course, there are lots of fake accounts on Twitter, but these are also robot accounts – programmed to post tweets at certain times.

To get a clearer picture of what is happening, I have now looked in detail at all the tweets posted on June 27 by nine accounts (@abdullahshokxqx, @romehfarragqiro, @gasemsheblikhau, @mahiobsahiardzk, @makbolmeshezgav, @khamissawwaevgm, @atefmelhemkodcc, @defallahattlvwq, @sharsalemaf44). 

Using data provided by Marc Owen Jones I have compiled a spreadsheet which lists the times of all the tweets from each account, and calculates the interval between tweets. The time intervals (in the second column under each account name) show that the tweets from all these accounts are controlled by an automated process.

The process has some features which I can't explain but what appears to be happening is this:

  • While the account is active, it checks every 23 minutes (plus or minus some seconds) to see if it has been asssigned any tweets to post.

  • If tweets have been assigned (usually no more than two) it posts them at intervals of four or five seconds.

  • If no tweets have been assigned it waits another 23 minutes.

Although I have no doubt that this is automated, the 23-minute waiting period is not perfectly regular. It fluctuates slightly, but normally by less than a minute. However, something anomalous happened shortly after 10 am on June 27 when the waiting period for five accounts was extended on one occasion to just over 25 minutes (highlighted in red).

I don't have the technical knowledge to explain these fluctuations and I would like to hear from anyone who has a theory about them – and what might be done to trace the source of these tweets.

Meanwhile, Twitter ought to be exploring better ways of identifying automated accounts.