Creating Equivalency?

There is an worrying amount of commentary in Germany these days that equates the actions of the far left with those of the far right. Simply put, many people seem to think that setting a car on fire to protest the G20 meeting is just as bad as setting a shelter for asylum seekers ablaze -- something neo-nazis in Germany have been doing at a frightening rate. I decided to take a look at the annual reports of the domestic intelligence office to see whether there is a trend on the official side toward creating an equivalency of left and right.

The Federal Office for the Protection of the Constitution, as the domestic intelligence office is known, monitors extremist and terrorist groups and publishes an annual report on their activities. The report for 2016 was published last month. While the report documents a decrease in violent crimes by so-called left-wing extremists, the report and its popular summary contain strongly worded warnings about the purported willingness of individuals on the far left to use violence, including deadly violence, to achieve their goals. This has already led to some alarmist headlines.

I thought I'd try out some of my toolkit to see what I can learn about the framing of the far left by the German federal government. Here are the beginnings of my exploration, focusing on reports issued at the federal level since 2012. In addition to the federal domestic intelligence office (BfV), each of the sixteen Länder also has a state-level office (LfV), and a systematic look at this issue should also take those into account.


Note: Since first posting this, I have updated this analysis by using a more sophisticated technique of extracting text from the PDF versions of the reports. (Previously I relied on the pstotext command line tool, which does a poor job with unicode text.) This update hasn't changed my conclusions drastically, but I have rephrased the discussion below.

In [1]:
%pylab inline
Populating the interactive namespace from numpy and matplotlib
In [2]:
import spacy
import requests
from pandas import Series
In [3]:
nlp = spacy.load('de')
In [4]:
def pdf_to_text(pdf_file):
    'See http://givemetext.okfnlabs.org/#api for a description of the API.'
    r = requests.put('http://beta.offenedaten.de:9998/tika', open(pdf_file, 'rb'))
    r.raise_for_status()
    r.encoding = 'utf-8'
    return r.text

After preparing the environment, the next step is to grab the reports.

In [5]:
report_url = 'https://www.verfassungsschutz.de/download/vsbericht-{}.pdf'
years = range(2012, 2017)
In [6]:
for y in years:
    !wget -q {report_url.format(y)}
In [7]:
corpus = {}
for y in years:
    corpus[yr] = nlp(pdf_to_text('vsbericht-{}.pdf'.format(y)))

All of the above was just to build up a dataset, or corpus, of BfV reports. First I downloaded them from the BfV website, then I extracted the text from the PDF files using my pdf_to_text helper function to read the contents of each of the reports into my corpus variable. I apply natural language processing from the spacy library to clean up the text.

In the following, I filter out only words starting with linksextrem and rechtsextrem, the terms the BfV uses to refer to far left and far right, respectively. I can then calculate annual ratios of the relative annual usage of these terms in the reports. If there is an increasing tendency toward equating left and right, we might see a rise in the use of linksextrem relative to rechtsextrem, effectively suggesting that they are painted as similarly threatening to public security in Germany.

In [8]:
ratios = {}
for y in years:
    left = [w for w in corpus[y] if 'linksextrem' in w.lemma_]
    right = [w for w in corpus[y] if 'rechtsextrem' in w.lemma_]
    ratios[str(y)] = len(left)/len(right)
In [9]:
Series(ratios).plot()
Out[9]:
<matplotlib.axes._subplots.AxesSubplot at 0x74801f2d52b0>

This exploration has yielded some prima facie evidence that there is a tendency toward increased left--right equivalence in BfV reports. In 2012 there were two mentions of the far left for every three mentions of the far right, making for a ratio of 0.676. After a drop in 2013, that ratio started going up with the 2014 report, indicating an increase in the relative occurence of mentions of the far left with each passing year. With the 2015 report, there were already more than three mentions of the far left for four mentions of the far right, and the ratio is a bit higher still for the recently published 2016 report. The increase overall is not huge (roughly 13 percent), but these results suggest that, in purely quantitative terms, there is a greater equivalency between the far left and the far right in part of official German government discourse.

I should warn against reading too much into these results. Further exploration could look more closely at the terms used in the reports and whether they shift over time, and it could look at a longer time series as well as the state level to establish whether there is a broader pattern. If I have a chance to continue, I'll post some more results here.