The Internet Blacklist – Understanding Russia

Russia has been accused of many things over the past few years. Its government has been called fascist, autocratic, dictatorial and tyrannic. According to some in the Western hemisphere Russia and its demagogue-representative Putin are a threat worse than Hitler as well as more savage than ISIS. We learned from our media that the country is plagued with a crusade against non-traditional relationships, that dissent to the Russian narrative is crushed with the Kremlin controlling media, worldview and understanding as well as that Putin himself is intolerant to criticism. Putin, as we know, is the idol everybody needs craving and any voice dismantling the idea is silenced, murdered or imprisoned.

Fascist, autocratic regimes have a tendency to manipulate knowledge. Most sociologists would agree that fascism needs violent and aggressive oppression. In this day and age, that means controlling the internet and what people view. Consulting the Wikipedia page about Internet Censorship in Russia, one comes to such dire an understanding. As it should according to defenders of the above world view, the country adopted laws banning “extremism”, “illegal meetings”, “violations of the establish order” as well as “child pornography”, “advocating suicide” and “illegal drugs”. While the latter are to be found in-line with general expectation, it are the former which the public is made to question. Proper analysis of the blacklist which is publicly available despite Wikipedia denying it until recently, paints the picture. Is Russia cracking down on alternative views? Can or can things not be freely expressed? Where’s the limit?

Although the list has almost 75.000 entries, it’s clear that internet censorship in Russia is not as bad as the Wikipedia page makes us believe. The last topic of the page, Instances of censorship, shows an extensive list of examples of internet censorship. According to that list, Wikipedia itself also is targeted but it does not seem to be reflected in the actual blacklist itself; there’s not a single Wikipedia page which is unavailable in Russia. In fact, quite a lot of the examples given are nowhere to be found. We’ll need to understand that some of the examples have at one time occurred in the blacklist, but have since been removed.

Blocking the Media? – Controlling the narrative

Querying the list with some random keywords does not provide further indication that Russia is in the process of cracking down on dissenting views. Only opposition figure Navalny’s website, Kasparov.org, was easily found. If there are more, they are difficult to find. Kasparov.org brings alternative articles on Russia’s policies. One could easily argue that most, if not all, of the Western media does the same as alternative opinions are often not published. While it is argued that the Russian state actively controls the narrative not including Western mainstream news sites on the blacklist seems to be an omission to that very idea. Indeed, none of the Western media seems to be blocked. The BBC, the Washington Post, the New York Times, AP, Reuters, CNN, Fox News, The Guardian, The Independent etc., are all freely accessible from within Russia.

While it is absolutely true that most of the popular Russian-language television channels seem to be state-controlled or handled by a (partially) state-controlled business (Gazprom Media for example), there’s nothing withholding Russians from consulting video streams on the Internet if they are in need of veering off the enforced “state controlled narrative” even if the broadcaster has Russian language content like the BBC. This finding is in stark contradiction to what we learn from our media addressing “Russian propaganda” which feeds us with the idea that Russians are being fed controlled point of views on a massive scale. None of the Western media seems to be blocked.

If not the narrative, what’s your bet on what is?

Running through the list more structurally it is possible to get a good idea of what the Russian IT watchdog Roskomnadzor is enforced to block. The emerging picture is not as dark as one might believe. It’s abundantly clear that Russia is not keen on allowing its citizens to gamble. When the list is analysed by means of identification of frequently encountered keywords, 44 479 URLs can easily be isolated. It even takes only 16 words to isolate almost half (35 670) of the complete list. These are in order of frequency: bet (9 533 entries), casino (4 021), kupit (2 519), sport (2 506), club (2 216), diplom (2 028), play (1 768), slot (1 739), leon (1 561), vulkan (1 532), prostitut (1 458), parimatch (1 294), game (1 261), mirror (1 170) and porn (1 064).

When using the easily identifiable number of 44 479 URLs as a basis for comparison, over half (23 710 entries or 53.31%) of the websites on the blacklist are related to gambling. That figure even increases to 72.79% when not taking into account a “Miscellaneous” category which was used to identify frequently blocked general words which are often part of the URL like “club” (2 216 entries or 2.96% of the total, unfiltered, list), “kupit” which means “buy” (2 519 or 3.36%), “vulkan” (1 532 or 2.04%) or “leon” (1 561 or 2.08%) and so on. A URL with “Bet” is blocked 9 533 times, “casino” 4021 times, “sport” 2 506 times, “play” 1768 times and so on.

Let there be porn – even the gay kind

The state operator also is required to block quite a lot of porn sites, but the number already is far smaller than the one found for gambling: 4 705 entries precisely (10.58% or 14.44% without the miscellaneous category). It needs to be said that the biggest porn websites are not blocked entirely: pornhub, youporn, xvideos… are largely accessible and action is only undertaken towards videos which show extreme porn, incest, paedophilia or paedophile nudism. In effect, the largest part of the blocked sex sites have to do with prostitution: 1458 URLs have “prostitu” in them, so about 31% of all sites in the “Porn” category, second is “porn” with 1 064 hits or a bit over 22% of the porn category. Third on the list is websites with “sex” in their domain name which accounts for 782 hits in the blacklist, which is about 16% of that same subdivision. Funny fact: www.youporn.com seems to be freely accessible except for some dubious videos on it, but ru.youporn.com is blocked.

If we are to believe Western news, one would expect a large amount of blocked gay porn as well. That is not the case however; only 76 sites with “gay” in their URL are blocked and they mostly are combined with “teen”. If the website’s path also is counted, there’s 136 items of gay content inaccessible in Russia. A fair number of them seem to link to cartoons depictions of gay porn where the drawn subject looks young and underage. The LGBT keyword is not even found back in any of the 74 932 entries in the blacklist.

Russians can’t see movies

In third place we find movies being banned. More specifically illegal download ones, but the number again is far less than before and only 2 568 DNS entries or URLs (5.77% of total identified ones or 7.88% excluding “miscellaneous”) can easily be identified by using keywords like “kino” (975 entries, 1.30% or 2.19% without miscellaneous), “video” (462, 0.62% or 1.04%), “torrent” (422, 0.56% or 0.95%) and so on. The numbers for other keywords like “movie”, “pirat”, “mp3″… quickly become too small to be really significant in the Russian blacklist. Column “C” of the Excel file holds links to very specific addresses being blocked. It’s noteworthy to state that most “torrent” DNS entries also have an entry in that third column. This means that the complete site is not blocked, but rather a link to a very specific movie or series. This likely is a reaction to a very specific request of the, mostly, Western content-producers asking Russia to take action in removing copyrighted content. The matter became significant after Russia joined the World Trade Organisation on August 22, 2012,  after which “Russia started fighting copyright piracy” according to a January 25, 2016 article on Bloomberg.com which has been retracted in the mean time. It’s a good thing we have the Internet Archive at our disposal.

Russians need to study!

Taking the total list counting 74 932 entries, we are down at the next chunk of more or less easily identifiable items on the blacklist, namely “getting a hold of illegal diplomas”. Indeed, isolating only the keyword “diplom” by itself created the fourth biggest category – by itself. The keyword is blocked 2 028 times (2.71% or 4.56% without miscellaneous). It’s clear that Russia does not allow people to look for, let alone buy certificates of education.

The rest of it

While isolating more and more items by identifying keywords used in the URLs of the blacklist, they quickly go down into triple-digit numbers. The “Drugs” category consisting of keywords like “chem” (237 entries in the complete list), “drug” (114), as well as some quite negligible ones like “grow” (81), “stone” (55) or “cannabis” (33) only adds up to 538 entries which is only 1.21% of the identifiable blacklist or 1.65% of that same list without the “miscellaneous” category. To put that into perspective, we are now at only 0.7% of the complete, unfiltered, uncategorised list.

We have Politics there too (516 items), but we’d need to remark that the keyword added to that category also includes “legalnye” which not only is about politics. “Kiev” on the other hand is blocked an immense amount of 50 times. With a president not allowing criticism to his person the keyword “Putin” is blocked only 18 times. That’s 0.02% of the complete list, 0.04% of the identified list and a total of 0.06% of the list without the miscellaneous category. Things remain negligible when counting the occurrence of the keyword in the paths as well: 44 entries. Granted, clicking through them do point out that most of them are political in nature but, an arguable ironical entry, there’s at least one line blocking Oliver Stone’s Putin Interviews as well.

Other things Russians are not allowed to see

Further analysis of the blacklist proves difficult. It is rife with indistinguishable entries. Identification would need opening the website one by one in order for them to be identified correctly. What to make of the following examples which are included?

• rqut0xkmb.mo.betsbc-24.com
• hojo.pw
• ru.admiral-n.net
• sslll.ru
• bkr13.top
• zerkalo702.ru
• …

These kind of entries do not allow any further constructive analysis of the blacklist provided by Roskomnadzor. Scrolling through the list, it does seem to be clear that quite a lot of extremely troublesome imagery is blocked: “ISIS” is found in 229 paths while “behead” results in 80 blocked pages. The content on these webpages is not for the faint of heart as they serve gruesome, graphical imagery.

Crusade against domains

Looking at the domains which are actually withheld, there’s not a lot of surprises either. Dot coms are blocked most by far, tallying up to over a third of total websites: 30 686 ones are blocked. Second, unsurprisingly maybe since we are talking about Russia are .ru domains with 9 399 occurrences; then follows .net with 6 241 and going down to 4 961 .biz ones, 3 186 for .org’s and 3 150 for URLs ending in “.info”. Given the current political state one might expect Ukrainian entries to number a lot, yet only 713 are accounted for which is less than the general “.me” (882) for example.

Conclusion

If Russia indeed is cracking down on differing world views, it seems to be doing a bad job concerning internet blacklisting. Indeed, all of the foreign news websites are freely accessible with the exception probably confirming the rule. Gay websites can be consulted, as are foreign NGO ones. Russia, so it appears, does not like gambling, illegal downloads of movies, extreme porn, drugs or acquiring undeserved diplomas. Whether that is good or bad is for the reader to decide, but it does not seem to be worse than Western countries cracking down on Russian “propaganda” and illegal downloads.

The list seems to suggest Russia is not as oppressive or repressive as we are often told. It seems to allow citizens to largely make up their own minds by consulting almost all of the information available to us through the internet. Russia’s blacklist serves specific goals; none of which would seem out of place in the Western hemisphere unless double agendas are used.

Our Method

As of half of July 2017, the list counts a bit less than 75.000 individual entries. The very first addition to the site seems to have been made on January 15, 2004, although this date could have been created by human input as it seems to have remained the only item on the list for over half a year in 2004 when another one was added. Wikipedia, as well as further verification, points out that Russia started the blacklist in 2012. The latest additions of the list were added not even a month before this writing. The list consists of the following columns:

1. The IP address or addresses being blocked. Running this list through specific tools one could deduct which countries most of the blocked servers are found in. We did not go to this length though. It mostly is of little value; any site could be hosted in any which other country. The decision where to host smaller websites often is an economic one instead of a conscious decision.
2. The user-readable URL or DNS entry, which are the human readable parts of internet usage together with. Like: “www.rt.com or” “www.bbc.co.uk”
3. URL-paths which complete any which URL we visit with the exact page, like the previously mentioned “rt.com” added with the path “/about-us/” to create “http://rt.com/about-us/” or “http://www.bbc.co.uk/aboutthebbc/”
4. A codification which is used in the fourth and fifth column,
5. A date which is probably reflecting the date the line was added to the blacklist.

We focused on an analysis of the URL or DNS entry. Scrolling down all 75k entries, we jotted down keywords which were apparent to be frequently used. In some cases the path was also looked at to know what the blacklist actually blocked. Frequently imagery which is euphemistically named to be “not safe for work” was found. Quite a lot of them would compromise the computer we wrote this on should it be searched by officials in most any country the analysis was created in.