What the internet looks like to users in the U.S. can be quite different from the online experience of people in other countries. Some of those variations are due to government censorship of online services, which is a significant threat to internet freedom worldwide. But private companies – many based in the U.S. – are also building obstacles to users from around the world who want to freely explore the internet.

Website operators and internet traffic managers often choose to deny access to users based on their location. Users from certain countries can’t visit certain websites – not because their governments say so, or because their employers want them to focus on work, but because a corporation halfway around the world has made a decision to deny them access.

This geoblocking, as it’s called, is not always nefarious. U.S. companies may block traffic from certain countries to comply with federal economic sanctions. Shopping websites might choose not to have visitors from countries they don’t ship goods to. Media sites might not be able to comply with other nations’ privacy laws. But other times it’s out of convenience, or laziness: It may be easier to stop hacking attempts from a country by blocking every user from that country, rather than increasing security of vulnerable systems.

Whatever its justifications, this blocking is increasing on all kinds of websites and is affecting users from almost every country in the world. Geoblocking cuts people off from global markets and international communications just as effectively as government censorship. And it creates a more splintered internet, where each country has its own bubble of content and services, rather than sharing a global commons of information and interconnection.

Measuring geoblocking globally

As a team of internet freedom researchers, my colleagues and I investigated the mechanics of geoblocking, including where geoblocking is happening, what content was being blocked and how websites were practicing geoblocking.

We used a service called Luminati, which provides researchers remote, automated access to residential internet connections around the world. Our automated system used those connections to see what more than 14,000 sites look like from 177 countries, and compared the results in each country.

Websites that didn’t block traffic typically served us a large file providing rich internet content, including text, images and video. Websites that were blocked usually delivered just a short notice saying that access was denied because of the visitor’s location. When the same website delivered a large file to an address in one country and a very short one to another, we knew we had a good chance of finding that the site was conducted geoblocking.

We found that the internet does indeed look very different depending on where you’re connecting from. Users in countries under U.S. sanctions – Iran, Syria, Sudan and Cuba – had access to significantly fewer websites than in other countries. People in China and Russia faced similar restrictions, though not as many. Some countries are less affected, but of the 177 countries we studied, every one – except the Seychelles – was subjected to at least some geoblocking, including the U.S.

Shopping websites were the most likely to geoblock, perhaps because of economic sanctions or more straightforward business reasons. But some websites hosting news and educational resources chose to block users from specific countries, limiting those people’s access to outside information and perspectives.

The role of internet middlemen

We also found that many websites are taking advantage of geoblocking services provided by their hosting companies and online middleman firms called content delivery networks. These companies operate systems that preload web content at key locations around the world to speed service to nearby internet users, so an Australian looking for an article in the Washington Post doesn’t have to wait for the request to travel halfway around the world and back. With a content delivery network, there’s already an up-to-date electronic copy of the Washington Post stored in, say, Sydney.

Many content delivery network services include a dashboard where a site administrator can easily select which countries to deliver the website’s information to – and which ones to block. Content delivery networks in general are a lot cheaper than they used to be, which means more and more website operators are getting their hands on simple geoblocking tools.

In fact, based on data that were provided to us by Cloudflare, one of the world’s largest content delivery networks, this trend is only increasing. As of August 2018, more than 37 percent of Cloudflare’s large-business customers block their website in at least one country.

Sometimes an unavailable website is merely an inconvenience – I can’t order my Irish friends a pizza from the U.S., for example. Other times geoblocking can really cause problems. We encountered an Iranian student who couldn’t apply to graduate school abroad because the admissions website wouldn’t accept payment of the application fee from Iran. Another person may be unable to read the news from a major international city, or plan a trip abroad because travel websites are all unavailable from their home.

Geoblocking is ineffective

Restricting access based on geography is unlikely to affect all internet users equally. As when evading censorship, getting around a geoblock isn’t necessarily difficult. But it might be expensive, expose users to additional tracking of their online activity, or require a level of technical literacy that not everyone has. Even if a user can ultimately access the content they were originally denied, they may bear a significant burden to gain access to the wider internet.

It’s also not easy – or necessarily accurate – to identify an internet user’s physical location. Using a computer’s numeric IP address to estimate where in the world it’s being used is notoriously unreliable. At least some users are likely being unfairly denied access to online services because their network address is determined to be somewhere they are not. However, rather than expanding the accessibility and accuracy of geoblocking, our group is encouraging researchers to address the needs of websites while maintaining as open an access policy as possible.

The internet has indelibly changed the world and the way people connect and do business. Researchers are working hard to keep this valuable resource available to everyone. Companies shouldn’t thwart those efforts by discriminating against users only because of where they are when they connect.

Allison McDonald does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.