Internet

Exchange Points
A closer look at the differences between continental Europe and the rest of the world.

Stephanie Silvius Amsterdam, January 2011

Table of Contents
Abstract ................................................................................................................................. 4 1 Euro-IX ............................................................................................................................ 5 2 The research ................................................................................................................. 6 3 Transit, peering and IXPs ......................................................................................... 7 3.1 How the Internet works ................................................................................................. 7 3.2 Autonomous Systems (AS) ............................................................................................ 7 3.3 Tier hierarchy ................................................................................................................... 9 3.4 Transit vs. peering ........................................................................................................ 10 3.5 What is an IXP? .............................................................................................................. 10 3.5.1 The purpose of an IXP .......................................................................................................... 11 3.5.2 Connecting to an IXP ............................................................................................................ 12 4 A brief history of IXPs ............................................................................................. 13 4.1 Europe .............................................................................................................................. 13 4.1.1 Western Europe ...................................................................................................................... 13 4.1.2 Northern Europe .................................................................................................................... 14 4.1.3 Central Europe ........................................................................................................................ 14 4.1.4 Southern Europe .................................................................................................................... 15 4.1.5 Eastern Europe ....................................................................................................................... 15 4.2 North America ................................................................................................................ 16 4.3 Asia .................................................................................................................................... 17 4.3.1 China ............................................................................................................................................ 17 4.3.2 Japan ............................................................................................................................................ 19 4.3.3 India ............................................................................................................................................. 19 4.3.4 Nepal ............................................................................................................................................ 20 4.4 Latin America ................................................................................................................. 20 4.4.1 Brazil ........................................................................................................................................... 20 4.4.2 Argentina ................................................................................................................................... 21 4.5 Africa ................................................................................................................................. 22 5 IXP models .................................................................................................................. 24 5.1 Commercial vs. non-commercial ............................................................................. 24 5.1.1 Neutrality .................................................................................................................................. 24 5.1.2 Ownership and management ............................................................................................ 24 5.1.3 Fees .............................................................................................................................................. 25 5.1.4 Scope of activities .................................................................................................................. 25 5.1.5 Peering agreements and policies .................................................................................... 26 5.2 Europe .............................................................................................................................. 29 5.3 North America ................................................................................................................ 29 5.4 Asia .................................................................................................................................... 30 5.5 Latin America ................................................................................................................. 30 5.6 Africa ................................................................................................................................. 31 6 Current developments and trends ..................................................................... 32 6.1 IXP participants Point of View .................................................................................. 32 6.2 Trends ............................................................................................................................... 33 6.2.1 Japan ............................................................................................................................................ 34 7 Differences ................................................................................................................. 35 7.1 Connected parties ......................................................................................................... 35 7.2 Influence incumbents .................................................................................................. 36 7.3 North American vs. European IXPs ......................................................................... 36

7.4 Difference in ASNs in Europe vs. US ........................................................................ 37

8 Conclusion .................................................................................................................. 39 Glossary of terms ............................................................................................................ 40 Acknowledgements ........................................................................................................ 41 Bibliography ..................................................................................................................... 42

Abstract

This document describes the research that has been done between August 2010 and January 2011 for the European Internet Exchange Point association. Euro-IX gathers resources about the European IXP scene to be able to make future predictions and improve the IXP community. The research started at looking into the differences between IXPs in the US and Europe, however this soon developed into looking at the differences between IXPs around the world, as it was thought that the initial research would be either very limited or had to be too thorough for the available time. By looking at the entire world a better understanding of the global IXP scene would be gained which would also be of use for Euro-IX, since they have members from outside of Europe. Proper knowledge was needed on the purpose and workings of IXPs and this was gained rapidly. After this a brief history on IXPs in all continents was made to understand their history and current situation a little. After this the differences between IXPs were mapped and located in each continent. As a follow up current trends and developments were looked into, which led to being able to make some future predictions. After doing the research, it can be said that its very difficult to compare the different IXPs to each other, since they are simply too different. Over the next couple of years we can see more IXPs being established in Latin America, Africa and (parts of) Asia, where currently there are very few IXPs present. These IXPs will probably be of a non-profit nature, while in the future more commercial IXPs might also show up. In North America commercial IXPs are dominant, while in Europe there are more non-profit IXPs but the number of commercial IXPs is increasing and is likely to continue to expand over the next couple of years. In North America the connection between networks are mainly private interconnections, while in Europe public peering is more common. European IXPs are likely to offer more services towards their participants, which might also be the case for the now less-developed regions. More peering relationships will be established with the Asian IXP scene, though this will probably be a slow development due to the high transit prices in Asia. It can be said with some certainty, that a large increase in exchanged bandwidth will be seen over the next couple of years, which will lead to a need of IXP platform upgrading to keep up with this demand.

1 Euro-IX

In May 2001 European Internet Exchange Points (IXP) in Europe decided there was need to combine their resources to strengthen, develop and improve the IXP community. This led to the formation of the European Internet Exchange Association (Euro-IX). This development and improvement of the IXP community is achieved by the coordination of technical standards, the development of common procedures, sharing ideas, knowledge and sharing public statistics and information between IXPs. To help accomplish this Euro-IX organises biannual two-day long meetings that all Euro-IX members can attend. They also provide mailing lists and on-line resources. Euro-IX currently has 56 affiliated IXPs from 37 countries around the world and is located in Amsterdam, The Netherlands. There is interest in having a clearer understanding of the differences between the continents as this can help give a better overview of the current situation of IXPs and developments around the world. One of the questions that have been raised for some time now, is why there is a large difference between the amount of unique ASNs at IXPs in Europe compared to the amount connected to North American IXPs, while the total number of advertised ASNs on both continents is quite similar. This is one of the questions that have been examined in this research and an attempt to answer this question will be made.

2 The research

The European Internet Exchange Association (Euro-IX) has been gathering data about IXPs, the parties connected to them and infrastructure to exchange the traffic for almost 10 years now. This has allowed them to produce reports on the current developments and trends, which they now want to use to produce more insights into trends that have been occurring and work on future predictions. Euro-IX is interesting in usage trends that are occurring in other regions than Europe and how these compare to Europe. What the IXPs look like in the US, if there are differences in the way that Internet traffic is handled, if Asian ISPs are as likely to make use of IXPs as those in Europe and if future predictions in different regions look similar. Euro-IX hopes to gain a better picture of the differences between the European and US peering ecosystems, what has and still does influence the amount of organizations that are present at public peering points on both continents, what makes European IXPs, relatively speaking more popular than their counterparts in the US? With a better understanding of these peering ecosystems we hope to better understand and predict the future of the global peering scene. It has to be delivered in a document describing the results and outcomes. Euro-IX wishes to have better insights in trends occurring around IXPs, not only in Europe but also in the US. To be able to do this, better insights are needed of the past and current situation on this continent, what differences there are and how they compare to each other. It is assumed that the main difference in IXP ecosystems between Europe and the US is that in the US major peering points are in fact run by colocation operators with the goal of making a profit via offering collocation space rather than simply serving the community and this has a direct affect on the amount of ASNs seen in the US.

3 Transit, peering and IXPs

Internet is available all over the world, but how do you reach a website that is located on a different continent? Where does the traffic go when you make an information request and how does it reach its destination? All of this is done by Internet Service Providers (ISPs) who either peer or buy transit and is assisted by Internet Exchange Points (IXPs). This chapter explains the basics of how Internet traffic is transported over the world.

3.1 How the Internet works

The definition of The Internet from Dictionary.com [1]: a vast computer network
linking smaller computer networks worldwide (usually prec. by the ). The Internet includes commercial, educational, governmental, and other networks, all of which use the same set of communications protocols.

When you are sitting behind your computer and are connected to the Internet, you can type in a URL in your web browser to request a webpage, hit enter and almost instantly the requested webpage appears on your screen! Or when you message a friend with a chat client, your message appears almost immediately on his screen and he can even message you back. Whats happening behind the scenes is what we call the magic of the Internet. The moment you hit enter, your request or the message you want to send, is transported to your ISP. This is the company that you buy your Internet access from. As a customer of an ISP, you are part of the ISPs network. When you request a webpage, it will be transported to your ISP, from which it will be transported to other networks (The Internet). Before your request reaches its destination it probably went through a lot of different networks (depending on where your request is destined). All of these networks that your request may travel through are called Autonomous Systems. One Autonomous System may be made up of smaller networks, but this is not always the case. All of these Autonomous Systems connecting together make the Internet, as we know it today.

3.2 Autonomous Systems (AS)

Throughout the world there are networks that are used to distribute traffic to and from other networks. These networks are called Autonomous Systems (AS) [3]. Every AS has a unique Autonomous System Number (ASN), which is used to

identify the network. In general ISPs and content providers have their own unique ASN, which is used to identify their network to setup BGP [2] sessions. This makes it possible for other networks to connect to them and exchange traffic. Having an ASN is also a requirement of connecting to an Internet Exchange Point (IXP). Figure 2 is an example of how different ASes are connected to one IXP with peering relationships to another AS. The rules of who to peer with, using the IXP, really depends on the IXPs peering rules. Some want all their participants to peer with everybody, while others leave the option of who to peer with up to the participants themselves. Setting up a (peering) relationship with another ISP (or content provider) is usually entered into to decrease the costs for interconnection. Furthermore, the number of AS hops that are required for the traffic to traverse and reach its destination can be reduced. In general IXPs dont interfere with the relationships that are created between their participants.

3.3 Tier hierarchy

While in general Internet Service Providers (ISPs) offer Internet access to their customers (home-users or businesses), there is a difference in the status of an ISP. The status of an ISP is called a Tier. While there is no official definition of what exactly a Tier 1 ISP actually is, it is generally thought of as an ISP that can reach any other network on the Internet without the purchase of transit or any other form of paying settlements. This means Tier 1 ISPs only peer with other Tier 1 ISPs and require all other ISPs to pay transit to them for transporting their traffic and make them accessible to the rest of the Internet. Tier 2 ISPs are usually regional Internet Service Providers, they tend to peer most of their traffic with other networks. When traffic cannot reach its destination via peering, the ISP will have to purchase transit from an Upstream Service Provider (Tier 1 ISP) to gain connectivity to other networks. A Tier 3 ISP is usually known as a local Internet Service Provider that solely purchase transit from other networks to make connectivity for their customers available. They either buy transit from their Upstream Service Provider, which can be either a Tier 1 or a Tier 2 ISP, or they peer with other networks. Figure 3 shows a Tier 1 to Tier 3 hierarchy.

3.4 Transit vs. peering

Traffic can be transported over the Internet in two ways: over transit lines or via peering. Transit is used between two parties, usually from an ISP to an Upstream Service Provider (usually Tier 1 or Tier 2) to transport the traffic via a paid agreement. The ISP under contract has to pay for traffic that goes over the line in both directions, so both incoming as well as outgoing traffic. The ISP pays the Upstream Service Provider to transport the traffic further to its final destination and to have a better connectivity to the Internet. Peering is known as a way to transport traffic between two networks for free, however in some cases paid peering is also practised. When two parties peer and they have an equivalent amount of traffic exchanged, there is usually no payment involved. When the traffic ratios arent alike paid peering can be an alternative, the network receiving most incoming traffic will generally receive payment. Apart from the costs for peering (either free or paid), the network infrastructure between the two parties needs to be maintained as well, usually both parties are responsible for maintaining the infrastructure. A peering relationship is very common and useful between two parties that send a lot of traffic to each other, if they were to pay transit it could potentially cost a lot more money and the connected party that has to pay doesnt have that much influence on for example, line upgrades or switches (which they require when they grow rapidly). There are two different types of peering: public and private peering. When three or more parties decide to peer publicly they will connect to a single point (switch). This public point of interconnection is also called an Internet Exchange Point. Private peering happens when two networks decide to interconnect directly with each other. This is usually done at datacenters (also known as carrier hotels), where a direct connection can be provisioned between two participants, both located in the same building. This requires a physical cable to be laid between the two ports/switches/networks to make private peering possible. Most of the traffic that goes between the largest networks is accomplished via private peering.

3.5 What is an IXP?

IXP is an acronym for Internet eXchange Point, which is a physical point where Internet traffic can be exchanged. This can be something as simple as a switch in a basement, to a large network spanning multiple datacenters, with power backup and cooling systems. An IXP provides the physical infrastructure where ISPs, universities, incumbents (Telcos), content providers and many more entities can connect to each other, by setting up a (peering) relationship to exchange traffic. The word we use nowadays to describe the physical point of interconnection has changed over the years. In the very beginning when the first scientific networks were being created, they called these interconnection points Network Access Point (NAP). Later this changed to being a Metropolitan Area Network (MAE) when IXPs became commercial and in the present days we know it, particularly in Europe, as an Internet Exchange Point (IXP). However, they all mean the same

10

thing: a physical point where different networks can exchange traffic with each other. In Figure 4 you can see different parties connected to a switch at an IXP. When the IXP is non-profit the parties connected to the IXP are generally referred to as members. However, when the IXP is commercial, its participants are referred to as customers. In this paper we refer to the parties connected to the IXP as participants, irrespective of whether the IXP is non-profit or commercial.

3.5.1 The purpose of an IXP The idea of an IXP was started as a way of keeping local Internet traffic local. This means, not using international (transit) lines to reach a destination in the same region as where the request originally came from. IXPs were introduced so that all parties willing to participate could connect at one (local) point, exchange traffic, save money and decrease Round Trip Times (RTT). Using an IXP also resulted in better network performance and QoS (Quality of Service). International lines are always available to use in case an IXP happens to fail, so the end-user wont notice much in terms of performance should this occur. When customers of an ISP enjoy their Internet experience, they are generally willing to pay more for their connectivity and speed. Money paid by customers can be invested in the infrastructure and services to the customers and not simply in international transit costs. This is good for the local economy. With fewer costs

11

for international traffic (because almost all the traffic can be exchanged via the IXP), the prices for consumer internet can go down, which will lead to more customers for the ISP. When a good local Internet infrastructure is available, other businesses and organizations are more willing to establish themselves in the same region, which leads yet again to more customers and also more content for the ISP. 3.5.2 Connecting to an IXP An IXP may have certain requirements that potential participants must meet before they are able to connect to their network. One very common requirement is that the party wishing to connect needs to have an ASN. What the specific joining requirements are really depends on the individual IXP. While their main purpose might be the same, their business models may differ and this may have an influence on your decision to join an IXP or not. Besides that you may also need to have an idea of the amount of bandwidth you plan to exchange on your port at the IXP; if you have any requirements on your housing location its a good idea to look into this as well, since most IXPs have their infrastructure at spread over multiple datacenters.

12

4 A brief history of IXPs

4.1 Europe

In this section a brief introduction will be given to cover the history and the present day situation of IXPS in regions around the world. Many of the first Internet connections in Europe were initiated by academic- and research-networks [4] or by telecom operators. Together they gathered to create one local point of interconnection, which resulted in IXPs. In the early days of Internet Exchanges in Europe, there was less competition between ISPs, because the biggest telecommunications operator (Telcos) kept the costs of transport, both for transit as for transatlantic, very high. Most of the traffic around this time was destined for America, because all content was available there. The local ISPs were forced to work together to reduce the costs for transporting traffic to North America. After a few years more content became available in Europe, because there was more local interest and this started to play a part in the development. The availability of local content changed the traffic flow. More traffic stayed within the European borders and less traffic went overseas, this most likely played an important role in forming the interconnection landscape. The very first European IXPs were established in 1993. This was around the time when Internet access became more readily available to the home users and more traffic started being exchanged. More traffic had a local destination, which resulted in the need for a local exchange. Peering helped to keep the end-users costs low, the transit and transport costs were high while at the same time the ISPs stayed competitive towards North American based providers. Today, the Internet penetration [5] in Europe is 58.4%, and makes up 24.2% of the Internet users worldwide. 4.1.1 Western Europe In this document we define Western Europe as: Belgium, France, the Netherlands, Luxembourg and the United Kingdom. The very first Internet Exchanges in Western Europe were established in 1994. These IXPs were named LINX (London Internet Exchange) and AMS-IX (Amsterdam Internet Exchange). Today, they both have more than 300 connected members from diverse locations around the world, with traffic peaks of over 1.1Tbps at AMS-IX [6] and more than 700Gbps at LINX [7]. There are approximately 30 IXPs spread across Western Europe. When looking at the IXPs in France, you can see that almost all their IXPs are located in and around Paris, with the recent creation of France-IX supported by Akamai, Jaguar Network, Neo Telecoms and Interxion in the hope of gaining better connectivity around France. The Internet penetration [5] in Western Europe is 2nd highest in Europe, United Kingdom has 82.5%, the Netherlands with 88.6%, Belgium 77.8%, Luxembourg 85.3% and France 68.9%.

13

4.1.2 Northern Europe In this document we define Northern Europe as: Denmark, Finland, Iceland, Norway and Sweden Northern Europe is the most progressive region of Europe based on Internet access and technology. The first IXPs in Northern Europe were established in 1993 in Finland, going by the name of FICIX the Finish Commercial Internet Exchange, created by the Finish telecoms and NIX, the Norwegian Internet Exchange in Oslo, established by the University of Oslo. In 1994 the Danish Internet Exchange (DIX) was established in Copenhagen and a few years later, in 1996 two more IXPs were established Netnod in Sweden and NorrNod in Norway. After this the development of IXPs the Internet connectivity grew tremendously. Together with Western Europe, Northern Europe has always been one of the leading regions in Europe as far as Internet connectivity goes. The Internet penetration [5] in Northern Europe is the highest in Europe with a penetration of 86.1% in Denmark, 85.3% in Finland, 94.8% in Finland, 92.5% in Sweden and 97.6% in Iceland. This makes Northern Europe possibly the best- connected region in Europe with Western Europe following close behind. Figure 5.1 shows all European countries that have one or multiple IXPs. The countries in grey are those that do not yet have an IXP. The white countries are outside of European borders. 4.1.3 Central Europe In this document we define Central Europe as: Austria, Czech Republic, Germany, Hungary, Liechtenstein, Poland, Slovakia, Slovenia and Switzerland. In 1995 one of the todays largest IXPs was established in Frankfurt, Germany known as DE-CIX [8] (Deutsche Commercial Internet Exchange). They currently have more than 300 customers with IXP traffic exceeding 1.2Tbps. Germany has an Internet penetration of 79.1%. In other Central European countries many smaller to mid-sized IXPs can be found. After DE-CIX, the largest IXPs in Central Europe are NIX.CZ in Prague, Czech Republic and PLIX in Warsaw, Poland both with traffic peaks in excess of 100Gbps. In Central Europe the Internet

14

penetrations [4] is between 58.4% and 75.3% (Germany not included). Its interesting to see that the country with the lowest Internet penetration (Poland with just 58.4%) has one of the biggest traffic exchanging IXPs in Central Europe, while Slovakia; with their IXP exchanging less traffic (about 40Gbps) has an Internet penetration of 74.3%. 4.1.4 Southern Europe In this document we define Southern Europe as: Albania, Bosnia and Herzegovina, Bulgaria, Croatia, Greece, Macedonian, Italy, Portugal, Serbia and Spain. Most of the Southern European IXPs were established around the late 1990s. In 1997 ESPANIX (Spains Internet Neutral Exchange Point) was established in Madrid connecting the most important Spanish telecom operators and are currently one of the largest IXPs in the Southern European region. Other countries with notable amounts of traffic at their IXPs are Bulgaria, Italy and Greece. The Internet penetration [4] in Southern Europe is significantly lower compared to that of the Northern and Western European regions, here it lies between 40% and 65%, while Bosnia Herzegovina at just 31.2%. 4.1.5 Eastern Europe In this document we define Eastern Europe as: Belarus, Estonia, Latvia, Lithuania, Moldova, Romania, Russia and Ukraine. Most the IXPs in Eastern Europe were established around 2000 and 2001. When looking at all the countries in this region, you can see that the traffic exchanged in Ukraine and Russia are tremendously high compared to the other countries, where most of the time the traffic is between 1Gbps and 10Gbps. The UA-IX (Ukrainian Internet Exchange) exchanges about 220Gbps at its peak and the MSK-IX (Moscow Internet Exchange) has peak traffic of around 400Gbps. Its interesting to see that quite a few Eastern European telecom operators and businesses connect to DE-CIX in Germany to get more routes to the rest of the Internet. This is probably due to Germanys geographical proximity to the Eastern European region. The Internet penetration [4] in Eastern Europe is comparatively lower to that of other European regions, however its growing rapidly, thus the penetration will potentially be even higher in the next couple of years. Estonia, Latvia and Lithuania have the highest Internet penetration with 75.1%, 67.8% and 59.3% respectively. In Belarus this figure is 46.2%, Moldavia 30.0% and Romania 35.5%. Its interesting to note that the Internet penetration is somewhat lower in Russia with just 42.8% and Ukraine with 33.7%, while the amount traffic at their IXPs is substantially higher than other countries in not only the region but the whole of Europe.

15

4.2 North America

The North American history of Internet is sometimes referred to as the beginning of the Internet. People often seem to refer to the history of ARPANET and NSFNET [9]. The reason for this probably being that North America is seen as one of the first countries with Internet access. The history of North America goes something like this: In the late 1960s the US Department of Defenses Advanced Research Project Agency (ARPA) was unsatisfied how the many research institutions and universities used located around the country, were unable to easily exchange results of research with each other. This was due to the fact that all institutions and universities used their own customized computers, who Figure 5.2 shows the North American states fit the requirements for their own that have one or multiple IXPs. The red state research. To make the exchange (New York) has been calculated to have more of results easier, ARPA wanted than 6 IXPs. The green state (California) was a network that could easily counted to have above 10 IXPs. connect all the different computer systems to each other, so that exchanging information would be a lot faster and easier. This network was eventually developed and named ARPANET. In the late 1980s a new network replaced the ARPANET, because it couldnt suffice the needs and demands of the researchers and many universities anymore. The new network was sponsored by the National Science Foundation (NSF) and comprised of five North America. Together they formed the NSFNET Backbone, basically forming a spine for the Internet. In 1989 Federal Internet Exchanges (FIX) were built on the East and West Coast (FIX East and FIX West). Soon after this the first Commercial Internet Exchange (CIX) was built on the West Coast of North America, due to the fact that the people in charge of the FIXes were troubled about connecting commercial networks to the NSFNET, they wanted to keep the two of these separate. In 1992 Metropolitan Fiber Systems (MFS) built the first Metropolitan Area (MAE) network the Washington DC area. This quickly grew into a place where many different (commercial) networks were able to interconnect with each other. Before the 1990s the Internet was mainly used for research purposes. However, this changed when email penetrated into the general community. More and more businesses and organizations saw the benefit of the Internet and decided to

16

connect to the network. This became a burden for the NSFNET, as there were too many people while not enough capacity to handle all this traffic. To ensure connectivity to all connected companies and organizations the National Science Foundation awarded contracts to four Network Access Points (NAP). Different companies spread across North America ran all these NAPs. The companies that received these contracts were: The Pacific Bell NAP in San Jose (California), the Ameritech NAP in Chicago (Illinois), the Sprint NAP in Pennsauken (New Jersey) and the already existing MAE East, operated by MCI WorldCom in Vienna (Virginia). In the coming years the amount of traffic started to reduce, because larger networks decided to set up a private interconnection and the smaller networks were limited to regional public interconnections Today, approximately 90 Internet Exchanges are located in North America and most of them are operated as a commercial service. These commercial Internet Exchanges havent been confined to within the American borders, but have reached out to Europe and Asia as well. Non-commercial Internet Exchanges can also be found in North America, however numbers are far less. How much IXP traffic is being exchanged within this region is a little unclear, as many of the IXPs do not publicly display these statistics. About 5% of the worlds population is located in North America with an Internet penetration [5] of over 75%. The US makes up about 13% of the total number of Internet users in the world.

4.3 Asia
There is a large difference between the countries within Asia with their level of Internet access and connectivity. For instance countries like China and Japan are more developed than countries like Vietnam, Indonesia, Nepal and India, where the Internet penetration is significantly lower. When you look at the population statistics [5] from Asia, you can see that 56.0% of the world population is located in Asia. Of the total Asian population, only 21.5% has access to Internet. In the last 10 years the Internet growth has increased tremendously, and this seems to be a continuing trend. About 42.0% of the total Internet Users in the world are located in Asia. This would suggest that a lot of traffic is going through the Asian regions. 4.3.1 China One of the very first Internet links (64 kbps) in Hong Kong was in 1991 by the Chinese University of Hong Kong (CUHK) [10]. However, back in those days it was still under British rule. A year later all higher educational institutions in Hong Kong were linked to each other over this link. They called this network HARNET (Hong Kong Academic and Research NETwork). In 1994 when a few small ISPs were established the Internet on the non- academic side started to grow while serving mostly individual subscribers. The real growth of Internet started in 1995 as more and more companies started to see the importance of the Internet. This caused a major impact on the availability of local content and intra-Hong Kong communication became more of a necessity. However a large number of ISPs still had to rely on other ISPs that had a local presence to complete there do transit for them. This massive growth of Internet caused a need for a better network infrastructure so as to lower the costs of operations. It was not very effective for ISPs to route their intra-Hong Kong traffic overseas, because the links were

17

expensive and relatively slow, thus CUHK set up the framework for HKIX and by negotiating with newly established ISPs with links to overseas and agreed on getting a link to CUHK. The ISPs placed their own routers at CUHK, while they provided the space, electricity, and a network to connect all routers of the participants. The philosophy of HKIX is that every participant is equal and will be treated fairly and together make certain that traffic is routed over the fastest possible path and all participants can gain the greatest possible benefits. The growth of HKIX has led to having almost 90 licensed and over 40 non- licensed members with aggregated traffic peaks from over 120Gbps and still growing The largest network provider in China is ChinaNet, owned by China Telecom. They carry about 80% of traffic from the Chinese mainland, excluding traffic from Hong Kong. In 2000 Chinas first NAP was opened: Beijings National Internet Exchange Centre and connects all major network providers. In the same year the Shanghai NAP was established, mainly to connect major ISPs in Shanghai and its surrounding districts. The Internet penetration [5] within China is about 31.6%, this is about 50.9% of all Asian Internet users and is the leading country in Asia as far as the number of users is concerned.

Figure 5.3 shows the Asian countries that have one or more IXPs. The white countries have no known IXPs at this moment.

18

4.3.2 Japan In 1975 an academic project [11] was initiated by the Japanese Government, which was developed to connect Tokyo University and Kyoto University. Which were located 500km apart from each other. This network was given the name N1 Network. In 1981 the network between the universities was officially in use. However the N1 Network had certain limitations, such as: e-mail support, it couldnt interconnect with networks outside of Japan (due to the specific protocol use it wasnt compatible with other networks, which made it different from the ARPANET); and the encoding of the Chinese characters was a problem. A few years later, in 1984 JUNET (Japan University NETwork) was established, with a connection over a telephone line to the Tokyo Tech University, Keio University and Tokyo University. This network was supported by KDD Lab (now KDDI Lab) and NTT Musashino Lab, both being technology research centers. The first Internet Exchange built in Japan was in 1988, started as a project by WIDE (Widely Integrated Distributed Environments) to experiment with new computers and communication technologies. The first ISP business was started by AT&T Jens and IIJ (Internet Initiative Japan) in 1993 as the first Japanese Commercial Internet Service Provider. In 1997 the first commercial Internet Exchange was started, with the name JPIX with the purpose of providing neutral internet traffic exchange services for ISPs and Content Service Providers so they can exchange traffic economically and effectively. In 1994 NSPIXP was firstly launched as a Japanese IXP, designed and operated by WIDE. Over the years they have led to the current state of development of the commercial IX Service. 4.3.3 India In 1992 the National Centre for Software Development (NCST) assigned [12] and maintained the .IN Top Level Domain (TLD). However after some time these duties were transferred to the Centre for Development of Advanced Computer (C-DAC). This process of getting an .IN TLD was difficult, expensive and a very long process with restrictive eligibility requirements. During this time there were around 6.600 Domain names in the database. In 2004 the government of India decided to transfer the registration of the .IN Domain Name from the National Centre Of Software Technology to NIXI, the very first Internet Exchange in India, which was established in 2003. This was done with help from the Internet Service Providers Association of India (ISPAI) and the Department of Information Technology (DIT). NIXI is a non-profit company. Today NIXI has 30 major ISPs connected, with 67 connections at 7 different locations. At this moment there are about 50.000 Domain Names registered, with 57 Registrars appointed across the world. Qualified people have been assigned to reduce conflicts and maintain the management of the domain names. They have an Internet penetration [5] among their population of 6.9% and make up 9.8% of the total Internet users in Asia. This doesnt sound like a lot, but keep in mind that China already holds 50.9% of the total Internet users of Asia and Japan has only slightly more Internet users than India with 12% of the Asian total.

19

4.3.4 Nepal The Internet market in Nepal [13] is growing rapidly. With less than 1 percent of Nepals population using the Internet in 2005, this number is growing fast. A competitive market between ISPs and low prices to Internet access are the cause of this. Right now 31 private ISPs offer Internet to businesses and home-users, but only two of them dominate the market with a total share of over 70%. These ISPs are WorldLink and Nepal Telecom. An important source to Internet access for the Nepalese are cybercafs, it is believed to have one of the highest concentration of cybercafs in the world. The Kathmandu Valley region has a higher penetration of Internet access, because its more developed while being hard to gain access to the mountain regions where low incomes populations can be found. The Nepali Wireless Networking Project is running an initiative to make Internet access possible in the rural populated mountain regions. In 2002 the first Internet Exchange was established in Nepal, called npIX. Currently they have peak traffic of over 100Mbps with 17 connected ISPs, including the two most dominate ISPs. Nepal has an Internet penetration [5] of 2.2%, and make up about 0.1% of the total users in Asia.

4.4 Latin America

The Internet penetration in Latin America amongst its population is 39.5%. They hold 8.0% of the total Internet users in the world. However in the past 10 years, there has been a gigantic growth, in users, so Internet is becoming more and more common in this region. Of all the Latin American countries, Brazil is by far the largest of the Internet users with some 48.5% of the continents total and the Internet user penetration is at around 37.8%. The second biggest contributor to Internet usage in Latin America is Argentina, with 17% followed by Colombia with 13.7%. However, the Internet penetration amongst the population is significantly higher in Argentina, compared to Brazil, with 64.4% and Columbia with 48.7% respectively. 4.4.1 Brazil In 1962 a research foundation was established in the Sao Paulo region in Brazil, which was given the name FAPESP (Research Foundation Sao Paulo) [14]. In 1988 the ANSP (Academic Network Sao Paulo) was created as an academic network by FAPESP, which would be the first Brazilian network to be integrated with the Internet in 1991 and connected universities and research institutes with one another. The ANSP operated NAP do Brasil, in 2004 Terremark and FAPESP decided to work together by letting Terremark operate the NAP, this would benefit both parties concerned and the general development of Internet in Brazil. Besides FAPESP an Internet Steering Committee (CGI.br) was created in May 1995, with the aim of promoting and improving the participation of the society towards the implementation, management and usage of the Internet. One of the executive arms (Center of Study and Research in Network Technology and Operations, CEPTRO) of the Internet Steering Committee is responsible for projects and services that are aimed to improve the quality of Internet in Brazil. One of their projects was the creation of PTT Metro, which are the Brazilian

20

Internet Exchange Points, located in all of the major Brazilian cities. This project was started in 2004. The most important features of the PTT Metro are to be neutral and independent from commercial suppliers and offer high quality service. Today they have some 14 IXPs with an aggregated peak traffic of more than 30Gbps and more than 200 participants. The Internet Steering Committee is responsible for operating and managing the Brazilian Internet backbone. 4.4.2 Argentina In 1989 an association was formed by: ISPs, Datacenters, Solutions & Content providers and many other entities. This association was named CABASE and created NAP de CABASE in 1997. The purpose of NAP de CABASE was to provide a place where all Internet players Figure 5.4 shows the Latin American could peer efficiently. There were a countries that have one or more IXPs. few things to be considered before this The white countries have no known could be accomplished. The location IXPs operating within them. of the site was very important. It was eventually decided to locate it in Downtown Buenos Aires, the largest and most important city in Argentina and has many Internet players. The decision of the location was also based on the network availability for accessing NAP de CABASE. The next consideration was the model of the NAP. It was decided to use a cooperative and self-governed model. Any cost or investment would be distributed amongst the members of NAP de CABASE. Operating agreements as well as procedures are developed and voted on by the members. Each member has the right of one vote, regardless of its size. NAP de CABASE is also carrier neutral, so even today the site and infrastructure are not owned by any of their member, which minimizes the risk of being affected by problems of a member. The NAP has special agreements for either non-carrier or non-ISP members. They can only connect if their businesses dont affect any of the current ISPs or carriers that are connected. The average number of members connected to NAP de CABASE is 35. Since the beginning of the NAP multilateral peering was the only peering agreements possibility made available, but since two years ago, bilateral agreements have been permitted, but not

21

actively promoted. In November 2010 NAP de CABASE operated the first regional NAP in the Neuqun Province, starting with 7 members. It has a connection to the NAP in Downtown Buenos Aires with access to all the members and traffic there, with the same conditions as their members. This creates a huge cost improvement for the Argentina region. Plans for other regional NAPs have also been made, however no further details have come to hand. After the deployment of the first NAP de CABASE a growth in local hosting, content and e-commerce services was seen, this resulted in more local content becoming available.

4.5 Africa
In the last couple of years the Internet usage in Africa has grown tremendously, to such an extent that the current network capacity cant keep up with the demand, which actually led to falling behind on their growth. Right now they could be considered as being almost 10 years behind on Europe and North America, however this difference only looks set to grow bigger and bigger over the next couple of years. In the last couple of years the existing connectivity was mainly over satellites. This is very expensive, usually slow and sensitive to traffic congestion. Depending on the part of the African [15] region that you are looking at, the growth, state of Internet connectivity and access is very diverse. Some countries are well developed, while others are very poorly developed. This makes it very hard to connect all the countries to each other and provide better Internet connectivity and access within the African regions. One of the main reasons for the poor growth is poor infrastructure and very fragile end-to-end connectivity (for example, connectivity from the local ISP to the home-user), this is increased by a lack of adequate network backup routes, in case the power goes down or a cable is broken. Other disadvantages for Africa are lack of power, high import taxes (to get routers and switches into the country), lack of proper skilled technical staff (they tend to leave to more developed regions like North America or Europe, once they are skilled), corruption and monopolies. International companies and organizations have started projects to deploy proper Internet connectivity to a larger number of people throughout the region. Today, most of the African traffic is routed through Europe or North America even if the destination of the request is within African borders. Some of the traffic even needs to go through both Europe and North America to then go back to the African region. High international bandwidth prices are the biggest contributor to African ISPs, a lot of the traffic goes over satellite links, because fibre links are limited and expensive to deploy. High traffic costs and the poor connectivity can be improved by building regional and national IXPs, this also reduces the usage of international traffic lines. The few countries in Africa that do have access to international fibre dont feel the benefits of using this, because they are still charged at monopoly level prices by state owned operators, which still dominate most of Africa and in many other developing regions, so satellite links are used instead. This is beginning to change as many countries revise their competition frameworks as exclusivities, granted to incumbents, come to an end. In 2003 there were 10 countries of the 53 in the region that had an IXP, in 2006 this number had risen to 16 countries.

22

Today the total number of IXPs in Africa is 18 and this number looks to grow further. As more and more routes stabilise, more customers will subscribe to ISPs services. In time this will result in better QoS with a wider impact on the deployment of infrastructure in the regions. Local people are demanding better connectivity and the ISPs have no choice but to invest, so as to make more money. This is good for the local economy. Today fibre cables are being deployed right across Africa, however this is a long process where a lot can go wrong, such as sabotage on landlines that are, either broken or stolen, which keeps the incumbents from deploying faster. Some try to solve this by providing wireless access to their users, but this has other implications, which are out of the scope of this research. In the next couple of years the Internet connectivity should have improved from where it is today and it should be more accessible to everybody. At this moment some 14% of the total world population [5] is located in Africa, but only 10% of the total African population has access to Internet. This means there is a lot of potential to grow and create local Internet connectivity to home users compared to that of any other region in the world. Figure 5.5 shows the African countries that have an IXP. The white countries do not have an IXP. Some of the African countries have more than 1 IXP, however this is limited to just two countries, Egypt en South Africa.

23

5 IXP models

There are approximately 300 IXPs [16] around the world, however there are quite a number of differences between these IXPs. No IXP is the same, they may look quite a bit alike, but things will probably be just a bit different in each case. This section will take a look at the differences between the IXP models [16].

5.1 Commercial vs. non-commercial

A key factor to differentiating an IXP is whether it is commercial or not-for- profit. When an IXP is commercial its main goal is to make a profit and more often than not, operated by the same colocation company that houses them. One of the advantages of a commercial IXP is that they have the ability to charge strategic differential pricing, to attract more customers to connect to them. They also tend to approach their pricing based on what the market can withstand. Decisions about the commercial IXP are made by the commercial IX operator and not by the parties connected to them. Commercial IXPs also tend to compete with each other instead of cooperating. ISPs, content providers and other parties connected to a commercial IX are called customers, while for a non-commercial IXP these same participants are usually known as members. When an IXP is non-commercial, also known as non-profit (or not-for-profit) the members connected to the IXP generally have an influence on the decisions that may affect the future of the IXP. Non-commercial IXPs were established with the aim of assisting ISPs to save costs and to improve the end-users Internet connectivity speed, instead of being purely set-up to make a profit and this is possibly what has helped these IXPs become so successful. 5.1.1 Neutrality The neutrality of an IXP can be seen as an important factor of its successfulness. Some think its the key factor, while others see it as important, but not necessarily the main factor. There is no official definition of the neutrality of an IXP, this has to be decided by the IXP itself. For instance, it can decide to be carrier neutral, but not colocation neutral or possibly both carrier and colocation neutral. Or is the neutrality seen as only organizational? There is no right or wrong when you are talking about the neutrality, because it really depends on what you see as being neutral for an IXP. Other ways to stay neutral are not interfering with your customers or members, however it may be necessary to establish rules for your customers or membership base and inform them of what they can and cannot do. When the IXP decides to implement or change it services, do you listen to your members or do you stay neutral. All of this makes the definition of the neutrality hard to define. 5.1.2 Ownership and management When you look at the differences in ownership of IXPs around the world you can find quite some differences in the bodies running them. The body of the IXP is a very important factor. It decides who makes the decisions about the IXP and how these decisions are made. From an historic point of view the most common body is when an academic institution owns the IXP. This is where the idea originally

24

came from, however sometimes the ownership develops into a more formal body over the years. When this development occurs it usually leads to an IXP becoming Association based. Where a Board has been created who make major decisions about the future of the IXP. It is very common that the members of an association based IXP have voting powers and has some kind of influence on the future of their IXP. Another common body is the business unit. This type of IXPs aims to make a profit and usually provide the collocation space if the decisions made by the IXP management are aimed at making profit which they generally do by selling as much rack space as possible and tend to set their prices based on the market. 5.1.3 Fees There are a lot of differences between fees at IXPs. Some IXPs even claim to be completely free, meaning that they do not charge a port connection fee however they still may require the participants to pay for rack space and local transport to the IXP. In general, this is done by very few IXPs today. Depending on whether an IXP is commercial or non-commercial may also influence the level of the fees. Usually when an IXP is non-commercial you see a monthly fee you have to pay for access to your port, sometimes you also have to pay a connection or establishment fee. This is all really different per IXP. When looking at commercial IXPs, you can see a lot of differences in pricing as well. One of the reasons for this is that most of the commercial IXPs can do their pricing based on the market. This makes them more flexible in a way. Some IXPs want you to pay a one-time fee, and a monthly fee for your port. Some give the port away for free, because you are required to rent rack space in one of their colocation facilities, or dependent on certain services they offer. A very important factor when looking at the pricing between IXPs is whether they have volunteers working for them, or they are properly staffed. When you want to be sure of support, the decision is usually made to choose for dedicated personnel. It is also important to look at the equipment in this case, usually when an IXP is ran by volunteers, the equipment is usually funded by donations from companies, organizations and individuals, while a staffed IXP probably has more budget for equipment. Which means they are less dependant on third parties. 5.1.4 Scope of activities Which services an IXP offers to its participants is really dependent on the scope of their activities. These activities were decided upon when the IXP started, or when a We will see where it goes from there mentality didnt work anymore. What exactly is offered by an IXP is yet again dependant on whether it is commercial or non-commercial. Commercial IXPs usually provide their customers with colocation, hosting, security, customer support, SLAs etc. All these services are also used to make more profit. Even when looking at non-commercial IXPs the scope of activities also tends to differ somewhat, from just offering ports to full customer support, SLAs and rack space.

25

5.1.5 Peering agreements and policies Peering can be achieved on different levels, it can be implemented in a technical way, or it can also be implemented in a non-technical way, based on the peering policies from connected IXP participants. When its based on the peering policies of the IXP participant, it has nothing to do with the IXP itself, because its purely an agreement between two or more parties that connect to the IXP and describes how they want to exchange the traffic and under which conditions this has to be done. There are two ways to implement peering on a technical basis. This can either be done via private peering or through public peering where the traffic goes over an IXP. A definition of both can be found in the table below. Private peering Physical direct connection between only two parties Public peering (over IXP) Physical connection to a medium, managed by a third party, through which multiple parties are connected and approachable Table 6.1: Different ways to implement technical peering. The other possibility of implementing peering is in a non-technical way, where agreements are made between two or more parties. Every connected party to an IXP has a specific peering policy, in which it defines how they want to exchange traffic and under which conditions. The different peering policies are described in the table below. Every connected party can have one of these policies, based on their business model and peering conditions. This peering policy may differ per region. For example the local incumbent in a country in Europe can choose to have a restrictive peering policy in Europe, however when this incumbent expands it business to a different region, for example North America, it may decide to have an open peering policy as this may make more financial sense. The connected parties at IXPs in North America might be more interesting to peer with than the connected parties at the IXP in its home country, where they feel it will decrease their business. Policy Definition Open Peering Generally willing to peer with anyone, with no or few prerequisites. Selective Peering Generally willing to peer, however there are some prerequisites, once met this generally leads to peering. Restrictive Peering Generally not willing to peer extremely difficult to meet prerequisites with high possibility of denial once they are met. No Peering No intention to peer. Traffic is exchanged via transit agreements Table 6.2: Peering policies for IXP participants. The implementation of the peering policies can be done in a number of different ways. This happens on a non-technical way and is more agreement based on the

26

conditions made between different parties. The connected parties can implement exchanging traffic through buying transit, having a bi-lateral or multi-lateral peering agreement. The distinction between the three can be found below. Transit Gives access to the complete Internet Bi-Lateral Gives access to the networks of the parties you are peering with and their client-networks. Multi-Lateral Access to multiple networks from parties you are peering with, based on one agreement Table 6.3: Possible ways to implement peering policies. A current popular trend is paid-peering, where one party pays another party to peer with them, this is usually done on the difference in exchanged traffic between the two parties. This is a modification of the bi-lateral peering agreement. The term Mandatory Multi-Lateral peering is often used when every connected party at an IXP is forced to peer with every other party, this peering agreement can be seen as multi-lateral peering, where all connected parties agree to exchange traffic under the same agreement. If they do not agree to this agreement, there will be no peering available for them. Figure 6.1 shows a transit agreement. Network A buys transit from its Transit Provider, which provides Network A with connectivity to the Internet. .

27

Figure 6.2 shows a Bi- lateral peering agreement. Network C buys transit from Network B, when Network A and Network B have agreed to peer (bi- lateral peering), Network A will get access to Network C. Network C will also get access to Network A, because of the transit agreement between Network B and Network C.

Figure 6.3 shows a Multi- lateral peering agreement. Network B, D, E and G have agreed to peer under the same peering conditions, so they have access to each others networks. Network C still buys transit from Network B Network A and Network B still have a bi-lateral peering agreement, this provides network A with access to Network D, E and G and the other way around.

28

5.2 Europe
When looking at the different IXPs in Europe, you can see that quite a large proportion of IXPs are of a non-profit nature and most tend to be both colocation and carrier neutral. The main reasoning behind this neutrality is that the ISPs are more willing to join an IXP that isnt controlled or influenced by a third party. They maintain this neutrality by being located at one or more datacenters that are owned by separate entities. Thus no special treatment is available for any one participant nor the datacenter companies. Besides the neutrality there are also some differences in ownership of the IXP, however most IXPs in Europe are association based or are run by an academic institution. There are a small number of commercial IXPs in Europe, however this is generally limited to the smaller to mid-sized IXPs. There is a trend in some countries such as The Netherlands, Italy and France for region al governments to establish local IXPs, however to date, these havent grown to be all that significant. In general most of the IXPs in Europe just stick to their main goal of renting ports on their switching infrastructure, however there is a trend of late that has seen some not-for-profit IXPs extending their services to fulfil member needs.

5.3 North America

The majority of IXPs in North America are of a commercial nature. They generally dont use the word IXP nor NAP in their corporate name, which already suggests that being an IXP/NAP is not their primary goal. Their main aim goal is to make a profit by selling rack space to their customers. They also sell additional services like security, hosting and SLAs as an extra source of income. Being a datacenter makes interconnecting for two of the IXPs customers relatively easy, because only a physical fibre needs to be laid between the two AS networks. Quite a number of IXP customers locate themselves within a datacenter, to make interconnecting with other customers easier and to have better connectivity to other Autonomous Systems as the distance to other customers is relatively small at a datacenter. The huge competition between IXPs in North America makes pricing and selling the ports interesting. For example, they make it enticing for their customers to buy a 10Gbps port, compared to a 1Gbps port, by selling them very cheap or giving them away for free. However this leads to the customer having to buy more rack space as well and probably other extra services, before they can operate their port. Some non-profit IXPs can also be found in North America, a good example is SIX (Seattle IX), which is a neutral IXP with volunteers as their staff. Connected parties are dependant on the time the volunteers are willing to spend. The equipment used by them is based purely on donations. They are completely dependant on the donations of business, organisations and the time of their volunteers.

29

5.4 Asia
The Asian Pacific region is usually thought of as a closed community where interconnecting is hard to maintain and setup. This may be true, however its slowly becoming easier to interconnect in Asia. One of the biggest problems the Asian market is struggling with is the artificially high price of transit, transatlantic costs and interconnecting costs at IXPs. Most of the dominant ISPs still think: for you to get more routes to the Internet you have to buy transit from me. This is only partly true. It works for the ISPs, however it doesnt help achieve getting better connectivity in the country nor the region. The idea of peering does not interest them, either due to the fact that they dont know the benefits of connecting at an IXP or are afraid of losing business. This keeps the prices of Internet traffic very high for smaller ISPs and overseas companies. Creating an IXP would be easy, however a lot of parties dont see the benefit from doing this, because of the tremendous high prices of connecting to an IXP. Asia is also troubled with a lot of regulated telecoms. Prices in de-regulated regions are generally lower than regulated regions. Many countries in Asia are dependant on neighbouring countries for transit connectivity. Another problem in Asia are culture differences, To help solve all of the above mentioned problems its important for all involved parties to participate in such forums as APNIC, SANOG and Apricot. This helps to create a better understanding of the concept of IXPs and peering, but also gives people the opportunity to exchange ideas and talk about problems with more experienced people from all over the world. Its also a good source for future customers and peering relations. Both commercial as non-profit IXPs can be found throughout Asia. However, most of the more commercial IXPs can be found in wealthier regions, like China, Japan and Australia, while less developed regions like India, Nepal and the Philippines tend to have more non-profit IXPs.

5.5 Latin America

Most of the IXPs in Latin America are of a non-profit nature. The ownership of the IXP varies from government to association based to an organization of telecom operators. Most countries are heavily regulated by their governments. These regulations make it harder for companies (outside of Latin America) to connect to an IXP, or for local business to connect to an IXP in a different country. This holds back the region from making faster progress. Almost every country in Latin America has an IXP, Brazil has a large number of IXPs and are all operated by CGI.br. In Argentina NAP de CABASE operates the IXP. Some IXPs in Latin America force all their participants to peer with each other to encourage traffic on their platform, while other leave this option to their participants. When peering is forced, (international) companies tend to be less willing to peer at these IXPs. The growth of the Latin American Internet ecosystem will most probably grow faster, once one or more international companies have established themselves in this region and are interesting to peer with, this will attract even more international businesses. This will also make it easier to expand to other neighboring countries; which helps their Internet ecosystems grow even further. Currently large proportions of the Latin American

30

traffic goes via Miami and back because its cheaper than deploying fibers across the multiple countries.

5.6 Africa
The African region is poorly developed compared to other regions around the world. This makes it very interesting to take a look at how they are developing and building a peering ecosystem. All IXPs that are currently operating in Africa are non-profit neutral IXPs, this is probably also the only possible way to get Internet connectivity up and running. The threshold for ISPs is significantly lower to connect with each other when its a neutral exchange. The main reason would be the already existing (non) trust issues that are going on, doesnt make it much easier to start business with each other. Most of the IXPs in Africa are established either by the Government or a local organization of Telecommunications. There are a few countries in Africa that are well developed compared to other countries in the region. Nigeria is one of the best-developed countries in Africa as far as Internet penetration goes. Egypt has the 2nd best-developed Internet penetration in Africa. For the regions in Africa that dont have an IXP, special initiatives have been set up to establish IXPs with the local ISPs and provide connectivity between African countries. These initiatives are supported by international companies and organizations, which work together. Over the next few years a huge increase of African traffic can be expected due to the establishment of regional IXPs and the deployment of further Internet connectivity.

31

6 Current developments and trends

Currently a lot of developments are taking place in the world of the Internet. These changes, trends and developments are of importance to the future growth of IXPs around the world. This section will take a closer look at what developments and trends are occurring.

6.1 IXP participants Point of View

Over the past few years Euro-IX has invited IXP participants to attend the Euro- IX Forums [16] to share their opinions on IXPs. In this section the answers from these participants have been summarized to give a global overview on the participants point of view. During these discussions topics like: what IXPs should and shouldnt do; services they should offer to their customers; what their preference was based on (non-profit or commercial) IXPs; pricing; the future of IXPs and more were points of discussion. Besides the discussions at the Euro-IX Forums, other IXP participants were asked, by this research, to give their opinion about IXPs as well. The ideal IXP has a high uptime, low (port) costs and has lots of other participants to peer with. Furthermore, it should be easy for new participants to connect to with a fast connection procedure. It should have a skilled and functional 24/7 NOC, good infrastructure and resources, robust security, good stability with reliable services and it should also be physically near to ones equipment. There are also things IXPs should definitely not do, for example having all kinds of requirements (both for joining as well participating on the IXP) definitely keeps a party from joining an IXP, companies are definitely not interested in joining an IXP when they appear to look attractive but really are not (example: giving away free ports, which also requires the company to rent rack space or other unexpected costs), the IXP should also stick to their business by staying neutral and not competing with their participants. Cases have occurred where companies have left peering at an IXP due to operational costs. Opinions on whether or not an IXP has to offer additional services to their participants vary and were also limited. Replies varied from simply providing a looking glass or route server to Private Interconnection services, contact information of parties who other parties could connect with and VoIP peering facilitation. Generally participants dont have a preference as to whether an IXP is non-profit or commercial based, as long as the IXP does what it is supposed to do, is reliable and has a reasonable pricing. The price however can be an issue for the participants; it should be reasonable compared to the offered services and quality. So a good balance between those two seems to be of importance. Some things that participants feel that IXPs should be concerned about are rapid growth of exchanged traffic loads, which can cause the number of free ports to be limited. If not enough large interfaces are available the IXP participants will peel their traffic off the IXP and use Private Interconnections. Some also mentioned that the decrease in price of IP transit might influence the growth of IXPs, because itll be cheaper than connecting to an IXP. Another important factor is the scaling and capacity of the platforms. It is expected that more traffic will go

32

over Private Interconnections, it is also likely that this traffic will not return to the public fabric of the IXP. They also mention that IXPs should make sure they keep their current level of quality or even improve on this. The non-profit IXPs should work on transparency and IXPs should try to work together with Euro-IX to standardize service offerings to their participants, like IPv6, multicast, (virtual) PIs, etc.

6.2 Trends

The 2010 Euro-IX Report [17] on European IXPs shows some trends that are occurring in the IXP world, especially in the European region. One of the biggest changes since to last year is the huge increase of peak traffic at IXPs in Eastern Europe. Countries like Russia, Bulgaria, Ukraine, Romania and Latvia have grown tremendously and rapid developments can be expected over the next couple of years with a real contribution to the total European IXP traffic aggregate. Other regions, which are also developing rapidly, include Latin America (especially Brazil and Argentina) and the African region however lots of investments are needed to improve the infrastructure, although a huge potential is present. A large growth can also be seen in the number of unique ASNs that are peering at two or more IXPs in Europe. Between 2008 and 2009 this number had grown by about 20%, while from 2009 to 2010 this has grown some 24%. Between 2007 and 2008 certain growth factors of IXPs in Europe slowed down or even decreased somewhat. For instance the amount of traffic per unique ASN only slightly increased, as did the number of known operating IXPs, while the average traffic load per switch increase dramatically. While 2008 2010 saw strong growth in nearly all aspects of the European IXP scene. It is expected that the Eastern European region will maintain a strong growth rate in the coming years. When comparing the categorized and regional IXP annual growth rates, from 2009-2010, one can see the largest growth both in Medium and Small IXPs as well as those IXPs coming from the Southern Europe and Northern Europe regions. Comparing the global growth rated from 2009 and 2010 one can see that both European and North American IXPs have grown on average between 61% and 67%, Asia on the other hand saw growth rates of around 15% compared to those of last year, while Brazil saw growth rates of around 138% and the Global average was at around 57%. Another trend worth noting is the growth in route server usage by IXPs in Europe. At this moment 19 IXPs are known to be operating a route server, however it is estimated that more IXPs are in fact making use of them and it is expected that more IXPs will follow suit in the coming years. It is also apparent that the core business of the participants, connected to an IXP, has changed over the past 5-10 years. While a company used to cover just one particular area of business (for example a pure ISP), that same entity now operates in 2, 3 or more different areas of business (e.g. ISP, hosting provider, content provider etc). It is further worth noting that transit prices are still falling dramatically and it is uncertain what influence this will have on IXPs in the future. It might be possible that more traffic will go over transit lines, however it is also possible that the growth of traffic going over IXPs will continue. If over the next couple of years

33

the traffic exchanged at IXPs keeps growing at the current rate, this will result in IXPs having to upgrade their platforms to keep up with this fast development. It is also anticipated that IXPs may also offer more and a broader range of services to their connected participants, however this will naturally be different from IXP to IXP. 6.2.1 Japan A few years ago it was stated [18] that Japanese traffic stays for about 80% within Japan and about 5% of traffic destined to/from the US with estimated traffic of between three and four Gigabits per second. This would make Japan one of the more self-contained self-sustaining peering ecosystems. After calculations made from the statistics of the Ministry of Internal Affairs and Communications of Japan [19], it appears that this situation was indeed correct around 2005-2006, but the Internet situation has changed since then. May May May May May May 2005 2006 2007 2008 2009 2010 Inside 85% 78% 76% 75% 74% 69% Japan To 15% 22% 24% 25% 26% 31% outside Japan Table 7.1: Upload traffic from customers. May May May May May May 2005 2006 2007 2008 2009 2010 Inside 80% 74% 68% 67% 66% 57% Japan To 20% 26% 32% 33% 34% 43% outside Japan Table 7.2: Download traffic from customers. Above calculations represent only a fragments of the Japanese traffic from 6 ISPs.

34

7 Differences

As far as IXPs go, there are some major differences between the many regions of the world, however these differences can also extend to within a region. This section will cover some of the differences that are of importance, and some further questions have been raised by Euro-IX.

7.1 Connected parties

Over the last couple of years the diversity of participants connected to the IXPs have changed. In the table below you can see the type of parties connected to IXPs [16] in Europe, with the percentage of IXPs that have any of these type of connected parties. No such statistics are available for North America. 2006 2007 2008 2009 2010 Adult hosting company Content provider Dedicated hosting provider Dedicated VoIP provider Ecommerce companies (Ebay, Amazon etc.) Enterprises (banks, airlines etc.) Gaming Gambling Gaming/Gambling Government Search Engines 70.4 71.4 37.5 22.7 57.7 18.2 94.7 78.9 44.4 27.8 63.2 33.3 21.1 85.0 75.0 36.8 33.3 30.0 30.0 10.5 50.0 25.0 27.8 94.4 76.5 41.2 27.8 33.3 16.7 11.1 66.7 33.3 24.0 96.3 88.9 48.1 34.6 46.2 23.1 7.7 77.8 48.0

Note: Data for the empty fields was not available in that specific year. As can be seen from the table, a very large amount of the European IXPs have Content Providers and Dedicated Hosting and Governments are also very often connected to an IXP. Its important to note that in a country there are often multiple IXPs, parties such as Governments or Enterprises might decide not to connect to every single IXP in their country. As previously mentioned over the last 5-10 years a trend can be seen in IXP participants. Most participants have expanded their business, for example, a search engine 10 years ago, was just a search engine, however this same entity now also provides hosting or is a content provider and provides an email service

35

etc. This goes for a lot of participants; this also makes it harder to categorize the participants in any manner.

7.2 Influence incumbents

The incumbent is generally very restricted about whom they will peer with and under which conditions this is done. Just a select group of companies or organizations are able to peer with the incumbent. In general they only perform private peering, because public peering might decrease the value of their business or change the market for smaller ISPs who will only benefit from the incumbent, while they cant benefit from the smaller ISPs (because they are not interesting parties for them). The benefits from having the incumbent connected to the IXP and their peering policy differs per country. The market does have some influence on the peering policies of the incumbent, by influencing them to peer with other IXP participants. A lot of IXPs in Europe have their incumbent connected to them, though this isnt always the case. Generally it is seen that having the local incumbent or telecoms connected to an IXP really influences the growth and future of the IXP. This is especially true in a beginning stage of the IXP, however even later in the IXPs life the joining of an incumbent might still influence the growth of an IXP.

7.3 North American vs. European IXPs

In North America the new Internet backbone, which was formed by the four NAPS (Sprint, Ameritech, PacBell and MAE East), replaced the NSFNET backbone in mid 1990s. All public peering was done from inside the datacenters from commercial providers. A few years later the IXPs decided to start operating from inside their own datacenters, offering both private and public peering to their customers from inside their datacenters. Companies would generally limit their presence to only the most important locations to interconnect with other companies, because of the distances they had to cover. To reduce the costs for fiber, companies decided to set up Private Interconnections (PIs) to exchange traffic with other companies in the datacenter. Meanwhile when Europe developed its first Internet connectivity this was only for the universities, however later ISPs were also able to connect to the Internet. The ISPs then wished to interconnect with each other, however the incumbents were generally not interested in accommodating these wishes. The universities stepped in to provide a facility within their buildings where the ISPs could interconnect. This resulted in the first European IXPs. Since the universities cared less about the financial part for the interconnection this resulted in a non- profit model IXP. Since there was a lack of commercial data center providers, the non-profit model became the working standard, which became very successful. In general the idea was not to compete, but to work together to provide better connectivity. Europe has always had the convenience of the small proximity (compared to North America) between cities. This made it easier for companies to expand to different cities (also outside of their own country) to connect with other companies. This also made companies more open to exchange traffic over public lines, instead of only private peering. Which resulted in the IXPs being more open towards their participants but also towards each other.

36

7.4 Difference in ASNs in Europe vs. US

One question that is raised is why there is a big difference in unique ASNs at IXPs in Europe compared to those in North America. When looking at the table below, you can see that there are 3181 Unique ASNs at IXPs in Europe, while there are 849 Unique ASNs at IXPs in North America. When comparing this to the number of Advertised ASNs in Europe (15685) to North America (13979) the difference in unique ASNs peering at IXPs looks huge.

IXP Participants 5829 IXP Participants 1116 IXP Participants 2055 IXP Participants 313 IXP Participants 202 IXP Participants 9515

Europe Unique ASNs Advertised ASNs 3181 15685 Asia-Pacific Unique ASNs Advertised ASNs 647 4354 North America Unique ASNs Advertised ASNs 849 13979 South America Unique ASNs Advertised ASNs 186 1538 Africa Unique ASNs Advertised ASNs 114 436 Global Unique ASNs Advertised ASNs 4699 35992

% of Adv. ASNs 20.28 % of Adv. ASNs 14.86 % of Adv. ASNs 6.07 % of Adv. ASNs 12.09 % of Adv. ASNs 26.15 % of Adv. ASNs 13.06

Note: Numbers were taken on 20-12-2010 [16] In this table the IXP Participants are the total number of participants at the IXPs per region, when a participant is connected to multiple IXPs, it will be counted multiple times, since for every IXP this is a different participant. The Unique ASNs are the total number of unique ASNs at all IXPs in a particular region, so an ASN present at multiple IXPs in a region, is only counted here once. When comparing North America and Europe to each other, one can see that there are quite a number of differences present. For instance, in the U.S. one of the main ones would be the fact that the U.S. consists of states, however they belong to one country. While every state might have slightly different laws, they all speak the same language, generally require the same procedures to get a company started and often the people residing in the states have very similar needs to one another. This makes it easier for a company to expand to different states of the U.S. and run their services from there, because there will be less fuss with new laws and taxes etc. In Europe however this is completely different. Every single country has its own set of laws, language, culture, specific rules,

37

cultural needs and more. All of this makes it a lot harder for a company to expand to a new country. As can be seen in Figure 7.2 (Map of the U.S. states with an IXP), there are a large number of states that have no IXPs at all. This means that the physical location between cities (in the states) that already have an IXP (the ones without are probably less interesting for a company to establish themselves and do business) can be quite large, especially compared to Europe, where most of the larger cities have at least one IXP. This makes it more attractive for companies or organizations to expand to different countries. The costs of fibre will probably be less expensive, because the distance between two cities is relatively close, compared with those in the U.S. This makes it more expensive for companies to reach out to an IXP, let alone multiple IXPs. Especially for smaller companies its cheaper to buy transit instead of connecting to an IXP. The added value of the IXP wont bring enough reward to their business. When two companies do decide to exchange traffic in the same datacenter, this is usually done via cross connects, because the port prices are more expensive than the physical cable laid between their two networks (this cable is usually just within the datacenter itself, so not so expensive compared to renting a port at the IXP). Large quantities of traffic going between two networks often makes a cross connects more often and more financially interesting, but offering the IXP service can act as a great draw card.

38

8 Conclusion

During the process of researching these differences between IXPs around the globe following was discovered. Commercial IXPs have a dominant position in the US, while in Europe mainly non-profit IXPs can be found, however there are also some commercial IXPs present. In Asia there is a mix between commercial and non-profit IXPs. In more economically developed Asian counties (China, Japan, Australia etc.) it is more common to see commercial IXPs. In the economically less developed countries (Nepal, India, Malaysia, etc.) more non- profit IXPs are present. The main difference between commercial and non-profit is that when an IXP is commercial the colocation owner is also the owner of the IXP. In the US it is more common to use Private Interconnections instead of peering publicly over the exchange (which is more common at non-profit IXPs). In Europe IXPs were created as a way to decrease the costs that otherwise had to be paid to the incumbent to exchange the traffic, while in the US IXPs evolved simply as means of making profit. From what has been researched a few important predictions can be made about the future. Over the next 5-10 years, it is likely that we will see an increase of the number of commercial IXPs in Europe. It is likely that non-profit IXPs will establish in the countries across the world that currently have no IXP at all. Once these non-profit IXPs have been established the country will probably increase to be more interesting for commercial IXPs to establish as well. European IXPs will start to offer more services towards their participants, which overtime might also be the case in (now) less developed regions. It can be expected that more companies and organizations will establish peering relationships with Asian companies, though this might develop slow due to the slowly dropping prices of transpacific transit and colocation in Asia. Finally, there is no doubt that we will see a huge increase in Internet bandwidth, which will require platforms at IXPs to be upgraded to keep up with the demand.

39

Glossary of terms

Autonomous System (AS): a (large) network on the Internet that is used to distribute traffic to and from other networks. Autonomous System Number (ASN): number that is used to identify an Autonomous System. Border Gateway Protocol (BGP): Internet protocol that is used to announce tables, which hold IP address ranges that can be reached through a specific AS. Colocation: facility from where an IXP or datacenter can operate. They can have multiple of these facilities that are all connected to the same network. Often known as a datacenter facility. Content Provider: company or organization that makes content like video, social media, music accessible to the Internet. Incumbent: Telecom Company who used to have monopoly in a specific region or country. Interconnection: physical connection between an AS and equipment or a facility that does not belong to them (for example an IXP or datacenter). Internet Access Provider (IAP): company that offers Internet access to its customers. Often referred to as an ISP. Internet Exchange Point (IXP): Physical point of infrastructure where parties can connect to and exchange Internet traffic with each other. Internet Service Provider (ISP): Company that offers Internet access to its customers using techniques such as DSL, Cable, or FTTH. Currently most ISP's offer services like e-mail access, VoIP, Television and other additional services. Private Network Interconnection (PNI): A physical cable between two Autonomous Systems to facilitate private peering. Route server: a centralized server provided by an IXP that collects tables with IP address ranges and distributes them to other peers.

40

Acknowledgements

Special thanks to everybody who took time into answering my, often many (annoying) questions, requests and e-mails in general. You have all been very useful! I could write down a list of all people who have helped me out here, with the risk of forgetting names or making people upset if they do not wish to be named. People from the following companies, organizations and Forums have contributed to this document: - 17th Euro-IX Forum attendees - NANOG50 attendees who were willing to share their business cards - Diverse number of people participating in the Asian IXP scene - Akamai - Amazon - APNIC - Equinix - Hurricane Electric - IPLAN - JPIX - JPNAP - LACNIC - Limelight - PhoenixNAP - Ripe NCC - Telx - Yahoo Of course also all the people that I forgot to mention above. All your help has been very much appreciated!

41

Bibliography
[1] [2] [3] [4] [5] [6] [7] [8] [9] Random House, Dictionary.com, definition of Internet, the (accessed 01-11-2010) http://dictionary.reference.com/browse/internet Y. Rekhter et all. A Border Gateway Protocol 4 (BGP-4), 2006 http://www.ietf.org/rfc/rfc4271 J. Hawkinson et all. Guidelines for creation, selection and registration of an Autonomous System (AS) 1996 http://tools.ietf.org/search/rfc1930 K. Lindqvist, Netnod, IXPs in Europe and the Business case for peering, 2009 http://www.menog.net/menog-meetings/menog4/presentations/ ixps-europe-business-case-peering.pdf Miniwatts Marketing Group, Internet World Stats, 2011 http://www.inter networldstats.com/stats.htm Amsterdam Internet Exchange, Traffic Statistics, 2011 http://www.ams- ix.net/statistics/ London Internet Exchange, LINX Traffic Stats, 2011 https://www.linx.net /pubtools/trafficstats.html Deutsche Commercial Internet Exchange, DE-CIX, 2011 http://de- cix.net/content/network.html US National Science Foundation, The launch of NSFNET, http://www.nsf. gov/about/history/nsf0050/internet/launch.htm

[10] C. Cheng, Internet Exchange for local traffic: Hong Kongs Experience, http://www.isoc.org/inet96/proceedings/h1/h1_3.htm [11] H. Esaki, WIDE Project Activities, 2000 http://hiroshi1.hongo.wide.ad.jp/ hiroshi/wide-wp/China/wide.html [12] A. Singhal et all. National Internet Exchange of India (NIXI), 2009 http:// www.apricot.net/apricot2009/images/lecture_files/nixi_apnic_amitabh_s inghal_feb09.pdf [13] OpenNet Initiative, Nepal, 10-05-2007 opennet.net/sites/opennet.net/file s/nepal.pdf [14] Pesquisa FAPESP Online, Strategic Partnerships, 03-2002 http://revistap esquisa.fapesp.br/?art=509&bd=1&pg=1&lg=en

42

[15]

W. Stucke, Challenges facing ISPs in Africa: a view from an African ISP. 2008, http://www.afrispa.org/dpages/Powerpoint/Challenges%20facings %20ISPs%20in%20Africa.ppt

[16] European Internet Exchange Association, Euro-IX, http://www.euro- ix.net/ (see note below) [17] S. Radovcic, European Internet Exchange Association 2010 Report on European IXPs, 2010 http://www.euro-ix.net/resources/reports/euro- ix_report_2010.pdf [18] B. Norton, Japan Internet Peering Ecosystem, 2005 http://peering.drpeeri ng.net/white-papers/Ecosystems/Japan-Peering-Ecosystem.php [19] Ministry of Internal Affairs and Communications, Understanding of the total Internet traffic in our country, 2010 http://www.soumu.go.jp/main_ content/000080807.pdf (Japanese) [20] G. Huston, Distribution Reports, AS Resource Allocations, 2010 http://resources.potaroo.net/iso3166/archive/ Note: Some statistics from Euro-IX are only accessible with login credentials.

43

Appendix
Since 2006 statistics on all IXPs in every European country were gathered by Euro-IX. Together with other resources [20] a table has been put together to be able to show the growth in ASNs at IXPs from 2007 to 2010. Due to missing statistics on ASNs in 2006, no growth can be seen from before 2007. The column Unique ASNs indicates the total unique ASNs that are present at European IXPs. The column Allocated ASNs indicates the total number of allocated ASNs in this country, Advertised ASNs shows the total number of advertised ASNs in a country and the column %ASNs at IXPs shows the total percentage of ASNs that are present at IXPs in a country. 2007
Country Austria Belgium Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Luxembourg Malta Netherlands Norway Poland Portugal Romania Russia Slovakia Slovenia Spain Sweden Switzerland Ukraine United Kingdom Total Unique ASNs 90 79 17 0 72 0 17 26 220 318 15 47 14 33 89 118 13 0 353 63 64 19 41 206 44 0 52 82 111 81 415 2699 Allocated ASNs 309 116 52 47 137 142 23 121 486 1187 141 175 22 81 498 150 18 24 396 106 755 59 911 1867 61 110 255 351 363 1046 1446 11455 Advertised ASNs 245 89 41 28 115 97 20 89 329 859 93 119 17 55 374 114 14 20 290 70 631 40 657 1478 48 91 190 232 274 821 969 8509 % ASNs at IXPs 36.73% 88.76% 41.46% 0.00% 62.61% 0.00% 85.00% 29.21% 66.87% 37.02% 16.13% 39.50% 82.35% 60.00% 23.80% 103.51% 92.86% 0.00% 121.72% 90.00% 10.14% 47.50% 6.24% 13.94% 91.67% 0.00% 27.37% 35.34% 40.51% 9.87% 42.83% 45.26%

2008
Country Austria Belgium Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Luxembourg Malta Netherlands Norway Poland Portugal Romania Russia Slovakia Slovenia Spain Sweden Switzerland Ukraine United Kingdom Total Unique ASNs 105 78 17 7 85 0 16 29 233 388 15 50 15 36 99 118 13 0 394 69 69 20 62 232 48 15 54 90 136 127 458 3078 Allocated ASNs 318 125 58 48 161 166 29 122 526 1233 151 200 28 87 538 174 20 24 427 124 872 63 956 2279 66 136 286 384 390 1271 1613 12875 Advertised ASNs 255 103 46 31 135 120 22 96 372 898 100 139 23 67 414 134 16 19 314 80 742 47 736 1851 54 119 220 267 299 1031 1109 9859 % ASNs at IXPs 41.18% 75.73% 36.96% 22.58% 62.96% 0.00% 72.73% 30.21% 62.63% 43.21% 15.00% 35.97% 65.22% 53.73% 23.91% 88.06% 81.25% 0.00% 125.48% 86.25% 9.30% 42.55% 8.42% 12.53% 88.89% 12.61% 24.55% 33.71% 45.48% 12.32% 41.30% 43.70%

2009
Country Austria Belgium Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Luxembourg Malta Netherlands Norway Poland Portugal Romania Russia Slovakia Slovenia Spain Sweden Switzerland Ukraine United Kingdom Total Unique ASNs 111 62 15 6 93 50 18 28 234 470 15 47 15 47 135 158 16 0 447 72 110 20 69 400 47 16 60 102 159 161 493 3676 Allocated ASNs 335 146 75 54 396 187 28 143 602 1259 145 196 29 105 578 207 29 23 458 151 1035 64 1192 2918 79 176 324 421 450 1551 1760 15116 Advertised ASNs 274 122 64 39 275 134 23 114 442 963 105 148 27 83 464 157 24 17 340 101 884 54 883 2435 63 150 253 300 345 1173 1263 11719 % ASNs at IXPs 40.51% 50.82% 23.44% 15.38% 33.82% 37.31% 78.26% 24.56% 52.94% 48.81% 14.29% 31.76% 55.56% 56.63% 29.09% 100.64% 66.67% 0.00% 131.47% 71.29% 12.44% 37.04% 7.81% 16.43% 74.60% 10.67% 23.72% 34.00% 46.09% 13.73% 39.03% 41.25%

2010
Country Austria Belgium Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Luxembourg Malta Netherlands Norway Poland Portugal Romania Russia Slovakia Slovenia Spain Sweden Switzerland Ukraine United Kingdom Total Unique ASNs 109 66 17 7 100 50 18 30 262 514 12 52 15 57 165 168 24 12 540 74 155 20 87 465 53 16 66 116 167 174 518 4129 Allocated ASNs 365 157 78 61 666 211 36 158 673 1333 150 198 35 114 617 216 35 25 502 162 1234 64 925 3293 84 204 359 456 543 1720 1862 16536 Advertised ASNs 291 127 66 39 465 154 31 124 486 999 95 158 31 83 481 163 28 15 382 111 983 55 562 2713 72 166 276 324 383 1296 1295 12454 % ASNs at IXPs 37.46% 51.97% 25.76% 17.95% 21.51% 32.47% 58.06% 24.19% 53.91% 51.45% 12.63% 32.91% 48.39% 68.67% 34.30% 103.07% 85.71% 80.00% 141.36% 66.67% 15.77% 36.36% 15.48% 17.14% 73.61% 9.64% 23.91% 35.80% 43.60% 13.43% 40.00% 44.30%

Growth 2007-2010
Country Austria Belgium Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Luxembourg Malta Netherlands Norway Poland Portugal Romania Russia Slovakia Slovenia Spain Sweden Switzerland Ukraine United Kingdom Total Unique ASNs 21.11% -16.46% 0.00% 0.00% 38.89% 0.00% 5.88% 15.38% 19.09% 61.64% -20.00% 10.64% 7.14% 72.73% 85.39% 42.37% 84.62% 0.00% 52.97% 17.46% 142.19% 5.26% 112.20% 125.73% 20.45% 6.67% 26.92% 41.46% 50.45% 114.81% 24.82% 37.74% Allocated ASNs 18.12% 35.34% 50.00% 29.79% 386.13% 48.59% 56.52% 30.58% 38.48% 12.30% 6.38% 13.14% 59.09% 40.74% 23.90% 44.00% 94.44% 4.17% 26.77% 52.83% 63.44% 8.47% 1.54% 76.38% 37.70% 85.45% 40.78% 29.91% 49.59% 64.44% 28.77% 50.25% Advertised ASNs 18.78% 42.70% 60.98% 39.29% 304.35% 58.76% 55.00% 39.33% 47.72% 16.30% 2.15% 32.77% 82.35% 50.91% 28.61% 42.98% 100.00% -25.00% 31.72% 58.57% 55.78% 37.50% -14.46% 83.56% 50.00% 82.42% 45.26% 39.66% 39.78% 57.86% 33.64% 51.59% % ASNs at IXPs 1.97% -41.45% -37.88% -20.51% -65.65% -12.99% -31.69% -17.18% -19.38% 38.98% -21.68% -16.67% -41.24% 14.46% 44.15% -0.43% -7.69% 0.00% 16.13% -25.93% 55.46% -23.44% 148.06% 22.97% -19.70% -23.53% -12.63% 1.29% 7.63% 36.08% -6.60% -1.91%

Note: Statistics from the countries indicated in green were not present for certain years, this counts for: Cyprus, Denmark, Malta and Slovenia. The calculations have been made from the first year data was available for that specific country.