Archives

Connect Working Group
Wednesday, 22 May 2019
11 a.m.

CHAIR: Good morning everyone, if you could all find your seats. We'll get started in a minute. All right. Good morning and welcome to your secretary most favourite Working Group session of the week, the Connect Working Group of course the NCC service later today is everyone's favourite Working Group, according to Kurtis at least who is the Chair of that. Welcome to Iceland. It's lovely to see so many of you here.

This room has five emergency exhibits, two in the front, one in the middle and two in the back. We have a chat monitor sitting there, we have the lovely Rumy over there who is our scribe. I trust everyone has read the minutes of the previous session. Any comments on those? Good, because we didn't send those out. Sorry! I'll make it up to you.

So, as, you know, we have had a bit of a challenge getting people to comment on the presentations today in their sessions. So, I'll going to give a challenge to all of our presenters today and it is for every three comments you get on the microphone, I'll buy you a drink. And to make it up with the scribe, I am also going to buy her a drink for every three comments that she has to minute.

So, keep that in mind. Strategise around that if you will.

So, my lovely co‑chair will make the presentation deck for today. He made sure that at least the background is intellectual property free. Let's see. There is the agenda. First off, we have the opening, apparently this is it. We have a presentation by Ignatio, there is RPKI and route servers by Nick Hilliard, there is BGP filtering by Arturo, Florence is going to be a connect up update. Bijal is going to do a EURO‑IX update and Jessie is going to be something special about ‑‑ it's going to be interactive to it's going to cost me a lot of money at the end of it. At the end of it, as last time, we'll have an offline piece of this session that will not be recorded and put online in case anyone wants to say something that they don't want to see back on YouTube. And then we have the end of it.

Anything I forgot? I think we're good. So, with that, I'd like to hand over to Florence and introduce the first speaker.

CHAIR: Just a small comment before that, we would like to remind that you you can actually go and comment anonymously on the chat, if you use the RIPE chat on the website, you can go and connect and then you can actually drop a question and you can do that anonymously if you want. So... and just to Florence.

FLORENCE LAVROFF: Just one last thing before we get started with with the first presentation. Last time we had a request for some peering personnel during the session. While we do not think that this is really the scope of the Working Group, I still think this is a good idea to have a little bit of a better idea of who we are and what we do, and yes, so let's get started with some warm up, a show of hands. There are some new people here, maybe some students, some first timers that do not know who we are and who is doing what. So, if you are working for an IXP, could you please raise your hands so that people can see who you are. Okay, thanks for that.

If you are working for an ISP, could you please raise your hand?

Okay. So that's a good thing for the moment I think that people are only raising their hands once, that's cool.

If you are working for content, could you please raise your hand? Excellent. Thank you.

And if you are a student working for an international network could you plays raise your hand so that we can see you. Okay. Thank you guys for doing that.

And last question: If you are working for enterprise, could you please raise your hand?
Excellent. Thank you. All right. So, I hope that was useful so ‑‑ and that it helps you to better connect with each other. All right.

So now that we are all warmed up, let's get started with our first presentation.

So, Ignatio, you are from the rationing initiative, thanks for being here, you are our first speaker because we really believe it is important to highlight the work that students are doing, they are the next generation, and Ignatio, so, thanks for presenting at our Working Group. Your presentation is about 10 years of IXP growth. I pass you the mic.

IGNATIO CASTRO: Thank you very much for having me here. I have a feeling I am a little bit old to be the next generation, but let's do our best on that.

So I am going to be talking today about some work I have been doing recently about how has the Internet evolved and what's the role of IXP in that. Ten years is a little bit of an exaggeration.

So let me start by what I'm going to be talking about is how have IXPs grown. There has been a massive growth of IXPs in terms of number of IXPs and membership. However, reachability has stagnated. And then I wonder what's the macroscopic impact of this on the Internet and what we basically say is that there is certain path shortening, however there is not redaction on transit dependence. So let's get on with it.

How was the early Internet? The early Internet you had DevOps and if you wanted to reach another one you depended on these in order to be able to reach it. How have things changed? Well that was a rather hierarchical Internet and what we typically see is that nowadays, well, we don't have that many desktops and if Mr. trump wants to congratulate around his sad winning of elections, he will typically depend on a lesser number of intermediaries to reach it. Typically or to some extent thanks to IXP, or that's part of the question. So my question is, are there less hops due to IXPs? Is there less transit dependence? Is there a flat Internet nowadays thanks to IXPs? That's kind of the question I wanted to answer.

Let's start by the first. Well a bunch of you guys here are from IXPs so you pretty well know this. So, they have grown. They have roughly 3 billion in terms of number of IXPs. However, the radiance that were big in the number of terms of IXPs, usually suspects, America, Europe, Europe, North America, Asia, some new regions have emerged, particularly South America. It's not only the number of IXPs, it's also the number of members inside of them. So new regions have raised, in particular South America as I was saying. Big stay big gain, and Europe as we know tops the growth.

So, that's pretty fair, pretty well, a lot of IXPs, a lot of members in these IXPs. How many IPs can you reach through those IXPs? Well that's a big question, and what he we do is we try to have an idea what of the reachability of an IXP basically out of all the IPs that you could reach through Tier 1s, which fraction of those can I reach if I go to an IXP? And let's say that I peer with absolutely everyone there with the exception of the tier 1s. Of course I know that just because I go to an IXP doesn't mean that I will be able to peer with everyone. But this is just an upper bound of what you could achieve. What we see is that basically you don't need that many IXPs to achieve all the reachability that you could potentially reach in a given year.

Interestingly, there is a very clear stagnation around 80% of the reachable IPv4 address space. So, there is a lot of IXPs. Reachability from IXPs is rather large. What is the impact that this has on peering? So, if we look at the total number of peerings, I think that I can actually point here ‑‑ so, if we look at the total number of peerings and we see out of those, which are potentially enabled by an IXP and by potentially enable I mean that I am aware that there is a peering latency between them and at the same time I am aware that they happen to be co‑located by looking at peering DB at an IXP. And what you can see is that a large fraction of those corresponds to peering enabled IXPs. So, fair enough. Potentially IXPs have impact, they are big and they have a lot of reachability. So what I try to do now is see what is the impact of the IXPs.

And I have already said that. So, how do I look at it? Basically what I do is that I look at the reduction in path lengths and the reduction on transit dependence.

So, starting by path length. Basically what we do is that we take a lot of traceroutes, we clean them a little bit, I'm not going to get into the nitty gritty. Basically we identify IXPs along the path and we compare those traceroutes that have IXPs with those traceroutes that don't. And try to see what's the difference and what are the trends over I am.

So, basically what we see is that over time, for those situations where there is no IXP but there is a peering, and those that there is no peering and there is also no IXP, as you can see, it's pretty much stable. There is almost no change, path length remains around 4 and a half, 5 hops. If we look at those cases where we have a peering at an IXP, there is some reduction, not a massive reduction. I have to say that I was a bit surprised, might be less surprised, but I was expecting to see a very clear impact, and I just didn't see it, and to be honest, I did look into the data and see if there was some trend that I could find. I didn't really find it.

We did find a little bit of that when we look at very large networks, well very large networks you can imagine usual suspects, Google, Apple, Facebook, Netflix, a bunch of networks that we identified as hyper giants according to a work colleague of mine. And in those ones we can actually see that there is a very clear difference for those cases where the traceroutes transfers no IXP versus those where it goes through an IXP. And you can even see that there is a tendency towards less number of hops. How meaningful is this? Well that's a tricky question to say.

How meaningful are path lengths? Well, it's a tricky thing to say well, because if I am peering momently instead of being physically co‑located at the IXP, I have a magic intermediary in the middle that I cannot see when I look at my Layer3 data, well pat lengths are very hard to assess what's the importance that they have. I did look at this in 2014, and I look at the of IXPs and there was basically remote peering in almost all of them in some of the large ones, it was close to 20%. Some other colleagues, have looked at it recently and the trend continues as you will know. So how meanful are path lengths in an area where I can remotely peer with no one else knowing about it. At the same time, there is the CDN redirection. I did some work on reidentifying Netflix caches and by looking at the DNS redirection. Most of the CDNs that you reach them by ways that are not visible when I look at traceroute data. So, well, it's hard to say how meaningful it is. And at the same time, billion, path length does not correlate with performance, welcome to the Internet, just because I have data, doesn't mean that that data isn't biased and it's really difficult to know what are the biases there.

Plus, which maybe one, 10% of the traceroutes I'm looking at, 3%, 90% of the traffic of the Internet, probably there is no way for me to know much about it, in particular if I try to look at the historical evolution of this.

So that's for path length. How about transit dependence.

So, that's the next thing we look at and what we do is similar to the previous case. We separate those from those that have an IXP and we look at those that have tier 1 along the path, basically what's the mean number of hops along the path? And you can see that there is a clear tendency towards a fall, tier 1 transit providers in the path in both cases, and that in particular, there is a very clear difference for the case when you go through an IXP to those situation where is you don't go through an IXP.

It's not all about tier 1 so we also looked at what's the fraction of transit links in the traceroutes. And what we see is that around 2010, 2012, there is something weird that goes on, was that maybe someone kind of liked me on it and from there on, there is a clear tendency towards less number of transit links on those traceroutes that transfers IXPs.

Next we wanted to take a look at ‑‑ well, what could be the reasons? Maybe CDNs around that time is Netflix CDN deployment, also Google made a massive expansion around that time. Maybe that's one of the reasons, to be honest I'm not sure.

Next I wanted to have an idea of how central are those. I'm looking at and what's the relationship with regards to path lengths and IXPs. So, let me walk a little bit slowly through this figure because it's a little bit more difficult to say.

So, again, I separated those traceroutes that I go through an IXP from those that don't. I take the top ten ASes in terms of the number of traceroutes for that given year that transfers them, so typically this is in the centrality of those networks within the sample that I'm looking at. And then I basically sort them by the that they have in terms of the size of the customer cone. And what you can see is that there is a very clear divergence over time. Basically what we see is that the most central ASes nowadays are very large and they avoid IXPs. And let's see what happens to those networks that are less central? So if I look at the top 500, basically those networks that are a little bit less ‑‑ that are much less central within my sample in comparison to the previous ones, what we see is that this divergence vanishes. Basically, least central ASes are small and like IXPs, this is quite a line with the knowledge that I have on this that if we are both peering at an IXP and we are starting exchanging a sufficient amount of traffic, we will probably avoid the public fabric and we will move to a private peering.

This is what so far we have looked at. At the moment we are trying to look a little bit further into it. We are trying to get more data to see with more granularity we can make a more analysis on radiant to radiant for example. Also trying to assess the bias because, well, traceroutes, how representative they are of the complete Internet and how they represent activity evolves over time. It's quite a difficult thing to say.

What's the role of of CDNs? Unfortunately I don't know any way of redirection over time to be able to assess how meaningful are these traceroutes.

I would be curious to know what's the specific impact of IXPs, what's the local benefit. So if a country comes to me and say will you build and IXP I would love to be able to say this will build it because this is the very simple metric that you need to look at. That's originally the idea that I had and I thought that I would see that very clearly in path length, which I didn't. A little bit more clear on transit dependence, also what is the general benefit for the IXP, the fact that you create an IXP in the US meet benefit me here in Iceland because other path will go from the US to here.

And also, would we like to know if we can predict to some extent the growth of IXPs? We know that eventually most of Africa will be very well connected to the Internet, that might happen in five years, ten, or 20, but it will eventually happen. What will be the infrastructures that have to be developed there to interconnect with the rest of the world?

And that's all from my side. And I think we can use these extra minutes for questions in case you have them. Let me remind that you there are some beers on offer for those who make questions.

FLORENCE LAVROFF: All right, what are the questions?

AUDIENCE SPEAKER: It's very important to optimise Remco's bar bill, so here is a question. Or a comment as well. Thank you for doing this. I think it's an interesting work. I'm a little bit confused about some definitions, so I think it would be great if we could get a little bit clearer. When you talk about IXP, are you then talking about public peering so there is a common fabric?

IGNATIO CASTRO: Yes, basically from peering DB, public peerings from peering DB recollect that's what I used.

AUDIENCE SPEAKER: Okay. Cool. I think that makes some of your conclusions a little bit confusing, because ‑‑ and you talk a little about about it during your presentation when we have a big flow in an IXP we move it to private interconnections. So looking at the if the Internet is getting flatter or not I think you should include them. When you are looking at ‑‑ you could probably use the same method you are looking at edges between two ASNs and you can have a look at which data senders, which common locations does this network have and you might even be looking at the traceroutes and identifying those locations. Because, just going oh, everything on the Internet that's going flatter is because of the Internet Exchanges, I don't think is true. Because ‑‑

IGNATIO CASTRO: I agree.

AUDIENCE SPEAKER: Because we do, everyone in this room, is doing private interconnections as soon as you have a certain amount of volume. So I think it would be super useful if you could kind of adjust your methods into doing that. I'm not going to take up too much time, but just one more.

When you are going to look at the CDNs and you say the CDN redirection, please, please, please go in and study each of the CDNs and how they route their clients, because we're not doing it the same way. And actually Netflix, we're using BGP, so you can probably use a lot of the same methodology to figure out where our clients are being served from by looking at the routing table. And not necessarily thinking that we have some magic behind because we don't. Others do, but not us.

IGNATIO CASTRO: Yeah, thank you for the comments, I completely agree with the private peering. So one of the problems that by looking at historical data, I'm basically constrained by things that I can compare over time, so, for example, if you change your redirection strategy, I never see the end if it's in a different way, it's almost impossible to do it for the whole Internet for all these years. I completely agree. But there are some things that it's just very difficult to get into to be able to compare.

AUDIENCE SPEAKER: We do it differently but you can find friends, I'm quite sure.

AUDIENCE SPEAKER: That's two questions from Nina, so I make it a third, if I do my math for the bar bill correctly. Martin Levy from CloudFlare, if you go back a couple of pages to the graft that shows the large divergence between the, it's actually the top graph of that one but you have it as a full slide. Yeah, that will work. So, I don't think anybody disagrees with this slide. But, to me it doesn't tell the full story, and part of the full story is that basically everywhere on the Internet should have and for most of what you're talking about, a full routing table. In other words, it has connectivity to the rest of the full Internet. So, this divergence which is partly business, business model of the different players, which is not a reflection on the Internet Exchange points in any way, doesn't ‑‑ it is a correct graph but it doesn't, to me, tell the story of we are better connected today than we were ten, 12, 15, even further ago. We have always had a full routing table, but we have lower as path, we have better access, we are geographically less tromboning, for example, if we go back 20, 25 years to a Europe prior to the large amount of IXs. So, I'm sort of intrigued at the other non Internet Exchange graph overlaid with this which says are we better connected? Are people more multihomed? Are people connect to go more Internet Exchanges and therefore they are important routes which are not the number of routes but maybe the amount of traffic. Much harder for you to measure.

So, if you continue this work, this is the graph that I want to see doubled up so to speak, and sort of go okay, we actually have a better Internet today. Because this graph by itself sort of isn't flattering for Internet Exchanges, and we all know in the back of our minds, in fact in the front of our minds, that Internet Exchanges are the saving grace for how the Internet works today. And I just wonder if, you know, is there any data that you have got hidden away that's non IXP based that actually sort of says yes, we are better today than we were? I mean, you have put a lot of effort looking at routing tables, so, that's sort of my question.

IGNATIO CASTRO: Yeah, thank you very much for the comment. I completely empathise with it. One of the reasons why this figure does not say that is because I was simply not looking at it. So... I completely agree and to be honest, like, a big part of the work is trying to find in the data things that I know are happening. This is very difficult to find them when you look at the data with with all the limitation that it has. Indeed, looking at the redundancy multihoming and so on my big one extra metric that would be very interesting to look at because indeed, for example, I say that reach ability stagnates at around 80% in IXPs, but that's amazing, that means that with two or three IXPs I can reach 80% of the reachable Internet. So, indeed I was just trying to look at the two relatively simple metrics that are comparable over time and are easy to convey. But it is true that I still be looking at things like redundancy multi‑homing, unfortunately traffic I just don't know in weigh in which I could compare over time and look at the evolution, but...

MARTIN LEVY: I commend you on the study. The numbers are still fantastic. This was just my one little ‑‑ thank you very much.

FLORENCE LAVROFF: I can see we have a third and a fourth question here. So, please...

AUDIENCE SPEAKER: Andrew Sullivan and I am employed by the Internet Society, but I'm not speaking for them. So, what troubled me a little bit about that last discussion, and by the way thank you for this, it was fascinating work, but we keep saying we know that IXPs are, in fact, doing this work, and I totally believe it too, but your numbers do not show that right. I mean, what you have shown here is, in fact, depressing story for IXPs, that it doesn't look to me like they are worth at lot of the investment that we're putting into it at the rate that we're doing it. One of the conclusions that you might take from this is we could stop this now. I don't believe that. I don't believe that. But if that's what ‑‑ if that's what the numbers show when either we have got to figure out a way to find the things that you were just talking about in the previous exchange or else we have got you know we have got to sort of understand why it is that ‑‑ I mean this is pretty ‑‑ this slide in particular is pretty compelling evidence that you know the work is done, and if that is ‑‑ and even though I don't believe that, it seems to me that I can't really say well I know that this stuff is really working because you know if I put my management hat on, nobody is going to believe me because I really feel it strongly. So thank you for this but I really do encourage you to find that additional bit of numbers that you need. Thanks.

IGNATIO CASTRO: Yeah, thank you very much. I said from the beginning like my original intention ‑‑ attempt with this work was basically to have something like should I build an IXP? These are the metrics that you should be looking at. This is the actual impact. I could not find a specific metric, maybe I was looking at the wrong metrics. I don't agree that the story that I'm telling here is that IXPs don't matter, you can forget about it. I completely disagree with that. The only thing I'm saying is that in the specific metrics I'm looking at, the impact is not the same that I would expect, in particular with regards to path length. But with regards to transit dependence, there is a very clear impact. So, I mean, same story, different narratives, and at the same time, give me the data, I will try to squeeze it, but in one point like I cannot cook the data even if I know that it's not revealing the full extent of the story.

AUDIENCE SPEAKER: Blake winds daily. Thanks for this. It's pretty useful. Real quick. To build on what Nina was saying earlier about private interconnects, PNIs, if you were looking for a subject for further research on this, I think it would be really interesting if you could dig into your data and analyse like when a path moved from an IXP to a PNI somewhere and showed that people were moving traffic off of the that's maybe a way to analyse where the bigger flows are where more traffic is, etc. Thanks.

IGNATIO CASTRO: Thank you very much. It's a very good easy thing to look at as a matter of fact.

FLORENCE LAVROFF: All right. So I don't think we have more questions. So unless you have a question from remote participation we can move to next points.

(Applause)

So, please rate this presentation and let's move to the second point of this agenda, which is about RPKI implementation on route servers, so thanks a lot for continuing this topic from the session of last time, Nick.

NICK HILLIARD: Hello everybody. My name is Nick Hilliard, I am chief technical officer at INEX.

So, as Florence said, the last meeting in Amsterdam we went into what we thought we were probably going to do in terms of RPKI and INEX and IXP manager. And this is a follow‑on presentation to tell you what we actually did and what we actually learned.

Okay. So, this was taken in the context of a general route server refresh. The RPKI side was one bit of it, it was a very important part of it but it wasn't the only driver. A lot of the driver ‑‑ there were a lot of other drivers in terms of getting better quality filtering and rethinking about how exactly we manage the prefix ingress flow into the route server and then redistribution within the system. It required an upgrade from BIRD version 1.6 to Version 2. And we actually took the opportunity at the same time to rebuild how the IXP manager Looking Glass worked, and what sort of features were available, would be available to members and participants at the exchange.

So, in terms of Bird, there were a bunch of really quite important changes. The first is that the RPKI‑RTR protocol is finally supported. That's really important because that gives live feed to the validation caches, there are a few other minor syntax changes, there are a lot of changes under the hood, version 4 and version 6 were integrated into single daemon, which is a hugely beneficial thing to do. Mostly the configuration syntax changed or at least remained the same.

So, in terms of validation caches, this is roughly how it looks. So you have the trust anchors, managed by the RIRs at the moment, APNIC, the RIPE NCC, ARIN, LACNIC and AFRINIC. There are feeds from the local caches to those trust anchors and then the route servers will talk to the local caches. The local caches, when you start them off they will pull all of the RPKI data and then they'll send off just a local feed off to the routers, that local feed is kind of a lazy feed in the sense that the routers ‑‑ if the feed gets disconnected, the routers will say okay, we'll remember this data for a while. It's not a big issue if they do get disconnected, when they reconnect, we have a periodic script to revalidate. There is one shortcomingthing in Bird, to say revalidation doesn't happen automatically, we get around this by doing a refresh. It causes a blip but it's not a big issue.

So, we have a much shorter slide at the Amsterdam meeting about how we handled filtering but we decided we were going to go the whole hog this time. And this is how we do T so there is a lot of sanity and sanity checking an validation going on here and we think that this is something that we put into IXP manager in order to implement all of the good ideas that you know we have talked about to people in the community. And it's essentially a melting pot of other people's really good policies. So, first of all, we filter out very small prefixes. We filter out bogons, we do a Max prefix check. We also do a minimum prefix check because it turns out that on BGP, you can actually send an update with no ASN at all. We didn't actually realise this but of course we did realise it because we run route servers and this actually stops organisation from connecting route servers to our route servers and doing really weird stuff. So it's a sensible thing to do.

The peer AS must match the first AS, and again it sounds pretty obvious but some people do really weird things. We have talked previously at the Connect Working Group session about next hop hijacking. This has actually been done at other IXPs and people have found that their peering sessions have been hijacked by their competition. We don't want this to happen at any IXP manager instance so we have put in explicit checks to make sure that it doesn't happen.

One of the new big new things is filtering known transit networks. Now, this is a list of transit networks that ‑‑ these prefixes should never appear at an Internet Exchange. And if you see any of these prefixes at a route server, you know that there is a misconfiguration of some form. It might be a more specific leak, but probably this is something that you want to filter out anyway.

There is a shortcoming in RPKI, and that is that there is no facility to generate AS‑SETs. So, we actually have to use the IRR still for this. So this is an AS‑SET from HEAnet, one of our connected organisations, and when they connect into the route server at INEX, these are the ASes that we will actually accept from them. So, as I said, there is a shortcoming. There are various Internet drafts which are hopefully going to deal with this. They haven't been standardised yet. They have not really gone through the full IETF consensus process. But we are working on it, and it is a bit of a regression, and it does create a dependency on the traditional IRR that, you know, one day we just need to move away from fully. But it's not there yet and we are just going to live with that.

Finally, in terms of RPKI, if a prefix is valid, after all of these previous checks that we have done, which includes the AS‑SET and all of the filtering checks, next hop hijacking F it's valid at that stage then we will accept it unilaterally, if it's invalid we will drop it unilaterally, because an RPKI ROA is a cryptographic attestation from the resource holder that a prefix should have certain characteristics, and if it pops up as invalid here, you know, an IXP really has no right to accept something that's been declared as being invalid by the resource holder so we just drop it and that's its end of it. And if it's RPKI unknown, we'll just drop through to the standard IRRDB prefix filtering that we have always done.

So, this is how it works. This is sample AS, AS112, which everybody will recognise as being the reverse DNS AS for private IP address space.

So, the peer is on the left‑hand side over here, and on the first import policy, it checks all of the prefixes and whatever problems it sees, it doesn't actually do anything, it just assigns BGP large community tags to them, so that sometime later on down the chain, we can do a little bit of filtering, but we can also get a good idea of what's actually happening to prefixes inside the RIB and we can export that into the Looking Glass system.

At the second stage between the ingress RIB and the master routing table, we actually do the filtering. So that's where the, all of the invalids, the incoming invalids are dropped.

And then once they get into the master routing table they are redistributed to all of the other RS clients.

So, this is an example here of some of the BGP large communities that we use. It's based on a list which is defined by the EURO‑IX route server Working Group. This is a rather loose list of, you know, which has some semantics defined, we think it's useful enough to use here, but all of these are internal to IXP manager, they are not exported to anybody else. So you can see that some of them will be on the right‑hand side over there, you can see reasons why prefixes will be dropped, so, for example, you'll have RPKI valid or invalid, IRRDB invalid, Bogon prefix, and there will be a whole lot of other sample entries inside there. On the other side we will have informational tags like RPKI valid or RPKI unknown, and these are purely informational signals that we're attaching to the prefix that we can inspect later.

Those ones on the right‑hand side are actually filtered.

Okay, so, on the export to the member side, we'll take other prefixes in from other route server clients over on the right‑hand side, the pipe back from the master routing table to the lock RIB out applies the standard Internet Exchange community filtering system. So this has been fairly well defined. And then on the way out we strip all of the informational tags, because we don't want to say that our interpretation of these flags is canonical in any way, they are just used internally within IXP manager so we strip them out. They don't really from any business being in the routing table.

The standard egress community filtering mechanism is this. So we have an RFC 1997 sort of a community tags then we have large BGP community tags on the right‑hand side. The large BGP community tags are evaluated first because it's a newer system, so therefore they have to be examined first because otherwise it will create inconsistencies in the prefix filtering mechanism.

So, that was all the theory. How did it go in practice? Well the first thing is how do you set up a validator? It turns out this is easy, you should all do it. This is what happens with the RPKI validator version 3 from RIPE. You download it. You unTAR is. You start it off. And it's really easy. If you want to download the ARIN TAL, the Trust Anchor Location from ARIN, that's a separate download. There is a bit of community discussion about trying to get the Karen guys to be a little bit more unrestrictive about the licence to download it but that's an ongoing discussion. It's pretty easy to download but it just means at some stage in the future this is going to be packaged software and it's not going to be possible to just download all the package without having some manual intervention to do that.

We decided to run two different various, validators. The sucked one was this. It was easy to install. You just Curl RIPE 2 S H route, everybody does that, that's standard Dev ops practice. But I mean, what are you doing? I mean like, is that any different from you know, W get pipe to TAR extract and then configure make install and then run the binary? I don't know. This is a hard problem. I'm not going to get into that here. But it's pretty easy to run. It's got very little overhead and again there is a requirement to download the ARIN TAL separately.

In Bird, this is how you define your RPKI RTR protocol. You can see it's pretty easy. Once you are in the filter section, it's really simple. You just execute the ROA check function and you check the return result. It might be ROA invalid or ROA valid or ROA unknown. It's really simple, make your decisions based on that.

So, we have two route servers and a route collector. We had to ‑‑ in order to implement this we did the route collector first because that was the read only mechanism that we used to figure out what sort of problems there would be. We did find four problems that out of 80 peering sessions. There is the usual things that you might expect. None of them were terribly unexpected, surprising, dangerous or anything like that. But, we were able to reach out to all of the members in the community and get them fixed and we did the route server upgrades over a period of one week ‑‑ sorry, one week separated.

And in terms of the general implementation process, yeah, there's very little issues with Bird Version 2. We have done a lot of work ‑‑ this is condensing all of this down into a very short presentation and there was obviously a lot for work going on in the background but we have done the work now and you can actually just roll this out as IXPs.

In terms of Looking Glass. This is the new look Looking Glass, you can just click on the prefixes received and it will give you a list of what's going on here. So, anything with an exclamation mark means that there is a filter happening. You can drill down into the filter and it will highlight the problems in red. And this is on the public side. On the member portal side, you can click on the route server prefixes tool, and it will give you a list of all of your prefixes that you filtered. And it will actually tell you why, and if it filters the prefix, it will not just tell you why, it will give you all of the reasons in advance, so it's not a question of announcing a prefix and finding out that it's been filtered and then you renounce, you fix the problem and then you find out that there is another problem. It will give you all of the problems at once and you can see exactly why your prefix is being filtered.

So, all of this is available in IXP manager 5, which was released yesterday evening. Lots of new features, there is various new looks, there is an awful lot of back end stuff. This is just a very short summary. There's been I think about 90,000 code line differences, so quite substantial changes going on in the background. It's available now, so, if you are running IXP manager version 4 at your Internet Exchange, please do upgrade and we'd love to hear from you about any problems.

Great. So any questions?

CHAIR: Thank you Nick.

(Applause)
So I see we have questions.

AUDIENCE SPEAKER: Hello, I'm Meara from cz.nic, I am the developer of BIRD. I would like to suggest to you one thing. You are using an extended communities and large communities that you have set in at the beginning and you stripped them out when you are sending the route back, yes?

NICK HILLIARD: That's correct.

AUDIENCE SPEAKER: In BIRD Version 2, there is a special feature called Custom Route Attribute that you can name and use your own route attribute instead of these defined contents which I suggest to do instead of these, because it's much faster. For comparison, if you have six large communities, it's two times slower than adding six custom attributes. So a suggestion probably for almost everybody who does this, and feel free to ask me if anybody wants more.

Second thing, you are speaking about RPKI versus RIR DB, that it's something like going down, yes? That for now are filtering by RPKI is worse than before IRR DB.

NICK HILLIARD: Yes.

AUDIENCE SPEAKER: Why are you not doing both?

NICK HILLIARD: We do both. But we do the RPKI filtering first and then if it fails the RPKI validation, it will drop the prefix and if it's unknown it will pass through to you to the IRRDB, but the ASN check ‑‑ the AS‑SET check from the IRRDB has to be evaluated first so it's IRRDB AS check followed by RPKI validation, followed by regular IRRDB validation if it's RPKI unknown.

AUDIENCE SPEAKER: Okay. Thank you.

NICK HILLIARD: Thanks for the suggestion about the tags. We didn't realise they were so much faster so we'll definitely take a look at that.

AUDIENCE SPEAKER: Just a quick question. I know you are using BIRD, but do you comparison evaluations against, for instance, FRR or do you, how do you just ‑‑

NICK HILLIARD: Okay. So we haven't tried it with FRR because we have been kind of keeping a general eye on the code updates to FRR and as far as we can tell, there hasn't been a huge amount of code changes for the route server code paths in FRR. Now, we previously used Quagga and we ran into quite substantial scaling problems with Quagga. So we haven't done that with Quagga, or at least with FRR yet. Now there are other issues as well with FRR in terms of it being a little bit difficult to do atomic config changes and updates. It's much easier to, you know, write out an entire configuration file and then just hit a reload button. Whereas with FRR it's a little bit more troublesome. So, we haven't done that yet. We are going to implement this with open BGPd when BGPd has got to the point when it's feasible to do this, but we haven't targeted FRR yet. And I suspect that FRR will need a little bit more work before we can do it.

CHAIR: Now it means that Nick is getting a drink.

AUDIENCE SPEAKER: Hi Nick. This is from the Amsterdam Internet Exchange. First of all, thank you so much for the lovely presentation. My question is, whether you guys had a change of heart with regards to implementing RPKI in the EU route servers?

NICK HILLIARD: Yeah, that's actually a really good question. And it comes down to ‑‑ and this, by the way, is very much a policy question rather than a technical question. Previously, we have expressed a lot of scepticism about RPKI and whether you would want to implement RPKI because if you look at it from a sort of a very high level, it creates a single point of failure in that if ‑‑ if there is a compromise, say, at an RIR of some form, it means that there is essentially a big red switch that could be enabled to disable ‑‑ or could be pressed to disable a particular prefix, or a particular set of prefixes. Until now, the BGP inter‑domain ecosystem has been very much a sort of a peer‑to‑peer system, you know, organisations have had bilateral filtering arrangements or probably in most cases no filtering arrangements whatsoever. Whereas now we have a kind of a top‑down thing coming in from the ‑‑ which is essentially controlled by the RIRs.

Set against that, there has been an increase in the amount of BGP hijacking, and not just BGP hijacking, but BGP hijacking where it is very carefully targeted and very difficult to deal with unless you actually implement tools like this. So, I think, like any policy decision, it is a balance ‑‑ it's a question of trying to get the balance right between, you know, between not implementing something which you think is going to have problems, but also implementing things which you think are going to fix a problem that you already have. And I think probably in the last couple of years, RPKI, there's been various changes at a policy level. One, for example, would be that there is now a certain amount of juris prudence about how the courts can interact with an organisation like the RIPE NCC about deregistering prefixes. And I think that's actually quite comforting that this discussion, that this situation has happened with the RIPE NCC because now there are, there is a clear ‑‑ there is much clearer understanding from a legal point of view. And as I said, set against that, you have a continued threat of really serious BGP hijacking problems, which have a political, optical visibility problem that when they happen, and if they are allowed to continue to happen, ultimately the decision to implement technical means for hanling BGP hijacking, if they are not ‑‑ if those decisions aren't taken by the technical community at the Internet, the ability to make that decision is going to be taken away from us and it's going to be made by regulatory authorities and governments.

So, this would be, I think this would be a good industry way of sort of saying well look, you know, we have looked at all of the issues, we have looked at the issues from a technical, from a policy angle and we think that at this point, the balance is RPKI is probably a good way to go.

AUDIENCE SPEAKER: Thank you so much and I'm glad that we are on the same page. Thanks.

AUDIENCE SPEAKER: Randy Bush, IIJ and Orcus. Nick, interesting and we should talk more. But two small points. One, RPKI is a database and what we really mean is RPKI based origin validation.

NICK HILLIARD: Yes.

RANDY BUSH: Secondly, unfortunately it's not the magic bullet that the attackers will just be slightly more sophisticated. But it will help. And it will certainly stop a lot of fat fingers. But it's not a magic bullet.

NICK HILLIARD: Yeah, you are absolutely right. There are no magic bullets. If BGP hijacking or miss configurations were things which were easy to solve, with we would have solved them years ago, and I think Randy, you are spot on in that. It's just going to make things a little bit more difficult to just screw up badly.

CHAIR: I was a bit scared that you had a question, Rudiger, but that's fine. Okay. So thank you very much.

(Applause)

And so now following up, Arturo who will be talking to us about routing filtering at the edge at AS 15169. I will give you the Chair.

ARTURO SERVIN: Very nice to be here, I think it's the first time that I am present at a RIPE meeting. Normally I am in the other side of the ocean in Latin America. But, now I'm working at Google, which is AS15169. I am part of the team that is working with peers and partners for GGC, that is the Google Global Catch and the peering relations, and now I'm working in Europe and also in some internal projects and one is precisely this about route filtering.

The name of the presentation is kind of cryptic, but if I could summarise it in I don't know one minute, it's like we are going to start filtering all other BGP sessions, public and private. So, better start looking at your Internet routing registry data so make sure that it's up to date and it is valid, because we are going to start filtering using that data.

So that basically will be like my presentation. So it's also like 20 slides of why we are doing it that ‑‑ I guess the majority of the people here knows why we are doing it. And a little bit of how we are doing this and when.

So let's start. Talk about why. When I think it's quite obvious. We're, in fact, one of the study cases for routing validation with Pakistan Telekom, you know about routing validation, Google was affected in that time. Since 2008 we know about this. The attacks are misconfigurations and problems. They haven't just gone away. Right now it's not that it's more important but in the past we have a view we just catch speedy or some other stuff, but today also at Google, some people depends on us for, I don't know, Cloud computing and e‑mail and some other things. So we need to be more careful in which data or routing data we ingest and how we handle and how we react with that.

So that's why ‑‑ that's what is changing. Also we think that ‑‑ well, we are part of the community, the Internet community, and we want to be part of the solution and not part of the problem. So, we are doing some of these things that possibly we should have done in the past.

Well now is the time. So this is kind of the problems that we have today. Not necessarily we are going to fix everything what we are doing. Basically for example, indirect sessions, or like data that is in the Internet, what will happen? Well somebody can just announce my prefixes and well, make a problem. We are not going to solve this with this. Maybe we can create a mass utilisation or we can motivate more people to use RPKI and Internet routing registries and possibly the problem will go away. But it's not something that we are going to try to solve, or we are not trying to solve directly with this.

Also, we can leak prefixings of others in our direct sessions, it has happened. We are not proud of that, but yeah, we have some operational issues sometimes.

And others can leak others space, and that can affect, for example, open DNS, for example, can go there and then the DNS will go all around there. It has happened as well. We are not going to try to solve this but as kind of the indirect solution that maybe we can have in the future. The problem that we really want to solve is like, okay, what happened when somebody sends me the routing information that is not correct? It's like hey, I have this network, when in reality they don't have that network there. So basically what we are going to do is use a variety of databases, mostly in the beginning it's going to be the Internet Routing Registry in the different databases that are around there, and we are going to build our prefixes, well, our filters, to filter those prefixes.

So, as I said, we are going to start uplying filters in all of our BGP sessions. We are going to start marking, I guess, next month and then we will have a clear idea of the size of the problem. And later, hopefully in a few months, maybe in September, we will start dropping those announcements and for the ones that they don't have the data correctly in the databases, probably it will shift to other paths. Hopefully nothing will happen and nothing will shift, but it's something that it can happen.

Which data are we using? Basically, possibly there will be a lot of opinions about why we are using what and why. Initially, we are using the Internet Routing Registries. It's not perfect, it's not going to solve everything. But right now, it's the information that is the best available there. So, there are some issues there, but we think that is the basis that we can start using to build this filtering. We are going to use RPKI. It's not as easy as just starting ingesting ROAs and applying it in the routers. We have some, as you can imagine, Google has been adopting since a few years ago, so we have some interesting moving parts, so we cannot just like start using RPKI from now, as you can do in any normal router. But it's the plan to do it. So that's the second stage. We don't have a date yet to start using RPKI but also I will suggest to you, to start adopting it and ‑‑ because any time in the future is going to happen.

An the other one is like, okay, we have sometime ‑‑ we have data about routing intent or well routing information. Basically what we have here is that we want to know what is your routing intent. So that's why we're using the Internet Routing Registries and RPKI. But also we have some information, historical, that says oh, this ASN always has been announcing this prefix. Suddenly it changes. Why? Is it something that was intended or it was something that it was a mistake? So this is information that we are not using now, and I'm not saying that we are going to use it, but it's something that is in our minds, like what happens if something changes? And we knew that it was ‑‑ this has a particular behaviour in the past. So something that we are analysing if something, that another thing that we can use to try to infer the routing intent that you have in your announcements.

So, the plan is basically this one. First one. Notify peers. So, consider yourself notified. So, in a few months, after your traffic shifts and you remember that you were in Iceland, yeah, remember that I told you so, that you have to check your filters. We are going to different conference, we were in LACNIC, NANOG, and ‑‑ so we are moving in different conferences telling peers, but also we are plan to use our own tools. We are our peering database, internal that when you peer with Google, you have access to some of our tools so we are planning to use that one. We don't want just to spam you, so basically we are finding out how will be the best way to inform all of you that you have to do this? We are also, it's not ‑‑ I think for this community, you know very well how to use the Internet Routing Registries, so, this is not a surprise to you how to use it, what it's about or anything, but there are some other communities that we need to tell, explain to them more what is this about and how to use it. So we are planning to push some informational documents, basically what you can find in the Internet what we are going to put in in some places, so other peers that are not as experienced you can access it and create the filters and update the database.

So, we are already doing the number 2, so we are collecting data from these different databases. I'm going to show you which ones, it's not a complete list because I think that we added a few more in the last few weeks. We are going to parse and process all this in our internal service. And finally, we are going to create the filters for each one of our peers, and as you can imagine, one of the last steps is, well, just go and configure all our routers and apply those filters. I think that we haven't done number 5, but we are very close to doing it. And because we know that we are going to find some issues, there are going to be prefixes that are not going to be tagged correctly, because, well, the routing information doesn't match what we see in the registries. We are going to mark first to see what is the amount of kind of invalid is not the RPKI invalid but it's a prefix or a route that we detect that doesn't match the information that we have with the routing intent, and after that, we start dropping. So when we start dropping, then you will see if you don't have your prefixes correctly in the database, possibly your traffic will shift, to transit or to other peers depending on the interconnection at that we have to that prefix.

We are going to use our portal, basically this is the portal or the website that many of our peering partners or peers and DGCs and ISPs use to see what is the traffic to Google and some other stuff. So we are planning to enhance this service, so you can go there and like a Looking Glass and say hey, what are my prefix? Oh my prefix is valid, it's announced in these places, and how do we see that prefix? It's valid. Do we see a problem? Google is seeing a problem with that prefix. It has the correct routing information or not? So with that then you can make sure that your prefixes are correct. You don't have to wait for this to happen. There are good services like RIPE has one, you just go there and put your ASN and it will tell you what are your prefixes that are valid or invalid in RPKI; which information ‑‑ sorry, which prefixes are missing routing information in the registries. So you can start doing that. You don't have to wait to see how we see or how we will see these prefixes. You can just go now and check if everything is correct. But that will be in the future. Sometimes peers they announce more specific or less specific that the Internet, something, there are some changes, so you can make sure that what we see is valid and we are going to treat that prefix as valid.

Also, we are going to implement some way that you can request how to update the filter. As you can see it's not like realtime, so you just go to IRR DB or RIPE database and modify your prefix and magically it won't appear in our database, it has to wait until like all the processing time. Right now, I think it's ‑‑ we are thinking around 24 hours, but that can change in a few weeks, it can be shorter or longer depending on our internal mechanics in how we are processing this but we are going to provide some way, as a peer, to tell us also hey, I changed something, I have a problem, you need to update your filters quickly because this is political to me. I don't know much to tell us about this but it is something that we are planning to do.

And hopefully, if you have feedback, you can do it now. You can do it later. But this is going to be very important for us in the next few months to make sure that we do it correctly. We know that for some of you, the peering connections, they can move a lot of traffic, so it's not like, just Google, just moved my traffic to someplace else, that's fine. We know that that is not easy for many of you so we want to do this the best way so we don't have any negative impact to you.

Basically we are taking the data from RADB which is a mirror of a lot of databases. Right now in the time that we did this the slide, there was no databases, but I think we are adding a few more. We will provide the updated one possibly when we do our official announcement in the ISP, with the documentation that we have, we will try to update it there. Any ways, if you use one that is not there, just let me know or Chris that is over there or one of the Google people that you know that are here, and then we analyse and if everything goes well, we will add it then and we will process it as any other Internet Routing Registry database.

Basically, how we are going to do this. We need the AS‑SET. So basically, we are going to get it from peering DB, so we are going to go with H 1 of our peers, check the ASN, going to peering DB, which is your AS‑SET, go into the AS‑SET, get the information, process it, build the filter. So, make sure that also, that your Internet Routing Registry that your preference is updated, but also check that peering DB is also updated. It's one of the databases that we use, and we use, or we rely, so, yeah, yeah. Keep it updated, because we are going to use it as well, not only the IRR database, but also peering DB.

Which tools do we use? Well, we don't use any of those, because we have a complicated things inside Google, so, also it was easier just to build our because we need some special things there. And we also collaborated with ISOC in MANRS tried to build some, some tools that will help you maybe not you, because you are really are on top of this. But to help some ISPs to manage and update all this data.

Basically, how we are doing it. Basically we are using OpenConfig. We have different type of routers, so cannot just generate for one platform, so basically we are creating this one using OpenConfig, the configurations, and then we push all these internally. It sounds ‑‑ this is just very simplified, I think that Chris right now is thinking like this is much more complicated that you are saying that we are doing that.

So, I think that we are running out of time. But the message is update your filters, update peering DB, and I will roll this May, like the end of this month and September to start filtering, so, be quickly. Going to peering.google.com for more general information. These are some facts so you don't have your filters ‑‑ well, you don't have your information, we won't accept your prefixes. And check peering DB. Check IRRs. And thank you.

The slides are over there. So if anyone wants to check the links.

CHAIR: So any questions?

ARTURO SERVIN: That was the part that I was afraid of.

RUDIGER VOLK: Not quite sure whether I will disguise all comments into questions. One thing, well, okay, looking at what you are reporting, I like the approach of mark and report first, and drop then. If you are taking that approach, I wonder why you are not considering or maybe secretly you are considering to mark the regular RPKI evaluation early on. You don't need to drop if you feel that's too dangerous, whatever. But actually, marking and reporting about what looks invalid there I think would be a good idea and helpful from the start.

ARTURO SERVIN: I think that is maybe something we can do, I think that's something Chris and I will discuss. Chris has a plan in his mind and something we were probably going to do. We haven't discussed the specific plans, but something we are going to do, but not only marking in the future, we are just going to drop. But it's something that we want to do. I don't know exactly the complications because Chris is the mastermind with the technical stuff. But it's something that we are considering at least.

RUDIGER VOLK: Okay. Continuing in RPKI which should not surprise, as I am speaking, is, as you are talking about the sources you are using, why are you considering RPKI as a source only for invalidating IRR data? Why are you not including the RPKI ROA data as an additional excellent quality source of route information?

ARTURO SERVIN: We will, it's just we are not ready to use it. But we will.

RUDIGER VOLK: Okay. Kind of, if you don't have the tools, actually I have them.

ARTURO SERVIN: Also RPKI only has the origin, also we need a little bit more like ‑‑ but, it's going ‑‑ we are going to ‑‑ we are not going to use one signal. We need several signals to make it reliable.

RUDIGER VOLK: Okay, RPKI is not solar bullet and obviously it is not the only criteria for deciding whether something makes sense.

The last comment is a little bit higher level, in a presentation like this you are only talking about what you are doing about acceptance. It would be really nice thing if there was a slide telling what you are telling, what you are publishing as your authoritative information for your peers.

ARTURO SERVIN: Okay. I was very quickly, but one of the plans is also to ‑‑ well, we are fixing all our internal system to make sure that the data in the Internet Routing Registry is correct, that is what is there is exactly what we want to announce and also we are working in publishing our ROAs. We have one part of the ROAs for the corporate space for Google for the corporate. We need to sign the prod, but we are going to do it very quickly, hopefully in the next quarter we will do it. But we have a lot of parts we need to fix but we are going to do it as well.

RUDIGER VOLK: Sure. Lots of things. It would be nice to have a summary and, of course, we are all busy and not everything that we consider to be the right thing, we can do today. It may be tomorrow or next week.

ARTURO SERVIN: Maybe, because I escape this one, this is the other part that we are doing, we are joining MANRS, it was an internal discussion to join or not to join. We decided it was better to join and show our commitment that we wanted this problem to be fixed and we wanted to be part of the solution and not only the one part of the problem. So we are working very closely with the Internet Society to make MANRS more successful. And as I said, ROAs and cleaning our own stuff is one of the parts, so I didn't go into the detail but it's part of it, we want to do it it. It's not like we warned you to filter your things and Google won't do it. We want to do it as well. So... we are in that process. Obviously cannot do it from one day to the other but obviously it's something we are committed to do.

RANDY BUSH: I yeild to the online user, I believe she has somebody remote.

AUDIENCE SPEAKER: /TKPWRER Anna, with the chat monitor. I have a question from Magnus, could you open your IRR software?

ARTURO SERVIN: We are not having like a run of the software. We have a bunch of tools that we use to parse the information ‑‑ are you going to say something? Please.

AUDIENCE SPEAKER: Chris from Google. I have written some IRR parsing software. It's available on GitHub/bettteers/tools, it's got some work to do still, but it's getting to the point where you could actually kind of use it to parse the data and then create filters, that's the end goal. I have to spend more time on that and also the internal problems.

ARTURO SERVIN: I think that the link is in the presentation. I'm pretty sure ‑‑ I think it's there, but if not, just end us an e‑mail and ‑‑

RANDY BUSH: Could you go to your next to last slide.

The first bullet. Does that include the transitive closure? In other words, I am not your peer, she is, I am her customer, I have ‑‑ I am a BGP speaker. Will you accept my prefix from her if my prefix does not have an IRR object?

ARTURO SERVIN: I think we won't because then we won't have the authority to say ‑‑ you won't show that you have the authorisation, because it's not ‑‑

RANDY BUSH: Fine. Just checking.

AUDIENCE SPEAKER: Magnus is saying thank you, Chris, for the explanation. I have a question from Paul Hostedder. Can you explain what you mean with follow the AS/arrow maintainer?

ARTURO SERVIN: In the Internet Routing Registry, you have the maintainer and the AS‑SET and the ASN object. So basically, we just go to the ASN object, we just see like, okay, what as an ASN you have there. Does the ASN that we peer with? But in reality, we need to know all the routing information that you want to announce and that is in the AS‑SET. So basically when you peer with us, unless you tell us what is your ASN set, that is possibly a good idea to put in in the ISP.google.com, we need to go and find the ASN somewhere, one of the way we are trying to find that AS‑SET that has all the ASNs that you are supposed to announce or all the information related to that ASN that you are supposed to announce is going to peering DB, check your ASN and then a peering DB will tell us oh this is my ASN set. So hopefully that answers the question.

CHAIR: We still have one question at the back and then I'll take the last one.

AUDIENCE SPEAKER: Brian Dixon, GoDaddy. I just wanted to get clarification. If somebody is, for instance, a BGP speaker with just their own ASN peering with Google directly and has RPKI but no IRR object, are you saying that exceptional case, where there is no AS path, you're not going to accept routes?

ARTURO SERVIN: For now we won't. But I think that NTT has some way to export from RPKI to Internet Routing Registries. We haven't decided if we are going to use it but maybe it would be a good idea. It's something that we might discuss or we might do later or now. But, yeah, now like, today, like right now, you have to have it in the Internet Routing Registry.

AUDIENCE SPEAKER: And supposing AS cone gets formalised and adopted, will you use AS cone?

ARTURO SERVIN: Well ‑‑

AUDIENCE SPEAKER: Plus RPKI?

ARTURO SERVIN: The AS cone, I don't know if you can put it in the RPKI right now. So basically we just know this prefix has these origin and that's all.

AUDIENCE SPEAKER: What I'm saying is AS cone, if it goes into the RPKI.

ARTURO SERVIN: When it comes and also AS PA or whatever else comes in RPKI, yeah, maybe we'll adopt it, yeah.

AUDIENCE SPEAKER: It's an anonymous comment ‑‑ or question. Before this were you doing some filtering and if so based on what information?

ARTURO SERVIN: We didn't do any filtering, basically we just have the prefix, the number of prefixes, so if you had like 1,000 prefixes and suddenly you were ‑‑ Chris is going to maybe ‑‑

Chris: No not filtering, but less filtering. You can send me my own routes, you can't send me routes that you are clearly obviously broken, but otherwise we could be doing way better.

CHAIR: Okay. Thanks. I think we're done here. Thank you very much.

Now we are going on with the connect update by Florence. Welcome to the stage.

FLORENCE LAVROFF: Hi again everyone. So, time for the connect update indeed. We are trying to make this a tradition in the Connect Working Group to have a moment where we can get the relevant updates for this crowd.

For the sake of time, I'll go through that rapidly. So for today we have four bullet points.

The first one is about peering DB, as you may have noticed last February peering DB went through some rebranding, they even have peering product manager, which is Filiz Yilmaz. Filiz, could you plese raise your hand.

(Applause)

Second point, presentation that was presented during GP F from Martin Levy at CloudFlare about peering locally with local networks and local IXs or not.

And for the third and fourth point I have some publication on RIPE Labs. This one may look familiar because this is ‑‑ that was a RACI presentation of the past session in Amsterdam and covering remote peering at IXPs, and then we have some really cool publication from the RIPE stuff about some countries like Bosnia Herzegovina as well some interesting countries like Russia and Saudia Arabia. Not on that slide I also wanted to mention a presentation that may be of interest for you, so on Friday mornings, there will be an interesting presentation from Susan phony, which is called Peering Economics 101, I recommend you attend that.

And then the last point, Remco, I think you have something to tell us about address space?

REMCO VAN MOOK: Address space, the session you are all avoiding by sitting in this room actually had something interesting going on this morning. So, as you all may know, there is a specific piece of IPv4 policy and a specific pool for Internet Exchange points. This morning, a draft was presented by me to extend that pool, so, back in 2012, a /16 was dedicated to IXPs. In the last seven years about half of that has been used and the rate at which this pool is being used up is actually accelerating, so the proposal is to extend that to a 15, so basically double it in size and do a bunch of tweaks and fine tune, to make sure that pool gets used efficiently. You are welcome to have read. It's not being presented yet. The presentation is online in Address Policy, you want to have a look. That's it. Any questions? Comments? Okay. Back to you.

FLORENCE LAVROFF: Okay.

FILIZ YILMAZ: Thank you for that note. I just want to acknowledge that I am here for the whole week and if you have any questions or comments about the product, please talk to me, but it's not only me, our entire peering DB board is actually attending the RIPE 78 and they are here as well as many volunteers, so we are on the ground and if you have any questions, please let us know. Thank you.

CHAIR: Okay. So ‑‑

FLORENCE LAVROFF: Then Bijal is going to continue the IXP update with the EURO‑IX update.

BIJAL SANGHANI: Hello everyone. This update is not the actual EURO‑IX update. It's actually an update on IXPs that have some news to share. Some of you may remember a long time ago we used to have the Connect Working Group and lots of IXPs would come up and talk about new things that they're working on. This way we don't have a lot of people on the stage so you get to listen to me for a couple of minutes. As you know we are running really, really short on time, and I have got another presentation and we have got a nice interactive session at the end. So, I'm going to speed through this so that we can get onto the last part of this.

Right. First of all, we have some news from DE‑CIX. DE‑CIX is now live with 10‑plus peers in Lisbon. They have 400G ethernet interfaces. They also now have a paid peering product, and DE‑CIX now also partners with Data Centre one to be present in other parts of Germany. DE‑CIX also has gone 80 gigs plus and they have a tech meeting so if you are interested, please speak to somebody and join them on the 6th June.

TOP‑IX is now opened a new POP in data centre 4 in Milan and is also released a service using PtP version 3, which is pretty cool and they work in the Italian National Meteorological Institute to get that.

Moscow IX, it's the largest IX in Russia, started in 1995. They are also doing more on the TV and now have more, over 400 TV signals including ultra HD on their LAN. They now have 12 DNS nodes globally. We have a new peer‑to‑peer management system. And have just recently partnered with DE‑CIX to launch services in their two markets.

AMS‑IX, LINX and DE‑CIX are working on a common API has use for remote customers, AMS‑IX is also using a notification system, and have deployed Artemis, which is a RIPE supported BGP hijacking detection tool. AMS‑IX took the adoption of the extreme SLX 9850 to the next level with the creation of a supersite. I'm not going to read all the text on that, so the the slides are available and if you want to read it in more detail you can do and you can also speak to the IXs themselves.

INEX, we did hear from Nick earlier, but again, version 5 has just been released and this is actually dedicated to the memory of Barry Rhodes, which I think is very nice. There is over 500 commits, they have got RPKI support, they support Bird Version 2, and now is IXP Manager is used at over 70 IXPs worldwide. And Tehran is actually building it out at the moment. Just to add to this, this is really really great news if you are also using the IXP database, because IXP Manager has a built‑in export which guarantees the information gets exported to the database. And finally, INEX Cork has been completely rebuild.

And that's it, for the IXP updates. Now, I'm going to give ‑‑ is there any questions? Not that I can answer questions on behalf of the IXPs, they are all here so I'm sure you can speak to them at some point but I am going to rush on because I know we're really really short on time.

Now, I'm not going to give you the full history of the IXP database, why we built it because I have done that many many times. But what I haven't done in recent times is tell you what we have done. My previous presentations have been oh we're going to do this or that. So it's nice to stand here and say well, okay, this is what we have done.

So, we have an IXP directory. We have ‑‑ this is the IXF ID, which is a unique identifier for Internet Exchange points, and this is ‑‑ if you are using any other ways to search for IXPs, you can use the IXF ID. You can do searches for the IXP, the city, the country, anything you want to search for in IXP is possible. You can also ‑‑ so the idea of this database is that we are trying to make it as automated as possible, so, you can actually also search just by API, you can see which IXPs are only exporting ‑‑ are exporting the IX JSON schemer directly to the database. So if you are doing a search and you are looking for really accurate information, this is the best way to go. You can search for region, so if you are looking for ones in Asia, Europe, South America, you can use the searches there to specify that. And again, all the top headers are all sortable. So you can actually sort under IXF ID, IXP, city, country and so forth, all of these are sourceable, and we also show whether IXPs are MANRS compliant. I think there was a presentation earlier on in the day about MANRS and there are people in the room that know more about MANRS than me. All I'm going to say is that we highlight IXPs and I'll show you later, networks as well, that are MANRS compliant.

And of course a number of ASNs that are connected to that IX. Just to be clear, that the data in the IXP database comes from the IXP. So it is the IXP that controls this.

We have an ASN directory. And this again you can do searches for the ASN or country, city, name. And you can look at type. So, this is something that's quite interesting I think. We get the type actually we pull that data from peering DB where the network defines itself. So, you can actually now also sort under that and see how many particular types of networks are peering at that Internet Exchange point. We have the IP addresses and Mac addresses. And you can also see whether the network is MANRS compliant as well. And last but not least the number of connections that network has.

If we dive a little bit deeper into the IXP information, you can see there's a number of tabs there, but I'm going to just focus on the ASN tab because here we show again the type of network, whether it's MANRS compliant and this is new, just a couple of weeks old. We also actually now show whether the network is present in peering DB or not. So, this is a way for IXPs to look very quickly on this list and say oh, that network is not there, that network is not there, let's go speak to them and get them to add their entry in peering DB.

And finally, the number of connections.

Diving in a little bit deeper ‑‑ yeah, compare. Once we got the database in place, the key thing that we want to do now is actually work on the tools and analysis and do some reporting from this data. And Jessie is going to talk about that in a little bit more detail. But just to give you a quick run through, what you can do here is you can go to the IXP directory, click on the IXPs you want to compare, and then hit the compare button, and here we go. I have got this one, the Italian IXs because I was actually presenting this at IT Nog last week and I think it's quite a nice example because, first of all, you have APIs, so it's all accurate data, and you know, I think they have done a really good job in filling out as much information as they can. So what this means is if you are looking to peer with an IX you can get a real quick snapshot view of what the IXP does and also compare it to other IXs. I think what is really interesting is that ‑‑ ‑‑ so you get all the different services as well ‑‑ and you can also see ‑‑ you can also see the common ASN, so you can see that at these four Internet Exchange points in Italy, these are the common ASNs, but you can also see the unique ASNs, so you can see that actually there is value in peering all of the exchanges because you will reach networks that are not at all of them.

Okay. So, while we were comparing ASNs we thought well it might be interesting to ‑‑ it might be interesting to compare ASNs as well. So we built the compare function for ASNs as well. So again the same thing. You can click on the ASNs. As many as you like by the way. Hit compare. And you get a quick snapshot comparison of the networks. So where are the networks, you know, what kind of network are they? How many IXPs are they connected to? And again, we show the common IXPs that the networks are both connected to. So what this means is if you're a network and you're looking to peer with another network you can have a look here and say we can set up peering, here, here, here and here. And that's probably for ‑‑ I know the larger networks have their own internal tools, but you know for the smaller networks, this is actually really useful.

And we also show which IXs are unique. So here we go, we can see that, you know, I think I have got Google and Akamai, but you can see the exhibitions where Google is present and Akamai isn't, and vice versa. That's a little bit more information there.

So, what's happening next? We are looking to publish a new version of the JSON and this is going to include information about location of the IXs and route server information, further route server and switch information. We're looking at authentication, so the IXPs can actually choose what data they want to have public and what they want to keep private. This is important for IXs because some IXs are quite private in what they want to share and they have certain restrictions from their members, etc. So, this way it's actually up to the IXs who can control who sees what.

We're doing further work on APIs and we're looking at creating some standard APIs and one of these for example includes an X4 API for route server, we can do some further analysis on prefixes and ASNs. We're looking to do some analysis, tools, mapping, searches and Jessie again is going to talk a bit more about this because ‑‑ I don't want to repeat that.

And then the key thing is that, so today actually we have 96 exports, I have got 95 there but I was having a look while I was getting this, just sitting down actually and we have 96 today. And you know, what we want to do ‑‑ eats in a year by the way, so in one year we now have 95 exports in the database. So, what we want to do is obviously want to increase this. So we want to work with the IXPs. As I mentioned earlier, IXP Manager has a built‑in export. The Asteroid Solution, they have a built‑in export and support from our IXs. IXs are a friendly bunch, they like to help each other what we're doing is trying to work with each other to make this happen, so that we can provide accurate, complete and authoritative IXP data to the community.

And, lastly, from me, we have a mailing list for users. If you want to know what's actually in the JSON schemer, you can have a look; it's all available on GitHub. The API is live, you can access it there. We have a wonderful IXP admin team and you can contact them there. So if you have any questions about the JSON or anything related to the IXP database, come and speak to me or one of the admins. If you think this is good work like we do, then support us. We really couldn't do this without the support of the sponsors and for those of you in the room I would like to thank you very much.

And lastly, do we have time for this? Two minutes. I'm done ‑‑ I'm going to switch to Jessie. Okay.

Thank you and if you have any questions, I'm here for the rest of the week, so please come talk to me. Thank you.

JESSE SOWELL: Thank you for being resilient and sticking with us to do this interactive portion.

I have been around the community for a while doing work on Internet Exchanges and other things. And that work has evolved to understand what is the impact of better connectivity on, especially in developing regions on but on met RIS in those developing regions. The reason I am up here giving this part of the presentation is a lot of the work I do in my team is we are building tooling to use the EURO‑IX datasets to use IXP DB to use DS peering DB as well as to integrate other technical I sets, information about demographic information, information about places that do and do not have IXs so we can get a better feel how can we better identify where to place the next IX and whether there is a good chance that it will have an effect on connectivity in the region as well as the economy for that region. So, while I'm doing that, I may as well share some of the features I'm developing with the community. Some of these questions we're asking is which of the features below are the most important to you.

When we look at route server analysis I think most of you have know what we're talking about there.

Maps, which ASNs are present at which facility. So a lot of the data that we currently have we can see what ASNs are at on a particular platform, but is it useful to see, to be able to do searches to show you which particular POP in that particular platform that ASN is participating at?

Another aspect of that is, do you want to be able to do searches such as, we'll have a question about peering calculator shortly, but what kind of visualisation would you like to see? So is it useful whenever with we think back to Bijal's Italy example, would it be useful to transpose that data on to a map so you can actually like visually explore that data as well as have that nice column data available?

And finally, we'll have a question about searching geography in a second, but I guess we should just probably jump to that search by geography question.

If there are other features you would like, please go ahead and like jump in and add those now and we can ‑‑ this is just a quick collection of what other features might be interesting.

So, if you think searching by geography is important, thinking about this, if you have, if you were able to say, hey, I want to search for ‑‑ I want to qualify my searches with geographic information, do you want to be able to say city, metro, country, economy, sub region, is continent useful to you? Which of these scales is useful to you? I should highlight this one is kind of almost the catchall. Do you actually ‑‑ are you actually sufficiently interested in enough nuance of the geography of what you are looking at to be able to say I want this very precise bounding box, it's not necessarily a sub‑region or a metro area, but is there a desire to have to be able to just literally draw a box on map and say qualify my scope, my query, to this bounding box, is that a useful feature?

Just to kind of give a caveat, not a caveat but just a kind of background. That lat/long, that's essentially going to be the underlying feature of all the search queries that my team is building, so that will be there regardless, it's just a question of whether there is something that EURO‑IX should invest in in kind of building into the database.

So, it looks like city metro is the most important. So I guess with we should go. So, this is one question and this is again a caveat. This is something I'll be doing in my research to understand what kind of markets exist on ‑‑ on an IX as a platform. So, this gives me ‑‑ I know stats change quickly but this is giving me a Twitch. So we currently have in peering DB we have NSP content. All of know what these peering categorisations are, one of the things ill be doing is looking at individual participants and saying can we identify finer grain categorisations, so rather than just labelling something like Akamai as ‑‑ or Google as both content delivery networks, they both private content, can we have more finer grained labelling schemes for those actors. So for instance Akamai would be labelled ‑‑ one would be labelled something like a more general purpose content delivery network, whereas Google is more specific to their particular applications. Netflix would be also labelled as a content delivery network, video streaming etc., etc. We would label, say, ISPs in terms of in ISPs this geographic scoped to this region, what other services they provide. So would these things actually be useful? And I think we jumped past that question. So ‑‑

So, this is the final question. We have already had some people ask for this but I'm curious, would you like a peering calculator. This is a list of AS, which IXs do these participate in this would be kind much coupled with a visualisation of that to say hey, we want to, here is a list of AS that is I potentially want to interconnect with with or peer with, show me what IXs these guys are currently participating in, and would you like this kind of, on a nice visual ‑‑ nice map visualisation. It looks like there is a lot of people who would like this. In that case I think I will let you go. Thank you very much for the participation ‑‑

BIJAL SANGHANI: Thank you everyone for taking part. I want to say a big thank you to Amanda and to Serge because we literally put this together in half an hour this morning. So, thank you very much Amanda. And it was Serge's idea. We have kept it really short this time because we're very limited on time. As you all know, but thank you for taking part, because we will be looking at the answers and looking at where we're going next in terms of development with the IXP database. So, thank you. That's it from me.

CHAIR: Okay. And now I know we are running late. But ‑‑ and so, that's ‑‑ so we will skip the off line discussion part because, I mean, you all want to go and eat at some point. So, I would remind ‑‑ well, I'd like to remind you to rate the talks because that's really useful for us to know what's going on, and I guess then we will have to see you at the next RIPE meeting in Rotterdam. Thank you.



LIVE CAPTIONING BY
MARY McKEON, RMR, CRR, CBC
DUBLIN, IRELAND.