Real-time RIS Live Data with BGPKIT Parser

In terms of real-time BGP data processing, RIPE NCC provides a great data source: Routing Information Service Live (RIS Live).

To begin with, here is what RIS Live by the creators:

RIS Live is a feed that offers BGP messages in real-time. It collects information from the RIS Route Collectors (RRCs) and uses a WebSocket JSON API to monitor and detect routing events around the world. A non-interactive full stream (“firehose”) is also available.

In essence, RIS Live provides:

• a WebSocket interface to stream BGP messages in real-time
• ability to subscribe to “sub-streams” with custom filtering messages
• JSON-encoded BGP messages as the stream payload
• “firehose” HTTPS stream interface as well, without needing to work with websocket.

In this post, we will discuss how to use the RIS Live stream in practice.

RIS Live Message Format

RIS Live has client messages and server messages.

The client messages is used to setup or dismantle “subscriptions”, which essentially tell the server what kind of BGP messages a client would like to receive, and allow the server to send only the interested messages to the client.

A server acknowledges the requests from the client and afterwards start streaming requested data back to the client. At a high-level, a server sends either ris_message or ris_error messages. The ris_message is the main payload that we are interested in, while the ris_error message provides debugging messages for the scenarios where stream or subscription fails.

Subscribe to a WebSocket Stream

RIS Live provides great flexibility for the clients to specify/narrowdown the interested messages, allowing both the server and client process less messages during a streaming session.

• host: only messages collected from a particular RRC (e.g. rrc21)
• type: only messages of a given type, e.g. UPDATE , OPEN
• require : only messages containing a given key, e.g. withdrawals will return only message that contains any withdrawn prefixes
• peer: messages from a particular BGP peer
• path : ASN or pattern to match the AS Path attribute in BGP update messages
• prefix: only messages containing information for a given prefix
• moreSpecific and lessSpecific: only messages that are the subprefix or super-prefix of the specified prefix
• includeRaw: whether to include the Base64-encoded RAW BGP messages

As an example, let’s take a look at the following message from the official manual: {"host":"rrc01","type":"UPDATE","require":"announcements","path":"64496,64497$"}

Example subscription message composer on RIS Live official site

As an example, let’s take a look at the following message from the official manual: {"host":"rrc01","type":"UPDATE","require":"announcements","path":"64496,64497$"}

• collected by rrc01
• BGP UPDATE messages
• have at least one announced prefix
• the last two hops of the AS Path is 66496 and 64497 (the origin)

The ris_message consists of “common header” fields and “data” fields (although they’re on the same level).

The “common header” fields are present for all types of sub-type messages, including timestamp, peer, peer_asn, id, host, type . The rest of the fields are the data fields that are dependent on the type of the messages. For most people, the UPDATE message is what they need. The following JSON block is an example message pulled directly from the demo site.

Example JSON formatted RIS message:

This example shows a BGP announcement of AS132354 originating two prefixes 103.249.208.0/23 and 103.14.184.0/24 , with the next hop to be 37.49.237.228. At this point, the information we see here is pretty similar to what we can see from other BGP MRT reader’s output (e.g. from bgpdump or bgpreader), just in JSON format.

WebSocket or Firehose?

Provided that RIS Live provides both WebSocket and HTTP Firehose, one would naturally wonder which one is the right choice for their application. Here we have a brief comparison between the two in the context of RIS Live.

WebSocket

Good:

• easy to customize stream by composing a simple JSON subscribe message
• work with various toolings in languages like Python and JavaScript

Bad:

• requires extra library dependencies to work with WebSocket
• need to write somewhat lengthy to get started (comparing to firehose)

Firehose

Good:

• easy to consume by simply calling GET request on the URL
• simple single-liner commandline program can start the stream (e.g. a simple curl call), no need complex script

Bad:

• customizing stream is doable with XRIS-SUBSCRIBE HTTP request header, but feels clunky and limited
• in my personal tests, the stream get disconnected often due to the stream cannot keep up with the data producer. this did not happen with websocket tests.

Summary

If your application could afford additional dependencies or writing extra scripts, WebSocket is the better choice. RIS Live official manual also makes implication that the WebSocket format is the current formally-supported streaming method.

---

RIS Live Coding Example with BGPKIT Parser

Now that we have a basic idea of what is RIS Live and the basic message format, we can get started working on some code that will actually use RIS Live to do something useful.

In the following example, we will build a short monitoring service that alerts us when Facebook operators announces their DNS IP prefix (see what happened before here). We are going to build the service in Rust with BGPKIT Parser , WebSocket library Tungstenite.

First, lets collect some basic information about what we are going to monitor here:

1. Facebook’s autonomous system number is 32934. So we will watch for all messages that was originated from AS32934.
2. Facebook’s DNS server IP prefixes involved in the previous incidence are 129.134.30.0/23 and 185.89.218.0/23 . So we want to carefully watch these two prefixes in our monitoring system.
3. We want to use one of the RIPE RIS collectors’ data for monitoring, rrc21 is a good choice since it’s being used by RIS Live’s demonstration. You can easily extend this service by tweaking the subscription message later.

OK, we are good to go. Let’s do it!

Setting up the stream

We picked the Tungstenite library as our WebSocket library of choice, partly because it has a very straightforward API design.

Let’s first connect to the websocket server by calling connect function given a websocket URL. One thing to notice is that the URL protocol section here is ws as opposed to the wss mentioned in the RIS Live documentation. For some reason, Tungstenite does not work with wss protocol (with SSL).

use tungstenite::{connect, Message};
constRIS_LIVE_URL: &str = "ws://ris-live.ripe.net/v1/ws/?client=rust-bgpkit-parser";let (mut socket, _response) = connect(Url::parse(RIS_LIVE_URL).unwrap()).expect("Can't connect to RIS Live websocket server");

Now, with a socket ready, we will first send a subscription message to let server know that we want some messages and we are ready to receive.

let msg = json!({"type": "ris_subscribe", "data": {"host": "rrc21"}}).to_string();
socket.write_message(Message::Text(msg)).unwrap();

Here we composed a simple subscription message that limits the stream to have messages only from rrc21 collector.

Parsing JSON messages

At this point, we have a WebSocket connection to RIS Live server, and have sent out a subscription message to the server. The server should be sending back messages anytime now, and we are ready to consume the stream.

We would like to code the following behavior:

1. continuously reading the websocket messages;
2. parse JSON string into internal BGP structs;
3. check each message if it contains origins (withdraw-only messages does not contain AS paths, and thus no origins either;
4. if the origin AS is AS32934, and the announced prefix is 129.134.30.0/23 or 185.89.218.0/23 , then we print out the message to output.

loop {
    let msg = socket.read_message().expect("Error reading message").to_string();
    if letOk(elems) = parse_ris_live_message(msg.as_str()) {
        for elem in elems {
            if letSome(origins) = elem.origin_asns.as_ref() {
                if origins.contains(&32934) &&
                    ( elem.prefix.to_string() ==  "129.134.30.0/23".to_string() ||
                        elem.prefix.to_string() ==  "185.89.218.0/23".to_string() )
                {
                    println!("{}", elem);
                }
            }
        }
    }
}

The full example code can be found here:

---

Building More with BGPKIT Tools

As introduced in our previous blog post, we added support of real-time BMP stream to BGPKIT Parser as well. Combining with RIPE RIS Live, and RouteViews BMP stream, we can build a powerful real-time BGP monitoring service directly within BGPKIT Parser. We also offer indexing and processing of historical BGP data as well with BGPKIT Broker.

Our goal at BGPKIT is to design, develop, and deploy the most developer-friendly BGP data processing toolkit. To learn more about our offerings, please check out our website and official Twitter account.