Real-time BGP data processing is very critical on building monitoring services that can detect BGP issues quickly with minimum delay and react to anomalies quickly and mitigate potential issues.

We are creating a new series of posts describing how we design our software to work with real-time BGP data streams. As an opening, we will describe how we handle data streams with BMP protocol and OpenBMP messages.

---

BMP

The BGP Monitoring Protocol (BMP) is a protocol that allows monitoring of BGP devices.

The RFC7854 describes the purpose of BMP as:

Many researchers and network operators wish to have access to the contents of routers’ BGP Routing Information Bases (RIBs) as well as a view of protocol updates the router is receiving. This monitoring task cannot be realized by standard protocol mechanisms. Prior to the introduction of BMP, this data could only be obtained through screen scraping.

BMP provides access to the Adj-RIB-In of a peer on an ongoing basis and a periodic dump of certain statistics the monitoring station can use for further analysis. From a high level, BMP can be thought of as the result of multiplexing together the messages received on the various monitored BGP sessions.

There are multiple types of BMP messages, each serving different purposes.

• Peer up and down notification: notification about the status of peering sessions to a monitored router;
• Initiation message: inform the monitoring station of the routers vendor, software version, and so on;
• Termination message: provides information on why a monitored router is terminating a session;
• Route monitoring:initial synchronization of the routing table;
• Route mirroring: verbatim duplication of messages as received.

For real-time BGP data processing, we are specifically interested in the route monitoring and route-mirroring messages, as we provide the routing information encoded as actual BGP messages.

---

OpenBMP

OpenBMP is a software implementation of the BMP protocol. It is an open-source project created by Cisco and currently maintained by nice folks from CAIDA/UCSD and RouteViews. It is implemented in C++, can be used with any compliant BMP sender (e.g., router).

OpenBMP provides multiple formats for outputting the BMP messages collected from the connected routers, one of which is the raw_bmp format, which is a thin wrapper of the raw BMP messages. The raw_bmp format provides the best performance and allows use to handle the BMP messages directly without having to write a different parser for the plaintext messages.

RouteViews currently provides a OpenBMP Kafka stream that streams BMP messages from their collectors.

---

BGPKIT Parser with BMP/OpenBMP Support

We develop BGPKIT Parser to provide a one-stop solution for handling all parsing tasks regarding BGP data. Supporting real-time data like BMP is a very important milestone for us.

We have recently developed the full support for BMP messages, and partial support for OpenBMP messages (for raw_bmp type only). This enables us to start working with real-time BMP streams like RouteViews’ Kafka stream.

Below is an example code that takes RouteViews’ Kafka OpenBMP stream and parse the messages into internal data structures:

let mut reader = Cursor::new(Vec::from(kafka_payload));
let header = parse_openbmp_header(&mut reader).unwrap();
if let Ok(msg) = parse_bmp_msg(&mut reader) {
    info!("Parsing OK: {:?}", msg.common_header.msg_type);
    match msg.message_body {
        MessageBody::RouteMonitoring(m) => {
            dbg!(m.bgp_update);
        }
        _ => {}
    }
}

Here is a break down of what it does:

• it first creates a bytes reader from the raw Kafka message payload;
• then parse OpenBMP message header, which contains some basic information about the BMP session;
• then it calls the parse_bmp_msg function to parse the embedded raw BMP messages and print out the BGP update messages if the parsing is successful.

Here is a full code example:

We have published the SDK on crates.io and GitHub. Feel free the check out the example code at examples/routeviews-kafka.rs if you are interested.