Introducing BGPKIT Parser

BGPKIT Parser is an open-source Rust-based MRT/BGP data parser that takes a MRT formatted binary file and turns it into BGP messages. It is one of the most important building block software that enables BGP data processing and analysis tasks.

Design and Features

As mentioned in our previous post introducing BGPKIT Broker, the most used BGP data collection projects, RouteViews and RIPE RIS, both publish their collected BGP data in MRT format on their data platform. The BGPKIT Parser is designed to handle parsing tasks for these data sources.

We design our parser to strictly follow the industry standard, e.g. RFC4271 and RFC6396. BGPKIT Parser is also designed with the following goals:

• performant: comparable to C-based implementations like bgpdump or bgpreader.
• actively maintained: we consistently introduce feature updates and bug fixes, and support most of the relevant BGP RFCs.
• ergonomic API: a three-line for loop can already get you started.
• battery-included: ready to handle remote or local, bzip2 or gz data files out of the box.
• open-source: we want people to use our parser freely and we can continue develop and improve it based on community feedbacks.

To demonstrate how easy it is to get started using BGPKIT Parser, check out the example below where we print out all BGP messages from a remote MRT file on RouteViews:

There are a number of things happens when we call for elem in BgpkitParser::new(url) :

1. it creates a new BgpkitParser struct instance with the provided URL to the data file;
2. it tries to retrieve the content of the remote file and download the raw compressed bytes into memory;
3. it determines the compression type by file suffix and calls corresponding decompression library to create a buffered reader;
4. it then creates an iterator (used by the for loop) that continuous return new parsed items until it reaches the end of the data stream.

We determined that it is worth the extra binary file size to bring in the network and compression libraries into the project so that the library users will never have to worry about handling data downloading and decompression by themselves again.

The Future

The future of the BGPKIT Parser lies on continuous performance and statbility improvements, as well as some exciting features that we are currently planning. Some of the coming features include

1. adding capability of handling real-time data streams coming from RIPE RIS Live and RotueViews’s Kafka BMP stream;
2. supporting data serialization back to MRT files (reverse-parsing), which allows users to produce customized MRT files after data processing;
3. adding WASM support to allow BGP data parsing directly on the web with JavaScript.

Because we are building our software in Rust, we can effortlessly tapping into Rust’s great software ecosystem and continue introducing new features and improvements. The future of BGPKIT Parser is exciting and we can’t wait to bring more features for you to try out!

For more details about the BGPKIT Parser, check out our GitHub repo and our website. Feedbacks are highly appreciated!

GitHub - bgpkit/bgpkit-parser: MRT/BGP data parser written in Rust.

MRT/BGP data parser written in Rust. Contribute to bgpkit/bgpkit-parser development by creating an account on GitHub.

GitHubbgpkit

BGPKIT Parser

Rust-based MRT/BGP Data Parser