Introducing BGPKIT Broker
BGPKIT Broker is a data API service that focuses on building a BGP data file index to enable searching for public/private BGP data files with custom filters. It is one of the building block components designed by BGPKIT to facilitate BGP data processing with ease.
The first step to investigate a BGP event.
Imagine this scenario: a malicious player just attempted to hijack a IP prefix using BGP announcements, and you are interested in learning what exactly happened during that half-hour down time of the victim network, how would you start investigating?
Collecting evidences. Luckily for us, the actors on the Internet, good and bad, always leave traces behind if they use BGP as their method. There are a number of reputable public BGP route collectors operating for years collecting all BGP messages received from their connected router peers and dump them into regular dump files. The most used projects are RouteViews and RIPE RIS Data.
Files are all over the places. There are more than 60 different data collectors from the two projects alone, and each publishing their data under separate sites. The two projects also have different data file structure and compression algorithms for their data dump files. It is not hard to go find one data file during the interested event time from one collector, but it will be a real hassle to gather URLs toward all data files that include information within a specified time range.
BGPKIT Broker — A BGP Data File Index API Service
We designed the BGPKIT Broker to resolve one and only one problem: quickly collect links to the BGP data files that matches a filtering criteria.
You can filter BGP data using multiple criteria:
start_ts: UNIX timestamp that all files must be dumped afterend_ts: UNIX timestamp that all files must be dumped beforedata_type: the type of the data file, can beupdateorribcollector: the collector ID that the files are generated fromproject: the data collection project, can beroute-viewsorriperispageandpage_size: the pagination control for collecting a large number of files
Here is an example REST API call: https://api.broker.bgpkit.com/v1/search?data_type=update&start_ts=1633046400&end_ts=1633132800&collector=rrc00&project=riperis&page%20=2&page_size=3
It asks for all data updates files dumped between 1633046400 and 1633132800, from RIPE RIS’s collector rrc00. It also requested for the second page of the results and each page contains 3 items.
DEPRECATED: BGPKIT Broker API service is freely available to use, hosted at https://api.broker.bgpkit.com/v1/. The documentation is available at https://docs.broker.bgpkit.com/
BGPKIT Broker API and SDK has been upgraded to V2 now. The V1 examples are left up for legacy services that depends on it. Please check out the current API documentation at for more: https://api.broker.bgpkit.com/v2/
BGPKIT Broker Rust API
The BGPKIT Broker API service is built entirely using Rust, and of course we also developed native Rust API to access the broker data with ease using Rust.
The Rust API is open source under MIT license, and with the free API, you can already build your own workflow today!
bgpkitEasy On-premise Deployment
For API service like this, especially that it also collects and indexes data of over 10 years span, one might imagine the deployment could be complex and slow.
For BGPKIT Broker, we spent extra efforts to make the API deployment process as quickly as possible, and also efficient on resource consumption. For references, we bootstrapped the entirety of the database in under 5 minutes, and cost less than 1 GB storage to store the information in PostgreSQL database. The whole database and API run fluently with less than 500MB of RAM usage. This allows use to deployment extra instances when under heavy load without costing a fortune.
Users who need dedicated resource allocation for query performance can contact us for private API hosting that are not shared by others. Enterprise option is also available for on-premise deployment with customization consultation available. If you are interested in testing it out, feel free to shoot us an email at contact@bgpkit.com.
For more information, checkout our website!
