Rayhaan JaufeerallyandClaude Fable 5.1
b0dd357986
A router feeding a collector every path (all_paths) was 4.8 GB resident
with a table that should take a third of that. A heap profile of the
daemon in the Linux VM, with four full tables and such a collector,
showed where it went: the exporter kept, for every route it had
advertised, the route's outgoing form, the parsed attribute list it was
dressed with, through an `Arc<Dressed>` in the per-route record. One
such form lived for every distinct attribute set advertised, about a
kilobyte each, for the life of the session, on top of the exporter's
cache of 262,144 of them. The record needs only to tell whether a route
changed, and the table's own attribute set, which is in memory anyway,
tells it that: the outgoing form is a function of the set, the peer the
route came from and the policy.
This change:
* `Sent` holds an `Arc<AttrSet>` instead of an `Arc<Dressed>`, and a
route is sent again when its set differs. `Verdict::All` carries both
while a flush is on.
* `DRESSED_KEPT` goes from 262,144 to 65,536: a kilobyte a set, so 64 MB
rather than 256 MB a session at most; what a flush is sending holds
its own.
* The status page's header shows the daemon's memory as jemalloc
accounts for it (allocated and resident), `/api/status` has the six
figures (`memory`: allocated, active, metadata, resident, mapped,
retained), and `bgp_memory_bytes{kind}` exports them. jemalloc is built
with statistics for this, and `bgp-web` reads them through
`tikv-jemalloc-ctl`.
* `scripts/profile/full-table.sh` gains `COLLECTOR=1`, a collector
session over IPv6 that takes both families with ADD-PATH, and `HEAP=1`,
which builds the daemon with jemalloc's heap profiler (the new
`heap-profiling` feature of the daemon crate, not for production) and
turns the profile into a flame graph with jeprof; and it reports
memory with the table in and again 30 s later. The feeder
(`crates/bench/examples/feed.rs`) plays the collector with
`FEED_COLLECTOR=1`.
Measured in the VM, four peers' full tables and a collector sent every
path (4.9 million routes advertised): 1,723 MiB resident before, 1,286
MiB after, with jemalloc's allocated figure from 1,500 MiB to 946 MiB.
Without the collector the same tables take 630 MiB, so the collector's
bookkeeping is now about 50 bytes a route advertised. Of the resident
figure, some 400 MiB was pages jemalloc had freed and not yet returned:
it returns them on allocator activity only, and a router that has its
tables in allocates little. jemalloc is now built with its background
thread on (the `background_threads` feature), which returns them on
time: 1,020 MiB resident 30 s after the load, against 1,286 MiB without
it.
Documentation: `memory` in the HTTP API reference, `bgp_memory_bytes`
in the metrics reference and the monitoring guide, the script's new
settings in the testing guide.
Testing: the ADD-PATH integration tests pass, including the collector
ones; the heap profile and the figures above are from
`PEERS=4 COLLECTOR=1 HEAP=1 scripts/profile/full-table.sh` in the VM.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Ferrix BGP
Ferrix is a BGP daemon for Linux routers, written in Rust. It exchanges routes with neighbors, filters them by policy and the RPKI, selects the best path for each prefix, installs it in the kernel, and passes routes on as BGP roles and communities permit. It aims to be small, safe, observable, and configured from one file that can be read by someone who did not write it.
Status
Ferrix routes the author's network, AS210036: three routers carrying full
tables from transit providers, route servers at several Internet exchanges,
and bilateral peers, with routes reflected between the routers over iBGP.
It is young software all the same. Read the limitations
before relying on it, and treat version 0.x as meaning that configuration
and APIs may still change between releases.
What it does:
- BGP-4 over IPv4 and IPv6, with four-octet AS numbers, multiprotocol reach for both families, IPv4 routes with IPv6 next hops (RFC 8950), route refresh, graceful restart and ADD-PATH receive.
- Import policy per neighbor: prefix limits, prefix and origin AS lists, bogon checks, AS path checks, and communities added on the way in.
- Route origin validation and ASPA path verification against an RTR cache.
- BGP roles (RFC 9234) and the Only to Customer attribute, with export limits as a second line of defence against route leaks.
- Best path selection and installation of the result in the Linux kernel over netlink, in a routing table of your choice.
- Route reflection between routers of the same AS, and passive neighbors.
- A status page, a JSON API, server-sent events, and Prometheus metrics.
- Graceful restart of the daemon itself for upgrades, with routes left in the kernel while it is away.
Not supported
- Per-client best path selection for route servers, and ADD-PATH for advertised routes
- TCP-AO, TCP MD5, and GTSM
- Long-Lived Graceful Restart, BFD, and configuration changes without a restart
- Address families other than IPv4 and IPv6 unicast
Building
You need a stable Rust toolchain from rustup.rs, and a
C compiler and make for the jemalloc allocator the daemon uses (for
example build-essential on Debian and Ubuntu, base-devel on Arch Linux,
or the Xcode command line tools on macOS). The Protocol Buffers compiler is
not needed: the configuration schema is compiled in pure Rust.
git clone https://git.rayhaan.net/rayhaan/ferrix.git
cd ferrix
cargo build --release --locked -p bgp
The binary is target/release/bgp. --locked builds with the dependency
versions recorded in Cargo.lock, which are the versions CI tested.
To try it without touching any router, run the quick-start configuration. It listens on loopback, needs no privileges, and serves the status page at http://127.0.0.1:8080; the quick start guide adds a test peer that announces routes to it.
target/release/bgp --check docs/examples/quick-start.textpb
target/release/bgp docs/examples/quick-start.textpb
Other ways to build:
- A static Linux executable that runs on any x86_64 distribution with
nothing else installed:
scripts/build-static.sh. It needs thex86_64-unknown-linux-musltarget and a musl C compiler; the script's header says how to install them, including on macOS. - A container image with the provided
Dockerfile:docker build -t ferrix .
Kernel route installation needs Linux and CAP_NET_ADMIN; everywhere else
Ferrix is a BGP speaker only. The installation
guide and the
deployment guide cover
privileges, the systemd unit, and upgrading without withdrawing routes.
Documentation
The documentation is a website built from docs/ with
Zola: zola --root docs serve, then open
http://127.0.0.1:1111. The sources are Markdown with Zola front matter,
shortcodes and internal links, so the forge shows them with those parts
unrendered; docs/README.md explains what each is. The
pages:
- Overview, installation and quick start
- User guide: BGP neighbors, import policy, BGP roles, RPKI, kernel route installation, route advertisement, graceful restart, configuration examples
- Deployment and monitoring
- Reference: configuration, command line, HTTP API, metrics, supported standards
- Architecture, testing and contributing
Complete configurations for an edge router, an exchange router and a
transit customer are in docs/examples/; CI checks that
each of them loads.
Security
- The status page and HTTP API are not authenticated and show the whole
configuration and routing table. Bind
http_endpointto a loopback address, or put it behind a proxy that authenticates. - Ferrix does not implement TCP-MD5 or TCP-AO. Filter TCP port 179 in the host firewall so that only configured neighbors can connect.
- Everything read from the network, the RPKI cache, and the kernel is
parsed without panicking; each parser has property tests and a fuzz
target.
unsafecode is forbidden throughout.
If you find a vulnerability, please contact the maintainer directly rather than opening a public issue.
Layout
| Crate | What is in it |
|---|---|
crates/primitives |
Prefixes, address families, parsing helpers |
crates/packet |
The wire format: messages, attributes, capabilities |
crates/rib |
The routing table, best route selection, the journal of changes |
crates/rpki |
The RTR client; origin validation and ASPA path verification |
crates/fib |
The best routes in the Linux kernel, over netlink |
crates/server |
Sessions, the state machine, policy, configuration |
crates/web |
The status page, HTTP API and metrics |
crates/integration-tests |
A test server and test peers, and the tests that run the daemon end to end |
crates/bench |
Benchmarks at the scale of the Internet routing table |
bin/bgp |
The daemon |
Development
CI runs these on every push, and fails on any warning:
cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --locked --workspace
RUSTDOCFLAGS="-D warnings" cargo doc --locked --workspace --no-deps --document-private-items
zola --root docs build
It also runs the tests that need a real Linux kernel
(scripts/kernel-tests.sh) and a lab that peers the daemon with FRR, BIRD
and Routinator (scripts/interop/lab.sh); scripts/linux-vm.sh runs
either from macOS in a Lima VM.
cargo test includes property tests: generated messages must survive being
written and read back, and no input, however damaged, may make a parser
panic. They take about a second; for a longer hunt:
PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties
Coverage-guided fuzz targets for the same parsers are in fuzz/; see
fuzz/README.md. CI runs each of them for a minute.
Development with AI agents
Much of Ferrix is written with an AI coding agent (Claude Code) working
alongside the maintainer, who directs the work, reviews every change, and
runs the result on real routers. Nothing is merged on the agent's say-so:
every change passes the checks above, the kernel tests and the
interoperability lab, and the changes that matter are tried on the network
first. Commits made with an agent carry a Co-Authored-By trailer that
says so. If you contribute, you are welcome to work the same way, or not;
what is reviewed is the change, not how it was produced.
License
Apache License 2.0; see LICENSE.