0 Fork 0

Files

T

Rayhaan JaufeerallyandClaude Fable 5.1 3608897245 fib: resolve next hops through the kernel's routing table

Before this change, Ferrix only installed a route if its BGP next hop was
directly connected: an address in the subnet of an interface that is up.
A route with any other next hop was held at import and never installed.
This made routes from iBGP neighbors unusable unless the neighbor set
itself as the next hop, and it is why the route reflector rewrites the
next hop on every reflected route.

Ferrix now resolves a next hop that is not directly connected through the
kernel's own routes, as other BGP implementations do:

* The route is looked up in one kernel table, the "resolution table"
  (`kernel { resolution_table: N }`, default 254 = main). The longest
  prefix that contains the next hop wins; of several routes for that
  prefix, the one with the lowest metric.
* Because the Linux kernel does not resolve gateways recursively, the BGP
  route is installed with the gateway and interface of the kernel route
  that covers the next hop. A kernel route without a gateway (a route
  onto a point-to-point or tunnel interface) makes the next hop itself
  the gateway on that interface. A kernel route that names a nexthop
  object is resolved through the object.
* Routes with protocol `bgp` never resolve a next hop, whether Ferrix's
  own or another daemon's. A next hop that is reachable only through a
  BGP route depends on another BGP next hop, and such chains are not
  predictable.
* The default route does not resolve a next hop unless
  `kernel { resolve_via_default_route: true }` is set. On a router the
  default route points at a transit provider, and a next hop that is
  reachable only through it is not usable.
* Blackhole, unreachable and prohibit routes make the next hop
  unresolvable, as does a route whose interface is down.

The same resolution is used by the import policy and the revalidator, so
a route whose next hop cannot be resolved is still held rather than
selected (RFC 4271 section 9.1.2.1), and is taken when a kernel route
makes its next hop resolvable.

Implementation:

* `bgp_fib::routes::KernelRoutes` holds the routes of the resolution
  table and answers longest-prefix lookups.
* `bgp_fib::topology::Topology` bundles the interfaces, the kernel routes
  and the kernel's nexthop objects into one value. The kernel task
  publishes it through `Fib::topology()` (formerly `Fib::interfaces()`),
  and `plan::resolve` takes it in place of `Interfaces`.
* The kernel task learns the resolution table with a dump of that table
  alone, using NETLINK_GET_STRICT_CHK (Linux 4.20 and later; older
  kernels get a dump of every table, filtered in user space). It asks
  again, debounced, whenever a dedicated netlink socket that is a member
  of the route multicast groups receives anything. That socket is given a
  64 KiB receive buffer on purpose: the kernel reports Ferrix's own route
  installs on it too, up to a million at a time, and an ENOBUFS overflow
  is treated as the same hint as a message. Its contents are never
  parsed. An earlier attempt to receive route notifications on the
  socket that writes routes failed, because the kernel only excludes the
  requesting socket from a notification when NLM_F_ECHO was requested.
* `Unresolved::NotConnected` is replaced by `NoRoute`, `OnlyDefault` and
  `Discarded`, which appear as the reasons in `/api/kernel` and on the
  status page.

Configuration: two new `kernel` fields, `resolution_table` and
`resolve_via_default_route`, documented in the configuration reference
and in the kernel guide, whose "Next Hop Resolution" section is rewritten.
Existing configurations behave as before except that routes with far
next hops may now be installed where a static or connected route covers
the next hop.

Testing: unit tests for `KernelRoutes`, `Topology` and resolution
through static, tunnel, nexthop-object, blackhole and default routes; a
kernel test (`a_far_next_hop_goes_the_way_the_kernels_routes_say`) that
checks the installed routes follow static, tunnel and nexthop-object
routes as they are added, replaced and removed; the existing kernel
tests, the interoperability lab with FRR and BIRD, and the workspace
tests, all passing. Installing 200,000 routes still runs at about
186,000 routes per second.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-10-08 22:23:27 +02:00

9.0 KiB

Raw Permalink Blame History

Ferrix BGP

Ferrix is a BGP daemon for Linux routers, written in Rust. It exchanges routes with neighbors, filters them by policy and the RPKI, selects the best path for each prefix, installs it in the kernel, and passes routes on as BGP roles and communities permit. It aims to be small, safe, observable, and configured from one file that can be read by someone who did not write it.

Status

Ferrix routes the author's network, AS210036: three routers carrying full tables from transit providers, route servers at several Internet exchanges, and bilateral peers, with routes reflected between the routers over iBGP. It is young software all the same. Read the limitations before relying on it, and treat version 0.x as meaning that configuration and APIs may still change between releases.

What it does:

BGP-4 over IPv4 and IPv6, with four-octet AS numbers, multiprotocol reach for both families, IPv4 routes with IPv6 next hops (RFC 8950), route refresh, graceful restart and ADD-PATH receive.
Import policy per neighbor: prefix limits, prefix and origin AS lists, bogon checks, AS path checks, and communities added on the way in.
Route origin validation and ASPA path verification against an RTR cache.
BGP roles (RFC 9234) and the Only to Customer attribute, with export limits as a second line of defence against route leaks.
Best path selection and installation of the result in the Linux kernel over netlink, in a routing table of your choice.
Route reflection between routers of the same AS, and passive neighbors.
A status page, a JSON API, server-sent events, and Prometheus metrics.
Graceful restart of the daemon itself for upgrades, with routes left in the kernel while it is away.

Not supported

Per-client best path selection for route servers, and ADD-PATH for advertised routes
TCP-AO, TCP MD5, and GTSM
Long-Lived Graceful Restart, BFD, and configuration changes without a restart
Address families other than IPv4 and IPv6 unicast

Building

You need a stable Rust toolchain from rustup.rs, and a C compiler and make for the jemalloc allocator the daemon uses (for example build-essential on Debian and Ubuntu, base-devel on Arch Linux, or the Xcode command line tools on macOS). The Protocol Buffers compiler is not needed: the configuration schema is compiled in pure Rust.

git clone https://git.rayhaan.net/rayhaan/ferrix.git
cd ferrix
cargo build --release --locked -p bgp

The binary is target/release/bgp. --locked builds with the dependency versions recorded in Cargo.lock, which are the versions CI tested.

To try it without touching any router, run the quick-start configuration. It listens on loopback, needs no privileges, and serves the status page at http://127.0.0.1:8080; the quick start guide adds a test peer that announces routes to it.

target/release/bgp --check docs/examples/quick-start.textpb
target/release/bgp docs/examples/quick-start.textpb

Other ways to build:

A static Linux executable that runs on any x86_64 distribution with nothing else installed: scripts/build-static.sh. It needs the x86_64-unknown-linux-musl target and a musl C compiler; the script's header says how to install them, including on macOS.
A container image with the provided Dockerfile: docker build -t ferrix .

Kernel route installation needs Linux and CAP_NET_ADMIN; everywhere else Ferrix is a BGP speaker only. The installation guide and the deployment guide cover privileges, the systemd unit, and upgrading without withdrawing routes.

Documentation

The documentation is a website built from docs/ with Zola: zola --root docs serve, then open http://127.0.0.1:1111. The sources are Markdown with Zola front matter, shortcodes and internal links, so the forge shows them with those parts unrendered; docs/README.md explains what each is. The pages:

Overview, installation and quick start
User guide: BGP neighbors, import policy, BGP roles, RPKI, kernel route installation, route advertisement, graceful restart, configuration examples
Deployment and monitoring
Reference: configuration, command line, HTTP API, metrics, supported standards
Architecture, testing and contributing

Complete configurations for an edge router, an exchange router and a transit customer are in docs/examples/; CI checks that each of them loads.

Security

The status page and HTTP API are not authenticated and show the whole configuration and routing table. Bind http_endpoint to a loopback address, or put it behind a proxy that authenticates.
Ferrix does not implement TCP-MD5 or TCP-AO. Filter TCP port 179 in the host firewall so that only configured neighbors can connect.
Everything read from the network, the RPKI cache, and the kernel is parsed without panicking; each parser has property tests and a fuzz target. unsafe code is forbidden throughout.

If you find a vulnerability, please contact the maintainer directly rather than opening a public issue.

Layout

Crate	What is in it
`crates/primitives`	Prefixes, address families, parsing helpers
`crates/packet`	The wire format: messages, attributes, capabilities
`crates/rib`	The routing table, best route selection, the journal of changes
`crates/rpki`	The RTR client; origin validation and ASPA path verification
`crates/fib`	The best routes in the Linux kernel, over netlink
`crates/server`	Sessions, the state machine, policy, configuration
`crates/web`	The status page, HTTP API and metrics
`crates/integration-tests`	A test server and test peers, and the tests that run the daemon end to end
`crates/bench`	Benchmarks at the scale of the Internet routing table
`bin/bgp`	The daemon

Development

CI runs these on every push, and fails on any warning:

cargo fmt --all -- --check
cargo clippy --locked --workspace --all-targets -- -D warnings
cargo test --locked --workspace
RUSTDOCFLAGS="-D warnings" cargo doc --locked --workspace --no-deps --document-private-items
zola --root docs build

It also runs the tests that need a real Linux kernel (scripts/kernel-tests.sh) and a lab that peers the daemon with FRR, BIRD and Routinator (scripts/interop/lab.sh); scripts/linux-vm.sh runs either from macOS in a Lima VM.

cargo test includes property tests: generated messages must survive being written and read back, and no input, however damaged, may make a parser panic. They take about a second; for a longer hunt:

PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties

Coverage-guided fuzz targets for the same parsers are in fuzz/; see fuzz/README.md. CI runs each of them for a minute.

Development with AI agents

Much of Ferrix is written with an AI coding agent (Claude Code) working alongside the maintainer, who directs the work, reviews every change, and runs the result on real routers. Nothing is merged on the agent's say-so: every change passes the checks above, the kernel tests and the interoperability lab, and the changes that matter are tried on the network first. Commits made with an agent carry a Co-Authored-By trailer that says so. If you contribute, you are welcome to work the same way, or not; what is reviewed is the change, not how it was produced.

License

Apache License 2.0; see LICENSE.

9.0 KiB Raw Permalink Blame History