Rhai made a decision about a route cost one to three microseconds,
some two hundred times what the same costs in Rust. That was bearable
where a function is asked once for an UPDATE, and is not where every
route is weighed against every peer, as it is when routes are passed
on. What is wanted of a policy (mark routes by where they came from,
mark what is announced, let communities say who is told what) needs no
language; it is to be configuration, done natively.
Kept from that work: route selection goes by a LOCAL_PREF that policy
set on a route from an external peer, and the lab's configuration can
be added to by a phase.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:
fn from_peer(route, peer) {
if route.has_community("65535:666") { return false; }
route.add_community(LEARNED_FROM[peer.role]);
route.local_pref = PREFERENCE[peer.role];
true
}
Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.
Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.
What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.
Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.
Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A peer's endpoint may be a link-local IPv6 address with the interface it
is on. Such a peer is the address and the interface together, wherever
peers are told apart: admission, collisions, hold-downs, the refusal log,
the status page. The interface is looked up by name each time it is
needed, so one that appears later or is made anew is found.
Our routes go out with our link-local address in both places of
MP_REACH_NLRI's next hop, as FRR sends its own. A peer's link-local next
hop is taken alone, twice, or after "::" (BIRD), and resolved in the
kernel on the interface of the session rather than by looking fe80::/64
up, which every interface has.
The lab has a new phase, linklocal: sessions with FRR (by interface, with
radvd beside the daemon) and BIRD over a LAN with link-local addresses
only, both families, pings across, the interface down and up.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
With rpki { caches { ... } } in the configuration the server keeps the
validated ROA payloads of a validator and validates every route from a
peer outside its own AS (RFC 6811). An invalid route is refused, unless
its peer has accept_rpki_invalid. Valid and not-found routes are taken;
validity does not otherwise enter route selection.
Validity is not settled when a route arrives: the data changes every few
minutes. The routes follow, without the peer sending anything again.
- A route refused as invalid is held by its session, out of the table.
When the data changes the session looks at what it holds, and what is
no longer invalid goes into the table. A held route is the peer's to
withdraw or replace like any other.
- A route in the table that becomes invalid is found by a task that
looks through the table when the data changes, at the prefixes the
changed VRPs cover, and is taken out and held by its session.
- If the data expires because no cache confirms it, nothing is invalid
any more and everything held is taken.
- A session holds at most 200,000 routes; past that it asks the peer for
a route refresh when the data changes.
So that the table can be looked through without parsing attributes, an
attribute set's key now has the route's origin AS. Memory per route is
unchanged at 52 bytes on the real table.
Shown: /api/rpki and a section of the status page with the cache, its
state and the age of the data; each route's validity; each peer's count
of held routes. Refusals are recorded under "RPKI invalid". Metrics:
bgp_rpki_vrps and bgp_rpki_updates_total.
Also: Negotiated says whether the peer supports route refresh, and
docs/RFCs.md is brought up to date.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The work of the speaker-cleanup branch as one commit; the branch keeps
the step-by-step history.
Sessions
- The state machine is a pure function from events to actions, driven by
one task per connection. Idle, Connect and Active belong to a dialer
that keeps configured peers up with backoff.
- Capabilities are negotiated: address families, four-octet AS numbers
with AS_TRANS and AS4_PATH, route refresh, ADD-PATH per family and
direction.
- Collision detection per RFC 4271 section 6.8, by address and
identifier.
- Connections are taken only from configured peers, a few per address.
- A panic in a session is contained and answered with a Cease; a core
task that stops shuts the server down with a Cease to every peer.
- Writes that make no progress are given up (RFC 9687).
Codec
- Malformed UPDATEs are handled as RFC 7606 says: treat-as-withdraw,
attribute discard, or session reset, with the NOTIFICATION to send.
- Address families are open types; any can be named and ignored.
- Prefixes are fixed-size values, normalised, with path identifiers.
- Lengths are filled in after writing; one definition of each type code.
- Property tests and fuzz targets for every decoder.
Routing table
- Adj-RIBs-In and Loc-RIB in one sharded table that sessions write to
directly. Routes are sixteen bytes; attribute sets are interned in
wire form. Best-path selection per RFC 4271 section 9.1.2.2 with
deterministic MED.
- A journal per shard lets followers read changes at their own pace;
one that falls behind is resynchronised.
- On a RIPE RIS dump of 148 peers and 39.3M routes: loaded in 2.0 s on
12 cores, about 52 bytes a route.
Announcing
- Originated routes come from the configuration. Each session has its
own exporter with an Adj-RIB-Out, so a slow peer delays only itself.
- Routes to a peer in our own AS have an empty path and a LOCAL_PREF.
Operations
- HTTP status page, JSON API, event stream and Prometheus metrics.
- Benchmarks on synthetic and real tables.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- Implement passive-side BGP FSM (OpenSent → OpenConfirm → Established)
with hold timer, keepalive timer, and NOTIFICATION on expiry
- Add bgp-server crate: config validation, TCP listener accept loop
(BgpServerBuilder → BgpServer), connection dispatch
- Add bgp-rib crate: Adj-RIB-In actor owning route state with event
broadcast for streaming consumers and snapshot for introspection
- Wire Adj-RIB-In into FSM: announce/withdraw on UPDATE for both legacy
IPv4 NLRI fields and MP_REACH/MP_UNREACH path attributes
- Add bgp-integration-tests crate with 15 end-to-end tests covering
session establishment, timers, UPDATE handling, and RIB introspection
- Fix require_empty! macro bug in open.rs (checked wrong buffer)
- Fix FourByteAsn capability parser returning consumed buffer as remainder
- Fix tracing subscriber not being installed in TracingArgs::install
- Expand docs/RFCs.md with full implementation status across 15 RFCs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
