Add an interoperability lab with FRR, BIRD and Routinator

scripts/interop/lab.sh runs the daemon against FRR, BIRD and Routinator,
each in a network namespace on one exchange LAN, and checks what every
side ends up with: sessions and routes over IPv4 and IPv6 with their
attributes, four-octet AS numbers, ADD-PATH, RFC 9234 roles and Only to
Customer, RTR version 2 and invalid routes, route refresh, session resets,
killed peers, a cut wire, the daemon stopped and killed, a table of a
million routes and a restart under it, role mismatch and the prefix limit.
All of it passes against FRR 10.5.1, BIRD 2.18 and Routinator 0.15.2.

scripts/interop/real-rpki.sh checks the RTR client against a Routinator
with the real RPKI: 713,258 IPv4 and 187,479 IPv6 VRPs and 3,379 ASPAs,
the same numbers on both sides.

What the lab found, fixed here:

- A daemon restarted with its routes in the kernel removed them all at
  once, its table being empty until the peers had sent theirs again, and
  then put them back: no forwarding meanwhile. Routes of an earlier run
  are now kept for `stale_seconds` (120 by default) and only those nobody
  announced again are removed. A restart under a million routes takes
  1.1 s to be in step, with nothing removed.
- FRR ends a session it loses to a collision with a state machine error
  without a subcode, which was shown as "error code 5 subcode 0".

CI gets a job that runs the lab.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Rayhaan JaufeerallyandClaude Fable 5.1 committed 2026-10-02 17:44:49 +02:00
1 parent 1de75fc610
commit 321af51e4b
16 files changed
+1024 -17

No files matched your search

+30 -1
View File
@@ -205,7 +205,9 @@ kernel {
run two daemons that install BGP routes into the same table with the same
metric.
- **Starting and stopping.** At startup, routes of ours left by an earlier
run are kept if they are still right and removed if not. On SIGINT or
run are kept, so that the machine goes on forwarding while the peers send
their tables again; those that no peer has announced again after
`stale_seconds` (120 unless set) are removed. On SIGINT or
SIGTERM the daemon closes its sessions and then removes its routes. After
a crash they stay until the next start.
- **Interfaces.** The daemon learns the machine's interfaces and addresses
@@ -251,6 +253,33 @@ and two daemons that route for each other with pings across. From macOS,
`scripts/linux-vm.sh` runs it (or any command, such as `cargo test`) in a
Lima virtual machine that it creates on first use.
## Interoperability
`scripts/interop/lab.sh` runs the daemon against FRR, BIRD and Routinator,
each in a network namespace of its own on one exchange LAN, and checks what
every side ends up with. It needs Linux (`scripts/linux-vm.sh
scripts/interop/lab.sh` from macOS; `scripts/interop/install.sh` installs
the others on Ubuntu). Last run against FRR 10.5.1, BIRD 2.18 and
Routinator 0.15.2. What it covers:
| Phase | What is checked |
|-------|-----------------|
| `basics` | Sessions over IPv4 and IPv6; routes both ways with path, next hop, communities and large communities intact; four-octet AS numbers; ADD-PATH from BIRD; pings across by the installed routes |
| `roles` | RFC 9234 roles agree with FRR (peer) and BIRD (provider); Only to Customer is set where it should be, in both directions |
| `rpki` | RTR version 2 with Routinator; an invalid route from FRR is refused and held, and taken after Routinator restarts with data that allows it |
| `refresh` | Routes added and withdrawn by FRR; FRR's ROUTE-REFRESH answered; BIRD re-sending |
| `flaps` | Five session resets; FRR and BIRD killed and restarted; a cut wire noticed by the hold timer |
| `restart` | The daemon stopped in good order (kernel emptied, peers told) and killed (routes kept, stale one cleared by the next start) |
| `scale` | A table from BIRD (`SCALE_ROUTES`, a million for the Internet's size) into the table and the kernel; the daemon killed and restarted under it; the table withdrawn |
| `limits` | A peer whose role does not fit gets no session; a peer over its prefix limit gets Cease / Maximum Number of Prefixes and is held down |
`scripts/interop/real-rpki.sh` checks the RTR client against a Routinator
that has the real RPKI, ASPAs included: the daemon has to end up with as
many VRPs and ASPAs as the validator says it has.
Not covered: routers other than FRR and BIRD; IPv6 routes over an IPv4
session (the daemon does not send them); long runs.
## Finding a route
The status page has a look-up box over the received routes, and