ferrix/docs/content/docs/development/testing.md at 99a47da2af73918fe1cd7c6aa70048fcaf5e4a04

0 Fork 0

Files

T

Rayhaan JaufeerallyandClaude Fable 5.1 ec5c9be454 export: advertise every route for a prefix with ADD-PATH (all_paths)

Before this change, a session on which ADD-PATH (RFC 7911) was
negotiated in the sending direction was sent path identifiers, but only
the best route for each prefix. A route collector such as bgp.tools
wants every route a router has accepted, from every neighbor, and from
one session: without this the collector saw one path per prefix, nearly
always the same neighbor's.

This change adds `all_paths` to the export policy. On a session with it,
and ADD-PATH send negotiated for a family, every accepted route for a
prefix of that family is advertised, each with a path identifier of its
own; for a family without ADD-PATH send the best route goes as before.
`all_paths` needs `learned_routes` (a configuration error otherwise) and
the peer's `add_path` to offer sending. The export policy, the role
rules, the well-known communities and the export limits apply to every
route alike, and the export limits count every route.

Implementation, in the advertiser:

* An exporter with `all_paths` follows the table's journal of all routes
  (`RibTable::reader_of_all_routes`, until now unused) instead of the
  best-route journal, and is given all the routes of a prefix whenever
  one of them changes (`News::Routes`).
* What the peer holds is `Announced`: the set of prefixes for a
  best-route session, as before; or for an all-paths session, for each
  prefix the routes announced, keyed by the peer they came from and that
  peer's own identifier, with the identifier they went out with and the
  form they went out in. A route keeps its identifier for as long as it
  is announced, goes out again only if it changed, and is withdrawn by
  its identifier; a new route gets the lowest identifier not in use for
  the prefix, from 1.
* Verdicts carry either kind (`Verdict::Best`, `Verdict::All`), and a
  reset of the table puts what was announced in doubt until heard of
  again, as before, with the same identifiers kept across the reset.
* The record for an all-paths session is about 50 bytes a route, some
  tens of megabytes for a full table from several neighbors; the
  best-route bookkeeping is unchanged in size.

Also, so that a collector session can be watched: the number of routes
advertised to each peer and not withdrawn since, every route counted
where several are sent for a prefix. The exporter keeps it in the
session's counters after every change; it is `advertised_routes` in
`/api/peers`, the "sent" column of the status page's neighbor table,
and the gauge `bgp_peer_advertised_routes{peer}`.

Documentation: a "Feeding a Route Collector" section in the guide to
passing routes on, with the configuration for a collector; `all_paths`
in the configuration reference; the ADD-PATH sections of the neighbor
guide and the standards reference; the limitation that was removed;
the new field, column and metric in the HTTP API, monitoring and
metrics references.

Testing: an integration test with two feeders and a collector that
negotiated ADD-PATH receive, which checks the two routes with their
identifiers, the withdrawal of one by its identifier, a changed route
keeping its identifier and a new route taking the free one; a
configuration test for `all_paths` without `learned_routes`; a new
`collector` phase of the interoperability lab in which FRR, as a route
collector, is told both of BIRD's routes for a prefix from BIRD's two
sessions and holds them apart, and loses one when its session is
disabled, with the advertised count checked at each step. The workspace tests, the kernel tests and every lab phase pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-10-09 01:16:48 +02:00

13 KiB

Raw Blame History

+++ title = "Testing" description = "Describes the Ferrix test suites: unit and integration tests, property tests, kernel tests, the interoperability lab, fuzz testing, and benchmarks." weight = 2 +++

Ferrix is tested at several levels, from individual functions to networks of routers. Continuous integration runs all levels on every push and fails on any compiler warning. Table 1 summarizes the test suites.

Suite	Scope	Command
Unit and integration tests	All crates, and the server against in-process peers	`cargo test --workspace`
Property tests	Encoding and decoding of generated messages	Included in `cargo test`
Kernel tests	Route installation in a Linux kernel, in network namespaces	`scripts/kernel-tests.sh`
Interoperability lab	The daemon with FRR, BIRD, and Routinator	`scripts/interop/lab.sh`
Fuzz tests	All parsers of external input	`cargo fuzz run <target>`
Benchmarks	Route processing performance	`cargo bench -p bgp-bench`

Unit and Integration Tests

To run the unit and integration tests:

cargo test --workspace

The crates/integration-tests crate runs the server in-process against test peers that exchange BGP messages over sockets. It covers session establishment, import policy, RPKI validation, BGP roles, route advertisement, connection collisions, and prefix limits. The BGP finite state machine has no input or output, so its state transitions are tested with direct function calls.

Property Tests

The property tests verify that generated messages are decoded to the same value after encoding, and that no input causes a parser to panic. They run as part of cargo test in approximately 1 second. To run more cases:

PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties

Kernel Tests

The tests that modify a kernel routing table are not run by cargo test. The script scripts/kernel-tests.sh runs them on Linux, each in its own network namespace. The script scripts/in-namespace.sh runs a single command in the same way. The kernel tests include:

Route installation and removal, verified with ip route.
The daemon with a BGP peer.
Two daemons that exchange routes, verified with ping (scripts/two-routers.sh).

On macOS, the script scripts/linux-vm.sh runs the kernel tests, or any other command, in a Lima virtual machine. The script creates the virtual machine on first use.

scripts/linux-vm.sh                  # kernel tests
scripts/linux-vm.sh cargo test       # all tests, on Linux
scripts/linux-vm.sh bash             # interactive shell

Interoperability Lab

The script scripts/interop/lab.sh tests the daemon with FRR, BIRD, and Routinator. Each router runs in its own network namespace, connected to a shared exchange LAN. Routinator serves RPKI data defined for the lab, and does not retrieve data from the Internet.

The lab requires Linux, sudo, and a release build of the daemon. The script scripts/interop/install.sh installs FRR, BIRD, radvd, and Routinator on Ubuntu.

cargo build --release -p bgp
scripts/interop/lab.sh                          # all phases
PHASES="basics transit" scripts/interop/lab.sh  # selected phases
scripts/interop/lab.sh up                       # set up the lab and leave it running

On macOS, prefix each command with scripts/linux-vm.sh. The lab was last run with FRR 10.5.1, BIRD 2.18, and Routinator 0.15.2. Table 2 lists the test phases.

Phase	Verified behavior
`basics`	IPv4 and IPv6 sessions; routes in both directions with AS path, next hop, communities, and large communities preserved; four-octet AS numbers; ADD-PATH from BIRD; forwarding with `ping` over the installed routes
`roles`	BGP role negotiation with FRR (peer) and BIRD (provider); OTC attribute added and verified in both directions
`rpki`	RTR version 2 with Routinator; an RPKI invalid route from FRR is rejected and held, and accepted after Routinator restarts with data that authorizes it
`refresh`	Routes added and withdrawn by FRR; ROUTE-REFRESH from FRR; route readvertisement by BIRD
`flaps`	Five session resets; FRR and BIRD terminated and restarted; link failure detected by the hold timer
`restart`	Daemon stopped normally (kernel routes removed, neighbors notified) and terminated (routes retained, stale route removed at the next start)
`scale`	A routing table from BIRD (`SCALE_ROUTES`; 1 million routes for a full table) installed in the routing table and the kernel; daemon terminated and restarted under load; table withdrawn
`limits`	Session rejected on role mismatch; Cease NOTIFICATION with Maximum Number of Prefixes Reached, and hold-down
`hijacks`	FRR advertises a local prefix, a route with BIRD as third-party next hop, and the exchange LAN prefix; all are rejected
`nexthops`	A preferred route from FRR with an unresolvable next hop is held, and the route from BIRD is selected and installed; the selection changes when the next hop becomes resolvable, and reverts when it becomes unresolvable
`transit`	FRR as customer and BIRD as provider: FRR's routes are advertised to BIRD with the local AS prepended; BIRD's routes are advertised to FRR with the OTC attribute; a control community prevents the advertisement of one FRR route to BIRD; traffic is forwarded between their networks
`collector`	FRR as a route collector with ADD-PATH and `all_paths`: BIRD peers twice, FRR receives both of BIRD's routes for a prefix with distinct path identifiers, and one is withdrawn by its identifier when its session is disabled
`v6only`	Exchange LAN without IPv4 addresses: one IPv6 session with each router for both address families; IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel; IPv4 forwarding verified with `ping`
`linklocal`	Exchange LAN with link-local addresses only, with the same addresses on a second interface: sessions with FRR (by interface, with radvd) and BIRD at `fe80::…%ix0`; both address families in BGP and in the kernel; forwarding verified with `ping`; interface disabled and enabled

The script scripts/interop/real-rpki.sh tests the RTR client against a Routinator instance with the global RPKI data, including ASPAs. The test verifies that the daemon receives the same number of VRPs and ASPAs that the validator reports.

The lab does not cover routers other than FRR and BIRD, IPv6 routes over IPv4 sessions (which Ferrix does not advertise), an interface that is deleted and recreated during a session, or long-running operation.

Fuzz Testing

The fuzz/ directory contains coverage-guided fuzz targets for all parsers of external input. Table 3 lists the targets.

Target	Input
`message`	Raw bytes received on a BGP connection
`message_body`	A message body with a valid header, to reach the body parsers
`attrs`	Path attribute bytes stored in the routing table
`prefix_text`	A prefix in text form
`config`	The configuration file
`rtr`	A stream of RTR PDUs from an RPKI cache
`netlink`	Messages from the kernel routing socket

For the message and message_body targets, a failure is either a panic or a message that is decoded but cannot be encoded and decoded back to the same value.

Fuzz testing requires a nightly toolchain and cargo-fuzz:

rustup toolchain install nightly
cargo install cargo-fuzz
cd fuzz
cargo +nightly fuzz run message_body corpus/message_body seeds/message_body -- -max_total_time=300

The first directory is the working corpus, which is not stored in the repository. The seeds/ directory contains valid inputs. Continuous integration runs each target for 1 minute.

When a target fails, the input is saved in artifacts/<target>/. To reproduce the failure, run cargo +nightly fuzz run <target> <file>. After you fix the defect, add a unit test with the input next to the affected code.

Benchmarks

Table 4 lists the benchmarks.

Benchmark	Measurement
`full_table`	Processing of a full routing table, from received bytes to best path selection. `BENCH_PEERS` sets the number of neighbors (default: 4); `BENCH_SCALE_DOWN` reduces the table size.
`transit`	Route advertisement: one neighbor advertises the table, and a second neighbor receives every route. The time is measured from the first byte sent to the last route received.
`real_table`	The `full_table` measurement on a RIPE RIS routing table dump. See `crates/bench/data/README.md` for how to obtain a dump.

cargo bench -p bgp-bench --bench full_table
cargo bench -p bgp-bench --bench transit
cargo bench -p bgp-bench --bench real_table

The full_table and transit benchmarks use a synthetic routing table, so that they require no data files and produce reproducible results. The table has the characteristics of the Internet routing table that affect performance: approximately 1 million IPv4 prefixes, mostly /24, and 250,000 IPv6 prefixes, mostly /48; prefixes with shared attributes grouped in UPDATE messages; AS paths of several hops; and communities on most routes.

Table 5 lists full_table results on an Apple M-series computer with a release build: 1.23 million prefixes from each of four neighbors, 4.92 million routes.

Measurement	Result
Decoding the UPDATE messages, one thread	0.43 seconds, 11 million routes per second
Decoding, interning attributes, inserting and selecting, one thread	1.9 seconds, 2.5 million routes per second
The same, 12 threads	0.7 seconds, 6.9 million routes per second
Four neighbors send the table over TCP to the in-process server	0.9 seconds, 5.2 million routes per second
Memory per route, attribute sets included (`--features count-memory`)	77 bytes

Profiling

The scripts/profile/full-table.sh script measures the daemon itself, as built for production, while one or more neighbors send it a full routing table, and records where its CPU time goes. It requires Linux with perf, inferno (cargo install inferno), curl, jq, and sudo. On another operating system, run it in the Linux virtual machine:

scripts/linux-vm.sh scripts/profile/full-table.sh

The script builds the daemon with the profiling Cargo profile, which is the release profile with line tables, and with frame pointers, so that perf can attribute samples to functions. It starts the daemon in a network namespace of its own, connected by a virtual Ethernet pair to a second namespace in which the load generator (crates/bench/examples/feed.rs) connects as the neighbors and sends the synthetic table. Routes that the daemon installs go into the first namespace's routing table and do not affect the host. Table 6 lists the environment variables that control the run.

Variable	Effect	Default
`PEERS`	The number of neighbors that send the table	1
`KERNEL`	`1` installs the best routes in the kernel; `0` keeps them in the routing table only	1
`SCALE_DOWN`	Divides the size of the table	1
`HOLD_SECS`	How long the sessions stay open after the table is sent	30
`FREQ`	The sampling frequency in hertz	1999
`OUT`	The directory for the results	`~/profile`
`JEMALLOC_CONF`	jemalloc options for the daemon (its `MALLOC_CONF`), for example `thp:always`	none

When run through scripts/linux-vm.sh, pass the variables with env, because the virtual machine does not receive the host's environment:

scripts/linux-vm.sh env PEERS=4 KERNEL=0 scripts/profile/full-table.sh

The script prints how long the routes take to arrive in the routing table and in the kernel, and the daemon's resident memory. It writes the following files to the results directory:

File	Contents
`flame.svg`	An interactive flame graph of the daemon's threads
`top-self.txt`	Functions by the time spent in their own code
`top-children.txt`	Functions by the time spent in them and everything they call
`stacks.folded`	The collapsed stacks the flame graph is drawn from
`perf.data`, `daemon.log`, `feed.log`	The raw samples and the logs

Table 7 lists results in the Linux virtual machine (8 virtual CPUs on an Apple M-series computer) with the synthetic table of 1.23 million prefixes.

Measurement	Result
One neighbor; routes in the routing table	0.8 seconds, 1.5 million routes per second
One neighbor; the best routes installed in the kernel	2.0 seconds after the first route
One neighbor, without the kernel; routes in the routing table	0.6 seconds, 2.0 million routes per second
Four neighbors; 4.92 million routes in the routing table	1.1 seconds, 4.4 million routes per second
Four neighbors; the best routes installed in the kernel	2.2 seconds after the first route
Four neighbors; routes the kernel was asked to install	1.25 million, for 1.23 million prefixes
Resident memory with one neighbor's table installed	440 MB
Resident memory with four neighbors' tables installed	630 MB

A session is processed on one thread, so one neighbor's table arrives at the rate of one CPU core, and several neighbors' tables arrive in parallel. Installing routes in the kernel takes longer than receiving them, and continues after the last route has arrived.

13 KiB Raw Blame History