ferrix/docs/content/docs/development/testing.md

+++ title = "Testing" description = "Describes the Ferrix test suites: unit and integration tests, property tests, kernel tests, the interoperability lab, fuzz testing, and benchmarks." weight = 2 +++

Ferrix is tested at several levels, from individual functions to networks of routers. Continuous integration runs all levels on every push and fails on any compiler warning. Table 1 summarizes the test suites.

Suite	Scope	Command
Unit and integration tests	All crates, and the server against in-process peers	cargo test --workspace
Property tests	Encoding and decoding of generated messages	Included in cargo test
Kernel tests	Route installation in a Linux kernel, in network namespaces	scripts/kernel-tests.sh
Interoperability lab	The daemon with FRR, BIRD, and Routinator	scripts/interop/lab.sh
Fuzz tests	All parsers of external input	cargo fuzz run <target>
Benchmarks	Route processing performance	cargo bench -p bgp-bench

Unit and Integration Tests

To run the unit and integration tests:

cargo test --workspace

The crates/integration-tests crate runs the server in-process against test peers that exchange BGP messages over sockets. It covers session establishment, import policy, RPKI validation, BGP roles, route advertisement, connection collisions, and prefix limits. The BGP finite state machine has no input or output, so its state transitions are tested with direct function calls.

Property Tests

The property tests verify that generated messages are decoded to the same value after encoding, and that no input causes a parser to panic. They run as part of cargo test in approximately 1 second. To run more cases:

PROPTEST_CASES=1000000 cargo test --release -p bgp-packet --test properties

Kernel Tests

The tests that modify a kernel routing table are not run by cargo test. The script scripts/kernel-tests.sh runs them on Linux, each in its own network namespace. The script scripts/in-namespace.sh runs a single command in the same way. The kernel tests include:

• Route installation and removal, verified with ip route.
• The daemon with a BGP peer.
• Two daemons that exchange routes, verified with ping (scripts/two-routers.sh).

On macOS, the script scripts/linux-vm.sh runs the kernel tests, or any other command, in a Lima virtual machine. The script creates the virtual machine on first use.

scripts/linux-vm.sh                  # kernel tests
scripts/linux-vm.sh cargo test       # all tests, on Linux
scripts/linux-vm.sh bash             # interactive shell

Interoperability Lab

The script scripts/interop/lab.sh tests the daemon with FRR, BIRD, and Routinator. Each router runs in its own network namespace, connected to a shared exchange LAN. Routinator serves RPKI data defined for the lab, and does not retrieve data from the Internet.

The lab requires Linux, sudo, and a release build of the daemon. The script scripts/interop/install.sh installs FRR, BIRD, radvd, and Routinator on Ubuntu.

cargo build --release -p bgp
scripts/interop/lab.sh                          # all phases
PHASES="basics transit" scripts/interop/lab.sh  # selected phases
scripts/interop/lab.sh up                       # set up the lab and leave it running

On macOS, prefix each command with scripts/linux-vm.sh. The lab was last run with FRR 10.5.1, BIRD 2.18, and Routinator 0.15.2. Table 2 lists the test phases.

Phase	Verified behavior
basics	IPv4 and IPv6 sessions; routes in both directions with AS path, next hop, communities, and large communities preserved; four-octet AS numbers; ADD-PATH from BIRD; forwarding with ping over the installed routes
roles	BGP role negotiation with FRR (peer) and BIRD (provider); OTC attribute added and verified in both directions
rpki	RTR version 2 with Routinator; an RPKI invalid route from FRR is rejected and held, and accepted after Routinator restarts with data that authorizes it
refresh	Routes added and withdrawn by FRR; ROUTE-REFRESH from FRR; route readvertisement by BIRD
flaps	Five session resets; FRR and BIRD terminated and restarted; link failure detected by the hold timer
restart	Daemon stopped normally (kernel routes removed, neighbors notified) and terminated (routes retained, stale route removed at the next start)
scale	A routing table from BIRD (SCALE_ROUTES; 1 million routes for a full table) installed in the routing table and the kernel; daemon terminated and restarted under load; table withdrawn
limits	Session rejected on role mismatch; Cease NOTIFICATION with Maximum Number of Prefixes Reached, and hold-down
hijacks	FRR advertises a local prefix, a route with BIRD as third-party next hop, and the exchange LAN prefix; all are rejected
nexthops	A preferred route from FRR with an unresolvable next hop is held, and the route from BIRD is selected and installed; the selection changes when the next hop becomes resolvable, and reverts when it becomes unresolvable
transit	FRR as customer and BIRD as provider: FRR's routes are advertised to BIRD with the local AS prepended; BIRD's routes are advertised to FRR with the OTC attribute; a control community prevents the advertisement of one FRR route to BIRD; traffic is forwarded between their networks
v6only	Exchange LAN without IPv4 addresses: one IPv6 session with each router for both address families; IPv4 routes with IPv6 next hops (RFC 8950) in BGP and in the kernel; IPv4 forwarding verified with ping
linklocal	Exchange LAN with link-local addresses only, with the same addresses on a second interface: sessions with FRR (by interface, with radvd) and BIRD at fe80::…%ix0; both address families in BGP and in the kernel; forwarding verified with ping; interface disabled and enabled

The script scripts/interop/real-rpki.sh tests the RTR client against a Routinator instance with the global RPKI data, including ASPAs. The test verifies that the daemon receives the same number of VRPs and ASPAs that the validator reports.

The lab does not cover routers other than FRR and BIRD, IPv6 routes over IPv4 sessions (which Ferrix does not advertise), an interface that is deleted and recreated during a session, or long-running operation.

Fuzz Testing

The fuzz/ directory contains coverage-guided fuzz targets for all parsers of external input. Table 3 lists the targets.

Target	Input
message	Raw bytes received on a BGP connection
message_body	A message body with a valid header, to reach the body parsers
attrs	Path attribute bytes stored in the routing table
prefix_text	A prefix in text form
config	The configuration file
rtr	A stream of RTR PDUs from an RPKI cache
netlink	Messages from the kernel routing socket

For the message and message_body targets, a failure is either a panic or a message that is decoded but cannot be encoded and decoded back to the same value.

Fuzz testing requires a nightly toolchain and cargo-fuzz:

rustup toolchain install nightly
cargo install cargo-fuzz
cd fuzz
cargo +nightly fuzz run message_body corpus/message_body seeds/message_body -- -max_total_time=300

The first directory is the working corpus, which is not stored in the repository. The seeds/ directory contains valid inputs. Continuous integration runs each target for 1 minute.

When a target fails, the input is saved in artifacts/<target>/. To reproduce the failure, run cargo +nightly fuzz run <target> <file>. After you fix the defect, add a unit test with the input next to the affected code.

Benchmarks

Table 4 lists the benchmarks.

Benchmark	Measurement
full_table	Processing of a full routing table, from received bytes to best path selection. BENCH_PEERS sets the number of neighbors (default: 4); BENCH_SCALE_DOWN reduces the table size.
transit	Route advertisement: one neighbor advertises the table, and a second neighbor receives every route. The time is measured from the first byte sent to the last route received.
real_table	The full_table measurement on a RIPE RIS routing table dump. See crates/bench/data/README.md for how to obtain a dump.

cargo bench -p bgp-bench --bench full_table
cargo bench -p bgp-bench --bench transit
cargo bench -p bgp-bench --bench real_table

The full_table and transit benchmarks use a synthetic routing table, so that they require no data files and produce reproducible results. The table has the characteristics of the Internet routing table that affect performance: approximately 1 million IPv4 prefixes, mostly /24, and 250,000 IPv6 prefixes, mostly /48; prefixes with shared attributes grouped in UPDATE messages; AS paths of several hops; and communities on most routes.

Table 5 lists results on an Apple M-series computer with a release build.

Measurement	Result
1.23 million routes from one neighbor, accepted into the routing table	0.47 seconds
The same routes received by a second neighbor (transit)	2.1 seconds, approximately 4 routes per UPDATE message
1 million IPv4 routes installed in the kernel (Linux virtual machine)	Slightly more than 1 second
The same routes removed from the kernel	0.5 seconds