Install best routes in the Linux kernel over netlink
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Rayhaan JaufeerallyandClaude Fable 5.1 committed
1 parent
1f84235e87
commit
1fc44798b7
38 files changed
+3745
-16
No files matched your search
Executable
+38
@@ -0,0 +1,38 @@
|
||||
#!/bin/sh
|
||||
# Runs the tests that need a real Linux kernel to change the routes of:
|
||||
# those of the bgp-fib crate that `cargo test` skips, and the end-to-end
|
||||
# test of two daemons that route for each other.
|
||||
#
|
||||
# They run in a network namespace made for the purpose, so the machine's
|
||||
# own interfaces and routes are out of their reach, and with the one
|
||||
# capability the daemon needs, CAP_NET_ADMIN, rather than as root.
|
||||
#
|
||||
# Run it on Linux, as a user who can sudo. On another system,
|
||||
# scripts/linux-vm.sh runs it in a virtual machine.
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
[ "$(uname -s)" = Linux ] || { echo "this needs Linux; see scripts/linux-vm.sh" >&2; exit 1; }
|
||||
|
||||
# Built as the user, so that nothing in the target directory is root's.
|
||||
cargo build --locked -p bgp
|
||||
cargo test --locked -p bgp-fib --test kernel --no-run
|
||||
cargo test --locked -p bgp-integration-tests --test kernel --no-run
|
||||
|
||||
namespace="ferrix-test-$$"
|
||||
sudo modprobe dummy 2>/dev/null || true
|
||||
sudo ip netns add "$namespace"
|
||||
trap 'sudo ip netns del "$namespace"' EXIT
|
||||
sudo ip netns exec "$namespace" ip link set lo up
|
||||
|
||||
# In the namespace, as the user, with CAP_NET_ADMIN and nothing else.
|
||||
in_namespace() {
|
||||
sudo ip netns exec "$namespace" \
|
||||
setpriv --reuid "$(id -u)" --regid "$(id -g)" --init-groups \
|
||||
--inh-caps +net_admin --ambient-caps +net_admin \
|
||||
env PATH="$PATH" HOME="$HOME" CARGO_TARGET_DIR="${CARGO_TARGET_DIR:-}" "$@"
|
||||
}
|
||||
|
||||
in_namespace cargo test --locked -p bgp-fib --test kernel -- --include-ignored --nocapture "$@"
|
||||
in_namespace cargo test --locked -p bgp-integration-tests --test kernel -- --include-ignored --nocapture "$@"
|
||||
scripts/two-routers.sh
|
||||
Executable
+34
@@ -0,0 +1,34 @@
|
||||
#!/bin/sh
|
||||
# Runs a command in a Linux virtual machine with this repository mounted
|
||||
# at the same path: for the tests that need a Linux kernel, from a machine
|
||||
# that is not one. Without arguments it runs scripts/kernel-tests.sh.
|
||||
#
|
||||
# scripts/linux-vm.sh the kernel tests
|
||||
# scripts/linux-vm.sh cargo test the whole test suite, on Linux
|
||||
# scripts/linux-vm.sh bash a shell
|
||||
#
|
||||
# The machine is made on first use with Lima (https://lima-vm.io; on macOS
|
||||
# `brew install lima`): Ubuntu, with a Rust toolchain. It sees the
|
||||
# repository read-only and builds into a directory of its own.
|
||||
# `limactl stop ferrix` stops it and `limactl delete ferrix` removes it.
|
||||
set -eu
|
||||
|
||||
vm=ferrix
|
||||
repository="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
command -v limactl >/dev/null || { echo "limactl not found: install Lima first" >&2; exit 1; }
|
||||
|
||||
if ! limactl list --quiet | grep -qx "$vm"; then
|
||||
limactl start --name="$vm" --cpus=8 --memory=8 --disk=30 --tty=false template://ubuntu-lts
|
||||
limactl shell "$vm" bash -c '
|
||||
set -eu
|
||||
sudo apt-get update -qq
|
||||
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq build-essential iproute2 iputils-ping curl
|
||||
curl --proto "=https" --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal -q
|
||||
'
|
||||
elif [ "$(limactl list --format '{{.Status}}' "$vm")" != Running ]; then
|
||||
limactl start "$vm"
|
||||
fi
|
||||
|
||||
[ $# -gt 0 ] || set -- scripts/kernel-tests.sh
|
||||
limactl shell --workdir "$repository" "$vm" \
|
||||
bash -lc 'export CARGO_TARGET_DIR="$HOME/target"; exec "$@"' -- "$@"
|
||||
Executable
+144
@@ -0,0 +1,144 @@
|
||||
#!/bin/sh
|
||||
# Two daemons that route for each other, on a real kernel: the proof that
|
||||
# what BGP learns ends up as forwarding.
|
||||
#
|
||||
# namespace a namespace b
|
||||
# lan 10.1.0.1/24 lan 10.2.0.1/24
|
||||
# 2001:db8:a::1/48 2001:db8:b::1/48
|
||||
# a0 10.0.0.1/24 ------ veth ------ b0 10.0.0.2/24
|
||||
# 2001:db8::1/64 2001:db8::2/64
|
||||
#
|
||||
# Each originates its own "lan" prefixes and installs what it learns in
|
||||
# its kernel. The test passes when each kernel has the other's prefixes as
|
||||
# BGP routes and a ping from one lan address to the other gets through,
|
||||
# over IPv4 and IPv6; and when the routes go again as a daemon is stopped.
|
||||
#
|
||||
# Run it on Linux as a user who can sudo, after `cargo build -p bgp`;
|
||||
# scripts/kernel-tests.sh does both.
|
||||
set -eu
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
bgp="${CARGO_TARGET_DIR:-target}/debug/bgp"
|
||||
[ -x "$bgp" ] || { echo "$bgp not found: cargo build -p bgp" >&2; exit 1; }
|
||||
|
||||
work="$(mktemp -d)"
|
||||
a="ferrix-a-$$"
|
||||
b="ferrix-b-$$"
|
||||
pids=""
|
||||
cleanup() {
|
||||
status=$?
|
||||
for pid in $pids; do sudo kill "$pid" 2>/dev/null || true; done
|
||||
sudo ip netns del "$a" 2>/dev/null || true
|
||||
sudo ip netns del "$b" 2>/dev/null || true
|
||||
if [ "$status" -ne 0 ]; then
|
||||
echo "--- FAILED; the daemons said:" >&2
|
||||
tail -n 30 "$work"/*.log >&2 || true
|
||||
fi
|
||||
rm -rf "$work"
|
||||
exit "$status"
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
in_a() { sudo ip netns exec "$a" "$@"; }
|
||||
in_b() { sudo ip netns exec "$b" "$@"; }
|
||||
|
||||
sudo modprobe dummy 2>/dev/null || true
|
||||
sudo ip netns add "$a"
|
||||
sudo ip netns add "$b"
|
||||
sudo ip link add a0 netns "$a" type veth peer name b0 netns "$b"
|
||||
for side in a b; do
|
||||
case $side in a) n=1; run=in_a ;; b) n=2; run=in_b ;; esac
|
||||
$run ip link set lo up
|
||||
$run ip addr add "10.0.0.$n/24" dev "${side}0"
|
||||
$run ip -6 addr add "2001:db8::$n/64" dev "${side}0" nodad
|
||||
$run ip link set "${side}0" up
|
||||
$run ip link add lan type dummy
|
||||
$run ip addr add "10.$n.0.1/24" dev lan
|
||||
$run ip -6 addr add "2001:db8:$side::1/48" dev lan nodad
|
||||
$run ip link set lan up
|
||||
done
|
||||
|
||||
# One session for each IP version: routes are sent with the session's own
|
||||
# address as next hop, which has to be of the routes' version.
|
||||
config() {
|
||||
side=$1; n=$2; other=$3
|
||||
cat > "$work/$side.textpb" <<END
|
||||
router_id: "10.0.0.$n"
|
||||
local_asn: 6500$n
|
||||
listen_endpoints { address: "10.0.0.$n" port: 179 }
|
||||
listen_endpoints { address: "2001:db8::$n" port: 179 }
|
||||
peers {
|
||||
name: "v4" endpoint { address: "10.0.0.$other" } remote_asn: 6500$other
|
||||
address_families: IPV4_UNICAST
|
||||
import { accept_bogons: true max_prefixes: 100 }
|
||||
}
|
||||
peers {
|
||||
name: "v6" endpoint { address: "2001:db8::$other" } remote_asn: 6500$other
|
||||
address_families: IPV6_UNICAST
|
||||
import { accept_bogons: true max_prefixes: 100 }
|
||||
}
|
||||
connect_retry_seconds: 1
|
||||
originate { prefix: "10.$n.0.0/24" }
|
||||
originate { prefix: "2001:db8:$side::/48" }
|
||||
kernel {}
|
||||
END
|
||||
}
|
||||
config a 1 2
|
||||
config b 2 1
|
||||
|
||||
# As the user, with what the daemon needs and no more: CAP_NET_ADMIN for
|
||||
# the routes, CAP_NET_BIND_SERVICE for port 179.
|
||||
start() {
|
||||
namespace=$1; side=$2
|
||||
sudo ip netns exec "$namespace" \
|
||||
setpriv --reuid "$(id -u)" --regid "$(id -g)" --init-groups \
|
||||
--inh-caps +net_admin,+net_bind_service --ambient-caps +net_admin,+net_bind_service \
|
||||
"$bgp" "$work/$side.textpb" > "$work/$side.log" 2>&1 &
|
||||
pids="$pids $!"
|
||||
}
|
||||
start "$a" a
|
||||
pid_a=$!
|
||||
start "$b" b
|
||||
|
||||
# Waits until `ip route` in a namespace shows, or no longer shows, a route.
|
||||
expect() {
|
||||
run=$1; family=$2; want=$3; route=$4
|
||||
for _ in $(seq 100); do
|
||||
if $run ip "$family" route show proto bgp | sed 's/ */ /g; s/ pref medium//; s/ *$//' | grep -qxF "$route"; then
|
||||
[ "$want" = present ] && return 0
|
||||
else
|
||||
[ "$want" = absent ] && return 0
|
||||
fi
|
||||
sleep 0.2
|
||||
done
|
||||
echo "expected the route \"$route\" to be $want; the kernel's BGP routes are:" >&2
|
||||
$run ip "$family" route show proto bgp >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
expect in_a -4 present "10.2.0.0/24 via 10.0.0.2 dev a0 metric 20"
|
||||
expect in_b -4 present "10.1.0.0/24 via 10.0.0.1 dev b0 metric 20"
|
||||
expect in_a -6 present "2001:db8:b::/48 via 2001:db8::2 dev a0 metric 20"
|
||||
expect in_b -6 present "2001:db8:a::/48 via 2001:db8::1 dev b0 metric 20"
|
||||
echo "ok: each kernel has the other's prefixes as BGP routes"
|
||||
|
||||
# From one lan address to the other: there and back by the routes BGP put
|
||||
# there.
|
||||
in_a ping -q -c 2 -W 2 -I 10.1.0.1 10.2.0.1 > /dev/null
|
||||
in_b ping -q -c 2 -W 2 -I 10.2.0.1 10.1.0.1 > /dev/null
|
||||
in_a ping -q -c 2 -W 2 -I 2001:db8:a::1 2001:db8:b::1 > /dev/null
|
||||
echo "ok: packets get through, over IPv4 and IPv6"
|
||||
|
||||
# One daemon is stopped in good order. It takes its routes out of its
|
||||
# kernel; the other, told the session is over, does the same with what it
|
||||
# learned.
|
||||
sudo ip netns pids "$a" | xargs sudo kill -TERM
|
||||
expect in_a -4 absent "10.2.0.0/24 via 10.0.0.2 dev a0 metric 20"
|
||||
expect in_a -6 absent "2001:db8:b::/48 via 2001:db8::2 dev a0 metric 20"
|
||||
expect in_b -4 absent "10.1.0.0/24 via 10.0.0.1 dev b0 metric 20"
|
||||
expect in_b -6 absent "2001:db8:a::/48 via 2001:db8::1 dev b0 metric 20"
|
||||
if in_b ping -q -c 1 -W 1 -I 10.2.0.1 10.1.0.1 > /dev/null 2>&1; then
|
||||
echo "packets still get through with the routes gone" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "ok: the routes go when a daemon stops"
|
||||
Reference in new issue
Block a user