Rayhaan JaufeerallyandClaude Fable 5.1
6d8f89e509
Before this change, a route carrying the BLACKHOLE community (65535:666,
RFC 7999) from an EBGP neighbor was always rejected with the reason
"blackhole request", and nothing discarded traffic. A customer under a
denial-of-service attack had no way to ask this router to drop traffic
to an address.
This change adds `accept_blackholes` to the import policy. When it is
set, a route with the BLACKHOLE community is accepted if:
* its prefix is a single address, a /32 or a /128 (otherwise it is
rejected with the new reason "blackhole for more than one address");
the `max_ipv4_prefix_length` and `max_ipv6_prefix_length` limits do
not apply, as RFC 7999 section 3.2 recommends;
* the address is inside a prefix the neighbor may announce, if
`prefixes` is configured, regardless of the entry's `max_length`
(new `PrefixSet::covers`); without a prefix list, an address inside
the local AS's own prefixes is rejected as before ("prefix is our
own");
* the AS path, origin and RPKI checks that apply to every route pass.
The next hop of an accepted blackhole route is not checked for
third-party addresses and is not resolved, because traffic is never
forwarded to it. The route takes part in best path selection as usual;
a host route is normally the active route for its address.
The kernel task installs the active route as a kernel blackhole route
(`RTN_BLACKHOLE`, shown by `ip route` as `blackhole PREFIX proto bgp`).
Blackhole routes left by an earlier run are adopted at startup like
other routes of ours. The route is advertised to other neighbors as the
export policy and export limits permit, with the community intact, so a
provider that honors BLACKHOLE can discard the traffic upstream.
Implementation:
* `plan::Target` becomes an enum: `Via { gateway, interface }` or
`Discard`. `plan::resolve` takes a `blackhole` flag and returns
`Discard` for such routes without looking at the next hop (the
own-network check still applies). No nexthop object is made for a
discard target.
* `AttrSet::is_blackhole` reads the community off the stored attribute
bytes, like `next_hop` and `as_path`, so the kernel task, the
revalidator and the RPKI revalidation can ask it of every route.
* `ImportPolicy::is_blackhole_request` and `check_blackhole_prefix`
carry the import side; the import flow (`routes.rs`) computes once per
UPDATE whether its routes are an accepted blackhole request and skips
the next hop checks for them.
* `Status::blackholes` and the `blackholes` field of `/api/kernel` count
the installed blackhole routes; the status page shows the count next
to the installed routes.
Not covered: a prefix this router originates with the BLACKHOLE
community is advertised but not installed as a blackhole route. The
documentation says to add a kernel blackhole route by hand for that
case.
Documentation: the "Blackhole Routes" section of the import policy
guide replaces "Blackhole Requests", the kernel guide gains a "Blackhole
Routes" section, the configuration reference describes
`accept_blackholes`, the HTTP API reference describes `blackholes`, and
the RFC 7999 rows of the standards reference are updated.
Testing: unit tests for the attribute check, the prefix checks and the
plan; integration tests that a blackhole request is refused by default
and, with `accept_blackholes`, taken for one address inside the peer's
space and refused otherwise, with the recorded reasons; a kernel test
that a blackhole request becomes a kernel blackhole route whatever its
next hop, is removed on withdrawal and is adopted after a restart. The
existing kernel tests, the interoperability lab and the workspace tests
pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
911 lines
34 KiB
Rust
911 lines
34 KiB
Rust
//! The kernel's routes against a real kernel: what `ip route` shows after
|
|
//! the routing table changes.
|
|
//!
|
|
//! These change the machine's interfaces and routes, so they are not run
|
|
//! by `cargo test`. `scripts/kernel-tests.sh` runs them in a network
|
|
//! namespace of their own, with the capability they need
|
|
//! (CAP_NET_ADMIN) and nothing more.
|
|
#![cfg(target_os = "linux")]
|
|
|
|
use std::collections::BTreeSet;
|
|
use std::net::{IpAddr, Ipv4Addr};
|
|
use std::process::Command;
|
|
use std::sync::Arc;
|
|
use std::time::{Duration, Instant};
|
|
|
|
use bgp_fib::{Fib, FibConfig, FibError};
|
|
use bgp_packet::message::{
|
|
AsPathAttribute, CommunitiesPathAttribute, MultiExitDiscPathAttribute, NextHopPathAttribute,
|
|
NlriNextHop, OriginPathAttribute, PathAttribute,
|
|
};
|
|
use bgp_primitives::ip_prefix::IpPrefix;
|
|
use bgp_rib::table::{AttrSet, PeerIndex, PeerInfo, RibTable};
|
|
use tokio::task::JoinHandle;
|
|
|
|
const TIMEOUT: Duration = Duration::from_secs(10);
|
|
|
|
fn prefix(s: &str) -> IpPrefix {
|
|
IpPrefix::try_from(s).unwrap()
|
|
}
|
|
|
|
/// Runs `ip` and returns what it printed. Panics if it fails.
|
|
#[track_caller]
|
|
fn ip(arguments: &str) -> String {
|
|
let output = Command::new("ip")
|
|
.args(arguments.split_whitespace())
|
|
.output()
|
|
.expect("the ip command is there");
|
|
assert!(
|
|
output.status.success(),
|
|
"ip {arguments}: {}",
|
|
String::from_utf8_lossy(&output.stderr)
|
|
);
|
|
String::from_utf8_lossy(&output.stdout).into_owned()
|
|
}
|
|
|
|
/// What a test has of the machine: an interface of its own with
|
|
/// 10.`n`.0.1/24 and 2001:db8:`n`::1/64 on it, and a routing table of its
|
|
/// own. Tests run side by side in one network namespace, and `n` keeps
|
|
/// them apart.
|
|
struct Machine {
|
|
n: u8,
|
|
rib: Arc<RibTable>,
|
|
/// Whether routes name nexthop objects, which is how it is unless a
|
|
/// test says otherwise.
|
|
nexthop_objects: bool,
|
|
/// How long routes of an earlier run are kept: not at all, unless a
|
|
/// test says otherwise.
|
|
stale_time: Duration,
|
|
}
|
|
|
|
impl Machine {
|
|
fn new(n: u8) -> Self {
|
|
let machine = Self {
|
|
n,
|
|
rib: Arc::new(RibTable::new(4)),
|
|
nexthop_objects: true,
|
|
stale_time: Duration::ZERO,
|
|
};
|
|
ip(&format!("link add {} type dummy", machine.interface()));
|
|
machine.add_addresses();
|
|
ip(&format!("link set {} up", machine.interface()));
|
|
machine
|
|
}
|
|
|
|
fn interface(&self) -> String {
|
|
format!("ferrix{}", self.n)
|
|
}
|
|
|
|
fn add_addresses(&self) {
|
|
let (n, interface) = (self.n, self.interface());
|
|
ip(&format!("addr add 10.{n}.0.1/24 dev {interface}"));
|
|
ip(&format!(
|
|
"-6 addr add 2001:db8:{n}::1/64 dev {interface} nodad"
|
|
));
|
|
}
|
|
|
|
fn add_addresses_v4(&self) {
|
|
ip(&format!(
|
|
"addr add 10.{}.0.1/24 dev {}",
|
|
self.n,
|
|
self.interface()
|
|
));
|
|
}
|
|
|
|
fn table(&self) -> u32 {
|
|
1000 + u32::from(self.n)
|
|
}
|
|
|
|
/// A neighbor on the test's interface.
|
|
fn neighbor(&self, host: u8) -> Ipv4Addr {
|
|
Ipv4Addr::new(10, self.n, 0, host)
|
|
}
|
|
|
|
fn start(&self) -> (Fib, JoinHandle<Result<(), FibError>>) {
|
|
Fib::spawn(
|
|
Arc::clone(&self.rib),
|
|
FibConfig {
|
|
table: self.table(),
|
|
metric: 20,
|
|
nexthop_objects: self.nexthop_objects,
|
|
stale_time: self.stale_time,
|
|
..FibConfig::default()
|
|
},
|
|
)
|
|
.expect("the kernel can be talked to")
|
|
}
|
|
|
|
fn peer(&self, host: u8) -> PeerIndex {
|
|
self.rib
|
|
.add_peer(PeerInfo {
|
|
asn: 65000 + u32::from(host),
|
|
router_id: self.neighbor(host),
|
|
address: self.neighbor(host).into(),
|
|
interface: 0,
|
|
ibgp: false,
|
|
reflector_client: false,
|
|
})
|
|
.unwrap()
|
|
}
|
|
|
|
/// Attributes with the next hop `next_hop` and, to say which of two
|
|
/// routes is the better, `med`.
|
|
fn via(&self, next_hop: IpAddr, med: u32) -> Arc<AttrSet> {
|
|
let mut attributes = vec![
|
|
PathAttribute::Origin(OriginPathAttribute::IGP),
|
|
AsPathAttribute::from_asns(vec![65001]),
|
|
PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(med)),
|
|
];
|
|
let mp_next_hop = match next_hop {
|
|
IpAddr::V4(v4) => {
|
|
attributes.push(PathAttribute::NextHop(NextHopPathAttribute(v4)));
|
|
None
|
|
}
|
|
IpAddr::V6(v6) => Some(NlriNextHop::Ipv6(v6)),
|
|
};
|
|
self.rib
|
|
.attrs()
|
|
.intern(&attributes, mp_next_hop.as_ref())
|
|
.unwrap()
|
|
}
|
|
|
|
/// The routes in the test's table, as `ip route` prints them.
|
|
fn kernel_routes(&self) -> BTreeSet<String> {
|
|
let table = self.table();
|
|
let mut routes = BTreeSet::new();
|
|
for family in ["-4", "-6"] {
|
|
// A table nothing was ever put in does not exist yet.
|
|
let output = Command::new("ip")
|
|
.args([family, "route", "show", "table", &table.to_string()])
|
|
.output()
|
|
.unwrap();
|
|
routes.extend(
|
|
String::from_utf8_lossy(&output.stdout)
|
|
.lines()
|
|
.map(|line| line.split_whitespace().collect::<Vec<_>>().join(" "))
|
|
// IPv6 routes are shown with "pref medium".
|
|
.map(|line| line.replace(" pref medium", ""))
|
|
// A route that names a nexthop object is shown with
|
|
// its number, which is the kernel's to choose.
|
|
.map(|line| match line.split_once(" nhid ") {
|
|
Some((prefix, rest)) => {
|
|
let rest = rest.split_once(' ').map_or("", |(_, rest)| rest);
|
|
format!("{prefix} {rest}")
|
|
}
|
|
None => line,
|
|
})
|
|
.filter(|line| !line.is_empty()),
|
|
);
|
|
}
|
|
routes
|
|
}
|
|
|
|
/// Waits until the kernel's table holds exactly `expected`.
|
|
#[track_caller]
|
|
fn kernel_becomes(&self, expected: &[String]) {
|
|
let expected: BTreeSet<String> = expected.iter().cloned().collect();
|
|
let start = Instant::now();
|
|
while self.kernel_routes() != expected {
|
|
assert!(
|
|
start.elapsed() < TIMEOUT,
|
|
"the kernel has {:#?}; expected {expected:#?}",
|
|
self.kernel_routes()
|
|
);
|
|
std::thread::sleep(Duration::from_millis(20));
|
|
}
|
|
}
|
|
|
|
/// How `ip route` shows a route of ours.
|
|
fn shown(&self, prefix: &str, gateway: impl std::fmt::Display) -> String {
|
|
format!(
|
|
"{prefix} via {gateway} dev {} proto bgp metric 20",
|
|
self.interface()
|
|
)
|
|
}
|
|
}
|
|
|
|
impl Drop for Machine {
|
|
fn drop(&mut self) {
|
|
let _ = Command::new("ip")
|
|
.args(["link", "del", &self.interface()])
|
|
.output();
|
|
for family in ["-4", "-6"] {
|
|
let _ = Command::new("ip")
|
|
.args([family, "route", "flush", "table", &self.table().to_string()])
|
|
.output();
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Waits for `condition` to hold of the status.
|
|
#[track_caller]
|
|
fn status_becomes(fib: &Fib, what: &str, condition: impl Fn(&bgp_fib::Status) -> bool) {
|
|
let start = Instant::now();
|
|
while !condition(&fib.status()) {
|
|
assert!(
|
|
start.elapsed() < TIMEOUT,
|
|
"{what}: the status is {:#?}",
|
|
fib.status()
|
|
);
|
|
std::thread::sleep(Duration::from_millis(20));
|
|
}
|
|
}
|
|
|
|
/// The best route of each prefix is in the kernel, and follows the table:
|
|
/// when the best route changes, when it goes, and when there is none whose
|
|
/// next hop can be reached.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn the_kernel_has_the_best_route_of_each_prefix() {
|
|
let machine = Machine::new(1);
|
|
let (near, far) = (machine.peer(2), machine.peer(3));
|
|
let via_near = machine.via(machine.neighbor(2).into(), 50);
|
|
// What was there before the daemon started is installed too.
|
|
machine
|
|
.rib
|
|
.announce(near, &via_near, &[prefix("203.0.113.0/24")]);
|
|
let (fib, task) = machine.start();
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
status_becomes(&fib, "in step", |status| {
|
|
status.synced && status.installed == 1
|
|
});
|
|
|
|
// More routes, IPv6 among them.
|
|
let v6_neighbor: IpAddr = "2001:db8:1::2".parse().unwrap();
|
|
machine.rib.announce(
|
|
near,
|
|
&machine.via(v6_neighbor, 50),
|
|
&[prefix("2001:db8:100::/48")],
|
|
);
|
|
machine
|
|
.rib
|
|
.announce(near, &via_near, &[prefix("198.18.0.0/15")]);
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
machine.shown("2001:db8:100::/48", v6_neighbor),
|
|
]);
|
|
|
|
// A better route from another peer takes the first one's place.
|
|
let via_far = machine.via(machine.neighbor(3).into(), 10);
|
|
machine
|
|
.rib
|
|
.announce(far, &via_far, &[prefix("203.0.113.0/24")]);
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", machine.neighbor(3)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
machine.shown("2001:db8:100::/48", v6_neighbor),
|
|
]);
|
|
// It goes: the first is the best again.
|
|
machine.rib.withdraw(far, &[prefix("203.0.113.0/24")]);
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
machine.shown("2001:db8:100::/48", v6_neighbor),
|
|
]);
|
|
|
|
// A best route whose next hop is not a neighbor is not installed, and
|
|
// the route it replaced does not stay.
|
|
let unreachable = machine.via("172.31.9.9".parse().unwrap(), 5);
|
|
machine
|
|
.rib
|
|
.announce(far, &unreachable, &[prefix("198.18.0.0/15")]);
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("2001:db8:100::/48", v6_neighbor),
|
|
]);
|
|
status_becomes(&fib, "one unresolved", |status| {
|
|
status.installed == 2 && status.unresolved == [("no route to the next hop", 1)]
|
|
});
|
|
|
|
// A session ends: its routes go.
|
|
machine.rib.remove_peer(near);
|
|
machine.kernel_becomes(&[]);
|
|
status_becomes(&fib, "nothing installed", |status| status.installed == 0);
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
}
|
|
|
|
/// Routes of ours left by a run that ended badly are removed at the next
|
|
/// start unless they are still wanted. Routes that are not ours are left
|
|
/// alone, whatever they are for.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_left_behind_are_cleared_and_others_are_not_touched() {
|
|
let machine = Machine::new(2);
|
|
let (table, interface) = (machine.table(), machine.interface());
|
|
let add = |route: &str| ip(&format!("route add {route} dev {interface} table {table}"));
|
|
// Ours, from before: one that is still right, one that has another
|
|
// next hop now, one that nobody announces any more.
|
|
add("203.0.113.0/24 via 10.2.0.2 proto bgp metric 20");
|
|
add("198.18.0.0/15 via 10.2.0.9 proto bgp metric 20");
|
|
add("192.88.99.0/24 via 10.2.0.2 proto bgp metric 20");
|
|
// Not ours: someone's static route, and a BGP route with another
|
|
// metric, which is how another daemon's would differ.
|
|
add("192.0.2.0/24 via 10.2.0.7 proto static metric 20");
|
|
add("192.88.99.0/24 via 10.2.0.7 proto bgp metric 30");
|
|
let others = [
|
|
format!("192.0.2.0/24 via 10.2.0.7 dev {interface} proto static metric 20"),
|
|
format!("192.88.99.0/24 via 10.2.0.7 dev {interface} proto bgp metric 30"),
|
|
];
|
|
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine.rib.announce(
|
|
peer,
|
|
&via,
|
|
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
|
|
);
|
|
let (fib, task) = machine.start();
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
others[0].clone(),
|
|
others[1].clone(),
|
|
]);
|
|
|
|
// A run that ends in good order takes its routes with it, and only
|
|
// its own.
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&others);
|
|
}
|
|
|
|
/// What a next hop resolves to depends on the machine's interfaces. When
|
|
/// they change, the routes are worked out again.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_follow_the_machines_interfaces() {
|
|
let machine = Machine::new(3);
|
|
let interface = machine.interface();
|
|
// The interface has no addresses yet: nothing is a neighbor.
|
|
ip(&format!("addr flush dev {interface}"));
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "unresolved", |status| {
|
|
status.synced && status.unresolved == [("no route to the next hop", 1)]
|
|
});
|
|
assert!(machine.kernel_routes().is_empty());
|
|
|
|
// It gets its address: the next hop is a neighbor now.
|
|
machine.add_addresses();
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
status_becomes(&fib, "the interface is shown", |status| {
|
|
status.unresolved.is_empty()
|
|
&& status.interfaces.iter().any(|shown| {
|
|
shown.name == interface
|
|
&& shown.up
|
|
&& shown
|
|
.addresses
|
|
.contains(&(IpAddr::from(machine.neighbor(1)), 24))
|
|
})
|
|
});
|
|
|
|
// It loses it. The kernel drops the route by itself; what matters is
|
|
// that we know it has.
|
|
ip(&format!("addr del 10.3.0.1/24 dev {interface}"));
|
|
machine.kernel_becomes(&[]);
|
|
status_becomes(&fib, "unresolved again", |status| {
|
|
status.installed == 0 && status.unresolved.len() == 1
|
|
});
|
|
|
|
// The interface goes down and comes straight back, faster than we
|
|
// work the routes out again. The kernel dropped the route when it went
|
|
// down, though nothing about the route has changed for us.
|
|
machine.add_addresses_v4();
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
ip(&format!("link set {interface} down"));
|
|
ip(&format!("link set {interface} up"));
|
|
assert!(machine.kernel_routes().is_empty(), "the kernel dropped it");
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
ip(&format!("addr del 10.3.0.1/24 dev {interface}"));
|
|
machine.kernel_becomes(&[]);
|
|
|
|
// And gets it back, with the interface having been down meanwhile.
|
|
ip(&format!("link set {interface} down"));
|
|
ip(&format!("addr add 10.3.0.1/24 dev {interface}"));
|
|
ip(&format!("link set {interface} up"));
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&[]);
|
|
}
|
|
|
|
/// A next hop that is no neighbor goes the way the kernel's own routes
|
|
/// say: a static route's gateway, a tunnel's interface, a nexthop object.
|
|
/// When those routes change, so do ours; a route that discards packets is
|
|
/// the way to nowhere.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn a_far_next_hop_goes_the_way_the_kernels_routes_say() {
|
|
let machine = Machine::new(10);
|
|
let interface = machine.interface();
|
|
let peer = machine.peer(2);
|
|
// Next hops that are not in any network of ours.
|
|
let far = machine.via("172.31.10.9".parse().unwrap(), 0);
|
|
let farther = machine.via("172.31.11.9".parse().unwrap(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &far, &[prefix("203.0.113.0/24")]);
|
|
machine
|
|
.rib
|
|
.announce(peer, &farther, &[prefix("198.18.0.0/15")]);
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "no way to either", |status| {
|
|
status.synced && status.unresolved == [("no route to the next hop", 2)]
|
|
});
|
|
assert!(machine.kernel_routes().is_empty());
|
|
|
|
// A static route with a gateway says the way to one: the route goes
|
|
// in with that gateway.
|
|
ip(&format!(
|
|
"route add 172.31.10.0/24 via 10.10.0.9 dev {interface}"
|
|
));
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", "10.10.0.9")]);
|
|
// A route onto the interface, as a tunnel's is, says the way to the
|
|
// other: the next hop itself is the neighbor there.
|
|
ip(&format!("route add 172.31.11.0/24 dev {interface}"));
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", "10.10.0.9"),
|
|
machine.shown("198.18.0.0/15", "172.31.11.9"),
|
|
]);
|
|
status_becomes(&fib, "both resolved", |status| {
|
|
status.unresolved.is_empty() && status.installed == 2
|
|
});
|
|
|
|
// The way changes: so does the route. Then it goes, and so does the
|
|
// route.
|
|
ip(&format!(
|
|
"route replace 172.31.10.0/24 via 10.10.0.8 dev {interface}"
|
|
));
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", "10.10.0.8"),
|
|
machine.shown("198.18.0.0/15", "172.31.11.9"),
|
|
]);
|
|
ip("route del 172.31.10.0/24");
|
|
machine.kernel_becomes(&[machine.shown("198.18.0.0/15", "172.31.11.9")]);
|
|
status_becomes(&fib, "one unresolved again", |status| {
|
|
status.unresolved == [("no route to the next hop", 1)]
|
|
});
|
|
|
|
// A route that names a nexthop object goes where the object says.
|
|
ip(&format!(
|
|
"nexthop add id 1010 via 10.10.0.7 dev {interface}"
|
|
));
|
|
ip("route add 172.31.10.0/24 nhid 1010");
|
|
machine.kernel_becomes(&[
|
|
machine.shown("203.0.113.0/24", "10.10.0.7"),
|
|
machine.shown("198.18.0.0/15", "172.31.11.9"),
|
|
]);
|
|
|
|
// The tunnel's route becomes a blackhole: the way to nowhere.
|
|
ip("route replace blackhole 172.31.11.0/24");
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", "10.10.0.7")]);
|
|
status_becomes(&fib, "discarded", |status| {
|
|
status.unresolved == [("the route to the next hop discards packets", 1)]
|
|
});
|
|
|
|
ip("route del 172.31.10.0/24");
|
|
ip("route del blackhole 172.31.11.0/24");
|
|
ip("nexthop del id 1010");
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&[]);
|
|
}
|
|
|
|
/// A route that asks for its packets to be dropped goes into the kernel
|
|
/// as a blackhole route, whatever its next hop, and comes out again with
|
|
/// the request.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn a_blackhole_request_becomes_a_blackhole_route() {
|
|
let machine = Machine::new(13);
|
|
let peer = machine.peer(2);
|
|
// Asked for from a next hop nothing is a neighbor of, and with one
|
|
// ordinary route beside it to show the two apart.
|
|
let mut attributes = vec![
|
|
PathAttribute::Origin(OriginPathAttribute::IGP),
|
|
AsPathAttribute::from_asns(vec![65002]),
|
|
PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(172, 31, 11, 9))),
|
|
PathAttribute::Communities(CommunitiesPathAttribute(vec![(65002, 1), (0xFFFF, 0x029A)])),
|
|
];
|
|
let blackhole = machine.rib.attrs().intern(&attributes, None).unwrap();
|
|
attributes.pop();
|
|
attributes.push(PathAttribute::Communities(CommunitiesPathAttribute(vec![
|
|
(65002, 1),
|
|
])));
|
|
let plain = machine.rib.attrs().intern(&attributes, None).unwrap();
|
|
machine.rib.announce(
|
|
peer,
|
|
&blackhole,
|
|
&[prefix("203.0.113.9/32"), prefix("2001:db8:100::9/128")],
|
|
);
|
|
machine
|
|
.rib
|
|
.announce(peer, &plain, &[prefix("198.18.0.0/15")]);
|
|
let (fib, task) = machine.start();
|
|
// The kernel puts an IPv6 blackhole route on lo, and says so.
|
|
machine.kernel_becomes(&[
|
|
"blackhole 203.0.113.9 proto bgp metric 20".into(),
|
|
"blackhole 2001:db8:100::9 dev lo proto bgp metric 20".into(),
|
|
]);
|
|
status_becomes(&fib, "two blackholes, one unresolved", |status| {
|
|
status.synced
|
|
&& status.installed == 2
|
|
&& status.blackholes == 2
|
|
&& status.unresolved == [("no route to the next hop", 1)]
|
|
});
|
|
|
|
// The request is withdrawn: the blackhole goes.
|
|
machine.rib.withdraw(peer, &[prefix("203.0.113.9/32")]);
|
|
machine.kernel_becomes(&["blackhole 2001:db8:100::9 dev lo proto bgp metric 20".into()]);
|
|
|
|
// A blackhole left by an earlier run is found and kept like any other.
|
|
fib.leave().await;
|
|
task.await.unwrap().unwrap();
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "the blackhole is adopted", |status| {
|
|
status.synced && status.installed == 1 && status.blackholes == 1
|
|
});
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&[]);
|
|
}
|
|
|
|
/// A table's worth of routes goes in, and comes out, in reasonable time.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn many_routes_are_installed_and_removed() {
|
|
// FERRIX_ROUTES=1000000 for the size of the Internet's IPv4 table.
|
|
let routes: usize = std::env::var("FERRIX_ROUTES")
|
|
.ok()
|
|
.and_then(|routes| routes.parse().ok())
|
|
.unwrap_or(200_000);
|
|
let machine = Machine::new(4);
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
let prefixes: Vec<IpPrefix> = (0..routes as u32)
|
|
.map(|n| {
|
|
let [_, a, b, c] = n.to_be_bytes();
|
|
IpPrefix::new(
|
|
bgp_primitives::constants::IpVersion::Ipv4,
|
|
[100 + a, b, c, 0],
|
|
24,
|
|
)
|
|
.unwrap()
|
|
})
|
|
.collect();
|
|
machine.rib.announce(peer, &via, &prefixes);
|
|
|
|
let started = Instant::now();
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "all installed", |status| {
|
|
status.synced && status.installed == routes
|
|
});
|
|
let installed_in = started.elapsed();
|
|
assert_eq!(machine.kernel_routes().len(), routes);
|
|
assert_eq!(fib.status().errors, 0, "{:?}", fib.status().last_error);
|
|
|
|
let started = Instant::now();
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
let removed_in = started.elapsed();
|
|
assert!(machine.kernel_routes().is_empty());
|
|
println!(
|
|
"{routes} routes: installed in {installed_in:?} ({:.0}/s), removed in {removed_in:?}",
|
|
routes as f64 / installed_in.as_secs_f64()
|
|
);
|
|
}
|
|
|
|
/// Someone empties the table behind our back (`ip route flush`), and then
|
|
/// the peers withdraw everything: the kernel refuses every removal, a
|
|
/// table's worth of rejections at once. That is more than the socket holds
|
|
/// answers for, and must be no more than a reason to look again.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn a_flood_of_refusals_does_not_stop_the_daemon() {
|
|
const ROUTES: u32 = 20_000;
|
|
let mut machine = Machine::new(11);
|
|
machine.nexthop_objects = false;
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
let prefixes: Vec<IpPrefix> = (0..ROUTES)
|
|
.map(|n| {
|
|
let [_, a, b, c] = n.to_be_bytes();
|
|
IpPrefix::new(
|
|
bgp_primitives::constants::IpVersion::Ipv4,
|
|
[100 + a, b, c, 0],
|
|
24,
|
|
)
|
|
.unwrap()
|
|
})
|
|
.collect();
|
|
machine.rib.announce(peer, &via, &prefixes);
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "all installed", |status| {
|
|
status.synced && status.installed == ROUTES as usize
|
|
});
|
|
|
|
ip(&format!("route flush table {}", machine.table()));
|
|
assert!(machine.kernel_routes().is_empty());
|
|
let routes: Vec<bgp_primitives::ip_prefix::Nlri> =
|
|
prefixes.iter().map(|prefix| (*prefix).into()).collect();
|
|
machine.rib.withdraw(peer, &routes);
|
|
status_becomes(&fib, "nothing installed", |status| {
|
|
status.synced && status.installed == 0
|
|
});
|
|
assert!(
|
|
!task.is_finished(),
|
|
"the task that talks to the kernel ended"
|
|
);
|
|
|
|
// And it goes on working.
|
|
machine.rib.announce(peer, &via, &prefixes[..3]);
|
|
status_becomes(&fib, "three installed", |status| {
|
|
status.synced && status.installed == 3
|
|
});
|
|
assert_eq!(machine.kernel_routes().len(), 3);
|
|
assert!(!task.is_finished());
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
}
|
|
|
|
/// Routes name a nexthop object for their gateway where the kernel has
|
|
/// such objects. The object is ours: made when first needed, put back if
|
|
/// someone removes it, and removed when we stop. Other objects are not
|
|
/// touched.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_name_a_nexthop_object_that_is_ours_to_keep() {
|
|
let machine = Machine::new(5);
|
|
let interface = machine.interface();
|
|
// Someone else's object, to the same gateway even.
|
|
ip(&format!(
|
|
"nexthop add id 50005 via 10.5.0.2 dev {interface} protocol bgp"
|
|
));
|
|
let nexthops = || -> Vec<String> {
|
|
ip("nexthop show")
|
|
.lines()
|
|
.filter(|line| line.contains(&format!("dev {interface} ")))
|
|
.map(|line| line.trim().to_string())
|
|
.collect()
|
|
};
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine.rib.announce(
|
|
peer,
|
|
&via,
|
|
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
|
|
);
|
|
let (fib, task) = machine.start();
|
|
let both = [
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
];
|
|
machine.kernel_becomes(&both);
|
|
|
|
// One object of ours beside the other, and both routes name it.
|
|
let ours: Vec<String> = nexthops()
|
|
.into_iter()
|
|
.filter(|line| !line.starts_with("id 50005 "))
|
|
.collect();
|
|
assert_eq!(ours.len(), 1, "{:?}", nexthops());
|
|
assert!(ours[0].contains("via 10.5.0.2") && ours[0].contains("proto bgp"));
|
|
let id = ours[0].split_whitespace().nth(1).unwrap().to_string();
|
|
let raw = ip(&format!("route show table {}", machine.table()));
|
|
assert_eq!(raw.matches(&format!("nhid {id} ")).count(), 2, "{raw}");
|
|
|
|
// Someone removes our object, and the kernel our routes with it. We
|
|
// are told, and put both back.
|
|
ip(&format!("nexthop del id {id}"));
|
|
machine.kernel_becomes(&both);
|
|
|
|
// When we stop, our object goes with our routes; the other stays.
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&[]);
|
|
assert_eq!(nexthops().len(), 1, "{:?}", nexthops());
|
|
assert!(nexthops()[0].starts_with("id 50005 "));
|
|
ip("nexthop del id 50005");
|
|
}
|
|
|
|
/// On a kernel without nexthop objects, or where the configuration says
|
|
/// so, every route carries its own gateway.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_can_carry_their_own_gateway() {
|
|
let mut machine = Machine::new(6);
|
|
machine.nexthop_objects = false;
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
|
|
let (fib, task) = machine.start();
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
let raw = ip(&format!("route show table {}", machine.table()));
|
|
assert!(!raw.contains("nhid"), "{raw}");
|
|
assert!(
|
|
!ip("nexthop show").contains(&format!("dev {} ", machine.interface())),
|
|
"no nexthop object was made"
|
|
);
|
|
|
|
// The next hop changes, and the route goes.
|
|
let other = machine.via(machine.neighbor(3).into(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &other, &[prefix("203.0.113.0/24")]);
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(3))]);
|
|
machine.rib.withdraw(peer, &[prefix("203.0.113.0/24")]);
|
|
machine.kernel_becomes(&[]);
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
}
|
|
|
|
/// When the daemon starts, its table is empty until its peers have sent
|
|
/// theirs. The routes an earlier run left are kept for a while, so that
|
|
/// the machine goes on forwarding; then those nobody announced again go.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_of_an_earlier_run_are_kept_while_peers_come_back() {
|
|
let mut machine = Machine::new(7);
|
|
machine.stale_time = Duration::from_secs(3);
|
|
let (table, interface) = (machine.table(), machine.interface());
|
|
let add = |route: &str| ip(&format!("route add {route} dev {interface} table {table}"));
|
|
add("203.0.113.0/24 via 10.7.0.2 proto bgp metric 20");
|
|
add("198.18.0.0/15 via 10.7.0.2 proto bgp metric 20");
|
|
let left = [
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
];
|
|
|
|
// The daemon starts with nothing in its table: both stay.
|
|
let (fib, task) = machine.start();
|
|
status_becomes(&fib, "in step, keeping two", |status| {
|
|
status.synced && status.installed == 2 && status.kept == 2
|
|
});
|
|
assert_eq!(machine.kernel_routes(), left.iter().cloned().collect());
|
|
|
|
// A peer comes back with one of them.
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
|
|
status_becomes(&fib, "one confirmed", |status| status.kept == 1);
|
|
assert_eq!(machine.kernel_routes(), left.iter().cloned().collect());
|
|
|
|
// Time is up: the one nobody announced goes.
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
status_becomes(&fib, "none kept", |status| {
|
|
status.installed == 1 && status.kept == 0
|
|
});
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
}
|
|
|
|
/// A restart that is to go unnoticed (graceful restart): the routes are
|
|
/// left in the kernel, by way of their nexthop objects; the next run
|
|
/// finds them, says how many, and lets those nobody announced again go
|
|
/// as soon as it is told the peers have all sent theirs.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn routes_stay_in_the_kernel_over_a_restart() {
|
|
let mut machine = Machine::new(12);
|
|
machine.stale_time = Duration::from_secs(60);
|
|
let (fib, task) = machine.start();
|
|
assert_eq!(fib.adopted().await, 0);
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine.rib.announce(
|
|
peer,
|
|
&via,
|
|
&[prefix("203.0.113.0/24"), prefix("198.18.0.0/15")],
|
|
);
|
|
let both = [
|
|
machine.shown("198.18.0.0/15", machine.neighbor(2)),
|
|
machine.shown("203.0.113.0/24", machine.neighbor(2)),
|
|
];
|
|
machine.kernel_becomes(&both);
|
|
|
|
fib.leave().await;
|
|
task.await.unwrap().unwrap();
|
|
assert_eq!(machine.kernel_routes(), both.iter().cloned().collect());
|
|
|
|
// The next run, with a table that is empty until the peers are back.
|
|
machine.rib = Arc::new(RibTable::new(4));
|
|
let (fib, task) = machine.start();
|
|
assert_eq!(fib.adopted().await, 2);
|
|
status_becomes(&fib, "keeping both", |status| status.kept == 2);
|
|
let peer = machine.peer(2);
|
|
let via = machine.via(machine.neighbor(2).into(), 0);
|
|
machine
|
|
.rib
|
|
.announce(peer, &via, &[prefix("203.0.113.0/24")]);
|
|
status_becomes(&fib, "one confirmed", |status| status.kept == 1);
|
|
|
|
// Every peer has sent its routes: what was not among them goes now,
|
|
// and not a minute later.
|
|
fib.release_kept();
|
|
machine.kernel_becomes(&[machine.shown("203.0.113.0/24", machine.neighbor(2))]);
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
}
|
|
|
|
/// An IPv4 route whose next hop is an IPv6 neighbor: routes over a link
|
|
/// that has only IPv6 addresses (RFC 8950). The kernel takes such a route,
|
|
/// by way of a nexthop object or with its gateway spelled out.
|
|
#[tokio::test(flavor = "multi_thread")]
|
|
#[ignore = "changes the machine's routes; run scripts/kernel-tests.sh"]
|
|
async fn an_ipv4_route_goes_by_way_of_an_ipv6_neighbor() {
|
|
for (n, nexthop_objects) in [(8, true), (9, false)] {
|
|
let mut machine = Machine::new(n);
|
|
machine.nexthop_objects = nexthop_objects;
|
|
// Only IPv6 on the link.
|
|
ip(&format!(
|
|
"addr del 10.{n}.0.1/24 dev {}",
|
|
machine.interface()
|
|
));
|
|
let neighbor: IpAddr = format!("2001:db8:{n}::2").parse().unwrap();
|
|
let peer = machine
|
|
.rib
|
|
.add_peer(PeerInfo {
|
|
asn: 65002,
|
|
router_id: machine.neighbor(2),
|
|
address: neighbor,
|
|
interface: 0,
|
|
ibgp: false,
|
|
reflector_client: false,
|
|
})
|
|
.unwrap();
|
|
// Both families by way of the same neighbor.
|
|
let via = machine.via(neighbor, 0);
|
|
machine.rib.announce(
|
|
peer,
|
|
&via,
|
|
&[prefix("203.0.113.0/24"), prefix("2001:db8:100::/48")],
|
|
);
|
|
// And an IPv4 route whose next hop has a link-local address too,
|
|
// with a global one that is not on any link of ours.
|
|
let link_local = machine
|
|
.rib
|
|
.attrs()
|
|
.intern(
|
|
&[
|
|
PathAttribute::Origin(OriginPathAttribute::IGP),
|
|
AsPathAttribute::from_asns(vec![65002]),
|
|
],
|
|
Some(&NlriNextHop::Ipv6WithLl {
|
|
global: "2001:db8:ffff::2".parse().unwrap(),
|
|
link_local: "fe80::2".parse().unwrap(),
|
|
}),
|
|
)
|
|
.unwrap();
|
|
machine
|
|
.rib
|
|
.announce(peer, &link_local, &[prefix("198.18.0.0/15")]);
|
|
|
|
let (fib, task) = machine.start();
|
|
let interface = machine.interface();
|
|
machine.kernel_becomes(&[
|
|
format!("203.0.113.0/24 via inet6 {neighbor} dev {interface} proto bgp metric 20"),
|
|
format!("198.18.0.0/15 via inet6 fe80::2 dev {interface} proto bgp metric 20"),
|
|
machine.shown("2001:db8:100::/48", neighbor),
|
|
]);
|
|
assert_eq!(fib.status().errors, 0, "{:?}", fib.status().last_error);
|
|
|
|
fib.stop().await;
|
|
task.await.unwrap().unwrap();
|
|
machine.kernel_becomes(&[]);
|
|
}
|
|
}
|