policy, config, web: the TCP gauges named once, one revalidator, one peer-naming rule

A code-quality pass over policy, configuration and the status API.
The twenty TCP gauges were spelled out in the struct, the
constructor, the removal, the sampler in the web crate and the status
view; one macro names each once with its help text, `PeerTcp::set`
fills them from a connection, and the view is a `From`. Every metric
and the list that forces them at start come from one declaration,
with a test that every declared family is registered. The two
revalidators, for next hops and for RPKI, share the task body that
scans the table off the sessions' threads and fans the findings out
to every session; the RPKI scan judges a route with the same `Checks`
the import path uses. Which peer a query names is one rule for the
received, rejected and TCP endpoints, so `/api/tcp` takes a name or an
address like the others. The config parser says proto3's "zero is
unset" in three helpers instead of a dozen shapes, and `parse_peer`
is three named parts shorter. The policy's test-only next-hop flag is
named for what it does, `accept_own_host_next_hop`; the "listed, or
else ours" prefix rule and the mark-then-prefer step each exist once.
The status page derives its spanning rows from the table's header,
shares one key-value grid and one detail-row rule, and defines its
helpers before their use. The web crate no longer re-exports views
nothing uses, and the TCP endpoint sits above the test module.

Testing: the TCP view's arithmetic (a share of no busy time is zero,
not NaN), the peer-naming rule, the RPKI scan finding only what a
change can have made invalid, and `/api/tcp` by address, by name and
for an unknown peer. The server and web tests, the HTTP, roles, RPKI
and import-policy integration tests, the docs build and every lab
phase pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

crates/bench/benches/full_table.rs

@@ -18,6 +18,7 @@ use std::sync::atomic::{AtomicUsize, Ordering};
 use std::time::{Duration, Instant};
 
 use bgp_bench::memory::{Region, describe};
+use bgp_bench::report::{mib, per_second, row};
 use bgp_bench::{Prefixes, Shape, announcements, encoded, env, wire_context};
 use bgp_integration_tests::test_peer::TestPeer;
 use bgp_integration_tests::test_server::TestServer;
@@ -26,23 +27,6 @@ use bgp_primitives::ip_prefix::IpPrefix;
 use bgp_rib::table::{PeerIndex, PeerInfo, RibTable};
 use bytes::BytesMut;
 
-fn mib(bytes: usize) -> f64 {
-    bytes as f64 / (1024.0 * 1024.0)
-}
-
-fn per_second(count: usize, elapsed: Duration) -> String {
-    let rate = count as f64 / elapsed.as_secs_f64();
-    if rate >= 1e6 {
-        format!("{:.2} M/s", rate / 1e6)
-    } else {
-        format!("{:.0} k/s", rate / 1e3)
-    }
-}
-
-fn row(what: &str, elapsed: Duration, detail: String) {
-    println!("{what:<44} {:>9.3} s   {detail}", elapsed.as_secs_f64());
-}
-
 fn main() {
     let peers = env("BENCH_PEERS", 4) as u32;
     let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
@@ -110,6 +94,8 @@ fn main() {
                 .handshake(64512 + peer, Ipv4Addr::new(10, 0, 0, peer as u8), 90)
                 .await
                 .expect("a session");
+            // The same bytes `table_stream` would make, from the messages
+            // already encoded for the decoding above.
             let stream: Vec<u8> = messages.iter().flat_map(|m| m.iter().copied()).collect();
             senders.push((session, stream));
         }

crates/bench/benches/mesh.rs

@@ -32,17 +32,21 @@ use std::sync::Arc;
 use std::sync::atomic::{AtomicU64, Ordering};
 use std::time::{Duration, Instant};
 
-use bgp_bench::{Prefixes, Shape, announcements, encoded, env};
+use bgp_bench::report::row;
+use bgp_bench::{Prefixes, Shape, announcements, encoded, env, table_stream};
 use bgp_integration_tests::peer_config;
-use bgp_integration_tests::test_peer::TestPeer;
+use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
 use bgp_integration_tests::test_server::TestServer;
 use bgp_packet::attributes::PathAttribute;
-use bgp_packet::open::{AddPathEntry, Capability, SendReceive};
+use bgp_packet::message::msg_type;
+use bgp_packet::open::SendReceive;
 use bgp_server::config::PeerConfig;
 use bgp_server::family::SUPPORTED;
 use bgp_server::policy::{ExportPolicy, ImportPolicy, PeerRole};
 use bgp_server::session::SessionRegistry;
 use tokio::io::{AsyncReadExt, AsyncWriteExt, BufReader};
+use tokio::net::tcp::OwnedWriteHalf;
+use tokio::task::JoinHandle;
 
 /// An AS number that is in none of the synthetic paths.
 const ASN: u32 = 4_200_000_000;
@@ -58,6 +62,9 @@ fn collector_address(n: u32) -> Ipv4Addr {
 /// The collectors' AS numbers start here.
 const COLLECTOR_ASN: u32 = 65000;
 
+/// How often the tables are looked at while they fill.
+const POLL: Duration = Duration::from_millis(5);
+
 /// What the routers send each other, and the receiver: every path.
 fn every_path() -> Arc<ExportPolicy> {
     Arc::new(ExportPolicy {
@@ -77,12 +84,12 @@ fn free_port() -> u16 {
 
 /// A router of the mesh, listening for feeders on IPv4 loopback and for
 /// the other router on IPv6 loopback at `port`, which it dials at
-/// `other`.
+/// `other_port`, knowing it as `other_name`.
 async fn router(
-    name: &'static str,
+    other_name: &'static str,
     id: u8,
     port: u16,
-    other: u16,
+    other_port: u16,
     with_receiver: bool,
     collectors: u32,
 ) -> TestServer {
@@ -99,8 +106,8 @@ async fn router(
             import: ImportPolicy::permissive(),
             export: every_path(),
             ..peer_config(
-                name,
+                other_name,
-                SocketAddr::new(Ipv6Addr::LOCALHOST.into(), other),
+                SocketAddr::new(Ipv6Addr::LOCALHOST.into(), other_port),
                 ASN,
             )
         }];
@@ -144,20 +151,22 @@ async fn feeders(
 ) -> Vec<(TestPeer, Vec<u8>)> {
     let mut sessions = Vec::new();
     for peer in first..first + count {
-        let mut updates = announcements(prefixes, shape, peer);
+        let stream = if marked {
-        if marked {
             // As having come across from a peer (RFC 9234): not for the
             // collectors, which are peers too.
+            let mut updates = announcements(prefixes, shape, peer);
             for update in &mut updates {
                 update
                     .path_attributes
                     .push(PathAttribute::OnlyToCustomer(64512 + peer));
             }
-        }
+            encoded(&updates)
-        let stream: Vec<u8> = encoded(&updates)
+                .iter()
-            .iter()
+                .flat_map(|m| m.iter().copied())
-            .flat_map(|m| m.iter().copied())
+                .collect()
-            .collect();
+        } else {
+            table_stream(prefixes, shape, peer)
+        };
         let mut session = TestPeer::connect(router.addr).await.expect("a connection");
         // No hold timer: a feeder has nothing to say after its table.
         session
@@ -169,39 +178,112 @@ async fn feeders(
     sessions
 }
 
-fn main() {
+/// Every session of `registry`, one line each, for the panic when a
-    // The routers' logs, with `BENCH_LOG` as a `RUST_LOG` filter.
+/// deadline passes: where the routes have got to, and what each session
-    if let Ok(filter) = std::env::var("BENCH_LOG") {
+/// was to be sent.
-        tracing_subscriber::fmt()
+fn sessions_report(registry: &SessionRegistry) -> Vec<String> {
-            .with_env_filter(filter)
+    registry
-            .with_writer(std::io::stderr)
+        .list()
-            .init();
+        .iter()
-    }
+        .map(|s| {
-    let peers_a = env("BENCH_PEERS", 4) as u32;
+            format!(
-    let peers_b = env("BENCH_PEERS_B", 2) as u32;
+                "{} {:?} export {:?} add_path_send {:?} routes {} rejected {} advertised {}",
-    let shape = Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1));
+                s.peer,
-    let per_peer = shape.prefixes();
+                s.name,
-    println!(
+                s.export,
-        "== Two routers of AS{ASN}, every path both ways: A fed by {peers_a} peers, B by {peers_b}, {} prefixes each",
+                s.add_path_send,
-        per_peer
+                s.counters.routes(),
-    );
+                s.counters.rejected(),
-    let prefixes = Prefixes::generate(shape);
+                s.counters.advertised()
+            )
+        })
+        .collect()
+}
 
-    let runtime = tokio::runtime::Builder::new_multi_thread()
+/// How big a run is, from the environment.
-        .enable_all()
+struct Settings {
-        .build()
+    peers_a: u32,
-        .expect("a runtime");
+    peers_b: u32,
-    runtime.block_on(async {
+    shape: Shape,
+    collectors: u32,
+}
+
+impl Settings {
+    fn from_env() -> Self {
+        Self {
+            peers_a: env("BENCH_PEERS", 4) as u32,
+            peers_b: env("BENCH_PEERS_B", 2) as u32,
+            shape: Shape::INTERNET.scaled_down(env("BENCH_SCALE_DOWN", 1)),
+            collectors: env("BENCH_COLLECTORS", 0) as u32,
+        }
+    }
+
+    /// How many routes A's own feeders bring, and B's.
+    fn own_routes(&self) -> (usize, usize) {
+        let per_peer = self.shape.prefixes();
+        (
+            per_peer * self.peers_a as usize,
+            per_peer * self.peers_b as usize,
+        )
+    }
+
+    /// How many routes the receiver is to be sent: A's IPv4 routes from
+    /// its own peers.
+    fn receiver_routes(&self) -> usize {
+        self.shape.ipv4_prefixes * self.peers_a as usize
+    }
+}
+
+/// The two routers, up and peered, with the collectors' sessions once
+/// they are connected.
+struct Mesh {
+    settings: Settings,
+    prefixes: Prefixes,
+    a: TestServer,
+    b: TestServer,
+    /// The collectors' connections, drained, held until the end.
+    collectors: Vec<(OwnedWriteHalf, JoinHandle<()>)>,
+}
+
+/// The feeders sending, from when they began.
+struct Feeding {
+    started: Instant,
+    /// Each hands its session back once its table is written.
+    sending: Vec<JoinHandle<TestPeer>>,
+}
+
+/// When each router had what: its own peers' routes, and the other's too.
+struct Tables {
+    a_own: Duration,
+    b_own: Duration,
+    a_all: Duration,
+    b_all: Duration,
+}
+
+impl Mesh {
+    /// Starts the routers, with the collectors and receiver configured at
+    /// A, and waits for them to peer.
+    async fn start(settings: Settings, prefixes: Prefixes) -> Self {
         let (port_a, port_b) = (free_port(), free_port());
-        let collectors = env("BENCH_COLLECTORS", 0) as u32;
+        let a = router("b", 1, port_a, port_b, true, settings.collectors).await;
-        let a = router("b", 1, port_a, port_b, true, collectors).await;
         let b = router("a", 2, port_b, port_a, false, 0).await;
         a.wait_for_sessions(1).await.expect("the mesh session at A");
         b.wait_for_sessions(1).await.expect("the mesh session at B");
-        // The collectors are there before the feeders, as a router's
+        Self {
-        // peers are before its tables arrive after a restart.
+            settings,
-        let mut draining = Vec::new();
+            prefixes,
-        for n in 1..=collectors {
+            a,
+            b,
+            collectors: Vec::new(),
+        }
+    }
+
+    /// Connects the collectors to A. They are there before the feeders, as
+    /// a router's peers are before its tables arrive after a restart. An
+    /// address that is not this machine's ends the connecting, with a
+    /// word.
+    async fn connect_collectors(&mut self) {
+        for n in 1..=self.settings.collectors {
             let socket = tokio::net::TcpSocket::new_v4().expect("a socket");
             if let Err(e) = socket.bind(SocketAddr::new(collector_address(n).into(), 0)) {
                 println!(
@@ -210,61 +292,82 @@ fn main() {
                 );
                 break;
             }
-            let stream = socket.connect(a.addr).await.expect("A listens");
+            let stream = socket.connect(self.a.addr).await.expect("A listens");
             let mut session = TestPeer::from_stream(stream);
             session
                 .handshake(COLLECTOR_ASN + n, Ipv4Addr::new(192, 0, 3, n as u8), 0)
                 .await
                 .expect("A takes a collector's session");
-            let (mut reader, writer) = session.into_stream().into_split();
+            self.collectors.push(session.drain());
-            draining.push((
-                writer,
-                tokio::spawn(async move {
-                    let mut sink = vec![0u8; 1 << 16];
-                    while matches!(reader.read(&mut sink).await, Ok(n) if n > 0) {}
-                }),
-            ));
         }
-        if collectors > 0 {
+        if !self.collectors.is_empty() {
-            a.wait_for_sessions(1 + draining.len())
+            self.a
+                .wait_for_sessions(1 + self.collectors.len())
                 .await
                 .expect("the collectors' sessions at A");
         }
+    }
 
-        let feed_a = feeders(&a, &prefixes, shape, 1, peers_a, collectors > 0).await;
+    /// Connects the feeders to both routers and starts them sending.
-        let feed_b = feeders(&b, &prefixes, shape, 1 + peers_a, peers_b, false).await;
+    async fn feed(&self) -> Feeding {
-        let own_a = per_peer * peers_a as usize;
+        let Settings {
-        let own_b = per_peer * peers_b as usize;
+            peers_a,
-
+            peers_b,
-        let receiver_routes = shape.ipv4_prefixes * peers_a as usize;
+            shape,
+            ..
+        } = self.settings;
+        let marked = !self.collectors.is_empty();
+        let feed_a = feeders(&self.a, &self.prefixes, shape, 1, peers_a, marked).await;
+        let feed_b = feeders(&self.b, &self.prefixes, shape, 1 + peers_a, peers_b, false).await;
         let started = Instant::now();
-        // Early: as the feeders start, or (2) once A has its own peers'
+        let sending = feed_a
-        // routes and B's are still coming, as a router's sessions come
+            .into_iter()
-        // up after a restart while its peers' tables arrive.
+            .chain(feed_b)
-        let when = env("BENCH_RECEIVER_EARLY", 0);
+            .map(|(mut session, stream)| {
-        let early_receiver = (when > 0).then(|| {
+                tokio::spawn(async move {
-            let (addr, registry, rib) = (a.addr, a.registry.clone(), Arc::clone(&a.rib));
+                    session.send_raw(&stream).await.expect("the router reads");
-            tokio::spawn(async move {
+                    session
-                while when == 2 && rib.len() < own_a {
+                })
-                    tokio::time::sleep(Duration::from_millis(5)).await;
-                }
-                resync(addr, registry, receiver_routes).await
             })
-        });
+            .collect();
-        let mut sending = Vec::new();
+        Feeding { started, sending }
-        for (mut session, stream) in feed_a.into_iter().chain(feed_b) {
+    }
-            sending.push(tokio::spawn(async move {
+
-                session.send_raw(&stream).await.expect("the router reads");
+    /// The receiver taking A's table while it fills, if `BENCH_RECEIVER_EARLY`
-                session
+    /// asks for that: as the feeders start (1), or once A has its own
-            }));
+    /// peers' routes and B's are still coming (2), as a router's sessions
+    /// come up after a restart while its peers' tables arrive.
+    fn early_receiver(&self, deadline: Duration) -> Option<JoinHandle<()>> {
+        let when = env("BENCH_RECEIVER_EARLY", 0);
+        if when == 0 {
+            return None;
         }
+        let (own_a, _) = self.settings.own_routes();
+        let routes = self.settings.receiver_routes();
+        let (addr, registry, rib) = (
+            self.a.addr,
+            self.a.registry.clone(),
+            Arc::clone(&self.a.rib),
+        );
+        Some(tokio::spawn(async move {
+            while when == 2 && rib.len() < own_a {
+                tokio::time::sleep(POLL).await;
+            }
+            resync(addr, registry, routes, deadline).await
+        }))
+    }
+
+    /// Waits until both routers have every route, noting when each had
+    /// what; or panics once `deadline` has passed since `started`.
+    async fn wait_for_tables(&self, started: Instant, deadline: Duration) -> Tables {
+        let (own_a, own_b) = self.settings.own_routes();
         let mut a_own = None;
         let mut b_own = None;
         let mut a_all = None;
         let mut b_all = None;
-        while a_all.is_none() || b_all.is_none() {
+        let (a_all, b_all) = loop {
-            tokio::time::sleep(Duration::from_millis(5)).await;
+            tokio::time::sleep(POLL).await;
-            let (at_a, at_b) = (a.rib.len(), b.rib.len());
+            let (at_a, at_b) = (self.a.rib.len(), self.b.rib.len());
             let now = started.elapsed();
             if a_own.is_none() && at_a >= own_a {
                 a_own = Some(now);
@@ -278,192 +381,279 @@ fn main() {
             if b_all.is_none() && at_b >= own_a + own_b {
                 b_all = Some(now);
             }
-            if now > Duration::from_secs(env("BENCH_DEADLINE", 600) as u64) {
+            if let (Some(a_all), Some(b_all)) = (a_all, b_all) {
-                let sessions = |router: &TestServer| -> Vec<String> {
+                break (a_all, b_all);
-                    router
+            }
-                        .registry
+            if now > deadline {
-                        .list()
-                        .iter()
-                        .map(|s| {
-                            format!(
-                                "{} routes {} rejected {}",
-                                s.peer,
-                                s.counters.routes(),
-                                s.counters.rejected()
-                            )
-                        })
-                        .collect()
-                };
                 panic!(
                     "not all routes arrived: A has {at_a} {:?}, B has {at_b} {:?}",
-                    sessions(&a),
+                    sessions_report(&self.a.registry),
-                    sessions(&b)
+                    sessions_report(&self.b.registry)
                 );
             }
-        }
-        let row = |what: &str, when: Option<Duration>, routes: usize| {
-            let when = when.unwrap_or_default();
-            println!(
-                "{what:<52} {:>8.3} s   {:.2} M routes/s",
-                when.as_secs_f64(),
-                routes as f64 / when.as_secs_f64() / 1e6
-            );
         };
-        row("A has its own peers' routes", a_own, own_a);
+        Tables {
-        row("B has its own peers' routes", b_own, own_b);
+            a_own: a_own.unwrap_or_default(),
+            b_own: b_own.unwrap_or_default(),
+            a_all,
+            b_all,
+        }
+    }
+
+    /// One row for each of the four moments of `tables`, with the rate the
+    /// routes that made the moment came at.
+    fn report(&self, tables: &Tables) {
+        let (own_a, own_b) = self.settings.own_routes();
+        let rate = |when: Duration, routes: usize| {
+            format!("{:.2} M routes/s", routes as f64 / when.as_secs_f64() / 1e6)
+        };
+        row(
+            "A has its own peers' routes",
+            tables.a_own,
+            rate(tables.a_own, own_a),
+        );
+        row(
+            "B has its own peers' routes",
+            tables.b_own,
+            rate(tables.b_own, own_b),
+        );
         row(
             "A has B's routes too (B's exporter, A's session)",
-            a_all,
+            tables.a_all,
-            own_b,
+            rate(tables.a_all, own_b),
         );
         row(
             "B has A's routes too (A's exporter, B's session)",
-            b_all,
+            tables.b_all,
-            own_a,
+            rate(tables.b_all, own_a),
         );
-        // The feeders' sessions stay up until the end: a feeder going
+    }
-        // would have its routes withdrawn from both tables meanwhile.
+}
+
+impl Feeding {
+    /// Waits for every feeder to have written its table, and returns the
+    /// sessions, which are to stay up until the end: a feeder going would
+    /// have its routes withdrawn from both tables meanwhile.
+    async fn finished(self) -> Vec<TestPeer> {
         let mut feeders = Vec::new();
-        for session in sending {
+        for session in self.sending {
             feeders.push(session.await.expect("a feeder is done"));
         }
+        feeders
+    }
+}
+
+fn main() {
+    // The routers' logs, with `BENCH_LOG` as a `RUST_LOG` filter.
+    if let Ok(filter) = std::env::var("BENCH_LOG") {
+        tracing_subscriber::fmt()
+            .with_env_filter(filter)
+            .with_writer(std::io::stderr)
+            .init();
+    }
+    let settings = Settings::from_env();
+    let deadline = Duration::from_secs(env("BENCH_DEADLINE", 600) as u64);
+    println!(
+        "== Two routers of AS{ASN}, every path both ways: A fed by {} peers, B by {}, {} prefixes each",
+        settings.peers_a,
+        settings.peers_b,
+        settings.shape.prefixes()
+    );
+    let prefixes = Prefixes::generate(settings.shape);
+
+    let runtime = tokio::runtime::Builder::new_multi_thread()
+        .enable_all()
+        .build()
+        .expect("a runtime");
+    runtime.block_on(async {
+        let mut mesh = Mesh::start(settings, prefixes).await;
+        mesh.connect_collectors().await;
+        let early_receiver = mesh.early_receiver(deadline);
+        let feeding = mesh.feed().await;
+        let tables = mesh.wait_for_tables(feeding.started, deadline).await;
+        mesh.report(&tables);
+        let feeders = feeding.finished().await;
         match early_receiver {
             Some(receiver) => receiver.await.expect("the receiver is done"),
-            None => resync(a.addr, a.registry.clone(), receiver_routes).await,
+            None => {
+                resync(
+                    mesh.a.addr,
+                    mesh.a.registry.clone(),
+                    mesh.settings.receiver_routes(),
+                    deadline,
+                )
+                .await
+            }
         }
         drop(feeders);
-        drop(draining);
+        drop(mesh);
     });
 }
 
+/// The new peer taking a router's table: counting the UPDATEs it is sent
+/// while keeping the session alive.
+struct Receiver {
+    started: Instant,
+    /// When the first UPDATE arrived, in microseconds after `started`; 0
+    /// until it has.
+    first: Arc<AtomicU64>,
+    updates: Arc<AtomicU64>,
+    counting: JoinHandle<()>,
+    keepalives: JoinHandle<()>,
+}
+
+impl Receiver {
+    /// A session with the router at `addr` from [`RECEIVER`], as a
+    /// collector of every path; or `None`, with a word, where that address
+    /// is not this machine's to connect from.
+    async fn connect(addr: SocketAddr) -> Option<TestPeer> {
+        let socket = tokio::net::TcpSocket::new_v4().expect("a socket");
+        if let Err(e) = socket.bind(SocketAddr::new(RECEIVER.into(), 0)) {
+            println!("A new peer taking A's routes: skipped, {RECEIVER} is not ours ({e})");
+            return None;
+        }
+        let stream = socket.connect(addr).await.expect("A listens");
+        let mut session = TestPeer::from_stream(stream);
+        session
+            .handshake_with(
+                ASN,
+                Ipv4Addr::new(192, 0, 2, 9),
+                90,
+                collector_capabilities(ASN, &SUPPORTED, false),
+            )
+            .await
+            .expect("A takes the receiver's session");
+        Some(session)
+    }
+
+    /// Starts counting what `session` is sent, from now. Every millionth
+    /// UPDATE is noted, as is a NOTIFICATION or the connection closing:
+    /// what went wrong, if the table never all arrives.
+    fn count_updates(session: TestPeer) -> Self {
+        let started = Instant::now();
+        let (reader, mut writer) = session.into_stream().into_split();
+        // KEEPALIVEs, so that the session outlives the hold time.
+        let keepalives = tokio::spawn(async move {
+            let keepalive = TestPeer::keepalive_bytes();
+            loop {
+                tokio::time::sleep(Duration::from_secs(20)).await;
+                if writer.write_all(&keepalive).await.is_err() {
+                    break;
+                }
+            }
+        });
+        let first = Arc::new(AtomicU64::new(0));
+        let updates = Arc::new(AtomicU64::new(0));
+        let counting = tokio::spawn({
+            let (first, updates) = (Arc::clone(&first), Arc::clone(&updates));
+            async move {
+                let mut reader = BufReader::with_capacity(1 << 20, reader);
+                let mut header = [0u8; 19];
+                let mut body = vec![0u8; 4096];
+                while reader.read_exact(&mut header).await.is_ok() {
+                    let length = usize::from(u16::from_be_bytes([header[16], header[17]]));
+                    if reader.read_exact(&mut body[..length - 19]).await.is_err() {
+                        break;
+                    }
+                    match header[18] {
+                        msg_type::UPDATE => {
+                            let _ = first.compare_exchange(
+                                0,
+                                started.elapsed().as_micros() as u64,
+                                Ordering::Relaxed,
+                                Ordering::Relaxed,
+                            );
+                            let n = updates.fetch_add(1, Ordering::Relaxed) + 1;
+                            if n % 1_000_000 == 0 {
+                                println!(
+                                    "  the receiver has read {n} UPDATEs after {:.2} s",
+                                    started.elapsed().as_secs_f64()
+                                );
+                            }
+                        }
+                        msg_type::NOTIFICATION => println!(
+                            "the receiver was sent a NOTIFICATION, error {} subcode {}, after {} UPDATEs",
+                            body[0],
+                            body[1],
+                            updates.load(Ordering::Relaxed)
+                        ),
+                        _ => {}
+                    }
+                }
+                println!(
+                    "the receiver's connection closed after {} UPDATEs",
+                    updates.load(Ordering::Relaxed)
+                );
+            }
+        });
+        Self {
+            started,
+            first,
+            updates,
+            counting,
+            keepalives,
+        }
+    }
+
+    /// Waits until the router whose sessions are in `registry` has
+    /// advertised `routes` routes to the receiver, and returns how many it
+    /// has; or panics once `deadline` has passed.
+    async fn wait_for_advertised(
+        &self,
+        registry: &SessionRegistry,
+        routes: usize,
+        deadline: Duration,
+    ) -> u64 {
+        loop {
+            tokio::time::sleep(POLL).await;
+            let advertised = registry
+                .list()
+                .iter()
+                .find(|s| s.peer.addr.ip() == RECEIVER)
+                .map_or(0, |s| s.counters.advertised());
+            if advertised as usize >= routes {
+                return advertised;
+            }
+            if self.started.elapsed() > deadline {
+                panic!(
+                    "the receiver has {advertised} of {routes} routes and read {} UPDATEs; sessions at A: {:#?}",
+                    self.updates.load(Ordering::Relaxed),
+                    sessions_report(registry)
+                );
+            }
+        }
+    }
+
+    /// Prints the row for the table being all there, `advertised` routes
+    /// of it, and stops counting.
+    fn report(self, advertised: u64) {
+        let took = self.started.elapsed();
+        self.counting.abort();
+        self.keepalives.abort();
+        let first = Duration::from_micros(self.first.load(Ordering::Relaxed));
+        row(
+            "A new peer has A's IPv4 routes (A's exporter)",
+            took,
+            format!(
+                "{:.2} M routes/s   first UPDATE after {:.0} ms, {} UPDATEs",
+                advertised as f64 / took.as_secs_f64() / 1e6,
+                first.as_secs_f64() * 1e3,
+                self.updates.load(Ordering::Relaxed),
+            ),
+        );
+    }
+}
+
 /// A new peer takes the table of the router at `addr`, whose sessions
 /// are in `registry`: `routes` routes with every path. How long until
 /// the first UPDATE, and until all of it is sent.
-async fn resync(addr: SocketAddr, registry: SessionRegistry, routes: usize) {
+async fn resync(addr: SocketAddr, registry: SessionRegistry, routes: usize, deadline: Duration) {
-    let socket = tokio::net::TcpSocket::new_v4().expect("a socket");
+    let Some(session) = Receiver::connect(addr).await else {
-    if let Err(e) = socket.bind(SocketAddr::new(RECEIVER.into(), 0)) {
-        println!("A new peer taking A's routes: skipped, {RECEIVER} is not ours ({e})");
         return;
-    }
-    let stream = socket.connect(addr).await.expect("A listens");
-    let mut session = TestPeer::from_stream(stream);
-    let mut capabilities = vec![Capability::FourByteAsn { asn: ASN }];
-    capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
-    capabilities.push(Capability::AddPath {
-        entries: SUPPORTED
-            .iter()
-            .map(|family| AddPathEntry {
-                afi: family.afi,
-                safi: family.safi,
-                send_recv: SendReceive::Receive,
-            })
-            .collect(),
-    });
-    session
-        .handshake_with(ASN, Ipv4Addr::new(192, 0, 2, 9), 90, capabilities)
-        .await
-        .expect("A takes the receiver's session");
-    let started = Instant::now();
-    let (reader, mut writer) = session.into_stream().into_split();
-    // KEEPALIVEs, so that the session outlives the hold time.
-    let keepalives = tokio::spawn(async move {
-        let mut keepalive = [0xffu8; 19];
-        keepalive[16..19].copy_from_slice(&[0, 19, 4]);
-        loop {
-            tokio::time::sleep(Duration::from_secs(20)).await;
-            if writer.write_all(&keepalive).await.is_err() {
-                break;
-            }
-        }
-    });
-    // When the first UPDATE arrived, in microseconds; 0 until it has.
-    let first = Arc::new(AtomicU64::new(0));
-    let updates = Arc::new(AtomicU64::new(0));
-    let counting = tokio::spawn({
-        let (first, updates) = (Arc::clone(&first), Arc::clone(&updates));
-        async move {
-            let mut reader = BufReader::with_capacity(1 << 20, reader);
-            let mut header = [0u8; 19];
-            let mut body = vec![0u8; 4096];
-            while reader.read_exact(&mut header).await.is_ok() {
-                let length = usize::from(u16::from_be_bytes([header[16], header[17]]));
-                if reader.read_exact(&mut body[..length - 19]).await.is_err() {
-                    break;
-                }
-                match header[18] {
-                    // An UPDATE (RFC 4271 §4.1).
-                    2 => {
-                        let _ = first.compare_exchange(
-                            0,
-                            started.elapsed().as_micros() as u64,
-                            Ordering::Relaxed,
-                            Ordering::Relaxed,
-                        );
-                        let n = updates.fetch_add(1, Ordering::Relaxed) + 1;
-                        if n % 1_000_000 == 0 {
-                            println!(
-                                "  the receiver has read {n} UPDATEs after {:.2} s",
-                                started.elapsed().as_secs_f64()
-                            );
-                        }
-                    }
-                    3 => println!(
-                        "the receiver was sent a NOTIFICATION, error {} subcode {}, after {} UPDATEs",
-                        body[0],
-                        body[1],
-                        updates.load(Ordering::Relaxed)
-                    ),
-                    _ => {}
-                }
-            }
-            println!(
-                "the receiver's connection closed after {} UPDATEs",
-                updates.load(Ordering::Relaxed)
-            );
-        }
-    });
-    let deadline = Duration::from_secs(env("BENCH_DEADLINE", 600) as u64);
-    let advertised = loop {
-        tokio::time::sleep(Duration::from_millis(5)).await;
-        let advertised = registry
-            .list()
-            .iter()
-            .find(|s| s.peer.addr.ip() == RECEIVER)
-            .map_or(0, |s| s.counters.advertised());
-        if advertised as usize >= routes {
-            break advertised;
-        }
-        if started.elapsed() > deadline {
-            let sessions: Vec<String> = registry
-                .list()
-                .iter()
-                .map(|s| {
-                    format!(
-                        "{} {:?} export {:?} add_path_send {:?} advertised {}",
-                        s.peer,
-                        s.name,
-                        s.export,
-                        s.add_path_send,
-                        s.counters.advertised()
-                    )
-                })
-                .collect();
-            panic!(
-                "the receiver has {advertised} of {routes} routes and read {} UPDATEs; sessions at A: {sessions:#?}",
-                updates.load(Ordering::Relaxed)
-            );
-        }
     };
-    let took = started.elapsed();
+    let receiver = Receiver::count_updates(session);
-    counting.abort();
+    let advertised = receiver
-    keepalives.abort();
+        .wait_for_advertised(&registry, routes, deadline)
-    let first = Duration::from_micros(first.load(Ordering::Relaxed));
+        .await;
-    println!(
+    receiver.report(advertised);
-        "{:<52} {:>8.3} s   {:.2} M routes/s   first UPDATE after {:.0} ms, {} UPDATEs",
-        "A new peer has A's IPv4 routes (A's exporter)",
-        took.as_secs_f64(),
-        advertised as f64 / took.as_secs_f64() / 1e6,
-        first.as_secs_f64() * 1e3,
-        updates.load(Ordering::Relaxed),
-    );
 }

crates/bench/benches/real_table.rs

@@ -11,30 +11,20 @@ use std::net::Ipv4Addr;
 use std::path::PathBuf;
 use std::sync::Arc;
 use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
-use std::time::{Duration, Instant};
+use std::time::Instant;
 
 use bgp_bench::memory::Region;
 use bgp_bench::mrt::{Dump, parse_attributes};
+use bgp_bench::report::{mib, per_second, row};
 use bgp_packet::message::{NlriNextHop, PathAttribute};
 use bgp_primitives::ip_prefix::IpPrefix;
 use bgp_rib::table::{Event, PeerIndex, PeerInfo, RibTable};
 
 /// Bytes allocated and not freed since `region` began; zero when memory
-/// is not being counted (see `bgp_bench::memory`).
+/// is not being counted (see `bgp_bench::memory`), and zero too in the
-fn retained(region: &Region) -> i64 {
+/// odd case of more freed than allocated, which is nothing held.
-    region.held().unwrap_or(0)
+fn retained(region: &Region) -> usize {
-}
+    usize::try_from(region.held().unwrap_or(0)).unwrap_or(0)
-
-fn mib(bytes: i64) -> f64 {
-    bytes as f64 / (1024.0 * 1024.0)
-}
-
-fn per_second(count: usize, elapsed: Duration) -> String {
-    format!("{:.2} M/s", count as f64 / elapsed.as_secs_f64() / 1e6)
-}
-
-fn row(what: &str, elapsed: Duration, detail: String) {
-    println!("{what:<44} {:>9.3} s   {detail}", elapsed.as_secs_f64());
 }
 
 fn dump_path() -> Option<PathBuf> {

crates/bench/examples/feed.rs

@@ -32,32 +32,18 @@
 use std::net::{Ipv4Addr, SocketAddr};
 use std::time::{Duration, Instant};
 
-use bgp_bench::{Prefixes, Shape, announcements, encoded, env};
+use bgp_bench::report::mib;
-use bgp_integration_tests::test_peer::TestPeer;
+use bgp_bench::{Prefixes, Shape, env, table_stream};
-use bgp_packet::open::{AddPathEntry, Capability, ExtendedNextHopEntry, SendReceive};
+use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
-use bgp_primitives::constants::{Afi, AfiSafi, Safi};
+use bgp_primitives::constants::AfiSafi;
-use bgp_server::family::SUPPORTED;
+use tokio::io::AsyncWriteExt;
-use tokio::io::{AsyncReadExt, AsyncWriteExt};
 use tokio::net::TcpSocket;
 use tokio::net::tcp::OwnedWriteHalf;
 
-/// A KEEPALIVE message: the marker, the length (19) and the type (4).
-const KEEPALIVE: [u8; 19] = {
-    let mut bytes = [0xff; 19];
-    bytes[16] = 0;
-    bytes[17] = 19;
-    bytes[18] = 4;
-    bytes
-};
-
 /// How often KEEPALIVEs go out while the sessions are held open: well
 /// inside the 90 s hold time.
 const KEEPALIVE_EVERY: Duration = Duration::from_secs(20);
 
-fn mib(bytes: usize) -> f64 {
-    bytes as f64 / (1024.0 * 1024.0)
-}
-
 #[tokio::main]
 async fn main() {
     let to: SocketAddr = std::env::var("FEED_TO")
@@ -73,12 +59,7 @@ async fn main() {
     let started = Instant::now();
     let prefixes = Prefixes::generate(shape);
     let tables: Vec<Vec<u8>> = (1..=peers)
-        .map(|peer| {
+        .map(|peer| table_stream(&prefixes, shape, peer))
-            encoded(&announcements(&prefixes, shape, peer))
-                .iter()
-                .flat_map(|message| message.iter().copied())
-                .collect()
-        })
         .collect();
     let routes = shape.prefixes() * peers as usize;
     let bytes: usize = tables.iter().map(Vec::len).sum();
@@ -109,38 +90,21 @@ async fn main() {
         }
         let stream = socket.connect(to).await.expect("the daemon listens");
         let mut session = TestPeer::from_stream(stream);
-        let mut capabilities = vec![Capability::FourByteAsn { asn: 212_232 }];
-        capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
-        capabilities.push(Capability::AddPath {
-            entries: [AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST]
-                .into_iter()
-                .map(|family| AddPathEntry {
-                    afi: family.afi,
-                    safi: family.safi,
-                    send_recv: SendReceive::Receive,
-                })
-                .collect(),
-        });
-        capabilities.push(Capability::ExtendedNextHop {
-            entries: vec![ExtendedNextHopEntry {
-                afi: Afi::IPV4,
-                safi: Safi::UNICAST,
-                nh_afi: Afi::IPV6,
-            }],
-        });
         session
-            .handshake_with(212_232, Ipv4Addr::new(10, 0, 0, 200), 90, capabilities)
+            .handshake_with(
+                212_232,
+                Ipv4Addr::new(10, 0, 0, 200),
+                90,
+                collector_capabilities(
+                    212_232,
+                    &[AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST],
+                    true,
+                ),
+            )
             .await
             .expect("the daemon takes the collector's session");
-        let (mut reader, writer) = session.into_stream().into_split();
+        let (writer, sink) = session.drain();
-        draining.push(tokio::spawn(async move {
+        draining.push(sink);
-            let mut sink = [0u8; 65536];
-            while let Ok(n) = reader.read(&mut sink).await {
-                if n == 0 {
-                    break;
-                }
-            }
-        }));
         println!("collector session established");
         collector_writer = Some(writer);
     }
@@ -166,7 +130,11 @@ async fn main() {
             )
             .await
             .expect("the daemon takes the session");
-        sessions.push(session.into_stream());
+        // Whatever the daemon sends back, KEEPALIVEs as a rule, is read
+        // and dropped, so that the writer is never held up.
+        let (writer, sink) = session.drain();
+        draining.push(sink);
+        sessions.push(writer);
     }
     println!("{peers} session(s) established; sending");
 
@@ -174,19 +142,8 @@ async fn main() {
     let senders: Vec<_> = sessions
         .into_iter()
         .zip(tables)
-        .map(|(stream, table)| {
+        .map(|(mut writer, table)| {
             tokio::spawn(async move {
-                let (mut reader, mut writer) = stream.into_split();
-                // Whatever the daemon sends back, KEEPALIVEs as a rule, is
-                // read and dropped, so that its writer is never held up.
-                tokio::spawn(async move {
-                    let mut sink = [0u8; 65536];
-                    while let Ok(n) = reader.read(&mut sink).await {
-                        if n == 0 {
-                            break;
-                        }
-                    }
-                });
                 let started = Instant::now();
                 writer.write_all(&table).await.expect("the daemon reads");
                 (writer, started.elapsed())
@@ -210,12 +167,13 @@ async fn main() {
         routes as f64 / elapsed.as_secs_f64() / 1e6
     );
 
+    let keepalive = TestPeer::keepalive_bytes();
     let deadline = Instant::now() + hold;
     while Instant::now() < deadline {
         tokio::time::sleep(KEEPALIVE_EVERY.min(deadline - Instant::now())).await;
         for writer in &mut writers {
             // A session the daemon closed is left alone.
-            let _ = writer.write_all(&KEEPALIVE).await;
+            let _ = writer.write_all(&keepalive).await;
         }
     }
     println!("sessions closed after {} s", hold.as_secs());

crates/bench/src/lib.rs

@@ -16,6 +16,7 @@
 
 pub mod memory;
 pub mod mrt;
+pub mod report;
 
 // The benchmarks allocate as the daemon does (bin/bgp/src/main.rs), with
 // jemalloc; the C library's allocator would put its own locks in the
@@ -266,6 +267,16 @@ pub fn announcements(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<Update
     updates
 }
 
+/// The whole table as the peer numbered `peer` sends it, one stream of
+/// bytes: [`announcements`], [`encoded`] and run together, for a session
+/// that writes the table in one go.
+pub fn table_stream(prefixes: &Prefixes, shape: Shape, peer: u32) -> Vec<u8> {
+    encoded(&announcements(prefixes, shape, peer))
+        .iter()
+        .flat_map(|message| message.iter().copied())
+        .collect()
+}
+
 /// `updates` as they are on the wire, on a session with four-octet AS
 /// numbers.
 pub fn encoded(updates: &[UpdateMessage]) -> Vec<BytesMut> {
@@ -300,3 +311,68 @@ pub fn env(name: &str, default: usize) -> usize {
         .and_then(|v| v.parse().ok())
         .unwrap_or(default)
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// A small table, enough for every prefix length and attribute to
+    /// come up.
+    fn shape() -> Shape {
+        Shape::INTERNET.scaled_down(1000)
+    }
+
+    /// The benchmarks are compared from one run to the next, so the table
+    /// they are fed must not change between runs: the generator is
+    /// seeded, and nothing in it depends on where things are in memory.
+    #[test]
+    fn the_table_is_the_same_every_run() {
+        let shape = shape();
+        let once = Prefixes::generate(shape);
+        let again = Prefixes::generate(shape);
+        assert_eq!(once.ipv4, again.ipv4);
+        assert_eq!(once.ipv6, again.ipv6);
+        assert_eq!(once.ipv4.len(), shape.ipv4_prefixes);
+        assert_eq!(once.ipv6.len(), shape.ipv6_prefixes);
+        for peer in [1, 2] {
+            assert_eq!(
+                table_stream(&once, shape, peer),
+                table_stream(&again, shape, peer)
+            );
+        }
+        assert_ne!(
+            table_stream(&once, shape, 1),
+            table_stream(&once, shape, 2),
+            "each peer has attributes of its own"
+        );
+    }
+
+    /// What a peer of the benchmarks sends is well formed, and says what
+    /// it was meant to: every UPDATE reads back as it was made.
+    #[test]
+    fn every_announcement_decodes_to_itself() {
+        let shape = shape();
+        let prefixes = Prefixes::generate(shape);
+        let updates = announcements(&prefixes, shape, 1);
+        let ctx = wire_context();
+        let announced: usize = updates
+            .iter()
+            .map(|update| {
+                update.announced_routes.len()
+                    + update
+                        .path_attributes
+                        .iter()
+                        .map(|attribute| match attribute {
+                            PathAttribute::MpReachNlri(reach) => reach.prefixes.len(),
+                            _ => 0,
+                        })
+                        .sum::<usize>()
+            })
+            .sum();
+        assert_eq!(announced, shape.prefixes());
+        for (update, wire) in updates.iter().zip(encoded(&updates)) {
+            let decoded = Message::decode(&ctx, &wire).expect("well formed");
+            assert_eq!(decoded, Message::Update(update.clone()));
+        }
+    }
+}

crates/bench/src/mrt.rs

@@ -10,7 +10,7 @@ use std::io::{self, BufReader, Read};
 use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
 use std::path::Path;
 
-use bgp_packet::message::{NlriNextHop, PathAttribute};
+use bgp_packet::message::{NlriNextHop, PathAttribute, attr_type, split_attribute};
 use bgp_primitives::constants::IpVersion;
 use bgp_primitives::ip_prefix::IpPrefix;
 use flate2::read::MultiGzDecoder;
@@ -21,7 +21,6 @@ const TABLE_DUMP_V2: u16 = 13;
 const PEER_INDEX_TABLE: u16 = 1;
 const RIB_IPV4_UNICAST: u16 = 2;
 const RIB_IPV6_UNICAST: u16 = 4;
-const MP_REACH_NLRI: u8 = 14;
 
 /// A peer of the collector.
 #[derive(Debug, Clone)]
@@ -205,32 +204,23 @@ pub fn parse_attributes(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<N
     let mut attributes = Vec::new();
     let mut next_hop = None;
     while !wire.is_empty() {
-        if wire.get(1) == Some(&MP_REACH_NLRI) {
+        let (raw, rest) = split_attribute(wire).ok()?;
-            let extended = wire[0] & 0x10 != 0;
+        if raw.type_code == attr_type::MP_REACH_NLRI {
-            let (length, header) = if extended {
+            // The value is the next hop's length and the next hop, and
-                (
+            // nothing of the family, reserved octet and prefixes that
-                    usize::from(u16::from_be_bytes([*wire.get(2)?, *wire.get(3)?])),
+            // follow in an UPDATE, so the attribute's own parser would
-                    4,
+            // refuse it.
-                )
+            let (length, address) = raw.value.split_first()?;
-            } else {
+            if usize::from(*length) != address.len() {
-                (usize::from(*wire.get(2)?), 3)
+                return None;
+            }
+            next_hop = Some(NlriNextHop::from_bytes(address)?);
+        } else {
+            let ([], attribute) = PathAttribute::from_wire(&ctx, raw.wire).ok()? else {
+                return None;
             };
-            let value = wire.get(header..header + length)?;
+            attributes.push(attribute);
-            let address = |bytes: &[u8]| Some(Ipv6Addr::from(<[u8; 16]>::try_from(bytes).ok()?));
-            next_hop = match value {
-                [4, a, b, c, d] => Some(NlriNextHop::Ipv4(Ipv4Addr::new(*a, *b, *c, *d))),
-                [16, rest @ ..] => Some(NlriNextHop::Ipv6(address(rest)?)),
-                [32, rest @ ..] if rest.len() == 32 => Some(NlriNextHop::Ipv6WithLl {
-                    global: address(&rest[..16])?,
-                    link_local: address(&rest[16..])?,
-                }),
-                _ => return None,
-            };
-            wire = &wire[header + length..];
-            continue;
         }
-        let (rest, attribute) = PathAttribute::from_wire(&ctx, wire).ok()?;
-        attributes.push(attribute);
         wire = rest;
     }
     Some((attributes, next_hop))

crates/bench/src/report.rs

@@ -0,0 +1,31 @@
+//! How the benchmarks print what they measured: one row a measurement,
+//! what was measured in a column wide enough for the longest of them, the
+//! time, and whatever else there is to say about it. The rows are what a
+//! run is compared against, so their shape is settled here once.
+
+use std::time::Duration;
+
+/// How wide the column that says what was measured is. The widest row is
+/// the mesh benchmark's, so that is what every benchmark lines up to.
+pub const WIDTH: usize = 52;
+
+/// `bytes` in mebibytes.
+pub fn mib(bytes: usize) -> f64 {
+    bytes as f64 / (1024.0 * 1024.0)
+}
+
+/// `count` things in `elapsed`, as a rate: millions a second where there
+/// are that many, thousands otherwise.
+pub fn per_second(count: usize, elapsed: Duration) -> String {
+    let rate = count as f64 / elapsed.as_secs_f64();
+    if rate >= 1e6 {
+        format!("{:.2} M/s", rate / 1e6)
+    } else {
+        format!("{:.0} k/s", rate / 1e3)
+    }
+}
+
+/// Prints one row: `what` took `elapsed`, and `detail` says more.
+pub fn row(what: &str, elapsed: Duration, detail: impl std::fmt::Display) {
+    println!("{what:<WIDTH$} {:>8.3} s   {detail}", elapsed.as_secs_f64());
+}

crates/fib/src/kernel.rs

@@ -498,9 +498,53 @@ struct Task {
     /// them means looking at every route, and is not done once there are
     /// none to find.
     keeping: bool,
+    /// What `follow` takes out of the table, and what it makes of it. Kept
+    /// from one wake to the next so that their room is not found anew each
+    /// time the table has something, which is many times a second while
+    /// it fills.
+    news: Vec<News>,
+    ops: Vec<Op>,
 }
 
 impl Task {
+    /// A task that is to keep the kernel in step with `rib`, talking to it
+    /// over `control` and `resolution` and writing routes over `routes`
+    /// from a task of its own. Returned with it is where that task
+    /// reports what became of the routes.
+    fn new(
+        rib: Arc<RibTable>,
+        fib: Fib,
+        control: Socket,
+        routes: Socket,
+        resolution: Option<Socket>,
+    ) -> (Self, mpsc::UnboundedReceiver<Report>) {
+        let (queue, batches) = mpsc::channel(QUEUE);
+        let (reports, heard) = mpsc::unbounded_channel();
+        let config = fib.config();
+        let task = Self {
+            config,
+            rib,
+            fib,
+            control,
+            resolution,
+            queue: Some(queue),
+            writer: tokio::spawn(write(routes, batches, reports)),
+            topology: Topology::new(config.resolution_table, config.resolve_via_default),
+            resolved: FastHashMap::default(),
+            plan: Plan::default(),
+            nexthops: config.nexthop_objects.then(FastHashMap::default),
+            owned: HashSet::new(),
+            awaiting: None,
+            marks: 0,
+            synced: false,
+            mended_at: None,
+            keeping: false,
+            news: Vec::with_capacity(BATCH),
+            ops: Vec::with_capacity(BATCH),
+        };
+        (task, heard)
+    }
+
     /// Whether a route the kernel has is one of ours.
     fn is_ours(&self, route: &Route) -> bool {
         route.protocol == PROTOCOL_BGP
@@ -631,6 +675,19 @@ impl Task {
         Ok(true)
     }
 
+    /// Learns what the kernel has that bears on our routes: the routes
+    /// that say the way to next hops, its nexthop objects, and the routes
+    /// of ours it has from before, which are kept for the peers to announce
+    /// again if `keep` says so. Done at the start, and again whenever what
+    /// the kernel has is no longer known.
+    async fn learn(&mut self, keep: bool) -> Result<(), FibError> {
+        self.learn_routes().await?;
+        let nexthops = self.kernel_nexthops().await?;
+        self.adopt_routes(&nexthops, keep).await?;
+        self.use_nexthops(&nexthops);
+        Ok(())
+    }
+
     /// Finds the routes of ours that the kernel has from before.
     async fn adopt_routes(
         &mut self,
@@ -751,17 +808,21 @@ impl Task {
         Ok(Ok((change, route)))
     }
 
-    /// Hands `ops` over to be written to the kernel, in order. With
+    /// Hands `ops` over to be written to the kernel, in order, and leaves
-    /// `whole_table` they complete a whole table, and the kernel is in
+    /// the vector empty for the caller to fill again. With `whole_table`
-    /// step once it has done them.
+    /// they complete a whole table, and the kernel is in step once it has
-    async fn send(&mut self, ops: Vec<Op>, whole_table: bool) -> Result<(), FibError> {
+    /// done them.
+    async fn send(&mut self, ops: &mut Vec<Op>, whole_table: bool) -> Result<(), FibError> {
         let mut batch = Batch {
             ops: Vec::with_capacity(ops.len()),
             requests: Vec::with_capacity(ops.len()),
             mark: None,
         };
-        let mut ops = std::collections::VecDeque::from(ops);
+        // By place rather than by draining: what takes a refused op's
-        while let Some(op) = ops.pop_front() {
+        // place is put at the end, and is dealt with in its turn.
+        let mut at = 0;
+        while let Some(op) = ops.get(at).copied() {
+            at += 1;
             match self.request(&op).await? {
                 Ok(request) => {
                     CHANGES
@@ -777,6 +838,7 @@ impl Task {
                 Err(errno) => ops.extend(self.refused(&op, errno)?),
             }
         }
+        ops.clear();
         if whole_table {
             self.marks += 1;
             batch.mark = Some(self.marks);
@@ -850,7 +912,12 @@ impl Task {
     /// hands over what brings the kernel in step with them. Returns whether
     /// the reader has read everything the table has for it.
     async fn follow(&mut self, reader: &mut Reader) -> Result<bool, FibError> {
-        let mut news = Vec::with_capacity(BATCH);
+        // Taken out of `self` while they are worked through, since working
+        // through them is what the rest of `self` is for.
+        let mut news = std::mem::take(&mut self.news);
+        let mut ops = std::mem::take(&mut self.ops);
+        news.clear();
+        ops.clear();
         // Only what is needed is taken while the table is held.
         let progress = reader.poll(BATCH, |event| {
             news.push(match event {
@@ -872,9 +939,8 @@ impl Task {
                 Event::Routes { .. } => return,
             });
         });
-        let mut ops = Vec::with_capacity(news.len());
         let mut whole_table = false;
-        for item in news {
+        for item in news.drain(..) {
             match item {
                 News::Reset => {
                     self.plan.reset();
@@ -928,7 +994,10 @@ impl Task {
                 News::Withdrawn(prefix) => ops.extend(self.plan.withdrawn(prefix)),
             }
         }
-        self.send(ops, whole_table).await?;
+        let sent = self.send(&mut ops, whole_table).await;
+        self.news = news;
+        self.ops = ops;
+        sent?;
         Ok(progress.caught_up)
     }
 
@@ -957,7 +1026,7 @@ impl Task {
             }
             Report::Refused { op, errno } => {
                 let instead = self.refused(&op, errno)?;
-                self.send(instead.into_iter().collect(), false).await?;
+                self.send(&mut instead.into_iter().collect(), false).await?;
                 // A refused route may have named a nexthop object that
                 // went away. Working the routes out again mends that.
                 let due = matches!(op, Op::Install { .. })
@@ -1026,18 +1095,14 @@ impl Task {
         self.flush(heard).await?;
         self.synced = false;
         self.plan = Plan::default();
-        self.learn_routes().await?;
-        let nexthops = self.kernel_nexthops().await?;
         // While routes of an earlier run are being kept for the peers to
         // announce again, they are kept through this too.
-        self.adopt_routes(&nexthops, self.keeping).await?;
+        self.learn(self.keeping).await
-        self.use_nexthops(&nexthops);
-        Ok(())
     }
 
     /// Waits until the kernel has done everything handed over so far.
     async fn flush(&mut self, heard: &mut mpsc::UnboundedReceiver<Report>) -> Result<(), FibError> {
-        self.send(vec![], true).await?;
+        self.send(&mut Vec::new(), true).await?;
         while let Some(mark) = self.awaiting {
             match heard.recv().await {
                 Some(Report::Done(done)) if done == mark => self.awaiting = None,
@@ -1090,8 +1155,8 @@ impl Task {
             self.plan = Plan::default();
             self.adopt_routes(&HashMap::new(), false).await?;
         }
-        let ops = self.plan.clear();
+        let mut ops = self.plan.clear();
-        self.send(ops, false).await?;
+        self.send(&mut ops, false).await?;
         self.finish_writer().await
     }
 
@@ -1153,38 +1218,15 @@ pub(crate) async fn run(
         resolution,
         mut hints,
     } = sockets;
-    let (queue, batches) = mpsc::channel(QUEUE);
+    let (mut task, mut heard) = Task::new(rib, fib, control, routes, resolution);
-    let (reports, mut heard) = mpsc::unbounded_channel();
+    let config = task.config;
-    let config = fib.config();
-    let mut task = Task {
-        config,
-        rib,
-        fib,
-        control,
-        resolution,
-        queue: Some(queue),
-        writer: tokio::spawn(write(routes, batches, reports)),
-        topology: Topology::new(config.resolution_table, config.resolve_via_default),
-        resolved: FastHashMap::default(),
-        plan: Plan::default(),
-        nexthops: config.nexthop_objects.then(FastHashMap::default),
-        owned: HashSet::new(),
-        awaiting: None,
-        marks: 0,
-        synced: false,
-        mended_at: None,
-        keeping: false,
-    };
     // The socket for news was bound before this: what changes while the
     // interfaces are being learned is told there.
     task.learn_interfaces().await?;
-    task.learn_routes().await?;
-    let nexthops = task.kernel_nexthops().await?;
     // Kept for a while, if there is to be any keeping: our table is empty
     // until the peers have sent theirs again.
     let keep = !config.stale_time.is_zero();
-    task.adopt_routes(&nexthops, keep).await?;
+    task.learn(keep).await?;
-    task.use_nexthops(&nexthops);
     let mut release_at = None;
     if task.plan.installed() > 0 {
         info!(
@@ -1284,12 +1326,12 @@ pub(crate) async fn run(
                 // taken out only to be put back.
                 while !task.follow(&mut reader).await? {}
                 task.keeping = false;
-                let stale = task.plan.release();
+                let mut stale = task.plan.release();
                 info!(
                     removed = stale.len(),
                     "Routes of an earlier run that were not announced again are removed"
                 );
-                task.send(stale, false).await?;
+                task.send(&mut stale, false).await?;
                 task.publish(false);
             }
             ended = &mut task.writer => return Err(writer_error(ended)),

crates/fib/src/netlink.rs

@@ -29,9 +29,13 @@ pub const KIND_UNICAST: u8 = 1;
 /// `RTN_BLACKHOLE`: a route that discards the packets for its prefix.
 pub const KIND_BLACKHOLE: u8 = 6;
 
-const HEADER_LEN: usize = 16;
+/// The netlink header: length, type, flags, sequence number and the
+/// sender's port, four octets each but for the two-octet type and flags.
+/// The socket diagnostics family (`sockdiag`) frames its messages the
+/// same way.
+pub(crate) const HEADER_LEN: usize = 16;
 
-mod message_type {
+pub(crate) mod message_type {
     pub const ERROR: u16 = 2;
     pub const DONE: u16 = 3;
     pub const NEW_LINK: u16 = 16;
@@ -48,7 +52,7 @@ mod message_type {
     pub const GET_NEXTHOP: u16 = 106;
 }
 
-mod flag {
+pub(crate) mod flag {
     pub const REQUEST: u16 = 0x1;
     pub const ACK: u16 = 0x4;
     /// Send the sender what everyone else is told of the change.
@@ -76,8 +80,8 @@ pub mod group {
 /// joined once the socket is bound.
 pub const GROUP_NEXTHOP: u32 = 32;
 
-const AF_INET: u8 = 2;
+pub(crate) const AF_INET: u8 = 2;
-const AF_INET6: u8 = 10;
+pub(crate) const AF_INET6: u8 = 10;
 
 /// `RT_TABLE_COMPAT`: the table's number is in an attribute, being too
 /// large for the octet the fixed part has for it.
@@ -219,22 +223,31 @@ pub enum NetlinkError {
     Truncated { message_type: u16 },
 }
 
-fn ne_u16(bytes: &[u8]) -> u16 {
+// The kernel writes its numbers in the machine's byte order. Each reader
+// takes the first octets of what it is given, so that a caller can pass
+// the rest of a buffer rather than cut the field out first.
+pub(crate) fn ne_u16(bytes: &[u8]) -> u16 {
     u16::from_ne_bytes([bytes[0], bytes[1]])
 }
 
-fn ne_u32(bytes: &[u8]) -> u32 {
+pub(crate) fn ne_u32(bytes: &[u8]) -> u32 {
     u32::from_ne_bytes([bytes[0], bytes[1], bytes[2], bytes[3]])
 }
 
+pub(crate) fn ne_u64(bytes: &[u8]) -> u64 {
+    u64::from_ne_bytes([
+        bytes[0], bytes[1], bytes[2], bytes[3], bytes[4], bytes[5], bytes[6], bytes[7],
+    ])
+}
+
 /// Rounds a length up to the four octets everything is aligned to.
-fn aligned(length: usize) -> usize {
+pub(crate) fn aligned(length: usize) -> usize {
     (length + 3) & !3
 }
 
 /// The attributes in `bytes`: each its type and value. Stops at what is
 /// not an attribute.
-fn attributes(mut bytes: &[u8]) -> impl Iterator<Item = (u16, &[u8])> {
+pub(crate) fn attributes(mut bytes: &[u8]) -> impl Iterator<Item = (u16, &[u8])> {
     std::iter::from_fn(move || {
         let header = bytes.first_chunk::<4>()?;
         let length = usize::from(ne_u16(header));

crates/fib/src/sockdiag.rs

@@ -7,6 +7,10 @@
 use std::io;
 use std::net::{IpAddr, SocketAddr};
 
+use crate::netlink::{
+    AF_INET, AF_INET6, HEADER_LEN, attributes, flag, message_type, ne_u16, ne_u32, ne_u64,
+};
+
 /// A TCP connection as the kernel sees it. Times are in microseconds,
 /// sizes in octets, rates in octets per second, unless said otherwise.
 /// Fields the running kernel does not report are zero.
@@ -90,17 +94,12 @@ impl TcpConnection {
 }
 
 const SOCK_DIAG_BY_FAMILY: u16 = 20;
-const NLMSG_ERROR: u16 = 2;
-const NLM_F_REQUEST: u16 = 1;
 const IPPROTO_TCP: u8 = 6;
-const AF_INET: u8 = 2;
-const AF_INET6: u8 = 10;
 const INET_DIAG_INFO: u16 = 2;
 const INET_DIAG_CONG: u16 = 4;
 /// `idiag_ext` bits asking for those two.
 const EXTENSIONS: u8 = (1 << (INET_DIAG_INFO - 1)) | (1 << (INET_DIAG_CONG - 1));
 const NO_COOKIE: u32 = u32::MAX;
-const HEADER_LEN: usize = 16;
 const REQUEST_LEN: usize = HEADER_LEN + 56;
 const DIAG_MSG_LEN: usize = 72;
 
@@ -113,7 +112,7 @@ pub fn request(local: SocketAddr, peer: SocketAddr, sequence: u32) -> Vec<u8> {
     let mut buf = Vec::with_capacity(REQUEST_LEN);
     buf.extend_from_slice(&(REQUEST_LEN as u32).to_ne_bytes());
     buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
-    buf.extend_from_slice(&NLM_F_REQUEST.to_ne_bytes());
+    buf.extend_from_slice(&flag::REQUEST.to_ne_bytes());
     buf.extend_from_slice(&sequence.to_ne_bytes());
     buf.extend_from_slice(&0u32.to_ne_bytes());
     // inet_diag_req_v2
@@ -162,11 +161,11 @@ pub fn parse(buf: &[u8]) -> io::Result<Answer> {
         .get(..HEADER_LEN)
         .ok_or_else(|| io::Error::other("short netlink answer"))?;
     let length = ne_u32(&header[0..4]) as usize;
-    let message_type = ne_u16(&header[4..6]);
+    let kind = ne_u16(&header[4..6]);
     let message = buf
         .get(HEADER_LEN..length.min(buf.len()))
         .ok_or_else(|| io::Error::other("short netlink answer"))?;
-    if message_type == NLMSG_ERROR {
+    if kind == message_type::ERROR {
         let errno = -(message
             .first_chunk::<4>()
             .map_or(0, |e| i32::from_ne_bytes(*e)));
@@ -177,7 +176,7 @@ pub fn parse(buf: &[u8]) -> io::Result<Answer> {
             Answer::Refused(errno)
         });
     }
-    if message_type != SOCK_DIAG_BY_FAMILY || message.len() < DIAG_MSG_LEN {
+    if kind != SOCK_DIAG_BY_FAMILY || message.len() < DIAG_MSG_LEN {
         return Err(io::Error::other("not a socket diagnostic"));
     }
     let mut connection = TcpConnection {
@@ -186,14 +185,7 @@ pub fn parse(buf: &[u8]) -> io::Result<Answer> {
         send_queue: ne_u32(&message[60..64]),
         ..TcpConnection::default()
     };
-    let mut attributes = &message[DIAG_MSG_LEN..];
+    for (kind, value) in attributes(&message[DIAG_MSG_LEN..]) {
-    while attributes.len() >= 4 {
-        let length = usize::from(ne_u16(&attributes[0..2]));
-        let kind = ne_u16(&attributes[2..4]);
-        if length < 4 || length > attributes.len() {
-            break;
-        }
-        let value = &attributes[4..length];
         match kind {
             INET_DIAG_INFO => fill_from_tcp_info(&mut connection, value),
             INET_DIAG_CONG => {
@@ -202,7 +194,6 @@ pub fn parse(buf: &[u8]) -> io::Result<Answer> {
             }
             _ => {}
         }
-        attributes = &attributes[(length + 3) & !3..];
     }
     Ok(Answer::Connection(Box::new(connection)))
 }
@@ -250,20 +241,6 @@ fn fill_from_tcp_info(c: &mut TcpConnection, info: &[u8]) {
     c.receive_window = u32_at(232);
 }
 
-fn ne_u16(bytes: &[u8]) -> u16 {
-    u16::from_ne_bytes([bytes[0], bytes[1]])
-}
-
-fn ne_u32(bytes: &[u8]) -> u32 {
-    u32::from_ne_bytes([bytes[0], bytes[1], bytes[2], bytes[3]])
-}
-
-fn ne_u64(bytes: &[u8]) -> u64 {
-    let mut array = [0u8; 8];
-    array.copy_from_slice(bytes);
-    u64::from_ne_bytes(array)
-}
-
 /// Asks the kernel about the TCP connection from `local` to `peer`.
 /// `Ok(None)` if there is no such connection, or on an operating system
 /// that cannot be asked.
@@ -304,9 +281,10 @@ pub async fn tcp_connection(
 mod tests {
     use super::*;
 
-    /// A reply as the kernel writes it, for an established connection
+    /// A reply as the kernel writes it, for an established connection,
-    /// with a `tcp_info` of the full modern length.
+    /// with the first `info_len` octets of a `tcp_info` of the full
-    fn reply() -> Vec<u8> {
+    /// modern length: an older kernel sends a shorter one.
+    fn reply(info_len: usize) -> Vec<u8> {
         let mut info = vec![0u8; 248];
         info[0] = 1; // established
         info[6] = 0x7a; // snd_wscale 10, rcv_wscale 7
@@ -326,6 +304,7 @@ mod tests {
         put64(&mut info, 168, 1_000_000); // busy
         put32(&mut info, 228, 131072); // snd_wnd
         put32(&mut info, 232, 262144); // rcv_wnd
+        info.truncate(info_len);
         let cong = b"cubic\0";
         let mut message = vec![0u8; DIAG_MSG_LEN];
         message[1] = 1;
@@ -345,7 +324,9 @@ mod tests {
         let mut buf = Vec::new();
         buf.extend_from_slice(&(length as u32).to_ne_bytes());
         buf.extend_from_slice(&SOCK_DIAG_BY_FAMILY.to_ne_bytes());
-        buf.extend_from_slice(&[0, 0, 1, 0, 0, 0, 0, 0, 0, 0]);
+        buf.extend_from_slice(&0u16.to_ne_bytes()); // flags
+        buf.extend_from_slice(&1u32.to_ne_bytes()); // sequence
+        buf.extend_from_slice(&0u32.to_ne_bytes()); // port
         buf.extend_from_slice(&message);
         buf.extend_from_slice(&attrs);
         buf
@@ -353,7 +334,7 @@ mod tests {
 
     #[test]
     fn a_reply_is_read_field_by_field() {
-        let Answer::Connection(c) = parse(&reply()).unwrap() else {
+        let Answer::Connection(c) = parse(&reply(248)).unwrap() else {
             panic!("a connection");
         };
         assert_eq!(c.state_name(), "established");
@@ -367,12 +348,25 @@ mod tests {
         assert_eq!((c.send_window, c.receive_window), (131072, 262144));
     }
 
+    /// Linux 4.x sends 104 octets of `tcp_info`, up to `tcpi_total_retrans`:
+    /// what is there is read, and what came later is left at zero rather
+    /// than read from the attribute after it.
+    #[test]
+    fn a_short_tcp_info_from_an_old_kernel_leaves_later_fields_zero() {
+        let Answer::Connection(c) = parse(&reply(104)).unwrap() else {
+            panic!("a connection");
+        };
+        assert_eq!(c.congestion.as_deref(), Some("cubic"));
+        assert_eq!((c.mss, c.rtt, c.cwnd), (1448, 2300, 42));
+        assert_eq!((c.busy, c.send_window), (0, 0));
+    }
+
     #[test]
     fn no_such_connection_and_a_refusal_are_told_apart() {
         let error = |errno: i32| {
             let mut buf = vec![0u8; 20];
             buf[0..4].copy_from_slice(&20u32.to_ne_bytes());
-            buf[4..6].copy_from_slice(&NLMSG_ERROR.to_ne_bytes());
+            buf[4..6].copy_from_slice(&message_type::ERROR.to_ne_bytes());
             buf[16..20].copy_from_slice(&(-errno).to_ne_bytes());
             buf
         };

crates/integration-tests/src/test_peer.rs

@@ -2,16 +2,55 @@ use std::net::{Ipv4Addr, SocketAddr};
 use std::time::Duration;
 
 use bgp_packet::message::{BGP_HEADER_LEN, Message, NotificationMessage, UpdateMessage};
-use bgp_packet::open::{Capability, OpenMessage, OpenOption};
+use bgp_packet::open::{
+    AddPathEntry, Capability, ExtendedNextHopEntry, OpenMessage, OpenOption, SendReceive,
+};
 use bgp_primitives::constants::{Afi, AfiSafi, Safi};
 use bgp_primitives::parser::ParserContext;
 use bytes::BytesMut;
 use eyre::{Result, bail, eyre};
 use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::net::tcp::OwnedWriteHalf;
 use tokio::net::{TcpListener, TcpStream};
+use tokio::task::JoinHandle;
 
 use crate::TIMEOUT;
 
+/// The capabilities a route collector offers in its OPEN: four-octet AS
+/// numbers as `asn`, each of `families`, and to be sent several routes
+/// for a prefix (ADD-PATH, RFC 7911) in each of them; with
+/// `ipv4_over_ipv6`, IPv4 routes with IPv6 next hops too (RFC 8950), as a
+/// collector on an IPv6-only session wants. For the tests and benchmarks
+/// that play a collector taking a router's whole table.
+pub fn collector_capabilities(
+    asn: u32,
+    families: &[AfiSafi],
+    ipv4_over_ipv6: bool,
+) -> Vec<Capability> {
+    let mut capabilities = vec![Capability::FourByteAsn { asn }];
+    capabilities.extend(families.iter().copied().map(Capability::multiprotocol));
+    capabilities.push(Capability::AddPath {
+        entries: families
+            .iter()
+            .map(|family| AddPathEntry {
+                afi: family.afi,
+                safi: family.safi,
+                send_recv: SendReceive::Receive,
+            })
+            .collect(),
+    });
+    if ipv4_over_ipv6 {
+        capabilities.push(Capability::ExtendedNextHop {
+            entries: vec![ExtendedNextHopEntry {
+                afi: Afi::IPV4,
+                safi: Safi::UNICAST,
+                nh_afi: Afi::IPV6,
+            }],
+        });
+    }
+    capabilities
+}
+
 /// A minimal BGP peer for use in tests. Connects over TCP and can exchange
 /// framed BGP messages using the `bgp-packet` wire format.
 pub struct TestPeer {
@@ -57,6 +96,32 @@ impl TestPeer {
         self.stream
     }
 
+    /// The connection with a task reading and dropping whatever the server
+    /// sends over it, and the half that is left for writing: for a peer
+    /// that has nothing to hear, as one feeding a table or taking one by
+    /// the megabyte has not, and whose writer must never be held up by
+    /// what it has not read. The task ends with the connection, or when
+    /// its handle is dropped.
+    pub fn drain(self) -> (OwnedWriteHalf, JoinHandle<()>) {
+        let (mut reader, writer) = self.stream.into_split();
+        let sink = tokio::spawn(async move {
+            let mut sink = vec![0u8; 1 << 16];
+            while matches!(reader.read(&mut sink).await, Ok(n) if n > 0) {}
+        });
+        (writer, sink)
+    }
+
+    /// A KEEPALIVE as it goes on the wire, for whoever writes to the
+    /// connection without the framing and has to keep the session alive
+    /// past its hold time.
+    pub fn keepalive_bytes() -> Vec<u8> {
+        let mut wire = BytesMut::new();
+        Message::KeepAlive
+            .to_wire(&ParserContext::default(), &mut wire)
+            .expect("a KEEPALIVE is nineteen octets");
+        wire.to_vec()
+    }
+
     /// Our end of the connection: what the server sees the peer as.
     pub fn local_addr(&self) -> Result<SocketAddr> {
         Ok(self.stream.local_addr()?)
@@ -145,7 +210,9 @@ impl TestPeer {
     pub async fn recv_open(&mut self) -> Result<OpenMessage> {
         match self.recv().await? {
             Message::Open(o) => {
-                // Enable 4-octet ASN parsing if the peer advertised it.
+                // AS numbers are four octets wide only if both sides
+                // offered it (RFC 6793 §4.1): the server's offer is read
+                // here, ours was noted when the OPEN went out.
                 let four_byte = o.options.iter().any(|opt| {
                     matches!(opt, OpenOption::Capabilities(caps)
                         if caps.iter().any(|c| matches!(c, Capability::FourByteAsn { .. })))
@@ -243,7 +310,8 @@ impl TestPeer {
 
     async fn finish_handshake(&mut self) -> Result<OpenMessage> {
         let server_open = self.recv_open().await?;
-        // Expect server KEEPALIVE
+        // The server's KEEPALIVE says it accepted our OPEN; anything else
+        // is its refusal, or a server out of step.
         match self.recv().await? {
             Message::KeepAlive => {}
             other => {
@@ -253,12 +321,15 @@ impl TestPeer {
                 ));
             }
         }
-        // Send our KEEPALIVE
+        // Ours says the same of its OPEN, and the session is established.
         self.send(&Message::KeepAlive).await?;
         Ok(server_open)
     }
 
-    /// Send raw bytes without framing — useful for testing invalid input.
+    /// Writes `bytes` as they are, with no framing of its own: for a test
+    /// that sends what no message would be, and for the benchmarks, which
+    /// send a table encoded once as one stream rather than message by
+    /// message.
     pub async fn send_raw(&mut self, bytes: &[u8]) -> Result<()> {
         self.stream.write_all(bytes).await?;
         Ok(())

crates/integration-tests/tests/add_path.rs

@@ -7,16 +7,15 @@ use std::sync::Arc;
 use std::time::Duration;
 
 use bgp_integration_tests::announcement::{Announcement, withdrawal};
-use bgp_integration_tests::test_peer::TestPeer;
+use bgp_integration_tests::test_peer::{TestPeer, collector_capabilities};
 use bgp_integration_tests::test_server::TestServer;
 use bgp_integration_tests::{peer_config, prefix, wait_until};
 use bgp_packet::message::{
     AsPathAttribute, Message, MpReachNlriPathAttribute, NlriNextHop, OriginPathAttribute,
     PathAttribute, UpdateMessage,
 };
-use bgp_packet::open::{AddPathEntry, Capability, ExtendedNextHopEntry, OpenMessage, SendReceive};
+use bgp_packet::open::{AddPathEntry, Capability, OpenMessage, SendReceive};
 use bgp_primitives::constants::AfiSafi;
-use bgp_primitives::constants::{Afi, Safi};
 use bgp_primitives::ip_prefix::Nlri;
 use bgp_rib::{LocalRoute, LocalRouteAttrs};
 use bgp_server::config::PeerConfig;
@@ -350,25 +349,16 @@ async fn test_every_route_goes_to_a_collector_told_apart_by_path_identifiers() -
     b.announce(route(&[PEER_ASN + 1, 64510])).await?;
     server.wait_for_routes(2).await?;
 
-    let mut capabilities = vec![Capability::FourByteAsn { asn: COLLECTOR_ASN }];
-    capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
-    capabilities.push(Capability::AddPath {
-        entries: vec![AddPathEntry {
-            afi: AfiSafi::IPV4_UNICAST.afi,
-            safi: AfiSafi::IPV4_UNICAST.safi,
-            send_recv: SendReceive::Receive,
-        }],
-    });
     let mut collector = TestPeer::connect(server.addr).await?;
     collector
         .handshake_with(
             COLLECTOR_ASN,
             Ipv4Addr::new(10, 0, 0, 100),
             90,
-            capabilities,
+            collector_capabilities(COLLECTOR_ASN, &SUPPORTED, false),
         )
         .await?;
-    collector.ctx.add_path = vec![AfiSafi::IPV4_UNICAST];
+    collector.ctx.add_path = SUPPORTED.to_vec();
 
     // Both routes, told apart.
     let mut held = Held::new();
@@ -437,25 +427,16 @@ async fn test_a_large_table_reaches_a_collector_that_was_there_first() -> Result
         c.connect_retry = Duration::from_secs(3600);
     })
     .await?;
-    let mut capabilities = vec![Capability::FourByteAsn { asn: COLLECTOR_ASN }];
-    capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
-    capabilities.push(Capability::AddPath {
-        entries: vec![AddPathEntry {
-            afi: AfiSafi::IPV4_UNICAST.afi,
-            safi: AfiSafi::IPV4_UNICAST.safi,
-            send_recv: SendReceive::Receive,
-        }],
-    });
     let mut collector = TestPeer::connect(server.addr).await?;
     collector
         .handshake_with(
             COLLECTOR_ASN,
             Ipv4Addr::new(10, 0, 0, 100),
             90,
-            capabilities,
+            collector_capabilities(COLLECTOR_ASN, &SUPPORTED, false),
         )
         .await?;
-    collector.ctx.add_path = vec![AfiSafi::IPV4_UNICAST];
+    collector.ctx.add_path = SUPPORTED.to_vec();
     server.wait_for_sessions(1).await?;
 
     // Two feeders, each sending the same prefixes a hundred to an UPDATE.
@@ -518,35 +499,16 @@ async fn test_both_families_reach_a_collector_over_one_ipv6_session() -> Result<
         c.connect_retry = Duration::from_secs(3600);
     })
     .await?;
-    let mut capabilities = vec![Capability::FourByteAsn { asn: COLLECTOR_ASN }];
-    capabilities.extend(SUPPORTED.into_iter().map(Capability::multiprotocol));
-    capabilities.push(Capability::AddPath {
-        entries: [AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST]
-            .into_iter()
-            .map(|family| AddPathEntry {
-                afi: family.afi,
-                safi: family.safi,
-                send_recv: SendReceive::Receive,
-            })
-            .collect(),
-    });
-    capabilities.push(Capability::ExtendedNextHop {
-        entries: vec![ExtendedNextHopEntry {
-            afi: Afi::IPV4,
-            safi: Safi::UNICAST,
-            nh_afi: Afi::IPV6,
-        }],
-    });
     let mut collector = TestPeer::connect(server.addrs[1]).await?;
     collector
         .handshake_with(
             COLLECTOR_ASN,
             Ipv4Addr::new(10, 0, 0, 100),
             90,
-            capabilities,
+            collector_capabilities(COLLECTOR_ASN, &SUPPORTED, true),
         )
         .await?;
-    collector.ctx.add_path = vec![AfiSafi::IPV4_UNICAST, AfiSafi::IPV6_UNICAST];
+    collector.ctx.add_path = SUPPORTED.to_vec();
     server.wait_for_sessions(1).await?;
     let session = &server.registry.list()[0];
     assert!(session.extended_next_hop, "RFC 8950 agreed");

crates/integration-tests/tests/http_api.rs

@@ -52,6 +52,27 @@ async fn test_status_and_peers() -> Result<()> {
         peers[0]["address_families"],
         serde_json::json!(["IPv4 unicast", "IPv6 unicast"])
     );
+
+    // The session's TCP connection, by the address the peer list gives.
+    // What the kernel says of it depends on the platform; that it was
+    // asked, and why it had nothing to say if it had not, does not.
+    let peer_addr = peers[0]["peer_addr"].as_str().unwrap();
+    let tcp = get_json(&client, &format!("{base}/api/tcp?peer={peer_addr}")).await?;
+    let available = tcp["available"].as_bool().expect("available is a bool");
+    if available {
+        assert!(tcp["connection"].is_object(), "{tcp}");
+    } else {
+        assert!(tcp["reason"].is_string(), "{tcp}");
+    }
+    // The address alone names the peer too, as it does elsewhere.
+    let by_address = get_json(&client, &format!("{base}/api/tcp?peer=127.0.0.1")).await?;
+    assert_eq!(by_address["available"], tcp["available"]);
+    // A peer there is no session with is a bad request, not an answer.
+    let nobody = client
+        .get(format!("{base}/api/tcp?peer=nobody"))
+        .send()
+        .await?;
+    assert_eq!(nobody.status(), 400);
     Ok(())
 }
 

crates/packet/src/attributes.rs

@@ -18,7 +18,6 @@ use bgp_primitives::constants::{AfiSafi, IpVersion, Safi};
 use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
 use bgp_primitives::parser::{BgpParserError, EncodeError, ParserContext};
 
-use crate::message::write_nlri;
 use crate::wire::{
     Parsed, list, malformed, parse_afi_safi, parse_asn, put_afi_safi, put_asn, put_length, records,
     whole,
@@ -833,6 +832,19 @@ pub(crate) fn parse_nlri_list(
     list(buf, |buf| parse_nlri(version, with_path_id, buf))
 }
 
+/// Writes prefixes as `parse_nlri_list` reads them: for each, its path
+/// identifier if the session uses them for the family (RFC 7911 §3), its
+/// length in bits, and the bytes that length covers.
+pub fn put_nlri_list(nlri: &[Nlri], with_path_id: bool, out: &mut BytesMut) {
+    for n in nlri {
+        if with_path_id {
+            out.put_u32(n.path_id);
+        }
+        out.put_u8(n.prefix.length());
+        out.put(n.prefix.bytes());
+    }
+}
+
 /// The IP version of the prefixes that make up the NLRI of `family`.
 ///
 /// Any family can be named in MP_REACH_NLRI and MP_UNREACH_NLRI, but only
@@ -897,7 +909,7 @@ impl MpReachNlriPathAttribute {
         out.put_u8(self.next_hop.wire_len());
         self.next_hop.to_wire(out);
         out.put_u8(0);
-        write_nlri(&self.prefixes, ctx.has_add_path(self.family), out);
+        put_nlri_list(&self.prefixes, ctx.has_add_path(self.family), out);
     }
 }
 
@@ -919,6 +931,6 @@ impl MpUnreachNlriPathAttribute {
 
     pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) {
         put_afi_safi(self.family, out);
-        write_nlri(&self.prefixes, ctx.has_add_path(self.family), out);
+        put_nlri_list(&self.prefixes, ctx.has_add_path(self.family), out);
     }
 }

crates/packet/src/message.rs

@@ -15,7 +15,7 @@ use crate::errors::DecodeError;
 use crate::errors::MsgHeaderSubcode;
 use crate::errors::OpenMsgSubcode;
 use crate::open::OpenMessage;
-use crate::wire::{fill_length, put_length};
+use crate::wire::{body_has_path_id, fill_length, put_length};
 
 /// A sentinel AS number used to denote that the actual AS number is provided as
 /// a 4 byte value in the capabilities instead.
@@ -30,10 +30,15 @@ pub const BGP_HEADER_LEN: u16 = 19;
 /// Maximum BGP message length per RFC 4271 §4.1.
 pub const BGP_MAX_MSG_LEN: u16 = 4096;
 
+/// The sixteen octets of ones that open every message (RFC 4271 §4.1).
+/// Whoever frames a message by hand wants [`begin_message`] and
+/// [`finish_message`] rather than this.
 pub const BGP_MARKER: [u8; 16] = [0xFF; 16];
 
-/// BGP message type codes (RFC 4271 §4.1, RFC 2918).
+/// BGP message type codes (RFC 4271 §4.1, RFC 2918), for whoever frames
-mod msg_type {
+/// a message with [`begin_message`] or tells messages apart by their
+/// header.
+pub mod msg_type {
     pub const OPEN: u8 = 1;
     pub const UPDATE: u8 = 2;
     pub const NOTIFICATION: u8 = 3;
@@ -162,23 +167,10 @@ impl Message {
     /// cannot be encoded, `out` is left as it was.
     pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) -> Result<(), EncodeError> {
         let start = out.len();
-        out.put_slice(&BGP_MARKER);
+        let length_at = begin_message(out, self.type_code());
-        // The length, filled in once the body is written. It counts the
+        let written = self
-        // header too.
+            .body_to_wire(ctx, out)
-        let length_at = out.len();
+            .and_then(|()| finish_message(out, length_at));
-        out.put_u16(0);
-        out.put_u8(self.type_code());
-        let written = self.body_to_wire(ctx, out).and_then(|()| {
-            let length = out.len() - start;
-            put_length(
-                out,
-                length_at,
-                2,
-                length,
-                BGP_MAX_MSG_LEN.into(),
-                "BGP message",
-            )
-        });
         if let Err(e) = written {
             out.truncate(start);
             return Err(e);
@@ -215,6 +207,35 @@ impl Message {
     }
 }
 
+/// Writes the header of a message of type `kind` to the end of `out`,
+/// with its length left at zero, and returns where the length goes. The
+/// body follows, and then [`finish_message`] fills the length in: for
+/// whoever builds a message out of pieces that are not a [`Message`], as
+/// an exporter streaming a table does.
+pub fn begin_message(out: &mut BytesMut, kind: u8) -> usize {
+    out.put_slice(&BGP_MARKER);
+    let length_at = out.len();
+    out.put_u16(0);
+    out.put_u8(kind);
+    length_at
+}
+
+/// Fills in the length of the message whose header [`begin_message`] put
+/// at `length_at`, counting the header too, once the body is written. A
+/// message longer than [`BGP_MAX_MSG_LEN`] cannot be sent, and is an
+/// error; `out` is left for the caller to cut back.
+pub fn finish_message(out: &mut BytesMut, length_at: usize) -> Result<(), EncodeError> {
+    let length = out.len() - length_at + BGP_MARKER.len();
+    put_length(
+        out,
+        length_at,
+        2,
+        length,
+        BGP_MAX_MSG_LEN.into(),
+        "BGP message",
+    )
+}
+
 /// Decodes the body of an OPEN, with the NOTIFICATION that reports a
 /// failure (RFC 4271 §6.2).
 fn decode_open(body: &[u8]) -> Result<OpenMessage, DecodeError> {
@@ -256,7 +277,10 @@ pub struct Decoded {
     pub message: Message,
     /// The errors in an UPDATE that were handled without resetting the
     /// session (RFC 7606). Empty for a well-formed UPDATE and for every
-    /// other message.
+    /// other message. These are
+    /// [`DecodedUpdate::errors`](crate::update_decode::DecodedUpdate::errors),
+    /// named here for the one message that can have them, since this is
+    /// the result for any message.
     pub update_errors: Vec<UpdateError>,
     /// See [`crate::update_decode::DecodedUpdate::attributes_wire`].
     pub attributes_wire: Option<Vec<u8>>,
@@ -332,14 +356,13 @@ impl UpdateMessage {
 
     /// Writes the body of the UPDATE (what follows the 19-octet header).
     pub fn to_wire(&self, ctx: &ParserContext, out: &mut BytesMut) -> Result<(), EncodeError> {
-        // The prefixes in the body are IPv4 unicast (RFC 4271 §4.3).
+        let with_path_id = body_has_path_id(ctx);
-        let with_path_id = ctx.has_add_path(AfiSafi::IPV4_UNICAST);
 
         // Each field's two-octet length is filled in once the field is
         // written.
         let withdrawn = out.len();
         out.put_u16(0);
-        write_nlri(&self.withdrawn_routes, with_path_id, out);
+        put_nlri_list(&self.withdrawn_routes, with_path_id, out);
         fill_length(out, withdrawn, 2, u16::MAX.into(), "withdrawn routes")?;
 
         let attributes = out.len();
@@ -350,23 +373,14 @@ impl UpdateMessage {
         fill_length(out, attributes, 2, u16::MAX.into(), "path attributes")?;
 
         // The NLRI take up the rest of the message.
-        write_nlri(&self.announced_routes, with_path_id, out);
+        put_nlri_list(&self.announced_routes, with_path_id, out);
         Ok(())
     }
 }
 
-/// Writes prefixes as an UPDATE names them: for each, its path identifier
+/// The name [`put_nlri_list`] went by before it moved next to the parser
-/// if the session uses them for the family (RFC 7911 §3), its length in
+/// it is the pair of; the exporter still uses it.
-/// bits, and the bytes that length covers.
+pub use crate::attributes::put_nlri_list as write_nlri;
-pub fn write_nlri(nlri: &[Nlri], with_path_id: bool, out: &mut BytesMut) {
-    for n in nlri {
-        if with_path_id {
-            out.put_u32(n.path_id);
-        }
-        out.put_u8(n.prefix.length());
-        out.put(n.prefix.bytes());
-    }
-}
 
 /// A parser failure as text for a log line.
 pub(crate) fn describe(e: &nom::Err<BgpParserError<&[u8]>>) -> String {
@@ -409,7 +423,7 @@ mod tests {
         str::FromStr,
     };
 
-    use bytes::BytesMut;
+    use bytes::{BufMut, BytesMut};
     use eyre::{Result, eyre};
 
     use crate::message::{
@@ -1133,6 +1147,31 @@ mod tests {
         Ok(())
     }
 
+    /// A message framed by hand, as the exporter frames an UPDATE it
+    /// builds from pieces, is the message `to_wire` writes; and one that
+    /// grows past the maximum is refused with its length filled in.
+    #[test]
+    fn a_message_framed_by_hand_is_the_one_to_wire_writes() {
+        use crate::message::{begin_message, finish_message, msg_type};
+        let ctx = default_v6_context();
+        let mut whole = BytesMut::new();
+        Message::Update(UpdateMessage::end_of_rib(AfiSafi::IPV4_UNICAST))
+            .to_wire(&ctx, &mut whole)
+            .unwrap();
+
+        let mut by_hand = BytesMut::new();
+        let length_at = begin_message(&mut by_hand, msg_type::UPDATE);
+        assert_eq!(length_at, 16);
+        by_hand.put_u32(0);
+        finish_message(&mut by_hand, length_at).unwrap();
+        assert_eq!(by_hand, whole);
+
+        let mut too_long = BytesMut::new();
+        let length_at = begin_message(&mut too_long, msg_type::UPDATE);
+        too_long.put_bytes(0, 4096);
+        assert!(finish_message(&mut too_long, length_at).is_err());
+    }
+
     #[test]
     fn test_message_rejects_bad_marker() {
         let ctx = default_v6_context();

crates/packet/src/update_decode.rs

@@ -19,7 +19,7 @@
 //! be acted on, with treat-as-withdraw and attribute discard already carried
 //! out, together with a report of what was wrong, for the log.
 
-use bgp_primitives::constants::{AfiSafi, IpVersion};
+use bgp_primitives::constants::IpVersion;
 use bgp_primitives::ip_prefix::Nlri;
 use bgp_primitives::parser::ParserContext;
 
@@ -29,6 +29,7 @@ use crate::message::{
     MpUnreachNlriPathAttribute, NotAnAttribute, PathAttribute, RawAttribute, UpdateMessage,
     defined_flags, describe, parse_nlri_list, split_attribute,
 };
+use crate::wire::body_has_path_id;
 
 /// How an error in an UPDATE was handled, short of resetting the session.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
@@ -62,7 +63,9 @@ pub struct UpdateError {
 pub struct DecodedUpdate {
     /// The UPDATE after treat-as-withdraw and attribute discard.
     pub update: UpdateMessage,
-    /// Empty when the UPDATE was well formed.
+    /// What was wrong, in the order met; empty when the UPDATE was well
+    /// formed. Where a message of any type was decoded this is
+    /// [`Decoded::update_errors`](crate::message::Decoded::update_errors).
     pub errors: Vec<UpdateError>,
     /// The attributes in `update`, as they were on the wire, for a table
     /// that keeps attributes in that form and need not write them out
@@ -102,9 +105,7 @@ fn reset(subcode: UpdateMsgSubcode, data: Vec<u8>, detail: String) -> DecodeErro
 /// error here cannot be recovered from: the prefixes are what
 /// treat-as-withdraw would act on.
 fn prefixes(ctx: &ParserContext, field: &str, buf: &[u8]) -> Result<Vec<Nlri>, DecodeError> {
-    // The prefixes in the body are IPv4 unicast (RFC 4271 §4.3).
+    match parse_nlri_list(IpVersion::Ipv4, body_has_path_id(ctx), buf) {
-    let with_path_id = ctx.has_add_path(AfiSafi::IPV4_UNICAST);
-    match parse_nlri_list(IpVersion::Ipv4, with_path_id, buf) {
         Ok((_, prefixes)) => Ok(prefixes),
         Err(e) => Err(reset(
             UpdateMsgSubcode::InvalidNetworkField,
@@ -234,7 +235,10 @@ struct AttributeReader<'a> {
     attributes: Vec<PathAttribute>,
     errors: Vec<UpdateError>,
     /// The bytes of the attributes that are kept and are a property of a
-    /// route: all but MP_REACH_NLRI and MP_UNREACH_NLRI.
+    /// route: all but MP_REACH_NLRI and MP_UNREACH_NLRI. Left empty on a
+    /// session with two-octet AS numbers, whose bytes no table keeps (see
+    /// [`DecodedUpdate::attributes_wire`]), so that nothing is copied or
+    /// reserved for them.
     kept_wire: Vec<u8>,
     /// The type codes met so far, one bit each.
     seen: [u64; 4],
@@ -246,7 +250,7 @@ impl<'a> AttributeReader<'a> {
             ctx,
             attributes: Vec::new(),
             errors: Vec::new(),
-            kept_wire: Vec::with_capacity(wire_len),
+            kept_wire: Vec::with_capacity(if ctx.four_octet_asn { wire_len } else { 0 }),
             seen: [0; 4],
         }
     }
@@ -291,7 +295,7 @@ impl<'a> AttributeReader<'a> {
                 route_property,
             } = self.judge(&raw)?
             {
-                if route_property {
+                if route_property && self.ctx.four_octet_asn {
                     self.kept_wire.extend_from_slice(raw.wire);
                 }
                 self.attributes.push(attribute);
@@ -494,6 +498,7 @@ mod tests {
     use std::net::Ipv4Addr;
 
     use crate::message::PathAttributeFlags;
+    use bgp_primitives::constants::AfiSafi;
     use bgp_primitives::ip_prefix::IpPrefix;
 
     use super::*;

crates/packet/src/wire.rs

@@ -9,7 +9,7 @@ use nom::IResult;
 use nom::number::complete::{be_u8, be_u16, be_u32};
 
 use bgp_primitives::constants::{Afi, AfiSafi, Safi};
-use bgp_primitives::parser::{BgpParserError, EncodeError};
+use bgp_primitives::parser::{BgpParserError, EncodeError, ParserContext};
 
 use crate::message::AS_TRANS;
 
@@ -91,6 +91,15 @@ pub(crate) fn put_asn(four_octet: bool, asn: u32, out: &mut BytesMut) {
     }
 }
 
+/// Whether the prefixes in the body of an UPDATE carry path identifiers
+/// on a session with `ctx`. The body holds IPv4 unicast routes and nothing
+/// else (RFC 4271 §4.3), so that is the family whose ADD-PATH agreement
+/// decides it; every other family's prefixes are in MP_REACH_NLRI and
+/// MP_UNREACH_NLRI, which name their family.
+pub(crate) fn body_has_path_id(ctx: &ParserContext) -> bool {
+    ctx.has_add_path(AfiSafi::IPV4_UNICAST)
+}
+
 /// Parses an address family as the multiprotocol attributes and most
 /// capabilities name it: a two-octet AFI and a one-octet SAFI.
 pub(crate) fn parse_afi_safi(buf: &[u8]) -> Parsed<'_, AfiSafi> {

crates/rib/src/table/attrs.rs

@@ -13,7 +13,7 @@
 use std::hash::BuildHasher;
 use std::sync::{Arc, Mutex, MutexGuard, PoisonError};
 
-use bgp_packet::message::{NlriNextHop, PathAttribute, attr_type, split_attribute};
+use bgp_packet::message::{NlriNextHop, PathAttribute, RawAttribute, attr_type, split_attribute};
 use bgp_primitives::parser::{EncodeError, ParserContext};
 use bytes::{BufMut, BytesMut};
 use foldhash::fast::RandomState;
@@ -21,6 +21,10 @@ use hashbrown::HashTable;
 
 const OPTIONAL: u8 = 0x80;
 
+/// The type of an AS_PATH segment whose AS numbers are in order; the other
+/// type, AS_SET, is an unordered handful (RFC 4271 §4.3).
+const AS_SEQUENCE: u8 = 2;
+
 /// What route selection reads from a set of attributes (RFC 4271 §9.1.2.2).
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub struct SelectionKey {
@@ -116,7 +120,6 @@ impl SelectionKey {
 
 /// Reads an AS_PATH value with four-octet AS numbers.
 fn scan_as_path(mut segments: &[u8]) -> Option<PathScan> {
-    const AS_SEQUENCE: u8 = 2;
     let mut hops = 0usize;
     let (mut first, mut last) = (0, 0);
     let mut at_start = true;
@@ -150,7 +153,6 @@ fn scan_as_path(mut segments: &[u8]) -> Option<PathScan> {
 /// AS numbers to `out`, and returns whether any segment is not a sequence.
 /// Stops at what is not a segment.
 fn read_as_path(mut segments: &[u8], out: &mut Vec<u32>) -> bool {
-    const AS_SEQUENCE: u8 = 2;
     let mut has_set = false;
     while let [kind, count, rest @ ..] = segments {
         let Some((asns, rest)) = rest.split_at_checked(usize::from(*count) * 4) else {
@@ -187,24 +189,37 @@ impl AttrSet {
         &self.wire
     }
 
+    /// The attributes one by one, as they lie in the stored bytes. The
+    /// bytes were checked when the set was made, so this is all of them;
+    /// it stops at the first that does not split as an attribute all the
+    /// same, rather than go on from the middle of one.
+    fn raw_attributes(&self) -> impl Iterator<Item = RawAttribute<'_>> {
+        let mut wire = &*self.wire;
+        std::iter::from_fn(move || {
+            let (attribute, rest) = split_attribute(wire).ok()?;
+            wire = rest;
+            Some(attribute)
+        })
+    }
+
+    /// The value of the attribute of type `type_code`, if the set has one.
+    fn attribute(&self, type_code: u8) -> Option<&[u8]> {
+        self.raw_attributes()
+            .find(|attribute| attribute.type_code == type_code)
+            .map(|attribute| attribute.value)
+    }
+
     /// The route's next hop: the one that came in MP_REACH_NLRI, or else
     /// the NEXT_HOP attribute. Read off the stored bytes without building
     /// anything else, for those who ask it of every route.
     pub fn next_hop(&self) -> Option<NlriNextHop> {
         let mut plain = None;
-        let mut wire = &*self.wire;
+        for attribute in self.raw_attributes() {
-        while let Ok((attribute, rest)) = split_attribute(wire) {
             match attribute.type_code {
-                attr_type::MP_REACH_NLRI => {
+                attr_type::MP_REACH_NLRI => return decode_next_hop(attribute.value),
-                    // Kept as the next hop's length and the next hop.
-                    let (length, address) = attribute.value.split_first()?;
-                    return NlriNextHop::from_bytes(address)
-                        .filter(|_| usize::from(*length) == address.len());
-                }
                 attr_type::NEXT_HOP => plain = NlriNextHop::from_bytes(attribute.value),
                 _ => {}
             }
-            wire = rest;
         }
         plain
     }
@@ -215,14 +230,7 @@ impl AttrSet {
     /// for those who ask it of every route: a session to a peer or a
     /// provider asks it of most of the table, to pass over them.
     pub fn is_only_to_customer(&self) -> bool {
-        let mut wire = &*self.wire;
+        self.attribute(attr_type::ONLY_TO_CUSTOMER).is_some()
-        while let Ok((attribute, rest)) = split_attribute(wire) {
-            if attribute.type_code == attr_type::ONLY_TO_CUSTOMER {
-                return true;
-            }
-            wire = rest;
-        }
-        false
     }
 
     /// Whether the route carries the BLACKHOLE community (RFC 7999), which
@@ -231,15 +239,8 @@ impl AttrSet {
     /// of every route.
     pub fn is_blackhole(&self) -> bool {
         const BLACKHOLE: [u8; 4] = [0xFF, 0xFF, 0x02, 0x9A];
-        let mut wire = &*self.wire;
+        self.attribute(attr_type::COMMUNITIES)
-        while let Ok((attribute, rest)) = split_attribute(wire) {
+            .is_some_and(|communities| communities.as_chunks::<4>().0.contains(&BLACKHOLE))
-            if attribute.type_code == attr_type::COMMUNITIES {
-                let (communities, _) = attribute.value.as_chunks::<4>();
-                return communities.contains(&BLACKHOLE);
-            }
-            wire = rest;
-        }
-        false
     }
 
     /// Puts the AS numbers of the route's path in `out`, in place of what
@@ -253,14 +254,10 @@ impl AttrSet {
     /// else, for those who ask it of every route.
     pub fn as_path(&self, out: &mut Vec<u32>) -> bool {
         out.clear();
-        let mut wire = &*self.wire;
+        match self.attribute(attr_type::AS_PATH) {
-        while let Ok((attribute, rest)) = split_attribute(wire) {
+            Some(segments) => read_as_path(segments, out),
-            if attribute.type_code == attr_type::AS_PATH {
+            None => false,
-                return read_as_path(attribute.value, out);
-            }
-            wire = rest;
         }
-        false
     }
 
     /// The attributes, parsed, and the next hop if it came in
@@ -307,7 +304,8 @@ fn encode(
     Ok(())
 }
 
-/// Writes a next hop as an MP_REACH_NLRI that holds nothing else.
+/// Writes a next hop as an MP_REACH_NLRI that holds nothing else: the
+/// next hop's length, and the next hop.
 fn encode_next_hop(next_hop: &NlriNextHop, out: &mut BytesMut) {
     out.put_u8(OPTIONAL);
     out.put_u8(attr_type::MP_REACH_NLRI);
@@ -316,6 +314,13 @@ fn encode_next_hop(next_hop: &NlriNextHop, out: &mut BytesMut) {
     next_hop.to_wire(out);
 }
 
+/// Reads the next hop out of the value of an MP_REACH_NLRI as
+/// [`encode_next_hop`] writes it. `None` if the value is not one.
+fn decode_next_hop(value: &[u8]) -> Option<NlriNextHop> {
+    let (length, address) = value.split_first()?;
+    NlriNextHop::from_bytes(address).filter(|_| usize::from(*length) == address.len())
+}
+
 /// Parses path attributes in the form [`AttrSet`] keeps and table dumps
 /// use (RFC 6396 §4.3.4): four-octet AS numbers, and an MP_REACH_NLRI that
 /// holds only a next hop. `None` if they are malformed.
@@ -326,11 +331,7 @@ fn parse(mut wire: &[u8]) -> Option<(Vec<PathAttribute>, Option<NlriNextHop>)> {
     while !wire.is_empty() {
         let (raw, rest) = split_attribute(wire).ok()?;
         if raw.type_code == attr_type::MP_REACH_NLRI {
-            let (length, address) = raw.value.split_first()?;
+            next_hop = Some(decode_next_hop(raw.value)?);
-            if usize::from(*length) != address.len() {
-                return None;
-            }
-            next_hop = Some(NlriNextHop::from_bytes(address)?);
         } else {
             let (_, attribute) = PathAttribute::from_wire(&ctx, raw.wire).ok()?;
             attributes.push(attribute);
@@ -650,6 +651,67 @@ mod tests {
         );
     }
 
+    /// The OTC attribute is read off the stored bytes, wherever in them it
+    /// lies.
+    #[test]
+    fn otc_is_told_from_the_bytes() {
+        let store = AttrStore::new();
+        let without = store.intern(&attributes(vec![3333]), None).unwrap();
+        assert!(!without.is_only_to_customer());
+
+        let mut with = attributes(vec![3333]);
+        with.insert(1, PathAttribute::OnlyToCustomer(3333));
+        assert!(store.intern(&with, None).unwrap().is_only_to_customer());
+
+        // After COMMUNITIES, which is the last of the usual attributes.
+        let mut after = attributes(vec![3333]);
+        after.push(PathAttribute::OnlyToCustomer(3333));
+        assert!(store.intern(&after, None).unwrap().is_only_to_customer());
+    }
+
+    /// What a session received, with attributes of our own added after it,
+    /// is one set with the parsed form of the whole; the bytes are not
+    /// parsed to find that out, so the bytes must say the same thing.
+    #[test]
+    fn received_attributes_with_ours_added_are_the_same_set_as_the_parsed_form() {
+        let store = AttrStore::new();
+        let ctx = four_octet();
+        let received = attributes(vec![65001, 65002]);
+        let mut wire = BytesMut::new();
+        encode(&received, None, &mut wire).unwrap();
+        let ours = [
+            PathAttribute::OnlyToCustomer(65001),
+            PathAttribute::LocalPref(LocalPrefPathAttribute(150)),
+        ];
+        let mut added = BytesMut::new();
+        for attribute in &ours {
+            attribute.to_wire(&ctx, &mut added).unwrap();
+        }
+        let next_hop = NlriNextHop::Ipv6("2001:db8::1".parse().unwrap());
+
+        let mut whole = received.clone();
+        whole.extend(ours.iter().cloned());
+        let parsed = store.intern(&whole, Some(&next_hop)).unwrap();
+        let from_bytes = store
+            .intern_received_with(&wire, &added, Some(&next_hop))
+            .unwrap();
+        assert!(Arc::ptr_eq(&parsed, &from_bytes));
+        assert!(from_bytes.is_only_to_customer());
+        assert_eq!(from_bytes.next_hop(), Some(next_hop));
+
+        // Without a next hop too.
+        let parsed = store.intern(&whole, None).unwrap();
+        let from_bytes = store.intern_received_with(&wire, &added, None).unwrap();
+        assert!(Arc::ptr_eq(&parsed, &from_bytes));
+
+        // An attribute of ours of a type the peer sent is not a set.
+        let mut repeated = BytesMut::new();
+        PathAttribute::MultiExitDisc(MultiExitDiscPathAttribute(1))
+            .to_wire(&ctx, &mut repeated)
+            .unwrap();
+        assert!(store.intern_received_with(&wire, &repeated, None).is_none());
+    }
+
     /// Two of the same attribute are not a set: its readers could each
     /// take a different one.
     #[test]

crates/rib/src/table/journal.rs

@@ -95,7 +95,7 @@ struct Entry {
 }
 
 /// One journal of one shard.
-struct Journal {
+pub(super) struct Journal {
     kind: Kind,
     /// The sequence number of the latest change. Counts every change this
     /// journal is for, whether or not anyone was reading.
@@ -113,10 +113,6 @@ struct Journal {
     /// Where each reader has read to, by reader slot. Written by the
     /// readers themselves while they hold the shard for reading.
     cursors: Vec<AtomicU64>,
-    /// `seq`, for reading without the shard's lock: the table holds the
-    /// other end, so that a reader can tell a shard with nothing new for
-    /// it from one with something without stopping the shard's writers.
-    latest: Arc<AtomicU64>,
     /// How many reader slots are in use.
     readers: usize,
     limits: Limits,
@@ -129,7 +125,6 @@ impl Journal {
         Self {
             kind,
             limits,
-            latest: Arc::new(AtomicU64::new(0)),
             seq: 0,
             floor: 0,
             entries: VecDeque::new(),
@@ -144,7 +139,6 @@ impl Journal {
     /// number for the row to remember.
     fn changed(&mut self, prefix: IpPrefix) -> u64 {
         self.seq += 1;
-        self.latest.store(self.seq, Ordering::Release);
         if self.readers > 0 {
             self.entries.push_back(Entry {
                 seq: self.seq,
@@ -167,7 +161,6 @@ impl Journal {
     /// Notes that `prefix` lost its last route and has no row any more.
     fn disappeared(&mut self, prefix: IpPrefix) {
         self.seq += 1;
-        self.latest.store(self.seq, Ordering::Release);
         if self.readers > 0 {
             self.entries.push_back(Entry {
                 seq: self.seq,
@@ -178,25 +171,35 @@ impl Journal {
         }
     }
 
-    /// Forgets that `entry`'s prefix had gone, if `entry` is what said so.
+    /// Voids the note that `entry`'s prefix had gone, if `entry` is what
-    fn drop_entry(gone: &mut HashMap<IpPrefix, u64, RandomState>, entry: &Entry) {
+    /// said so: an entry that is dropped must not leave the note behind,
+    /// or the prefix would be given out as gone on the strength of a later
+    /// entry that says something else.
+    fn void_gone_note(gone: &mut HashMap<IpPrefix, u64, RandomState>, entry: &Entry) {
         if gone.get(&entry.prefix) == Some(&entry.seq) {
             gone.remove(&entry.prefix);
         }
     }
 
-    /// What `entry` says, if it still says something: it is the latest
+    /// What `entry` says: something only if it is the latest change of its
-    /// change of its prefix. The row, or `None` for a prefix that has gone.
+    /// prefix.
-    fn live<'a>(
+    fn says<'a>(&self, entry: &Entry, rows: &'a HashTable<Row>, hasher: &RandomState) -> Says<'a> {
-        &self,
-        entry: &Entry,
-        rows: &'a HashTable<Row>,
-        hasher: &RandomState,
-    ) -> Option<Option<&'a Row>> {
         let hash = hasher.hash_one(entry.prefix);
         match rows.find(hash, |row| row.prefix == entry.prefix) {
-            Some(row) => (row.seqs.of(self.kind) == entry.seq).then_some(Some(row)),
+            Some(row) if *row.seqs.of(self.kind) == entry.seq => Says::Row(row),
-            None => (self.gone.get(&entry.prefix) == Some(&entry.seq)).then_some(None),
+            None if self.gone.get(&entry.prefix) == Some(&entry.seq) => Says::Gone,
+            _ => Says::Nothing,
+        }
+    }
+
+    /// Where a reader whose cursor is `cursor` is to read from.
+    fn place_of(&self, cursor: u64) -> Place {
+        if cursor < self.floor {
+            Place::Lagged
+        } else if cursor == self.seq {
+            Place::Nothing
+        } else {
+            Place::From(self.entries.partition_point(|entry| entry.seq <= cursor))
         }
     }
 
@@ -206,7 +209,26 @@ impl Journal {
         if self.entries.len() <= self.limits.tidy_at {
             return;
         }
-        // What every reader has passed can go.
+        self.drop_passed();
+        // Entries for prefixes that have a row are at most as many as the
+        // rows once the dead ones are out. Entries for prefixes that have
+        // gone are not counted in: they are what a reader that has stopped
+        // reading makes pile up. Twice that, so that the pass over every
+        // entry that finding the dead ones takes is made only once the
+        // journal has doubled, and the cost per change stays small.
+        let limit = 2 * (rows.len() + self.limits.slack);
+        if self.entries.len() <= limit {
+            return;
+        }
+        self.drop_dead(rows, hasher);
+        if self.entries.len() <= limit {
+            return;
+        }
+        self.drop_oldest_half();
+    }
+
+    /// Drops what every reader has passed.
+    fn drop_passed(&mut self) {
         let passed = self
             .cursors
             .iter()
@@ -217,40 +239,57 @@ impl Journal {
             if front.seq > passed {
                 break;
             }
-            Self::drop_entry(&mut self.gone, front);
+            Self::void_gone_note(&mut self.gone, front);
             self.entries.pop_front();
         }
+    }
 
-        // Entries for prefixes that have a row are at most as many as the
+    /// Drops the dead entries: those whose prefix has changed again since,
-        // rows once the dead ones are out. Entries for prefixes that have
+    /// so that a later entry says what they said. This looks every entry
-        // gone are not counted in: they are what a reader that has stopped
+    /// up.
-        // reading makes pile up.
+    fn drop_dead(&mut self, rows: &HashTable<Row>, hasher: &RandomState) {
-        let limit = 2 * (rows.len() + self.limits.slack);
-        if self.entries.len() <= limit {
-            return;
-        }
-        // Dead entries: their prefix has changed again since. This looks
-        // every entry up, but only once the journal has doubled, so the
-        // cost per change stays small.
         let mut entries = std::mem::take(&mut self.entries);
-        entries.retain(|entry| self.live(entry, rows, hasher).is_some());
+        entries.retain(|entry| !matches!(self.says(entry, rows, hasher), Says::Nothing));
         self.entries = entries;
+    }
 
-        if self.entries.len() <= limit {
+    /// Drops the older half of the entries without regard to the readers,
-            return;
+    /// for a journal that is too long even without its dead entries: a
-        }
+    /// reader is not reading while prefixes come and go, or more prefixes
-        // Still too long: a reader is not reading while prefixes come and
+    /// went at once than are left. A reader that had not read what goes
-        // go, or more prefixes went at once than are left. The older half
+    /// loses its place. Half, so that readers that are only a little
-        // goes, and a reader that had not read it loses its place. Half,
+    /// behind keep theirs, and so that getting a long journal down to size
-        // so that readers that are only a little behind keep theirs, and
+    /// takes few passes.
-        // so that getting a long journal down to size takes few passes.
+    fn drop_oldest_half(&mut self) {
         for entry in self.entries.drain(..self.entries.len() / 2) {
-            Self::drop_entry(&mut self.gone, &entry);
+            Self::void_gone_note(&mut self.gone, &entry);
             self.floor = entry.seq;
         }
     }
 }
 
+/// What an entry says; see [`Journal::says`].
+enum Says<'a> {
+    /// The prefix's row, whose latest change the entry is.
+    Row(&'a Row),
+    /// The prefix has no route any more, and the entry is what says so.
+    Gone,
+    /// Nothing: the prefix has changed again since, and a later entry has
+    /// it.
+    Nothing,
+}
+
+/// Where a reader is to read a journal from; see [`Journal::place_of`].
+enum Place {
+    /// Entries the reader had not read were dropped; it has lost its
+    /// place.
+    Lagged,
+    /// The reader has read everything up to the latest change.
+    Nothing,
+    /// The index of the first entry the reader has not read.
+    From(usize),
+}
+
 /// Which of a shard's two journals.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub(super) enum Kind {
@@ -260,39 +299,36 @@ pub(super) enum Kind {
     All,
 }
 
-impl Kind {
+/// One of something for each [`Kind`] of journal.
-    /// Its place in what the table keeps for each shard's two journals.
+#[derive(Debug, Clone, Copy, Default)]
-    pub(super) fn index(self) -> usize {
+pub(super) struct ByKind<T> {
-        match self {
+    best: T,
-            Self::Best => 0,
+    all: T,
-            Self::All => 1,
+}
+
+impl<T> ByKind<T> {
+    pub(super) fn of(&self, kind: Kind) -> &T {
+        match kind {
+            Kind::Best => &self.best,
+            Kind::All => &self.all,
+        }
+    }
+
+    pub(super) fn of_mut(&mut self, kind: Kind) -> &mut T {
+        match kind {
+            Kind::Best => &mut self.best,
+            Kind::All => &mut self.all,
         }
     }
 }
 
 /// The sequence numbers a row remembers, one for each journal: of the
 /// latest change of its best route, and of any of its routes.
-#[derive(Debug, Clone, Copy)]
+pub(super) type Seqs = ByKind<u64>;
-pub(super) struct Seqs {
-    best: u64,
-    all: u64,
-}
-
-impl Seqs {
-    fn of(self, kind: Kind) -> u64 {
-        match kind {
-            Kind::Best => self.best,
-            Kind::All => self.all,
-        }
-    }
-}
 
 /// The two journals of a shard, written together: every change to a row is
 /// a change of one of its routes, and some are a change of its best route.
-pub(super) struct Journals {
+pub(super) type Journals = ByKind<Journal>;
-    best: Journal,
-    all: Journal,
-}
 
 impl Journals {
     pub(super) fn new(limits: Limits) -> Self {
@@ -302,24 +338,13 @@ impl Journals {
         }
     }
 
-    /// The latest sequence number of each journal, by [`Kind::index`],
+    /// Copies the journals' sequence numbers to `latest`, which the table
-    /// as something to read without the shard's lock.
+    /// keeps outside the shard's lock for readers to look at
-    pub(super) fn latest(&self) -> [Arc<AtomicU64>; 2] {
+    /// ([`RibTable::latest_seq`]). Called with the lock held, after every
-        [Arc::clone(&self.best.latest), Arc::clone(&self.all.latest)]
+    /// change to the shard.
-    }
+    pub(super) fn publish(&self, latest: &ByKind<AtomicU64>) {
-
+        latest.best.store(self.best.seq, Ordering::Release);
-    fn of(&self, kind: Kind) -> &Journal {
+        latest.all.store(self.all.seq, Ordering::Release);
-        match kind {
-            Kind::Best => &self.best,
-            Kind::All => &self.all,
-        }
-    }
-
-    fn of_mut(&mut self, kind: Kind) -> &mut Journal {
-        match kind {
-            Kind::Best => &mut self.best,
-            Kind::All => &mut self.all,
-        }
     }
 
     /// The time by the table's clock that the changes made from now on
@@ -385,18 +410,11 @@ pub(super) struct Mark(u64, u64);
 pub struct Routes<'a>(&'a Row);
 
 impl<'a> Routes<'a> {
+    /// The routes: at least one, since a prefix with none has no row, and
+    /// is given out as [`Event::Withdrawn`] instead.
     pub fn iter(&self) -> impl Iterator<Item = RouteRef<'a>> + use<'a> {
         self.0.routes()
     }
-
-    /// How many there are: at least one.
-    pub fn len(&self) -> usize {
-        self.0.paths.len()
-    }
-
-    pub fn is_empty(&self) -> bool {
-        self.len() == 0
-    }
 }
 
 impl std::fmt::Debug for Routes<'_> {
@@ -447,19 +465,15 @@ pub struct Progress {
 }
 
 enum Phase {
-    /// Giving out the whole table; `next` is the shard to do next, and
+    /// Giving out the whole table; `next` is the shard to do next. `upto`
-    /// `announced` whether [`Event::Reset`] has been given. `upto` is the
+    /// is the time on the table's clock the table is given out as of, and
-    /// time on the table's clock the table is given out as of: the shards
+    /// is taken when [`Event::Reset`] is given, so `None` until then: the
-    /// are gone through one at a time while the table changes, and the
+    /// shards are gone through one at a time while the table changes, and
-    /// place taken in each shard's journal is as of that one time, so that
+    /// the place taken in each shard's journal is as of that one time, so
-    /// what follows is consistent from shard to shard. A change after it
+    /// that what follows is consistent from shard to shard. A change after
-    /// that a later shard's rows already show is read again from the
+    /// it that a later shard's rows already show is read again from the
     /// journal; that is harmless.
-    Resync {
+    Resync { next: usize, upto: Option<u64> },
-        next: usize,
-        announced: bool,
-        upto: u64,
-    },
     /// Following the journals; `next` is the shard to read next, and
     /// `upto` the time by the table's clock up to which this pass reads
     /// (see [`Entry::stamp`]).
@@ -470,9 +484,11 @@ impl Phase {
     /// The beginning of a resynchronisation.
     const RESYNC: Self = Self::Resync {
         next: 0,
-        announced: false,
+        upto: None,
-        upto: 0,
     };
+
+    /// The beginning of a pass over the journals.
+    const FOLLOW: Self = Self::Follow { next: 0, upto: 0 };
 }
 
 /// Follows the routes of a table: the best of each prefix, or all of them.
@@ -489,7 +505,10 @@ pub struct Reader {
     /// is for the shard's benefit.
     cursors: Vec<u64>,
     phase: Phase,
-    /// The table's version when the reader was last found caught up.
+    /// The table's version as it was when the current pass over the
+    /// journals began. [`Reader::changed`] waits while the version is still
+    /// this one: whatever was written since the pass began has moved it
+    /// on, and is there to be read.
     seen: u64,
     caught_up: bool,
 }
@@ -505,7 +524,7 @@ impl RibTable {
     /// those who want the news and not the table. If it falls too far
     /// behind it is reset like any other, and is then given the table.
     pub fn reader_from_now(self: &Arc<Self>) -> Reader {
-        let mut reader = self.register(Kind::Best, Phase::Follow { next: 0, upto: 0 });
+        let mut reader = self.register(Kind::Best, Phase::FOLLOW);
         for (index, shard) in self.shards.iter().enumerate() {
             let shard = read(shard);
             reader.set_cursor(index, &shard, shard.journals.best.seq);
@@ -625,114 +644,124 @@ impl Reader {
         // shard of every pass, by every reader, was the most of what a
         // router with many peers did.
         let table = Arc::clone(&self.table);
-        let shards = table.shards.len();
         let mut events = 0;
         loop {
-            match self.phase {
+            let done = match self.phase {
-                Phase::Resync {
+                Phase::Resync { next, upto } => {
-                    next,
+                    self.resync_step(&table, next, upto, budget, &mut events, &mut visit)
-                    announced,
-                    upto,
-                } => {
-                    let upto = if announced {
-                        upto
-                    } else {
-                        self.forget_place();
-                        visit(Event::Reset);
-                        events += 1;
-                        self.table.settled()
-                    };
-                    if next == shards {
-                        visit(Event::Synced);
-                        events += 1;
-                        self.phase = Phase::Follow { next: 0, upto: 0 };
-                        continue;
-                    }
-                    events += self.scan(&table, next, upto, &mut visit);
-                    self.phase = Phase::Resync {
-                        next: next + 1,
-                        announced: true,
-                        upto,
-                    };
-                    if events >= budget {
-                        return self.progress(events, false);
-                    }
                 }
                 Phase::Follow { next, upto } => {
-                    // Read before the pass: a change made during it shows as
+                    self.follow_pass(&table, next, upto, budget, &mut events, &mut visit)
-                    // a newer version, and `changed` then returns at once.
+                }
-                    // And the pass reads up to one time in every shard
+            };
-                    // alike, chosen so that about the budget is read.
+            if let Some(progress) = done {
-                    let (version, upto) = if next == 0 {
+                return progress;
-                        (
+            }
-                            self.table.version.load(Ordering::Acquire),
+        }
-                            self.time_for(budget - events),
+    }
-                        )
+
-                    } else {
+    /// One step of a resynchronisation: the shard `next`, or the end of it
-                        (self.seen, upto)
+    /// once every shard has been given out. Returns what to tell the
-                    };
+    /// caller if the budget is spent, and `None` to go on.
-                    self.seen = version;
+    fn resync_step(
-                    let mut shard = next;
+        &mut self,
-                    let mut lagged = false;
+        table: &RibTable,
-                    let mut consumed_any = false;
+        next: usize,
-                    let mut more_somewhere = false;
+        upto: Option<u64>,
-                    while shard < shards {
+        budget: usize,
-                        if events >= budget {
+        events: &mut usize,
-                            self.phase = Phase::Follow { next: shard, upto };
+        visit: &mut impl FnMut(Event<'_>),
-                            return self.progress(events, false);
+    ) -> Option<Progress> {
-                        }
+        let upto = upto.unwrap_or_else(|| {
-                        // Nothing new in this shard, by the number that is
+            self.forget_place();
-                        // read without its lock: it is left alone, and so
+            visit(Event::Reset);
-                        // are the sessions writing to it.
+            *events += 1;
-                        if self.cursors[shard] == self.table.latest_seq(shard, self.kind) {
+            table.settled()
-                            shard += 1;
+        });
-                            continue;
+        if next == table.shards.len() {
-                        }
+            visit(Event::Synced);
-                        match self.drain(&table, shard, upto, &mut visit) {
+            *events += 1;
-                            Drained::Lagged => {
+            self.phase = Phase::FOLLOW;
-                                lagged = true;
+            return None;
-                                break;
+        }
-                            }
+        *events += self.scan(table, next, upto, visit);
-                            Drained::Read {
+        self.phase = Phase::Resync {
-                                events: read,
+            next: next + 1,
-                                consumed,
+            upto: Some(upto),
-                                more,
+        };
-                            } => {
+        (*events >= budget).then(|| self.progress(*events, false))
-                                events += read;
+    }
-                                consumed_any |= consumed;
+
-                                more_somewhere |= more;
+    /// A pass over the journals from shard `next` on, or the rest of one
-                            }
+    /// that an earlier call cut short. Returns what to tell the caller if
-                        }
+    /// the pass is to be the call's last, and `None` to go round again.
-                        shard += 1;
+    fn follow_pass(
-                    }
+        &mut self,
-                    if lagged {
+        table: &RibTable,
-                        self.phase = Phase::RESYNC;
+        next: usize,
-                        continue;
+        upto: u64,
-                    }
+        budget: usize,
-                    self.phase = Phase::Follow { next: 0, upto: 0 };
+        events: &mut usize,
-                    if next == 0 && !consumed_any && more_somewhere {
+        visit: &mut impl FnMut(Event<'_>),
-                        // A whole pass got nowhere, though there is more:
+    ) -> Option<Progress> {
-                        // it is all beyond the earliest change still under
+        // Read before the pass: a change made during it shows as a newer
-                        // way, which no pass can read past until it ends.
+        // version, and `changed` then returns at once. And the pass reads
-                        // Asking again at once would spin until then. The
+        // up to one time in every shard alike, chosen so that about the
-                        // change wakes the readers when it ends.
+        // budget is read.
-                        return self.progress(events, true);
+        let (version, upto) = if next == 0 {
-                    }
+            (
-                    if next != 0 || more_somewhere {
+                table.version.load(Ordering::Acquire),
-                        // This pass took up where an earlier call stopped,
+                self.time_for(budget - *events),
-                        // so the shards before that point have not been
+            )
-                        // looked at in this call; or some shard had more
+        } else {
-                        // than its share, or changes later than this
+            (self.seen, upto)
-                        // pass's time. Go round again from the first.
+        };
-                        if events >= budget {
+        self.seen = version;
-                            return self.progress(events, false);
+        let mut consumed_any = false;
-                        }
+        let mut more_somewhere = false;
-                        continue;
+        for shard in next..table.shards.len() {
-                    }
+            if *events >= budget {
-                    return self.progress(events, true);
+                self.phase = Phase::Follow { next: shard, upto };
+                return Some(self.progress(*events, false));
+            }
+            // Nothing new in this shard, by the number that is read
+            // without its lock: it is left alone, and so are the sessions
+            // writing to it.
+            if self.cursors[shard] == table.latest_seq(shard, self.kind) {
+                continue;
+            }
+            match self.drain(table, shard, upto, visit) {
+                Drained::Lagged => {
+                    self.phase = Phase::RESYNC;
+                    return None;
+                }
+                Drained::Read {
+                    events: read,
+                    consumed,
+                    more,
+                } => {
+                    *events += read;
+                    consumed_any |= consumed;
+                    more_somewhere |= more;
                 }
             }
         }
+        self.phase = Phase::FOLLOW;
+        if next == 0 && !consumed_any && more_somewhere {
+            // A whole pass got nowhere, though there is more: it is all
+            // beyond the earliest change still under way, which no pass
+            // can read past until it ends. Asking again at once would spin
+            // until then. The change wakes the readers when it ends.
+            return Some(self.progress(*events, true));
+        }
+        if next != 0 || more_somewhere {
+            // This pass took up where an earlier call stopped, so the
+            // shards before that point have not been looked at in this
+            // call; or some shard had more than its share, or changes
+            // later than this pass's time. Go round again from the first.
+            return (*events >= budget).then(|| self.progress(*events, false));
+        }
+        Some(self.progress(*events, true))
     }
 
     fn progress(&mut self, events: usize, caught_up: bool) -> Progress {
@@ -847,10 +876,9 @@ impl Reader {
             }
             let shard = read(shard);
             let journal = shard.journals.of(self.kind);
-            if cursor < journal.floor || cursor == journal.seq {
+            let Place::From(start) = journal.place_of(cursor) else {
                 continue;
-            }
+            };
-            let start = journal.entries.partition_point(|entry| entry.seq <= cursor);
             if let Some(next) = journal.entries.get(start) {
                 earliest = Some(earliest.map_or(next.stamp, |e: u64| e.min(next.stamp)));
             }
@@ -874,18 +902,17 @@ impl Reader {
         let shard = read(&table.shards[index]);
         let journal = shard.journals.of(self.kind);
         let mut cursor = self.cursors[index];
-        if cursor < journal.floor {
+        let start = match journal.place_of(cursor) {
-            return Drained::Lagged;
+            Place::Lagged => return Drained::Lagged,
-        }
+            Place::Nothing => {
-        if cursor == journal.seq {
+                return Drained::Read {
-            return Drained::Read {
+                    events: 0,
-                events: 0,
+                    consumed: false,
-                consumed: false,
+                    more: false,
-                more: false,
+                };
-            };
+            }
-        }
+            Place::From(start) => start,
-
+        };
-        let start = journal.entries.partition_point(|entry| entry.seq <= cursor);
         let mut events = 0;
         let mut more = false;
         for entry in journal.entries.range(start..) {
@@ -894,13 +921,12 @@ impl Reader {
                 break;
             }
             cursor = entry.seq;
-            let event = match journal.live(entry, &shard.rows, &shard.hasher) {
+            let event = match journal.says(entry, &shard.rows, &shard.hasher) {
-                Some(Some(row)) => self.event_for(row),
+                Says::Row(row) => self.event_for(row),
-                Some(None) => Event::Withdrawn {
+                Says::Gone => Event::Withdrawn {
                     prefix: entry.prefix,
                 },
-                // The prefix has changed again; a later entry has it.
+                Says::Nothing => continue,
-                None => continue,
             };
             visit(event);
             events += 1;

crates/rib/src/table/journal_tests.rs

@@ -8,7 +8,7 @@ use proptest::prelude::*;
 
 use super::super::tests::{Fixture, Route, external, internal, prefix, route};
 use super::super::{AttrSet, PeerIndex, PeerInfo, RibTable};
-use super::{Event, Limits, Reader};
+use super::{Event, Kind, Limits, Reader, read};
 
 /// What a follower would keep: its own copy of the best routes, brought
 /// up to date from the events it is given.
@@ -53,7 +53,10 @@ impl Mirror {
             }
             Event::Routes { prefix, routes } => {
                 self.routes_events += 1;
-                assert!(!routes.is_empty(), "a prefix with no routes is withdrawn");
+                assert!(
+                    routes.iter().next().is_some(),
+                    "a prefix with no routes is withdrawn"
+                );
                 self.routes.insert(
                     prefix,
                     sorted(
@@ -115,9 +118,6 @@ impl Mirror {
             );
         }
     }
-}
-
-impl Mirror {
     /// For the mirror of a follower of all routes: it holds every route in
     /// the table, and knows which is best.
     #[track_caller]
@@ -213,6 +213,107 @@ fn a_reader_from_now_is_given_the_changes_and_not_the_table() {
     assert_eq!((best, withdrawn), (vec![new], vec![old]));
 }
 
+/// The number a reader checks without a shard's lock must be the journal's
+/// own after every kind of change, whether or not anyone is reading: a
+/// reader skips a shard on the strength of it, and a shard with news that
+/// said it had none would never be read until the next change.
+#[test]
+fn the_lock_free_number_is_the_journals_after_every_kind_of_change() {
+    let mut fixture = Fixture::new(&[external(1), external(2)]);
+    let prefixes: Vec<IpPrefix> = (0..16).map(nth_prefix).collect();
+    let agree = |table: &RibTable| {
+        for (index, shard) in table.shards.iter().enumerate() {
+            let shard = read(shard);
+            for kind in [Kind::Best, Kind::All] {
+                assert_eq!(
+                    table.latest_seq(index, kind),
+                    shard.journals.of(kind).seq,
+                    "shard {index}, {kind:?}"
+                );
+            }
+        }
+    };
+
+    // Without a reader, with one of each kind, and with both.
+    for readers in 0..4 {
+        let _best = (readers & 1 != 0).then(|| fixture.table.reader());
+        let _all = (readers & 2 != 0).then(|| fixture.table.reader_of_all_routes());
+        agree(&fixture.table);
+
+        for p in &prefixes {
+            fixture.announce(0, p, &Route::via(&[1]));
+        }
+        agree(&fixture.table);
+        // The same prefixes again, with other attributes.
+        for p in &prefixes {
+            fixture.announce(0, p, &Route::via(&[1, 2]));
+        }
+        agree(&fixture.table);
+        // A second route for each, then the last route of one taken away.
+        for p in &prefixes {
+            fixture.announce(1, p, &Route::via(&[2, 3]));
+        }
+        agree(&fixture.table);
+        fixture.withdraw(0, &prefixes[0]);
+        agree(&fixture.table);
+        fixture.withdraw(1, &prefixes[0]);
+        agree(&fixture.table);
+
+        fixture.table.remove_peer(fixture.peers[1]);
+        agree(&fixture.table);
+        fixture.peers[1] = fixture.table.add_peer(external(2)).unwrap();
+
+        fixture
+            .table
+            .mark_stale(fixture.peers[0], &[IpVersion::Ipv4]);
+        agree(&fixture.table);
+        fixture
+            .table
+            .sweep_stale(fixture.peers[0], &[IpVersion::Ipv4]);
+        agree(&fixture.table);
+        assert!(
+            fixture.table.is_empty(),
+            "round {readers} left routes behind"
+        );
+    }
+}
+
+/// A shard that had nothing new for a reader is passed over without its
+/// lock. The next change to it must bring the reader back.
+#[test]
+fn a_shard_passed_over_is_read_again_once_it_has_news() {
+    let fixture = Fixture::on(RibTable::new(1), &[external(1)]);
+    let mut reader = fixture.table.reader();
+    let mut mirror = Mirror::default();
+    mirror.catch_up(&mut reader, 100);
+    assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
+
+    fixture.announce(0, &prefix("10.0.0.0/8"), &Route::via(&[1]));
+    let mut events = Vec::new();
+    let progress = reader.poll(100, |event| events.push(format!("{event:?}")));
+    assert!(progress.caught_up);
+    assert_eq!(events.len(), 1, "{events:?}");
+    assert!(events[0].starts_with("Best"), "{events:?}");
+}
+
+/// As [`a_shard_passed_over_is_read_again_once_it_has_news`], for a reader
+/// that begins where the journals are rather than with the table.
+#[test]
+fn a_shard_passed_over_by_a_reader_from_now_is_read_again_once_it_has_news() {
+    let fixture = Fixture::on(RibTable::new(1), &[external(1)]);
+    fixture.announce(0, &prefix("192.0.2.0/24"), &Route::via(&[1]));
+    let mut reader = fixture.table.reader_from_now();
+    assert!(reader.poll(100, |_| panic!("no event expected")).caught_up);
+    assert_eq!(reader.poll(100, |_| panic!("no event expected")).events, 0);
+
+    fixture.announce(0, &prefix("10.0.0.0/8"), &Route::via(&[1]));
+    let mut events = Vec::new();
+    let progress = reader.poll(100, |event| events.push(format!("{event:?}")));
+    assert!(progress.caught_up);
+    assert_eq!(events.len(), 1, "{events:?}");
+    assert!(events[0].starts_with("Best"), "{events:?}");
+}
+
 /// A change to a route that is not the best is nothing to a follower of
 /// the best routes, and is what a follower of all routes is there for.
 #[test]

crates/rib/src/table/mod.rs

@@ -313,13 +313,21 @@ struct Shard {
     journals: journal::Journals,
 }
 
-/// The routing table. Shared between threads by reference; every method
+/// A shard, and beside its lock what is read without taking it.
-/// takes `&self`.
+struct ShardSlot {
+    lock: RwLock<Shard>,
+    /// The sequence number of each of the shard's journals as of the last
+    /// change to the shard, stored by the change while it still holds the
+    /// lock ([`RibTable::change`]). A reader looks at it first, and passes
+    /// over a shard with nothing new for it without stopping the shard's
+    /// writers.
+    latest: journal::ByKind<AtomicU64>,
+}
+
+/// The routing table. Shared between threads by reference: nothing takes
+/// `&mut self`, and the readers ([`Reader`]) keep an `Arc` of their own.
 pub struct RibTable {
-    shards: Box<[RwLock<Shard>]>,
+    shards: Box<[ShardSlot]>,
-    /// The latest sequence number of each shard's two journals, kept by
-    /// the journals themselves, for reading without the shard's lock.
-    latest: Box<[[Arc<AtomicU64>; 2]]>,
     hasher: RandomState,
     attrs: AttrStore,
     /// Which peer numbers are in use. Changing the peer list also takes
@@ -358,23 +366,20 @@ impl RibTable {
     /// sizes. Tests use small ones to get there quickly.
     fn with_journal_limits(shards: usize, limits: journal::Limits) -> Self {
         let hasher = RandomState::default();
-        let mut latest = Vec::with_capacity(shards.max(1));
         Self {
             shards: (0..shards.max(1))
-                .map(|_| {
+                .map(|_| ShardSlot {
-                    let journals = journal::Journals::new(limits);
+                    lock: RwLock::new(Shard {
-                    latest.push(journals.latest());
-                    RwLock::new(Shard {
                         rows: HashTable::new(),
                         routes: 0,
                         peers: Vec::new(),
                         hasher: hasher.clone(),
                         scratch: select::Scratch::default(),
-                        journals,
+                        journals: journal::Journals::new(limits),
-                    })
+                    }),
+                    latest: journal::ByKind::default(),
                 })
                 .collect(),
-            latest: latest.into_boxed_slice(),
             hasher,
             attrs: AttrStore::new(),
             peers: Mutex::default(),
@@ -523,7 +528,7 @@ impl RibTable {
     /// How many routes of `peer` are stale; see [`RibTable::mark_stale`].
     /// Looks at every prefix in the table.
     #[cfg(test)]
-    pub(super) fn count_stale(&self, peer: PeerIndex) -> usize {
+    fn count_stale(&self, peer: PeerIndex) -> usize {
         self.shards
             .iter()
             .map(|shard| {
@@ -623,18 +628,22 @@ impl RibTable {
     /// journal one of them follows was written to.
     fn change<'a, T>(
         &'a self,
-        parts: impl IntoIterator<Item = (&'a RwLock<Shard>, T)>,
+        parts: impl IntoIterator<Item = (&'a ShardSlot, T)>,
         mut apply: impl FnMut(&mut Shard, T),
     ) {
         let change = self.begin_change();
         let mut news = false;
-        for (shard, part) in parts {
+        for (slot, part) in parts {
-            let mut shard = write(shard);
+            let mut shard = write(slot);
             shard.journals.set_stamp(change.stamp);
             let before = shard.journals.mark();
             apply(&mut shard, part);
             shard.tidy_journals();
             news |= shard.journals.has_news_since(before);
+            // Once per change, not once per route; and before the lock is
+            // let go, so that a reader that sees the new number and takes
+            // the lock finds the entries behind it.
+            shard.journals.publish(&slot.latest);
         }
         drop(change);
         if news {
@@ -658,15 +667,18 @@ impl RibTable {
         }
     }
 
-    /// The latest sequence number of shard `shard`'s journal of `kind`,
+    /// The sequence number of shard `shard`'s journal of `kind` as of the
-    /// without the shard's lock.
+    /// last change to the shard, without the shard's lock.
     fn latest_seq(&self, shard: usize, kind: journal::Kind) -> u64 {
-        self.latest[shard][kind.index()].load(std::sync::atomic::Ordering::Acquire)
+        self.shards[shard]
+            .latest
+            .of(kind)
+            .load(std::sync::atomic::Ordering::Acquire)
     }
 
     /// The time on the clock up to which every change is complete in every
     /// shard: everything before the earliest one still under way.
-    pub(super) fn settled(&self) -> u64 {
+    fn settled(&self) -> u64 {
         let in_progress = self
             .in_progress
             .lock()
@@ -709,11 +721,8 @@ impl RibTable {
     }
 
     /// The best route for `prefix`: who it is from, and its attributes.
-    pub fn best(&self, prefix: &IpPrefix) -> Option<(PeerIndex, Arc<AttrSet>)> {
+    pub fn best(&self, prefix: &IpPrefix) -> Option<Route> {
-        self.with_row(prefix, |row| {
+        self.with_row(prefix, |row| row.route(row.best()).to_owned())
-            let best = &row.paths[row.best()];
-            (best.peer(), Arc::clone(&best.attrs))
-        })
     }
 
     /// Every route for `prefix`.
@@ -811,14 +820,14 @@ impl RibTable {
     }
 }
 
-fn read(shard: &RwLock<Shard>) -> RwLockReadGuard<'_, Shard> {
+fn read(shard: &ShardSlot) -> RwLockReadGuard<'_, Shard> {
     // A panic under the lock cannot leave a row half changed in a way that
     // is unsafe to read; the poison is ignored, as for the server's tables.
-    shard.read().unwrap_or_else(PoisonError::into_inner)
+    shard.lock.read().unwrap_or_else(PoisonError::into_inner)
 }
 
-fn write(shard: &RwLock<Shard>) -> RwLockWriteGuard<'_, Shard> {
+fn write(shard: &ShardSlot) -> RwLockWriteGuard<'_, Shard> {
-    shard.write().unwrap_or_else(PoisonError::into_inner)
+    shard.lock.write().unwrap_or_else(PoisonError::into_inner)
 }
 
 /// A change to the table that is under way; see [`RibTable::begin_change`].

crates/rib/src/table/tests.rs

@@ -162,7 +162,7 @@ impl Fixture {
 
     /// Which peer (by order added) the best route for `prefix` is from.
     fn best(&self, prefix: &IpPrefix) -> Option<usize> {
-        let (peer, _) = self.table.best(prefix)?;
+        let peer = self.table.best(prefix)?.peer;
         self.peers.iter().position(|p| *p == peer)
     }
 }

crates/server/src/address.rs

@@ -54,6 +54,14 @@ impl From<IpAddr> for PeerAddress {
     }
 }
 
+/// Whether `wanted`, as someone typed it, names the peer at `address`
+/// that is configured as `name`, if it is: by that name, or by the address
+/// as [`PeerAddress::is_called`] reads it. The one rule for every place a
+/// peer is asked for by name.
+pub fn names_peer(wanted: &str, name: Option<&str>, address: PeerAddress) -> bool {
+    name == Some(wanted) || address.is_called(wanted)
+}
+
 /// The address, and the interface by name if there is one: `fe80::1%eth0`.
 /// An interface that is no longer there goes by its number.
 impl fmt::Display for PeerAddress {
@@ -325,6 +333,12 @@ mod tests {
         assert!(there.is_called("fe80::1"));
         assert!(!there.is_called("fe80::1%other"));
         assert!(!there.is_called("fe80::2"));
+        // By its configured name as well, where it has one.
+        assert!(names_peer("upstream", Some("upstream"), there));
+        assert!(names_peer("fe80::1", Some("upstream"), there));
+        assert!(names_peer("fe80::1", None, there));
+        assert!(!names_peer("upstream", None, there));
+        assert!(!names_peer("other", Some("upstream"), there));
         assert_eq!(
             socket_text(peer.to_dial().unwrap()),
             format!("[fe80::1%{name}]:179")

crates/server/src/config.rs

@@ -11,6 +11,7 @@ use bgp_primitives::constants::{AfiSafi, IpVersion};
 use bgp_primitives::ip_prefix::IpPrefix;
 use bgp_rib::{LocalRoute, LocalRouteAttrs};
 use bgp_rpki::RpkiConfig;
+use protobuf::EnumOrUnknown;
 
 use crate::address::{Endpoint, PeerAddress};
 use crate::error::{ConfigError, ExportProblem, ImportProblem, OriginateProblem, PeerProblem};
@@ -141,20 +142,28 @@ impl Config {
         let listen_endpoints = parse_listen_endpoints(&proto.listen_endpoints)?;
 
         let defaults = PeerDefaults {
-            export: Arc::new(match proto.export.as_ref() {
+            export: Arc::new(
-                Some(export) => parse_export(export)?,
+                proto
-                None => ExportPolicy::default(),
+                    .export
-            }),
+                    .as_ref()
-            export_limits: match proto.export_limits.as_ref() {
+                    .map(parse_export)
-                Some(limits) => parse_export_limits(limits)
+                    .transpose()?
-                    .map_err(ConfigError::ExportLimits)?
+                    .unwrap_or_default(),
-                    .map(Arc::new),
+            ),
-                None => None,
+            export_limits: proto
-            },
+                .export_limits
-            graceful_restart: match proto.graceful_restart.as_ref() {
+                .as_ref()
-                Some(own) => GracefulRestartConfig::default().overridden_by(own)?,
+                .map(parse_export_limits)
-                None => GracefulRestartConfig::default(),
+                .transpose()
-            },
+                .map_err(ConfigError::ExportLimits)?
+                .flatten()
+                .map(Arc::new),
+            graceful_restart: proto
+                .graceful_restart
+                .as_ref()
+                .map(|own| GracefulRestartConfig::default().overridden_by(own))
+                .transpose()?
+                .unwrap_or_default(),
         };
         let mut peers = parse_peers(proto.peers, proto.local_asn, &defaults)?;
         check_roles(
@@ -179,16 +188,7 @@ impl Config {
             None => None,
         };
 
-        let rpki = match proto.rpki.as_ref() {
+        let rpki = proto.rpki.as_ref().map(parse_rpki).transpose()?;
-            Some(rpki) => Some(parse_rpki(rpki)?),
-            None => None,
-        };
-
-        let connect_retry = if proto.connect_retry_seconds == 0 {
-            DEFAULT_CONNECT_RETRY
-        } else {
-            Duration::from_secs(proto.connect_retry_seconds as u64)
-        };
 
         Ok(Self {
             router_id,
@@ -197,7 +197,7 @@ impl Config {
             hold_time_secs: DEFAULT_HOLD_TIME_SECS,
             listen_endpoints,
             peers,
-            connect_retry,
+            connect_retry: seconds(proto.connect_retry_seconds).unwrap_or(DEFAULT_CONNECT_RETRY),
             collision_detect_established_state: proto.collision_detect_established_state,
             accept_unconfigured_peers: proto.accept_unconfigured_peers,
             originate,
@@ -398,20 +398,37 @@ fn parse_identifier(text: &str) -> Option<Ipv4Addr> {
         .filter(|id| !id.is_unspecified())
 }
 
+/// A number the configuration has as zero is a field left out: the
+/// protocol buffer has no way of telling the two apart, and nothing
+/// configured here is meaningfully zero.
+fn given(n: u32) -> Option<u32> {
+    (n > 0).then_some(n)
+}
+
+/// A time given in seconds, if it was given.
+fn seconds(n: u32) -> Option<Duration> {
+    given(n).map(|n| Duration::from_secs(n.into()))
+}
+
+/// A list of prefixes, if any are listed: an empty list leaves the setting
+/// out rather than allowing nothing.
+fn parse_prefixes_if_any(
+    ranges: &[proto::PrefixRange],
+) -> Result<Option<PrefixSet>, ImportProblem> {
+    if ranges.is_empty() {
+        return Ok(None);
+    }
+    parse_prefixes(ranges).map(Some)
+}
+
 fn parse_kernel(kernel: &proto::Kernel) -> FibConfig {
     let defaults = FibConfig::default();
     FibConfig {
         // The kernel has no table 0 to put routes in.
-        table: match kernel.table {
+        table: given(kernel.table).unwrap_or(defaults.table),
-            0 => defaults.table,
-            table => table,
-        },
         metric: kernel.metric.unwrap_or(defaults.metric),
         nexthop_objects: !kernel.without_nexthop_objects,
-        resolution_table: match kernel.resolution_table {
+        resolution_table: given(kernel.resolution_table).unwrap_or(defaults.resolution_table),
-            0 => defaults.resolution_table,
-            table => table,
-        },
         resolve_via_default: kernel.resolve_via_default_route,
         stale_time: kernel.stale_seconds.map_or(defaults.stale_time, |seconds| {
             Duration::from_secs(seconds.into())
@@ -452,12 +469,8 @@ fn parse_export_limits(
     limits: &proto::ExportLimits,
 ) -> Result<Option<ExportLimits>, ImportProblem> {
     let limits = ExportLimits {
-        prefixes: if limits.prefixes.is_empty() {
+        prefixes: parse_prefixes_if_any(&limits.prefixes)?,
-            None
+        max_prefixes: given(limits.max_prefixes),
-        } else {
-            Some(parse_prefixes(&limits.prefixes)?)
-        },
-        max_prefixes: Some(limits.max_prefixes).filter(|limit| *limit > 0),
     };
     Ok((limits != ExportLimits::default()).then_some(limits))
 }
@@ -472,38 +485,17 @@ fn parse_peer(peer: Peer, defaults: &PeerDefaults) -> Result<PeerConfig, PeerPro
         return Err(PeerProblem::RemoteAsn);
     }
 
-    let hold_time_secs = match peer.hold_time_seconds {
+    let hold_time_secs = parse_hold_time(peer.hold_time_seconds)?;
-        None => DEFAULT_HOLD_TIME_SECS,
+    let local_address = (!peer.local_address.is_empty())
-        // RFC 4271 §4.2: zero, or at least three seconds.
+        .then(|| {
-        Some(1 | 2) => return Err(PeerProblem::HoldTime),
+            IpAddr::from_str(&peer.local_address)
-        Some(secs) => u16::try_from(secs).map_err(|_| PeerProblem::HoldTime)?,
+                .map_err(|_| PeerProblem::LocalAddress(peer.local_address.clone()))
-    };
+        })
-
+        .transpose()?;
-    let local_address = if peer.local_address.is_empty() {
+    if local_address.is_some_and(|ip| ip.is_ipv4() != endpoint.ip.is_ipv4()) {
-        None
+        return Err(PeerProblem::AddressFamilyMismatch);
-    } else {
+    }
-        let ip = IpAddr::from_str(&peer.local_address)
+    let families = parse_families(&peer.address_families)?;
-            .map_err(|_| PeerProblem::LocalAddress(peer.local_address.clone()))?;
-        if ip.is_ipv4() != endpoint.ip.is_ipv4() {
-            return Err(PeerProblem::AddressFamilyMismatch);
-        }
-        Some(ip)
-    };
-
-    let families = if peer.address_families.is_empty() {
-        FamilySet::supported()
-    } else {
-        peer.address_families
-            .iter()
-            .map(|family| match family.enum_value() {
-                Ok(proto::AddressFamily::IPV4_UNICAST) => Ok(AfiSafi::IPV4_UNICAST),
-                Ok(proto::AddressFamily::IPV6_UNICAST) => Ok(AfiSafi::IPV6_UNICAST),
-                Ok(proto::AddressFamily::ADDRESS_FAMILY_UNSPECIFIED) | Err(_) => {
-                    Err(PeerProblem::AddressFamily)
-                }
-            })
-            .collect::<Result<_, _>>()?
-    };
 
     let add_path = match peer.add_path.enum_value() {
         Ok(proto::AddPath::ADD_PATH_OFF) => None,
@@ -513,20 +505,14 @@ fn parse_peer(peer: Peer, defaults: &PeerDefaults) -> Result<PeerConfig, PeerPro
         Err(_) => return Err(PeerProblem::AddPath),
     };
 
-    let import = match peer.import.as_ref() {
+    let import = peer
-        Some(import) => parse_import(import)?,
+        .import
-        None => ImportPolicy::default(),
+        .as_ref()
-    };
+        .map(parse_import)
+        .transpose()?
+        .unwrap_or_default();
 
-    let role = match peer.role.enum_value() {
+    let role = parse_role(peer.role)?;
-        Ok(proto::Role::ROLE_UNSPECIFIED) => None,
-        Ok(proto::Role::PROVIDER) => Some(PeerRole::Provider),
-        Ok(proto::Role::CUSTOMER) => Some(PeerRole::Customer),
-        Ok(proto::Role::PEER) => Some(PeerRole::Peer),
-        Ok(proto::Role::ROUTE_SERVER) => Some(PeerRole::RouteServer),
-        Ok(proto::Role::ROUTE_SERVER_CLIENT) => Some(PeerRole::RouteServerClient),
-        Err(_) => return Err(PeerProblem::Role),
-    };
     if peer.require_role && role.is_none() {
         return Err(PeerProblem::RequireRoleWithoutRole);
     }
@@ -550,24 +536,74 @@ fn parse_peer(peer: Peer, defaults: &PeerDefaults) -> Result<PeerConfig, PeerPro
         import,
         role,
         require_role: peer.require_role,
-        export: match peer.export.as_ref() {
+        export: peer
-            Some(own) => Arc::new(parse_export(own)?),
+            .export
-            None => Arc::clone(&defaults.export),
+            .as_ref()
-        },
+            .map(parse_export)
-        export_limits: match peer.export_limits.as_ref() {
+            .transpose()?
-            Some(own) => parse_export_limits(own)
+            .map_or_else(|| Arc::clone(&defaults.export), Arc::new),
-                .map_err(PeerProblem::ExportLimits)?
+        // A block of the peer's own takes the place of the top-level one,
-                .map(Arc::new),
+        // an empty block included: that lifts the limits.
-            None => defaults.export_limits.clone(),
+        export_limits: peer
-        },
+            .export_limits
-        graceful_restart: match peer.graceful_restart.as_ref() {
+            .as_ref()
-            Some(own) => defaults.graceful_restart.overridden_by(own)?,
+            .map(parse_export_limits)
-            None => defaults.graceful_restart,
+            .transpose()
-        },
+            .map_err(PeerProblem::ExportLimits)?
+            .map_or_else(|| defaults.export_limits.clone(), |own| own.map(Arc::new)),
+        graceful_restart: peer
+            .graceful_restart
+            .as_ref()
+            .map(|own| defaults.graceful_restart.overridden_by(own))
+            .transpose()?
+            .unwrap_or(defaults.graceful_restart),
         groups: peer.groups,
         reflector_client: peer.route_reflector_client,
         passive: peer.passive,
-        expected_routes: Some(peer.expected_routes).filter(|n| *n > 0),
+        expected_routes: given(peer.expected_routes),
+    })
+}
+
+/// The hold time a peer is offered: the default where none is given.
+fn parse_hold_time(seconds: Option<u32>) -> Result<u16, PeerProblem> {
+    match seconds {
+        None => Ok(DEFAULT_HOLD_TIME_SECS),
+        // RFC 4271 §4.2: zero, or at least three seconds.
+        Some(1 | 2) => Err(PeerProblem::HoldTime),
+        Some(secs) => u16::try_from(secs).map_err(|_| PeerProblem::HoldTime),
+    }
+}
+
+/// The address families offered to a peer: every one there is, unless
+/// some are listed.
+fn parse_families(
+    families: &[EnumOrUnknown<proto::AddressFamily>],
+) -> Result<FamilySet, PeerProblem> {
+    if families.is_empty() {
+        return Ok(FamilySet::supported());
+    }
+    families
+        .iter()
+        .map(|family| match family.enum_value() {
+            Ok(proto::AddressFamily::IPV4_UNICAST) => Ok(AfiSafi::IPV4_UNICAST),
+            Ok(proto::AddressFamily::IPV6_UNICAST) => Ok(AfiSafi::IPV6_UNICAST),
+            Ok(proto::AddressFamily::ADDRESS_FAMILY_UNSPECIFIED) | Err(_) => {
+                Err(PeerProblem::AddressFamily)
+            }
+        })
+        .collect()
+}
+
+/// What the peer is to us, if that was said.
+fn parse_role(role: EnumOrUnknown<proto::Role>) -> Result<Option<PeerRole>, PeerProblem> {
+    Ok(match role.enum_value() {
+        Ok(proto::Role::ROLE_UNSPECIFIED) => None,
+        Ok(proto::Role::PROVIDER) => Some(PeerRole::Provider),
+        Ok(proto::Role::CUSTOMER) => Some(PeerRole::Customer),
+        Ok(proto::Role::PEER) => Some(PeerRole::Peer),
+        Ok(proto::Role::ROUTE_SERVER) => Some(PeerRole::RouteServer),
+        Ok(proto::Role::ROUTE_SERVER_CLIENT) => Some(PeerRole::RouteServerClient),
+        Err(_) => return Err(PeerProblem::Role),
     })
 }
 
@@ -584,7 +620,6 @@ fn parse_rpki(rpki: &proto::Rpki) -> Result<RpkiConfig, ConfigError> {
             port => parse_endpoint(&cache.address, port),
         })
         .collect::<Result<_, _>>()?;
-    let seconds = |seconds: u32| (seconds > 0).then(|| Duration::from_secs(seconds.into()));
     Ok(RpkiConfig {
         caches,
         refresh: seconds(rpki.refresh_seconds),
@@ -595,20 +630,20 @@ fn parse_rpki(rpki: &proto::Rpki) -> Result<RpkiConfig, ConfigError> {
 
 fn parse_import(import: &proto::ImportPolicy) -> Result<ImportPolicy, ImportProblem> {
     let defaults = ImportPolicy::default();
-    // A length of zero is the field left out.
+    let longest = |field, value: u32, version: IpVersion, default: u8| {
-    let longest = |field, value: u32, version: IpVersion, default: u8| match value {
+        given(value).map_or(Ok(default), |value| {
-        0 => Ok(default),
+            u8::try_from(value)
-        value => u8::try_from(value)
+                .ok()
-            .ok()
+                .filter(|length| *length <= version.bits())
-            .filter(|length| *length <= version.bits())
+                .ok_or(ImportProblem::PrefixLength {
-            .ok_or(ImportProblem::PrefixLength {
+                    field,
-                field,
+                    value,
-                value,
+                    longest: version.bits(),
-                longest: version.bits(),
+                })
-            }),
+        })
     };
 
-    let prefixes = parse_prefixes(&import.prefixes)?;
+    let prefixes = parse_prefixes_if_any(&import.prefixes)?;
     let rules = import
         .rules
         .iter()
@@ -622,11 +657,8 @@ fn parse_import(import: &proto::ImportPolicy) -> Result<ImportPolicy, ImportProb
         .collect::<Result<Vec<_>, _>>()?;
 
     Ok(ImportPolicy {
-        max_prefixes: Some(import.max_prefixes).filter(|limit| *limit > 0),
+        max_prefixes: given(import.max_prefixes),
-        hold_down: match import.hold_down_seconds {
+        hold_down: seconds(import.hold_down_seconds).unwrap_or(defaults.hold_down),
-            0 => defaults.hold_down,
-            seconds => Duration::from_secs(seconds.into()),
-        },
         accept_bogons: import.accept_bogons,
         accept_default_route: import.accept_default_route,
         max_ipv4_length: longest(
@@ -646,7 +678,7 @@ fn parse_import(import: &proto::ImportPolicy) -> Result<ImportPolicy, ImportProb
         accept_aspa_invalid: import.accept_aspa_invalid,
         accept_as_sets: import.accept_as_sets,
         accept_third_party_next_hop: import.accept_third_party_next_hop,
-        accept_loopback_next_hop: false,
+        accept_own_host_next_hop: false,
         accept_blackholes: import.accept_blackholes,
         own_prefixes: Arc::default(),
         originated: Arc::default(),
@@ -660,13 +692,13 @@ fn parse_import(import: &proto::ImportPolicy) -> Result<ImportPolicy, ImportProb
         local_pref: import.local_pref,
         rules,
         reflection: None,
-        max_as_path_length: match import.max_as_path_length {
+        max_as_path_length: given(import.max_as_path_length)
-            0 => defaults.max_as_path_length,
+            .map(|length| u16::try_from(length).map_err(|_| ImportProblem::PathLength(length)))
-            length => u16::try_from(length).map_err(|_| ImportProblem::PathLength(length))?,
+            .transpose()?
-        },
+            .unwrap_or(defaults.max_as_path_length),
-        prefixes: Some(prefixes).filter(|prefixes| !prefixes.is_empty()),
+        prefixes,
-        origin_asns: Some(import.origin_asns.iter().copied().collect())
+        origin_asns: (!import.origin_asns.is_empty())
-            .filter(|asns: &std::collections::HashSet<u32>| !asns.is_empty()),
+            .then(|| import.origin_asns.iter().copied().collect()),
     })
 }
 
@@ -681,17 +713,17 @@ fn parse_prefixes(ranges: &[proto::PrefixRange]) -> Result<PrefixSet, ImportProb
                 problem,
             })?;
         let longest = prefix.address_family().bits();
-        let max_length = match range.max_length {
+        // Left out, the prefix itself is the longest allowed.
-            0 => prefix.length(),
+        let max_length = given(range.max_length).map_or(Ok(prefix.length()), |max_length| {
-            max_length => u8::try_from(max_length)
+            u8::try_from(max_length)
                 .ok()
                 .filter(|max| (prefix.length()..=longest).contains(max))
                 .ok_or(ImportProblem::MaxLength {
                     prefix: range.prefix.clone(),
                     max_length,
                     longest,
-                })?,
+                })
-        };
+        })?;
         prefixes.allow(prefix, max_length);
     }
     Ok(prefixes)
@@ -700,21 +732,16 @@ fn parse_prefixes(ranges: &[proto::PrefixRange]) -> Result<PrefixSet, ImportProb
 fn parse_rule(rule: &proto::ImportRule) -> Result<ImportRule, ImportProblem> {
     Ok(ImportRule {
         communities: Patterns::parse(&rule.match_communities)?,
-        prefixes: if rule.match_prefixes.is_empty() {
+        prefixes: parse_prefixes_if_any(&rule.match_prefixes)?,
-            None
-        } else {
-            Some(parse_prefixes(&rule.match_prefixes)?)
-        },
         local_pref: rule.local_pref,
         marks: Marks::parse(&[], &rule.add_communities, &rule.add_large_communities)?,
     })
 }
 
 fn parse_port(port: u32) -> Result<u16, ConfigError> {
-    match port {
+    given(port).map_or(Ok(DEFAULT_BGP_PORT), |port| {
-        0 => Ok(DEFAULT_BGP_PORT),
+        u16::try_from(port).map_err(|_| ConfigError::Port(port))
-        port => u16::try_from(port).map_err(|_| ConfigError::Port(port)),
+    })
-    }
 }
 
 fn parse_endpoint(address: &str, port: u32) -> Result<SocketAddr, ConfigError> {

crates/server/src/dialer.rs

@@ -18,7 +18,7 @@ use tracing::{info, warn};
 
 use crate::collision::Direction;
 use crate::config::PeerConfig;
-use crate::connection::{CLOSED_AT_OUR_REQUEST, PeerSettings, ServerContext, run_session};
+use crate::connection::{PeerSettings, ServerContext, run_session};
 use crate::session::DialState;
 
 /// Upper bound for exponential backoff between connection attempts.
@@ -102,7 +102,9 @@ async fn dial_loop(peer: PeerConfig, server: Arc<ServerContext>, connect_retry:
         registry.set_dial_state(&peer.name, DialState::Connecting);
         info!(peer = %peer.name, endpoint = %peer.endpoint, "Connecting to peer");
 
-        let ended = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
+        // Why the attempt ended, and whether that is a failure worth a
+        // line in the log.
+        let (reason, failed) = match tokio::time::timeout(CONNECT_TIMEOUT, connect(&peer)).await {
             Ok(Ok(stream)) => match stream.peer_addr() {
                 Ok(peer_addr) => {
                     info!(peer = %peer.name, endpoint = %peer.endpoint, "TCP connected, starting session");
@@ -121,25 +123,17 @@ async fn dial_loop(peer: PeerConfig, server: Arc<ServerContext>, connect_retry:
                     if end.established {
                         backoff = connect_retry;
                     }
-                    if end.result.is_ok() {
+                    let failed = end.result.is_err();
-                        Ok(())
+                    (end.reason(), failed)
-                    } else {
-                        Err(end.reason())
-                    }
                 }
-                Err(e) => Err(format!("connection lost at once: {e}")),
+                Err(e) => (format!("connection lost at once: {e}"), true),
             },
-            Ok(Err(e)) => Err(format!("connect failed: {e}")),
+            Ok(Err(e)) => (format!("connect failed: {e}"), true),
-            Err(_) => Err(format!("connect timed out after {CONNECT_TIMEOUT:?}")),
+            Err(_) => (format!("connect timed out after {CONNECT_TIMEOUT:?}"), true),
-        };
-
-        let reason = match ended {
-            Ok(()) => CLOSED_AT_OUR_REQUEST.to_string(),
-            Err(reason) => {
-                warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %reason, "Session attempt ended");
-                reason
-            }
         };
+        if failed {
+            warn!(peer = %peer.name, endpoint = %peer.endpoint, err = %reason, "Session attempt ended");
+        }
 
         // Not all at the same moment, when many peers went away together.
         let wait = jittered(backoff);

crates/server/src/error.rs

@@ -41,6 +41,9 @@ pub enum SessionError {
     Decode(#[from] DecodeError),
     #[error("could not encode a message: {0}")]
     Encode(#[from] EncodeError),
+    /// A received UPDATE's attributes could not be kept in the table.
+    #[error(transparent)]
+    Unstorable(#[from] crate::routes::Unstorable),
     /// The peer sent nothing for as long as the pre-negotiation hold time.
     #[error("timed out waiting for a message in the {0} state")]
     Timeout(State),
@@ -101,9 +104,10 @@ impl SessionError {
     pub fn notification(&self) -> Option<NotificationMessage> {
         let error = match self {
             Self::Io(_) | Self::SendHoldTimerExpired(_) | Self::NotifiedByPeer(_) => return None,
-            // Something of ours could not be written out, which is a bug
+            // Something of ours could not be written out, or the peer's
-            // here: as for a panic, the session is reset.
+            // routes could not be kept, which is a bug here: as for a
-            Self::Encode(_) => BgpError::Cease(CeaseSubcode::AdminReset),
+            // panic, the session is reset.
+            Self::Encode(_) | Self::Unstorable(_) => BgpError::Cease(CeaseSubcode::AdminReset),
             Self::Decode(e) => {
                 return Some(NotificationMessage {
                     error: e.notification.clone(),

crates/server/src/export.rs

@@ -33,9 +33,9 @@
 use std::net::{IpAddr, Ipv4Addr};
 
 use bgp_packet::message::{
-    AsPathAttribute, AsPathSegment, BGP_MARKER, LocalPrefPathAttribute, MpReachNlriPathAttribute,
+    AsPathAttribute, AsPathSegment, LocalPrefPathAttribute, MpReachNlriPathAttribute,
     MpUnreachNlriPathAttribute, NextHopPathAttribute, NlriNextHop, PathAttribute, UpdateMessage,
-    attr_type, write_nlri,
+    attr_type, begin_message, finish_message, msg_type, put_nlri_list,
 };
 use bgp_primitives::constants::{AfiSafi, IpVersion};
 use bgp_primitives::ip_prefix::Nlri;
@@ -50,7 +50,7 @@ use crate::policy::{ExportPolicy, PeerRole};
 /// The session a route may go out on, as far as that decides whether it
 /// does and as what.
 #[derive(Debug, Clone, Copy)]
-pub struct Egress<'a> {
+pub(crate) struct Egress<'a> {
     pub towards: Towards,
     pub peer_asn: u32,
     /// The peer's number in the routing table: what its own routes there
@@ -69,7 +69,7 @@ pub struct Egress<'a> {
 
 /// Route reflection (RFC 4456), as far as one session goes.
 #[derive(Debug, Clone, Copy)]
-pub struct Reflection {
+pub(crate) struct Reflection {
     /// The cluster this router reflects routes for.
     pub cluster_id: Ipv4Addr,
     /// Whether the peer is a route reflector client of ours.
@@ -81,7 +81,7 @@ pub struct Reflection {
 
 /// Where a learned route came from.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
-pub struct Source {
+pub(crate) struct Source {
     pub peer: PeerIndex,
     /// Whether that peer is in our own AS.
     pub ibgp: bool,
@@ -95,7 +95,7 @@ pub struct Source {
 /// A route ready to be announced on a session: its attributes, but for
 /// the ones that depend on which prefixes go with them.
 #[derive(Debug, Clone, PartialEq, Eq)]
-pub struct Outgoing {
+pub(crate) struct Outgoing {
     pub attributes: Vec<PathAttribute>,
     pub next_hop: NlriNextHop,
 }
@@ -396,6 +396,14 @@ pub(crate) fn finish(attributes: &mut Vec<PathAttribute>, towards: &Towards) {
 /// lengths every UPDATE has.
 const UPDATE_ROOM: usize = 4096 - 19 - 4;
 
+/// The most octets a route's attributes may take on the wire for the
+/// route to go out at all. A message is 4,096 octets at most (RFC 4271
+/// §4.1) and must hold the attributes, the next hop and at least one
+/// prefix; three quarters of it leaves [`UPDATE_ROOM`] a thousand octets
+/// for the next hop and the prefixes, which is some forty IPv6 prefixes
+/// with path identifiers a message, enough to keep a table's UPDATEs few.
+pub(crate) const ATTRIBUTES_AT_MOST: usize = 3072;
+
 /// The most octets a prefix takes in an UPDATE: a path identifier, a
 /// length, and the address.
 fn nlri_octets(routes: IpVersion) -> usize {
@@ -409,16 +417,16 @@ fn nlri_octets(routes: IpVersion) -> usize {
 /// MP_REACH_NLRI (type 14) go, so that the attributes stay in order of
 /// type code.
 #[derive(Debug)]
-pub struct EncodedAttributes {
+pub(crate) struct EncodedAttributes {
-    pub bytes: Bytes,
+    bytes: Bytes,
-    pub at_next_hop: usize,
+    at_next_hop: usize,
-    pub at_mp_reach: usize,
+    at_mp_reach: usize,
 }
 
 impl EncodedAttributes {
     /// Encodes `attributes`, which are `finish`ed, for a session with
     /// `four_octet_asn`. `None` if they cannot be encoded.
-    pub fn encode(attributes: &[PathAttribute], four_octet_asn: bool) -> Option<Self> {
+    pub(crate) fn encode(attributes: &[PathAttribute], four_octet_asn: bool) -> Option<Self> {
         let ctx = ParserContext {
             four_octet_asn,
             ibgp: false,
@@ -444,9 +452,81 @@ impl EncodedAttributes {
         })
     }
 
-    pub fn octets(&self) -> usize {
+    pub(crate) fn octets(&self) -> usize {
         self.bytes.len()
     }
+
+    /// The UPDATE messages, complete with their headers, that announce
+    /// `routes`, all of IP version `version`, with these attributes and
+    /// `next_hop`: as few as they fit in, in pieces of [`ENCODED_PIECE`]
+    /// octets or so, each of whole messages. `ctx` is the session's, and
+    /// says whether path identifiers go with the family's prefixes
+    /// (RFC 7911).
+    ///
+    /// What `announcements`, kept under test as the reference, builds as
+    /// messages, built as bytes: the attributes are not parsed, cloned or
+    /// encoded again for each message.
+    pub(crate) fn announcements(
+        &self,
+        next_hop: NlriNextHop,
+        version: IpVersion,
+        routes: &[Nlri],
+        ctx: &ParserContext,
+    ) -> Vec<BytesMut> {
+        // Attributes that would leave no room for a prefix do not go out
+        // at all; see `Learned::dressed`.
+        debug_assert!(self.octets() < ATTRIBUTES_AT_MOST);
+        let room = UPDATE_ROOM.saturating_sub(self.octets() + NEXT_HOP_ROOM);
+        let at_once = (room / nlri_octets(version)).max(1);
+        let in_body = matches!(next_hop, NlriNextHop::Ipv4(_)) && version == IpVersion::Ipv4;
+        let with_path_id = ctx.add_path.contains(&AfiSafi::unicast(version));
+        let mut pieces = Vec::new();
+        let mut out = BytesMut::with_capacity(ENCODED_PIECE.min(routes.len() * 64 + 4096));
+        for routes in routes.chunks(at_once) {
+            {
+                let length_at = begin_message(&mut out, msg_type::UPDATE);
+                // No withdrawn routes.
+                out.put_u16(0);
+                let attributes_at = out.len();
+                out.put_u16(0);
+                if in_body {
+                    let NlriNextHop::Ipv4(next_hop) = next_hop else {
+                        unreachable!("in_body means an IPv4 next hop");
+                    };
+                    out.put_slice(&self.bytes[..self.at_next_hop]);
+                    PathAttribute::NextHop(NextHopPathAttribute(next_hop))
+                        .to_wire(ctx, &mut out)
+                        .expect("a next hop encodes");
+                    out.put_slice(&self.bytes[self.at_next_hop..]);
+                } else {
+                    out.put_slice(&self.bytes[..self.at_mp_reach]);
+                    PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
+                        family: AfiSafi::unicast(version),
+                        next_hop,
+                        prefixes: routes.to_vec(),
+                    })
+                    .to_wire(ctx, &mut out)
+                    .expect("a chunk that fits encodes");
+                    out.put_slice(&self.bytes[self.at_mp_reach..]);
+                }
+                let attributes_len = out.len() - attributes_at - 2;
+                out[attributes_at..attributes_at + 2]
+                    .copy_from_slice(&(attributes_len as u16).to_be_bytes());
+                if in_body {
+                    put_nlri_list(routes, with_path_id, &mut out);
+                }
+                // As many prefixes went in as fit beside the attributes.
+                finish_message(&mut out, length_at).expect("a chunk that fits is a message");
+            }
+            if out.len() >= ENCODED_PIECE {
+                pieces.push(out.split());
+            }
+        }
+        if !out.is_empty() {
+            pieces.push(out);
+        }
+        pieces
+    }
 }
 
 /// What the next hop takes, in its own attribute or with the prefixes,
@@ -458,84 +538,12 @@ const NEXT_HOP_ROOM: usize = 64;
 /// so it runs over by at most one.
 pub(crate) const ENCODED_PIECE: usize = 64 * 1024;
 
-/// The UPDATE messages, complete with their headers, that announce
-/// `routes`, all of IP version `version`, with the attributes `encoded`
-/// and `next_hop`: as few as they fit in, in pieces of
-/// [`ENCODED_PIECE`] octets or so, each of whole messages. `ctx` is the
-/// session's, and says whether path identifiers go with the family's
-/// prefixes (RFC 7911).
-///
-/// What [`announcements`] builds as messages, built as bytes: the
-/// attributes are not parsed, cloned or encoded again for each message.
-pub(crate) fn encoded_announcements(
-    encoded: &EncodedAttributes,
-    next_hop: NlriNextHop,
-    version: IpVersion,
-    routes: &[Nlri],
-    ctx: &ParserContext,
-) -> Vec<BytesMut> {
-    let room = UPDATE_ROOM.saturating_sub(encoded.octets() + NEXT_HOP_ROOM);
-    let at_once = (room / nlri_octets(version)).max(1);
-    let in_body = matches!(next_hop, NlriNextHop::Ipv4(_)) && version == IpVersion::Ipv4;
-    let with_path_id = ctx.add_path.contains(&AfiSafi::unicast(version));
-    let mut pieces = Vec::new();
-    let mut out = BytesMut::with_capacity(ENCODED_PIECE.min(routes.len() * 64 + 4096));
-    for routes in routes.chunks(at_once) {
-        {
-            out.put_slice(&BGP_MARKER);
-            let length_at = out.len();
-            out.put_u16(0);
-            // The type of an UPDATE (RFC 4271 §4.1).
-            out.put_u8(2);
-            // No withdrawn routes.
-            out.put_u16(0);
-            let attributes_at = out.len();
-            out.put_u16(0);
-            if in_body {
-                let NlriNextHop::Ipv4(next_hop) = next_hop else {
-                    unreachable!("in_body means an IPv4 next hop");
-                };
-                out.put_slice(&encoded.bytes[..encoded.at_next_hop]);
-                PathAttribute::NextHop(NextHopPathAttribute(next_hop))
-                    .to_wire(ctx, &mut out)
-                    .expect("a next hop encodes");
-                out.put_slice(&encoded.bytes[encoded.at_next_hop..]);
-            } else {
-                out.put_slice(&encoded.bytes[..encoded.at_mp_reach]);
-                PathAttribute::MpReachNlri(MpReachNlriPathAttribute {
-                    family: AfiSafi::unicast(version),
-                    next_hop,
-                    prefixes: routes.to_vec(),
-                })
-                .to_wire(ctx, &mut out)
-                .expect("a chunk that fits encodes");
-                out.put_slice(&encoded.bytes[encoded.at_mp_reach..]);
-            }
-            let attributes_len = out.len() - attributes_at - 2;
-            out[attributes_at..attributes_at + 2]
-                .copy_from_slice(&(attributes_len as u16).to_be_bytes());
-            if in_body {
-                write_nlri(routes, with_path_id, &mut out);
-            }
-            let length = out.len() - length_at + BGP_MARKER.len();
-            out[length_at..length_at + 2].copy_from_slice(&(length as u16).to_be_bytes());
-        }
-        if out.len() >= ENCODED_PIECE {
-            pieces.push(out.split());
-        }
-    }
-    if !out.is_empty() {
-        pieces.push(out);
-    }
-    pieces
-}
-
 /// The UPDATEs that announce `routes`, all of IP version `version`, as
 /// `outgoing`: as few as they fit in. `attributes_octets` is how long
 /// `outgoing`'s attributes are on this session's wire.
 ///
-/// What [`encoded_announcements`] does, built as messages: kept as what
+/// What [`EncodedAttributes::announcements`] does, built as messages:
-/// its bytes are checked against.
+/// kept as what its bytes are checked against.
 #[cfg(test)]
 pub(crate) fn announcements(
     outgoing: &Outgoing,
@@ -544,9 +552,6 @@ pub(crate) fn announcements(
     towards: &Towards,
     attributes_octets: usize,
 ) -> Vec<UpdateMessage> {
-    // What the next hop takes, in its own attribute or with the prefixes,
-    // and a little to spare.
-    const NEXT_HOP_ROOM: usize = 64;
     let room = UPDATE_ROOM.saturating_sub(attributes_octets + NEXT_HOP_ROOM);
     let at_once = (room / nlri_octets(version)).max(1);
     let in_body = matches!(outgoing.next_hop, NlriNextHop::Ipv4(_)) && version == IpVersion::Ipv4;
@@ -684,31 +689,47 @@ mod tests {
         PathAttribute::Communities(CommunitiesPathAttribute(communities.to_vec()))
     }
 
-    /// What `encoded_announcements` writes is, byte for byte, what the
+    /// What `EncodedAttributes::announcements` writes is, byte for byte,
-    /// messages `announcements` builds encode to: with the next hop in
+    /// what the messages `announcements` builds encode to: with the next
-    /// its own attribute (IPv4) and in MP_REACH_NLRI (IPv6), with and
+    /// hop in its own attribute (IPv4) and in MP_REACH_NLRI (IPv6, and
-    /// without path identifiers, to two-octet and four-octet AS speakers
+    /// IPv4 with an IPv6 next hop, RFC 8950), with and without path
-    /// (so with the attributes around NEXT_HOP, MP_REACH_NLRI and
+    /// identifiers, to two-octet and four-octet AS speakers (so with the
-    /// AS4_PATH in order), and over as many messages when the routes do
+    /// attributes around NEXT_HOP, MP_REACH_NLRI and AS4_PATH in order),
-    /// not fit in one.
+    /// and over as many messages when the routes do not fit in one.
     #[test]
     fn encoded_announcements_are_the_messages_on_the_wire() {
         use std::net::Ipv6Addr;
 
+        // The routes' IP version, their next hop, and whether the session
+        // takes IPv4 routes with IPv6 next hops.
+        let layouts = [
+            (
+                IpVersion::Ipv4,
+                NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 1)),
+                false,
+            ),
+            (
+                IpVersion::Ipv6,
+                NlriNextHop::Ipv6WithLl {
+                    global: Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1),
+                    link_local: Ipv6Addr::new(0xfe80, 0, 0, 0, 0, 0, 0, 1),
+                },
+                false,
+            ),
+            (
+                IpVersion::Ipv4,
+                NlriNextHop::Ipv6(Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1)),
+                true,
+            ),
+        ];
         for four_octet_asn in [true, false] {
             for with_path_id in [false, true] {
-                for version in [IpVersion::Ipv4, IpVersion::Ipv6] {
+                for (version, next_hop, extended_next_hop) in layouts {
                     let towards = Towards {
                         four_octet_asn,
+                        extended_next_hop,
                         ..towards(None, false)
                     };
-                    let next_hop = match version {
-                        IpVersion::Ipv4 => NlriNextHop::Ipv4(Ipv4Addr::new(192, 0, 2, 1)),
-                        IpVersion::Ipv6 => NlriNextHop::Ipv6WithLl {
-                            global: Ipv6Addr::new(0x2001, 0xdb8, 0, 0, 0, 0, 0, 1),
-                            link_local: Ipv6Addr::new(0xfe80, 0, 0, 0, 0, 0, 0, 1),
-                        },
-                    };
                     let attributes = vec![
                         PathAttribute::Origin(OriginPathAttribute::IGP),
                         AsPathAttribute::from_asns(vec![4_200_000_000, 3356, 64700]),
@@ -753,7 +774,7 @@ mod tests {
                     let mut finished = attributes.clone();
                     finish(&mut finished, &towards);
                     let encoded = EncodedAttributes::encode(&finished, four_octet_asn).unwrap();
-                    let bytes = encoded_announcements(&encoded, next_hop, version, &routes, &ctx);
+                    let bytes = encoded.announcements(next_hop, version, &routes, &ctx);
 
                     let outgoing = Outgoing {
                         attributes,
@@ -771,7 +792,8 @@ mod tests {
                         .collect();
 
                     let what = format!(
-                        "{version:?}, path ids {with_path_id}, four-octet {four_octet_asn}"
+                        "{version:?} with next hop {next_hop:?}, path ids {with_path_id}, \
+                         four-octet {four_octet_asn}"
                     );
                     assert!(expected.len() > 1, "{what}: more than one message");
                     for update in &expected {

crates/server/src/marks.rs

@@ -15,7 +15,8 @@
 
 use bgp_packet::community::{CommunityError, parse_large, parse_standard};
 use bgp_packet::message::{
-    CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity, PathAttribute,
+    CommunitiesPathAttribute, LargeCommunitiesPathAttribute, LargeCommunity,
+    LocalPrefPathAttribute, PathAttribute,
 };
 
 /// A standard community, or all those with one half or both left open:
@@ -186,6 +187,19 @@ impl Marks {
     }
 }
 
+/// Marks `attributes` as a policy or one of its rules says: with `marks`
+/// on their communities, and with `local_pref` as their LOCAL_PREF if one
+/// is given, in place of whatever they had.
+pub fn mark(attributes: &mut Vec<PathAttribute>, marks: &Marks, local_pref: Option<u32>) {
+    marks.apply(attributes);
+    if let Some(local_pref) = local_pref {
+        put(
+            attributes,
+            PathAttribute::LocalPref(LocalPrefPathAttribute(local_pref)),
+        );
+    }
+}
+
 /// Takes the standard communities out of `attributes`.
 fn take_communities(attributes: &mut Vec<PathAttribute>) -> Vec<(u16, u16)> {
     let at = attributes

crates/server/src/metrics.rs

@@ -2,391 +2,296 @@
 
 use std::sync::LazyLock;
 
+use bgp_fib::sockdiag::TcpConnection;
 use bgp_packet::message::Message;
+#[cfg(test)]
+use prometheus::core::Collector;
 use prometheus::{
     GaugeVec, IntCounter, IntCounterVec, IntGauge, IntGaugeVec, register_gauge_vec,
     register_int_counter, register_int_counter_vec, register_int_gauge, register_int_gauge_vec,
 };
 
-pub static MESSAGES_SENT: LazyLock<IntCounterVec> = LazyLock::new(|| {
+/// What a declared metric puts on the registry: one collector for a
-    register_int_counter_vec!(
+/// metric, and one for each gauge of a group of them.
+#[cfg(test)]
+trait Collectors {
+    fn collectors(&self) -> Vec<Box<dyn Collector>>;
+}
+
+#[cfg(test)]
+impl<T: Collector + Clone + 'static> Collectors for T {
+    fn collectors(&self) -> Vec<Box<dyn Collector>> {
+        vec![Box::new(self.clone())]
+    }
+}
+
+/// Declares the metrics: a static for each, made on first use, and
+/// [`init`], which forces every one of them so that nothing is left out
+/// of a scrape for not having been used yet. Each is declared once, here,
+/// so that the two cannot drift apart.
+macro_rules! metrics {
+    ($(
+        $(#[$meta:meta])*
+        $name:ident: $type:ty = $register:expr;
+    )*) => {
+        $(
+            $(#[$meta])*
+            pub static $name: LazyLock<$type> =
+                LazyLock::new(|| $register.expect("metric registration"));
+        )*
+
+        /// Every metric declared above, as the collectors the registry
+        /// holds, for the tests to check it against.
+        #[cfg(test)]
+        fn declared() -> Vec<Box<dyn Collector>> {
+            let mut all = Vec::new();
+            $(all.extend($name.collectors());)*
+            all
+        }
+
+        /// Forces registration of every metric so they appear in scrapes
+        /// before their first observation: a counter that is not there
+        /// until it first counts hides that first count from `increase()`.
+        pub fn init() {
+            $(LazyLock::force(&$name);)*
+            BUILD_INFO
+                .with_label_values(&[env!("CARGO_PKG_VERSION")])
+                .set(1);
+        }
+    };
+}
+
+metrics! {
+    MESSAGES_SENT: IntCounterVec = register_int_counter_vec!(
         "bgp_messages_sent_total",
         "BGP messages sent, by type",
         &["type"]
-    )
+    );
-    .expect("metric registration")
+    MESSAGES_RECEIVED: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static MESSAGES_RECEIVED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_messages_received_total",
         "BGP messages received, by type",
         &["type"]
-    )
+    );
-    .expect("metric registration")
+    UPDATE_ERRORS: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static UPDATE_ERRORS: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_update_errors_total",
         "Errors in received UPDATE messages that did not reset the session, by how they were handled (RFC 7606)",
         &["handling"]
-    )
+    );
-    .expect("metric registration")
+    SESSIONS_ESTABLISHED_TOTAL: IntCounter = register_int_counter!(
-});
-
-pub static SESSIONS_ESTABLISHED_TOTAL: LazyLock<IntCounter> = LazyLock::new(|| {
-    register_int_counter!(
         "bgp_sessions_established_total",
         "Sessions that reached Established"
-    )
+    );
-    .expect("metric registration")
+    SESSIONS_ESTABLISHED: IntGauge = register_int_gauge!(
-});
-
-pub static SESSIONS_ESTABLISHED: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_sessions_established",
         "Sessions currently in Established"
-    )
+    );
-    .expect("metric registration")
+    ROUTES_REJECTED: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static ROUTES_REJECTED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_routes_rejected_total",
         "Routes announced by a peer and refused by its import policy, by why",
         &["peer", "reason"]
-    )
+    );
-    .expect("metric registration")
+    PEER_ROUTES: IntGaugeVec = register_int_gauge_vec!(
-});
-
-pub static PEER_ROUTES: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_peer_routes",
         "Routes of an established peer that are in the routing table",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    PEER_ADVERTISED: IntGaugeVec = register_int_gauge_vec!(
-});
-
-pub static PEER_ADVERTISED: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_peer_advertised_routes",
         "Routes announced to an established peer and not withdrawn since",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    PEER_TCP: PeerTcp = PeerTcp::new();
-});
+    /// The allocator's account of the daemon's memory, by `kind`: allocated,
-
+    /// active, metadata, resident, mapped, retained (jemalloc's `stats.*`).
-/// A session's TCP connection as the kernel reports it (`TCP_INFO`),
+    MEMORY: IntGaugeVec = register_int_gauge_vec!(
-/// sampled when the metrics are scraped. Seconds, octets, segments, and
-/// octets per second; the busy, limited and byte figures are cumulative
-/// over the connection's life.
-pub struct PeerTcp {
-    pub rtt_seconds: GaugeVec,
-    pub min_rtt_seconds: GaugeVec,
-    pub rtt_variance_seconds: GaugeVec,
-    pub cwnd_segments: IntGaugeVec,
-    pub mss_bytes: IntGaugeVec,
-    pub send_window_bytes: IntGaugeVec,
-    pub receive_window_bytes: IntGaugeVec,
-    pub send_queue_bytes: IntGaugeVec,
-    pub not_sent_bytes: IntGaugeVec,
-    pub receive_queue_bytes: IntGaugeVec,
-    pub unacked_segments: IntGaugeVec,
-    pub retransmitted_segments: IntGaugeVec,
-    pub lost_segments: IntGaugeVec,
-    pub busy_seconds: GaugeVec,
-    pub receive_window_limited_seconds: GaugeVec,
-    pub send_buffer_limited_seconds: GaugeVec,
-    pub sent_bytes: IntGaugeVec,
-    pub received_bytes: IntGaugeVec,
-    pub delivery_rate_bytes_per_second: IntGaugeVec,
-    pub pacing_rate_bytes_per_second: IntGaugeVec,
-}
-
-impl PeerTcp {
-    /// Forgets the peer's series, when its session has ended.
-    pub fn remove(&self, label: &str) {
-        for gauge in [
-            &self.rtt_seconds,
-            &self.min_rtt_seconds,
-            &self.rtt_variance_seconds,
-            &self.busy_seconds,
-            &self.receive_window_limited_seconds,
-            &self.send_buffer_limited_seconds,
-        ] {
-            let _ = gauge.remove_label_values(&[label]);
-        }
-        for gauge in [
-            &self.cwnd_segments,
-            &self.mss_bytes,
-            &self.send_window_bytes,
-            &self.receive_window_bytes,
-            &self.send_queue_bytes,
-            &self.not_sent_bytes,
-            &self.receive_queue_bytes,
-            &self.unacked_segments,
-            &self.retransmitted_segments,
-            &self.lost_segments,
-            &self.sent_bytes,
-            &self.received_bytes,
-            &self.delivery_rate_bytes_per_second,
-            &self.pacing_rate_bytes_per_second,
-        ] {
-            let _ = gauge.remove_label_values(&[label]);
-        }
-    }
-}
-
-pub static PEER_TCP: LazyLock<PeerTcp> = LazyLock::new(|| {
-    let seconds = |name: &str, help: &str| {
-        register_gauge_vec!(format!("bgp_peer_tcp_{name}"), help, &["peer"])
-            .expect("metric registration")
-    };
-    let count = |name: &str, help: &str| {
-        register_int_gauge_vec!(format!("bgp_peer_tcp_{name}"), help, &["peer"])
-            .expect("metric registration")
-    };
-    PeerTcp {
-        rtt_seconds: seconds(
-            "rtt_seconds",
-            "Smoothed round-trip time of the session's TCP connection",
-        ),
-        min_rtt_seconds: seconds(
-            "min_rtt_seconds",
-            "Least round-trip time seen on the connection",
-        ),
-        rtt_variance_seconds: seconds(
-            "rtt_variance_seconds",
-            "Round-trip time variance of the connection",
-        ),
-        cwnd_segments: count("cwnd_segments", "Congestion window, in segments"),
-        mss_bytes: count("mss_bytes", "Maximum segment size"),
-        send_window_bytes: count(
-            "send_window_bytes",
-            "The peer's receive window: what it lets us send",
-        ),
-        receive_window_bytes: count(
-            "receive_window_bytes",
-            "Our receive window: what we let the peer send",
-        ),
-        send_queue_bytes: count(
-            "send_queue_bytes",
-            "Octets written and not yet acknowledged by the peer",
-        ),
-        not_sent_bytes: count("not_sent_bytes", "Octets written and not yet sent"),
-        receive_queue_bytes: count("receive_queue_bytes", "Octets received and not yet read"),
-        unacked_segments: count("unacked_segments", "Segments sent and not yet acknowledged"),
-        retransmitted_segments: count(
-            "retransmitted_segments",
-            "Segments retransmitted over the connection's life",
-        ),
-        lost_segments: count("lost_segments", "Segments thought lost now"),
-        busy_seconds: seconds(
-            "busy_seconds",
-            "Time the connection had something to send, over its life",
-        ),
-        receive_window_limited_seconds: seconds(
-            "receive_window_limited_seconds",
-            "Of the busy time, time held up by the peer's receive window",
-        ),
-        send_buffer_limited_seconds: seconds(
-            "send_buffer_limited_seconds",
-            "Of the busy time, time held up by our send buffer being full",
-        ),
-        sent_bytes: count("sent_bytes", "Octets sent over the connection's life"),
-        received_bytes: count(
-            "received_bytes",
-            "Octets received over the connection's life",
-        ),
-        delivery_rate_bytes_per_second: count(
-            "delivery_rate_bytes_per_second",
-            "Delivery rate the kernel measured most recently",
-        ),
-        pacing_rate_bytes_per_second: count(
-            "pacing_rate_bytes_per_second",
-            "Rate the kernel paces the connection at",
-        ),
-    }
-});
-
-/// The allocator's account of the daemon's memory, by `kind`: allocated,
-/// active, metadata, resident, mapped, retained (jemalloc's `stats.*`).
-pub static MEMORY: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_memory_bytes",
         "The daemon's memory as its allocator accounts for it, by kind",
         &["kind"]
-    )
+    );
-    .expect("metric registration")
+    PEER_MAX_PREFIXES: IntGaugeVec = register_int_gauge_vec!(
-});
-
-pub static PEER_MAX_PREFIXES: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_peer_max_prefixes",
         "The most routes a configured peer may have (import.max_prefixes), where there is a limit",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    PREFIX_LIMIT_EXCEEDED: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static PREFIX_LIMIT_EXCEEDED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_prefix_limit_exceeded_total",
         "Sessions ended because the peer went over its limit of routes",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    EXPORT_LIMIT_EXCEEDED: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static EXPORT_LIMIT_EXCEEDED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_export_limit_exceeded_total",
         "Sessions ended because more routes were to be announced to the peer than its export limit allows",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    RPKI_VRPS: IntGaugeVec = register_int_gauge_vec!(
-});
-
-pub static RPKI_VRPS: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_rpki_vrps",
         "Validated ROA payloads in use for route origin validation, by address family",
         &["family"]
-    )
+    );
-    .expect("metric registration")
+    RPKI_ASPAS: IntGauge = register_int_gauge!(
-});
-
-pub static RPKI_ASPAS: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_rpki_aspas",
         "Customer ASes with an ASPA in use for AS path verification"
-    )
+    );
-    .expect("metric registration")
+    RPKI_UPDATES: IntCounter = register_int_counter!(
-});
+        "bgp_rpki_updates_total",
-
+        "Times the VRPs or ASPAs changed"
-pub static RPKI_UPDATES: LazyLock<IntCounter> = LazyLock::new(|| {
+    );
-    register_int_counter!("bgp_rpki_updates_total", "Times the VRPs or ASPAs changed")
+    CONNECTIONS_REFUSED: IntCounterVec = register_int_counter_vec!(
-        .expect("metric registration")
-});
-
-pub static CONNECTIONS_REFUSED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_connections_refused_total",
         "Incoming connections closed without a session, by why",
         &["reason"]
-    )
+    );
-    .expect("metric registration")
+    COLLISIONS: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static COLLISIONS: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_connection_collisions_total",
         "Connection collisions resolved, by which connection was closed",
         &["closed"]
-    )
+    );
-    .expect("metric registration")
+    LOCAL_ROUTES: IntGauge = register_int_gauge!(
-});
+        "bgp_local_routes",
-
+        "Locally originated routes"
-pub static LOCAL_ROUTES: LazyLock<IntGauge> = LazyLock::new(|| {
+    );
-    register_int_gauge!("bgp_local_routes", "Locally originated routes")
+    RIB_ROUTES: IntGauge = register_int_gauge!(
-        .expect("metric registration")
-});
-
-pub static RIB_ROUTES: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_rib_routes",
         "Routes learned from peers, counting each peer's separately"
-    )
+    );
-    .expect("metric registration")
+    RIB_PREFIXES: IntGauge = register_int_gauge!(
-});
-
-pub static RIB_PREFIXES: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_rib_prefixes",
         "Prefixes with at least one route, and so with a best route"
-    )
+    );
-    .expect("metric registration")
+    RIB_ATTRIBUTE_SETS: IntGauge = register_int_gauge!(
-});
-
-pub static RIB_ATTRIBUTE_SETS: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_rib_attribute_sets",
         "Distinct sets of path attributes held for the routes"
-    )
+    );
-    .expect("metric registration")
+    RIB_JOURNAL_ENTRIES: IntGauge = register_int_gauge!(
-});
-
-pub static RIB_JOURNAL_ENTRIES: LazyLock<IntGauge> = LazyLock::new(|| {
-    register_int_gauge!(
         "bgp_rib_journal_entries",
         "Changes of best route waiting for a follower to read them"
-    )
+    );
-    .expect("metric registration")
+    ROUTES_EXPORTED: IntCounterVec = register_int_counter_vec!(
-});
-
-pub static ROUTES_EXPORTED: LazyLock<IntCounterVec> = LazyLock::new(|| {
-    register_int_counter_vec!(
         "bgp_routes_exported_total",
         "Routes learned from one peer and announced to, or withdrawn from, another",
         &["change"]
-    )
+    );
-    .expect("metric registration")
+    /// Whether each configured peer's session is up: what to alert on when a
-});
+    /// peer goes away, which a count of sessions cannot say.
-
+    PEER_UP: IntGaugeVec = register_int_gauge_vec!(
-/// Whether each configured peer's session is up: what to alert on when a
-/// peer goes away, which a count of sessions cannot say.
-pub static PEER_UP: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_peer_up",
         "Whether the session with a configured peer is Established (1) or not (0)",
         &["peer"]
-    )
+    );
-    .expect("metric registration")
+    BUILD_INFO: IntGaugeVec = register_int_gauge_vec!(
-});
-
-pub static BUILD_INFO: LazyLock<IntGaugeVec> = LazyLock::new(|| {
-    register_int_gauge_vec!(
         "bgp_build_info",
         "Always 1; the version of the daemon is in the label",
         &["version"]
-    )
+    );
-    .expect("metric registration")
+}
-});
 
-/// Forces registration of every metric so they appear in scrapes before
+/// The type of a gauge of [`PeerTcp`] by what it holds: a time, in
-/// their first observation: a counter that is not there until it first
+/// seconds, or a count.
-/// counts hides that first count from `increase()`.
+macro_rules! tcp_gauge {
-pub fn init() {
+    (seconds) => {
-    LazyLock::force(&MESSAGES_SENT);
+        GaugeVec
-    LazyLock::force(&MESSAGES_RECEIVED);
+    };
-    LazyLock::force(&UPDATE_ERRORS);
+    (count) => {
-    LazyLock::force(&SESSIONS_ESTABLISHED_TOTAL);
+        IntGaugeVec
-    LazyLock::force(&SESSIONS_ESTABLISHED);
+    };
-    LazyLock::force(&ROUTES_REJECTED);
+}
-    LazyLock::force(&PEER_ROUTES);
+
-    LazyLock::force(&PEER_MAX_PREFIXES);
+/// Registers a gauge of [`PeerTcp`], with the peer as its one label.
-    LazyLock::force(&PREFIX_LIMIT_EXCEEDED);
+macro_rules! register_tcp_gauge {
-    LazyLock::force(&EXPORT_LIMIT_EXCEEDED);
+    (seconds, $name:expr, $help:expr) => {
-    LazyLock::force(&RPKI_VRPS);
+        register_gauge_vec!($name, $help, &["peer"])
-    LazyLock::force(&RPKI_ASPAS);
+    };
-    LazyLock::force(&RPKI_UPDATES);
+    (count, $name:expr, $help:expr) => {
-    LazyLock::force(&CONNECTIONS_REFUSED);
+        register_int_gauge_vec!($name, $help, &["peer"])
-    LazyLock::force(&COLLISIONS);
+    };
-    LazyLock::force(&LOCAL_ROUTES);
+}
-    LazyLock::force(&RIB_ROUTES);
+
-    LazyLock::force(&RIB_PREFIXES);
+/// What the kernel's figure becomes on the gauge: microseconds become
-    LazyLock::force(&RIB_ATTRIBUTE_SETS);
+/// seconds, and a count is clamped into what a gauge holds.
-    LazyLock::force(&RIB_JOURNAL_ENTRIES);
+macro_rules! tcp_gauge_value {
-    LazyLock::force(&ROUTES_EXPORTED);
+    (seconds, $microseconds:expr) => {
-    LazyLock::force(&PEER_UP);
+        $microseconds as f64 / 1e6
-    LazyLock::force(&PEER_ADVERTISED);
+    };
-    LazyLock::force(&PEER_TCP);
+    (count, $count:expr) => {
-    LazyLock::force(&MEMORY);
+        i64::try_from($count).unwrap_or(i64::MAX)
-    BUILD_INFO
+    };
-        .with_label_values(&[env!("CARGO_PKG_VERSION")])
+}
-        .set(1);
+
+/// Declares the gauges of [`PeerTcp`], each named once with its help text
+/// and the field of [`TcpConnection`] it is set from: the struct, how it is
+/// made, and what sets and removes a peer's series all follow from the one
+/// list. The gauge `name` is `bgp_peer_tcp_name`.
+macro_rules! peer_tcp {
+    ($connection:ident; $(
+        $name:ident: $kind:ident from $source:ident = $help:literal,
+    )*) => {
+        /// A session's TCP connection as the kernel reports it (`TCP_INFO`),
+        /// sampled when the metrics are scraped. Seconds, octets, segments, and
+        /// octets per second; the busy, limited and byte figures are cumulative
+        /// over the connection's life.
+        pub struct PeerTcp {
+            $(pub $name: tcp_gauge!($kind),)*
+        }
+
+        #[cfg(test)]
+        impl Collectors for PeerTcp {
+            fn collectors(&self) -> Vec<Box<dyn Collector>> {
+                vec![$(Box::new(self.$name.clone()),)*]
+            }
+        }
+
+        impl PeerTcp {
+            fn new() -> prometheus::Result<Self> {
+                Ok(Self {
+                    $($name: register_tcp_gauge!(
+                        $kind,
+                        concat!("bgp_peer_tcp_", stringify!($name)),
+                        $help
+                    )?,)*
+                })
+            }
+
+            /// Sets the peer's gauges from what the kernel reports of its
+            /// connection.
+            pub fn set(&self, label: &str, $connection: &TcpConnection) {
+                let label = [label];
+                $(
+                    self.$name
+                        .with_label_values(&label)
+                        .set(tcp_gauge_value!($kind, u64::from($connection.$source)));
+                )*
+            }
+
+            /// Forgets the peer's series, when its session has ended.
+            pub fn remove(&self, label: &str) {
+                $(let _ = self.$name.remove_label_values(&[label]);)*
+            }
+        }
+    };
+}
+
+peer_tcp! {
+    connection;
+    rtt_seconds: seconds from rtt = "Smoothed round-trip time of the session's TCP connection",
+    min_rtt_seconds: seconds from min_rtt = "Least round-trip time seen on the connection",
+    rtt_variance_seconds: seconds from rtt_variance = "Round-trip time variance of the connection",
+    cwnd_segments: count from cwnd = "Congestion window, in segments",
+    mss_bytes: count from mss = "Maximum segment size",
+    send_window_bytes: count from send_window = "The peer's receive window: what it lets us send",
+    receive_window_bytes: count from receive_window = "Our receive window: what we let the peer send",
+    send_queue_bytes: count from send_queue = "Octets written and not yet acknowledged by the peer",
+    not_sent_bytes: count from not_sent = "Octets written and not yet sent",
+    receive_queue_bytes: count from recv_queue = "Octets received and not yet read",
+    unacked_segments: count from unacked = "Segments sent and not yet acknowledged",
+    retransmitted_segments: count from total_retrans = "Segments retransmitted over the connection's life",
+    lost_segments: count from lost = "Segments thought lost now",
+    busy_seconds: seconds from busy = "Time the connection had something to send, over its life",
+    receive_window_limited_seconds: seconds from limited_by_receive_window = "Of the busy time, time held up by the peer's receive window",
+    send_buffer_limited_seconds: seconds from limited_by_send_buffer = "Of the busy time, time held up by our send buffer being full",
+    sent_bytes: count from bytes_sent = "Octets sent over the connection's life",
+    received_bytes: count from bytes_received = "Octets received over the connection's life",
+    delivery_rate_bytes_per_second: count from delivery_rate = "Delivery rate the kernel measured most recently",
+    pacing_rate_bytes_per_second: count from pacing_rate = "Rate the kernel paces the connection at",
 }
 
 pub fn message_type_name(msg: &Message) -> &'static str {
@@ -398,3 +303,67 @@ pub fn message_type_name(msg: &Message) -> &'static str {
         Message::RouteRefresh(_) => "route_refresh",
     }
 }
+
+#[cfg(test)]
+mod tests {
+    use std::collections::HashSet;
+
+    use super::*;
+
+    /// Every metric family the declaration lists is on the registry once
+    /// `init` has run, each under a name of its own: the registry refuses
+    /// a second collector for any of them. The registry is the process's,
+    /// so what else is on it is not looked at.
+    #[test]
+    fn init_registers_every_declared_metric_family() {
+        init();
+        let declared = declared();
+        let names: HashSet<String> = declared
+            .iter()
+            .flat_map(|collector| collector.desc())
+            .map(|desc| desc.fq_name.clone())
+            .collect();
+        assert_eq!(names.len(), declared.len(), "a name is declared twice");
+        for collector in declared {
+            assert!(matches!(
+                prometheus::register(collector),
+                Err(prometheus::Error::AlreadyReg)
+            ));
+        }
+        // A metric without labels is in a scrape from the start, which is
+        // what `init` is for; one with labels is there once it has one.
+        let scraped: HashSet<String> = prometheus::gather()
+            .iter()
+            .map(|family| family.name().to_string())
+            .collect();
+        for name in [
+            "bgp_sessions_established",
+            "bgp_rib_routes",
+            "bgp_build_info",
+        ] {
+            assert!(scraped.contains(name), "{name} is not in a scrape");
+        }
+    }
+
+    /// The kernel's microseconds become seconds, and its counts are set as
+    /// they are; a peer whose session ended loses every series.
+    #[test]
+    fn a_connection_sets_and_a_removal_clears_every_gauge() {
+        let tcp = &*PEER_TCP;
+        let connection = TcpConnection {
+            rtt: 1500,
+            cwnd: 10,
+            bytes_sent: 1 << 40,
+            ..TcpConnection::default()
+        };
+        tcp.set("metrics-test", &connection);
+        let label = ["metrics-test"];
+        assert_eq!(tcp.rtt_seconds.with_label_values(&label).get(), 0.0015);
+        assert_eq!(tcp.cwnd_segments.with_label_values(&label).get(), 10);
+        assert_eq!(tcp.sent_bytes.with_label_values(&label).get(), 1 << 40);
+        tcp.remove("metrics-test");
+        // There is nothing left to remove.
+        assert!(tcp.rtt_seconds.remove_label_values(&label).is_err());
+        assert!(tcp.sent_bytes.remove_label_values(&label).is_err());
+    }
+}

crates/server/src/policy.rs

@@ -29,9 +29,7 @@ use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
 use std::sync::{Arc, LazyLock};
 use std::time::Duration;
 
-use bgp_packet::message::{
+use bgp_packet::message::{AS_TRANS, AsPathAttribute, NlriNextHop, PathAttribute};
-    AS_TRANS, AsPathAttribute, LocalPrefPathAttribute, NlriNextHop, PathAttribute,
-};
 use bgp_packet::open::BgpRole;
 use bgp_primitives::constants::IpVersion;
 use bgp_primitives::ip_prefix::IpPrefix;
@@ -91,10 +89,10 @@ pub struct ImportPolicy {
     /// who never asked for it. A route server's routes always have its
     /// clients as next hops, and are taken either way.
     pub accept_third_party_next_hop: bool,
-    /// Take a next hop that is a loopback address, or one of our own: for
+    /// Take a next hop that is on this host: a loopback address, or one of
-    /// routers that share a host, as those of a test do. Not a setting;
+    /// our own. For routers that share a host, as those of a test do. Not
-    /// only [`ImportPolicy::permissive`] has it.
+    /// a setting; only [`ImportPolicy::permissive`] has it.
-    pub accept_loopback_next_hop: bool,
+    pub accept_own_host_next_hop: bool,
     /// Take routes with the BLACKHOLE community for single addresses, and
     /// have the kernel drop the traffic to them (RFC 7999).
     pub accept_blackholes: bool,
@@ -174,13 +172,7 @@ impl ImportRule {
     }
 
     pub fn apply(&self, attributes: &mut Vec<PathAttribute>) {
-        self.marks.apply(attributes);
+        marks::mark(attributes, &self.marks, self.local_pref);
-        if let Some(local_pref) = self.local_pref {
-            marks::put(
-                attributes,
-                PathAttribute::LocalPref(LocalPrefPathAttribute(local_pref)),
-            );
-        }
     }
 }
 
@@ -203,7 +195,7 @@ impl Default for ImportPolicy {
             accept_aspa_invalid: false,
             accept_as_sets: false,
             accept_third_party_next_hop: false,
-            accept_loopback_next_hop: false,
+            accept_own_host_next_hop: false,
             accept_blackholes: false,
             own_prefixes: Arc::default(),
             originated: Arc::default(),
@@ -233,7 +225,7 @@ impl ImportPolicy {
             accept_aspa_invalid: true,
             accept_as_sets: true,
             accept_third_party_next_hop: true,
-            accept_loopback_next_hop: true,
+            accept_own_host_next_hop: true,
             accept_blackholes: false,
             ..Self::default()
         }
@@ -256,13 +248,7 @@ impl ImportPolicy {
         if let Some(scrub) = self.scrub {
             scrub.apply(attributes);
         }
-        self.marks.apply(attributes);
+        marks::mark(attributes, &self.marks, self.local_pref);
-        if let Some(local_pref) = self.local_pref {
-            marks::put(
-                attributes,
-                PathAttribute::LocalPref(LocalPrefPathAttribute(local_pref)),
-            );
-        }
     }
 }
 
@@ -660,11 +646,24 @@ impl ImportPolicy {
                 return Err(Rejection::BogonPrefix);
             }
         }
+        self.listed_or_ours(prefix, PrefixSet::allows)
+    }
+
+    /// Whether `prefix` is one the peer may announce by its list, where it
+    /// has one, and else not one of ours. `inside` says what being in a
+    /// set means: [`PrefixSet::allows`] for a route, which is bound by the
+    /// lengths allowed, [`PrefixSet::covers`] for a blackhole request,
+    /// which is for one address whatever the lengths.
+    fn listed_or_ours(
+        &self,
+        prefix: &IpPrefix,
+        inside: fn(&PrefixSet, &IpPrefix) -> bool,
+    ) -> Result<(), Rejection> {
         match &self.prefixes {
-            Some(allowed) if !allowed.allows(prefix) => Err(Rejection::PrefixNotAllowed),
+            Some(allowed) if !inside(allowed, prefix) => Err(Rejection::PrefixNotAllowed),
             // Said to be this peer's to announce, ours or not.
             Some(_) => Ok(()),
-            None if self.own_prefixes.allows(prefix) => Err(Rejection::OwnPrefix),
+            None if inside(&self.own_prefixes, prefix) => Err(Rejection::OwnPrefix),
             None => Ok(()),
         }
     }
@@ -690,12 +689,7 @@ impl ImportPolicy {
         if !self.accept_bogons && is_bogon_prefix(prefix) {
             return Err(Rejection::BogonPrefix);
         }
-        match &self.prefixes {
+        self.listed_or_ours(prefix, PrefixSet::covers)
-            Some(allowed) if !allowed.covers(prefix) => Err(Rejection::PrefixNotAllowed),
-            Some(_) => Ok(()),
-            None if self.own_prefixes.covers(prefix) => Err(Rejection::OwnPrefix),
-            None => Ok(()),
-        }
     }
 
     /// Judges the next hop of routes of IP version `routes`.
@@ -735,7 +729,7 @@ impl ImportPolicy {
             let usable = !address.is_unspecified()
                 && !address.is_multicast()
                 && address != IpAddr::V4(Ipv4Addr::BROADCAST)
-                && (self.accept_loopback_next_hop
+                && (self.accept_own_host_next_hop
                     || (!address.is_loopback() && address != from.local_address));
             if !usable {
                 return Err(Rejection::NextHopUnusable);

crates/server/src/reach.rs

@@ -29,9 +29,9 @@ use bgp_primitives::constants::IpVersion;
 use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
 use bgp_rib::table::{PeerIndex, RibTable};
 use tokio::task::JoinHandle;
-use tracing::{info, warn};
 
 use crate::address::PeerAddress;
+use crate::rpki::{by_rib_peer, group_by_peer, revalidate_all};
 use crate::session::SessionRegistry;
 
 /// How long after the interfaces change the table is looked through:
@@ -130,37 +130,18 @@ pub(crate) fn spawn_revalidator(
                 continue;
             };
             let sessions = registry.list();
-            // By peer, for looking up as often as there are routes.
+            let neighbors = by_rib_peer(
-            let mut neighbors: Vec<Option<Neighbor>> = Vec::new();
+                sessions
-            for session in &sessions {
+                    .iter()
-                let at = session.rib_peer.as_usize();
+                    .map(|session| (session.rib_peer, neighbor_at(session.peer.addr))),
-                if neighbors.len() <= at {
-                    neighbors.resize(at + 1, None);
-                }
-                neighbors[at] = Some(neighbor_at(session.peer.addr));
-            }
-            let table = Arc::clone(&rib);
-            // Looks at every route: not for a thread that sessions run on.
-            let found =
-                tokio::task::spawn_blocking(move || now_unreachable(&table, &topology, &neighbors))
-                    .await;
-            let mut suspects = match found {
-                Ok(suspects) => suspects,
-                Err(e) => {
-                    warn!(err = %e, "Looking for routes whose next hop is gone failed");
-                    HashMap::new()
-                }
-            };
-            info!(
-                now_unreachable = suspects.values().map(Vec::len).sum::<usize>(),
-                sessions = sessions.len(),
-                "The machine's interfaces or routes changed; looking at next hops again"
             );
-            // Every session, also those with nothing found: each has its
+            revalidate_all(
-            // held routes to look at again.
+                sessions,
-            for session in sessions {
+                &rib,
-                session.revalidate(suspects.remove(&session.rib_peer).unwrap_or_default());
+                "The machine's interfaces or routes changed; looking at next hops again",
-            }
+                move |rib| now_unreachable(rib, &topology, &neighbors),
+            )
+            .await;
         }
     })
 }
@@ -211,14 +192,7 @@ fn now_unreachable(
             .is_ok()
         })
     });
-    let mut by_peer: HashMap<PeerIndex, Vec<Nlri>> = HashMap::new();
+    group_by_peer(routes)
-    for route in routes {
-        by_peer.entry(route.peer).or_default().push(Nlri {
-            prefix: route.prefix,
-            path_id: route.path_id,
-        });
-    }
-    by_peer
 }
 
 #[cfg(test)]
@@ -370,7 +344,7 @@ mod tests {
         }
 
         fn best(&self, route: &str) -> Option<PeerIndex> {
-            self.table.best(&prefix(route)).map(|(peer, _)| peer)
+            self.table.best(&prefix(route)).map(|best| best.peer)
         }
     }
 

crates/server/src/restart.rs

@@ -244,6 +244,30 @@ pub struct Kept {
     pub until: SystemTime,
 }
 
+impl Kept {
+    /// What was kept of a peer: `routes` routes of the IP versions
+    /// `versions`, under `rib_peer`, of the peer at `addr` called `name`.
+    /// Which restart they are of and until when they are kept are the
+    /// table's to say, in [`Restarts::keep`].
+    pub(crate) fn new(
+        addr: SocketAddr,
+        name: Option<String>,
+        rib_peer: PeerIndex,
+        versions: Vec<IpVersion>,
+        routes: usize,
+    ) -> Self {
+        Self {
+            id: 0,
+            addr,
+            rib_peer,
+            name,
+            versions,
+            routes,
+            until: SystemTime::UNIX_EPOCH,
+        }
+    }
+}
+
 /// What there is of graceful restart while the server runs: the routes
 /// kept of peers that are restarting, and how far the peers are with
 /// sending their routes after this daemon started.
@@ -298,33 +322,14 @@ impl Restarts {
         })
     }
 
-    /// Notes that the routes of `peer` under `rib_peer` have been kept
+    /// Notes that `kept` says what routes of `peer` have been kept, until
-    /// (`routes` of them, of the IP versions `versions`) until `for_at_most`
+    /// `for_at_most` from now, and has them removed then unless the peer
-    /// from now, and has them removed then unless the peer is back.
+    /// is back.
-    #[allow(clippy::too_many_arguments)]
+    pub(crate) fn keep(self: &Arc<Self>, peer: PeerKey, mut kept: Kept, for_at_most: Duration) {
-    pub(crate) fn keep(
-        self: &Arc<Self>,
-        peer: PeerKey,
-        addr: SocketAddr,
-        name: Option<String>,
-        rib_peer: PeerIndex,
-        versions: Vec<IpVersion>,
-        routes: usize,
-        for_at_most: Duration,
-    ) {
         let id = self.ids.fetch_add(1, Ordering::Relaxed);
-        self.kept.locked().insert(
+        kept.id = id;
-            peer,
+        kept.until = SystemTime::now() + for_at_most;
-            Kept {
+        self.kept.locked().insert(peer, kept);
-                id,
-                addr,
-                rib_peer,
-                name,
-                versions,
-                routes,
-                until: SystemTime::now() + for_at_most,
-            },
-        );
         let restarts = Arc::clone(self);
         tokio::spawn(async move {
             tokio::time::sleep(for_at_most).await;

crates/server/src/routes.rs

@@ -21,6 +21,14 @@ use crate::rpki::Checks;
 /// than a peer sending the whole Internet has; a bound all the same.
 const MAX_HELD: usize = 200_000;
 
+/// Why an UPDATE could not be applied: the attributes its routes were to
+/// be kept with could not be written in the form the table keeps them in.
+/// Attributes that were decoded from a message always can be, so this is
+/// a bug here, and not anything the peer did.
+#[derive(Debug, thiserror::Error)]
+#[error("could not keep the routes' attributes: {0}")]
+pub struct Unstorable(#[from] pub(crate) EncodeError);
+
 /// What applying an UPDATE did.
 #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
 pub(crate) struct Applied {
@@ -113,7 +121,7 @@ pub(crate) fn apply(
     held: &mut Held,
     update: &UpdateMessage,
     attributes_wire: Option<&[u8]>,
-) -> Result<Applied, EncodeError> {
+) -> Result<Applied, Unstorable> {
     let mut applying = Applying {
         table,
         import,
@@ -598,3 +606,254 @@ pub(crate) fn revalidate(
     }
     applied
 }
+
+#[cfg(test)]
+mod tests {
+    use std::net::{IpAddr, Ipv4Addr};
+
+    use bgp_packet::message::{AsPathAttribute, NextHopPathAttribute, OriginPathAttribute};
+    use bgp_rib::table::PeerInfo;
+    use bgp_rpki::{Vrp, VrpTable};
+
+    use super::*;
+    use crate::rejections::RejectionLog;
+
+    const PEER_ASN: u32 = 65001;
+    const PEER_ADDRESS: Ipv4Addr = Ipv4Addr::new(192, 0, 2, 1);
+
+    fn prefix(text: &str) -> IpPrefix {
+        IpPrefix::try_from(text).unwrap()
+    }
+
+    fn nlri(text: &str) -> Nlri {
+        prefix(text).into()
+    }
+
+    fn attrs_of(route: &str) -> Arc<AttrSet> {
+        RibTable::new(1)
+            .attrs()
+            .intern(&announcement(route).path_attributes, None)
+            .unwrap()
+    }
+
+    /// An UPDATE as the peer announces `route`, originating it itself.
+    fn announcement(route: &str) -> UpdateMessage {
+        UpdateMessage {
+            withdrawn_routes: vec![],
+            path_attributes: vec![
+                PathAttribute::Origin(OriginPathAttribute::IGP),
+                AsPathAttribute::from_asns(vec![PEER_ASN]),
+                PathAttribute::NextHop(NextHopPathAttribute(PEER_ADDRESS)),
+            ],
+            announced_routes: vec![nlri(route)],
+        }
+    }
+
+    /// VRPs that allow `route` to be originated by `asn` and nobody else.
+    fn vrps(route: &str, asn: u32) -> VrpTable {
+        let prefix = prefix(route);
+        VrpTable::new([Vrp {
+            prefix,
+            max_length: prefix.length(),
+            asn,
+        }])
+    }
+
+    /// A table with the peer in it, and what is needed to apply its
+    /// UPDATEs by a permissive policy, checking only against `vrps`.
+    struct Fixture {
+        table: RibTable,
+        peer: PeerIndex,
+        policy: ImportPolicy,
+        rejections: RejectionLog,
+    }
+
+    impl Fixture {
+        fn new() -> Self {
+            let table = RibTable::new(1);
+            let peer = table
+                .add_peer(PeerInfo {
+                    asn: PEER_ASN,
+                    router_id: Ipv4Addr::new(10, 0, 0, 2),
+                    address: PEER_ADDRESS.into(),
+                    interface: 0,
+                    ibgp: false,
+                    reflector_client: false,
+                })
+                .unwrap();
+            Self {
+                table,
+                peer,
+                policy: ImportPolicy::permissive(),
+                rejections: RejectionLog::default(),
+            }
+        }
+
+        fn with_import<T>(
+            &self,
+            vrps: Option<&VrpTable>,
+            held: &mut Held,
+            work: impl FnOnce(&RibTable, &Import<'_>, &mut Held) -> T,
+        ) -> T {
+            let import = Import {
+                peer: self.peer,
+                from: RouteSource {
+                    peer_asn: PEER_ASN,
+                    local_asn: 65000,
+                    ibgp: false,
+                    four_octet_asn: true,
+                    role: None,
+                    peer_address: PEER_ADDRESS.into(),
+                    local_address: Ipv4Addr::new(192, 0, 2, 9).into(),
+                    extended_next_hop: false,
+                },
+                policy: &self.policy,
+                checks: Checks {
+                    vrps,
+                    ..Checks::NONE
+                },
+                rejections: &self
+                    .rejections
+                    .of_peer(IpAddr::from(PEER_ADDRESS).into(), "test"),
+            };
+            work(&self.table, &import, held)
+        }
+
+        fn announce(&self, vrps: Option<&VrpTable>, held: &mut Held, route: &str) -> Applied {
+            self.with_import(vrps, held, |table, import, held| {
+                apply(table, import, held, &announcement(route), None).unwrap()
+            })
+        }
+
+        fn revalidate(
+            &self,
+            vrps: Option<&VrpTable>,
+            held: &mut Held,
+            suspects: &[Nlri],
+        ) -> Applied {
+            self.with_import(vrps, held, |table, import, held| {
+                revalidate(table, import, held, suspects)
+            })
+        }
+    }
+
+    /// Past the bound, a new route is dropped and the overflow noted, so
+    /// that the peer can be asked for it later; one already held is still
+    /// replaced, since that takes no more room.
+    #[test]
+    fn holding_stops_at_the_bound_and_says_so() {
+        let mut held = Held::default();
+        let attrs = attrs_of("198.51.100.0/24");
+        for n in 0..MAX_HELD as u32 {
+            let [a, b, c, d] = (0x0a00_0000 + n).to_be_bytes();
+            held.hold(
+                IpPrefix::new(IpVersion::Ipv4, [a, b, c, d], 32)
+                    .unwrap()
+                    .into(),
+                &attrs,
+            );
+        }
+        assert_eq!(held.len(), MAX_HELD);
+        assert!(!held.take_overflowed());
+
+        held.hold(nlri("198.51.100.0/24"), &attrs);
+        assert_eq!(held.len(), MAX_HELD);
+        assert!(held.take_overflowed());
+        assert!(
+            !held.take_overflowed(),
+            "noted once, until it happens again"
+        );
+
+        let other = attrs_of("203.0.113.0/24");
+        held.hold(nlri("10.0.0.0/32"), &other);
+        assert_eq!(held.len(), MAX_HELD);
+        assert!(!held.take_overflowed());
+        assert!(Arc::ptr_eq(&held.routes[&nlri("10.0.0.0/32")], &other));
+    }
+
+    #[test]
+    fn forgetting_takes_out_what_is_named_and_nothing_else() {
+        let mut held = Held::default();
+        let attrs = attrs_of("198.51.100.0/24");
+        held.hold(nlri("198.51.100.0/24"), &attrs);
+        held.hold(nlri("203.0.113.0/24"), &attrs);
+
+        held.forget(&[nlri("198.51.100.0/24"), nlri("192.0.2.0/24")]);
+        assert_eq!(held.len(), 1);
+        assert!(held.routes.contains_key(&nlri("203.0.113.0/24")));
+
+        held.forget(&[nlri("203.0.113.0/24")]);
+        held.forget(&[nlri("203.0.113.0/24")]);
+        assert_eq!(held.len(), 0);
+    }
+
+    /// A route the VRPs find invalid stays out of the table but is not
+    /// lost: when the VRPs change in its favour it goes in, without the
+    /// peer having to send it again.
+    #[test]
+    fn a_held_route_is_released_once_the_checks_pass_it() {
+        let fixture = Fixture::new();
+        let mut held = Held::default();
+        let route = "198.51.100.0/24";
+        let against = vrps(route, PEER_ASN + 1);
+        let applied = fixture.announce(Some(&against), &mut held, route);
+        assert_eq!(applied.rejected, 1);
+        assert_eq!(applied.outcome.added, 0);
+        assert!(fixture.table.is_empty());
+        assert_eq!(held.len(), 1);
+
+        // Nothing has changed in the route's favour yet.
+        let applied = fixture.revalidate(Some(&against), &mut held, &[]);
+        assert_eq!(applied, Applied::default());
+        assert_eq!(held.len(), 1);
+
+        let for_it = vrps(route, PEER_ASN);
+        let applied = fixture.revalidate(Some(&for_it), &mut held, &[]);
+        assert_eq!(applied.outcome.added, 1);
+        assert_eq!(applied.rejected, 0);
+        assert_eq!(held.len(), 0);
+        assert_eq!(fixture.table.routes(&prefix(route)).len(), 1);
+    }
+
+    /// The other way round: a route in the table that the VRPs come to
+    /// find invalid is taken out and held, and comes back once they
+    /// change again.
+    #[test]
+    fn a_route_turned_invalid_is_held_and_comes_back() {
+        let fixture = Fixture::new();
+        let mut held = Held::default();
+        let route = "198.51.100.0/24";
+        let for_it = vrps(route, PEER_ASN);
+        let applied = fixture.announce(Some(&for_it), &mut held, route);
+        assert_eq!(applied.outcome.added, 1);
+        assert_eq!(fixture.table.len(), 1);
+
+        let against = vrps(route, PEER_ASN + 1);
+        let applied = fixture.revalidate(Some(&against), &mut held, &[nlri(route)]);
+        assert_eq!(applied.rejected, 1);
+        assert_eq!(applied.outcome.removed, 1);
+        assert!(fixture.table.is_empty());
+        assert_eq!(held.len(), 1);
+
+        let applied = fixture.revalidate(Some(&for_it), &mut held, &[]);
+        assert_eq!(applied.outcome.added, 1);
+        assert_eq!(held.len(), 0);
+        assert_eq!(fixture.table.len(), 1);
+    }
+
+    /// Without anything to check against there is nothing to look at
+    /// again: what is held stays held, and what is in the table stays.
+    #[test]
+    fn revalidation_without_checks_changes_nothing() {
+        let fixture = Fixture::new();
+        let mut held = Held::default();
+        let route = "198.51.100.0/24";
+        fixture.announce(Some(&vrps(route, PEER_ASN + 1)), &mut held, route);
+        assert_eq!(held.len(), 1);
+
+        let applied = fixture.revalidate(None, &mut held, &[nlri(route)]);
+        assert_eq!(applied, Applied::default());
+        assert_eq!(held.len(), 1);
+        assert!(fixture.table.is_empty());
+    }
+}

crates/server/src/rpki.rs

@@ -22,7 +22,7 @@ use std::sync::Arc;
 use std::time::Duration;
 
 use bgp_primitives::ip_prefix::{IpPrefix, Nlri};
-use bgp_rib::table::{AttrSet, PeerIndex, RibTable};
+use bgp_rib::table::{AttrSet, PeerIndex, RibTable, Route};
 use bgp_rpki::{Algorithm, AspaTable, Changed, PathValidity, Rpki, Update, Validity, VrpTable};
 use tokio::task::JoinHandle;
 use tracing::{info, warn};
@@ -239,52 +239,92 @@ pub(crate) fn spawn_revalidator(
             RPKI_UPDATES.inc();
 
             tokio::time::sleep(SETTLE).await;
-            let sessions: Vec<(SessionHandle, Subject)> = registry
+            // The sessions whose routes are subject to the checks at all.
+            let sessions: Vec<SessionHandle> = registry
                 .list()
                 .into_iter()
-                .filter_map(|session| {
+                .filter(|session| session.validated().is_some())
-                    let subject = session.validated()?;
-                    Some((session, subject))
-                })
                 .collect();
-            // By peer, for looking up as often as there are routes.
+            let subjects = by_rib_peer(
-            let mut subjects: Vec<Option<Subject>> = Vec::new();
+                sessions
-            for (session, subject) in &sessions {
+                    .iter()
-                let at = session.rib_peer.as_usize();
+                    .filter_map(|session| Some((session.rib_peer, session.validated()?))),
-                if subjects.len() <= at {
-                    subjects.resize(at + 1, None);
-                }
-                subjects[at] = Some(*subject);
-            }
-            let (vrps, aspas) = (update.vrps.len(), update.aspas.len());
-            let rib = Arc::clone(&rib);
-            // Looks at every route, or at every prefix at least: not for a
-            // thread that sessions run on.
-            let found =
-                tokio::task::spawn_blocking(move || newly_invalid(&rib, &update, &subjects)).await;
-            let mut suspects = match found {
-                Ok(suspects) => suspects,
-                Err(e) => {
-                    warn!(err = %e, "Looking for routes that are now invalid failed");
-                    HashMap::new()
-                }
-            };
-            info!(
-                vrps,
-                aspas,
-                now_invalid = suspects.values().map(Vec::len).sum::<usize>(),
-                sessions = sessions.len(),
-                "RPKI data changed; revalidating routes"
             );
-            // Every session, also those with nothing found: each has its
+            let what = format!(
-            // held routes to look at again.
+                "RPKI data changed, to {} VRPs and {} ASPAs; revalidating routes",
-            for (session, _) in sessions {
+                update.vrps.len(),
-                session.revalidate(suspects.remove(&session.rib_peer).unwrap_or_default());
+                update.aspas.len()
-            }
+            );
+            revalidate_all(sessions, &rib, &what, move |rib| {
+                newly_invalid(rib, &update, &subjects)
+            })
+            .await;
         }
     })
 }
 
+/// `what` of each session, by its peer's number in the table, for looking
+/// up as often as there are routes; `None` where no session has that
+/// number.
+pub(crate) fn by_rib_peer<T>(sessions: impl IntoIterator<Item = (PeerIndex, T)>) -> Vec<Option<T>> {
+    let mut by_peer: Vec<Option<T>> = Vec::new();
+    for (peer, what) in sessions {
+        let at = peer.as_usize();
+        if by_peer.len() <= at {
+            by_peer.resize_with(at + 1, || None);
+        }
+        by_peer[at] = Some(what);
+    }
+    by_peer
+}
+
+/// `routes` as what each session is to take out of the table, by peer.
+pub(crate) fn group_by_peer(
+    routes: impl IntoIterator<Item = Route>,
+) -> HashMap<PeerIndex, Vec<Nlri>> {
+    let mut by_peer: HashMap<PeerIndex, Vec<Nlri>> = HashMap::new();
+    for route in routes {
+        by_peer.entry(route.peer).or_default().push(Nlri {
+            prefix: route.prefix,
+            path_id: route.path_id,
+        });
+    }
+    by_peer
+}
+
+/// One round of revalidation, for whichever data changed: `find` looks
+/// through the table for the routes that are to go, by peer, and every one
+/// of `sessions` is told of its own, also those with nothing found, since
+/// each has its held routes to look at again. `what` is for the log.
+pub(crate) async fn revalidate_all<F>(
+    sessions: Vec<SessionHandle>,
+    rib: &Arc<RibTable>,
+    what: &str,
+    find: F,
+) where
+    F: FnOnce(&RibTable) -> HashMap<PeerIndex, Vec<Nlri>> + Send + 'static,
+{
+    let table = Arc::clone(rib);
+    // Looks at every route, or at every prefix at least: not for a thread
+    // that sessions run on.
+    let found = tokio::task::spawn_blocking(move || find(&table)).await;
+    let mut suspects = match found {
+        Ok(suspects) => suspects,
+        Err(e) => {
+            warn!(err = %e, "Looking through the table for routes to take out failed");
+            HashMap::new()
+        }
+    };
+    info!(
+        found = suspects.values().map(Vec::len).sum::<usize>(),
+        sessions = sessions.len(),
+        "{what}"
+    );
+    for session in sessions {
+        session.revalidate(suspects.remove(&session.rib_peer).unwrap_or_default());
+    }
+}
+
 /// The routes in the table that `data` finds invalid, by peer. `subjects`
 /// says, for each peer by its number in the table, which checks its routes
 /// are subject to.
@@ -306,42 +346,54 @@ fn newly_invalid(
         path.iter().any(|asn| changed.aspa_customers.contains(asn))
     };
     let path = RefCell::new(Vec::new());
+    // What each peer's routes are checked against: the data, for the
+    // checks its routes are subject to.
+    let checks: Vec<Option<Checks<'_>>> = subjects
+        .iter()
+        .map(|subject| {
+            subject.map(|subject| Checks {
+                vrps: subject.origin.then_some(&*data.vrps),
+                aspas: subject.path.map(|check| (&*data.aspas, check)),
+                next_hops: None,
+            })
+        })
+        .collect();
 
     let routes = rib.routes_where(usize::MAX, |route| {
-        let Some(Some(subject)) = subjects.get(route.peer.as_usize()) else {
+        let Some(Some(checks)) = checks.get(route.peer.as_usize()) else {
             return false;
         };
-        if subject.origin
+        let path = &mut *path.borrow_mut();
-            && vrps_may_matter
+        // Only the checks whose verdict the change can have altered for
-            && changed.is_none_or(|changed| changed.vrps.covers(&route.prefix))
+        // this route are made.
-        {
+        let checks = Checks {
-            let origin = route.attrs.key().origin();
+            vrps: checks.vrps.filter(|_| {
-            if data.vrps.validate(&route.prefix, origin) == Validity::Invalid {
+                vrps_may_matter && changed.is_none_or(|changed| changed.vrps.covers(&route.prefix))
-                return true;
+            }),
-            }
+            aspas: checks.aspas.filter(|_| {
-        }
+                aspas_may_matter
-        if let Some(check) = subject.path
+                    && changed.is_none_or(|changed| {
-            && aspas_may_matter
+                        route.attrs.as_path(path);
-        {
+                        on_changed_path(changed, path)
-            let path = &mut *path.borrow_mut();
+                    })
-            let has_set = route.attrs.as_path(path);
+            }),
-            return changed.is_none_or(|changed| on_changed_path(changed, path))
+            next_hops: None,
-                && check.verify(&data.aspas, path, has_set) == PathValidity::Invalid;
+        };
-        }
+        checks.rejection(&route.prefix, route.attrs, path).is_some()
-        false
     });
-    let mut by_peer: HashMap<PeerIndex, Vec<Nlri>> = HashMap::new();
+    group_by_peer(routes)
-    for route in routes {
-        by_peer.entry(route.peer).or_default().push(Nlri {
-            prefix: route.prefix,
-            path_id: route.path_id,
-        });
-    }
-    by_peer
 }
 
 #[cfg(test)]
 mod tests {
+    use std::net::Ipv4Addr;
+
+    use bgp_packet::message::{
+        AsPathAttribute, NextHopPathAttribute, OriginPathAttribute, PathAttribute,
+    };
+    use bgp_rib::table::PeerInfo;
+    use bgp_rpki::Vrp;
+
     use super::*;
 
     fn aspas() -> AspaTable {
@@ -463,4 +515,89 @@ mod tests {
             })
         );
     }
+
+    fn prefix(s: &str) -> IpPrefix {
+        IpPrefix::try_from(s).unwrap()
+    }
+
+    /// Where it is known which VRPs came or went, only the routes those
+    /// cover are looked at: the others are as valid or invalid as they
+    /// were, and were dealt with then. Without that knowledge every route
+    /// is looked at; and a peer whose routes are not subject to the check
+    /// has none found.
+    #[test]
+    fn only_routes_a_change_can_have_made_invalid_are_found() {
+        let table = RibTable::new(2);
+        let peer = table
+            .add_peer(PeerInfo {
+                asn: 65001,
+                router_id: Ipv4Addr::new(10, 0, 0, 1),
+                address: Ipv4Addr::new(192, 0, 2, 1).into(),
+                interface: 0,
+                ibgp: false,
+                reflector_client: false,
+            })
+            .unwrap();
+        let announce = |route: &str, origin: u32| {
+            let attrs = table
+                .attrs()
+                .intern(
+                    &[
+                        PathAttribute::Origin(OriginPathAttribute::IGP),
+                        AsPathAttribute::from_asns(vec![65001, origin]),
+                        PathAttribute::NextHop(NextHopPathAttribute(Ipv4Addr::new(192, 0, 2, 1))),
+                    ],
+                    None,
+                )
+                .unwrap();
+            table.announce(peer, &attrs, &[prefix(route)]);
+        };
+        announce("10.0.0.0/24", 65010);
+        announce("10.1.0.0/24", 65011);
+        // Both prefixes are someone else's to originate.
+        let vrp = |covered: &str| Vrp {
+            prefix: prefix(covered),
+            max_length: 24,
+            asn: 65009,
+        };
+        let update = Update {
+            version: 2,
+            vrps: Arc::new(VrpTable::new([vrp("10.0.0.0/24"), vrp("10.1.0.0/24")])),
+            aspas: Arc::new(AspaTable::default()),
+            changed: Some(Changed {
+                vrps: Arc::new(VrpTable::new([vrp("10.1.0.0/24")])),
+                aspa_customers: Arc::default(),
+            }),
+        };
+        let origins = by_rib_peer([(
+            peer,
+            Subject {
+                origin: true,
+                path: None,
+            },
+        )]);
+        let found = newly_invalid(&table, &update, &origins);
+        assert_eq!(
+            found.get(&peer).map(Vec::as_slice),
+            Some(&[Nlri::from(prefix("10.1.0.0/24"))][..])
+        );
+
+        let whole = Update {
+            changed: None,
+            ..update
+        };
+        let found = newly_invalid(&table, &whole, &origins);
+        let routes = found.get(&peer).expect("the peer's routes");
+        assert_eq!(routes.len(), 2);
+        assert!(routes.contains(&Nlri::from(prefix("10.0.0.0/24"))));
+
+        let paths_only = by_rib_peer([(
+            peer,
+            Subject {
+                origin: false,
+                path: None,
+            },
+        )]);
+        assert!(newly_invalid(&table, &whole, &paths_only).is_empty());
+    }
 }

crates/server/src/server.rs

@@ -237,7 +237,7 @@ impl BgpServer {
             };
             match result {
                 Ok((stream, peer_addr)) => {
-                    match admit(&self.config, &context, peer_addr.into(), &connections) {
+                    match screen(&self.config, &context, peer_addr.into(), &connections) {
                         Ok((peer, slot)) => {
                             let context = Arc::clone(&context);
                             tokio::spawn(serve(stream, peer_addr, context, peer, slot));
@@ -461,9 +461,10 @@ fn listen(addr: SocketAddr) -> io::Result<TcpListener> {
     socket.listen(LISTEN_BACKLOG)
 }
 
-/// Decides whether to take a connection from `address`, and if so with what
+/// Screens a connection from `address`: whether to take it, and if so
-/// settings.
+/// with what settings. Not the collision check, which is
-fn admit(
+/// [`crate::collision::CollisionTable::admit`].
+fn screen(
     config: &Config,
     context: &ServerContext,
     address: PeerAddress,

crates/server/src/session.rs

@@ -295,8 +295,6 @@ impl SessionHandle {
         }
     }
 
-    /// Whether `other` is a handle to this same session, and not to another
-    /// session with the same peer.
     /// What the peer goes by in the metrics: its configured name, or its
     /// address.
     pub fn metric_label(&self) -> String {
@@ -305,6 +303,8 @@ impl SessionHandle {
             .unwrap_or_else(|| crate::address::PeerAddress::from(self.peer.addr).to_string())
     }
 
+    /// Whether `other` is a handle to this same session, and not to another
+    /// session with the same peer.
     pub fn is_same_session(&self, other: &SessionHandle) -> bool {
         self.outbound.same_channel(&other.outbound)
     }
@@ -460,10 +460,6 @@ impl SessionRegistry {
         self.shutting_down.load(Ordering::Relaxed)
     }
 
-    /// Closes every session politely: each peer is sent Cease /
-    /// Administrative Shutdown (RFC 4486) so it can drop the session at once
-    /// instead of discovering a dead TCP connection. Waits up to `grace` for
-    /// the sessions to finish; returns how many were still open.
     /// Closes every connection because the daemon is going, as `leave`
     /// says: for good, or to restart (see [`Leave`]). Waits up to `grace`
     /// for the sessions to finish; returns how many were still open.

crates/web/src/api.rs

@@ -9,6 +9,7 @@ use axum::Json;
 use axum::extract::{Query, State};
 use axum::http::StatusCode;
 use axum::response::{IntoResponse, Response};
+use bgp_fib::sockdiag::TcpConnection;
 use bgp_packet::community;
 use bgp_packet::message::{
     AsPathAttribute, ExtendedCommunity, NlriNextHop, OriginPathAttribute, PathAttribute,
@@ -18,7 +19,8 @@ use bgp_primitives::ip_prefix::IpPrefix;
 use bgp_rib::table::{AttrSet, PeerIndex, RibTable, Route, RouteRef, SelectionKey};
 use bgp_rib::{LocalRoute, RibStopped};
 use bgp_rpki::{CacheState, Update};
-use bgp_server::address::{PeerAddress, socket_text};
+use bgp_server::address::{names_peer, socket_text};
+use bgp_server::family::FamilySet;
 use bgp_server::policy::PeerRole;
 use bgp_server::restart::{Agreed, Restarts};
 use bgp_server::rpki::PathCheck;
@@ -148,51 +150,46 @@ impl From<&Agreed> for GracefulRestartView {
 
 impl From<&SessionHandle> for PeerView {
     fn from(s: &SessionHandle) -> Self {
+        let families =
+            |families: &FamilySet| Some(families.iter().map(|f| f.to_string()).collect());
         Self {
-            name: s.name.clone(),
-            peer_addr: socket_text(s.peer.addr),
-            peer_asn: s.peer.asn,
-            state: "Established".into(),
             direction: Some(s.direction.as_str().into()),
             local_addr: Some(socket_text(s.local_addr)),
             established_at_unix: Some(unix_secs(s.established_at)),
             uptime_secs: Some(s.established_at.elapsed().map(|d| d.as_secs()).unwrap_or(0)),
             four_octet_asn: Some(s.four_octet_asn),
             extended_next_hop: Some(s.extended_next_hop),
-            address_families: Some(s.families.iter().map(|f| f.to_string()).collect()),
+            address_families: families(&s.families),
-            add_path_receive: Some(s.add_path_receive.iter().map(|f| f.to_string()).collect()),
+            add_path_receive: families(&s.add_path_receive),
-            add_path_send: Some(s.add_path_send.iter().map(|f| f.to_string()).collect()),
+            add_path_send: families(&s.add_path_send),
             routes: Some(s.counters.routes()),
             rejected_routes: Some(s.counters.rejected()),
             advertised_routes: Some(s.counters.advertised()),
             held_routes: Some(s.counters.held()),
-            expected_routes: None,
             role: s.role.map(PeerRole::as_str),
             role_confirmed: s.role.map(|_| s.role_confirmed),
-            last_error: None,
-            retry_at_unix: None,
             graceful_restart: s.graceful_restart.as_ref().map(GracefulRestartView::from),
             stale_routes: Some(s.counters.stale()).filter(|stale| *stale > 0),
-            stale_until_unix: None,
+            ..Self::known(
+                s.name.clone(),
+                socket_text(s.peer.addr),
+                s.peer.asn,
+                "Established",
+            )
         }
     }
 }
 
 impl PeerView {
-    fn from_configured(peer: &ConfiguredPeer, state: &DialState) -> Self {
+    /// What is known of any peer, with or without a session: who it is
-        let (state_name, last_error, retry_at_unix) = match state {
+    /// and where it stands. Everything else is `None`, for the caller to
-            DialState::Idle => ("Idle", None, None),
+    /// fill in what it knows besides.
-            DialState::Connecting => ("Connecting", None, None),
+    fn known(name: Option<String>, peer_addr: String, peer_asn: u32, state: &str) -> Self {
-            DialState::Backoff { until, last_error } => {
-                ("Backoff", Some(last_error.clone()), Some(unix_secs(*until)))
-            }
-            DialState::Passive { last_error } => ("Passive", last_error.clone(), None),
-        };
         Self {
-            name: Some(peer.name.clone()),
+            name,
-            peer_addr: peer.endpoint.with_port(),
+            peer_addr,
-            peer_asn: peer.remote_asn,
+            peer_asn,
-            state: state_name.into(),
+            state: state.into(),
             direction: None,
             local_addr: None,
             established_at_unix: None,
@@ -206,16 +203,39 @@ impl PeerView {
             rejected_routes: None,
             advertised_routes: None,
             held_routes: None,
-            expected_routes: peer.expected_routes,
+            expected_routes: None,
-            role: peer.role.map(PeerRole::as_str),
+            role: None,
             role_confirmed: None,
-            last_error,
+            last_error: None,
-            retry_at_unix,
+            retry_at_unix: None,
             graceful_restart: None,
             stale_routes: None,
             stale_until_unix: None,
         }
     }
+
+    fn from_configured(peer: &ConfiguredPeer, state: &DialState) -> Self {
+        let (state_name, last_error, retry_at_unix) = match state {
+            DialState::Idle => ("Idle", None, None),
+            DialState::Connecting => ("Connecting", None, None),
+            DialState::Backoff { until, last_error } => {
+                ("Backoff", Some(last_error.clone()), Some(unix_secs(*until)))
+            }
+            DialState::Passive { last_error } => ("Passive", last_error.clone(), None),
+        };
+        Self {
+            expected_routes: peer.expected_routes,
+            role: peer.role.map(PeerRole::as_str),
+            last_error,
+            retry_at_unix,
+            ..Self::known(
+                Some(peer.name.clone()),
+                peer.endpoint.with_port(),
+                peer.remote_asn,
+                state_name,
+            )
+        }
+    }
 }
 
 /// Configured peers first (with their live session when established), then
@@ -541,9 +561,7 @@ impl PeerNames {
     fn matching(&self, wanted: &str) -> Vec<PeerIndex> {
         self.peers
             .iter()
-            .filter(|(_, (peer, name, _))| {
+            .filter(|(_, (peer, name, _))| names_peer(wanted, name.as_deref(), peer.addr.into()))
-                name.as_deref() == Some(wanted) || PeerAddress::from(peer.addr).is_called(wanted)
-            })
             .map(|(index, _)| *index)
             .collect()
     }
@@ -1226,7 +1244,7 @@ pub async fn rejected_routes(
             query
                 .peer
                 .as_deref()
-                .is_none_or(|wanted| record.label() == wanted || address.is_called(wanted))
+                .is_none_or(|wanted| names_peer(wanted, Some(record.label()), *address))
         })
         .map(|(address, record)| RejectionsView {
             peer: record.label().to_string(),
@@ -1256,6 +1274,168 @@ pub async fn rejected_routes(
     Json(peers)
 }
 
+/// `GET /api/tcp?peer=NAME`: what the kernel knows of the TCP connection of
+/// the session with that peer.
+#[derive(Debug, Deserialize)]
+pub struct TcpQuery {
+    /// The peer whose session it is: its configured name, or its address.
+    /// The address with its port, as `peer_addr` of `/api/peers` has it,
+    /// names it too.
+    pub peer: String,
+}
+
+/// A session's TCP connection, as the kernel reports it (Linux only).
+/// Times in milliseconds, sizes in octets, rates in octets per second.
+#[derive(Debug, Serialize)]
+pub struct TcpView {
+    pub state: &'static str,
+    pub congestion: Option<String>,
+    pub rtt_ms: f64,
+    pub rtt_variance_ms: f64,
+    pub min_rtt_ms: f64,
+    pub rto_ms: f64,
+    pub mss: u32,
+    pub pmtu: u32,
+    pub cwnd_segments: u32,
+    pub ssthresh_segments: Option<u32>,
+    /// The peer's receive window: what it lets us send.
+    pub send_window: u32,
+    /// Our receive window: what we let the peer send.
+    pub receive_window: u32,
+    pub receive_space: u32,
+    pub send_window_scale: u8,
+    pub receive_window_scale: u8,
+    /// Octets written by us and not yet acknowledged, and of those the
+    /// octets not even sent yet.
+    pub send_queue: u32,
+    pub not_sent: u32,
+    pub recv_queue: u32,
+    pub unacked_segments: u32,
+    pub retrans_segments: u32,
+    pub lost_segments: u32,
+    pub total_retrans: u32,
+    pub bytes_retrans: u64,
+    pub pacing_rate: u64,
+    pub delivery_rate: u64,
+    pub bytes_sent: u64,
+    pub bytes_acked: u64,
+    pub bytes_received: u64,
+    pub segments_out: u32,
+    pub segments_in: u32,
+    /// How long the connection had something to send, and of that how
+    /// long it was held up by the peer's window and by our send buffer.
+    pub busy_ms: f64,
+    pub limited_by_receive_window_ms: f64,
+    pub limited_by_send_buffer_ms: f64,
+    /// The same two, as shares of the busy time, 0 to 1.
+    pub limited_by_receive_window_share: f64,
+    pub limited_by_send_buffer_share: f64,
+    pub last_data_sent_ms: u32,
+    pub last_data_received_ms: u32,
+    pub last_ack_received_ms: u32,
+}
+
+impl From<&TcpConnection> for TcpView {
+    fn from(c: &TcpConnection) -> Self {
+        // The kernel's times are in microseconds.
+        let ms = |us: u64| us as f64 / 1000.0;
+        // A connection that was never busy was never held up: a share of
+        // nothing is nothing, not a division by zero.
+        let share = |part: u64| {
+            if c.busy == 0 {
+                0.0
+            } else {
+                part as f64 / c.busy as f64
+            }
+        };
+        Self {
+            state: c.state_name(),
+            congestion: c.congestion.clone(),
+            rtt_ms: ms(c.rtt.into()),
+            rtt_variance_ms: ms(c.rtt_variance.into()),
+            min_rtt_ms: ms(c.min_rtt.into()),
+            rto_ms: ms(c.rto.into()),
+            mss: c.mss,
+            pmtu: c.pmtu,
+            cwnd_segments: c.cwnd,
+            ssthresh_segments: c.ssthresh,
+            send_window: c.send_window,
+            receive_window: c.receive_window,
+            receive_space: c.receive_space,
+            send_window_scale: c.send_window_scale,
+            receive_window_scale: c.receive_window_scale,
+            send_queue: c.send_queue,
+            not_sent: c.not_sent,
+            recv_queue: c.recv_queue,
+            unacked_segments: c.unacked,
+            retrans_segments: c.retrans,
+            lost_segments: c.lost,
+            total_retrans: c.total_retrans,
+            bytes_retrans: c.bytes_retrans,
+            pacing_rate: c.pacing_rate,
+            delivery_rate: c.delivery_rate,
+            bytes_sent: c.bytes_sent,
+            bytes_acked: c.bytes_acked,
+            bytes_received: c.bytes_received,
+            segments_out: c.segments_out,
+            segments_in: c.segments_in,
+            busy_ms: ms(c.busy),
+            limited_by_receive_window_ms: ms(c.limited_by_receive_window),
+            limited_by_send_buffer_ms: ms(c.limited_by_send_buffer),
+            limited_by_receive_window_share: share(c.limited_by_receive_window),
+            limited_by_send_buffer_share: share(c.limited_by_send_buffer),
+            last_data_sent_ms: c.last_data_sent_ms,
+            last_data_received_ms: c.last_data_received_ms,
+            last_ack_received_ms: c.last_ack_received_ms,
+        }
+    }
+}
+
+/// The answer to `/api/tcp`.
+#[derive(Debug, Serialize)]
+pub struct TcpAnswer {
+    /// Whether the kernel could be asked and knew the connection.
+    pub available: bool,
+    /// Why not, if not.
+    pub reason: Option<String>,
+    pub connection: Option<TcpView>,
+}
+
+pub async fn tcp(
+    State(state): State<AppState>,
+    Query(query): Query<TcpQuery>,
+) -> Result<Json<TcpAnswer>, QueryError> {
+    let session = state
+        .registry
+        .list()
+        .into_iter()
+        .find(|s| {
+            names_peer(&query.peer, s.name.as_deref(), s.peer.addr.into())
+                || socket_text(s.peer.addr) == query.peer
+        })
+        .ok_or_else(|| QueryError::Bad(format!("no session with {}", query.peer)))?;
+    let unavailable = |reason: String| TcpAnswer {
+        available: false,
+        reason: Some(reason),
+        connection: None,
+    };
+    let answer =
+        match bgp_fib::sockdiag::tcp_connection(session.local_addr, session.peer.addr).await {
+            Ok(Some(c)) => TcpAnswer {
+                available: true,
+                reason: None,
+                connection: Some(TcpView::from(&c)),
+            },
+            Ok(None) => unavailable(if cfg!(target_os = "linux") {
+                "the kernel has no such connection".to_string()
+            } else {
+                "only on Linux".to_string()
+            }),
+            Err(e) => unavailable(format!("the kernel could not be asked: {e}")),
+        };
+    Ok(Json(answer))
+}
+
 #[cfg(test)]
 mod tests {
     use std::net::Ipv4Addr;
@@ -1580,156 +1760,34 @@ mod tests {
         .map(|(name, value)| (name.to_string(), value.to_string()));
         assert_eq!(other, expected);
     }
-}
 
-/// `GET /api/tcp?peer=ADDRESS:PORT`: what the kernel knows of the TCP
+    /// The kernel's microseconds are shown as milliseconds, and the time
-/// connection of the session with that peer, as `/api/peers` names it.
+    /// held up as a share of the busy time; a connection that was never
-#[derive(Debug, Deserialize)]
+    /// busy has a share of zero, not of NaN.
-pub struct TcpQuery {
+    #[test]
-    pub peer: String,
+    fn a_tcp_view_has_milliseconds_and_shares_of_the_busy_time() {
-}
+        let idle = TcpConnection {
-
+            rtt: 1500,
-/// A session's TCP connection, as the kernel reports it (Linux only).
+            busy: 0,
-/// Times in milliseconds, sizes in octets, rates in octets per second.
+            limited_by_receive_window: 0,
-#[derive(Debug, Serialize)]
+            ..TcpConnection::default()
-pub struct TcpView {
-    pub state: &'static str,
-    pub congestion: Option<String>,
-    pub rtt_ms: f64,
-    pub rtt_variance_ms: f64,
-    pub min_rtt_ms: f64,
-    pub rto_ms: f64,
-    pub mss: u32,
-    pub pmtu: u32,
-    pub cwnd_segments: u32,
-    pub ssthresh_segments: Option<u32>,
-    /// The peer's receive window: what it lets us send.
-    pub send_window: u32,
-    /// Our receive window: what we let the peer send.
-    pub receive_window: u32,
-    pub receive_space: u32,
-    pub send_window_scale: u8,
-    pub receive_window_scale: u8,
-    /// Octets written by us and not yet acknowledged, and of those the
-    /// octets not even sent yet.
-    pub send_queue: u32,
-    pub not_sent: u32,
-    pub recv_queue: u32,
-    pub unacked_segments: u32,
-    pub retrans_segments: u32,
-    pub lost_segments: u32,
-    pub total_retrans: u32,
-    pub bytes_retrans: u64,
-    pub pacing_rate: u64,
-    pub delivery_rate: u64,
-    pub bytes_sent: u64,
-    pub bytes_acked: u64,
-    pub bytes_received: u64,
-    pub segments_out: u32,
-    pub segments_in: u32,
-    /// How long the connection had something to send, and of that how
-    /// long it was held up by the peer's window and by our send buffer.
-    pub busy_ms: f64,
-    pub limited_by_receive_window_ms: f64,
-    pub limited_by_send_buffer_ms: f64,
-    /// The same two, as shares of the busy time, 0 to 1.
-    pub limited_by_receive_window_share: f64,
-    pub limited_by_send_buffer_share: f64,
-    pub last_data_sent_ms: u32,
-    pub last_data_received_ms: u32,
-    pub last_ack_received_ms: u32,
-}
-
-/// The answer to `/api/tcp`.
-#[derive(Debug, Serialize)]
-pub struct TcpAnswer {
-    /// Whether the kernel could be asked and knew the connection.
-    pub available: bool,
-    /// Why not, if not.
-    pub reason: Option<String>,
-    pub connection: Option<TcpView>,
-}
-
-pub async fn tcp(
-    State(state): State<AppState>,
-    Query(query): Query<TcpQuery>,
-) -> Result<Json<TcpAnswer>, QueryError> {
-    let session = state
-        .registry
-        .list()
-        .into_iter()
-        .find(|s| socket_text(s.peer.addr) == query.peer)
-        .ok_or_else(|| QueryError::Bad(format!("no session with {}", query.peer)))?;
-    let answer =
-        match bgp_fib::sockdiag::tcp_connection(session.local_addr, session.peer.addr).await {
-            Ok(Some(c)) => {
-                let ms = |us: u64| us as f64 / 1000.0;
-                let share = |part: u64| {
-                    if c.busy == 0 {
-                        0.0
-                    } else {
-                        part as f64 / c.busy as f64
-                    }
-                };
-                TcpAnswer {
-                    available: true,
-                    reason: None,
-                    connection: Some(TcpView {
-                        state: c.state_name(),
-                        congestion: c.congestion.clone(),
-                        rtt_ms: ms(c.rtt.into()),
-                        rtt_variance_ms: ms(c.rtt_variance.into()),
-                        min_rtt_ms: ms(c.min_rtt.into()),
-                        rto_ms: ms(c.rto.into()),
-                        mss: c.mss,
-                        pmtu: c.pmtu,
-                        cwnd_segments: c.cwnd,
-                        ssthresh_segments: c.ssthresh,
-                        send_window: c.send_window,
-                        receive_window: c.receive_window,
-                        receive_space: c.receive_space,
-                        send_window_scale: c.send_window_scale,
-                        receive_window_scale: c.receive_window_scale,
-                        send_queue: c.send_queue,
-                        not_sent: c.not_sent,
-                        recv_queue: c.recv_queue,
-                        unacked_segments: c.unacked,
-                        retrans_segments: c.retrans,
-                        lost_segments: c.lost,
-                        total_retrans: c.total_retrans,
-                        bytes_retrans: c.bytes_retrans,
-                        pacing_rate: c.pacing_rate,
-                        delivery_rate: c.delivery_rate,
-                        bytes_sent: c.bytes_sent,
-                        bytes_acked: c.bytes_acked,
-                        bytes_received: c.bytes_received,
-                        segments_out: c.segments_out,
-                        segments_in: c.segments_in,
-                        busy_ms: ms(c.busy),
-                        limited_by_receive_window_ms: ms(c.limited_by_receive_window),
-                        limited_by_send_buffer_ms: ms(c.limited_by_send_buffer),
-                        limited_by_receive_window_share: share(c.limited_by_receive_window),
-                        limited_by_send_buffer_share: share(c.limited_by_send_buffer),
-                        last_data_sent_ms: c.last_data_sent_ms,
-                        last_data_received_ms: c.last_data_received_ms,
-                        last_ack_received_ms: c.last_ack_received_ms,
-                    }),
-                }
-            }
-            Ok(None) => TcpAnswer {
-                available: false,
-                reason: Some(if cfg!(target_os = "linux") {
-                    "the kernel has no such connection".to_string()
-                } else {
-                    "only on Linux".to_string()
-                }),
-                connection: None,
-            },
-            Err(e) => TcpAnswer {
-                available: false,
-                reason: Some(format!("the kernel could not be asked: {e}")),
-                connection: None,
-            },
         };
-    Ok(Json(answer))
+        let view = TcpView::from(&idle);
+        assert_eq!(view.rtt_ms, 1.5);
+        assert_eq!(view.busy_ms, 0.0);
+        assert_eq!(view.limited_by_receive_window_share, 0.0);
+        assert_eq!(view.limited_by_send_buffer_share, 0.0);
+
+        let busy = TcpConnection {
+            busy: 4_000_000,
+            limited_by_receive_window: 1_000_000,
+            limited_by_send_buffer: 3_000_000,
+            ..TcpConnection::default()
+        };
+        let view = TcpView::from(&busy);
+        assert_eq!(view.busy_ms, 4000.0);
+        assert_eq!(view.limited_by_receive_window_ms, 1000.0);
+        assert_eq!(view.limited_by_receive_window_share, 0.25);
+        assert_eq!(view.limited_by_send_buffer_share, 0.75);
+    }
 }

crates/web/src/lib.rs

@@ -28,11 +28,6 @@ use serde::Serialize;
 use tokio::net::TcpListener;
 use tracing::info;
 
-pub use api::{
-    AttrSummary, ConfigView, KernelView, LocalRouteView, PeerView, ReceivedRouteView,
-    RejectedRouteView, RejectionsView, RoutesPage, RpkiView, StatusView,
-};
-
 /// Static facts about the router shown in status output.
 #[derive(Debug, Clone)]
 pub struct RouterInfo {
@@ -98,8 +93,8 @@ impl RouterInfo {
     }
 }
 
-/// Best-effort host name without pulling in a dependency: ask the `hostname`
+/// The machine's host name as the kernel has it, for display; a machine
-/// utility, which exists on every platform we run on.
+/// without one, or with one that is not text, is "unknown host".
 fn hostname() -> String {
     nix::unistd::gethostname()
         .ok()

crates/web/src/metrics.rs

@@ -63,70 +63,9 @@ async fn sample_tcp(state: &AppState) {
         let label = session.metric_label();
         let connection =
             bgp_fib::sockdiag::tcp_connection(session.local_addr, session.peer.addr).await;
-        let Ok(Some(c)) = connection else {
+        match connection {
-            tcp.remove(&label);
+            Ok(Some(c)) => tcp.set(&label, &c),
-            continue;
+            _ => tcp.remove(&label),
-        };
+        }
-        let seconds = |us: u64| us as f64 / 1e6;
-        let int = |n: u64| i64::try_from(n).unwrap_or(i64::MAX);
-        let label = [label.as_str()];
-        tcp.rtt_seconds
-            .with_label_values(&label)
-            .set(seconds(c.rtt.into()));
-        tcp.min_rtt_seconds
-            .with_label_values(&label)
-            .set(seconds(c.min_rtt.into()));
-        tcp.rtt_variance_seconds
-            .with_label_values(&label)
-            .set(seconds(c.rtt_variance.into()));
-        tcp.cwnd_segments
-            .with_label_values(&label)
-            .set(c.cwnd.into());
-        tcp.mss_bytes.with_label_values(&label).set(c.mss.into());
-        tcp.send_window_bytes
-            .with_label_values(&label)
-            .set(c.send_window.into());
-        tcp.receive_window_bytes
-            .with_label_values(&label)
-            .set(c.receive_window.into());
-        tcp.send_queue_bytes
-            .with_label_values(&label)
-            .set(c.send_queue.into());
-        tcp.not_sent_bytes
-            .with_label_values(&label)
-            .set(c.not_sent.into());
-        tcp.receive_queue_bytes
-            .with_label_values(&label)
-            .set(c.recv_queue.into());
-        tcp.unacked_segments
-            .with_label_values(&label)
-            .set(c.unacked.into());
-        tcp.retransmitted_segments
-            .with_label_values(&label)
-            .set(c.total_retrans.into());
-        tcp.lost_segments
-            .with_label_values(&label)
-            .set(c.lost.into());
-        tcp.busy_seconds
-            .with_label_values(&label)
-            .set(seconds(c.busy));
-        tcp.receive_window_limited_seconds
-            .with_label_values(&label)
-            .set(seconds(c.limited_by_receive_window));
-        tcp.send_buffer_limited_seconds
-            .with_label_values(&label)
-            .set(seconds(c.limited_by_send_buffer));
-        tcp.sent_bytes
-            .with_label_values(&label)
-            .set(int(c.bytes_sent));
-        tcp.received_bytes
-            .with_label_values(&label)
-            .set(int(c.bytes_received));
-        tcp.delivery_rate_bytes_per_second
-            .with_label_values(&label)
-            .set(int(c.delivery_rate));
-        tcp.pacing_rate_bytes_per_second
-            .with_label_values(&label)
-            .set(int(c.pacing_rate));
     }
 }

crates/web/static/statusz.html

@@ -68,12 +68,15 @@ table.data th { background: var(--head); font-weight: bold; position: sticky; to
 table.data > tbody > tr:hover > td { background: var(--hover); }
 table.data td.num, table.data th.num { text-align: right; font-variant-numeric: tabular-nums; white-space: nowrap; }
 table.data tr.group > td { background: var(--band); font-weight: bold; padding: 3px 6px; }
-table.data tr.tcp-row > td { background: var(--band); padding: 6px 10px 8px 28px; }
+/* A row of detail under a row of the table: a session's TCP connection, a
+   route in full. The route's is indented past the rank column. */
+table.data tr.detail > td { background: var(--band); padding: 6px 10px 8px 28px; }
+table.data tr.route-detail > td { padding-left: 40px; }
 table.data td.attrs { cursor: pointer; }
 table.data td.attrs:hover { background: var(--hover); }
-table.data tr.route-detail > td { background: var(--band); padding: 6px 10px 8px 40px; }
+/* Names and values, two columns, as the detail rows show them. */
-.tcp-grid { display: grid; grid-template-columns: max-content 1fr; gap: 1px 16px; font-size: 12px; }
+.kv-grid { display: grid; grid-template-columns: max-content 1fr; gap: 1px 16px; font-size: 12px; }
-.tcp-grid > div:nth-child(odd) { color: var(--fg-soft); white-space: nowrap; }
+.kv-grid > div:nth-child(odd) { color: var(--fg-soft); white-space: nowrap; }
 a.tcp-link { color: var(--dim); font-size: 11.5px; text-decoration: none; border: 1px solid var(--line); border-radius: 3px; padding: 0 4px; margin-left: 4px; }
 a.tcp-link:hover, a.tcp-link.open { color: var(--link); border-color: var(--link); }
 table.data tr.best > td { background: var(--best-bg); }
@@ -276,6 +279,17 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     if (state.status) $('s-uptime').textContent = fmtDur(t - state.status.started_at_unix);
   }
   const fmtWhen = unix => new Date(unix * 1000).toLocaleString();
+  const fmtOctets = n => n >= 1048576 ? `${(n / 1048576).toFixed(1)} MiB` : n >= 1024 ? `${(n / 1024).toFixed(1)} KiB` : `${n} B`;
+  const fmtBits = n => { const b = n * 8; return b >= 1e9 ? `${(b / 1e9).toFixed(2)} Gbit/s` : b >= 1e6 ? `${(b / 1e6).toFixed(1)} Mbit/s` : `${(b / 1e3).toFixed(0)} kbit/s`; };
+  const fmtMs = n => n >= 1000 ? `${(n / 1000).toFixed(1)} s` : `${n.toFixed(n < 10 ? 2 : 0)} ms`;
+  const ago = ms => ms >= 60000 ? `${Math.round(ms / 60000)}m` : ms >= 1000 ? `${(ms / 1000).toFixed(1)}s` : `${ms}ms`;
+  // A cell across every column of the table that `body` is the body of,
+  // and a row of one: for a heading, a note or a detail row. The width is
+  // the table's header's, so a column added there is covered here.
+  const fullCell = (body, html, cls = '') => `<td colspan="${body.closest('table').querySelectorAll('thead th').length}"${cls ? ` class="${cls}"` : ''}>${html}</td>`;
+  const fullRow = (body, html, rowClass = '', cellClass = '') => `<tr${rowClass ? ` class="${rowClass}"` : ''}>${fullCell(body, html, cellClass)}</tr>`;
+  // Names and values in two columns, for a detail row.
+  const kvGrid = rows => `<div class="kv-grid mono">${rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join('')}</div>`;
 
   // What a community of ours asks for where control communities are on
   // (control_communities), or ''. They are acted on where a route leaves
@@ -498,8 +512,10 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     'route server': ['RS', 'route server of an Internet exchange'],
     'route server client': ['RSC', 'client of a route server we run'],
   };
+  // A router of our own AS, by its AS number.
+  const isIbgp = p => p.peer_asn === (state.status || {}).local_asn;
   function roleGlyph(p) {
-    if (p.peer_asn === (state.status || {}).local_asn) return glyph('iBGP', 'a router of our own AS');
+    if (isIbgp(p)) return glyph('iBGP', 'a router of our own AS');
     const r = ROLE_GLYPH[p.role];
     if (!r) return glyph('?', 'no role configured', 'amber');
     const confirmed = p.role_confirmed ? '; the peer says so too (RFC 9234)' : p.state === 'Established' ? '; by our configuration, the peer has not said what it is' : '';
@@ -531,7 +547,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
   }
   const ROLE_ORDER = ['iBGP', 'provider', 'route server', 'peer', 'customer', 'route server client', 'none'];
   function peerGroup(p) {
-    if (p.peer_asn === (state.status || {}).local_asn) return 'iBGP';
+    if (isIbgp(p)) return 'iBGP';
     return p.role || 'none';
   }
   const GROUP_TITLE = { iBGP: 'Our own routers', provider: 'Providers', 'route server': 'Route servers', peer: 'Peers', customer: 'Customers', 'route server client': 'Route server clients', none: 'Without a role' };
@@ -542,7 +558,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     const down = peers.filter(p => peerRank(p) === 0).length;
     const waiting = peers.filter(p => p.state === 'Passive').length;
     $('peers-count').textContent = `${up} of ${peers.length} established` + (down ? `, ${down} down` : '') + (waiting ? `, ${waiting} waiting` : '');
-    if (!peers.length) { setHTML(b, '<tr><td colspan="10" class="dim">No peers configured and none connected.</td></tr>'); return; }
+    if (!peers.length) { setHTML(b, fullRow(b, 'No peers configured and none connected.', '', 'dim')); return; }
     const problemsOnly = $('peers-problems').checked;
     // The IPv4 and IPv6 sessions of one neighbour, named <stem>-v4 and
     // <stem>-v6, are one neighbour to the reader: listed together, IPv6
@@ -585,16 +601,16 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     // they are to us.
     const problems = peers.filter(p => peerRank(p) <= 1).sort(byName);
     if (problems.length) {
-      html += `<tr class="group"><td colspan="10"><span class="red">Needing attention</span> (${problems.length})</td></tr>` + rows(problems);
+      html += fullRow(b, `<span class="red">Needing attention</span> (${problems.length})`, 'group') + rows(problems);
     }
     if (!problemsOnly) {
       for (const group of ROLE_ORDER) {
         const members = peers.filter(p => peerRank(p) >= 2 && peerGroup(p) === group).sort(byName);
         if (!members.length) continue;
-        html += `<tr class="group"><td colspan="10">${GROUP_TITLE[group]} (${members.length})</td></tr>` + rows(members);
+        html += fullRow(b, `${GROUP_TITLE[group]} (${members.length})`, 'group') + rows(members);
       }
     } else if (!problems.length) {
-      html += '<tr><td colspan="10" class="dim">Nothing needs attention.</td></tr>';
+      html += fullRow(b, 'Nothing needs attention.', '', 'dim');
     }
     setHTML(b, html);
     renderTcpRows();
@@ -610,10 +626,6 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     const open = state.tcpOpen.has(p.peer_addr);
     return `<a href="#" class="tcp-link${open ? ' open' : ''}" data-peer="${esc(p.peer_addr)}" title="TCP_INFO for this session's socket">tcp</a>`;
   }
-  const fmtOctets = n => n >= 1048576 ? `${(n / 1048576).toFixed(1)} MiB` : n >= 1024 ? `${(n / 1024).toFixed(1)} KiB` : `${n} B`;
-  const fmtBits = n => { const b = n * 8; return b >= 1e9 ? `${(b / 1e9).toFixed(2)} Gbit/s` : b >= 1e6 ? `${(b / 1e6).toFixed(1)} Mbit/s` : `${(b / 1e3).toFixed(0)} kbit/s`; };
-  const fmtMs = n => n >= 1000 ? `${(n / 1000).toFixed(1)} s` : `${n.toFixed(n < 10 ? 2 : 0)} ms`;
-  const ago = ms => ms >= 60000 ? `${Math.round(ms / 60000)}m` : ms >= 1000 ? `${(ms / 1000).toFixed(1)}s` : `${ms}ms`;
   function tcpDetail(a) {
     if (!a.available) return `<span class="dim">${esc(a.reason || 'n/a')}</span>`;
     const c = a.connection;
@@ -634,24 +646,25 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
       ['bytes', `${kv('sent', fmtOctets(c.bytes_sent))} ${kv('acked', fmtOctets(c.bytes_acked))} ${kv('received', fmtOctets(c.bytes_received))} ${kv('segs', `${fmtCount(c.segments_out)}↑ ${fmtCount(c.segments_in)}↓`)}`],
       ['last', `${kv('snd', ago(c.last_data_sent_ms))} ${kv('rcv', ago(c.last_data_received_ms))} ${kv('ack', ago(c.last_ack_received_ms))}`],
     ];
-    return `<div class="tcp-grid mono">${rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join('')}</div>`;
+    return kvGrid(rows);
   }
   async function renderTcpRows() {
+    const b = $('peers-body');
     for (const addr of state.tcpOpen) {
-      const tr = document.querySelector(`#peers-body tr[data-peer-addr="${CSS.escape(addr)}"]`);
+      const tr = b.querySelector(`tr[data-peer-addr="${CSS.escape(addr)}"]`);
       if (!tr) continue;
       let row = tr.nextElementSibling;
       if (!row || !row.classList.contains('tcp-row')) {
         row = document.createElement('tr');
-        row.className = 'tcp-row';
+        row.className = 'detail tcp-row';
-        row.innerHTML = '<td colspan="10"><span class="dim">asking the kernel…</span></td>';
+        row.innerHTML = fullCell(b, '<span class="dim">asking the kernel…</span>');
         tr.after(row);
       }
       try {
         const answer = await getJSON(`/api/tcp?peer=${encodeURIComponent(addr)}`);
-        row.innerHTML = `<td colspan="10">${tcpDetail(answer)}</td>`;
+        row.innerHTML = fullCell(b, tcpDetail(answer));
       } catch (e) {
-        row.innerHTML = `<td colspan="10"><span class="red">${esc(String(e))}</span></td>`;
+        row.innerHTML = fullCell(b, `<span class="red">${esc(String(e))}</span>`);
       }
     }
   }
@@ -790,7 +803,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
 
   function renderLocal() {
     const b = $('local-body'); $('local-count').textContent = state.local.length;
-    if (!state.local.length) { setHTML(b, '<tr><td colspan="7" class="dim">This router originates no routes.</td></tr>'); return; }
+    if (!state.local.length) { setHTML(b, fullRow(b, 'This router originates no routes.', '', 'dim')); return; }
     const local = state.status ? state.status.local_asn : '?';
     setHTML(b, state.local.map(r => `<tr>
       <td class="mono">${esc(r.prefix)}</td><td>${esc(r.origin)}</td>
@@ -826,7 +839,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
       : `${shown}${st ? `, for ${fmtCount(st.received_prefixes)} prefixes` : ''}`;
     renderPager();
     if (!state.received.length) {
-      setHTML(b, `<tr><td colspan="12" class="dim">${looking ? 'No route matches.' : 'No routes received from peers.'}</td></tr>`); return;
+      setHTML(b, fullRow(b, looking ? 'No route matches.' : 'No routes received from peers.', '', 'dim')); return;
     }
     setHTML(b, state.received.map(r => { const s = r.summary; return `<tr class="${r.best ? 'best' : ''}" data-route="${esc(routeKey(r))}">
       <td class="nowrap">${rankGlyph(r)}${r.stale ? glyph('stale', 'Kept from a session of the peer’s that ended (graceful restart), and not sent again since', 'amber') : ''}</td>
@@ -863,18 +876,19 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     if (r.path_id) rows.push(['PATH_ID', `${r.path_id}`]);
     rows.push(['selection', `${r.best ? 'best' : `rank ${r.rank}`}${r.stale ? ', stale' : ''}`]);
     rows.push(['rpki', r.rpki ? esc(r.rpki) : DASH], ['aspa', r.aspa ? esc(r.aspa) : DASH]);
-    return `<div class="tcp-grid mono">${rows.map(([k, v]) => `<div>${k}</div><div>${v}</div>`).join('')}</div>`;
+    return kvGrid(rows);
   }
   function renderRouteDetails() {
+    const b = $('received-body');
     for (const key of state.routeOpen) {
       const r = state.received.find(r => routeKey(r) === key);
-      const tr = document.querySelector(`#received-body tr[data-route="${CSS.escape(key)}"]`);
+      const tr = b.querySelector(`tr[data-route="${CSS.escape(key)}"]`);
       if (!r || !tr) continue;
       const next = tr.nextElementSibling;
       if (next && next.classList.contains('route-detail')) continue;
       const row = document.createElement('tr');
-      row.className = 'route-detail';
+      row.className = 'detail route-detail';
-      row.innerHTML = `<td colspan="12">${routeDetail(r)}</td>`;
+      row.innerHTML = fullCell(b, routeDetail(r));
       tr.after(row);
     }
   }
@@ -961,7 +975,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
     const total = state.rejected.reduce((n, p) => n + p.total, 0);
     const announcements = n => `${fmtCount(n)} announcement${n === 1 ? '' : 's'}`;
     $('rejected-count').textContent = total ? `${announcements(total)} from ${state.rejected.length} peer${state.rejected.length === 1 ? '' : 's'}${chosen ? `; showing ${chosen}` : ''}` : '0';
-    if (!peers.length) { setHTML(b, `<tr><td colspan="11" class="dim">${chosen ? `Nothing ${esc(chosen)} announced has been rejected.` : 'Nothing a peer announced has been rejected.'}</td></tr>`); return; }
+    if (!peers.length) { setHTML(b, fullRow(b, chosen ? `Nothing ${esc(chosen)} announced has been rejected.` : 'Nothing a peer announced has been rejected.', '', 'dim')); return; }
     setHTML(b, peers.map(p => {
       // Alike: the same prefix, for the same reason, by the same path.
       const alike = new Map();
@@ -974,7 +988,7 @@ from <select id="lookup-peer"><option value="">any peer</option></select>
       const lines = [...alike.values()].sort((a, c) => c.at_unix - a.at_unix).slice(0, REJECTED_SHOWN);
       const more = alike.size > REJECTED_SHOWN
         ? ` <span class="dim">${REJECTED_SHOWN} of ${alike.size} shown; <a href="/api/routes/rejected?peer=${encodeURIComponent(p.peer)}">the latest ${p.recent.length} as json</a>.</span>` : '';
-      return `<tr class="group"><td colspan="11"><a href="#received" class="peer-link" data-peer="${esc(p.peer)}">${esc(p.peer)}</a> <span class="dim" style="font-weight: normal">${esc(p.peer_addr)}</span>: <span style="font-weight: normal">${announcements(p.total)} rejected</span> ${reasonBar(p.by_reason, p.total)}${more}</td></tr>` +
+      return fullRow(b, `<a href="#received" class="peer-link" data-peer="${esc(p.peer)}">${esc(p.peer)}</a> <span class="dim" style="font-weight: normal">${esc(p.peer_addr)}</span>: <span style="font-weight: normal">${announcements(p.total)} rejected</span> ${reasonBar(p.by_reason, p.total)}${more}`, 'group') +
         lines.map(r => { const s = r.summary; return `<tr><td></td>
           <td class="mono">${esc(r.prefix)}${r.path_id ? ` <span class="dim">path ${r.path_id}</span>` : ''}</td>
           <td class="mono path">${clamp(asPath(r.as_path))}</td><td>${esc(r.reason)}</td>

docs/content/docs/reference/http-api.md

@@ -233,9 +233,11 @@ routes. To return the records for one neighbor, add `?peer=NAME`.
 
 ## TCP Connection
 
-`/api/tcp?peer=ADDRESS` returns what the kernel reports about the TCP
+`/api/tcp?peer=PEER` returns what the kernel reports about the TCP
-connection of the session with the neighbor at `ADDRESS`, given as
+connection of the session with the neighbor `PEER`, named as for
-`peer_addr` in `/api/peers` shows it: the round-trip time, the congestion
+`/api/routes/received`: its configured name, or its address, with or
+without the port as `peer_addr` in `/api/peers` shows it. The report
+covers the round-trip time, the congestion
 and receive windows, the queues, the retransmissions, and how long the
 connection has been held up by the neighbor's receive window or by the
 local send buffer. Linux only; on other systems, and for a session the