rib: size the routing table for the routes the peers are expected to send

The routing table's shards started empty and grew as routes came in,
each growth moving every row of the shard into a table twice the size.
In a profile of the daemon taking in a full table, the growth showed as
up to a seventh of the session's time, and the daemon's resident memory
had the old and the new table of each shard side by side for a moment.

This change adds `RibTable::expect_prefixes`, which reserves room for
that many prefixes spread over the shards, and has the server call it at
startup with the largest `expected_routes` among the configured peers:
the prefixes of several full tables are mostly the same ones. Without
`expected_routes` nothing changes. The `full_table` benchmark does the
same unless `BENCH_PRESIZE=0`.

Measured, the saving in time is within the run-to-run noise: in the
Linux VM one peer's table is in the routing table in 0.81 s instead of
0.89 s, and the kernel is in step at about the same time as before;
three pairs of runs of the benchmark on a Mac average 0.33 s against
0.35 s. The profile's attribution to the growth was mostly sampling skid.
The memory saving is real: 442 MiB resident with one peer's table in
instead of 483 MiB, and 626 MiB instead of 632 MiB with four.

Also: the profiling script printed nothing for the kernel's change
counts because it asked `/metrics` for them; the endpoint is `/metricz`.
With that fixed, a four-peer load shows 1.25 million installs for 1.23
million prefixes: the journal already gives the kernel task only the
latest change of a prefix when it lags, so no further coalescing of
installs is called for.

Documentation: `expected_routes` in the configuration reference, the
neighbor guide and the proto comment say that the table is sized by it;
the testing guide's VM results are brought up to date.

Testing: a table test checks that the room is there and is kept; the
workspace tests pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

crates/bench/benches/full_table.rs

@@ -3,7 +3,8 @@
 //!
 //! Run with `cargo bench -p bgp-bench --bench full_table`. Set
 //! `BENCH_PEERS` (default 4) and `BENCH_SCALE_DOWN` (default 1; 10 makes the
-//! table ten times smaller) to change the size. The table is synthetic (see
+//! table ten times smaller) to change the size. `BENCH_PRESIZE=0` leaves
+//! the table to grow as it fills, as it does without `expected_routes`. The table is synthetic (see
 //! the crate documentation); `real_table` runs on a real one.
 //!
 //! This is a stopwatch around whole operations at full size, not a
@@ -148,6 +149,9 @@ fn main() {
 fn into_table(wire: &[Vec<BytesMut>], routes: usize, threads: usize) {
     let region = Region::begin();
     let table = RibTable::new(threads * 16);
+    if env("BENCH_PRESIZE", 1) == 1 {
+        table.expect_prefixes(routes / wire.len());
+    }
     let peers: Vec<PeerIndex> = (0..wire.len())
         .map(|peer| {
             table

crates/rib/src/table/mod.rs

@@ -670,6 +670,23 @@ impl RibTable {
         self.len() == 0
     }
 
+    /// Makes room for `prefixes` prefixes, spread over the shards, so that
+    /// a table of that size goes in without the shards growing as it
+    /// does. Growing a shard moves every row it has into a bigger table;
+    /// over a full table that was a seventh of what a session spent
+    /// taking it in. Room that is already there is kept.
+    pub fn expect_prefixes(&self, prefixes: usize) {
+        let each = prefixes.div_ceil(self.shards.len());
+        for shard in &self.shards {
+            let mut shard = write(shard);
+            let hasher = shard.hasher.clone();
+            let additional = each.saturating_sub(shard.rows.len());
+            shard
+                .rows
+                .reserve(additional, |row| hasher.hash_one(row.prefix));
+        }
+    }
+
     /// How many prefixes have at least one route.
     pub fn prefix_count(&self) -> usize {
         self.shards.iter().map(|shard| read(shard).rows.len()).sum()

crates/rib/src/table/tests.rs

@@ -1002,3 +1002,31 @@ fn routes_can_be_picked_out_of_the_table() {
     );
     assert!(table.routes_where(0, |_| true).is_empty());
 }
+
+#[test]
+fn a_table_told_what_to_expect_has_the_room_before_the_routes_come() {
+    let table = RibTable::new(8);
+    table.expect_prefixes(100_000);
+    let room: usize = table
+        .shards
+        .iter()
+        .map(|shard| read(shard).rows.capacity())
+        .sum();
+    assert!(room >= 100_000, "room for {room} prefixes");
+    // Asking again for less, or for the same, takes nothing away.
+    table.expect_prefixes(10);
+    let still: usize = table
+        .shards
+        .iter()
+        .map(|shard| read(shard).rows.capacity())
+        .sum();
+    assert_eq!(still, room);
+    // And it is a table like any other.
+    let peer = table.add_peer(external(1)).unwrap();
+    let attrs = table
+        .attrs()
+        .intern(&Route::via(&[64512]).attributes(), None)
+        .unwrap();
+    table.announce(peer, &attrs, &[Nlri::from(prefix("192.0.2.0/24"))]);
+    assert_eq!(table.prefix_count(), 1);
+}

crates/server/proto/config.proto

@@ -407,7 +407,9 @@ message Peer {
   // the table with it, and shows an established session with fewer than
   // half as needing attention: a peer that is up but has sent next to
   // nothing looks, without this, exactly like one that is fine. Zero, the
-  // default, compares nothing.
+  // default, compares nothing. The routing table is also made with room
+  // for the largest of these from the start, so that it need not grow,
+  // and so use twice its memory for a moment, while the tables come in.
   uint32 expected_routes = 17;
 }
 

crates/server/src/server.rs

@@ -118,7 +118,7 @@ impl BgpServer {
         for route in &config.originate {
             local_rib.announce(route.clone()).await?;
         }
-        let configured = config
+        let configured: Vec<ConfiguredPeer> = config
             .peers
             .iter()
             .map(|p| ConfiguredPeer {
@@ -134,6 +134,11 @@ impl BgpServer {
         // first routes are.
         let (rpki, rpki_client) = config.rpki.clone().map(Rpki::spawn).unzip();
         let rib = Arc::new(RibTable::new(table_shards()));
+        // Sized for the largest table a peer is expected to send: the
+        // prefixes of the others are mostly the same ones.
+        if let Some(expected) = configured.iter().filter_map(|p| p.expected_routes).max() {
+            rib.expect_prefixes(expected as usize);
+        }
         // Started now, so that a machine on which it cannot be done stops
         // the daemon from starting. There is nothing to install yet.
         let (kernel, kernel_task) = match config.kernel {