Rayhaan JaufeerallyandClaude Fable 5.1
7425b101ed
A table given out anew to a session, at its start or when the journal had let changes go, was gathered as a verdict for each prefix, each with the route dressed for the session, and sent once the table was all there. Dressing a route parses its attributes and encodes them for the session, so a table of five million routes took seconds before the first UPDATE went, on one worker thread without a yield. And the routes that share attributes went out in separate UPDATEs after all: they were grouped by the dressed form, of which only so many are kept, and the routes of a set are spread over the whole table, so by the time the next was reached the form had been let go of and made anew. The mesh benchmark's new peer got four million routes in 3.8 million UPDATEs; on a RIPE RIS table, grouped whole, a full table is 160,000 UPDATEs and 21 MB, and grouped in slices of 4,096 routes it is 845,000 UPDATEs and 72 MB, so slices are not the answer either. The table is now gathered by what dresses its routes: the attribute set as the table has it, who the route is from, and its IP version. Nothing is dressed while gathering, which is a map insert a route. Once the table is all there, each set is dressed once, with the thread lent to the other tasks every few thousand, and then the sets go out one after the other, each as few UPDATEs as its routes fit in, with the peer reading the first while the rest are sent. A prefix heard of again while the table is read supersedes what was heard before: each word on a prefix is numbered, and only the latest goes. Path identifiers are given at gathering, in the order the table has a prefix's routes, so that they are as before. What was announced before and not heard of again is withdrawn after the table, as before. The exporter never blocks the thread on a table: the steady state's batch is small, and the gathered table is gone through with yields. Testing: a new test gives a session a table of 300 routes sharing three sets of attributes, spread over the shards, and checks they go in three UPDATEs. The mesh benchmark has a new phase in which a peer takes the first router's table once the mesh is complete, timing the first UPDATE and the whole; its feeders now negotiate no hold timer, since they say nothing after their tables, and `BENCH_LOG` prints the routers' logs. A new example, packing, counts the UPDATEs and octets a RIS table takes grouped whole and in slices. The server tests, the integration tests, the kernel tests and every lab phase pass. In the Linux VM, three rounds alternating with the previous commit, the new peer's first UPDATE comes 2.9 s after its session is up instead of 5.3 s, and it has all four million routes in 4.0 s instead of 8.0 s, in 667,000 UPDATEs instead of 3.76 million; the whole run takes 22.4 s of CPU time instead of 26.2 s. The routers' exchange of their tables, which does not go through this path once their sessions are up, is unchanged within its noise. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
72 lines
2.7 KiB
Rust
72 lines
2.7 KiB
Rust
//! How many UPDATEs, and how many octets, a table takes when the routes
|
|
//! that share attributes are grouped only within slices of the table
|
|
//! taken in shard order, for a few slice sizes: what streaming a table
|
|
//! out in slices costs against grouping it whole.
|
|
|
|
use std::collections::HashMap;
|
|
use std::hash::{BuildHasher, RandomState};
|
|
use std::path::PathBuf;
|
|
|
|
use bgp_bench::mrt::Dump;
|
|
|
|
fn main() {
|
|
let path = std::env::var("BENCH_MRT")
|
|
.map(PathBuf::from)
|
|
.unwrap_or_else(|_| "crates/bench/data/rrc03-bview.20260915.0000.gz".into());
|
|
let dump = Dump::read(&path).expect("a readable dump");
|
|
let peers = dump.largest_peers(usize::MAX);
|
|
let full: Vec<usize> = peers
|
|
.iter()
|
|
.copied()
|
|
.filter(|&p| dump.routes[p].len() > 800_000)
|
|
.collect();
|
|
println!("{} peers with a full table", full.len());
|
|
let hasher = RandomState::new();
|
|
for (what, chosen) in [
|
|
("one full table", vec![full[0]]),
|
|
("every full table, every path", full.clone()),
|
|
] {
|
|
// (hash of prefix, attribute set, prefix octets) in shard order.
|
|
let mut routes: Vec<(u64, u32, usize)> = chosen
|
|
.iter()
|
|
.flat_map(|&p| dump.routes[p].iter())
|
|
.map(|(prefix, set)| {
|
|
(
|
|
hasher.hash_one(prefix),
|
|
*set,
|
|
1 + usize::from(prefix.length()).div_ceil(8) + 4,
|
|
)
|
|
})
|
|
.collect();
|
|
routes.sort_unstable_by_key(|r| r.0);
|
|
println!("== {what}: {} routes", routes.len());
|
|
for slice in [4_096usize, 16_384, 65_536, 262_144, usize::MAX] {
|
|
let mut updates = 0usize;
|
|
let mut octets = 0usize;
|
|
for chunk in routes.chunks(slice.min(routes.len())) {
|
|
let mut groups: HashMap<u32, usize> = HashMap::new();
|
|
for &(_, set, prefix_octets) in chunk {
|
|
*groups.entry(set).or_insert(0) += prefix_octets;
|
|
}
|
|
for (set, prefix_octets) in groups {
|
|
// One UPDATE per 4096 octets of prefixes, roughly.
|
|
let attrs = dump.attribute_sets[set as usize].len();
|
|
let messages = prefix_octets.div_ceil(4000 - attrs.min(3000)).max(1);
|
|
updates += messages;
|
|
octets += messages * (23 + attrs) + prefix_octets;
|
|
}
|
|
}
|
|
println!(
|
|
" slice {:>9}: {:>9} UPDATEs, {:>5.1} MB on the wire",
|
|
if slice == usize::MAX {
|
|
"whole".to_string()
|
|
} else {
|
|
slice.to_string()
|
|
},
|
|
updates,
|
|
octets as f64 / 1e6
|
|
);
|
|
}
|
|
}
|
|
}
|