Install a full table in the kernel in about a second

A million IPv4 routes took 1.6 s to install (release build); they take
1.1 s now, and 0.5 s to remove instead of 1.15 s. Most of what is left is
the kernel's own work on each route.

- Nexthop objects (Linux 5.3 and later): a gateway is put in the kernel
  once and routes name it, so the kernel checks it once and not for every
  route. The daemon makes its own objects, uses nobody else's, is told
  when one of its own is removed and puts it back, and removes them when
  it stops, which takes all the routes that name them along at once.
  `without_nexthop_objects` turns this off; a kernel without them is
  noticed and served as before.
- Writing to the kernel is a task of its own, fed through a short queue,
  so the next batch is worked out while the kernel is busy with the last.
  "In step" is said when the kernel has done the batch that completes the
  table, not when it was handed over.
- What a next hop resolves to is kept, by next hop: it was looked up among
  the machine's addresses for every route.
- A quicker hash for the plan's maps, and no lookups in the ones that are
  empty.

Fixes a hole found on the way: when an interface went down and came back
within the second it takes us to react, the kernel had dropped our routes
and we did not put them back, since nothing about them had changed for
us. After such a change the kernel is now asked what it actually has.

scripts/in-namespace.sh runs one command in a scratch namespace;
FERRIX_ROUTES sets the size of the timing test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

bin/bgp/README.md

@@ -190,6 +190,15 @@ kernel {
   installed, and neither is whatever it replaced: looking up the way to such
   a next hop through other routes is not done yet. The status page counts
   these, by reason.
+- **Nexthop objects.** Where the kernel has them (Linux 5.3 and later) a
+  gateway is put there once, as a nexthop object, and routes name it; `ip
+  route` shows such routes with `nhid`, and `ip nexthop` the objects. The
+  kernel then checks the gateway once and not for every route, which makes
+  a full table about a third faster to install, and lets all of them be
+  removed at once when the daemon stops. The objects the daemon makes are
+  its own: it does not use anyone else's, puts its own back if one is
+  removed, and removes them when it stops. `without_nexthop_objects: true`
+  has every route carry its own gateway instead, as on older kernels.
 - **What is ours.** Routes are marked `proto bgp`. Every route with that
   mark in the configured table with the configured metric is taken to be
   this daemon's, and no other route is ever changed or removed. So do not
@@ -220,11 +229,24 @@ To forward packets for others the machine also needs
 `net.ipv4.ip_forward=1` and `net.ipv6.conf.all.forwarding=1`; the daemon does
 not set them.
 
+### How long it takes
+
+A million IPv4 routes, about the size of the Internet's table, go into the
+kernel in a little over a second and come out in half a second (release
+build, in a virtual machine on an Apple M-series laptop; the test
+`many_routes_are_installed_and_removed` with `FERRIX_ROUTES=1000000`).
+About three quarters of that is the kernel's own work on each route.
+What keeps the rest small: requests go to the kernel hundreds to a write,
+with one acknowledgement for each write and not for each route; a task of
+its own does the writing, so the next batch is worked out while the kernel
+is busy with the last; and what a next hop resolves to is worked out once
+for each next hop, not once for each route.
+
 ### Testing against a real kernel
 
 The tests that change a kernel's routes are not run by `cargo test`.
 `scripts/kernel-tests.sh` runs them on Linux, each in a network namespace
-made for it: the route installer against `ip route`, the daemon with a peer,
+made for it (`scripts/in-namespace.sh` runs any one command that way): the route installer against `ip route`, the daemon with a peer,
 and two daemons that route for each other with pings across. From macOS,
 `scripts/linux-vm.sh` runs it (or any command, such as `cargo test`) in a
 Lima virtual machine that it creates on first use.