fib: resolve next hops through the kernel's routing table

Before this change, Ferrix only installed a route if its BGP next hop was
directly connected: an address in the subnet of an interface that is up.
A route with any other next hop was held at import and never installed.
This made routes from iBGP neighbors unusable unless the neighbor set
itself as the next hop, and it is why the route reflector rewrites the
next hop on every reflected route.

Ferrix now resolves a next hop that is not directly connected through the
kernel's own routes, as other BGP implementations do:

* The route is looked up in one kernel table, the "resolution table"
  (`kernel { resolution_table: N }`, default 254 = main). The longest
  prefix that contains the next hop wins; of several routes for that
  prefix, the one with the lowest metric.
* Because the Linux kernel does not resolve gateways recursively, the BGP
  route is installed with the gateway and interface of the kernel route
  that covers the next hop. A kernel route without a gateway (a route
  onto a point-to-point or tunnel interface) makes the next hop itself
  the gateway on that interface. A kernel route that names a nexthop
  object is resolved through the object.
* Routes with protocol `bgp` never resolve a next hop, whether Ferrix's
  own or another daemon's. A next hop that is reachable only through a
  BGP route depends on another BGP next hop, and such chains are not
  predictable.
* The default route does not resolve a next hop unless
  `kernel { resolve_via_default_route: true }` is set. On a router the
  default route points at a transit provider, and a next hop that is
  reachable only through it is not usable.
* Blackhole, unreachable and prohibit routes make the next hop
  unresolvable, as does a route whose interface is down.

The same resolution is used by the import policy and the revalidator, so
a route whose next hop cannot be resolved is still held rather than
selected (RFC 4271 section 9.1.2.1), and is taken when a kernel route
makes its next hop resolvable.

Implementation:

* `bgp_fib::routes::KernelRoutes` holds the routes of the resolution
  table and answers longest-prefix lookups.
* `bgp_fib::topology::Topology` bundles the interfaces, the kernel routes
  and the kernel's nexthop objects into one value. The kernel task
  publishes it through `Fib::topology()` (formerly `Fib::interfaces()`),
  and `plan::resolve` takes it in place of `Interfaces`.
* The kernel task learns the resolution table with a dump of that table
  alone, using NETLINK_GET_STRICT_CHK (Linux 4.20 and later; older
  kernels get a dump of every table, filtered in user space). It asks
  again, debounced, whenever a dedicated netlink socket that is a member
  of the route multicast groups receives anything. That socket is given a
  64 KiB receive buffer on purpose: the kernel reports Ferrix's own route
  installs on it too, up to a million at a time, and an ENOBUFS overflow
  is treated as the same hint as a message. Its contents are never
  parsed. An earlier attempt to receive route notifications on the
  socket that writes routes failed, because the kernel only excludes the
  requesting socket from a notification when NLM_F_ECHO was requested.
* `Unresolved::NotConnected` is replaced by `NoRoute`, `OnlyDefault` and
  `Discarded`, which appear as the reasons in `/api/kernel` and on the
  status page.

Configuration: two new `kernel` fields, `resolution_table` and
`resolve_via_default_route`, documented in the configuration reference
and in the kernel guide, whose "Next Hop Resolution" section is rewritten.
Existing configurations behave as before except that routes with far
next hops may now be installed where a static or connected route covers
the next hop.

Testing: unit tests for `KernelRoutes`, `Topology` and resolution
through static, tunnel, nexthop-object, blackhole and default routes; a
kernel test (`a_far_next_hop_goes_the_way_the_kernels_routes_say`) that
checks the installed routes follow static, tunnel and nexthop-object
routes as they are added, replaced and removed; the existing kernel
tests, the interoperability lab with FRR and BIRD, and the workspace
tests, all passing. Installing 200,000 routes still runs at about
186,000 routes per second.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

docs/content/docs/development/architecture.md

@@ -163,9 +163,12 @@ UPDATE message, and waits when the session's output queue is full.
 ### Kernel Route Installation
 
 The `fib` crate reads best path changes and maintains the kernel routing
-table over rtnetlink, with one route for each prefix through the next hop
-of its best path. It also receives the router's interfaces and addresses
-from the kernel, which determine whether a next hop is resolvable.
+table over rtnetlink, with one route for each prefix through the resolved
+next hop of its best path. It also receives the router's interfaces,
+addresses and the routes of the resolution table from the kernel, which
+determine whether and how a next hop is resolvable; the import policy
+consults the same view, so that a route with an unresolvable next hop is
+held rather than selected.
 
 ### HTTP Interface
 

docs/content/docs/getting-started/introduction.md

@@ -67,7 +67,6 @@ Ferrix 0.2.5 does not support the following features:
   Generalized TTL Security Mechanism (GTSM).
 - Long-Lived Graceful Restart, Bidirectional Forwarding Detection (BFD),
   and configuration changes without a restart.
-- Recursive resolution of next hops that are not directly connected.
 
 For the implementation status of each RFC, see
 [Supported Standards](@/docs/reference/rfcs.md).

docs/content/docs/guides/kernel.md

@@ -33,6 +33,8 @@ kernel {
 | `metric` | Metric of the installed routes. The kernel prefers the route with the lowest metric. | 20 |
 | `without_nexthop_objects` | Installs each route with its own gateway instead of a nexthop object | Disabled |
 | `stale_seconds` | Time for which routes from a previous run are retained at startup | 120 seconds |
+| `resolution_table` | Kernel routing table whose routes resolve next hops that are not directly connected | 254 (`main`) |
+| `resolve_via_default_route` | Permits the default route to resolve a next hop | Disabled |
 
 Ferrix requires the `CAP_NET_ADMIN` capability to modify the kernel routing
 table. If the kernel rejects a request for lack of privileges, the daemon
@@ -44,15 +46,55 @@ exits. See [Granting Privileges](@/docs/getting-started/installation.md#granting
 
 ## Next Hop Resolution
 
-Ferrix installs a route only if its next hop is directly connected: the
-next hop must be an address in a subnet of an interface that is up. For an
-IPv6 route with a link-local next hop, the outgoing interface is the
-interface on which the neighbor's session is established.
+Ferrix resolves the next hop of each active route to a gateway and an
+outgoing interface before it installs the route:
 
-Ferrix does not resolve next hops recursively through other routes.
-Therefore, a route whose next hop is not directly connected, such as a
-route from an IBGP neighbor that does not set itself as the next hop, is
-not installed.
+- A next hop that is directly connected, that is, an address in a subnet
+  of an interface that is up, is the gateway, and the interface is that
+  interface. For an IPv6 route with a link-local next hop, the outgoing
+  interface is the interface on which the neighbor's session is
+  established.
+- A next hop that is not directly connected is resolved through the
+  routes of the resolution table (`resolution_table`, by default the
+  `main` table). Ferrix selects the route with the longest prefix that
+  contains the next hop, and of several routes for that prefix the one
+  with the lowest metric. If the route has a gateway, the gateway and the
+  route's interface are used. If the route has no gateway, as a route to a
+  point-to-point interface or a tunnel does not, the next hop itself is
+  the gateway and the route's interface is used. A route that names a
+  kernel nexthop object is resolved through that object.
+
+The Linux kernel does not resolve gateways recursively, so Ferrix installs
+the route with the resolved gateway and interface rather than with the
+BGP next hop. For example, a route from an IBGP neighbor with the next hop
+`192.0.2.9`, when the kernel has the static route
+`192.0.2.0/24 via 10.0.0.2 dev wg0`, is installed as a route
+`via 10.0.0.2 dev wg0`. When the static route changes or is removed,
+Ferrix resolves the next hop again and updates or removes the installed
+route.
+
+The following routes do not resolve a next hop:
+
+- Routes with protocol `bgp`, whether installed by Ferrix or by another
+  BGP daemon. A next hop that is reachable only through a BGP route would
+  depend on another next hop, and such dependencies between routes are not
+  predictable.
+- The default route, unless `resolve_via_default_route` is `true`. On a
+  router, the default route typically points to a transit provider, and a
+  next hop that is reachable only through it is not a usable next hop.
+- Blackhole, unreachable and prohibit routes. A next hop covered by such a
+  route is unresolvable.
+- Routes whose interface is down.
+
+Ferrix learns the routes of the resolution table from the kernel at
+startup and is notified of changes. Changes to the routes take effect
+within a few seconds.
+
+> [!NOTE]
+> Routes from an IBGP neighbor that does not set itself as the next hop,
+> such as a route reflector that preserves the next hop, require a route to
+> the next hop in the resolution table: a static route, a route to the
+> tunnel on which the next hop is reached, or a route from an IGP.
 
 ### Exclusion of Unresolvable Routes
 

docs/content/docs/guides/passing-routes-on.md

@@ -367,10 +367,12 @@ not affected.
 
 Ferrix advertises a reflected route with the local address of the session
 as next hop, as it does every route. A standard route reflector does not
-change the next hop. Ferrix does, because it does not resolve next hops
-that are not directly connected: a client that is not connected to the
-router that learned the route can then still use it, and sends the
-traffic through the route reflector.
+change the next hop. Ferrix does, so that a client needs no route to the
+router that learned the route: the client sends the traffic through the
+route reflector, which forwards it. Preserving the next hop on reflection
+is not yet configurable; clients that have a route to the originating
+router, for example over a tunnel, would then forward to it directly. See
+[Next Hop Resolution](@/docs/guides/kernel.md#next-hop-resolution).
 
 ### Loop Prevention
 

docs/content/docs/reference/configuration.md

@@ -214,6 +214,8 @@ Statements for kernel route installation. See
 | `metric` <span class="type">uint32, optional</span> | 20 | Specifies the metric of installed routes. |
 | `without_nexthop_objects` <span class="type">bool</span> | `false` | Installs each route with its own gateway instead of a nexthop object. |
 | `stale_seconds` <span class="type">uint32, optional</span> | 120 | Specifies the time, in seconds, for which routes installed by a previous run are retained at startup. A value of 0 removes them immediately. |
+| `resolution_table` <span class="type">uint32</span> | 254 (`main`) | Specifies the kernel routing table whose routes resolve next hops that are not directly connected. Routes with protocol `bgp` in this table are not used. See [Next Hop Resolution](@/docs/guides/kernel.md#next-hop-resolution). |
+| `resolve_via_default_route` <span class="type">bool</span> | `false` | Permits the default route of the resolution table to resolve a next hop. |
 
 ## GracefulRestart
 

docs/content/docs/reference/rfcs.md

@@ -38,9 +38,9 @@ The base protocol specification.
 | §8.1.1 | Optional `CollisionDetectEstablishedState` (`collision_detect_established_state`) | ✅ |
 | §6.2 | Bad BGP Identifier check (neighbor uses the local identifier, or 0.0.0.0) | ✅ |
 | §9 | Decision process / best-path selection | ✅ |
-| §9.1.2 | Best routes installed for forwarding (Linux kernel, over netlink), where the next hop is directly connected | ✅ |
+| §9.1.2 | Best routes installed for forwarding (Linux kernel, over netlink) | ✅ |
 | §9.1.2.1 | Routes with an unresolvable next hop excluded from selection | ✅ where routes are installed in the kernel: such a route is held out of the table until its next hop can be reached |
-| §9.1.2.1 | Next hops resolved through other routes | ❌ (only next hops that are neighbors count as reachable) |
+| §9.1.2.1 | Next hops resolved through other routes | ✅ (through the kernel's connected, static and IGP routes; not through BGP routes) |
 | §9 | Adj-RIB-In (per-peer table of received routes) | ✅ |
 | §9 | Locally originated routes, advertised to all established peers | ✅ |
 | §9 | Advertisement of the best path for each prefix to neighbors (`export.learned_routes`), with the local AS prepended and the local address as next hop; MED and LOCAL_PREF not sent to EBGP neighbors; LOCAL_PREF sent to IBGP neighbors (§5.1.4, §5.1.5, §9.1.3) | ✅ |
@@ -116,7 +116,7 @@ The base protocol specification.
 | ORIGINATOR_ID and CLUSTER_LIST added on reflection, not sent to EBGP neighbors | ✅ |
 | Routes with the local router ID as ORIGINATOR_ID, or the local cluster ID in CLUSTER_LIST, rejected | ✅ |
 | Configurable cluster ID | ✅ |
-| Next hop unchanged on reflection | ❌ The route reflector is the next hop: next hops that are not directly connected are not resolved |
+| Next hop unchanged on reflection | ❌ The route reflector sets itself as the next hop |
 | ORIGINATOR_ID and CLUSTER_LIST length as best path tie breakers (§9) | ❌ |
 
 ### RFC 8538 — Notification Message Support for BGP Graceful Restart