Before this change, Ferrix only installed a route if its BGP next hop was
directly connected: an address in the subnet of an interface that is up.
A route with any other next hop was held at import and never installed.
This made routes from iBGP neighbors unusable unless the neighbor set
itself as the next hop, and it is why the route reflector rewrites the
next hop on every reflected route.
Ferrix now resolves a next hop that is not directly connected through the
kernel's own routes, as other BGP implementations do:
* The route is looked up in one kernel table, the "resolution table"
(`kernel { resolution_table: N }`, default 254 = main). The longest
prefix that contains the next hop wins; of several routes for that
prefix, the one with the lowest metric.
* Because the Linux kernel does not resolve gateways recursively, the BGP
route is installed with the gateway and interface of the kernel route
that covers the next hop. A kernel route without a gateway (a route
onto a point-to-point or tunnel interface) makes the next hop itself
the gateway on that interface. A kernel route that names a nexthop
object is resolved through the object.
* Routes with protocol `bgp` never resolve a next hop, whether Ferrix's
own or another daemon's. A next hop that is reachable only through a
BGP route depends on another BGP next hop, and such chains are not
predictable.
* The default route does not resolve a next hop unless
`kernel { resolve_via_default_route: true }` is set. On a router the
default route points at a transit provider, and a next hop that is
reachable only through it is not usable.
* Blackhole, unreachable and prohibit routes make the next hop
unresolvable, as does a route whose interface is down.
The same resolution is used by the import policy and the revalidator, so
a route whose next hop cannot be resolved is still held rather than
selected (RFC 4271 section 9.1.2.1), and is taken when a kernel route
makes its next hop resolvable.
Implementation:
* `bgp_fib::routes::KernelRoutes` holds the routes of the resolution
table and answers longest-prefix lookups.
* `bgp_fib::topology::Topology` bundles the interfaces, the kernel routes
and the kernel's nexthop objects into one value. The kernel task
publishes it through `Fib::topology()` (formerly `Fib::interfaces()`),
and `plan::resolve` takes it in place of `Interfaces`.
* The kernel task learns the resolution table with a dump of that table
alone, using NETLINK_GET_STRICT_CHK (Linux 4.20 and later; older
kernels get a dump of every table, filtered in user space). It asks
again, debounced, whenever a dedicated netlink socket that is a member
of the route multicast groups receives anything. That socket is given a
64 KiB receive buffer on purpose: the kernel reports Ferrix's own route
installs on it too, up to a million at a time, and an ENOBUFS overflow
is treated as the same hint as a message. Its contents are never
parsed. An earlier attempt to receive route notifications on the
socket that writes routes failed, because the kernel only excludes the
requesting socket from a notification when NLM_F_ECHO was requested.
* `Unresolved::NotConnected` is replaced by `NoRoute`, `OnlyDefault` and
`Discarded`, which appear as the reasons in `/api/kernel` and on the
status page.
Configuration: two new `kernel` fields, `resolution_table` and
`resolve_via_default_route`, documented in the configuration reference
and in the kernel guide, whose "Next Hop Resolution" section is rewritten.
Existing configurations behave as before except that routes with far
next hops may now be installed where a static or connected route covers
the next hop.
Testing: unit tests for `KernelRoutes`, `Topology` and resolution
through static, tunnel, nexthop-object, blackhole and default routes; a
kernel test (`a_far_next_hop_goes_the_way_the_kernels_routes_say`) that
checks the installed routes follow static, tunnel and nexthop-object
routes as they are added, replaced and removed; the existing kernel
tests, the interoperability lab with FRR and BIRD, and the workspace
tests, all passing. Installing 200,000 routes still runs at about
186,000 routes per second.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
