ferrix/docs/content/docs/guides/kernel.md

+++ title = "Kernel Route Installation" description = "Describes how Ferrix installs active routes in the Linux kernel routing table over netlink, and how it handles next hop resolution, restarts, and kernel errors." weight = 5

[extra] nav = "Kernel Route Installation" +++

Ferrix can install the active route for each prefix in the Linux kernel routing table, so that the router forwards traffic according to BGP. Ferrix communicates with the kernel over rtnetlink. Kernel route installation is supported on Linux only.

If the kernel statement is not configured, Ferrix operates as a BGP speaker only and does not modify the kernel routing table.

Configuring Kernel Route Installation

The following example enables kernel route installation with the default values:

kernel {
  table: 254
  metric: 20
}

Statement	Description	Default
table	Kernel routing table in which routes are installed	254 (main)
metric	Metric of the installed routes. The kernel prefers the route with the lowest metric.	20
without_nexthop_objects	Installs each route with its own gateway instead of a nexthop object	Disabled
stale_seconds	Time for which routes from a previous run are retained at startup	120 seconds
resolution_table	Kernel routing table whose routes resolve next hops that are not directly connected	254 (main)
resolve_via_default_route	Permits the default route to resolve a next hop	Disabled

Ferrix requires the CAP_NET_ADMIN capability to modify the kernel routing table. If the kernel rejects a request for lack of privileges, the daemon exits. See Granting Privileges.

Note

Ferrix does not enable IP forwarding. Set net.ipv4.ip_forward=1 and net.ipv6.conf.all.forwarding=1 to forward transit traffic.

Next Hop Resolution

Ferrix resolves the next hop of each active route to a gateway and an outgoing interface before it installs the route:

• A next hop that is directly connected, that is, an address in a subnet of an interface that is up, is the gateway, and the interface is that interface. For an IPv6 route with a link-local next hop, the outgoing interface is the interface on which the neighbor's session is established.
• A next hop that is not directly connected is resolved through the routes of the resolution table (resolution_table, by default the main table). Ferrix selects the route with the longest prefix that contains the next hop, and of several routes for that prefix the one with the lowest metric. If the route has a gateway, the gateway and the route's interface are used. If the route has no gateway, as a route to a point-to-point interface or a tunnel does not, the next hop itself is the gateway and the route's interface is used. A route that names a kernel nexthop object is resolved through that object.

The Linux kernel does not resolve gateways recursively, so Ferrix installs the route with the resolved gateway and interface rather than with the BGP next hop. For example, a route from an IBGP neighbor with the next hop 192.0.2.9, when the kernel has the static route 192.0.2.0/24 via 10.0.0.2 dev wg0, is installed as a route via 10.0.0.2 dev wg0. When the static route changes or is removed, Ferrix resolves the next hop again and updates or removes the installed route.

The following routes do not resolve a next hop:

• Routes with protocol bgp, whether installed by Ferrix or by another BGP daemon. A next hop that is reachable only through a BGP route would depend on another next hop, and such dependencies between routes are not predictable.
• The default route, unless resolve_via_default_route is true. On a router, the default route typically points to a transit provider, and a next hop that is reachable only through it is not a usable next hop.
• Blackhole, unreachable and prohibit routes. A next hop covered by such a route is unresolvable.
• Routes whose interface is down.

Ferrix learns the routes of the resolution table from the kernel at startup and is notified of changes. Changes to the routes take effect within a few seconds.

Note

Routes from an IBGP neighbor that does not set itself as the next hop, such as a route reflector that preserves the next hop, require a route to the next hop in the resolution table: a static route, a route to the tunnel on which the next hop is reached, or a route from an IGP.

Exclusion of Unresolvable Routes

If the kernel statement is configured, Ferrix excludes routes with an unresolvable next hop from best path selection (RFC 4271 Section 9.1.2.1). This ensures that such a route is not selected while a usable route for the same prefix exists.

A route with an unresolvable next hop is a held route. Ferrix counts it as rejected with the reason next hop unreachable, and displays it as held on the status page. When an interface or address change makes the next hop resolvable, Ferrix accepts the route. A route in the routing table whose next hop becomes unresolvable is also moved to the held state.

If the kernel statement is not configured, Ferrix has no information about next hop reachability and accepts all routes.

Connected Networks

Ferrix never installs a route for a subnet of a local interface, or for a prefix within such a subnet, even if the import policy accepts it. The kernel reaches these destinations directly. Without this rule, a neighbor on a shared LAN that advertises part of the LAN could attract the traffic for other hosts on the LAN, including the BGP sessions with them.

Ferrix holds such a route, and counts it as rejected with the reason prefix is a network of this router. If the subnet is removed from the interface, Ferrix accepts the route.

Route Ownership

Ferrix installs routes with the protocol bgp (proto bgp). Ferrix considers every route with this protocol, in the configured table and with the configured metric, to be its own. It does not modify or remove any other route.

Warning

Do not run two routing daemons that install BGP routes in the same table with the same metric. Each daemon would remove the other's routes.

Nexthop Objects

On Linux 5.3 and later, Ferrix installs each gateway once as a nexthop object, and installs routes that reference the object. The kernel then validates each gateway once instead of once for each route. This reduces the time to install a full routing table by approximately one third, and allows all routes to be removed at once when the daemon stops.

The command ip route displays these routes with an nhid attribute, and the command ip nexthop displays the objects. Ferrix uses only the nexthop objects that it creates, recreates them if they are removed, and removes them when it stops.

To install each route with its own gateway, as on older kernels, configure without_nexthop_objects: true.

Startup and Shutdown Behavior

Ferrix handles its kernel routes as follows:

• At startup, Ferrix retains the routes installed by a previous run, so that the router continues to forward traffic while the neighbors readvertise their routes. When every configured neighbor has sent an End-of-RIB marker for each address family, or after stale_seconds, whichever is first, Ferrix removes the routes that no neighbor has readvertised.
• On SIGINT or SIGTERM, Ferrix closes its BGP sessions and removes its routes from the kernel.
• On SIGUSR1, Ferrix leaves its routes in the kernel and exits, to be started again (graceful restart). See Graceful Restart.
• After a crash, the routes remain in the kernel until the next start.

Interface Changes

Ferrix receives the router's interfaces and addresses from the kernel, and receives a notification when they change. Ferrix reevaluates all routes 1 second after the last change, and at most 5 seconds after the first change, because a change can affect the resolution of any next hop.

Kernel Error Handling

The kernel reports each request that it rejects. If the kernel's notifications exceed the socket buffer, or if the kernel does not respond within 10 seconds, Ferrix reads the kernel routing table and corrects any difference from the expected state.

Verifying Installed Routes

To display the installed routes, use the following commands:

ip route show proto bgp
ip -6 route show proto bgp
ip nexthop show

The status page and the HTTP API endpoint /api/kernel display the number of installed routes, the number of routes not installed and the reason, the last error reported by the kernel, and the interfaces. Table 2 lists the related metrics.

Metric	Description
bgp_kernel_routes	Number of routes installed by Ferrix
bgp_kernel_routes_unresolved	Number of prefixes whose active route is not installed because its next hop is unresolvable
bgp_kernel_routes_refused	Number of prefixes whose active route the kernel rejected
bgp_kernel_route_changes_total{change}	Number of route installations and removals requested
bgp_kernel_errors_total	Number of requests that the kernel rejected

Performance

In a virtual machine on an Apple M-series computer, with a release build, Ferrix installs 1 million IPv4 routes in slightly more than 1 second and removes them in approximately 0.5 seconds. Approximately three quarters of this time is spent in the kernel. Ferrix reduces its own share as follows:

• It sends hundreds of requests in each write to the netlink socket, and requests one acknowledgment for each write instead of each route.
• A dedicated task writes to the socket, so that the next batch is prepared while the kernel processes the previous one.
• It resolves each next hop once, not once for each route.