ferrix/docs/content/docs/guides/kernel.md

+++ title = "Kernel Route Installation" description = "Describes how Ferrix installs active routes in the Linux kernel routing table over netlink, and how it handles next hop resolution, restarts, and kernel errors." weight = 5

[extra] nav = "Kernel Route Installation" +++

Ferrix can install the active route for each prefix in the Linux kernel routing table, so that the router forwards traffic according to BGP. Ferrix communicates with the kernel over rtnetlink. Kernel route installation is supported on Linux only.

If the kernel statement is not configured, Ferrix operates as a BGP speaker only and does not modify the kernel routing table.

Configuring Kernel Route Installation

The following example enables kernel route installation with the default values:

kernel {
  table: 254
  metric: 20
}

Statement	Description	Default
table	Kernel routing table in which routes are installed	254 (main)
metric	Metric of the installed routes. The kernel prefers the route with the lowest metric.	20
without_nexthop_objects	Installs each route with its own gateway instead of a nexthop object	Disabled
stale_seconds	Time for which routes from a previous run are retained at startup	120 seconds

Ferrix requires the CAP_NET_ADMIN capability to modify the kernel routing table. If the kernel rejects a request for lack of privileges, the daemon exits. See Granting Privileges.

Note

Ferrix does not enable IP forwarding. Set net.ipv4.ip_forward=1 and net.ipv6.conf.all.forwarding=1 to forward transit traffic.

Next Hop Resolution

Ferrix installs a route only if its next hop is directly connected: the next hop must be an address in a subnet of an interface that is up. For an IPv6 route with a link-local next hop, the outgoing interface is the interface on which the neighbor's session is established.

Ferrix does not resolve next hops recursively through other routes. Therefore, a route whose next hop is not directly connected, such as a route from an IBGP neighbor that does not set itself as the next hop, is not installed.

Exclusion of Unresolvable Routes

If the kernel statement is configured, Ferrix excludes routes with an unresolvable next hop from best path selection (RFC 4271 Section 9.1.2.1). This ensures that such a route is not selected while a usable route for the same prefix exists.

A route with an unresolvable next hop is a held route. Ferrix counts it as rejected with the reason next hop unreachable, and displays it as held on the status page. When an interface or address change makes the next hop resolvable, Ferrix accepts the route. A route in the routing table whose next hop becomes unresolvable is also moved to the held state.

If the kernel statement is not configured, Ferrix has no information about next hop reachability and accepts all routes.

Connected Networks

Ferrix never installs a route for a subnet of a local interface, or for a prefix within such a subnet, even if the import policy accepts it. The kernel reaches these destinations directly. Without this rule, a neighbor on a shared LAN that advertises part of the LAN could attract the traffic for other hosts on the LAN, including the BGP sessions with them.

Ferrix holds such a route, and counts it as rejected with the reason prefix is a network of this router. If the subnet is removed from the interface, Ferrix accepts the route.

Route Ownership

Ferrix installs routes with the protocol bgp (proto bgp). Ferrix considers every route with this protocol, in the configured table and with the configured metric, to be its own. It does not modify or remove any other route.

Warning

Do not run two routing daemons that install BGP routes in the same table with the same metric. Each daemon would remove the other's routes.

Nexthop Objects

On Linux 5.3 and later, Ferrix installs each gateway once as a nexthop object, and installs routes that reference the object. The kernel then validates each gateway once instead of once for each route. This reduces the time to install a full routing table by approximately one third, and allows all routes to be removed at once when the daemon stops.

The command ip route displays these routes with an nhid attribute, and the command ip nexthop displays the objects. Ferrix uses only the nexthop objects that it creates, recreates them if they are removed, and removes them when it stops.

To install each route with its own gateway, as on older kernels, configure without_nexthop_objects: true.

Startup and Shutdown Behavior

Ferrix handles its kernel routes as follows:

• At startup, Ferrix retains the routes installed by a previous run, so that the router continues to forward traffic while the neighbors readvertise their routes. When every configured neighbor has sent an End-of-RIB marker for each address family, or after stale_seconds, whichever is first, Ferrix removes the routes that no neighbor has readvertised.
• On SIGINT or SIGTERM, Ferrix closes its BGP sessions and removes its routes from the kernel.
• On SIGUSR1, Ferrix leaves its routes in the kernel and exits, to be started again (graceful restart). See Graceful Restart.
• After a crash, the routes remain in the kernel until the next start.

Interface Changes

Ferrix receives the router's interfaces and addresses from the kernel, and receives a notification when they change. Ferrix reevaluates all routes 1 second after the last change, and at most 5 seconds after the first change, because a change can affect the resolution of any next hop.

Kernel Error Handling

The kernel reports each request that it rejects. If the kernel's notifications exceed the socket buffer, or if the kernel does not respond within 10 seconds, Ferrix reads the kernel routing table and corrects any difference from the expected state.

Verifying Installed Routes

To display the installed routes, use the following commands:

ip route show proto bgp
ip -6 route show proto bgp
ip nexthop show

The status page and the HTTP API endpoint /api/kernel display the number of installed routes, the number of routes not installed and the reason, the last error reported by the kernel, and the interfaces. Table 2 lists the related metrics.

Metric	Description
bgp_kernel_routes	Number of routes installed by Ferrix
bgp_kernel_routes_unresolved	Number of prefixes whose active route is not installed because its next hop is unresolvable
bgp_kernel_routes_refused	Number of prefixes whose active route the kernel rejected
bgp_kernel_route_changes_total{change}	Number of route installations and removals requested
bgp_kernel_errors_total	Number of requests that the kernel rejected

Performance

In a virtual machine on an Apple M-series computer, with a release build, Ferrix installs 1 million IPv4 routes in slightly more than 1 second and removes them in approximately 0.5 seconds. Approximately three quarters of this time is spent in the kernel. Ferrix reduces its own share as follows:

• It sends hundreds of requests in each write to the netlink socket, and requests one acknowledgment for each write instead of each route.
• A dedicated task writes to the socket, so that the next batch is prepared while the kernel processes the previous one.
• It resolves each next hop once, not once for each route.