The routing table's shards started empty and grew as routes came in, each growth moving every row of the shard into a table twice the size. In a profile of the daemon taking in a full table, the growth showed as up to a seventh of the session's time, and the daemon's resident memory had the old and the new table of each shard side by side for a moment. This change adds `RibTable::expect_prefixes`, which reserves room for that many prefixes spread over the shards, and has the server call it at startup with the largest `expected_routes` among the configured peers: the prefixes of several full tables are mostly the same ones. Without `expected_routes` nothing changes. The `full_table` benchmark does the same unless `BENCH_PRESIZE=0`. Measured, the saving in time is within the run-to-run noise: in the Linux VM one peer's table is in the routing table in 0.81 s instead of 0.89 s, and the kernel is in step at about the same time as before; three pairs of runs of the benchmark on a Mac average 0.33 s against 0.35 s. The profile's attribution to the growth was mostly sampling skid. The memory saving is real: 442 MiB resident with one peer's table in instead of 483 MiB, and 626 MiB instead of 632 MiB with four. Also: the profiling script printed nothing for the kernel's change counts because it asked `/metrics` for them; the endpoint is `/metricz`. With that fixed, a four-peer load shows 1.25 million installs for 1.23 million prefixes: the journal already gives the kernel task only the latest change of a prefix when it lags, so no further coalescing of installs is called for. Documentation: `expected_routes` in the configuration reference, the neighbor guide and the proto comment say that the table is sized by it; the testing guide's VM results are brought up to date. Testing: a table test checks that the room is there and is kept; the workspace tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
11 KiB
+++ title = "BGP Neighbors" description = "Describes how to configure BGP neighbors, address families, link-local neighbors, ADD-PATH, and session timers, and how Ferrix filters inbound connections." weight = 1
[extra] nav = "BGP Neighbors" +++
A BGP neighbor (peer) is a BGP speaker with which Ferrix establishes a
session over TCP. Each neighbor is defined by a peers statement in the
configuration.
Configuring a Neighbor
The following example configures an external BGP (EBGP) neighbor:
peers {
name: "transit-a"
endpoint { address: "203.0.113.1" }
remote_asn: 64496
local_address: "203.0.113.2"
hold_time_seconds: 90
role: PROVIDER
import { max_prefixes: 1500000 }
}
Table 1 describes the statements in the example.
| Statement | Description |
|---|---|
name |
Name of the neighbor in logs, metrics, and the status page. |
endpoint |
Address and TCP port of the neighbor. The default port is 179. |
remote_asn |
AS number of the neighbor. If it equals local_asn, the neighbor is an internal BGP (IBGP) neighbor. |
local_address |
Source address for connections to the neighbor, and the next hop of routes advertised to it. |
hold_time_seconds |
Hold time proposed in the OPEN message. |
role |
BGP role of the neighbor (RFC 9234). See BGP Roles. |
import |
Import policy for routes received from the neighbor. See Import Policy. |
Configure local_address when the router has several addresses and the
neighbor expects a specific one.
Session Establishment
Ferrix initiates a TCP connection to each configured neighbor and also accepts connections from it. If both connections succeed, Ferrix resolves the collision as specified in RFC 4271 Section 6.8: the connection initiated by the speaker with the higher BGP identifier is retained.
If a connection attempt fails, Ferrix retries after connect_retry_seconds
(default: 5 seconds). The interval doubles after each failure, up to a
maximum of 2 minutes.
Ferrix rejects a neighbor whose OPEN message contains an AS number other
than remote_asn, with the NOTIFICATION message OPEN Message Error / Bad
Peer AS.
Passive Neighbors
A passive neighbor is a neighbor to which Ferrix does not initiate connections. Ferrix accepts connections from the neighbor's address and establishes the session with the configured settings when the neighbor connects. Use a passive neighbor to configure a session before the neighbor's side is ready, or when the neighbor initiates the connections:
peers {
name: "new-peer"
endpoint { address: "192.0.2.10" }
remote_asn: 64510
role: PEER
passive: true
}
Until the neighbor connects, the status page and /api/peers show the
state Passive. After a session ends, they show the reason in the
neighbor's last error. Ferrix does not log connection attempts to a
passive neighbor, because it makes none.
Inbound Connection Filtering
Ferrix accepts inbound connections only from the addresses of configured
neighbors, and limits the number of concurrent connections from each
address. Ferrix closes any other connection before reading from it and
increments the bgp_connections_refused_total metric.
Ferrix treats each configured neighbor as a single router. A second connection from the same address with a different BGP identifier is rejected. This prevents a neighbor from exceeding its prefix limit by establishing additional sessions.
Caution
The
accept_unconfigured_peersstatement accepts sessions from any address, with any AS number, and accepts their routes. Use it only for testing on an isolated network.
Listening Addresses
By default, Ferrix listens on 0.0.0.0:179 and [::]:179. To listen on
specific addresses, configure listen_endpoints:
listen_endpoints { address: "192.0.2.1" }
listen_endpoints { address: "2001:db8::1" }
Address Families
Ferrix supports the IPv4 unicast and IPv6 unicast address families. Each speaker lists the families that it supports in its OPEN message, and a family is enabled on the session only if both speakers list it (RFC 4760). By default, Ferrix proposes both families. To propose only IPv6 unicast:
address_families: IPV6_UNICAST
If the neighbor supports none of the proposed families, Ferrix rejects the session.
IPv4 Routes with IPv6 Next Hops
Ferrix supports the advertisement of IPv4 routes with IPv6 next hops over a session between IPv6 addresses (RFC 8950). On every IPv6 session that carries IPv4 unicast, Ferrix proposes the Extended Next Hop capability; no configuration is required. On the neighbor, enable the capability as follows:
- FRR:
neighbor ... capability extended-nexthop - BIRD:
extended next hop onin the IPv4 channel
When the capability is negotiated, Ferrix advertises IPv4 routes with the
session's local IPv6 address as the next hop. It accepts IPv4 routes with an
IPv6 next hop, or with a global and a link-local next hop. The kernel
installs such routes through the IPv6 neighbor, for example
10.2.0.0/24 via inet6 2001:db8::2 dev eth0. This requires Linux 5.2 or
later. The status page displays "IPv4 with IPv6 next hops" for the session.
Link-Local Neighbors
Ferrix can establish a session with a neighbor at an IPv6 link-local
address (fe80::/10). Because a link-local address is significant only on
one link, you specify the interface with the address:
peers {
name: "upstream"
endpoint { address: "fe80::1%eno3" }
remote_asn: 64496
}
Link-local neighbors have the following behavior:
- Ferrix resolves the interface name to an interface index each time it is needed. If the interface does not exist, the neighbor state displays that the interface is missing; the session is established when the interface is created. An interface that is recreated with a new index, such as a restarted tunnel, is found automatically.
- The same link-local address on two interfaces identifies two neighbors. Ferrix rejects a link-local address without an interface, and an interface specified with a non-link-local address.
- To accept connections from link-local neighbors, Ferrix must listen on
::, which is the default. - Ferrix advertises routes with its link-local address as the next hop, in
both next hop fields of MP_REACH_NLRI. It accepts a link-local next hop in
any of the encodings in use: the link-local address alone, the link-local
address twice, or the link-local address after
::. - The kernel installs the routes through the link-local address on the
session's interface, for example
10.2.0.0/24 via inet6 fe80::1 dev eno3. - If
local_addressis a link-local address, Ferrix uses it on the neighbor's interface.
Tip
On FRR, configure the neighbor by interface (
neighbor eno3 interface remote-as ...). FRR discovers the neighbor's address from IPv6 router advertisements, which Ferrix does not send; runradvdon the interface. With theneighbor fe80::1 interface eno3form, FRR 10.5 establishes the session but marks the received routes as inaccessible. On BIRD, configureneighbor fe80::1%eno3 as ....
ADD-PATH
By default, a neighbor advertises one path for each prefix. With ADD-PATH (RFC 7911), a neighbor can advertise multiple paths for the same prefix. To accept multiple paths from a neighbor:
add_path: ADD_PATH_RECEIVE
ADD-PATH is enabled for each address family and direction only if the
neighbor also supports it. Ferrix negotiates ADD_PATH_SEND and
ADD_PATH_BOTH, but it advertises only one path for each prefix.
Hold Timer
hold_time_seconds specifies the hold time that Ferrix proposes in its
OPEN message. The session uses the lower of the two proposed values. If no
message is received from the neighbor within the hold time, Ferrix tears
down the session.
Range: 0, or 3 through 65535 seconds. A value of 0 disables keepalive messages and the hold timer. Default: 90 seconds.
Ferrix also implements the send hold timer (RFC 9687). If Ferrix cannot write to the neighbor for 8 minutes or twice the hold time, whichever is longer, it tears down the session. This prevents a neighbor that does not read from the connection from blocking route advertisement indefinitely.
Session Flap Damping
If sessions with a neighbor are established and then fail within 1 minute repeatedly, Ferrix places the neighbor in hold-down. The hold-down begins at the third such failure with a duration of 1 second, and doubles at each further failure, up to a maximum of 5 minutes. During hold-down, Ferrix does not establish a session with the neighbor.
The end of a session requires Ferrix to remove the neighbor's routes from the routing table. Hold-down limits the processing that a flapping neighbor causes.
A neighbor that exceeds its prefix limit is placed in hold-down for
hold_down_seconds. See Prefix Limits.
Administrative Shutdown
When the daemon receives SIGINT or SIGTERM, it sends a Cease NOTIFICATION
message with the subcode Administrative Shutdown to every neighbor. If the
kernel statement is configured, it then removes its routes from the
kernel. See Kernel Route Installation.
Expected Route Counts
An established session with no routes, or with a small fraction of the
routes that the neighbor normally advertises, indicates a problem that the
session state does not show: a filter on the neighbor's side, an import
policy that rejects everything, or a neighbor that has not finished
sending its table. The expected_routes statement specifies the
approximate number of routes that the neighbor advertises in normal
operation:
peers {
name: "transit-a"
endpoint { address: "192.0.2.1" }
remote_asn: 64496
role: PROVIDER
expected_routes: 1000000
import { max_prefixes: 1200000 }
}
The status page compares the number of routes accepted from the neighbor
with this value. An established session with fewer than half the expected
routes is listed under Needing attention, with the expected number
beside the actual one. The comparison starts one minute after the session
is established, so that a neighbor that is still sending its table is not
reported. The value does not affect which routes are accepted;
max_prefixes is the limit that Ferrix enforces. Ferrix also sizes the
routing table at startup for the largest expected_routes value among
the neighbors, so that the table does not grow while the tables are
received.
Verifying BGP Sessions
To verify the state of each neighbor, use one of the following:
- The status page,
/statusz, lists each neighbor with its session state, address families, role, and number of routes. - The HTTP API,
/api/peers. See HTTP API Reference. - The metric
bgp_peer_up{peer}, which is 1 while the session is established and 0 otherwise.