main
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
25597ab3ef |
packet, fib, bench: framing helpers, one netlink reader, a mesh with named phases
A code-quality pass over the periphery. The packet crate frames a message with `begin_message` and `finish_message`, which the exporter's hand-written UPDATE header can use instead of a bare type code, and keeps the NLRI reader and writer together as a pair; the rule that the body's prefixes are IPv4 unicast lives once; a session with two-octet AS numbers no longer reserves the attributes' wire copy it never keeps. The TCP diagnostics read netlink with the netlink module's own attribute walk and helpers instead of copies of them. The kernel follower learns what the kernel has in one named step used at start and on relearning, keeps its batch buffers between wakes, and builds its task in a constructor. The mesh benchmark, which had grown six phases in one main, is a `Mesh` with named phases and a `Receiver`, with one report of the sessions for both deadlines; the collector capabilities, the KEEPALIVE bytes, the draining of a session and the printing of a table row are helpers shared by the benches, the examples and the integration tests. The MRT reader uses the packet crate's attribute splitter. Testing: a message framed by hand is the one `to_wire` writes; a short tcp_info from an old kernel leaves the later fields zero; the synthetic table is the same every run and every announcement decodes to itself. The packing example gives the same figures on the RIS dump before and after. The crates' tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
99a47da2af |
status: a session's TCP connection as the kernel sees it; a route in full
When a session advertised a full table to a route collector at two thousand routes a second, nothing on the status page said whether the daemon or the peer set that pace. The answer is in the kernel: the peer's receive window, the queues, and how long the connection was held up by the window or by the send buffer, which `ss -ti` shows. This change asks the kernel the same way `ss` does, over the socket diagnostics netlink family (`NETLINK_SOCK_DIAG`, `SOCK_DIAG_BY_FAMILY` with the connection's addresses and ports), and reads `tcp_info` and the congestion control name from the answer. No external program is run and no unsafe code is needed: the request and the answer are byte layouts of the kernel's ABI, read field by field, with fields an older kernel does not send left at zero. A socket on a link-local address is found by its interface, which the request names. * `bgp_fib::sockdiag`: the request, the parser, `TcpConnection`, and `tcp_connection(local, peer)`; `Ok(None)` where there is no such connection or the system is not Linux. * `GET /api/tcp?peer=ADDRESS` in the HTTP API: the connection of the session with that peer, with times in milliseconds, the two windows named by what they allow, and the held-up times also as shares of the busy time. * The status page: a "tcp" link on each established session, which expands a row below it with the figures, re-asked on every refresh while open. The group rows' span is corrected for the "sent" column. Also on the status page: the attributes of a received route that have no column of their own were badges with their value in a tooltip, which is hard to read. Clicking the attrs cell now expands a row under the route with every attribute by its name (ORIGIN, AS_PATH, NEXT_HOP, MED, LOCAL_PREF, COMMUNITIES, LARGE_COMMUNITIES, AGGREGATOR and the rest), the path identifier, the selection rank and the RPKI and ASPA states; the badges stay. Both expansions use the terms of `ss` and the RFCs rather than sentences. Documentation: the endpoint in the HTTP API reference, and the link and the expansion in the monitoring guide. Testing: unit tests of the request and of the parser on a reply built field by field, and of a "no such socket" and a refused answer; a Linux-only test that asks the kernel about a loopback connection of the test's own and about one that does not exist; the lab's basics phase checks the endpoint for the FRR session; the page was checked by hand against the daemon in the Linux VM. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9e9e4f90ef |
fib: take best routes from the table 16384 at a time
The kernel task read changes of best route from the routing table 1024 at a time. A poll of the table does two binary searches and takes a read lock in every shard, however few events the shard has; with the 128 shards of an 8-core machine, that overhead was paid for every 8 events a shard, and the profile of a full-table load showed 8% to 9% of the daemon's CPU time in the reader's searches and locks. This change raises the batch to 16384. In the Linux VM, with four peers sending the synthetic table of 1.23 million prefixes, the kernel is in step 2.2 s after the first route instead of 2.9 s, and the daemon's CPU time over the load falls 13%; the reader's share drops from 9% to under 1%. With one peer the kernel-side share falls the same way and the CPU time 3%. A batch holds up to 16384 small records between polls, which is about a megabyte. Testing: the fib unit tests pass; measured with scripts/profile/full-table.sh with one and four peers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1bc71aaac5 |
fib, bench: profile the daemon taking in a full table; count blackholes instead of scanning
Before this change there was no way to see where the daemon's CPU time goes while it receives a full Internet routing table: the benchmarks in crates/bench time the routing table in-process, without the sessions, the policy, the kernel task or the production allocator. Profiling the daemon itself showed that a third of its CPU time during a full-table load went into `Plan::blackholes()`, which walked every installed route to count the blackhole ones each time the kernel task published its status, after every batch of changes; with 1.2 million routes installed that was 3.9 seconds from the first route to a synced kernel instead of 2.0. This change: * Keeps counts of blackhole routes and of routes kept unconfirmed from an earlier run in `Plan`, in step with the map through `Plan::put` and `Plan::take`, so that `Plan::blackholes()` and `Plan::kept()` cost nothing. `kept()` had the same scan; it ran only while routes of an earlier run were being kept, which is exactly during a full-table load after a restart. * Adds `scripts/profile/full-table.sh`, which builds the daemon with the new `profiling` Cargo profile (release with line tables) and frame pointers, runs it in a network namespace joined by a veth pair to a second namespace, feeds it the synthetic table from there as one or more neighbors, samples it with `perf`, and writes a flame graph (inferno), flat profiles by self and inclusive time, the time until the routes are in the table and in the kernel, and the daemon's resident memory. `PEERS`, `KERNEL`, `SCALE_DOWN`, `HOLD_SECS`, `FREQ` and `OUT` control it. From macOS it runs in the Linux VM through scripts/linux-vm.sh. * Adds the load generator `crates/bench/examples/feed.rs`: it connects from the peers' addresses, completes the handshakes, writes each neighbor's table, drains what the daemon sends back and holds the sessions open with KEEPALIVEs. `TestPeer::into_stream` gives it the connection after the handshake. * Updates the testing guide: the current `full_table` numbers, a "Profiling" section, and the daemon's numbers in the VM. Measured in the Linux VM (8 vCPUs) with the synthetic table of 1.23 million prefixes: one neighbor's table is in the routing table after 0.7 s and in the kernel after 2.0 s (before: 3.9 s); four neighbors' 4.9 million routes are in the table after 1.1 s and the kernel is synced after 3.0 s; 520 MB resident with one table, 680 MB with four. Without the kernel one session takes in 2.0 million routes per second on one core. A profile also attributed 14% of the session's time to `Journal::tidy`; inlining its early return changed nothing in an A/B run of the benchmark, so that was a sampling artefact and is left as it is. Testing: a unit test checks the counts against a scan of the entries through adopt, best, reset, synced, release, withdrawal, refusal and clear; the kernel test suite and the workspace tests pass; the script was run with one and four neighbors, with and without the kernel. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
6d8f89e509 |
policy, fib: blackhole routing for routes with the BLACKHOLE community
Before this change, a route carrying the BLACKHOLE community (65535:666,
RFC 7999) from an EBGP neighbor was always rejected with the reason
"blackhole request", and nothing discarded traffic. A customer under a
denial-of-service attack had no way to ask this router to drop traffic
to an address.
This change adds `accept_blackholes` to the import policy. When it is
set, a route with the BLACKHOLE community is accepted if:
* its prefix is a single address, a /32 or a /128 (otherwise it is
rejected with the new reason "blackhole for more than one address");
the `max_ipv4_prefix_length` and `max_ipv6_prefix_length` limits do
not apply, as RFC 7999 section 3.2 recommends;
* the address is inside a prefix the neighbor may announce, if
`prefixes` is configured, regardless of the entry's `max_length`
(new `PrefixSet::covers`); without a prefix list, an address inside
the local AS's own prefixes is rejected as before ("prefix is our
own");
* the AS path, origin and RPKI checks that apply to every route pass.
The next hop of an accepted blackhole route is not checked for
third-party addresses and is not resolved, because traffic is never
forwarded to it. The route takes part in best path selection as usual;
a host route is normally the active route for its address.
The kernel task installs the active route as a kernel blackhole route
(`RTN_BLACKHOLE`, shown by `ip route` as `blackhole PREFIX proto bgp`).
Blackhole routes left by an earlier run are adopted at startup like
other routes of ours. The route is advertised to other neighbors as the
export policy and export limits permit, with the community intact, so a
provider that honors BLACKHOLE can discard the traffic upstream.
Implementation:
* `plan::Target` becomes an enum: `Via { gateway, interface }` or
`Discard`. `plan::resolve` takes a `blackhole` flag and returns
`Discard` for such routes without looking at the next hop (the
own-network check still applies). No nexthop object is made for a
discard target.
* `AttrSet::is_blackhole` reads the community off the stored attribute
bytes, like `next_hop` and `as_path`, so the kernel task, the
revalidator and the RPKI revalidation can ask it of every route.
* `ImportPolicy::is_blackhole_request` and `check_blackhole_prefix`
carry the import side; the import flow (`routes.rs`) computes once per
UPDATE whether its routes are an accepted blackhole request and skips
the next hop checks for them.
* `Status::blackholes` and the `blackholes` field of `/api/kernel` count
the installed blackhole routes; the status page shows the count next
to the installed routes.
Not covered: a prefix this router originates with the BLACKHOLE
community is advertised but not installed as a blackhole route. The
documentation says to add a kernel blackhole route by hand for that
case.
Documentation: the "Blackhole Routes" section of the import policy
guide replaces "Blackhole Requests", the kernel guide gains a "Blackhole
Routes" section, the configuration reference describes
`accept_blackholes`, the HTTP API reference describes `blackholes`, and
the RFC 7999 rows of the standards reference are updated.
Testing: unit tests for the attribute check, the prefix checks and the
plan; integration tests that a blackhole request is refused by default
and, with `accept_blackholes`, taken for one address inside the peer's
space and refused otherwise, with the recorded reasons; a kernel test
that a blackhole request becomes a kernel blackhole route whatever its
next hop, is removed on withdrawal and is adopted after a restart. The
existing kernel tests, the interoperability lab and the workspace tests
pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3608897245 |
fib: resolve next hops through the kernel's routing table
Before this change, Ferrix only installed a route if its BGP next hop was
directly connected: an address in the subnet of an interface that is up.
A route with any other next hop was held at import and never installed.
This made routes from iBGP neighbors unusable unless the neighbor set
itself as the next hop, and it is why the route reflector rewrites the
next hop on every reflected route.
Ferrix now resolves a next hop that is not directly connected through the
kernel's own routes, as other BGP implementations do:
* The route is looked up in one kernel table, the "resolution table"
(`kernel { resolution_table: N }`, default 254 = main). The longest
prefix that contains the next hop wins; of several routes for that
prefix, the one with the lowest metric.
* Because the Linux kernel does not resolve gateways recursively, the BGP
route is installed with the gateway and interface of the kernel route
that covers the next hop. A kernel route without a gateway (a route
onto a point-to-point or tunnel interface) makes the next hop itself
the gateway on that interface. A kernel route that names a nexthop
object is resolved through the object.
* Routes with protocol `bgp` never resolve a next hop, whether Ferrix's
own or another daemon's. A next hop that is reachable only through a
BGP route depends on another BGP next hop, and such chains are not
predictable.
* The default route does not resolve a next hop unless
`kernel { resolve_via_default_route: true }` is set. On a router the
default route points at a transit provider, and a next hop that is
reachable only through it is not usable.
* Blackhole, unreachable and prohibit routes make the next hop
unresolvable, as does a route whose interface is down.
The same resolution is used by the import policy and the revalidator, so
a route whose next hop cannot be resolved is still held rather than
selected (RFC 4271 section 9.1.2.1), and is taken when a kernel route
makes its next hop resolvable.
Implementation:
* `bgp_fib::routes::KernelRoutes` holds the routes of the resolution
table and answers longest-prefix lookups.
* `bgp_fib::topology::Topology` bundles the interfaces, the kernel routes
and the kernel's nexthop objects into one value. The kernel task
publishes it through `Fib::topology()` (formerly `Fib::interfaces()`),
and `plan::resolve` takes it in place of `Interfaces`.
* The kernel task learns the resolution table with a dump of that table
alone, using NETLINK_GET_STRICT_CHK (Linux 4.20 and later; older
kernels get a dump of every table, filtered in user space). It asks
again, debounced, whenever a dedicated netlink socket that is a member
of the route multicast groups receives anything. That socket is given a
64 KiB receive buffer on purpose: the kernel reports Ferrix's own route
installs on it too, up to a million at a time, and an ENOBUFS overflow
is treated as the same hint as a message. Its contents are never
parsed. An earlier attempt to receive route notifications on the
socket that writes routes failed, because the kernel only excludes the
requesting socket from a notification when NLM_F_ECHO was requested.
* `Unresolved::NotConnected` is replaced by `NoRoute`, `OnlyDefault` and
`Discarded`, which appear as the reasons in `/api/kernel` and on the
status page.
Configuration: two new `kernel` fields, `resolution_table` and
`resolve_via_default_route`, documented in the configuration reference
and in the kernel guide, whose "Next Hop Resolution" section is rewritten.
Existing configurations behave as before except that routes with far
next hops may now be installed where a static or connected route covers
the next hop.
Testing: unit tests for `KernelRoutes`, `Topology` and resolution
through static, tunnel, nexthop-object, blackhole and default routes; a
kernel test (`a_far_next_hop_goes_the_way_the_kernels_routes_say`) that
checks the installed routes follow static, tunnel and nexthop-object
routes as they are added, replaced and removed; the existing kernel
tests, the interoperability lab with FRR and BIRD, and the workspace
tests, all passing. Installing 200,000 routes still runs at about
186,000 routes per second.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
23e8b4aadb |
One word for a route that is not taken: rejected
The API, the metrics, the status page and the documentation say a route is "rejected" (/api/routes/rejected, rejected_routes, bgp_routes_rejected_total, "Rejected routes"), and the reason type is Rejection. The code that keeps the record said "refused": RefusalLog, PeerRefusals, RefusedRoute, RefusalsView, RefusedRouteView, the refusals module. Those are RejectionLog, PeerRejections, RejectedRoute, RejectionsView, RejectedRouteView and rejections now, and the log line says "Rejected a route". Nothing the API or the metrics expose changes. Refusing stays what it was: a connection turned away at the door (server::Refusal), a kernel that would not take a route, an OPEN we would not have. The two aliases of SendReceive, AddPathSendRecv and OrfSendRecv, are gone; it is one type. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
724bb99f11 |
Kernel: read what the table has before letting kept routes go
Once every peer had sent all its routes, the kernel task let go of the routes kept from the earlier run that the table did not have, at once. But "the table has" and "the kernel task has read" are not the same moment: a route a peer sent just before its End-of-RIB could still be unread in the table's journal, in which case it was taken out of the kernel and put back a moment later. The interop lab saw this now and then as "N routes were taken out of the kernel over the restart". The task now reads the table to the end before it decides what is unconfirmed; follow() says whether the reader is caught up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
b467b136e4 |
Kernel and RPKI: say what went wrong, and once
The RPKI client had one Timeout error whose text said "did not answer within 60s" whatever had run out: the connect timeout, the time for a whole answer, or the data's lifetime. These are what /api/rpki shows as the last error; each is its own now. The protocol version shown is the one the cache answered in, not the one being tried. The VRP set, hashed by the hundred thousand with every answer, uses the quick hash the ASPA table already did. In the kernel installer the writer task's error was worked out in three places, and the next-hop resolution checked for our own networks twice on every cache miss. A refusal was logged twice, once with the status lock held. The interfaces are listed in order of index by the structure that holds them, from a function that can be tested on any OS, and is. EIO is with the other errnos; Plan has no installed_prefixes (no callers) and clears its records in one place, with the order of Unresolved::ALL asserted at compile time since indexing depends on it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
b16227c51f |
Route reflection, with ourselves as next hop
A route learned from one router of our AS went to no other (RFC 4271 §9.2.1), so every router needed a session with every other. A router at the end of a single link, with one session to its neighbour inside the AS, heard only of what that neighbour learned itself, and nothing of what the neighbour had from the routers behind it. A peer in our own AS can now be made a route reflector client (`route_reflector_client`, RFC 4456). What a client sends goes to all our routers; what the others send goes to the clients. A reflected route gets ORIGINATOR_ID (where it started, if it does not say) and our cluster ID at the front of CLUSTER_LIST; it is not sent back to the router it started at, nor round to a cluster it has been through, and one that comes back anyway is refused as "route reflected back to us". The cluster ID is the router ID unless `cluster_id` says otherwise. The two attributes are now decoded as such rather than kept as bytes, and are never sent to other networks. Unlike a standard reflector, this one is the next hop of what it reflects, as it is of everything it announces: there is no resolving a next hop that is not a neighbour, so a client reaches what lies behind another router through the reflector. Only peers in our own AS can be clients; the configuration refuses the setting for any other. Nothing changes in what other networks are told: Only to Customer marks go along with reflected routes, and every receiving session keeps its own max_prefixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
65f136048e |
Graceful restart (RFC 4724, RFC 8538)
A peer that restarts keeps its routes with us, and we keep ours with our peers while we restart, so that a restart withdraws nothing across the network and the machine goes on forwarding. On by default for every peer; `graceful_restart` sets the restart time we give, caps how long a peer's routes are kept, or turns it off. When a peer restarts: its routes are marked stale in the routing table, and chosen as before, for at most its restart time. When it is back, what it sends again is fresh, and the stale routes of a family go at its End-of-RIB marker, or after the cap; at once for a family whose forwarding it did not keep. Only a connection that failed counts as a restart, and, with RFC 8538 agreed, an expired hold timer or a NOTIFICATION other than a Hard Reset. The stale mark is a bit of the route's word that selection never looks at. When we restart: SIGUSR1 leaves the kernel's routes where they are, closes the sessions without a word where graceful restart is agreed, and exits with status 75, on which the systemd unit starts us again. The next start finds the routes in the kernel, says so in its OPENs (R, and F where forwarding was kept), and has them go once every peer has sent its End-of-RIB. Our own End-of-RIB markers for learned routes wait for that too. SIGTERM still stops for good, now with a Hard Reset where RFC 8538 is agreed, as are errors of ours. The status page and API show what was agreed with each peer, the routes kept while a peer restarts, and which routes are stale. Tested against FRR and BIRD in a new phase of the interoperability lab, and with routes kept in a real kernel over a restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f1d80c66df |
Do not choose a route whose next hop cannot be reached (RFC 4271 §9.1.2.1)
Where routes go into the kernel, a route the kernel could not be given is no longer taken into the table: its next hop is no neighbor on any interface, or its prefix is a network the machine is itself on. Such a route used to be able to win selection and then leave its prefix without a route in the kernel, though another peer had one that worked. It is held aside the way an RPKI-invalid route is, and by the same means: judged on arrival against the interfaces as the kernel last described them, looked for in the table when they change, and taken from what is held when its next hop can be reached. The fib publishes the interfaces for this; the next hop is resolved once for an UPDATE and not for each of its routes. The lab has a phase, nexthops: FRR's better route with a next hop that is nowhere is held and BIRD's is installed, and they change places as an address comes and goes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3d946c32ce |
Code health after review: docs, metrics, structure, one more fuzz target
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9ca86ede2f |
Harden import, the kernel task and the RTR client after review
Three independent reviews (hijack resistance, reliability, code health) of the whole tree. This is what they found that could let a bad route in or take the daemon down. Hijacks: - A route had two paths on a two-octet session, AS_PATH and AS4_PATH, read in two places by two rules. A peer could make the origin checked on import differ from the one checked at revalidation, and get an RPKI-invalid route into the table. AS4_PATH is now merged into AS_PATH once, on receipt (RFC 6793 §4.2.3), and nothing else knows of it. - The next hop is judged on import: it must be usable, agreed on (RFC 8950), and by default the peer itself. - Our own originated prefixes are not taken from other networks. - A route for a network the machine is on is never put in the kernel. - Paths with an AS_SET are refused (RFC 9774). - A configured peer gets one session, not one per BGP Identifier. - More IPv6 special-purpose prefixes are bogons. Reliability: - A batch of requests the kernel refused could overflow the netlink socket and end the daemon. The socket has room now, a lost answer or a silent kernel means asking what is there, and neither is fatal. - Working the routes out again within the grace period after a restart no longer drops the routes being kept. - That work can no longer be put off for ever by interface changes. - Peers whose sessions keep failing are slowed down. - Unused attribute sets are swept when they pile up, not only each minute. - Log lines a peer can cause are said once in a while. - An RTR answer is bounded in size and time; data cannot expire between two refreshes. - The shard of a key is taken from a stirred hash: neighbouring bits of the hash proved alike for prefixes in a row. The test that showed it, failing one run in sixteen, is deterministic now. The lab has a phase, hijacks, in which FRR tries the first three. route::apply is a struct with methods instead of nested closures, and the kernel task's loop is shorter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
fd2248feb5 |
Peer at link-local addresses: fe80::1%eth0
A peer's endpoint may be a link-local IPv6 address with the interface it is on. Such a peer is the address and the interface together, wherever peers are told apart: admission, collisions, hold-downs, the refusal log, the status page. The interface is looked up by name each time it is needed, so one that appears later or is made anew is found. Our routes go out with our link-local address in both places of MP_REACH_NLRI's next hop, as FRR sends its own. A peer's link-local next hop is taken alone, twice, or after "::" (BIRD), and resolved in the kernel on the interface of the session rather than by looking fe80::/64 up, which every interface has. The lab has a new phase, linklocal: sessions with FRR (by interface, with radvd beside the daemon) and BIRD over a LAN with link-local addresses only, both families, pings across, the interface down and up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
92e013ca01 |
Carry IPv4 routes over IPv6 sessions with IPv6 next hops (RFC 8950)
For a link that has only IPv6 addresses. - The Extended Next Hop capability is offered on every session over IPv6 whose families include IPv4 unicast, and used if the peer offers it too. Nothing to configure. - On such a session our IPv4 routes go out in MP_REACH_NLRI with the session's IPv6 address as next hop, and are withdrawn in MP_UNREACH_NLRI. Without the agreement they are not sent, as before. - IPv4 routes are read with an IPv6 next hop, global or global with link-local. - The kernel installer resolves an IPv4 route's IPv6 next hop like any other and installs it: by way of an IPv6 nexthop object, or with RTA_VIA where routes carry their own gateway. Tested against a real kernel in both modes, with a link-local next hop too, and in the lab against FRR and BIRD: the new `v6only` phase takes the IPv4 addresses off the exchange LAN, runs one IPv6 session with each router for both families, and pings IPv4 across. Sessions between link-local addresses are not supported yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
321af51e4b |
Add an interoperability lab with FRR, BIRD and Routinator
scripts/interop/lab.sh runs the daemon against FRR, BIRD and Routinator, each in a network namespace on one exchange LAN, and checks what every side ends up with: sessions and routes over IPv4 and IPv6 with their attributes, four-octet AS numbers, ADD-PATH, RFC 9234 roles and Only to Customer, RTR version 2 and invalid routes, route refresh, session resets, killed peers, a cut wire, the daemon stopped and killed, a table of a million routes and a restart under it, role mismatch and the prefix limit. All of it passes against FRR 10.5.1, BIRD 2.18 and Routinator 0.15.2. scripts/interop/real-rpki.sh checks the RTR client against a Routinator with the real RPKI: 713,258 IPv4 and 187,479 IPv6 VRPs and 3,379 ASPAs, the same numbers on both sides. What the lab found, fixed here: - A daemon restarted with its routes in the kernel removed them all at once, its table being empty until the peers had sent theirs again, and then put them back: no forwarding meanwhile. Routes of an earlier run are now kept for `stale_seconds` (120 by default) and only those nobody announced again are removed. A restart under a million routes takes 1.1 s to be in step, with nothing removed. - FRR ends a session it loses to a collision with a state machine error without a subcode, which was shown as "error code 5 subcode 0". CI gets a job that runs the lab. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1de75fc610 |
Install a full table in the kernel in about a second
A million IPv4 routes took 1.6 s to install (release build); they take 1.1 s now, and 0.5 s to remove instead of 1.15 s. Most of what is left is the kernel's own work on each route. - Nexthop objects (Linux 5.3 and later): a gateway is put in the kernel once and routes name it, so the kernel checks it once and not for every route. The daemon makes its own objects, uses nobody else's, is told when one of its own is removed and puts it back, and removes them when it stops, which takes all the routes that name them along at once. `without_nexthop_objects` turns this off; a kernel without them is noticed and served as before. - Writing to the kernel is a task of its own, fed through a short queue, so the next batch is worked out while the kernel is busy with the last. "In step" is said when the kernel has done the batch that completes the table, not when it was handed over. - What a next hop resolves to is kept, by next hop: it was looked up among the machine's addresses for every route. - A quicker hash for the plan's maps, and no lookups in the ones that are empty. Fixes a hole found on the way: when an interface went down and came back within the second it takes us to react, the kernel had dropped our routes and we did not put them back, since nothing about them had changed for us. After such a change the kernel is now asked what it actually has. scripts/in-namespace.sh runs one command in a scratch namespace; FERRIX_ROUTES sets the size of the timing test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1fc44798b7 |
Install best routes in the Linux kernel over netlink
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|