db8edaae8e5f98c0ddb90714d9a03d382770f3da
24
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
4cddd6fb04 |
export: one rule each for admission, identifiers and what the peer had
A code-quality pass over the exporter. The resync path (gather then stream) had grown beside the steady-state path (judge then flush) with its own copies of the admission predicate, the path identifier rule, the reconciliation of a route the peer already had, the parser contexts, the withdrawal loop and the advertised gauge. Each is one function now: `prefix_may_go`, `lowest_free_id`, `Sent::renewed`, `parser_contexts`, `send_withdrawals`, `within_limit` and `note_held`. The 3,072-octet ceiling on a route's attributes, which is what keeps an UPDATE within its size, is a named constant where it is relied on. The UPDATEs are framed with the packet crate's helpers instead of a bare type code and a length written by hand. The positional tuples for a prefix's paths are structs with names. A mismatched kind of bookkeeping, which nothing should produce, is asserted in tests and logged in release instead of dropped silently. Types used only inside the crate are `pub(crate)`. Five doc comments that had drifted onto neighbours or named former designs are put right; the exporter's reference encoder no longer breaks the CI doc build with a link to an item that exists only under test. Testing: ADD-PATH export has unit tests at last: two routes for a prefix go with their own identifiers and keep them, and the withdrawal of one carries its identifier; what is not heard of again after a reset is withdrawn after the table. The byte-for-byte test of the encoded UPDATEs covers IPv4 routes with an IPv6 next hop (RFC 8950) too. The test fixtures are shared helpers. The server tests and the ADD-PATH, transit, limits, roles and deadlock integration tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
b5ca42244d |
export: a lighter gather, and UPDATEs to the session by the piece
Two things kept an exporter from its peer. A table being gathered to go out anew kept a map of every prefix heard, to let a later word on a prefix supersede an earlier one, and dressed every set of attributes before the first UPDATE went: a map insert and a parse-and-encode a route that the peer waited on. And the session's queue took UPDATEs one by one, a thousand at most, so the exporter stood at the queue once a hundred kilobytes were in it, behind a writer that takes a megabyte: the two took turns instead of running together. The journal gives a table out shard by shard, each prefix once, and a change meanwhile comes after the table as news of its own, so there is no second word to keep track of while gathering: the map is gone, and a prefix the peer is not to have is simply not gathered, as what it held of it is withdrawn once the table is through. Each set is dressed as its turn to go comes, so the first UPDATE goes once the table is gathered; only when the gathered routes alone would be over the peer's limit are the sets dressed first, to know which cannot be and take their routes with them, so that none goes if too many would. The UPDATEs for a set now go to the session in pieces of 64 KiB or so, each of whole messages, and the queue takes 64 of those: a few megabytes ahead of the writer, not a hundred kilobytes. The session counts the messages in a piece by their headers. Testing: the byte-for-byte test of the encoded UPDATEs now encodes thirty thousand routes and checks the pieces are of whole messages and of the size meant. The server tests, the integration tests, the kernel tests and every lab phase pass. In the Linux VM, three rounds alternating with the previous commit, a new peer's first UPDATE comes 1.8 s after its session is up instead of 3.1 s, and it has its four million routes in 3.8 s instead of 4.1 s; the smaller of the two routers' tables crosses in 3.0 s instead of 4.2 s and the larger in 5.0 s instead of 5.2 s, at the same CPU time. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
381eabd886 |
export: encode a route's attributes once a session
The exporter built each UPDATE as a message: the attributes of the route's set, parsed from the table, were cloned, had the next hop added, were sorted, and were handed to the session, which encoded them to the wire with the rest of the message. For a table whose attribute sets each cover a few routes that is an encoding of the same octets for nearly every route, and a copy of the message's bytes on the way to the writer besides. A profile of a production router taking the other router's table with ADD-PATH had the exporters at 55% of the CPU time, their dressing of routes at 23%, and memcpy at 12%. A route's attributes are now encoded once for a session, when it is first dressed for it, and kept with where in the octets a NEXT_HOP or an MP_REACH_NLRI belongs so that the attributes stay in order of type code. An UPDATE is then its header, the encoded octets with the next hop or the MP_REACH_NLRI spliced in, and the prefixes: written straight into a buffer that goes to the session as it is, and from the session to the writer without a copy. The session's queue takes either a message or an encoded UPDATE; an encoded one for a session that is no longer established is dropped, as the exporter made it for the session as it was. What is kept for a route, now a few hundred octets instead of a kilobyte, lets twice as many be kept. The exporter's maps hash with foldhash instead of SipHash, and the parser context for an UPDATE is made once a flush instead of once an UPDATE. Testing: a new test encodes a thousand routes with each of the forms an UPDATE takes (IPv4 with the next hop in its own attribute, IPv6 in MP_REACH_NLRI, with and without path identifiers, to two-octet and four-octet AS speakers, over several messages) both ways and checks the octets are the same byte for byte; the message-building form is kept for the test. The exporter tests decode what the session is handed. The server tests, the integration tests, the kernel tests and every lab phase pass. On the mesh benchmark in the Linux VM, three rounds each alternating with the previous commit, the two routers' exchange of 7.4 million routes takes 20.6 s of CPU time instead of 25.4 s (-19%) over 9.0 s of wall-clock time instead of 11.1 s, and the larger of the two tables is across in 7.1 s instead of 9.3 s. The smaller table's time varies too much from run to run (5-8 s) for a difference to be read from it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
d7d7d39bb9 |
metrics: export each session's TCP connection figures
The kernel's view of a session's TCP connection, which the status page shows on request since the last change, could not be looked at after the fact: a feed that crawled overnight left no record of whether the peer's window or our own side held it up. This change samples it at every scrape of `/metricz`: for each established session, one `NETLINK_SOCK_DIAG` round trip, and twenty gauges labelled by peer, `bgp_peer_tcp_*`: smoothed, least and variance of the round-trip time; the congestion window and segment size; the peer's receive window and ours; the send queue, the part of it not yet sent, the receive queue and the segments in flight; retransmitted and lost segments; the busy time and the time held up by the peer's window and by our send buffer, cumulative, so that their rates give the share of time a session is held up; octets sent and received; and the delivery and pacing rates. A session the kernel has no connection for loses its series, as does one that ends. `SessionHandle::metric_label` is the one place a peer's metric label is made (its name, or its address), used by the advertiser's gauge and here. Documentation: the metrics reference lists them; the monitoring guide gives the query for the share of time a session is held up by the neighbor's receive window, which is the question the overnight feed raised. Testing: a daemon in the Linux VM with one session shows the series and drops them when the session ends; the lab's basics phase checks the scrape for the FRR session's round-trip time; the workspace tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
ec5c9be454 |
export: advertise every route for a prefix with ADD-PATH (all_paths)
Before this change, a session on which ADD-PATH (RFC 7911) was
negotiated in the sending direction was sent path identifiers, but only
the best route for each prefix. A route collector such as bgp.tools
wants every route a router has accepted, from every neighbor, and from
one session: without this the collector saw one path per prefix, nearly
always the same neighbor's.
This change adds `all_paths` to the export policy. On a session with it,
and ADD-PATH send negotiated for a family, every accepted route for a
prefix of that family is advertised, each with a path identifier of its
own; for a family without ADD-PATH send the best route goes as before.
`all_paths` needs `learned_routes` (a configuration error otherwise) and
the peer's `add_path` to offer sending. The export policy, the role
rules, the well-known communities and the export limits apply to every
route alike, and the export limits count every route.
Implementation, in the advertiser:
* An exporter with `all_paths` follows the table's journal of all routes
(`RibTable::reader_of_all_routes`, until now unused) instead of the
best-route journal, and is given all the routes of a prefix whenever
one of them changes (`News::Routes`).
* What the peer holds is `Announced`: the set of prefixes for a
best-route session, as before; or for an all-paths session, for each
prefix the routes announced, keyed by the peer they came from and that
peer's own identifier, with the identifier they went out with and the
form they went out in. A route keeps its identifier for as long as it
is announced, goes out again only if it changed, and is withdrawn by
its identifier; a new route gets the lowest identifier not in use for
the prefix, from 1.
* Verdicts carry either kind (`Verdict::Best`, `Verdict::All`), and a
reset of the table puts what was announced in doubt until heard of
again, as before, with the same identifiers kept across the reset.
* The record for an all-paths session is about 50 bytes a route, some
tens of megabytes for a full table from several neighbors; the
best-route bookkeeping is unchanged in size.
Also, so that a collector session can be watched: the number of routes
advertised to each peer and not withdrawn since, every route counted
where several are sent for a prefix. The exporter keeps it in the
session's counters after every change; it is `advertised_routes` in
`/api/peers`, the "sent" column of the status page's neighbor table,
and the gauge `bgp_peer_advertised_routes{peer}`.
Documentation: a "Feeding a Route Collector" section in the guide to
passing routes on, with the configuration for a collector; `all_paths`
in the configuration reference; the ADD-PATH sections of the neighbor
guide and the standards reference; the limitation that was removed;
the new field, column and metric in the HTTP API, monitoring and
metrics references.
Testing: an integration test with two feeders and a collector that
negotiated ADD-PATH receive, which checks the two routes with their
identifiers, the withdrawal of one by its identifier, a changed route
keeping its identifier and a new route taking the free one; a
configuration test for `all_paths` without `learned_routes`; a new
`collector` phase of the interoperability lab in which FRR, as a route
collector, is told both of BIRD's routes for a prefix from BIRD's two
sessions and holds them apart, and loses one when its session is
disabled, with the advertised count checked at each step. The workspace tests, the kernel tests and every lab phase pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
277b973f1a |
Status page: what moved, what is missing, and a dark room
The attention strip's names lead to the peer's row, which lights up for a moment. The IPv4 and IPv6 sessions of one neighbour (NAME-v4, NAME-v6) are listed together, joined down the side, the name said once in full. A route count that moved by more than 2% between refreshes carries ▲ or ▼ for two minutes; a row whose state or count changed while the tab was in the background carries ● until it is clicked. A compact mode drops the next hop and session columns, and is remembered by the browser. Colours are variables, with a second set for prefers-color-scheme: dark. A new per-peer `expected_routes` says about how many routes a peer sends when all is well. The daemon only passes it to the API; the page lists an established peer with under half of it as needing attention, a minute after the session came up, with the expected count beside the actual one. An established session with next to no routes looked, until now, exactly like one that was fine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3b22e318a2 |
Server: one way to work on the table off-thread, and smaller tidying
Removing a peer's routes, sweeping its stale ones and sweeping the attribute sets were each written out as their own spawn_blocking, and each dealt with a failed task differently: the periodic sweep dropped a panic on the floor, and a restarting peer whose kept routes could not be removed was logged as "removed 0 routes". off_thread does it once, and every caller logs a failure. Also: - the hard-reset wrapping of a NOTIFICATION is one function in the FSM; - the two ways to close every session are one, and the registry has one leave() rather than leave() and shutdown(); - the dialer says why a session ended with SessionEnd::reason, and no longer turns a result into a result; - as_path_prepend is parsed in one place, and the limits in error texts come from the constants rather than being written out; - router_id and cluster_id are parsed by one function; - the exporter builds its Egress in one place; - the fields a session keeps only to hand to its handle are fewer: route_refresh is a bool, not a copy of the families; - counting the stale routes removed cannot underflow; - docs that had gone stale: the proto still said there was no route reflection, the advertiser's module doc said it announced only our own routes, from_server listed two of its five sources, and so on. Tests for what had none: which session ends count as restarts, the Hard Reset wrapping, the graceful restart settings, the dialer's jitter, withdrawals in MP_UNREACH_NLRI, and AS_TRANS on a two-octet session. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
c9751883c4 |
Passive peers: configured, and left to connect
A session could only be configured for a peer we would then dial, every few seconds to two minutes, for as long as it was not up. A new peer whose side was not ready yet filled the log with failed attempts, and a peer that opens its sessions itself was dialled for nothing. A peer with `passive: true` is not dialled. A connection from its address is taken as for any configured peer, and the session comes up with all that is configured for it: role, import policy, export limits, hold-downs. Until then the status page and /api/peers show it as `Passive`, waiting for the peer to connect; after a session, with how that session ended, which nothing else would say since nobody dials it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
9cfd3f67f0 |
Export limits, and a role for every peer where routes are passed on
What other networks were told of rested on one thing: the Only to Customer mark that routes from providers, peers and route servers get as they come in. A route without it, from a session without a role or by a mistake anywhere along the way, went to every peer, from every router of ours, and nothing of ours stood in the way. export_limits are a second check, independent of the roles, the marks and the export policy. For peers above or beside us (PROVIDER, PEER, ROUTE_SERVER, and any in another AS without a role): - prefixes: only routes for these go out, learned or our own. With our and our customers' space listed, a leak of anyone else's routes goes nowhere above us. - max_prefixes: if more routes were to go out than this, none of them does. The session ends with a Cease / Administrative Shutdown whose Shutdown Communication says why (RFC 9003), and the peer is held down for its import.hold_down_seconds, so that it is told nothing rather than too much. The whole table is counted before any of it is sent. They are set at the top and hold for every such peer that has none of its own; unlike `export`, a peer's own export policy does not drop them. A peer's own export_limits take their place, and an empty one lifts them. Customers, route server clients and our own routers are not limited. A configuration in which learned routes are passed on and a peer in another AS has no role is now refused, as is accept_unconfigured_peers with learned routes: without a role, a peer's routes go to everyone and it is told of everything. This used to be a warning. At startup, a warning for each peer the limits apply to that has none, and for each originated route the limits leave out. A metric, bgp_export_limit_exceeded_total, counts the sessions ended for it. The interop lab has a phase in which a customer leaks: a route outside its space never reaches BIRD, and when more were to go than the limit, BIRD's session ends and its log has our reason. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
b16227c51f |
Route reflection, with ourselves as next hop
A route learned from one router of our AS went to no other (RFC 4271 §9.2.1), so every router needed a session with every other. A router at the end of a single link, with one session to its neighbour inside the AS, heard only of what that neighbour learned itself, and nothing of what the neighbour had from the routers behind it. A peer in our own AS can now be made a route reflector client (`route_reflector_client`, RFC 4456). What a client sends goes to all our routers; what the others send goes to the clients. A reflected route gets ORIGINATOR_ID (where it started, if it does not say) and our cluster ID at the front of CLUSTER_LIST; it is not sent back to the router it started at, nor round to a cluster it has been through, and one that comes back anyway is refused as "route reflected back to us". The cluster ID is the router ID unless `cluster_id` says otherwise. The two attributes are now decoded as such rather than kept as bytes, and are never sent to other networks. Unlike a standard reflector, this one is the next hop of what it reflects, as it is of everything it announces: there is no resolving a next hop that is not a neighbour, so a client reaches what lies behind another router through the reflector. Only peers in our own AS can be clients; the configuration refuses the setting for any other. Nothing changes in what other networks are told: Only to Customer marks go along with reflected routes, and every receiving session keeps its own max_prefixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
72444c2706 |
Control communities: session groups, prepends, and only ours to give
Control communities could say which peers a route goes to, by AS. Now
they can also name sessions by group, and ask for prepends:
L:11:P .. L:13:P our AS once to three times more towards AS P
(P 0: towards every peer in another AS)
L:20:G / L:21:G not to / only to the sessions of group G
L:31:G .. L:33:G our AS once to three times more towards group G
A group is a number given to sessions in the configuration (`groups` of
a peer): those at one exchange, or those over one link. The same network
at two exchanges is one AS but two sessions, which is what steering
inbound traffic between exchanges needs. Of several prepends that apply
the most counts, on top of the session's own as_path_prepend. Functions
2 to 9, as the documentation already promised, and 100 and up stay
informational; 10 to 99 are kept for actions to come.
They are acted on where a route leaves our AS, and taken off there; our
informational communities are not. To a peer in our own AS a route goes
with them, unchanged, so that a mark put on at one router is acted on at
another. Before, they were acted on and taken off on internal sessions
too, so a route marked "to nobody" did not even reach our own routers.
And they are ours to give: the communities of our own AS (L:n, L:n:m,
and 0:n where control communities are on) are taken off every route
from a peer in another AS as it comes in, before import marks are put
on. A peer at an exchange could otherwise decide where its routes go
from us, and set our informational marks. A customer that is to steer
its own routes is given import.accept_control_communities. An UPDATE
without any of ours is still taken as it came, without being written
out again.
This changes what a customer that set control communities gets: it now
needs accept_control_communities. The transit example and the interop
lab say so; the lab also has FRR, as a customer, ask for a prepend
towards BIRD's group, and BIRD, as a provider, have its "not to" taken
off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
65f136048e |
Graceful restart (RFC 4724, RFC 8538)
A peer that restarts keeps its routes with us, and we keep ours with our peers while we restart, so that a restart withdraws nothing across the network and the machine goes on forwarding. On by default for every peer; `graceful_restart` sets the restart time we give, caps how long a peer's routes are kept, or turns it off. When a peer restarts: its routes are marked stale in the routing table, and chosen as before, for at most its restart time. When it is back, what it sends again is fresh, and the stale routes of a family go at its End-of-RIB marker, or after the cap; at once for a family whose forwarding it did not keep. Only a connection that failed counts as a restart, and, with RFC 8538 agreed, an expired hold timer or a NOTIFICATION other than a Hard Reset. The stale mark is a bit of the route's word that selection never looks at. When we restart: SIGUSR1 leaves the kernel's routes where they are, closes the sessions without a word where graceful restart is agreed, and exits with status 75, on which the systemd unit starts us again. The next start finds the routes in the kernel, says so in its OPENs (R, and F where forwarding was kept), and has them go once every peer has sent its End-of-RIB. Our own End-of-RIB markers for learned routes wait for that too. SIGTERM still stops for good, now with a Hard Reset where RFC 8538 is agreed, as are errors of ours. The status page and API show what was agreed with each peer, the routes kept while a peer restarts, and which routes are stale. Tested against FRR and BIRD in a new phase of the interoperability lab, and with routes kept in a real kernel over a restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1e2e8f2d86 |
Pass learned routes on to peers, with marks and control communities
What was to be scripted is configuration, done natively:
- import { add_communities, add_large_communities, remove_communities,
local_pref }: every route taken from a peer is marked (where it was
learned, say), with nobody else's word for our marks taken, and given
its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
as_path_prepend, learned_routes }, for a peer or for all that have none
of their own.
- control_communities: the communities by which routes say which peers
they are passed on to, as exchanges' route servers have them (0:P, L:P,
0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
goes on.
And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.
Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.
One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9367c38907 |
Remove policy scripts
Rhai made a decision about a route cost one to three microseconds, some two hundred times what the same costs in Rust. That was bearable where a function is asked once for an UPDATE, and is not where every route is weighed against every peer, as it is when routes are passed on. What is wanted of a policy (mark routes by where they came from, mark what is announced, let communities say who is told what) needs no language; it is to be configuration, done natively. Kept from that work: route selection goes by a LOCAL_PREF that policy set on a route from an external peer, and the lab's configuration can be added to by a phase. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
cbf0df1eef |
Policy scripts: the operator's own rules for routes, in Rhai
A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:
fn from_peer(route, peer) {
if route.has_community("65535:666") { return false; }
route.add_community(LEARNED_FROM[peer.role]);
route.local_pref = PREFERENCE[peer.role];
true
}
Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.
Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.
What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.
Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.
Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f1d80c66df |
Do not choose a route whose next hop cannot be reached (RFC 4271 §9.1.2.1)
Where routes go into the kernel, a route the kernel could not be given is no longer taken into the table: its next hop is no neighbor on any interface, or its prefix is a network the machine is itself on. Such a route used to be able to win selection and then leave its prefix without a route in the kernel, though another peer had one that worked. It is held aside the way an RPKI-invalid route is, and by the same means: judged on arrival against the interfaces as the kernel last described them, looked for in the table when they change, and taken from what is held when its next hop can be reached. The fib publishes the interfaces for this; the next hop is resolved once for an UPDATE and not for each of its routes. The lab has a phase, nexthops: FRR's better route with a next hop that is nowhere is held and BIRD's is installed, and they change places as an address comes and goes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3d946c32ce |
Code health after review: docs, metrics, structure, one more fuzz target
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
fd2248feb5 |
Peer at link-local addresses: fe80::1%eth0
A peer's endpoint may be a link-local IPv6 address with the interface it is on. Such a peer is the address and the interface together, wherever peers are told apart: admission, collisions, hold-downs, the refusal log, the status page. The interface is looked up by name each time it is needed, so one that appears later or is made anew is found. Our routes go out with our link-local address in both places of MP_REACH_NLRI's next hop, as FRR sends its own. A peer's link-local next hop is taken alone, twice, or after "::" (BIRD), and resolved in the kernel on the interface of the session rather than by looking fe80::/64 up, which every interface has. The lab has a new phase, linklocal: sessions with FRR (by interface, with radvd beside the daemon) and BIRD over a LAN with link-local addresses only, both families, pings across, the interface down and up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
92e013ca01 |
Carry IPv4 routes over IPv6 sessions with IPv6 next hops (RFC 8950)
For a link that has only IPv6 addresses. - The Extended Next Hop capability is offered on every session over IPv6 whose families include IPv4 unicast, and used if the peer offers it too. Nothing to configure. - On such a session our IPv4 routes go out in MP_REACH_NLRI with the session's IPv6 address as next hop, and are withdrawn in MP_UNREACH_NLRI. Without the agreement they are not sent, as before. - IPv4 routes are read with an IPv6 next hop, global or global with link-local. - The kernel installer resolves an IPv4 route's IPv6 next hop like any other and installs it: by way of an IPv6 nexthop object, or with RTA_VIA where routes carry their own gateway. Tested against a real kernel in both modes, with a link-local next hop too, and in the lab against FRR and BIRD: the new `v6only` phase takes the IPv4 addresses off the exchange LAN, runs one IPv6 session with each router for both families, and pings IPv4 across. Sessions between link-local addresses are not supported yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f9ef781d51 |
Check roles in OPEN and prevent route leaks with Only to Customer (RFC 9234)
A peer's configured role now does three more things: - Our OPEN carries the BGP Role capability with what we are to the peer. A peer whose own says it is something that does not pair with that, or that sends two different roles, is refused with OPEN Message Error / Role Mismatch. A peer that says nothing is accepted unless require_role is set. A role on a peer in our own AS is a configuration error. - Routes from a provider, a peer or a route server are marked Only to Customer on the way in if they are not already, and our routes are marked on the way out to a customer, a peer or a route server client. - A marked route from a customer or route server client, or from a peer with someone else's mark, is a leak and is refused. No policy setting turns that off. The packet crate gains the Only to Customer attribute (malformed: treat-as-withdraw), and the Role Mismatch subcode is corrected from 8 to 11. The status page shows whether the peer confirmed its role. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
8086aa994f |
Verify AS paths with ASPAs, over RTR version 2
Origin validation says who may originate a prefix and nothing about the
rest of the path: it does not catch a route leak, nor a hijack that keeps
the right origin at the end of a made-up path. ASPAs, in which a network
lists its providers, do.
bgp-rpki:
- aspa: the ASPAs and the upstream and downstream verification procedures
of draft-ietf-sidrops-aspa-verification-24, tested against the draft's
wording as written.
- rtr: protocol version 2 (draft-ietf-sidrops-8210bis-23) with the ASPA
PDU and the new error codes.
- client: asks for version 2 and falls back to whatever older version the
cache speaks; keeps ASPAs with the VRPs, applied whole or not at all;
publishes which customers' ASPAs changed; tells the cache why it is
being left (Error Report); does not ask one version's serial of another.
Server:
- A peer has a role (provider, customer, peer, route server, route server
client), which decides how its paths are verified. Without one they are
not.
- A route whose path is invalid is refused ("ASPA invalid") and held, like
one with an invalid origin; accept_aspa_invalid exempts a peer.
- When ASPAs change, routes in the table whose path has a changed customer
on it are verified again, and held routes are released when they pass
both checks.
- The revalidator finds a route's peer by index instead of searching a
list for every route.
The status page and API show each peer's role, each route's path validity,
the number of ASPAs and the RTR version in use; bgp_rpki_aspas is on
/metricz.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
bdaf226a16 |
Validate route origins against the RPKI and refuse invalid routes
With rpki { caches { ... } } in the configuration the server keeps the
validated ROA payloads of a validator and validates every route from a
peer outside its own AS (RFC 6811). An invalid route is refused, unless
its peer has accept_rpki_invalid. Valid and not-found routes are taken;
validity does not otherwise enter route selection.
Validity is not settled when a route arrives: the data changes every few
minutes. The routes follow, without the peer sending anything again.
- A route refused as invalid is held by its session, out of the table.
When the data changes the session looks at what it holds, and what is
no longer invalid goes into the table. A held route is the peer's to
withdraw or replace like any other.
- A route in the table that becomes invalid is found by a task that
looks through the table when the data changes, at the prefixes the
changed VRPs cover, and is taken out and held by its session.
- If the data expires because no cache confirms it, nothing is invalid
any more and everything held is taken.
- A session holds at most 200,000 routes; past that it asks the peer for
a route refresh when the data changes.
So that the table can be looked through without parsing attributes, an
attribute set's key now has the route's origin AS. Memory per route is
unchanged at 52 bytes on the real table.
Shown: /api/rpki and a section of the status page with the cache, its
state and the age of the data; each route's validity; each peer's count
of held routes. Refusals are recorded under "RPKI invalid". Metrics:
bgp_rpki_vrps and bgp_rpki_updates_total.
Also: Negotiated says whether the peer supports route refresh, and
docs/RFCs.md is brought up to date.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
e2b1979e38 |
Judge every route a peer announces: import policy, prefix limit, hold-down
A configured peer could announce any prefix with any path, in any number, and all of it was stored and could be chosen as best. Each peer now has an import policy, and every route goes through it before it reaches the routing table. Checks that hold for any peer unless turned off for it: - no prefixes that are not routed on the Internet (private, loopback, link-local, documentation, multicast, reserved; for IPv6, anything outside global unicast); - nothing longer than /24 or /48, no default route, nothing shorter than /8 or /16; - no reserved, private or documentation AS number in the path; - a path that begins with the peer's own AS (turned off for a route server), and of at most 64 AS numbers. Always: a route with our own AS number in its path is refused. RFC 4271 requires this loop check and it was missing. Per peer, if configured: the prefixes it may announce, each with an optional longest more-specific, and the AS numbers its routes may originate from. This is the list generated from the routing registries. max_prefixes limits how many routes a peer may have. One more ends the session with Cease / Maximum Number of Prefixes Reached, and the peer is kept down for hold_down_seconds (ten minutes unless set): its connections are not taken and it is not dialled. Refused routes do not count towards the limit. A refused route is withdrawn rather than ignored, so that it replaces whatever the peer had announced for it before. On a session with two-octet AS numbers the path is judged with its AS4_PATH merged in, so that the real AS numbers are what is checked. Peers that are not configured (accept_unconfigured_peers, for test networks) get a policy that takes everything but loops. Refusals are counted in bgp_routes_rejected_total by reason. The status page and /api/peers show, for each session, how many routes the peer has and how many were refused. The server warns at startup about peers with no limit. The prefix check costs about 4 ns a route with the default policy; the end-to-end benchmark is unchanged. Also: the benchmark's server AS is one that is in none of its generated paths, and a table test that depended on the hasher's seed is fixed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
630c175918 |
Rework the speaker: sans-I/O FSM, sharded table, RFC 7606, ADD-PATH
The work of the speaker-cleanup branch as one commit; the branch keeps the step-by-step history. Sessions - The state machine is a pure function from events to actions, driven by one task per connection. Idle, Connect and Active belong to a dialer that keeps configured peers up with backoff. - Capabilities are negotiated: address families, four-octet AS numbers with AS_TRANS and AS4_PATH, route refresh, ADD-PATH per family and direction. - Collision detection per RFC 4271 section 6.8, by address and identifier. - Connections are taken only from configured peers, a few per address. - A panic in a session is contained and answered with a Cease; a core task that stops shuts the server down with a Cease to every peer. - Writes that make no progress are given up (RFC 9687). Codec - Malformed UPDATEs are handled as RFC 7606 says: treat-as-withdraw, attribute discard, or session reset, with the NOTIFICATION to send. - Address families are open types; any can be named and ignored. - Prefixes are fixed-size values, normalised, with path identifiers. - Lengths are filled in after writing; one definition of each type code. - Property tests and fuzz targets for every decoder. Routing table - Adj-RIBs-In and Loc-RIB in one sharded table that sessions write to directly. Routes are sixteen bytes; attribute sets are interned in wire form. Best-path selection per RFC 4271 section 9.1.2.2 with deterministic MED. - A journal per shard lets followers read changes at their own pace; one that falls behind is resynchronised. - On a RIPE RIS dump of 148 peers and 39.3M routes: loaded in 2.0 s on 12 cores, about 52 bytes a route. Announcing - Originated routes come from the configuration. Each session has its own exporter with an Adj-RIB-Out, so a slow peer delays only itself. - Routes to a peer in our own AS have an empty path and a LOCAL_PREF. Operations - HTTP status page, JSON API, event stream and Prometheus metrics. - Benchmarks on synthetic and real tables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |