main
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
bcfd6e013f |
rib: the lock-free number in a slot of its own, and the reader in named steps
A code-quality pass over the table. The latest sequence number of each journal, which readers compare their cursors with without the shard's lock, was an atomic inside the journal under the lock with the table holding the other end of an Arc to it, stored on every route; it is now in a slot beside each shard's lock, `ShardSlot`, stored once a shard a change while the write guard is still held and before the readers are woken. "One of each kind of journal" is one type, `ByKind<T>`, where three hand-written dispatchers and an array index were. `Reader::poll` is the loop skeleton over `resync_step` and `follow_pass`; the journal's `tidy` is three named stages; `live`'s nested Option is an enum that says what an entry says; the place of a cursor in a journal is found by one function for both the time chosen for a pass and the pass itself. The attribute set's four byte-readers share one walk over the raw attributes, with the next hop decoded once for both of its readers. `best` returns a `Route` like the other lookups. Docs that described an earlier design (the reader's `seen`, "every method takes &self", a `Routes::is_empty` that could not be true) are put right. Testing: new tests pin the lock-free skip, which nothing checked directly before: the number equals the journal's after every kind of change, with and without readers of each kind; a shard passed over is read again once it has news, from a reader of the whole table and from one that starts now. The OTC flag and the received-attributes interning gain tests. The crate's tests, both proptests included, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
c70a223d0d |
rib: a reader counts the table once a poll, not once a shard
A router with many peers has a reader of the table's journal for each session that is sent learned routes, seventy on ddln, and every change to the table wakes them all. Each pass of a reader went round every shard, and for each shard took a count on the table (an Arc clone) to hold it while the shard was read, and gave it back after: two atomic adds on one cache line, by seventy readers, 256 shards a pass. A profile of ddln taking a table from the other router had the reader's poll as the top function at two to three cores, and in the Linux VM with 64 such sessions half of all CPU time was in the two atomic add instructions. The sessions writing to the table waited behind it: the table came in at 58,000 routes a second. The count is now taken once a poll. And each journal keeps its latest sequence number in an atomic the table holds the other end of, so that a reader can tell a shard with nothing new for it from one with something without the shard's lock, and pass over it: on a quiet table, which is what the readers see most of the time, a pass is a load a shard and no lock. An exporter that is sent little of a table, as one to a peer at an exchange is, no longer makes a verdict of nothing for a prefix the peer does not hold, which cost a map insert and a lookup on the way out, and tells a route that came down or across (RFC 9234) from its attributes' bytes before parsing them. Testing: the mesh benchmark can have any number of collector sessions of the first router (BENCH_COLLECTORS): peers in the sense of RFC 9234, sent learned routes, with the feeders' routes marked as having come across from a peer, so that each has a reader following the table while it fills and passes nearly all of it over, as a router with many peers at exchanges has. In the Linux VM with 64 of them, three rounds alternating with the previous commit, the routers' exchange of their tables takes 11 s instead of 20 s, at 86 s of CPU time instead of 155 s; without them it is unchanged. The table's tests, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
92b9bdd3c5 |
export: keep the peer reading while the next batch is dressed
An exporter following the table took up to 65,536 changes at a time, dressed them all, and only then sent them; and before each batch it slept the 50 ms that lets a run of changes be gathered, whether or not the table was already far ahead of it. The session's queue holds about a thousand UPDATEs, so once the batch went out the exporter waited on the peer reading it, and while the next batch was dressed the peer had nothing to read. A profile of two routers exchanging their tables, sampled in 25 ms windows, showed the two stages taking turns: the exporters at two cores while the sessions were at none, then the other way round, with a gap of 50 to 75 ms at the start of each cycle for the sleep. Each task ran at about half a core. A batch is now 4,096 changes, so that what one batch sends is being read by the peer while the next is being dressed, and the gathering sleep is skipped while the reader is behind the table: a run is in hand already. A reader says whether its last poll caught up. Testing: the server tests, the integration tests, the kernel tests and every lab phase pass. On the mesh benchmark in the Linux VM, three rounds each alternating with the previous commit, the larger table (4.9 million routes) is across in 5.6 s instead of 6.7 s and the smaller (2.5 million) in 4.4 s instead of 6.4 s; the whole exchange takes 7.3 s of wall-clock time instead of 8.6 s, and 18.9 s of CPU time instead of 20.5 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f1aca37ebe |
rib: a short row grows by doubling
A prefix's row of paths is given room for as many paths as there are peers, and from there grows by a quarter at a time, one path at a time while it is short. A peer sending every path it has (ADD-PATH) gives a prefix several paths from one peer, so a row goes past the peers' count from the first few routes and was then reallocated for each path. A row shorter than the first allocation's room doubles instead, and grows by a quarter only from there. Measured on the mesh benchmark, the reallocations in the table's insert path fall by half; the memory held is as before, as the rows that reach the quarter-growth are the ones that were already allocated for it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
03f4f58ecb |
rib: give a row room for every peer's route at its first growth
A row's routes are in a vector that grew by one route at a time, so with four peers sending full tables every row's vector was reallocated three times as their routes arrived, and the reallocations were 7% of the daemon's CPU time in a profile of a four-peer load. This change has the first growth of a row take it to as many routes as the table has peers, at most 8 (`FIRST_ROOM`), since a prefix that one full table has, the others nearly always have too; after that it grows by a quarter as before. A table with one peer never grows a row and is unchanged. A route server's clients do not all announce every prefix, hence the cap: a row of a prefix that fewer peers announce has at most seven spare slots of 16 bytes. Measured in the Linux VM with four peers sending the synthetic table of 1.23 million prefixes: the kernel is in step 1.9 s after the first route instead of 2.2 s, the daemon's CPU time over the load is 5% lower, and it is 604 MiB resident instead of about 630, since there are no reallocation leftovers. With one peer nothing changes. Memory per route in the benchmarks, allocations counted: 77 bytes before and after with the synthetic table from four peers; 64 before and 65 after with the RIPE RIS dump's eight largest peers, whose load went from 2.8 s to 1.8 s on one thread, the allocation counting making every reallocation dearer than it is in the daemon. Also tried and not adopted, with the new `JEMALLOC_CONF` setting of the profiling script: transparent huge pages for the allocator (`thp:always,metadata_thp:always`). Over three pairs of runs with one peer the table arrived in 0.66, 1.17 and 0.70 s with them against 0.87, 0.75 and 0.87 s without, the CPU time was within the noise, and the daemon was 60 to 100 MiB larger. An arena for the routes would not help either: jemalloc's size classes already give rows exact 16, 32, 48 and 64 byte allocations, and the cost was the number of reallocations, not the allocator. Testing: the table tests, including the one that grows a row past 65,535 routes, pass; the kernel tests and the interoperability lab pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
48da8cc7d2 |
rib: size the routing table for the routes the peers are expected to send
The routing table's shards started empty and grew as routes came in, each growth moving every row of the shard into a table twice the size. In a profile of the daemon taking in a full table, the growth showed as up to a seventh of the session's time, and the daemon's resident memory had the old and the new table of each shard side by side for a moment. This change adds `RibTable::expect_prefixes`, which reserves room for that many prefixes spread over the shards, and has the server call it at startup with the largest `expected_routes` among the configured peers: the prefixes of several full tables are mostly the same ones. Without `expected_routes` nothing changes. The `full_table` benchmark does the same unless `BENCH_PRESIZE=0`. Measured, the saving in time is within the run-to-run noise: in the Linux VM one peer's table is in the routing table in 0.81 s instead of 0.89 s, and the kernel is in step at about the same time as before; three pairs of runs of the benchmark on a Mac average 0.33 s against 0.35 s. The profile's attribution to the growth was mostly sampling skid. The memory saving is real: 442 MiB resident with one peer's table in instead of 483 MiB, and 626 MiB instead of 632 MiB with four. Also: the profiling script printed nothing for the kernel's change counts because it asked `/metrics` for them; the endpoint is `/metricz`. With that fixed, a four-peer load shows 1.25 million installs for 1.23 million prefixes: the journal already gives the kernel task only the latest change of a prefix when it lags, so no further coalescing of installs is called for. Documentation: `expected_routes` in the configuration reference, the neighbor guide and the proto comment say that the table is sized by it; the testing guide's VM results are brought up to date. Testing: a table test checks that the room is there and is kept; the workspace tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
6d8f89e509 |
policy, fib: blackhole routing for routes with the BLACKHOLE community
Before this change, a route carrying the BLACKHOLE community (65535:666,
RFC 7999) from an EBGP neighbor was always rejected with the reason
"blackhole request", and nothing discarded traffic. A customer under a
denial-of-service attack had no way to ask this router to drop traffic
to an address.
This change adds `accept_blackholes` to the import policy. When it is
set, a route with the BLACKHOLE community is accepted if:
* its prefix is a single address, a /32 or a /128 (otherwise it is
rejected with the new reason "blackhole for more than one address");
the `max_ipv4_prefix_length` and `max_ipv6_prefix_length` limits do
not apply, as RFC 7999 section 3.2 recommends;
* the address is inside a prefix the neighbor may announce, if
`prefixes` is configured, regardless of the entry's `max_length`
(new `PrefixSet::covers`); without a prefix list, an address inside
the local AS's own prefixes is rejected as before ("prefix is our
own");
* the AS path, origin and RPKI checks that apply to every route pass.
The next hop of an accepted blackhole route is not checked for
third-party addresses and is not resolved, because traffic is never
forwarded to it. The route takes part in best path selection as usual;
a host route is normally the active route for its address.
The kernel task installs the active route as a kernel blackhole route
(`RTN_BLACKHOLE`, shown by `ip route` as `blackhole PREFIX proto bgp`).
Blackhole routes left by an earlier run are adopted at startup like
other routes of ours. The route is advertised to other neighbors as the
export policy and export limits permit, with the community intact, so a
provider that honors BLACKHOLE can discard the traffic upstream.
Implementation:
* `plan::Target` becomes an enum: `Via { gateway, interface }` or
`Discard`. `plan::resolve` takes a `blackhole` flag and returns
`Discard` for such routes without looking at the next hop (the
own-network check still applies). No nexthop object is made for a
discard target.
* `AttrSet::is_blackhole` reads the community off the stored attribute
bytes, like `next_hop` and `as_path`, so the kernel task, the
revalidator and the RPKI revalidation can ask it of every route.
* `ImportPolicy::is_blackhole_request` and `check_blackhole_prefix`
carry the import side; the import flow (`routes.rs`) computes once per
UPDATE whether its routes are an accepted blackhole request and skips
the next hop checks for them.
* `Status::blackholes` and the `blackholes` field of `/api/kernel` count
the installed blackhole routes; the status page shows the count next
to the installed routes.
Not covered: a prefix this router originates with the BLACKHOLE
community is advertised but not installed as a blackhole route. The
documentation says to add a kernel blackhole route by hand for that
case.
Documentation: the "Blackhole Routes" section of the import policy
guide replaces "Blackhole Requests", the kernel guide gains a "Blackhole
Routes" section, the configuration reference describes
`accept_blackholes`, the HTTP API reference describes `blackholes`, and
the RFC 7999 rows of the standards reference are updated.
Testing: unit tests for the attribute check, the prefix checks and the
plan; integration tests that a blackhole request is refused by default
and, with `accept_blackholes`, taken for one address inside the peer's
space and refused otherwise, with the recorded reasons; a kernel test
that a blackhole request becomes a kernel blackhole route whatever its
next hop, is removed on withdrawal and is adopted after a restart. The
existing kernel tests, the interoperability lab and the workspace tests
pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
35ce2e2e57 |
Status page: routes in order of selection, lookup by origin AS
Received routes are listed from the best down: the RIB gains ranked_routes, which runs selection again without the winners, so the second route is the one that would be installed were the best withdrawn. The API gives each route its rank, and the page shows it as a numbered badge after the star. The lookup box moves to the received-routes table, which is what it filters, and takes an AS number: a new `origin` query parameter finds every route whose path ends in that AS, with an empty path counting as the local AS. Long cells are clamped to a few lines with a control to open them, and an AS repeated along a path (prepending) is shown once with a ×N badge. Peer names are bold and dark, not underlined links, so a row can be found by its name among the addresses and numbers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
c69aa38090 |
Routing table: a reader waits out a change under way instead of spinning
A reader does not read past the earliest change still under way, which may not have reached every shard. A change to every shard, as when a peer's whole table goes, is under way for a while; changes others made meanwhile are beyond it. A reader that found only those saw "more to read" with nothing it could read, and asked again at once, round and round, taking every shard's read lock each time, until the change ended: every exporter and the kernel installer, spinning for as long as a full table took to remove. Now a pass that moves no cursor though there is more says it is caught up, and waits. And a change wakes the readers when it ends if any other change woke them while it was under way, since what they were woken for is only readable now. The three per-shard change loops are one (RibTable::change), which drops the change before waking, as only one of them did. Also: the add_peer doc was attached to peer(); some_routes had no callers; count_stale is for tests. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
b16227c51f |
Route reflection, with ourselves as next hop
A route learned from one router of our AS went to no other (RFC 4271 §9.2.1), so every router needed a session with every other. A router at the end of a single link, with one session to its neighbour inside the AS, heard only of what that neighbour learned itself, and nothing of what the neighbour had from the routers behind it. A peer in our own AS can now be made a route reflector client (`route_reflector_client`, RFC 4456). What a client sends goes to all our routers; what the others send goes to the clients. A reflected route gets ORIGINATOR_ID (where it started, if it does not say) and our cluster ID at the front of CLUSTER_LIST; it is not sent back to the router it started at, nor round to a cluster it has been through, and one that comes back anyway is refused as "route reflected back to us". The cluster ID is the router ID unless `cluster_id` says otherwise. The two attributes are now decoded as such rather than kept as bytes, and are never sent to other networks. Unlike a standard reflector, this one is the next hop of what it reflects, as it is of everything it announces: there is no resolving a next hop that is not a neighbour, so a client reaches what lies behind another router through the reflector. Only peers in our own AS can be clients; the configuration refuses the setting for any other. Nothing changes in what other networks are told: Only to Customer marks go along with reflected routes, and every receiving session keeps its own max_prefixes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Opus 5.5
|
65f136048e |
Graceful restart (RFC 4724, RFC 8538)
A peer that restarts keeps its routes with us, and we keep ours with our peers while we restart, so that a restart withdraws nothing across the network and the machine goes on forwarding. On by default for every peer; `graceful_restart` sets the restart time we give, caps how long a peer's routes are kept, or turns it off. When a peer restarts: its routes are marked stale in the routing table, and chosen as before, for at most its restart time. When it is back, what it sends again is fresh, and the stale routes of a family go at its End-of-RIB marker, or after the cap; at once for a family whose forwarding it did not keep. Only a connection that failed counts as a restart, and, with RFC 8538 agreed, an expired hold timer or a NOTIFICATION other than a Hard Reset. The stale mark is a bit of the route's word that selection never looks at. When we restart: SIGUSR1 leaves the kernel's routes where they are, closes the sessions without a word where graceful restart is agreed, and exits with status 75, on which the systemd unit starts us again. The next start finds the routes in the kernel, says so in its OPENs (R, and F where forwarding was kept), and has them go once every peer has sent its End-of-RIB. Our own End-of-RIB markers for learned routes wait for that too. SIGTERM still stops for good, now with a Hard Reset where RFC 8538 is agreed, as are errors of ours. The status page and API show what was agreed with each peer, the routes kept while a peer restarts, and which routes are stale. Tested against FRR and BIRD in a new phase of the interoperability lab, and with routes kept in a real kernel over a restart. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1e2e8f2d86 |
Pass learned routes on to peers, with marks and control communities
What was to be scripted is configuration, done natively:
- import { add_communities, add_large_communities, remove_communities,
local_pref }: every route taken from a peer is marked (where it was
learned, say), with nobody else's word for our marks taken, and given
its preference, which selection goes by.
- export { add_communities, add_large_communities, remove_communities,
as_path_prepend, learned_routes }, for a peer or for all that have none
of their own.
- control_communities: the communities by which routes say which peers
they are passed on to, as exchanges' route servers have them (0:P, L:P,
0:L, and the large forms L:0:P, L:1:P, L:0:0), taken off as the route
goes on.
And what the last of these is for: with export.learned_routes a peer is
told the best route of every prefix the others announced. The rules are
BGP's and cannot be turned off: not back where it came from, not to a
peer whose AS is on the path, not between two internal peers, not past
NO_EXPORT or NO_ADVERTISE; our AS on the path and ourselves as next hop
to another network, LOCAL_PREF to our own; to a route server's client
the path, MED and next hop as they came (RFC 7947). Which routes go where
follows from the roles of RFC 9234: a route that came down or across goes
on down only, so that with roles on every peer this makes us transit for
our customers and nothing else. A peer without a role is told everything,
and the daemon says so at startup.
Each session follows the table's best routes with a reader and keeps the
prefixes it has announced, so that it can withdraw them. Routes that go
out alike go in one UPDATE: changes are gathered for 50 ms first, and a
table that is given out anew is gathered whole (at most four sessions at
a time; it is tens of megabytes). For that to work the table's journal
had to give out together what was written together: every change now
carries a time on a table-wide clock, a reader's pass reads every shard
up to one time, no further than the changes that are complete in every
shard, and a resync takes its places in the journals as of one time.
One peer sending 1.23 M routes over a session, another told of them:
in our table after 0.47 s, all heard after 2.1 s, four routes an UPDATE.
The lab has a phase, transit: FRR as our customer and BIRD as our
provider, routes each way with the right paths and marks, one kept back
by a community, and pings between their networks through us.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
cbf0df1eef |
Policy scripts: the operator's own rules for routes, in Rhai
A new crate, bgp-policy, runs functions of a script on routes: on their
way in, after the general checks of a peer's import and before the
checks that can change (RPKI, next hop reachability); and on their way
out to a peer. A function is given the route and the peer, may refuse
the route, and may change its communities, large communities,
LOCAL_PREF and MED, and on the way out prepend our AS:
fn from_peer(route, peer) {
if route.has_community("65535:666") { return false; }
route.add_community(LEARNED_FROM[peer.role]);
route.local_pref = PREFERENCE[peer.role];
true
}
Configuration: policy { path | source, import_function, export_function }
and, for a peer, import { function } and export { function }. The script
is compiled at startup; `bgp --check` reads it and says which function
is whose. A function that fails on a route has refused it. Scripts have
no files, network or clock, and are stopped after 200,000 steps.
Route selection now goes by a LOCAL_PREF on a route from an external
peer: the decoder never lets a peer's own through, so one that is there
was set by policy.
What it costs was measured before it was built on (benches/policy.rs):
a call that tests a few communities and changes two takes 1 to 3
microseconds, some two hundred times what Rust would. So a function is
asked once for the routes of an UPDATE, which share their attributes,
unless it asks for the prefix. One peer sending 1.23 M routes over a
session: 0.37 s without a function, 0.80 s with one that marks and
ranks every route, 3.4 s if it also looks at the prefix.
Export applies to what is announced, which today is what the daemon
originates; learned routes are not passed on yet.
Tested by unit tests of the crate, sessions end to end, a fuzz target on
the example script (testdata/policy.rhai), and a lab phase in which a
script changes which of FRR's and BIRD's route is chosen and what each
of them is told.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3d946c32ce |
Code health after review: docs, metrics, structure, one more fuzz target
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9ca86ede2f |
Harden import, the kernel task and the RTR client after review
Three independent reviews (hijack resistance, reliability, code health) of the whole tree. This is what they found that could let a bad route in or take the daemon down. Hijacks: - A route had two paths on a two-octet session, AS_PATH and AS4_PATH, read in two places by two rules. A peer could make the origin checked on import differ from the one checked at revalidation, and get an RPKI-invalid route into the table. AS4_PATH is now merged into AS_PATH once, on receipt (RFC 6793 §4.2.3), and nothing else knows of it. - The next hop is judged on import: it must be usable, agreed on (RFC 8950), and by default the peer itself. - Our own originated prefixes are not taken from other networks. - A route for a network the machine is on is never put in the kernel. - Paths with an AS_SET are refused (RFC 9774). - A configured peer gets one session, not one per BGP Identifier. - More IPv6 special-purpose prefixes are bogons. Reliability: - A batch of requests the kernel refused could overflow the netlink socket and end the daemon. The socket has room now, a lost answer or a silent kernel means asking what is there, and neither is fatal. - Working the routes out again within the grace period after a restart no longer drops the routes being kept. - That work can no longer be put off for ever by interface changes. - Peers whose sessions keep failing are slowed down. - Unused attribute sets are swept when they pile up, not only each minute. - Log lines a peer can cause are said once in a while. - An RTR answer is bounded in size and time; data cannot expire between two refreshes. - The shard of a key is taken from a stirred hash: neighbouring bits of the hash proved alike for prefixes in a row. The test that showed it, failing one run in sixteen, is deterministic now. The lab has a phase, hijacks, in which FRR tries the first three. route::apply is a struct with methods instead of nested closures, and the kernel task's loop is shorter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
fd2248feb5 |
Peer at link-local addresses: fe80::1%eth0
A peer's endpoint may be a link-local IPv6 address with the interface it is on. Such a peer is the address and the interface together, wherever peers are told apart: admission, collisions, hold-downs, the refusal log, the status page. The interface is looked up by name each time it is needed, so one that appears later or is made anew is found. Our routes go out with our link-local address in both places of MP_REACH_NLRI's next hop, as FRR sends its own. A peer's link-local next hop is taken alone, twice, or after "::" (BIRD), and resolved in the kernel on the interface of the session rather than by looking fe80::/64 up, which every interface has. The lab has a new phase, linklocal: sessions with FRR (by interface, with radvd beside the daemon) and BIRD over a LAN with link-local addresses only, both families, pings across, the interface down and up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1fc44798b7 |
Install best routes in the Linux kernel over netlink
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1f84235e87 |
Page through the received routes, on the status page and in the API
/api/routes/received now returns one page: {routes, next_after,
prev_before}. Pages are in prefix order and begin at a prefix (after= or
before=), not at a count, so a table that changes between two pages does
not make them overlap or leave a gap. A page holds whole prefixes, about
`limit` routes' worth.
The table keeps prefixes in no order, so a page is found by looking at
every prefix once and keeping the nearest few; the routes of the others
are not touched (RibTable::for_each_prefix).
The status page shows about 100 routes with first / previous / next
links; the place is kept in the page's address with the lookup.
The response of /api/routes/received was an array before: this changes
its shape.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f9ef781d51 |
Check roles in OPEN and prevent route leaks with Only to Customer (RFC 9234)
A peer's configured role now does three more things: - Our OPEN carries the BGP Role capability with what we are to the peer. A peer whose own says it is something that does not pair with that, or that sends two different roles, is refused with OPEN Message Error / Role Mismatch. A peer that says nothing is accepted unless require_role is set. A role on a peer in our own AS is a configuration error. - Routes from a provider, a peer or a route server are marked Only to Customer on the way in if they are not already, and our routes are marked on the way out to a customer, a peer or a route server client. - A marked route from a customer or route server client, or from a peer with someone else's mark, is a leak and is refused. No policy setting turns that off. The packet crate gains the Only to Customer attribute (malformed: treat-as-withdraw), and the Role Mismatch subcode is corrected from 8 to 11. The status page shows whether the peer confirmed its role. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
8086aa994f |
Verify AS paths with ASPAs, over RTR version 2
Origin validation says who may originate a prefix and nothing about the
rest of the path: it does not catch a route leak, nor a hijack that keeps
the right origin at the end of a made-up path. ASPAs, in which a network
lists its providers, do.
bgp-rpki:
- aspa: the ASPAs and the upstream and downstream verification procedures
of draft-ietf-sidrops-aspa-verification-24, tested against the draft's
wording as written.
- rtr: protocol version 2 (draft-ietf-sidrops-8210bis-23) with the ASPA
PDU and the new error codes.
- client: asks for version 2 and falls back to whatever older version the
cache speaks; keeps ASPAs with the VRPs, applied whole or not at all;
publishes which customers' ASPAs changed; tells the cache why it is
being left (Error Report); does not ask one version's serial of another.
Server:
- A peer has a role (provider, customer, peer, route server, route server
client), which decides how its paths are verified. Without one they are
not.
- A route whose path is invalid is refused ("ASPA invalid") and held, like
one with an invalid origin; accept_aspa_invalid exempts a peer.
- When ASPAs change, routes in the table whose path has a changed customer
on it are verified again, and held routes are released when they pass
both checks.
- The revalidator finds a route's peer by index instead of searching a
list for every route.
The status page and API show each peer's role, each route's path validity,
the number of ASPAs and the RTR version in use; bgp_rpki_aspas is on
/metricz.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
bdaf226a16 |
Validate route origins against the RPKI and refuse invalid routes
With rpki { caches { ... } } in the configuration the server keeps the
validated ROA payloads of a validator and validates every route from a
peer outside its own AS (RFC 6811). An invalid route is refused, unless
its peer has accept_rpki_invalid. Valid and not-found routes are taken;
validity does not otherwise enter route selection.
Validity is not settled when a route arrives: the data changes every few
minutes. The routes follow, without the peer sending anything again.
- A route refused as invalid is held by its session, out of the table.
When the data changes the session looks at what it holds, and what is
no longer invalid goes into the table. A held route is the peer's to
withdraw or replace like any other.
- A route in the table that becomes invalid is found by a task that
looks through the table when the data changes, at the prefixes the
changed VRPs cover, and is taken out and held by its session.
- If the data expires because no cache confirms it, nothing is invalid
any more and everything held is taken.
- A session holds at most 200,000 routes; past that it asks the peer for
a route refresh when the data changes.
So that the table can be looked through without parsing attributes, an
attribute set's key now has the route's origin AS. Memory per route is
unchanged at 52 bytes on the real table.
Shown: /api/rpki and a section of the status page with the cache, its
state and the age of the data; each route's validity; each peer's count
of held routes. Refusals are recorded under "RPKI invalid". Metrics:
bgp_rpki_vrps and bgp_rpki_updates_total.
Also: Negotiated says whether the peer supports route refresh, and
docs/RFCs.md is brought up to date.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
599cd0ad8b |
Look routes up, and see and alert on the ones that were refused
Two things were hard to find out from the status page and the API: what the table has for a given prefix or address, and what a peer announced that was refused and why. Looking up. /api/routes/received takes a query, and the status page has a box for it: - prefix: every route for that prefix, the best first; - address: every prefix that covers the address, most specific first; - within: a prefix and everything inside it; - peer (by name or address) and best narrow any of these. A query that makes no sense is answered 400 with the reason, which the page shows. Results come in address order. Routes are shown with the peer's configured name. Refused routes. A refused route is not stored, so until now there was only a count. For each peer the server now keeps how many routes were refused for each reason, and the latest 200 with the AS path they came with. The record is kept by the peer's address and outlives the session, so that a peer that was cut off can still be looked into. It is at /api/routes/rejected and in a new section of the status page; the peers table links to it. Alerting. bgp_routes_rejected_total is now by peer as well as reason. New: bgp_peer_routes and bgp_peer_max_prefixes, to see a peer nearing its limit, and bgp_prefix_limit_exceeded_total. The README has rules to start from. The log has a line for a peer's first refused route and then at most one a minute, with the count of the others. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
e2b1979e38 |
Judge every route a peer announces: import policy, prefix limit, hold-down
A configured peer could announce any prefix with any path, in any number, and all of it was stored and could be chosen as best. Each peer now has an import policy, and every route goes through it before it reaches the routing table. Checks that hold for any peer unless turned off for it: - no prefixes that are not routed on the Internet (private, loopback, link-local, documentation, multicast, reserved; for IPv6, anything outside global unicast); - nothing longer than /24 or /48, no default route, nothing shorter than /8 or /16; - no reserved, private or documentation AS number in the path; - a path that begins with the peer's own AS (turned off for a route server), and of at most 64 AS numbers. Always: a route with our own AS number in its path is refused. RFC 4271 requires this loop check and it was missing. Per peer, if configured: the prefixes it may announce, each with an optional longest more-specific, and the AS numbers its routes may originate from. This is the list generated from the routing registries. max_prefixes limits how many routes a peer may have. One more ends the session with Cease / Maximum Number of Prefixes Reached, and the peer is kept down for hold_down_seconds (ten minutes unless set): its connections are not taken and it is not dialled. Refused routes do not count towards the limit. A refused route is withdrawn rather than ignored, so that it replaces whatever the peer had announced for it before. On a session with two-octet AS numbers the path is judged with its AS4_PATH merged in, so that the real AS numbers are what is checked. Peers that are not configured (accept_unconfigured_peers, for test networks) get a policy that takes everything but loops. Refusals are counted in bgp_routes_rejected_total by reason. The status page and /api/peers show, for each session, how many routes the peer has and how many were refused. The server warns at startup about peers with no limit. The prefix check costs about 4 ns a route with the default policy; the end-to-end benchmark is unchanged. Also: the benchmark's server AS is one that is in none of its generated paths, and a table test that depended on the hasher's seed is fixed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
630c175918 |
Rework the speaker: sans-I/O FSM, sharded table, RFC 7606, ADD-PATH
The work of the speaker-cleanup branch as one commit; the branch keeps the step-by-step history. Sessions - The state machine is a pure function from events to actions, driven by one task per connection. Idle, Connect and Active belong to a dialer that keeps configured peers up with backoff. - Capabilities are negotiated: address families, four-octet AS numbers with AS_TRANS and AS4_PATH, route refresh, ADD-PATH per family and direction. - Collision detection per RFC 4271 section 6.8, by address and identifier. - Connections are taken only from configured peers, a few per address. - A panic in a session is contained and answered with a Cease; a core task that stops shuts the server down with a Cease to every peer. - Writes that make no progress are given up (RFC 9687). Codec - Malformed UPDATEs are handled as RFC 7606 says: treat-as-withdraw, attribute discard, or session reset, with the NOTIFICATION to send. - Address families are open types; any can be named and ignored. - Prefixes are fixed-size values, normalised, with path identifiers. - Lengths are filled in after writing; one definition of each type code. - Property tests and fuzz targets for every decoder. Routing table - Adj-RIBs-In and Loc-RIB in one sharded table that sessions write to directly. Routes are sixteen bytes; attribute sets are interned in wire form. Best-path selection per RFC 4271 section 9.1.2.2 with deterministic MED. - A journal per shard lets followers read changes at their own pace; one that falls behind is resynchronised. - On a RIPE RIS dump of 148 peers and 39.3M routes: loaded in 2.0 s on 12 cores, about 52 bytes a route. Announcing - Originated routes come from the configuration. Each session has its own exporter with an Adj-RIB-Out, so a slow peer delays only itself. - Routes to a peer in our own AS have an empty path and a LOCAL_PREF. Operations - HTTP status page, JSON API, event stream and Prometheus metrics. - Benchmarks on synthetic and real tables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Sonnet 4.6
|
1ec60255ea |
feature: BGP server, FSM, Adj-RIB-In actor, and integration tests
- Implement passive-side BGP FSM (OpenSent → OpenConfirm → Established) with hold timer, keepalive timer, and NOTIFICATION on expiry - Add bgp-server crate: config validation, TCP listener accept loop (BgpServerBuilder → BgpServer), connection dispatch - Add bgp-rib crate: Adj-RIB-In actor owning route state with event broadcast for streaming consumers and snapshot for introspection - Wire Adj-RIB-In into FSM: announce/withdraw on UPDATE for both legacy IPv4 NLRI fields and MP_REACH/MP_UNREACH path attributes - Add bgp-integration-tests crate with 15 end-to-end tests covering session establishment, timers, UPDATE handling, and RIB introspection - Fix require_empty! macro bug in open.rs (checked wrong buffer) - Fix FourByteAsn capability parser returning consumed buffer as remainder - Fix tracing subscriber not being installed in TracingArgs::install - Expand docs/RFCs.md with full implementation status across 15 RFCs Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |