bcfd6e013f8a34d47f2c9c114cdc4588cb48724b
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rayhaan JaufeerallyandClaude Fable 5.1
|
bcfd6e013f |
rib: the lock-free number in a slot of its own, and the reader in named steps
A code-quality pass over the table. The latest sequence number of each journal, which readers compare their cursors with without the shard's lock, was an atomic inside the journal under the lock with the table holding the other end of an Arc to it, stored on every route; it is now in a slot beside each shard's lock, `ShardSlot`, stored once a shard a change while the write guard is still held and before the readers are woken. "One of each kind of journal" is one type, `ByKind<T>`, where three hand-written dispatchers and an array index were. `Reader::poll` is the loop skeleton over `resync_step` and `follow_pass`; the journal's `tidy` is three named stages; `live`'s nested Option is an enum that says what an entry says; the place of a cursor in a journal is found by one function for both the time chosen for a pass and the pass itself. The attribute set's four byte-readers share one walk over the raw attributes, with the next hop decoded once for both of its readers. `best` returns a `Route` like the other lookups. Docs that described an earlier design (the reader's `seen`, "every method takes &self", a `Routes::is_empty` that could not be true) are put right. Testing: new tests pin the lock-free skip, which nothing checked directly before: the number equals the journal's after every kind of change, with and without readers of each kind; a shard passed over is read again once it has news, from a reader of the whole table and from one that starts now. The OTC flag and the received-attributes interning gain tests. The crate's tests, both proptests included, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
c70a223d0d |
rib: a reader counts the table once a poll, not once a shard
A router with many peers has a reader of the table's journal for each session that is sent learned routes, seventy on ddln, and every change to the table wakes them all. Each pass of a reader went round every shard, and for each shard took a count on the table (an Arc clone) to hold it while the shard was read, and gave it back after: two atomic adds on one cache line, by seventy readers, 256 shards a pass. A profile of ddln taking a table from the other router had the reader's poll as the top function at two to three cores, and in the Linux VM with 64 such sessions half of all CPU time was in the two atomic add instructions. The sessions writing to the table waited behind it: the table came in at 58,000 routes a second. The count is now taken once a poll. And each journal keeps its latest sequence number in an atomic the table holds the other end of, so that a reader can tell a shard with nothing new for it from one with something without the shard's lock, and pass over it: on a quiet table, which is what the readers see most of the time, a pass is a load a shard and no lock. An exporter that is sent little of a table, as one to a peer at an exchange is, no longer makes a verdict of nothing for a prefix the peer does not hold, which cost a map insert and a lookup on the way out, and tells a route that came down or across (RFC 9234) from its attributes' bytes before parsing them. Testing: the mesh benchmark can have any number of collector sessions of the first router (BENCH_COLLECTORS): peers in the sense of RFC 9234, sent learned routes, with the feeders' routes marked as having come across from a peer, so that each has a reader following the table while it fills and passes nearly all of it over, as a router with many peers at exchanges has. In the Linux VM with 64 of them, three rounds alternating with the previous commit, the routers' exchange of their tables takes 11 s instead of 20 s, at 86 s of CPU time instead of 155 s; without them it is unchanged. The table's tests, the server tests, the integration tests, the kernel tests and every lab phase pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
6d8f89e509 |
policy, fib: blackhole routing for routes with the BLACKHOLE community
Before this change, a route carrying the BLACKHOLE community (65535:666,
RFC 7999) from an EBGP neighbor was always rejected with the reason
"blackhole request", and nothing discarded traffic. A customer under a
denial-of-service attack had no way to ask this router to drop traffic
to an address.
This change adds `accept_blackholes` to the import policy. When it is
set, a route with the BLACKHOLE community is accepted if:
* its prefix is a single address, a /32 or a /128 (otherwise it is
rejected with the new reason "blackhole for more than one address");
the `max_ipv4_prefix_length` and `max_ipv6_prefix_length` limits do
not apply, as RFC 7999 section 3.2 recommends;
* the address is inside a prefix the neighbor may announce, if
`prefixes` is configured, regardless of the entry's `max_length`
(new `PrefixSet::covers`); without a prefix list, an address inside
the local AS's own prefixes is rejected as before ("prefix is our
own");
* the AS path, origin and RPKI checks that apply to every route pass.
The next hop of an accepted blackhole route is not checked for
third-party addresses and is not resolved, because traffic is never
forwarded to it. The route takes part in best path selection as usual;
a host route is normally the active route for its address.
The kernel task installs the active route as a kernel blackhole route
(`RTN_BLACKHOLE`, shown by `ip route` as `blackhole PREFIX proto bgp`).
Blackhole routes left by an earlier run are adopted at startup like
other routes of ours. The route is advertised to other neighbors as the
export policy and export limits permit, with the community intact, so a
provider that honors BLACKHOLE can discard the traffic upstream.
Implementation:
* `plan::Target` becomes an enum: `Via { gateway, interface }` or
`Discard`. `plan::resolve` takes a `blackhole` flag and returns
`Discard` for such routes without looking at the next hop (the
own-network check still applies). No nexthop object is made for a
discard target.
* `AttrSet::is_blackhole` reads the community off the stored attribute
bytes, like `next_hop` and `as_path`, so the kernel task, the
revalidator and the RPKI revalidation can ask it of every route.
* `ImportPolicy::is_blackhole_request` and `check_blackhole_prefix`
carry the import side; the import flow (`routes.rs`) computes once per
UPDATE whether its routes are an accepted blackhole request and skips
the next hop checks for them.
* `Status::blackholes` and the `blackholes` field of `/api/kernel` count
the installed blackhole routes; the status page shows the count next
to the installed routes.
Not covered: a prefix this router originates with the BLACKHOLE
community is advertised but not installed as a blackhole route. The
documentation says to add a kernel blackhole route by hand for that
case.
Documentation: the "Blackhole Routes" section of the import policy
guide replaces "Blackhole Requests", the kernel guide gains a "Blackhole
Routes" section, the configuration reference describes
`accept_blackholes`, the HTTP API reference describes `blackholes`, and
the RFC 7999 rows of the standards reference are updated.
Testing: unit tests for the attribute check, the prefix checks and the
plan; integration tests that a blackhole request is refused by default
and, with `accept_blackholes`, taken for one address inside the peer's
space and refused otherwise, with the recorded reasons; a kernel test
that a blackhole request becomes a kernel blackhole route whatever its
next hop, is removed on withdrawal and is adopted after a restart. The
existing kernel tests, the interoperability lab and the workspace tests
pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
3d946c32ce |
Code health after review: docs, metrics, structure, one more fuzz target
What the code-health review found, and what a fuzz target written for it
found in its first seconds.
- A set of attributes with the same attribute twice is refused by the
routing table. Its readers (the selection key, the path, the next hop)
could each take a different one of the two. The decoder never lets one
through; the new fuzz target `attrs` found that the table would.
- bgp_peer_up{peer}, to alert on a peer going away, and bgp_build_info.
Every metric is there from the first scrape. Alerts for a peer down,
hijack attempts and kernel errors are in the README.
- `bgp --check config` validates a configuration without starting, and
`--version` says which build it is. CI runs the first on the example
configuration, which a test now also parses with its commented fields
put in; the example had to learn about the next hop check.
- Path attributes have a module of their own, packet::attributes,
re-exported from packet::message as before.
- policy::From, which shadowed the trait, is RouteSource. The server's
connection, dialer and fsm modules are private. Logging arguments live
in the binary; the server library no longer depends on clap.
- One is_unicast_link_local (the standard library's) instead of two of
our own; SelectionKey::origin(); one place where an Import is made.
- READMEs say what the daemon does today rather than what it did not do
a year ago. Documentation links are checked in CI. Unused dependencies
and bgp_web::serve are gone; the Docker build is --locked and does not
ship the fuzz corpus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
9ca86ede2f |
Harden import, the kernel task and the RTR client after review
Three independent reviews (hijack resistance, reliability, code health) of the whole tree. This is what they found that could let a bad route in or take the daemon down. Hijacks: - A route had two paths on a two-octet session, AS_PATH and AS4_PATH, read in two places by two rules. A peer could make the origin checked on import differ from the one checked at revalidation, and get an RPKI-invalid route into the table. AS4_PATH is now merged into AS_PATH once, on receipt (RFC 6793 §4.2.3), and nothing else knows of it. - The next hop is judged on import: it must be usable, agreed on (RFC 8950), and by default the peer itself. - Our own originated prefixes are not taken from other networks. - A route for a network the machine is on is never put in the kernel. - Paths with an AS_SET are refused (RFC 9774). - A configured peer gets one session, not one per BGP Identifier. - More IPv6 special-purpose prefixes are bogons. Reliability: - A batch of requests the kernel refused could overflow the netlink socket and end the daemon. The socket has room now, a lost answer or a silent kernel means asking what is there, and neither is fatal. - Working the routes out again within the grace period after a restart no longer drops the routes being kept. - That work can no longer be put off for ever by interface changes. - Peers whose sessions keep failing are slowed down. - Unused attribute sets are swept when they pile up, not only each minute. - Log lines a peer can cause are said once in a while. - An RTR answer is bounded in size and time; data cannot expire between two refreshes. - The shard of a key is taken from a stirred hash: neighbouring bits of the hash proved alike for prefixes in a row. The test that showed it, failing one run in sixteen, is deterministic now. The lab has a phase, hijacks, in which FRR tries the first three. route::apply is a struct with methods instead of nested closures, and the kernel task's loop is shorter. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
1fc44798b7 |
Install best routes in the Linux kernel over netlink
New crate bgp-fib. It follows the routing table's best routes and keeps
the kernel's routing table the same, and learns the machine's interfaces
and addresses from the kernel, which is what says whether a next hop can
be reached.
- netlink: the rtnetlink messages used (routes, addresses, links, errors),
as bytes and back. No I/O; property tested and fuzzed.
- interfaces: what is directly reachable, and from which interface.
- plan: what to ask the kernel for, given the best routes and what the
kernel has. No I/O; tested against a model kernel.
- kernel (Linux only): the sockets and the task. Requests go in batches;
routes of ours found at startup are kept if still right and removed if
not; a change of interfaces has every route worked out again; a clean
stop removes our routes.
Only routes marked proto bgp with the configured table and metric are
ever touched. A best route is installed if its next hop is a neighbor;
next hops further away are not resolved through other routes yet, and
such prefixes are counted and shown instead.
Configured with `kernel { table metric }`; off by default. The daemon
needs CAP_NET_ADMIN and stops if the kernel refuses it. On shutdown it
closes sessions and then removes its routes.
/api/kernel and the status page show installed, unresolved and refused
counts, the last error and the interfaces; five bgp_kernel_* metrics.
Testing: scripts/kernel-tests.sh runs, in network namespaces of their
own and with only the capabilities the daemon needs, the installer
against `ip route`, the daemon with a peer, and two daemons that route
for each other with pings across over IPv4 and IPv6.
scripts/linux-vm.sh runs that, or anything else, in a Lima VM from
macOS. CI runs the kernel tests and fuzzes the netlink decoder.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
f9ef781d51 |
Check roles in OPEN and prevent route leaks with Only to Customer (RFC 9234)
A peer's configured role now does three more things: - Our OPEN carries the BGP Role capability with what we are to the peer. A peer whose own says it is something that does not pair with that, or that sends two different roles, is refused with OPEN Message Error / Role Mismatch. A peer that says nothing is accepted unless require_role is set. A role on a peer in our own AS is a configuration error. - Routes from a provider, a peer or a route server are marked Only to Customer on the way in if they are not already, and our routes are marked on the way out to a customer, a peer or a route server client. - A marked route from a customer or route server client, or from a peer with someone else's mark, is a leak and is refused. No policy setting turns that off. The packet crate gains the Only to Customer attribute (malformed: treat-as-withdraw), and the Role Mismatch subcode is corrected from 8 to 11. The status page shows whether the peer confirmed its role. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
8086aa994f |
Verify AS paths with ASPAs, over RTR version 2
Origin validation says who may originate a prefix and nothing about the
rest of the path: it does not catch a route leak, nor a hijack that keeps
the right origin at the end of a made-up path. ASPAs, in which a network
lists its providers, do.
bgp-rpki:
- aspa: the ASPAs and the upstream and downstream verification procedures
of draft-ietf-sidrops-aspa-verification-24, tested against the draft's
wording as written.
- rtr: protocol version 2 (draft-ietf-sidrops-8210bis-23) with the ASPA
PDU and the new error codes.
- client: asks for version 2 and falls back to whatever older version the
cache speaks; keeps ASPAs with the VRPs, applied whole or not at all;
publishes which customers' ASPAs changed; tells the cache why it is
being left (Error Report); does not ask one version's serial of another.
Server:
- A peer has a role (provider, customer, peer, route server, route server
client), which decides how its paths are verified. Without one they are
not.
- A route whose path is invalid is refused ("ASPA invalid") and held, like
one with an invalid origin; accept_aspa_invalid exempts a peer.
- When ASPAs change, routes in the table whose path has a changed customer
on it are verified again, and held routes are released when they pass
both checks.
- The revalidator finds a route's peer by index instead of searching a
list for every route.
The status page and API show each peer's role, each route's path validity,
the number of ASPAs and the RTR version in use; bgp_rpki_aspas is on
/metricz.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
bdaf226a16 |
Validate route origins against the RPKI and refuse invalid routes
With rpki { caches { ... } } in the configuration the server keeps the
validated ROA payloads of a validator and validates every route from a
peer outside its own AS (RFC 6811). An invalid route is refused, unless
its peer has accept_rpki_invalid. Valid and not-found routes are taken;
validity does not otherwise enter route selection.
Validity is not settled when a route arrives: the data changes every few
minutes. The routes follow, without the peer sending anything again.
- A route refused as invalid is held by its session, out of the table.
When the data changes the session looks at what it holds, and what is
no longer invalid goes into the table. A held route is the peer's to
withdraw or replace like any other.
- A route in the table that becomes invalid is found by a task that
looks through the table when the data changes, at the prefixes the
changed VRPs cover, and is taken out and held by its session.
- If the data expires because no cache confirms it, nothing is invalid
any more and everything held is taken.
- A session holds at most 200,000 routes; past that it asks the peer for
a route refresh when the data changes.
So that the table can be looked through without parsing attributes, an
attribute set's key now has the route's origin AS. Memory per route is
unchanged at 52 bytes on the real table.
Shown: /api/rpki and a section of the status page with the cache, its
state and the age of the data; each route's validity; each peer's count
of held routes. Refusals are recorded under "RPKI invalid". Metrics:
bgp_rpki_vrps and bgp_rpki_updates_total.
Also: Negotiated says whether the peer supports route refresh, and
docs/RFCs.md is brought up to date.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
Rayhaan JaufeerallyandClaude Fable 5.1
|
630c175918 |
Rework the speaker: sans-I/O FSM, sharded table, RFC 7606, ADD-PATH
The work of the speaker-cleanup branch as one commit; the branch keeps the step-by-step history. Sessions - The state machine is a pure function from events to actions, driven by one task per connection. Idle, Connect and Active belong to a dialer that keeps configured peers up with backoff. - Capabilities are negotiated: address families, four-octet AS numbers with AS_TRANS and AS4_PATH, route refresh, ADD-PATH per family and direction. - Collision detection per RFC 4271 section 6.8, by address and identifier. - Connections are taken only from configured peers, a few per address. - A panic in a session is contained and answered with a Cease; a core task that stops shuts the server down with a Cease to every peer. - Writes that make no progress are given up (RFC 9687). Codec - Malformed UPDATEs are handled as RFC 7606 says: treat-as-withdraw, attribute discard, or session reset, with the NOTIFICATION to send. - Address families are open types; any can be named and ignored. - Prefixes are fixed-size values, normalised, with path identifiers. - Lengths are filled in after writing; one definition of each type code. - Property tests and fuzz targets for every decoder. Routing table - Adj-RIBs-In and Loc-RIB in one sharded table that sessions write to directly. Routes are sixteen bytes; attribute sets are interned in wire form. Best-path selection per RFC 4271 section 9.1.2.2 with deterministic MED. - A journal per shard lets followers read changes at their own pace; one that falls behind is resynchronised. - On a RIPE RIS dump of 148 peers and 39.3M routes: loaded in 2.0 s on 12 cores, about 52 bytes a route. Announcing - Originated routes come from the configuration. Each session has its own exporter with an Adj-RIB-Out, so a slow peer delays only itself. - Routes to a peer in our own AS have an empty path and a LOCAL_PREF. Operations - HTTP status page, JSON API, event stream and Prometheus metrics. - Benchmarks on synthetic and real tables. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> |